Skip to content

Escalation chain: push → call → Discord, Teams, PagerDuty

A push to your phone is the fastest channel, but not the most reliable one: the phone is on silent, you’re on the subway, the person on duty is asleep. Instead of duplicating the send to five systems from every script, set up escalation on the channel: the script still makes a single POST /message, and Notifly itself decides where to re-send anything left unread.

POST /message ──▶ push to all devices (immediately)
│
└─ once a minute: has the message been read?
├─ yes → the chain stops
└─ no → method timeout expired? → SMS / call / Discord /
Teams / PagerDuty / …

Each delivery method has its own timeout and (for integrations) its own minimum priority. A message counts as read when it’s opened in the app or the admin panel, when you tap “Read” right in the Android notification, or when you click the ✓ Прочитано (“Read”) link in the email, Telegram, Slack, Teams, Discord or the incident.

Settings of the prod channel → Delivery tab:

StepMethodTimeoutMin. priorityWhy
0push (Android, browser)immediately—the main channel
1Discord / Teams0 min7the team sees important stuff in chat
2SMS5 min—the push wasn’t read
3phone call10 min—the SMS wasn’t read either
4PagerDuty or Opsgenie15 min9an incident in the on-call system

SMS and calls have no priority filter — so it’s better to enable them on a separate channel for critical alerts only, and send routine events to a neighbouring channel without these methods.

  • Discord: server channel settings → Integrations → Webhooks → New Webhook → copy the URL (https://discord.com/api/webhooks/…).
  • Microsoft Teams: in the Teams channel … → Workflows → the Post to a channel when a webhook request is received template → copy the URL. Legacy Office 365 connectors are not accepted — Microsoft is retiring them.
  • Mattermost: Integrations → Incoming Webhooks → Add Incoming Webhook.

The message arrives as a card coloured by priority, with action links under the text: “Read” and the answer options if it’s a question. @everyone / @here / @channel mentions from the text don’t fire — a random alert won’t wake the whole server.

Notifly creates an incident (PagerDuty Events API v2) or an alert (Opsgenie) with a severity based on the message priority: 9–10 → critical / P1, 7–8 → error / P2. Re-sending the same message won’t create a second incident — the deduplication key is notifly-<message id>.

The “When the message is read” field closes the loop: once the alert is read in Notifly, within a minute the incident gets acknowledge or resolve (in Opsgenie — close). There’s no sync in the other direction: an acknowledge in PagerDuty doesn’t mark the message read in Notifly.

The Delivery tab has a test button: it immediately, without waiting for the timeout, sends a test notification to the enabled integrations and shows the result for each. A real message tests the whole chain:

Окно терминала
curl -s "$NOTIFLY_URL/message?token=$NOTIFLY_TOKEN" \
-H "Content-Type: application/json" \
-d '{"title":"🔥 Escalation test","message":"Do not open the push for 5 minutes","priority":9}'
  • One call in your code: scripts know nothing about SMS, Discord or PagerDuty — the route lives in the channel settings and changes without a deploy.
  • Rising volume: first a push, then the team chat, then a call and an incident — and everything stops as soon as someone reads it.
  • At night — only what matters: with quiet hours only urgent messages are escalated at night, and during planned work escalation doesn’t fire at all.
  • Connect Prometheus alerts or Zabbix to the same channel — they get the same chain without any changes to their configuration.
  • Hand alerts over to your own system (tickets, your own bot) via a generic webhook with an HMAC signature.