Fallback to secondary model triggered

A good system with a fallback doesn’t crash — it silently switches to a backup model or provider and keeps answering. That silence is the problem. You find out the primary provider went down only when the expensive backup’s bill arrives or when answer quality sags. You’d like to know about the switch right away — but without panic, since users aren’t at risk right now.

The perfect case for a quiet low-priority push: “running degraded, reason such-and-such”.

Push on switching to the backup

We wrap the “primary → backup” chain: if the fallback triggered — count the switches in a window and send a quiet alert with the reason.

import os, time, collections, requests
import openai, anthropic

NOTIFLY_URL   = os.environ["NOTIFLY_URL"]
NOTIFLY_TOKEN = os.environ["NOTIFLY_TOKEN"]

WINDOW_SEC = 300      # window in which we accumulate the switch counter
COOLDOWN   = 300      # no more than one alert per 5 minutes

_falls = collections.deque()   # timestamps of fallback triggers
_last_alert = 0.0

primary  = openai.OpenAI()
backup   = anthropic.Anthropic()

def notify(title, msg, prio):
    requests.post(f"{NOTIFLY_URL}/message",
                  params={"token": NOTIFLY_TOKEN},
                  json={"title": title, "message": msg, "priority": prio},
                  timeout=5)

def _on_fallback(reason):
    global _last_alert
    now = time.time()
    _falls.append(now)
    while _falls and _falls[0] < now - WINDOW_SEC:
        _falls.popleft()
    if now - _last_alert > COOLDOWN:
        _last_alert = now
        notify(
            "🟠 Работаем на резервной модели",
            f"Основной провайдер отвалился ({reason}). "
            f"Переключений за {WINDOW_SEC // 60} мин: {len(_falls)}.",
            priority=3,          # quiet: users are fine right now
        )

def ask(prompt):
    """Try the primary model; on failure — the backup, with a quiet notice."""
    try:
        r = primary.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "user", "content": prompt}])
        return r.choices[0].message.content
    except Exception as e:
        _on_fallback(f"{type(e).__name__}: {e}")
        r = backup.messages.create(
            model="claude-haiku-4-5", max_tokens=1024,
            messages=[{"role": "user", "content": prompt}])
        return r.content[0].text

print(ask("One sentence about reliability."))

priority=3 is a deliberately quiet level (the silent range is 0–3): the phone doesn’t ring, but the event is recorded. The counter over the window turns “one random switch” (usually noise) into a clear signal “the primary really went down, already N times in 5 minutes”.

Escalation: stuck on the backup for a long time

One switch is normal. But if the fallback holds for minutes, that’s already an incident: the backup is usually more expensive and/or weaker. Add a condition “share of fallback requests in the window > 50%” and raise it to priority=8 — now a loud alert “time to fix the primary, not live on the backup”.

The reason matters more than the fact

Put the parsed reason into reason, not just Exception: 529 overloaded (wait it out), insufficient_quota (see free-tier exhausted), timeout (see latency) — these are three different incidents, and they’re fixed differently.

What to put in the alert text

• which model you switched from and to;
• the parsed reason for the primary’s failure;
• the number of switches (or share of fallback requests) over the window;
• how much more expensive the backup is — to gauge the urgency of switching back.