Moderation queue backlog

Auto-moderation flags suspicious content, and a human reviews it. If the incoming stream outpaces the reviewers, the queue grows and a toxic response sits for hours before anyone looks. Catch two signals: the queue length and the age of the oldest unreviewed item. The second matters more — it’s about the SLA, not the volume.

1. A scheduled queue check

Every few minutes a cloud function with a timer-trigger looks at the length and the age of the head of the queue:

import os, time, requests

MAX_LEN       = 50               # alarm if more than this is waiting
SLA_MINUTES   = 30               # nothing should wait for review longer than this

def handler(event, context):
    pending = db_fetch_pending()          # list of dicts with a created_at field (epoch)
    n = len(pending)
    if n == 0:
        return {"statusCode": 200}

    oldest_age_min = (time.time() - min(p["created_at"] for p in pending)) / 60

    if oldest_age_min > SLA_MINUTES:
        notify("🛡️ Модерация: просрочен SLA",
               f"Самый старый элемент ждёт {oldest_age_min:.0f} мин "
               f"(SLA {SLA_MINUTES} мин).\nВ очереди: {n}.",
               priority=9)
    elif n > MAX_LEN:
        notify("🛡️ Очередь модерации растёт",
               f"В очереди: {n} (лимит {MAX_LEN}).\n"
               f"Самый старый: {oldest_age_min:.0f} мин.",
               priority=7)
    return {"statusCode": 200}

def notify(title, message, priority):
    requests.post(f"{os.environ['NOTIFLY_URL']}/message",
                  params={"token": os.environ["NOTIFLY_TOKEN"]},
                  json={"title": title, "message": message, "priority": priority},
                  timeout=5)

Age (SLA) takes priority over length: a small but stagnant queue is more dangerous than a large but quickly cleared one.

2. Deduplication until the queue is cleared

The check runs every 5 minutes, and the queue can stay full for an hour — you don’t want twelve identical pushes. A flag with a TTL:

import redis
R = redis.from_url(os.environ["REDIS_URL"])

def notify_once(title, message, priority, cooldown=1800):
    if R.set("alert:" + title, "1", nx=True, ex=cooldown):
        notify(title, message, priority)

3. Back it with a heartbeat: “are the reviewers even working?”

A separate quiet signal — not that the queue is long, but that it’s stopped being cleared (everyone left for the weekend, the worker hung). Have the code that processes an item ping a heartbeat on every review:

def on_item_reviewed(item):
    save_decision(item)
    requests.get(os.environ["MODERATION_PING_URL"], timeout=5)   # heartbeat

Set intervalSec to match the expected review pace — and if reviewing stalls, Notifly sends a push on its own, even while the queue is still within its limit.

What to put in the alert text

• the queue length and the limit;
• the age of the oldest item and the SLA;
• a breakdown by flag reason (toxicity, PII, spam);
• a link to the moderation dashboard via extras with a URL.

Related recipes

• LLM job queue is growing — the same trick for a task queue.
• PII in a prompt or response — a common reason for landing in the queue.
• Safety / prompt injection triggered — what fills the queue.