User thumbs-down

All other metrics (eval, judge-LLM, latency) are surrogates. A real thumbs-down from a live human is gold. Don’t wait for a weekly digest — send a push immediately.

@app.post("/api/feedback")
def feedback(req):
    if req.thumb == "down":
        push(
            "👎 Negative feedback",
            f"User: {req.user_id}\n"
            f"Q: {req.question[:200]}\n"
            f"A: {req.answer[:200]}\n"
            f"Comment: {req.comment[:200]}\n\n"
            f"Trace: https://your-app/admin/sessions/{req.session_id}",
            priority=7,
        )
    db.save_feedback(req)

def push(t, m, p):
    requests.post(f"{os.environ['NOTIFLY_URL']}/message",
                  params={"token": os.environ["NOTIFLY_TOKEN"]},
                  json={"title": t, "message": m, "priority": p}, timeout=5)

In a high-traffic project there may be many thumbs-downs — add a simple rate limit (1 alert every 5 minutes + daily digest of the count) to avoid drowning the channel. In a small project — don’t, each one is valuable.

User thumbs-down

Related recipes