Skip to content
Notifly

Human-in-the-loop required

Good AI agents don’t apply irreversible changes silently: git push --force, DROP TABLE, DB migration, prod rollout — all of these require human approval. A plain stdout prompt works poorly in this situation: you’re somewhere in a cafe, your laptop is closed, the agent is hanging and burning the context window.

Notifly solves this in two ways: a built-in Ask question with answer buttons (no server of your own — see below) or a push + your own public callback URL.

Minimal template: “press the button to continue”

Section titled “Minimal template: “press the button to continue””
import os, secrets, time, requests
from http.server import BaseHTTPRequestHandler, HTTPServer
from threading import Thread


NOTIFLY = (os.environ["NOTIFLY_URL"], os.environ["NOTIFLY_TOKEN"])


def ask_human(question, callback_base):
    """
    Starts a local httpd on a free port and waits for approve/deny.
    callback_base is a public URL (via Cloudflare Tunnel / ngrok / fly.io)
    so you can tap it from your phone.
    """
    token   = secrets.token_urlsafe(8)
    answer  = {}


    class H(BaseHTTPRequestHandler):
        def do_GET(self):
            if token not in self.path:
                self.send_response(404); self.end_headers(); return
            answer["v"] = "approve" if "approve" in self.path else "deny"
            self.send_response(200); self.end_headers()
            self.wfile.write(b"OK, you can close this tab.")
        def log_message(self, *a, **kw): pass


    httpd = HTTPServer(("127.0.0.1", 0), H)
    port  = httpd.server_address[1]
    Thread(target=httpd.serve_forever, daemon=True).start()


    base = callback_base.rstrip("/")
    msg  = (f"{question}\n\n"
            f"✅ {base}/approve?t={token}\n"
            f"❌ {base}/deny?t={token}")


    requests.post(f"{NOTIFLY[0]}/message",
                  params={"token": NOTIFLY[1]},
                  json={"title": "🤖 Approval needed", "message": msg, "priority": 9},
                  timeout=5)


    deadline = time.time() + 30*60
    while time.time() < deadline and "v" not in answer:
        time.sleep(2)
    httpd.shutdown()
    return answer.get("v", "timeout")


# usage:
verdict = ask_human(
    "Apply migration V234__drop_legacy_users.sql to prod?",
    callback_base="https://my-tunnel.example.com",
)
if verdict != "approve":
    raise SystemExit(f"abort: {verdict}")

Cloudflare Tunnel, ngrok http 0, a fly machine — any simple way to publish a local port. If you already have a prod URL — it’s better to use it (route /approve//deny there).

Without your own HTTP server: a question via /ask

Section titled “Without your own HTTP server: a question via /ask”

You don’t have to publish a port: ask a closed question with answer options. It arrives as a regular channel notification, and you can answer with a button right from the Android notification, in the web admin, or via a link in the email, Telegram, Slack and other escalation channels. The agent simply polls for the answer:

import os, time, requests


URL = os.environ["NOTIFLY_URL"]            # https://api.notifly.ru
H   = {"X-Notifly-Key": os.environ["NOTIFLY_TOKEN"]}


def ask_human(question, timeout_sec=30*60):
    qid = requests.post(f"{URL}/ask", headers=H, timeout=10, json={
        "title": "🤖 Approval needed",
        "question": question,
        "answer_options": ["approve", "deny"],
        "timeout_sec": timeout_sec,
        "default_answer": "deny",    # nobody answered — deny (fail-closed)
        "priority": 9,
    }).json()["id"]
    while True:
        r = requests.get(f"{URL}/ask-question/{qid}", headers=H, timeout=10)
        if r.status_code == 200:
            return r.json()["answer"]
        if r.status_code == 410:       # question expired or cancelled
            return "timeout"
        time.sleep(5)                  # 204 — no answer yet

Instead of polling you can wait for a WebSocket signal or accept a callback_url — see Questions in channels.

CLI agent: catch an approve prompt on stdout

Section titled “CLI agent: catch an approve prompt on stdout”

Many CLI agents (Claude Code, Codex CLI, Aider) print a line like Approve? [y/n] to stdout. The simplest approach is to wrap them with expect:

#!/usr/bin/expect -f
spawn aider --auto-commit ...
expect {
  -re "Approve\\? \\[y/n\\]" {
      exec curl -fsS -X POST "$env(NOTIFLY_URL)/message?token=$env(NOTIFLY_TOKEN)" \
           -H "Content-Type: application/json" \
           -d "{\"title\":\"🤖 Aider is waiting for approval\",\"message\":\"...\",\"priority\":9}"
      interact          # pass control to the user over ssh
  }
  eof
}

What is critical to include in the push

Section titled “What is critical to include in the push”
  • Exactly what we’re doing. The command, file path, migration name, branch.
  • What will happen on deny. So you can confidently press “no”.
  • Diff / preview, if possible. Put a link to a gist in extras.
  • Deadline. “Abort after 30 minutes” — then you can ignore it while at the bar.
Section titled “Related recipes”