Skip to content
Notifly

RAG retrieval returns nothing

Retrieval returned zero chunks above the similarity threshold — and the model answers “there’s no information in the provided context” or, worse, hallucinates. A single empty retrieval is normal (a query outside the knowledge base), but a spike in the empty rate almost always means something broke: reindexed into the wrong collection, embeddings changed, or the score_threshold was set too high. All HTTP statuses stay green.

1. Count the empty-retrieval rate in a window

Section titled “1. Count the empty-retrieval rate in a window”

Wrap the real retrieval and tag every query as empty or not.

import os, requests, redis


R         = redis.from_url(os.environ["REDIS_URL"])
KEY       = "rag-retrieval"      # "hit" / "miss"
WIN       = 300
MIN       = 40
THRESHOLD = 0.75                 # similarity threshold


def retrieve(query_vec, top_k=8):
    hits = vector_search(query_vec, top_k=top_k)        # your client
    good = [h for h in hits if h.score >= THRESHOLD]


    R.lpush(KEY, b"hit" if good else b"miss")
    R.ltrim(KEY, 0, WIN - 1)
    _maybe_alert()
    return good


def _maybe_alert():
    win = R.lrange(KEY, 0, WIN - 1)
    if len(win) < MIN:
        return
    miss = sum(1 for x in win if x == b"miss")
    rate = miss / len(win)
    # normal baseline of empty retrievals is 5–15%; alarm on a clear excess
    if rate > 0.35 and R.set("alert:retrieval-empty", "1", nx=True, ex=900):
        notify("🫥 RAG: empty retrieval",
               f"Queries with no context: {miss}/{len(win)} ({int(rate*100)}%).\n"
               f"Similarity threshold: {THRESHOLD}.\n\n"
               "Check: is it the right collection, does the embeddings model match, "
               "is the cutoff set too high?",
               priority=8)


def notify(title, message, priority):
    requests.post(f"{os.environ['NOTIFLY_URL']}/message",
                  params={"token": os.environ["NOTIFLY_TOKEN"]},
                  json={"title": title, "message": message, "priority": priority},
                  timeout=5)

2. A synthetic canary on known queries

Section titled “2. A synthetic canary on known queries”

Don’t wait for prod to degrade. Every 10 minutes a cloud function runs queries for which we know for sure the context is in the base:

CANARIES = [
    "how do I reset my password",
    "which payment methods do you support",
    "refund policy",
]


def handler(event, context):
    empty = []
    for q in CANARIES:
        vec  = embed(q)
        good = [h for h in vector_search(vec, top_k=8) if h.score >= THRESHOLD]
        if not good:
            empty.append(q)
    if empty:
        notify("🫥 Retrieval canary came back empty",
               "No context for queries that are known to be covered:\n"
               + "\n".join(f"• {q}" for q in empty),
               priority=9)
    return {"statusCode": 200}

If baseline queries stopped finding context, the index is almost certainly empty or incompatible, and that needs fixing immediately.

What to put in the alert text

Section titled “What to put in the alert text”
  • share of empty retrievals and the window size;
  • the current score_threshold;
  • collection name and embeddings model (cross-check with the drift check);
  • a couple of example queries left without context.
Section titled “Related recipes”