Embedding cost spike

Embedding cost hides inside the overall LLM bill, yet it behaves differently: chat tokens track user traffic, while embedding tokens fire in bursts during a reindex. One accidental full reindex inside a loop, a forgotten for doc in docs: reindex(), or ingest retries — and overnight you re-vectorize the whole base several times over. Count embedding spend separately.

1. A dedicated embedding token counter

Wrap every embeddings call and accumulate tokens over a sliding window (say, one hour). Keep the state in Redis.

import os, time, requests, redis

R = redis.from_url(os.environ["REDIS_URL"])

# $ per 1M tokens — set for your embeddings model
PRICE_PER_1M = 0.02
HOURLY_BUDGET = 1.50            # alarm if an hour exceeds this

def track_embeddings(resp, model="text-embedding-3-small"):
    tokens = resp.usage.total_tokens
    bucket = f"emb-tokens:{int(time.time() // 3600)}"      # bucket for the current hour
    total  = R.incrby(bucket, tokens)
    R.expire(bucket, 7200)

    cost = total / 1_000_000 * PRICE_PER_1M
    if cost > HOURLY_BUDGET and R.set("alert:emb-cost", "1", nx=True, ex=1800):
        notify("💸 Всплеск стоимости embeddings",
               f"За текущий час: {total:,} токенов ≈ ${cost:.2f}\n"
               f"Бюджет: ${HOURLY_BUDGET:.2f}/час, модель: {model}.\n\n"
               "Проверьте: не крутится ли лишняя переиндексация или цикл ретраев.",
               priority=8)

def notify(title, message, priority):
    requests.post(f"{os.environ['NOTIFLY_URL']}/message",
                  params={"token": os.environ["NOTIFLY_TOKEN"]},
                  json={"title": title, "message": message, "priority": priority},
                  timeout=5)

The key idea is to not mix this counter with chat tokens: otherwise a vectorization spike drowns in the noise of user traffic.

2. Guard against re-vectorizing the same content

The most common source of waste is re-embedding chunks that are already indexed. Cache by content hash:

import hashlib

def embed_cached(text: str):
    h = hashlib.sha256(text.encode()).hexdigest()
    cached = R.get(f"emb-cache:{h}")
    if cached:
        return decode_vec(cached)
    vec = embed(text)                       # paid call
    R.set(f"emb-cache:{h}", encode_vec(vec), ex=30 * 86400)
    return vec

If spend doesn’t drop after enabling the cache, the content genuinely changes every run (usually a chunking bug), and that alone is a reason to push.

3. A daily digest on a schedule

Once a day a cloud function with a timer-trigger sends the daily total — so you see the trend, not just the fires:

def handler(event, context):
    hours = [int(R.get(f"emb-tokens:{int(time.time()//3600) - i}") or 0) for i in range(24)]
    total = sum(hours)
    notify("📊 Embeddings за сутки",
           f"Токенов: {total:,} ≈ ${total/1_000_000*PRICE_PER_1M:.2f}\n"
           f"Пик за час: {max(hours):,}",
           priority=3)
    return {"statusCode": 200}

What to put in the alert text

• tokens and dollars over the window, separate from chat;
• the embeddings model and price per 1M;
• the suspected source: reindex, ingest, retry loop;
• the cache hit-rate, if enabled.

Related recipes

• LLM API spend — the overall token budget.
• Vector index rebuild finished — the planned source of the spike.
• Document ingestion failures — retries that burn tokens.