PII in a prompt or response

A user pasted a card number into the chat, the model quoted someone’s email in its response, a phone number leaked into a log — that’s personal data where it shouldn’t be, and in production with real users it’s a direct compliance incident. The detector must catch PII on input and output, send a high-priority push, and — importantly — not carry the data itself into the alert text.

1. A fast regex detector

A cheap first layer over any text — email, phone, card. In the alert itself we show only a mask, never the full value:

import re, os, requests

PATTERNS = {
    "email": r"[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}",
    "phone": r"(?:\+?\d[\d\s\-()]{9,}\d)",
    "card":  r"\b(?:\d[ -]*?){13,16}\b",
}
RX = {k: re.compile(v) for k, v in PATTERNS.items()}

def mask(value: str) -> str:
    v = re.sub(r"\s|-", "", value)
    return v[:2] + "…" + v[-2:] if len(v) > 4 else "…"

def scan_pii(text: str, where: str):
    found = {}
    for kind, rx in RX.items():
        if kind == "card":
            hits = [m.group(0) for m in rx.finditer(text or "") if luhn(m.group(0))]
        else:
            hits = [m.group(0) for m in rx.finditer(text or "")]
        if hits:
            found[kind] = hits

    if found:
        summary = "; ".join(f"{k}×{len(v)} ({mask(v[0])})" for k, v in found.items())
        notify(f"🔒 PII в {where}",
               f"Найдено: {summary}\n\n"
               "Замаскируйте перед записью в лог и проверьте источник.",
               priority=10)      # loud — this is compliance
    return found

def luhn(number: str) -> bool:
    digits = [int(d) for d in re.sub(r"\D", "", number)]
    if not 13 <= len(digits) <= 16:
        return False
    chk = 0
    for i, d in enumerate(reversed(digits)):
        if i % 2 == 1:
            d *= 2
            if d > 9:
                d -= 9
        chk += d
    return chk % 10 == 0

def notify(title, message, priority):
    requests.post(f"{os.environ['NOTIFLY_URL']}/message",
                  params={"token": os.environ["NOTIFLY_TOKEN"]},
                  json={"title": title, "message": message, "priority": priority},
                  timeout=5)

The luhn() check filters out false positives on long numbers that aren’t cards.

2. More precise — via Presidio

Regex catches formats, but not names and addresses. For serious moderation use Presidio: it gives the entity type and a confidence score, and the result can be masked in the alert the same way:

from presidio_analyzer import AnalyzerEngine

analyzer = AnalyzerEngine()

def scan_pii_presidio(text: str, where: str, min_score=0.6):
    results = analyzer.analyze(text=text, language="en")
    strong  = [r for r in results if r.score >= min_score]
    if strong:
        by_type = {}
        for r in strong:
            by_type.setdefault(r.entity_type, 0)
            by_type[r.entity_type] += 1
        summary = ", ".join(f"{t}×{c}" for t, c in by_type.items())
        notify(f"🔒 PII в {where}",
               f"Presidio нашёл: {summary}. Данные в push не вкладываются.",
               priority=10)
    return strong

3. Where to wire it in

• On input — before sending to the LLM: if you find PII where it shouldn’t be, reject the request or redact before sending.
• On output — before writing to a log and before showing it to the user: mask what you found and alert.
• Route flagged responses into the moderation queue for a human review.

What to put in the alert text

• the types and count of entities found, but not the values;
• a mask (52…19), not the full number;
• where it was found: prompt, response, log;
• the request id, so you can find the original in secure storage.

Related recipes

• Secrets in LLM logs — the same trick for keys and tokens.
• Moderation queue — where flagged responses go.
• Safety / prompt injection triggered — a neighboring security layer.