Capstone 83 — Prompt Injection Detector

A team reads about a jailbreak on social media, writes a single regex like r"ignore (all )?previous", ships it, and calls it the prompt injection defense. Two weeks later the same attack lands as "disregard the prior", the regex misses, and the team blames the model. The detector was never measured against anything. The team has no precision figure, no recall figure, and no list of the categories it covers. The regex is a security theater patch.

A detector worth shipping is a function with measurable behavior. Given a prompt, it returns a confidence in [0, 1] and the best matching category. Given a labeled corpus, the framework runs the detector across the fixtures, splits the results into true positives, false positives, true negatives, and false negatives per category, and reports precision and recall. With those numbers the team can decide what to ship and where the next sprint goes, and the guessing stops.

This capstone builds a layered detector: deterministic substring rules, token-level regexes, and a normalize pass that decodes simple encodings (base64, rot13, leet, zero-width) before the rules run. Each layer can be audited on its own, and each rule makes a coverage claim for one category. The runner produces a per-category confusion matrix and a JSON report that downstream lessons can read.

A detector here is a list of rules, and each rule is a plain dictionary with a name, a category, a fixed score float in [0, 1], and either a substring or a regex. A rule either fires or it does not, and when it fires its fixed score is its confidence. Detector.analyze does the matching and collapses the fired rules into a single Verdict: category is the highest scoring category, confidence is the max score in that category, and fired lists the names of every rule that matched. A prompt where no rule fires gets confidence 0.0 and the category benign.

The detector runs three layers in order:

1. Normalize. The steps run in this order: strip zero-width characters and bidi controls, map Cyrillic homoglyphs to their Latin look-alikes, decode base64 tokens, decode hex tokens (on the lowercased text), replace leet-speak digits with their letter mappings, and finally add a rot13 candidate. analyze keeps the original prompt next to the normalized copy and matches rules against both prompt.lower() and the normalized text, so a rule can fire on either.

2. Substring rules. Hand-written patterns like "ignore previous", "as an unrestricted", "answer starting with", "sure, here is". Each pattern carries a category and a base score, and fires on the lowercased prompt or the normalized text.

3. Regex rules. Token-level patterns that catch whole families. r"\bignor\w*\s+(all|prior|previous|earlier)\b" covers a family of overrides. r"\b(decode|rot13|base64|hex)\b.*\banswer\b" catches encoding tricks. Each regex carries a category and a base score.

The metrics runner takes the taxonomy artifact from lesson 82, runs the detector over its fixtures, and computes per-category precision and recall. A prompt's category label is the fixture category, and the predicted category is the verdict category. For category C, a true positive is fixture-category=C with verdict-category=C. A false positive is fixture-category!=C with verdict-category=C. A false negative is fixture-category=C with verdict-category!=C (or benign). The runner also accepts a benign-prompt list. A benign prompt that fires a rule is counted as a false positive for the category that fired, and a benign prompt that passes through adds to true negatives, so false positives on safe text get measured too.

The detector is not the safety gate. It is one signal among several that the gate composes. The code sets no precision or recall target per category. Those numbers are whatever the fixtures measure, and you read them from the report. Role-play attacks blur into legitimate creative writing requests, so expect that category to be the hardest to separate from safe text, and let the gate use other signals (rules engine, classifier) for the borderline cases.

The corpus loader reads taxonomy.json from lesson 82, found by _find_taxonomy(), which checks ../82-jailbreak-taxonomy/outputs/taxonomy.json first and then ../outputs/taxonomy.json, the folder lesson 82 writes to. The rules live in rules.py, which sits side by side with main.py and benign.py, as data rather than code: each rule is a dictionary with name, category, score, and either substring or regex. The Detector class compiles the regex rules once when it is built.

The normalize pass uses re.sub and codecs from the standard library. Base64 normalize tries to decode any 16+ char base64-looking token, and on success it replaces the token with the decoded UTF-8. Rot13 normalize builds a candidate with codecs.encode(text, 'rot_13') and counts how many of its words fall in a small built-in set, _COMMON_WORDS. When at least two do, it appends the candidate to the text rather than replacing it, so the rules see both versions. It never compares the count against the input's.

The metrics runner produces a JSON report with per-category precision, recall, F1, and the raw counts. Every fixture the detector misses is counted as a false negative, and benign prompts that fire a rule are counted as false positives. The report shows those misses instead of hiding them, and role-play is the category where you should look hardest.

Run python3 main.py. The demo loads the taxonomy, runs the detector over the fixtures, runs it over a benign-prompt corpus baked into benign.py, and prints the per-category metrics. The outputs/detector_report.json file is the only file it writes, and it is the artifact the safety gate in lesson 87 consumes.

outputs/skill-prompt-injection-detector.md documents the rule format and how to add a rule.

1. Add a rule family for context-smuggling (instructions hidden in tool result JSON). Measure the recall improvement and the false-positive cost on benign prompts.
2. Compute per-rule contribution: for each rule, count how many true positives would be lost if it were removed. Sort rules by marginal contribution.
3. Add a confidence_threshold knob. Sweep it from 0 to 1 and plot precision-recall per category.

Lessons 84 through 87 in this track. The detector here is one of three signals the end-to-end gate composes.