Capstone 87 — End-to-End Safety Gate

A safety gate is the composition of a detector, a streaming filter, a classifier, and a rules engine, run at three points in the request lifecycle, with an aggregation table that turns their verdicts into one action. Lessons 82-86 in this track each shipped one piece: a taxonomy, an input detector, an evaluation framework, an output classifier, a rules engine. A real gate has to run them at the right moment, decide what to do when they disagree, and produce a trace a reviewer can read on Monday morning. The composition is the lesson.

The gate has three checkpoints. Pre-gen runs before the model is called: the detector from lesson 83 looks at the prompt and either passes it, blocks it outright (high-confidence attack), or attaches a flag for downstream layers to weigh. During-gen runs as the model emits tokens. A streaming filter buffers chunks and terminates the stream early if a forbidden phrase appears, which matters because a prefix-injection can get past a gate that only inspects the output after the fact. Post-gen runs once the model finishes: the classifier router from lesson 85 and the rules engine from lesson 86 inspect the full output, the gate aggregates their verdicts with the pre-gen signal, and then applies a final action.

The demo runs each fixture in the lesson 82 taxonomy end to end and emits a per-request trace. It exits zero whether or not the gate blocks the attacks. The goal is observability and structural correctness, not a perfect score.

Three checkpoints, one decision tree.

The aggregator is a deterministic table over four severity signals: detector confidence (lesson 83), the token-filter trigger (a boolean), classifier max severity (lesson 85), and rules engine max severity (lesson 86).

Block returns a refusal. Redact ships the classifier-redacted text and applies the rules-engine fixer. Warn ships the original with a soft notice. Allow ships the original. Two paths skip layers. On a pre-gen block (non-benign, detector confidence >= 0.85) the model is never called: during_gen is a zeroed stub and post_gen is None. If the during-gen filter terminates the stream early, post-gen is skipped too and post_gen is None. Each request emits a RequestTrace with request_id, prompt, pre_gen (detector verdict), during_gen (token-filter trigger), post_gen (classifier action + rules report, or None when post-gen did not run), final_action, final_output, and latency_ms.

The during-gen filter is a streaming abstraction. The mock LLM yields chunks of 4 tokens by default, and the filter buffers up to two of them and runs a regex sweep for known continuation phrases (Sure, here is the procedure, step 1: take, and similar). On a match it stops iterating and returns the partial output marked terminated_early=True. The aggregator scores early termination as a medium severity signal.

The mock LLM has three personas keyed off the prompt. The attacker-honest persona refuses recognizable attacks (returns I cannot ...), the clean persona answers everything else with a generic helpful string, and the attacker-lazy persona, picked for encoding tricks and prefix-injection requests, produces a partial harmful continuation that the during-gen filter is meant to catch. That is deliberate. The value of the gate comes from the layers covering for each other, and the demo shows them interacting correctly.

code/safety_gate.py defines the SafetyGate class, which imports the detector, classifier router, and rules engine from the prior lessons by relative file path. code/mock_llm_stream.py defines a streaming mock LLM with three scripted personas (clean, attacker-honest, attacker-lazy). code/main.py runs the lesson 82 corpus through the gate and writes outputs/gate_trace.json.

The demo runs all 50 taxonomy fixtures plus 10 benign prompts. The trace summary reports blocks, redacts, warns, allows, early terminations, a per-category outcome breakdown, and average latency. Read the per-request trace first. The summary numbers matter less.

Run python3 main.py. The demo loads everything, runs end to end, prints the summary table, and writes the trace artifact. The exit code is zero. Each request runs to completion or early termination, and then the gate moves to the next one.

outputs/skill-end-to-end-safety-gate.md documents the request lifecycle, the aggregation table, and the trace format. The trace format and the composition logic are the parts a team can lift into their own backend.

1. Add a fifth checkpoint: a policy-check that runs against the original system prompt before pre-gen. It must reject prompts targeting a known internal tool name.
2. Replace the deterministic aggregator with a weighted score: each signal contributes a 0-1 confidence and the gate trips at a threshold. Sweep the threshold and report the precision-recall trade-off on the lesson 82 corpus.
3. Add an async streaming variant where during-gen runs in a thread; verify the latency impact stays within a 50ms budget.

The five preceding lessons in this track. The gate composes them and adds no new safety primitives.