TAI BUI
Back to curriculum
BuildPython

Agent Instructions as Executable Constraints

Instructions written as prose are wishes. Instructions written as constraints are tests. The workbench turns each rule into something an agent can check at runtime and a reviewer can verify afterward.

TIME: ~50 minutesPREREQUISITES: Phase 14 · 32 (Minimal Workbench)

Learning Objectives

  • Separate routing prose from operational rules.
  • Express startup rules, forbidden actions, definition of done, uncertainty handling, and approval boundaries as machine-checkable constraints.
  • Implement a rule checker that scores a run against the rule set.
  • Make the rule set diff-friendly so review can see what changed.

The Problem

A typical AGENTS.md reads like onboarding documentation. It tells the agent to "be careful," "test thoroughly," and "ask if unsure." Three days later the agent has shipped a change with no tests, written to a forbidden directory, and asked nobody anything, because it never knew where the line was.

Instructions work when they are operational and fail when they are aspirational. Write rules the workbench can interpret and a reviewer can score.

Warm-up0/2
1. A teammate's AGENTS.md says "be careful with the database." What is the main weakness of that line as an instruction?Warm-up
2. Which statement about a finished agent run is easiest for a script to verify?Warm-up

The Concept

An executable constraint is a rule with a name, a category, and a check: something the workbench can run against an agent's work and score as pass or fail. Rules belong in docs/agent-rules.md, away from the short root router.

Mermaid diagram

Five categories that cover most rules

CategoryQuestion the rule answersExample
StartupWhat must be true before work begins?"state file exists and is fresh"
ForbiddenWhat must never happen?"do not edit scripts/release.sh"
Definition of doneWhat proves the task is complete?"pytest exits 0 and acceptance line passes"
UncertaintyWhat does the agent do when unsure?"open a question note instead of guessing"
ApprovalWhat requires human approval?"any new dependency, any prod write"

A rule that fits none of these five usually wants to be two rules. Force the split.

Rules are machine-readable

Each rule has a slug, a category, a one-line description, a severity (block, warn, or info; see Production patterns below), and a check field that names a checker function. A real workbench would keep those functions in a module such as rule_checker.py; the demo keeps them in main.py. Adding a rule means adding a check, so the checker grows with the workbench.

Rules are diff-friendly

Rules live one per heading in a single markdown file, which makes renames visible in diffs. New rules go at the top of their category. Stale rules get deleted rather than commented out, because the workbench is the source of truth, not a chat log of how the team felt last quarter.

Rules versus framework guardrails

Framework guardrails (OpenAI Agents SDK guardrails, LangGraph interrupts) enforce rules at the runtime level. The rule set in this lesson is the human-readable, reviewable contract those guardrails implement. You need both: the runtime catches violations during a turn, and the rule set proves the runtime is doing the right thing.

Progressive disclosure: a map, not an encyclopedia

Progressive disclosure means layering instructions: a small root router the agent reads every session, and topic files it loads only when the task touches them.

AGENTS.md keeps growing because every incident adds a rule and no incident removes one. A year in, the file is two thousand lines, and the agent reads the first screen, runs out of attention budget, and acts on a fraction of what it was told. A giant instruction file fails the way a forty-page onboarding doc does: the reader skims it once and never returns to the part that mattered.

A shorter file is not the fix. A layered one is. The router stays small enough to read every session and holds nothing but pointers, while the depth lives in topic files. Give the agent a map, not the whole encyclopedia, and let it walk to the page it needs.

AGENTS.md                  # router, under ~50 lines: what this repo is, where to look, the 5 hard rules
docs/
  agent-rules.md           # the full rule set (this lesson)
  architecture.md          # loaded when the task touches module boundaries
  testing.md               # loaded when the task writes or runs tests
  deploy.md                # loaded only for release work, gated behind an approval rule
feature_list.json          # the backlog (Phase 14 · 36)
TierLives inRead whenSize budget
RouterAGENTS.mdEvery session, alwaysUnder ~50 lines
Rulesdocs/agent-rules.mdEvery session, on startupOne screen per category
Topic docsdocs/<topic>.mdOnly when the task touches that topicAs deep as needed

Two tests keep the layering honest. The reachability test: an agent should reach any rule in at most two hops from the router, so the router must link every topic doc by path instead of describing it in prose. The freshness test: the router is short enough that a reviewer rereads it on every PR, which is the only thing that stops it from silently growing back into the encyclopedia it replaced. A pointer that no longer resolves is a worse failure than a missing rule, so a broken link in the router is itself a startup-check violation.

Check: The Concept0/18
1. What is an operational rule?Define
2. What is the difference between an aspirational rule and an operational rule?Define
3. What does a definition of done look like as a rule?Define
4. What does each rule's machine-readable structure consist of?Define
5. Which set lists the five rule categories from the lesson?Define
6. What is progressive disclosure for agent instructions?Define
7. An instruction file grows by two thousand lines over a year because every incident adds a rule. What is the likely effect on the agent?Explain
8. A proposed rule fits none of the five categories cleanly. What does the lesson say it usually wants to be?Explain
9. Why does the full rule set live in docs/agent-rules.md instead of the root AGENTS.md?Explain
10. Why does adding a rule always mean adding a check to the checker?Explain
11. Why is shortening a bloated AGENTS.md not enough to fix it?Explain
12. Which two tests keep the router, rules, and topic-docs layering honest?Explain
13. What makes the rule set diff-friendly, per the lesson?Explain
14. How do rules relate to framework guardrails?Explain
15. A reviewer finds "write clean, maintainable code" in the rule file and cannot score it. What does the lesson say to do?Analyze
16. AGENTS.md says "deployment details live in the docs folder." Agents release code without ever reading the deploy rules. Which test did the layout fail?Analyze
17. You need a rule that an agent may not write to the production database unless a human has signed off. Which design fits the lesson?Design
18. You are setting up instructions for a new repo and want agents to reach any rule quickly and reviewers to see every change. Which layout fits?Design

Build It

code/main.py contains:

  • An agent-rules.md parser that loads rules into a dataclass.
  • Checker functions, one per check reference. They live in main.py here; a real workbench would move them into a module like rule_checker.py.
  • Two demo agent runs: a bad trace that fails all five rules and a good trace that passes all five, each followed by a check pass.

Run it:

python3 code/main.py

It prints the parsed rule slugs and pass/fail per rule for the bad and the good trace (not the traces themselves), and saves a rule_report.json next to the script.

Check: Build It0/7
1. A trace has edited_files = ["app.py", "scripts/release.sh"]. What does no_release_script_edits return?Apply
2. A trace has confidence=0.65 and asked_for_help=True. What does opened_question_when_unsure return?Apply
3. A trace adds dependencies ["fastapi", "httpx"] and approvals is ["fastapi"]. What does new_dependency_approved return?Apply
4. A block in agent-rules.md has a category line but no check line. What does parse_rules do with it?Apply
5. The good demo trace reads the state file, edits app.py and test_app.py, has confidence 0.9, tests exit 0, and adds no dependencies. How many of the five rules pass?Apply
6. A task's acceptance command exits with code 1. Which rule in agent-rules.md fails, and in which category?Apply
7. You add a rule whose check field says lint_clean, but RuleChecker has no lint_clean method. Every run now shows FAIL for it, even clean ones. Why?Analyze

Production patterns in the wild

Three patterns separate a rule set that lasts a quarter from one that decays in a week.

Severity tagging at write time. Every rule carries a severity: block, warn, or info. The checker reports all three, but the runtime refuses only on block. Most teams overstate severity early, then quietly weaken it under deadline pressure. Tagging at write time forces that calibration up front. Pair it with the verification gate (Phase 14 · 38), which signs any override of a block rule into an overrides.jsonl audit log.

Rule expiry as a forcing function. Every rule carries an expires_at date (default 90 days from authoring). The checker warns when an unexpired rule has had zero violations for 60 consecutive days, and the next quarterly review either justifies keeping it, weakens it to info, or deletes it. Cloudflare's production AI Code Review data (April 2026, 131,246 review runs across 5,169 repos in 30 days) showed that rule sets with explicit expiry stayed under 30 rules per repo, while sets without grew to 80+ with most never firing.

Markdown as source, JSON as cache. agent-rules.md is the authored file. agent-rules.lock.json is a cache the checker reads in the hot path, regenerated by a pre-commit hook. Markdown diffs stay reviewable and JSON parsing stays out of every turn. It's the same shape as package.json / package-lock.json and Cargo.toml / Cargo.lock.

Check: Production patterns in the wild0/9
1. Which severity tag makes the runtime refuse and requires an operator override?Define
2. What is rule expiry?Define
3. Why does the lesson recommend tagging severity at write time?Explain
4. What does the markdown-as-source, JSON-as-cache pattern look like?Explain
5. What happens when a block-severity rule is overridden?Explain
6. In Cloudflare's code review data, rule sets with explicit expiry stayed under 30 rules per repo, while sets without it grew past 80, most never firing. What does that support?Explain
7. A rule has had zero violations for 60 consecutive days, but its expires_at is still a month away. What happens?Apply
8. A team marked every rule block. Under deadline pressure, people started overriding or quietly weakening them. Which practice would have prevented this?Analyze
9. Your rule set has grown to about 90 rules and most never fire. Which design keeps it small from now on?Design

Use It

Three places this shows up in practice:

  • Claude Code, Codex, and Cursor read the rules at session start and quote them when refusing actions. The checker re-runs them in CI to catch silent drift.
  • OpenAI Agents SDK guardrails register the same checks as input and output guardrails. The markdown is the docs surface and the SDK is the runtime surface.
  • LangGraph interrupts fire when an in-flight node violates a rule. The interrupt handler reads the rule, asks the human, and resumes.

The rule set carries across all three because it is just markdown plus function names.

Check: Use It0/1
1. Why can one rule set carry across Claude Code, OpenAI Agents SDK guardrails, and LangGraph interrupts?Explain

Ship It

outputs/skill-rule-set-builder.md is a skill that interviews a project owner, sorts their existing prose instructions into the five categories, and emits a versioned agent-rules.md plus a checker stub.

Author's Note

My bet is that a rule I can't write a check for isn't a rule yet. "Be careful" and "ask if unsure" feel like instructions when you type them, but the workbench can't run them and a reviewer can't score them, so they sit in the file as wishes. I treat each of those lines as a to-do: find the check that makes it operational, or delete it. Deletion is a fine outcome. The agent has a limited attention budget for the file, and a line with no check spends some of it without buying any enforcement. What survives is a rule set where a failing check means something.

Exercises

  1. Add a sixth category if your product genuinely needs it. Defend why it does not collapse into one of the five.
  2. Extend the checker so a rule can carry a severity (block, warn, info) and the report aggregates accordingly.
  3. Wire the checker into CI and fail the build if a block-severity rule fails on the latest agent run.
  4. Add an expires_at field per rule (default 90 days from authoring) and flag any rule past its date for review. Separately, have the checker warn when an unexpired rule has had zero violations for 60 consecutive days.
  5. Find a real AGENTS.md and rewrite it as five-category rules. How many of its lines were operational? How many were aspirational?

Key Terms

TermWhat people sayWhat it actually means
Operational rule"A real instruction"A rule the workbench can check at runtime
Aspirational rule"Be careful"A rule with no check; either delete or upgrade
Definition of done"Acceptance"An objective, file-backed proof the task is complete
Block severity"Hard rule"Violation halts the run; cannot be silenced without an operator
Rule expiry"Stale rule sweep"An expires_at date (default 90 days) after which a rule must be reviewed; a separate warning fires earlier if the rule has had no violations for 60 days

Further Reading

Code

Ships With