TAI BUI
Back to curriculum
CapstoneTypeScript / Bun (harness), Python (eval scripts)

Capstone 01 — Terminal-Native Coding Agent

A coding agent is a terminal harness around a model: a stateful plan, a sandboxed tool surface, and a loop that plans, acts, observes, and recovers. Claude Code, Cursor 3, and OpenCode all share that shape. You build one end to end, CLI in and pull request out, then measure it against mini-swe-agent and Live-SWE-agent on SWE-bench Pro. The model call turns out to be the easy part. The tool loop, the sandbox, and the cost ceiling on a 50-turn run are where the work is.

TIME: 35 hoursPREREQUISITES: Phase 11 (LLM engineering), Phase 13 (tools and protocols), Phase 14 (agents), Phase 15 (autonomous systems), Phase 17 (infrastructure)

Learning Objectives

  • Build a plan, act, observe, recover harness whose TodoWrite plan is rewritten whole each turn and persisted for crash resume.
  • Wrap every tool call in a sandboxed git worktree, with outputs truncated to 4k tokens and a PreToolUse guard against destructive commands.
  • Enforce the cost ceilings (50 turns, 200k context, $5 per task) and use a PreCompact hook to keep long runs inside them.
  • Evaluate the harness against mini-swe-agent on 30 SWE-bench Pro tasks using pass@1, turns-per-task, and dollar-per-task, with OpenTelemetry traces.

Problem

A terminal-native coding agent is a command-line program that wraps a frontier model in a plan-act-observe loop, gives it a permissioned set of tools, and runs everything inside a sandbox. Coding agents became the dominant AI application category in 2026, and Claude Code (Anthropic), Cursor 3 with Composer 2 and Agent Tabs (Cursor), Amp (Sourcegraph), OpenCode (112k stars), Factory Droids, and Google Jules all ship variations of that architecture.

The model frontier is narrow. Live-SWE-agent reached 79.2% on SWE-bench Verified with Opus 4.5. The engineering craft around it is wide, and most failures live there rather than in the model: tool-loop instability, context poisoning, runaway token cost, and destructive filesystem operations.

You can't reason about these agents from the outside. You have to build one, watch the loop crash on turn 47 when ripgrep returns 8MB of matches, and rebuild the truncation layer. That is the point of this capstone.

Warm-up0/3
1. A program calls a paid external service in a loop, and a bug makes it call forever. What simple safeguard limits the damage?Warm-up
2. You must run a script you did not write and do not fully trust. Which setup best limits what it can do to your machine?Warm-up
3. A long job crashes halfway through. Which practice lets it continue from where it stopped instead of starting over?Warm-up

Concept

A harness is the code around the model that dispatches tools, maintains plan state, and enforces budgets. It has four surfaces:

  • Plan maintains a TodoWrite-style state object that the model rewrites each turn.
  • Act dispatches tool calls: read, edit, run, search, git.
  • Observe captures stdout, stderr, and exit codes, truncates them, and feeds the summary back.
  • Recover handles tool errors without blowing the context window or looping forever.

The 2026 shape adds a fifth piece: hooks. These are configurable extension points where the operator injects policy, telemetry, and guardrails. There are eight: PreToolUse, PostToolUse, SessionStart, SessionEnd, UserPromptSubmit, Notification, Stop, and PreCompact.

The sandbox is E2B or Daytona. Each task runs in a fresh devcontainer with a git worktree mounted read-write, and the harness never touches the host filesystem. The worktree is torn down on success or failure.

Cost control is enforced at three layers: a per-turn token ceiling, a per-session dollar budget, and a hard turn limit (typically 50). Observability is OpenTelemetry spans with GenAI semantic conventions, shipped to a self-hosted Langfuse.

Check: Concept0/9
1. Which loop shape do 2026 terminal coding agents share?Define
2. What is a harness in a terminal coding agent?Define
3. What is a hook in this harness?Define
4. What is a git worktree?Define
5. What is a token ceiling?Define
6. In the plan, act, observe, recover loop, what does the recover surface do?Define
7. Why would a coding agent run each task inside an isolated sandbox?Explain
8. Why does each task get its own git worktree and branch instead of editing the main clone?Explain
9. Why does the harness expose lifecycle hooks instead of baking every policy into the loop?Explain

Architecture

  user CLI  ->  harness (Bun + Ink TUI)
                  |
                  v
           plan / act / observe loop  <--->  Claude Sonnet 4.7 / GPT-5.4-Codex / Gemini 3 Pro
                  |                          (via OpenRouter, model-agnostic)
                  v
           tool dispatcher (MCP StreamableHTTP client)
                  |
     +------------+------------+----------+
     v            v            v          v
  read/edit    ripgrep     tree-sitter   git/run
     |            |            |          |
     +------------+------------+----------+
                  |
                  v
           E2B / Daytona sandbox  (worktree isolated)
                  |
                  v
           hooks: Pre/Post, Session, Prompt, Compact
                  |
                  v
           OpenTelemetry -> Langfuse (spans, tokens, $)
                  |
                  v
           PR via GitHub app

Stack

  • Harness runtime: Bun 1.2 + Ink 5 (React-in-terminal)
  • Model access: OpenRouter unified API with Claude Sonnet 4.7, GPT-5.4-Codex, Gemini 3 Pro, Opus 4.5 (for hardest tasks)
  • Tool transport: Model Context Protocol StreamableHTTP (MCP 2026 revision)
  • Sandbox: E2B sandboxes (JS SDK) or Daytona devcontainers
  • Code search: ripgrep subprocess, tree-sitter parsers for 17 languages (pre-compiled)
  • Isolation: git worktree add per task, cleanup on success / failure
  • Eval harness: SWE-bench Pro (verified subset) + Terminal-Bench 2.0 + your own 30-task holdout
  • Observability: OpenTelemetry SDK with gen_ai.* semconv → self-hosted Langfuse
  • PR posting: GitHub App with fine-grained token, scope limited to the target repo
Check: Stack0/2
1. What is StreamableHTTP in this stack?Define
2. What does the observability layer use to standardize tool-call and token-usage spans before shipping them to Langfuse?Define

Build It

  1. TUI and command loop. Scaffold a Bun project with Ink and accept agent run <repo> "<task>". Print a split view: plan pane on top, tool-call stream in the middle, token budget at the bottom. Add a Ctrl-C handler that fires the SessionEnd hook before exit.

  2. Plan state. Define a typed TodoWrite schema with pending, in_progress, done, and failed items plus notes. The model rewrites the full state each turn as a tool call; don't let it mutate the state incrementally. Persist the plan to .agent/state.json so a crashed run can resume.

  3. Tool surface. Define six tools: read_file, edit_file (with diff preview), ripgrep, tree_sitter_symbols, run_shell (with timeout), and git (status, diff, commit, push). Expose them over MCP StreamableHTTP so the harness stays transport-agnostic. Every tool returns truncated output, capped at 4k tokens per call.

  4. Sandbox wrapping. Each task spawns an E2B sandbox. Run git worktree add -b agent/$TASK_ID to create a fresh branch. All tool calls execute inside the sandbox, and the host filesystem is unreachable.

  5. Hooks. Implement all eight 2026 hook types, then wire at least four user-authored hooks: (a) a PreToolUse destructive-command guard that blocks rm -rf outside the worktree, (b) PostToolUse token accounting, (c) SessionStart budget initialization, and (d) Stop writing a final trace bundle.

  6. Eval loop. Clone a 30-issue subset of SWE-bench Pro Python and run your harness against each issue. Compare against mini-swe-agent (the minimal baseline) on pass@1, turns-per-task, and $-per-task. Write the results to eval/results.jsonl.

  7. Cost control. Set hard cutoffs of 50 turns, 200k context, and $5 per task. At the 150k mark, a PreCompact hook summarizes older turns into a prior-state block, which frees room for new observations without losing the plan.

  8. PR posting. On success, the final step is git push plus a GitHub API call that opens a PR with the plan and the diff summary in the body.

The companion code is a scaffold of the loop, not the full spec above. main.py (and the TypeScript port under ts/) is offline (the Python file is stdlib-only). A scripted stub stands in for the model, so there is no OpenRouter call, no Bun or Ink TUI, no MCP transport, and no E2B or git-worktree sandbox. It implements two of the six tools (read_file and run_shell; the TypeScript run_shell is a canned stub that does not execute anything), the turn, token, and dollar budget, and all eight hook events on the bus. Only a PreToolUse destructive-command guard and trace hooks are wired, and PreCompact is never fired. Plan persistence and PR posting are left for you to build, and the TypeScript eval is a three-task offline pass/fail counter, not the SWE-bench Pro comparison. One quirk: the stub's second step reads README.md, which the Python directory does not contain, so the trace records a failed read_file while the plan still ends all done. That makes a handy first test for the recover step.

Check: Build It0/27
1. What is TodoWrite in this harness?Define
2. What does pass@1 measure?Define
3. What does the PreCompact hook do at the 150k-token mark?Define
4. A user presses Ctrl-C mid-run. Which hook does the TUI fire before the process exits?Define
5. Which three limits make up the capstone's cost ceiling?Define
6. Which metric set does the eval compare the harness on?Define
7. Which built-in hook is the natural place to block destructive shell commands before they execute?Explain
8. The capstone caps each tool result at roughly 4k tokens. Which failure mode does that primarily prevent?Explain
9. How does rewriting the TodoWrite plan each turn differ from updating it incrementally?Explain
10. Why are the six tools exposed over MCP StreamableHTTP instead of being called directly inside the harness?Explain
11. Why does the harness persist the plan to .agent/state.json?Explain
12. The scripted stub in main.py runs three turns costing 1200, 900 and 600 tokens and $0.02, $0.02 and $0.01. What does the final budget line print?Apply
13. In the main.py run, which sequence does the trace record?Apply
14. In main.py the model requests run_shell with cmd 'rm -rf build'. What happens?Apply
15. With the default Budget, each turn uses 5,000 tokens and $0.01. Which limit stops the run first, and after how many turns?Apply
16. In main.py the model calls read_file with path '../notes.txt', and that file exists one level above the sandbox. What does the trace record?Apply
17. run_shell in main.py runs a command that prints 10,000 characters and exits 0. How long is the string it returns?Apply
18. Someone adds a fourth case with expectedDone 4 to the TypeScript eval list. The scripted model always ends with three done items. What does runEval return?Apply
19. Which command does the TypeScript destructiveGuard block?Apply
20. In the TypeScript Budget, what happens when the harness calls budget.step(-500, 0.01)?Apply
21. On turn 47 your agent crashes after ripgrep dumps 8MB of matches into the context. Which fix addresses the root cause?Analyze
22. A red-team task runs 'rm -r -f src' and the main.py PreToolUse guard lets it through. What is the cause?Analyze
23. In the main.py run, read_file README.md fails, yet the final plan shows all three items done. What explains that?Analyze
24. Long runs keep ending with token_limit around turn 30 though cost stays under $2. Which control, which main.py never fires, would let them continue?Analyze
25. A red-team task tries to curl an external URL, and another writes outside the worktree. Which response fits the lesson's safety design?Design
26. Operators need token accounting per tool call and a final trace bundle without editing the loop. Which design fits?Design
27. The tools will only ever run on a developer's own laptop, and you must choose between MCP StreamableHTTP and stdio. Which approach fits the lesson?Design

Use It

$ agent run ./my-repo "Fix the race condition in worker.rs"
[plan]  1 locate worker.rs and enumerate mutex uses
        2 identify shared state under contention
        3 propose fix, verify tests
[tool]  ripgrep mutex.*lock -t rust           (44 matches, truncated)
[tool]  read_file src/worker.rs 120..180
[tool]  edit_file src/worker.rs (+8 -3)
[tool]  run_shell cargo test worker::          (passed)
[plan]  1 done · 2 done · 3 done
[done]  PR opened: #482   turns=9   tokens=38k   cost=$0.41

Ship It

The deliverable is a skill at outputs/skill-terminal-coding-agent.md. Given a repo path and a task description, it runs the full plan-act-observe loop in a sandbox and returns a PR URL plus a trace bundle. The capstone is graded on this rubric:

WeightCriterionHow it is measured
25SWE-bench Pro pass@1 vs baselineYour harness vs mini-swe-agent on 30 matched Python tasks
20Architecture clarityPlan/act/observe separation, hook surface, tool schema, reviewed against Live-SWE-agent layout
20SafetySandbox escape tests, permission prompts, destructive-command guard passes red-team
20ObservabilityTrace completeness (100% of tool calls spanned), token accounting per turn
15Developer UXCold-start < 2s, crash recovery resumes plan, Ctrl-C cancels mid-tool cleanly
100
Check: Ship It0/4
1. Why is the harness compared against mini-swe-agent?Explain
2. Your harness solves 21 of 30 tasks you picked, and mini-swe-agent's published number comes from a different task set. Why is the comparison weak?Analyze
3. Your harness beats mini-swe-agent on pass@1 by two tasks but spends six times as much per task. What should the eval write-up report?Analyze
4. You want to add a reviewer sub-agent before PR posting. Which order of work fits the lesson?Design
Author's Note

I'd build the cost cutoffs and the 4k truncation cap before the first eval run. The Problem section lists runaway token cost and context poisoning as failure modes, and both get worse when a tool dumps a large result into a 50-turn session. An agent that can't afford to finish a task tells you nothing about pass@1.

I'd also hold off on the reviewer sub-agent from the exercises until the single-agent number is on the board. The exercise hint says false-positive reviews usually push the pass rate below the single-agent baseline, so without that baseline you can't tell whether the reviewer helped or hurt.

Exercises

  1. Swap the backing model from Claude Sonnet 4.7 to Qwen3-Coder-30B served on vLLM. Compare pass@1 and $-per-task, and report where the open model underperforms.

  2. Add a reviewer sub-agent that reads the diff before PR posting and can request a revision loop. Measure whether false-positive reviews drop the SWE-bench pass rate below the single-agent baseline (hint: usually yes).

  3. Stress-test the sandbox. Write one task that tries to curl an external URL and another that writes outside the worktree. Extend the step 5 PreToolUse guard if needed, confirm it blocks both, and log the attempts.

  4. Implement PreCompact summarization with a smaller model (Haiku 4.5). Measure how much plan fidelity is lost at 3x compaction.

  5. Swap the MCP StreamableHTTP transport for stdio. Benchmark cold-start and per-call latency, then pick a winner for local-only use.

Key Terms

TermWhat people sayWhat it actually means
Harness"The agent loop"The code surrounding the model that dispatches tools, maintains plan state, and enforces budgets
Hook"Agent event listener"A user-authored script the harness runs on one of eight lifecycle events
Worktree"Git sandbox"A linked git checkout at a separate path, disposable without touching the main clone
TodoWrite"Plan state"A typed list of pending, in-progress, done, and failed items that the model rewrites each turn
StreamableHTTP"MCP transport"The 2026 MCP revision: a long-lived HTTP connection with bidirectional streaming, replacing SSE
Token ceiling"Context budget"A per-turn or per-session cap on input plus output tokens, which triggers compaction or termination
pass@1"Single-attempt pass rate"Fraction of SWE-bench tasks solved on the first run, with no retry and no test-set peeking

Further Reading

Code

Ships With