BuildPython

Repo Memory and Durable State

Chat history is volatile and the repo is durable. The workbench keeps agent state in versioned files, so the next session, the next agent, and the next reviewer all read from the same source of truth.

TIME: ~60 minutesPREREQUISITES: Phase 14 · 32 (Minimal Workbench)

Learning Objectives

Define what belongs in repo memory and what belongs in chat history.
Author JSON Schemas for `agent_state.json` and `task_board.json`.
Build a state manager that loads, validates, mutates, and persists state atomically.
Use the schema to refuse bad writes before they corrupt the workbench.

The Problem

The agent finishes a session and the chat closes. The next session opens and asks where to start. The model says "let me check the files," reads stale notes, and re-does work that was already complete. Or it rewrites a finished file because nothing told it the file was finished.

The workbench fix is repo memory: state kept in JSON files in the repo, written under a schema, persisted atomically, and readable as a diff in code review. Chat is a transient feed. The repo is the system of record.

Warm-up0/3

1. An agent session ends and the chat window closes. Where should the next session look to learn which task was in progress?Warm-up

2. A program crashes while overwriting a data file with new contents. What is the main danger for whatever program reads that file next?Warm-up

3. A session stops midway through a task and a second session must continue it. What should the first session leave behind for it?Warm-up

Check: The Problem0/2

1. Why is chat treated as a transient feed instead of the record of progress?Explain

2. A new session reads old notes, decides a finished file is still unfinished, and rewrites it. Which ingredient was missing?Analyze

The Concept

Mermaid diagram

Repo memory is agent state stored in tracked files in the repo, under a schema. Only some information qualifies, so start with what goes in and what stays out.

What belongs in repo memory

Belongs	Does not belong
Active task id	Raw chat transcripts
Touched files this session	Token-level reasoning traces
Assumptions the agent made	"The user seemed frustrated"
Open blockers	Sampled completions
Next action	Vendor-specific model ids

The test is durability. Would this still be useful three months from now in a CI rerun? If yes, it goes in the repo. If no, it's telemetry.

Schema-first state

JSON Schema is the contract. Without one, every agent invents new fields, every reviewer learns a new shape, and every CI script special-cases past versions. With one, a bad write is a refused write.

The schema covers:

Required keys.
Allowed status values.
Forbidden values (e.g. null for arrays).
Pattern constraints (task ids match T-\d{3,}).
Version field for migrations.

Atomic writes

State writes have to survive partial failures, so write to a tempfile, fsync, and rename it over the target. The state file is the source of truth, and a half-written one is worse than no file at all.

Migrations

When the schema changes, ship a migration script alongside the schema bump. The state file carries a schema_version field, and the manager refuses to load a file from a version it cannot migrate.

Check: The Concept0/12

1. What is repo memory?Define

2. What does schema-first mean for agent state?Define

3. How does atomic write work?Define

4. What is a migration in the context of repo memory?Define

5. What does it mean to call the repo the system of record for the workbench?Define

6. Which schema constraints does the lesson name for agent state?Define

7. What is the durability test that decides whether a piece of information belongs in repo memory?Explain

8. What goes wrong when agent state has no schema?Explain

9. Why is a half-written state file worse than no file at all?Explain

10. What does the lesson say the manager does when it meets an unknown schema_version?Explain

11. A task on the board has status 'finished', which no tool understands, and CI scripts start special-casing it. Which schema feature would have refused the write?Analyze

12. You must rename `blockers` to `risks` while old state files exist in the repo. Which plan fits the lesson?Design

Build It

code/main.py implements the following:

agent_state.schema.json and task_board.schema.json.
A stdlib-only validator (subset of JSON Schema: required, type, enum, pattern, items).
StateManager.load and StateManager.commit, with atomic temp-and-rename writes. There is no update method: callers load the state, mutate the dict, and commit it back.
A demo that mutates state, persists, reloads, and proves the round-trip.

Run it:

python3 code/main.py

The script writes workdir/agent_state.json and workdir/task_board.json, activates T-001 in the state and commits it, prints the reloaded state and board, and finishes by showing a bad write (T-bogus) rejected before anything reaches disk.

Check: Build It0/7

1. Why does the state manager's commit validate the state before writing anything?Explain

2. In the demo, `bad` is a copy of the state with active_task_id set to 'T-bogus' and mgr.commit(bad) is called. What happens?Apply

3. The demo's initial state sets active_task_id to None. Does STATE_SCHEMA accept it?Apply

4. A task on the board has owner 'qa' and every other field is valid. What does board_mgr.commit do?Apply

5. A state dict carries every required key plus an extra key named 'notes'. What does validate do with it?Apply

6. A board task is committed with id 'T-12', everything else valid. What happens?Apply

7. Inside atomic_write, the write to the temp file raises an exception. What state is the target file in afterwards?Apply

Production patterns in the wild

Five patterns turn the lesson's minimum into something a multi-agent monorepo can survive.

Atomic temp-and-rename is not optional. A March 2026 Hive project bug report shows the failure mode cleanly. state.json was written via write_text() and the exceptions were caught and silenced. Partial writes left sessions resuming against corrupt state with no signal. The fix is always the same: tempfile.mkstemp in the same directory as the target, write, fsync, then os.replace (an atomic rename on POSIX and Windows). This lesson's atomic_write does exactly that.

Idempotency keys on every non-idempotent tool call. Say an agent crashes after calling a tool but before checkpointing the result. Recovery retries the call. That's safe for reads and dangerous for emails, DB inserts, and file uploads. Log every tool call ID into a pending_calls.jsonl before execution. On retry, check for the ID, and if it's there, skip the call and use the cached result. Anthropic and LangChain both call this out in 2026 guidance, and LangGraph's checkpointer persists pending writes for the same reason.

Separate large artifacts from state. Don't put CSVs, long transcripts, or generated files in agent_state.json. Save the artifact as its own file (or upload it to object storage) and keep only the path in state. Checkpoints stay small and fast while the artifacts grow independently.

Event sourcing for audit, snapshots for resume. Append to an event log (state.events.jsonl) on every mutation, and snapshot to state.json periodically. Resume reads the snapshot, then replays any events after the snapshot's timestamp. It costs more disk, but you can replay agent decisions verbatim, which matters when you're debugging long-horizon runs. Postgres uses the same shape internally for its WAL.

Schema migrations or refuse to load. The schema_version integer is the contract. When the manager meets a file at an unknown version, it refuses to read it. Ship a migration script next to the schema bump, and run tools/migrate_state.py idempotently on every startup.

Check: Production patterns in the wild0/7

1. Why are idempotency keys required for non-idempotent tool calls?Explain

2. Where should large artifacts (CSVs, long transcripts, generated files) live relative to state?Explain

3. What does event sourcing for audit plus snapshots for resume buy you?Explain

4. An agent crashes right after a tool call sends an email, before checkpointing. Recovery sends the email a second time. What was missing?Analyze

5. Resumed sessions keep reading corrupt state with no error signal. Which failure does the lesson cite as the cause?Analyze

6. agent_state.json has grown to tens of megabytes because the agent embeds generated reports in it, and checkpoints are slow. Which change fits the lesson?Design

7. A nightly CI agent must resume after a crash without redoing finished work or repeating side effects like emails. Which design fits the lesson?Design

Use It

Three places you'll meet this in production:

LangGraph checkpointers. Same idea, different storage. The checkpointer persists graph state to SQLite, Postgres, or a custom backend. This lesson's schema is what you reach for when the checkpointer dies and you need to read state by hand.
Letta memory blocks. Persistent blocks with structured schemas (Phase 14 · 08). The same discipline, scoped to long-running personas.
OpenAI Agents SDK session store. Pluggable backends, schema-aware. The state file in this lesson is the local-file backend.

Check: Use It0/1

1. How does the lesson describe LangGraph checkpointers relative to this lesson's schema?Explain

Ship It

outputs/skill-state-schema.md generates a project-specific JSON Schema pair (state and board), a Python StateManager wired to atomic writes, and a migration scaffold, so the next schema bump doesn't break the workbench.

Author's Note

My stance is schema-first: define the contract before the writer, and refuse drift. Once the schema exists, a bad write is a refused write, so drift gets caught at write time instead of at the next session's resume.

The same goes for versions. My bet is that a loud refusal is the right failure when the manager meets a schema_version it cannot migrate, with the migration shipped next to the schema bump. A half-written state file is worse than no file at all, so the manager should stop rather than load something it cannot vouch for.

Exercises

Add a last_human_touch timestamp. Refuse any agent write within five seconds of a human edit.
Extend the validator to support oneOf, so a task can be either a build task or a review task with different required fields.
Add a schema_version field and write the migration from v1 to v2 (rename blockers to risks).
Move the storage backend from a local file to SQLite while keeping the StateManager API identical.
Run two agents against the same state file with a 50 ms write race. What goes wrong, and how does the atomic rename save you?

Key Terms

Term	What people say	What it actually means
Repo memory	"Notes file"	State stored in tracked files in the repo, under schema
Schema-first	"Validate inputs"	Define the contract before the writer, refuse drift
Atomic write	"Just rename"	Write to temp, fsync, rename, so partial failures cannot corrupt
Migration	"Schema bump"	A script that turns vN state into v(N+1) state
System of record	"Source of truth"	The artifact the workbench treats as authoritative