Critic Loop

A structured critique here is a score vector: one number each for clarity, novelty, evidence, methodology, and related-work. A freeform critic returns a paragraph of suggestions instead, and the next round's revision treats that paragraph as ambient context. Nobody can verify whether the rewrite addressed the criticism, because the criticism never had a shape.

Five dimensions give the harness a contract.

The harness records every dimension in every round. A revision that raises clarity but tanks evidence shows up as a regression on evidence in the trace, and the target rule refuses to call that success. The plateau check is coarser: it looks only at the mean. A model-only critic cannot offer either guarantee.

A Suggestion carries the dimension it improves, the section it targets, and an edit instruction the reviser can apply. The reviser is a callable too. The code here ships a deterministic reviser that maps each edit name to a fixed operation, and a model-driven reviser would read the same field as a prompt. The contract stays the same either way.

The critic loop terminates when any one of three conditions fires.

Target is the strictest rule: all five dimensions (clarity, novelty, evidence, methodology, related_work) must reach >= target_score (default 8.0) before the loop returns success. A high mean with one weak dimension does not count. Plateau compares the current round's mean to the previous round's mean. If no round-over-round gain exceeds plateau_epsilon (default 0.1) across plateau_window (default 2) consecutive comparisons, the loop exits with plateau. That needs three trace entries, so plateau can fire at round three at the earliest. Budget is a hard cap on rounds (default 5), and the loop exits with budget when it runs out.

The order matters: target beats plateau, and plateau beats budget. If round three hits the target on the same iteration that would also trigger a plateau, the result is target, not plateau.

Plateau detection waits for two consecutive flat comparisons because one flat step can be a revision that happened to change little. The shipped scorer, deterministic_score, is a pure function of the paper, so the same draft always gets the same scores. Any change in the mean comes from the paper changing, not from scoring noise. A drop in the mean counts as flat too, since only a gain larger than plateau_epsilon resets the check. Requiring two in a row means the draft has stalled across several revisions, not just one.

This lesson does not call a model. The shipped critic is a callable that scores a draft from structural signals on a MiniPaper: average section body length (clarity), figure and citation counts (evidence), an originality_tag field (novelty), and the presence of method and related-work sections (methodology, related-work). The reviser knows how to push each score upward.

clarity      grows when the average section body length increases
novelty      grows when originality_tag is set to "high"
evidence     grows when a section's figure_refs is non-empty
methodology  grows when a section titled "Method" exists with body
related-work grows when a section titled "Related Work" exists with body

The exact scoring rules are numeric. Evidence is min(10, 2 + 2 * figure_refs + 1.5 * cites), novelty is 3, 6, or 9 for a low, medium, or high tag, and methodology matches any section whose title starts with "method". The reviser maps each suggestion's edit name to one operation. add-method-section and add-related-work-section append a new MiniSection (or fill an empty body), expand-body and add-figure-and-cite append to an existing section, and bump-originality mutates the paper's originality_tag one step per call (low to medium to high). After round one the harness can watch the score go up, and the tests assert exactly that: the loop reduces the gap.

The harness owns the round counter, the trace, and the convergence check. The critic owns the score. The reviser owns the diff. None of the three touches the others' state.

The trace is the per-round event log. Each round emits one event with the round number, the score vector, the mean, the suggestion count, and the convergence verdict, and the full trace comes back alongside the final draft. A downstream dashboard can render it as a score-per-round chart. The next lesson, the iteration scheduler, reads the trace to decide whether the branch is worth keeping.

A budget is the max-iteration ceiling that stops a bad critic from running the loop forever. With the default settings, a critic whose suggestions never improve the score trips the plateau rule first: the trace shows three rounds, scores flat, verdict plateau. The budget verdict fires when the mean keeps rising by more than plateau_epsilon each round but never reaches the target, so the rounds simply run out. Either way the user reads a flat trace as a critic bug, not a draft bug. Surfacing only the final draft would hide the diagnosis, so the design surfaces the trace first.

code/main.py defines Critique, Suggestion, the Critic and Reviser protocols, CriticLoop, and a make_deterministic_critic_pair factory that returns the deterministic critic with a matching reviser. Minimal MiniPaper and MiniSection shapes are included so the lesson stands alone.

code/tests/test_critic_loop.py covers monotone improvement after round one, target convergence on a tuned draft, plateau detection when the mean is stable, a no-progress critic stopping by plateau or budget within three rounds, suggestion application by the reviser, and trace shape.

Two extensions a real implementation will want. The first is dimension weights: a workshop paper weights novelty higher than methodology, and a journal weights the inverse, so the convergence check becomes a weighted mean. The second is paired critics, where one critic scores and a second adjudicates the suggestions before the reviser sees them. Both add value, and both compose on the same Critique shape.

The bet is the score vector. Once the critique is structured, every other improvement (a new convergence rule, a dashboard, a paired critic) drops in without changing the loop.