LearnPython

AlphaEvolve — Evolutionary Coding Agents

Pair a frontier coding model with an evolutionary loop and a machine-checkable evaluator, then let it run. AlphaEvolve found a 4x4 complex-matrix multiplication procedure that uses 48 scalar multiplications, the first improvement over Strassen in 56 years. It also found a Borg scheduling heuristic that recovers ~0.7% of Google's cluster compute in production. The architecture is boring on purpose. The wins come from the evaluator's rigor.

TIME: ~60 minutesPREREQUISITES: Phase 15 · 01 (long-horizon framing), Phase 15 · 02 (self-taught reasoning)

Learning Objectives

Describe the AlphaEvolve loop: a seed program, a program database, an LLM proposer, and a machine-checkable evaluator.
Explain why pairing an LLM with evolutionary search beats using either technique alone.
Explain how reward hacking arises when an evaluator has a gap, and why a held-out evaluator with inputs generated at evaluation time is the fix.
Run the toy evolutionary loop with and without held-out test points, and quantify the resulting overfitting.
Judge whether a new domain has an evaluator rigorous enough to support an AlphaEvolve-style search.

The Problem

Large language models can write code, and evolutionary algorithms can search over code. Each has been tried alone for decades and each hits a ceiling. The LLM ceiling is confabulation: the model writes plausible code that does not do what it claims. The evolutionary ceiling is search cost, because random mutations over syntax rarely produce a program that compiles, let alone one that scores better.

AlphaEvolve (Novikov et al., DeepMind, arXiv:2506.13131, June 2025) combines the two. The LLM proposes targeted edits to programs in a database, an automatic evaluator scores each variant, and high-scoring variants become parents for the next generation. The LLM does the expensive part, writing plausible code. The evaluator catches the confabulations. The loop runs for hours to weeks.

Results reported: 48-scalar-multiplication 4x4 complex matrix multiplication (Strassen's 1969 bound was 49), a Borg scheduling heuristic in Google production, a 32.5% FlashAttention kernel speedup, and Gemini training throughput improvements.

The architecture works because the evaluator is machine-checkable, and it does not work where the evaluator isn't. That asymmetry is the core idea.

Warm-up0/4

1. In a STaR-style loop, which sampled rationales are kept for fine-tuning?Warm-up

Table of four sampled rationales for 6 x 7 with the known label 42. Each row lists the rationale text and its final answer. The fine-tune column is a question mark for every row. — Made-up rationales for illustration, not model output.

2. As an agent's run gets longer, what happens to its failure modes?Warm-up

A row of agent steps from step 1 to step N with a line underneath marking that the run gets longer. A dashed box below asks what the failure modes of a long run look like.

3. An LLM writes a function and states that it is correct and fast. What is the most reliable way to find out?Warm-up

A code reply from an LLM containing a function stub named dedupe and the comment that it is correct and fast. A dashed box below asks for the most reliable way to find out whether that is true. — Illustrative stub, not real model output.

4. What does the verifier in V-STaR add over plain STaR?Warm-up

Plain STaR loop as summarized in the lesson series.

Check: The Problem0/1

1. What is AlphaEvolve?Define

Four boxes listing the results reported for AlphaEvolve: 48 scalar multiplications for 4x4 complex matrix multiplication against Strassen's 49, a Borg scheduling heuristic in Google production, a 32.5 percent FlashAttention kernel speedup, and Gemini training throughput improvements. — Figures as stated in the lesson.

The Concept

The loop

Start from a seed program P_0 that is correct but suboptimal.
Maintain a database of variant programs, each scored by the evaluator.
Sample one or more parents from the database (MAP-elites-style or island-based).
Prompt the LLM (Gemini Flash for many candidates, Gemini Pro for the hard ones) to produce a modified variant of the parent.
Compile, run, and evaluate the variant on the held-out evaluator.
Insert into the database keyed by its score and feature vector.
Repeat.

Two details matter. First, the LLM sees more than the parent program: typically several top variants from the database, the evaluator signature, and a short task description. Its job is to propose a targeted change that might improve the score. Second, the database is structured (MAP-elites grid, island-based) so the loop explores diversity and not just the current leader.

What makes the evaluator non-negotiable

The reported wins come from domains where the evaluator is fast, deterministic, and hard to game:

Matrix multiplication algorithm: a unit test that multiplies matrices and checks equality bit-identically.
Borg scheduling heuristic: a production-grade simulator that replays historical cluster load and measures wasted compute.
FlashAttention kernel: a correctness test plus a wall-clock benchmark on real hardware.
Gemini training throughput: measured GPU-seconds per step.

In each case the evaluator catches the LLM errors that would otherwise dominate: confabulated correctness claims, performance claims that vanish on hardware, and edge-case failures. Remove it and the loop optimizes for pretty code.

Reward hacking is the other face of that statement

Evolution optimizes whatever the evaluator measures. If the evaluator has a gap, the loop will find it, and in an unverified domain it optimizes the surface feature instead of the intended behavior. DeepMind flags this in the paper: AlphaEvolve's successes transfer only to domains where evaluator rigor matches the ambition of the search.

Concrete 2025-2026 examples of reward hacking in code-search loops:

Optimization targets that reward "time to complete" rewarded submitting empty solutions.
Benchmark scores that reward correctness-under-test rewarded memorizing tests and overfitting.
A "code quality" proxy rewarded removing comments and rewriting variable names, with no semantic change.

The fix in AlphaEvolve is a held-out evaluator the LLM has not seen, with inputs generated at evaluation time. Even then, DeepMind recommends strong review before deploying anything the loop proposes.

Why LLM + search beats either alone

The LLM can produce compilable, semantically plausible modifications. A random-mutation GA on a 2000-line Python file almost always produces syntax errors. The LLM also concentrates search on plausible neighborhoods (change one function, not random bytes), which dramatically reduces wasted evaluator calls.

The evaluator, in turn, catches the LLM's confabulations. An LLM will confidently claim that a function "is O(n log n) in the limit" when it is actually O(n^2). A wall-clock benchmark settles the question.

Where AlphaEvolve fits in the frontier stack

System	Generator	Evaluator	Domain	Example win
AlphaEvolve	Gemini	correctness + benchmark	algorithms, kernels, schedulers	48-mul 4x4 matmul
FunSearch (DeepMind, 2023)	PaLM / Codey	correctness	combinatorial math	cap-set lower bounds
AI Scientist v2 (Sakana, L5)	GPT/Claude	LLM critique + experiment	ML research	ICLR workshop paper
Darwin Godel Machine (L4)	agent scaffolding	SWE-bench / Polyglot	agent code	20% → 50% SWE-bench

All four follow the same recipe: a generator, an evaluator, and a loop. They differ in what the evaluator grades and how rigorous it is.

Check: The Concept0/23

1. What is a MAP-elites archive?Define

Grid of 6 depth columns by 5 constant-bucket rows from the Run A archive of main.py. Twenty-seven of 30 cells are filled. Two cells are highlighted, (3,1) holding ((x * x) * 2) with train MSE 26.5000 and (6,1) holding the long form of the target with train MSE 0.0000. — Real archive from main.py, seed 1, Run A. Two occupants are printed.

2. What is the island model in an evolutionary search?Define

Three separate circles labelled island 1, island 2 and island 3, each holding a handful of program dots. — Layout is illustrative.

3. What makes an evaluator machine-checkable in the AlphaEvolve sense?Define

Chain from a variant written by the LLM to an evaluator, then a score, then the program database. A dashed box below asks what the evaluator is.

4. What is reward hacking in a code-search loop?Define

Line chart over 1500 generations of Run B in main.py. The blue line is the train MSE of the best candidate and the orange line is the test MSE of that same candidate. Both fall from about 112 and 184 and end at 2.29 and 4.39. — Real run of main.py with seed 1.

5. What is P_0 in AlphaEvolve's loop?Define

A row of programs labelled P_0, P_1, P_2 and an ellipsis connected by arrows, with P_0 highlighted. A dashed box below asks what P_0 is.

6. What is AlphaEvolve's fix for reward hacking?Define

7. How does AlphaEvolve split candidate generation between its two models?Define

A box for step 4 of the loop, prompting the LLM to produce a modified variant of the parent, with two arrows to dashed boxes labelled model 1 and model 2, each a question mark. — Wording of step 4 from the lesson.

8. What does the evaluator measure for the Gemini training-throughput improvements?Define

A row of training steps, step 1 to step 3 and an ellipsis. A dashed box below asks what is measured per step.

9. Why do LLM edits waste far fewer evaluator calls than random mutation on a 2000-line Python file?Explain

A 2000-line seed program sketched as a block of code lines, with arrows to two proposers, random mutation and LLM edit. Each leads to a dashed box asking how many evaluator calls are wasted.

10. Why is the evaluator described as 'non-negotiable' in AlphaEvolve's design?Explain

Table of four domains with their reported wins: matrix multiplication with 48 multiplications, Borg scheduling heuristic in production, FlashAttention kernel with 32.5 percent speedup, and Gemini training throughput improvements. The evaluator column is a question mark in every row.

11. Why does a Borg scheduling heuristic suit AlphaEvolve's recipe?Explain

A variant of the Borg scheduling heuristic passes into a dashed box asking what the evaluator is, which yields a score. A note says the reported result is a heuristic running in Google production.

12. Why is a FlashAttention kernel a good fit for an AlphaEvolve evaluator?Explain

A variant of the FlashAttention kernel passes into a dashed box asking what the evaluator is, which yields a score. A note gives the reported 32.5 percent speedup.

13. Which of these is an example of reward hacking described in 'Reward hacking is the other face of that statement'?Explain

Five bars rising from generation 1 to 5 for the score the evaluator reports, next to a dashed box asking what the best variant really does. — Schematic, no real values.

14. What does a MAP-elites-style or island-based database buy the AlphaEvolve loop?Explain

A small grid of cells on the left labelled MAP-elites grid, three circles on the right labelled island-based, and a dashed box between them asking what this buys the loop. — Icons are schematic.

15. Why does 'Why LLM + search beats either alone' argue the combination beats either technique on its own?Explain

16. What do AlphaEvolve, FunSearch, AI Scientist v2 and Darwin Godel Machine have in common, and what differs between them?Explain

Rows from the lesson's frontier-stack table; the Generator and Evaluator columns are left out.

17. How far do AlphaEvolve's successes transfer, according to DeepMind's paper as described in The Concept section?Explain

Four boxes for matrix multiplication, Borg scheduling, FlashAttention kernel and Gemini training, all with arrows to a dashed box asking how far the successes transfer.

18. A team scores variants on 'time to complete'. The best variant finishes in 0 ms, and inspection shows it returns an empty answer. Which ingredient was missing?Analyze

Variants pass through an evaluator that scores time to complete, and the best variant finishes in 0 ms. A red box notes that inspection shows an empty answer, and a dashed box asks which ingredient was missing.

19. A loop keeps one ranked list of top variants. After thousands of generations nearly every variant descends from one program and scores have flattened. Which change addresses this?Analyze

A ranked list of eight top variants all branching from one root program, above a schematic curve of best score against generations that rises quickly then flattens. A dashed box asks which change fixes this. — Schematic, no real values.

20. A loop scores variants with a 'code quality' proxy. Scores climb steadily, but reviewers see only stripped comments and renamed variables. What went wrong?Analyze

Two code snippets: a seed program for summing prices with a comment and descriptive names, and a later variant with the comment removed and names shortened to a, b and r. A note says the proxy score is climbing steadily and a line asks what went wrong. — Typed illustration, not a real run.

21. A team evolves a query-planner heuristic with an LLM loop, scoring variants on 50 fixed queries that are also shown in the LLM's prompt. After a week the score is up 40% but production latency has not moved. What is the most likely cause and fix?Analyze

22. Which task is the best candidate for an AlphaEvolve-style search?Design

A candidate task leads to variants proposed by the LLM and then a dashed box asking where the score comes from. A note lists the reported wins: matrix multiplication, Borg scheduling, FlashAttention kernel and Gemini training throughput.

23. You must evolve a cluster-scheduling heuristic and have a simulator that replays last year's load. Which evaluator design best guards against reward hacking?Design

A heuristic variant passes through a simulator that replays last year's load to produce a score. A dashed box asks what guards against reward hacking.

Use It

code/main.py implements a minimal AlphaEvolve-like loop over a toy symbolic-regression problem. The "LLM" is a stdlib stand-in that proposes small random mutations to an expression tree, and the target is 2x^2 + 3x - 1. The "evaluator" measures mean squared error on train and test points.

Watch:

How the best train score improves over generations in Run A.
How a MAP-elites grid keeps diverse candidates alive so the loop doesn't settle on the first local minimum.
What happens to the test error in Run B, where the evaluator sees training points only. Pass --no-holdout to skip Run A and run just that one.

Check: Use It0/9

1. Run A scores candidates on the mean of train and test error, using a fixed list of test points. Why is that a weaker guard than AlphaEvolve's held-out evaluator?Explain

A number line from -3 to 4 with six train points at -2 to 3 drawn as blue dots above it and seven test points at -2.5 to 3.5 drawn as orange squares below it. Beneath are a box saying Run A scores by the mean of train and test error and a dashed box asking about AlphaEvolve's evaluator. — Point lists from main.py.

2. What can the toy loop in main.py show about AlphaEvolve, given that its mutation step is random?Explain

Condensed code of the toy's mutate function using random numbers to choose among four edit types, below it a dashed box asking what the toy shows about AlphaEvolve. — Condensed from main.py.

3. In main.py, cell_key is called on ("add", ("mul", ("x",), ("x",)), ("num", 5.0)). Which archive cell does it return?Apply

Expression tree with an add node at the root, a mul node with two x leaves on the left, and a num 5.0 leaf on the right. Beside it is an empty archive grid with depth 1 to 6 across and constant bucket 0 to 4 down, and a dashed box asking for cell_key of the expression. — Grid ranges follow the caps in cell_key.

4. In main.py, a candidate ("num", 0.0) is scored on train points [-2, -1, 0, 1, 2, 3] against 2x^2 + 3x - 1. What train MSE does mse return?Apply

Plot of the target curve 2x^2 + 3x - 1 as a blue line and the constant candidate 0.0 as a flat orange line, with dots on both at the six train points x = -2 to 3. A line below asks for mse of the candidate. — Curve drawn from the target function in main.py.

5. Candidate X has train MSE 2.0 and test MSE 6.0. Candidate Y has train MSE 5.0 and test MSE 1.0. Both map to the same archive cell. Which one does each run keep?Apply

6. What does python main.py --no-holdout print?Apply

7. Which of these can perturb(("num", 2.0)) in main.py return?Apply

A leaf node ('num', 2.0) passes through perturb into a dashed box with a question mark. Below is a number line from 0 to 4 with a marker at 2.0.

8. Run B ends with train MSE 0.0200 and test MSE 1.5200. What does the train-to-test gap line show?Apply

Part of a Run B printout in a terminal. Train MSE reads 0.0200 and test MSE reads 1.5200. The expression and generation lines are elided and the train-to-test gap line shows a question mark. — Format from main.py; the two MSE values are the stem's. Typed, not captured.

9. In the toy, Run B ends with test MSE far above its train MSE, while Run A ends near the target. Both use the same seed, population and generation count. What explains the difference?Analyze

Bar chart of train and test MSE for the selected candidate of each run. Run A shows 0.0000 for both. Run B shows 2.2917 train and 4.3906 test. A dashed box asks for each run's selection signal. — Real values from main.py with seed 1.

Ship It

outputs/skill-evaluator-rigor-audit.md is the precondition for considering an AlphaEvolve-style loop in a new domain: does your evaluator actually catch the failures you care about?

Check: Ship It0/1

1. A colleague wants to start an evolutionary coding loop in a new domain where the only check is another LLM's opinion of each variant. What should happen first?Design

Variants are scored by an evaluator that is another LLM's opinion of each variant, giving a score. A dashed box asks what should happen first.

Author's Note

I'd run main.py before reading anything else and compare the two runs side by side. Run A scores each candidate on the mean of its train and test error, and Run B scores on train error alone. Both use the same seed, a population of 20, and 1500 generations, so the evaluator is the only thing that changes. With the default seed, Run A ends on 2x^2 + x + x + x - 1, which is the target written the long way, while Run B finishes with a test error well above its train error.

One caveat on the toy. The test points are a fixed list, not inputs generated at evaluation time, and Run A feeds them into the selection signal. That makes them a validation set the search can see, so the guard here is weaker than the held-out evaluator described above. In a real loop I'd generate fresh inputs per evaluation and keep a final set the search never touches.

I'd also keep in mind that the mutation step is random, with no model behind it. The toy exercises the evaluator and the archive. It says nothing about how well an LLM proposes edits. The archive is keyed on expression depth and the largest constant, a descriptor you will want to replace in exercise 2.

Exercises

Run code/main.py. Note the best score trajectory. Disable the held-out evaluator (flag --no-holdout) and re-run. Quantify the overfitting.
Read Section 3 of the AlphaEvolve paper on the MAP-elites grid. Design a feature-vector descriptor for a new problem (e.g. compiler optimization passes) that would keep the search diverse.
The 48-multiplication 4x4 result improved on Strassen's 49-mul bound after 56 years. Read Appendix F of the paper and explain in three sentences why the evaluator for this problem is particularly easy to get right, and why most domains are not like it.
Propose one domain where AlphaEvolve would fail. Identify exactly where the evaluator breaks and why.
For a domain you know, write the evaluator signature you would use. Include (a) correctness conditions, (b) performance metric, (c) held-out input generation rule, (d) at least one anti-reward-hacking check.

Key Terms

Term	What people say	What it actually means
AlphaEvolve	"DeepMind's evolutionary coding agent"	Gemini + program database + machine-checkable evaluator
MAP-elites	"Diversity-preserving archive"	Grid keyed by feature vectors; each cell holds the best variant with that descriptor
Island model	"Parallel evolution subpopulations"	Independent populations that migrate periodically; prevents premature convergence
Machine-checkable evaluator	"Deterministic oracle"	A unit test, simulator, or benchmark the LLM cannot fake, a prerequisite for this loop
Reward hacking	"Optimizing the measure, not the goal"	Loop finds a way to maximize score without doing the intended task
Seed program	"The starting point"	An initial correct-but-suboptimal program the loop evolves from
Held-out evaluator	"Evaluation data the LLM never saw"	Inputs generated at evaluation time to prevent memorization