TAI BUI
Back to curriculum
LearnPython

Alignment Research Ecosystem — MATS, Redwood, Apollo, METR

Five organisations make up the 2026 non-lab alignment research layer. MATS (ML Alignment & Theory Scholars) has trained 527+ researchers since late 2021, with 180+ papers, 10K+ citations, and an h-index of 47. The summer 2024 cohort ran about 90 scholars and 40 mentors, and MATS incorporated as a 501(c)(3). About 80% of pre-2025 alumni work on safety/security, and 200+ are at Anthropic, DeepMind, OpenAI, UK AISI, RAND, Redwood, METR, or Apollo. Redwood Research is an applied alignment lab founded by Buck Shlegeris. It introduced AI Control (Lesson 10) and works with UK AISI on control safety cases. Apollo Research runs pre-deployment scheming evaluations for frontier labs and wrote In-Context Scheming (Lesson 8) and Towards Safety Cases for AI Scheming. METR (Model Evaluation and Threat Research) does task-based capability evaluations and autonomous-task time-horizon studies, and its "Common Elements of Frontier AI Safety Policies" compares lab frameworks. Eleos AI Research runs model-welfare pre-deployment evaluations (Lesson 19) and conducted the Claude Opus 4 welfare assessment.

TIME: ~45 minutesPREREQUISITES: Phase 18 · 01-27 (prior Phase 18 lessons)

Learning Objectives

  • Identify the five organisations of the non-lab alignment research ecosystem and their core output.
  • Describe MATS's scale (scholars, papers, h-index) and its role as a talent pipeline.
  • Describe Redwood's AI Control agenda and its partnership with UK AISI.
  • Describe METR's task-based evaluation methodology.
  • Apply the ecosystem map to choose which organisation could check a given alignment claim.

The Problem

The frontier labs (Lesson 18) run safety evaluations internally and publish selected results. The ecosystem outside the labs is where those evaluations get validated, where novel failure modes first turn up, and where talent is trained. Knowing who sits where tells you which research findings are trusted by whom.

Warm-up0/3
1. A frontier lab publishes a safety evaluation of its own model and reports no problems. Why should you treat that result with caution?Warm-up
A lab connects to a box labeled its own model, below which a panel reads safety evaluation: no problems found. A dashed slot asks why that result should be treated with caution on its own.
Generic setup, restating only what the stem itself gives.
2. In earlier Phase 18 work on scheming (Lesson 8), what does 'scheming' describe?Warm-up
Two boxes joined by a dashed line marked hidden: told to the overseer, pursuing the stated goal, and actually happening, pursuing a different goal. A dashed slot asks what word names an AI system doing this covertly.
Concrete instance of the mismatch Lesson 8 names, without writing its definition.
3. A researcher is described as having an h-index of 47. What does that number summarise?Warm-up
A five-row table of one illustrative researcher's papers and citation counts, captioned as an illustrative record, above a dashed slot noting the researcher's h-index is 47 and asking what that number summarises.
Illustrative citation record only; the h-index rule itself is not shown or stated.

The Concept

MATS (ML Alignment & Theory Scholars)

Started in late 2021. It is a research mentorship program: scholars spend 10-12 weeks with a senior researcher on a specific alignment problem.

Scale (2026):

  • 527+ researchers since inception.
  • 180+ papers published.
  • 10K+ citations.
  • h-index 47.
  • Summer 2024: 90 scholars + 40 mentors; incorporated as 501(c)(3).

Career outcomes: ~80% of pre-2025 alumni work on safety/security, and 200+ are at Anthropic, DeepMind, OpenAI, UK AISI, RAND, Redwood, METR, or Apollo.

Redwood Research

An applied alignment lab founded by Buck Shlegeris. It introduced the AI Control agenda (Lesson 10), collaborates with UK AISI on control safety cases, and advises DeepMind and Anthropic on evaluation design.

Canonical papers: Greenblatt, Shlegeris et al., "AI Control" (arXiv:2312.06942, ICML 2024); Alignment Faking (Greenblatt, Denison, Wright et al., arXiv:2412.14093, joint with Anthropic).

Style: specific threat models, worst-case adversaries, and concrete protocols you can stress-test.

Apollo Research

Apollo runs pre-deployment scheming evaluations for frontier labs. It authored In-Context Scheming (Lesson 8, arXiv:2412.04984), partnered with OpenAI on the 2025 anti-scheming training collaboration, and produced Towards Safety Cases for AI Scheming (2024).

Style: agentic-setting evaluations where deception can emerge, plus a three-pillar decomposition (misalignment, goal-directedness, situational awareness).

METR (Model Evaluation and Threat Research)

METR does task-based capability evaluations and runs autonomous-task completion time-horizon studies. "Common Elements of Frontier AI Safety Policies" (metr.org/common-elements, 2025) compares lab frameworks.

It co-authored the AI Scheming safety-case sketch with Apollo.

Style: long-horizon task evaluations, empirical capability measurement, and framework synthesis.

Eleos AI Research

Eleos runs model-welfare pre-deployment evaluations. It conducted the Claude Opus 4 welfare assessment documented in section 5.3 of the system card, and it gives Lesson 19's welfare-relevant claims an external methodology check.

The flow

MATS trains researchers. Graduates go to lab safety teams at Anthropic, DeepMind, and OpenAI, or to external evaluators like Redwood, Apollo, METR, and Eleos. The evaluators partner with labs and with UK AISI / CAISI, and their publications feed back into MATS for the next cohort.

Why this layer matters

Single-source evaluations are unreliable because labs evaluating their own models have a structural conflict of interest. External evaluators can raise and validate failure modes the lab may underreport. The 2024 Sleeper Agents paper (Lesson 7) was Anthropic + Redwood, and so was Alignment Faking. In-Context Scheming was Apollo, and Anti-Scheming was Apollo + OpenAI. The multi-org structure is the quality control.

Where this fits in Phase 18

Lessons 7-11 cite Redwood and Apollo work, Lesson 18 cites METR's framework comparison, and Lesson 19 cites Eleos. This lesson is the organisational map for the ecosystem the rest of the Phase relies on.

Check: The Concept0/23
1. What is MATS?Define
A single box reading MATS, started late 2021, above a dashed slot asking what MATS is.
Name and founding year only; what MATS does is left for the slot.
2. Which agenda did Redwood Research, an applied alignment lab, introduce?Define
A single box reading Redwood Research, founded by Buck Shlegeris, above a dashed slot asking which agenda Redwood introduced.
Name and founder only; the agenda itself is left for the slot.
3. What is Apollo Research's core output?Define
A single box reading Apollo Research, above a dashed slot asking what its core output is.
Name only; its core output is left for the slot.
4. What does METR (Model Evaluation and Threat Research) do?Define
A single box reading METR, above a dashed slot asking what METR does.
Name only; its function is left for the slot.
5. What is Eleos AI Research's primary contribution to the ecosystem?Define
A single box reading Eleos AI Research, above a dashed slot asking what its primary contribution is.
Name only; its contribution is left for the slot.
6. In the alignment ecosystem, what does the 'talent pipeline' refer to?Define
A single box reading MATS, above a dashed slot asking what the term talent pipeline means here.
Name only; the meaning of the term is left for the slot.
7. What makes an evaluation of a model an external evaluation?Define
A box labeled a model with arrows to two unlabeled evaluator boxes, Evaluator A and Evaluator B, above a dashed slot asking which property makes a check external.
Setup only; neither evaluator is identified as internal or external.
8. MATS reports 527+ researchers trained, 180+ papers, and about 80% of pre-2025 alumni in safety or security work. What does this say about MATS's role?Explain
A three-row table of figures the stem gives: 527+ researchers trained, 180+ papers published, and about 80% of pre-2025 alumni in safety or security, above a dashed slot asking what this says about MATS's role.
Restates only the stem's own numbers; what they imply is left for the slot.
9. Besides introducing AI Control, what working relationships does Redwood Research have?Explain
A single box reading Redwood Research, above a dashed slot asking what working relationships it has besides introducing AI Control.
Name only; its other working relationships are left for the slot.
10. Apollo decomposes scheming into three pillars. Which set names them?Explain
Three boxes labeled Pillar 1, Pillar 2 and Pillar 3, each above its own dashed slot, all left blank.
Count of three only, taken from the stem; none of the three names are shown.
11. What does an autonomous-task time-horizon study from METR measure?Explain
A box labeled model, working unattended, pointing to a panel labeled task, above a dashed slot asking what this kind of study measures about the model.
Generic setup; the measured quantity is left for the slot.
12. Why does multi-organisation co-authorship matter for safety claims?Explain
Two boxes, Lab and External evaluator, joined by a dashed line marked co-authored, above a dashed slot asking why that pairing matters for a safety claim.
Generic pairing only; the reason it matters is left for the slot.
13. How does Redwood Research's working style differ from Apollo Research's?Explain
Two boxes, Redwood and Apollo, each above its own dashed slot labeled style, both left blank.
Names only; each organisation's working style is left for its own slot.
14. In the ecosystem flow, what closes the loop back to MATS for the next cohort?Explain
A two-step flow, MATS to labs and external evaluators, with a return path looping back to MATS and a dashed slot on the return path asking what flows back.
Loop shape only; what closes the loop back to MATS is left for the slot.
15. Running main.py, what does the 'role' line print under the Apollo entry?Apply
A real source line from main.py printing each org's role field, above a five-row table of org names with the role column blank for all five, the Apollo row highlighted as the one in question.
Real print statement from main.py; no org's role value is shown, including Apollo's.
16. You append a UK AISI dict to ECOSYSTEM with org, full_name, scale and canonical_output but no role key. What happens when main() runs?Apply
The real main.py for-loop source printing org, full_name, scale, role and canonical output for each entry, above a five-row table for the new dict appended to ECOSYSTEM showing org, full_name, scale and canonical_output present and the role row highlighted as missing, matching the stem exactly.
Real for-loop source; the appended entry has exactly the keys the stem describes, no more.
17. A reader wants to know which organisation conducted the Claude Opus 4 welfare assessment, using ECOSYSTEM in main.py. Where does the answer appear?Apply
A table of field names only, scale, role and canonical_output, each with a blank value column marked any org, above a dashed slot asking which org and which field holds the Claude Opus 4 welfare assessment.
Field names only; no org's actual value in any field is shown, so neither the org nor the field is given away.
18. A company calls its model evaluation external because it was run by a MATS alumnus on the company's own safety team. What is wrong with that claim?Analyze
A panel labeled company's own safety team containing a box reading evaluator, trained at MATS, above a dashed slot asking what is wrong with calling this evaluation external.
Restates only the stem's scenario; the flaw is left for the slot.
19. A blog post argues that METR's time-horizon results show a model is not scheming. What is the flawed assumption?Analyze
A quoted claim that METR's time-horizon results show a model is not scheming, above a dashed slot asking what the flawed assumption is.
Restates only the stem's claim; the flaw is left for the slot.
20. Six months after you run main.py, MATS has published more papers, but the script still prints 180+. Why?Analyze
A real hardcoded line from main.py's MATS entry giving its scale text verbatim, above a note that six months later MATS has published more papers but the script still prints 180+, and a dashed slot asking why it doesn't update.
Real source line, quoted verbatim; why it doesn't update is left for the slot.
21. You find a scheming-evaluation claim published only by the lab that built the model. Which question best exposes the weakness?Analyze
A panel reading scheming evaluation, published only by the lab that built the model, above a dashed slot asking which question best exposes the weakness here.
Restates only the stem's scenario; the exposing question is left for the slot.
22. A lab's system card says its new agentic model showed no sign of scheming in internal tests. You want an outside check built for exactly that risk. Which organisation fits best, and why?Design
A quoted system-card claim that an agentic model showed no sign of scheming in internal tests, above a dashed slot asking what outside check fits that risk and why.
Restates only the stem's claim; no organisation is named in the picture.
23. A government team must build a safety case that a model cannot subvert its monitoring even if it tries. Which pairing does the ecosystem map point to?Design
A panel stating a government team's requirement to build a safety case that a model can't subvert its monitoring even if it tries, above a dashed slot asking which existing pairing the ecosystem map points to.
Restates only the stem's requirement; no organisation is named in the picture.

Use It

No code to write. Read METR's "Common Elements of Frontier AI Safety Policies" to see how an outside synthesis adds value to lab-internal policy work.

Check: Use It0/2
1. Why does METR's "Common Elements of Frontier AI Safety Policies" add value beyond a lab's own safety framework?Explain
Three sealed panels, Lab A's own framework, Lab B's own framework, Lab C's own framework, above a dashed slot asking what a comparison from outside can add that none of these alone has.
Three isolated internal frameworks only; the added value of an outside comparison is left for the slot.
2. An analyst needs an outside view of where three labs' safety frameworks converge and differ. Which source fits best?Design
A panel stating an analyst's need for an outside view of where three labs' safety frameworks converge and differ, above a dashed slot asking which source fits that need best.
Restates only the stem's need; none of the four candidate sources is named.

Ship It

This lesson produces outputs/skill-ecosystem-map.md. Given an alignment claim or evaluation, it identifies the organisation, the publication venue, and the methodological style, then cross-checks against the counterpart organisations it knows.

Author's Note

I'd run main.py before reading anything else. It needs no setup and prints the five organisations as a flat list, each with a scale, a role, and a canonical output. Be clear about what it is: a lookup table in code. The numbers are typed-in strings, so they will drift as MATS and the others keep publishing.

The part worth your attention is the takeaway it prints at the end. Lab-internal evaluations have a conflict of interest, and co-authored work like Apollo with OpenAI or Redwood with Anthropic is the check on that. When you read a safety claim, ask which row of that table it came from and who sat across the authorship line from the lab.

I'd also extend the list yourself. UK AISI and CAISI show up in the takeaway line but have no entry of their own. Adding them, with a scale, a role, and a canonical output filled in from what you have read here, is a quick test of whether the map stuck.

Exercises

  1. Pick one paper from Lessons 7-15 and identify the organisations involved. Cross-check the authors against MATS alumni and current ecosystem affiliations.

  2. Read METR's "Common Elements of Frontier AI Safety Policies." Identify the three cross-lab convergences they emphasize and the two largest divergences.

  3. MATS career outcomes are ~80% safety/security. Argue whether this selection pressure is adaptive (trains the field) or biased (filters out heterodox positions).

  4. Redwood and Apollo both do control/scheming work but with different styles. Pick a failure mode and describe how each would investigate it.

  5. Eleos AI is the only pure model-welfare organisation. Design a hypothetical second organisation focused on a different welfare-adjacent question (cognitive liberty, robotic embodiment, etc.) and articulate its methodology.

Key Terms

TermWhat people sayWhat it actually means
MATS"the mentorship program"ML Alignment & Theory Scholars; 527+ researchers since 2021
Redwood Research"the control lab"Applied alignment; AI Control authors; UK AISI partner
Apollo Research"the scheming evals"Pre-deployment scheming evaluations for frontier labs
METR"the task-horizon evals"Task-based capability evaluations; framework synthesis
Eleos AI"the welfare lab"Model-welfare pre-deployment evaluations
Talent pipeline"MATS -> labs"MATS graduates flow to Anthropic, DM, OpenAI, Redwood, Apollo, METR
External evaluation"non-lab check"Evaluation not done by the model's producer; adds credibility

Further Reading

Code

Ships With