BuildPython

Failure Modes: Why Agents Break

MASFT (Berkeley, 2025) catalogs 14 multi-agent failure modes in 3 categories. Microsoft's Taxonomy documents how existing AI failures amplify in agentic settings. Industry field data converges on five recurring modes: hallucinated actions, scope creep, cascading errors, context loss, and tool misuse.

TIME: ~60 minutesPREREQUISITES: Phase 14 · 05 (Self-Refine and CRITIC), Phase 14 · 24 (Agent Observability Platforms)

Learning Objectives

Describe MASFT, the taxonomy that sorts 14 multi-agent failure modes into three categories.
Explain why agentic failure amplifies existing AI failure modes (bias, hallucination).
Describe the five industry-recurring modes and their mitigations.
Implement a stdlib detector that tags agent traces with failure-mode labels.

The Problem

Teams ship agents that work on 90% of traces. The other 10% isn't random noise. Those failures fall into a small number of recurring categories, and once you can name them you can monitor for them and fix them.

Warm-up0/3

1. In CRITIC, what makes a model's critique of its own answer more reliable than plain Self-Refine?Warm-up

2. Why trace each step of an agent run instead of logging only the final answer?Warm-up

3. An HTTP API replies to a request with status 400. What does that status mean?Warm-up

The Concept

MASFT (Berkeley, arXiv:2503.13657)

MASFT, the Multi-Agent System Failure Taxonomy, sorts 14 failure modes into 3 categories. Inter-annotator Cohen's Kappa is 0.88, so the categories are reliably distinguishable.

Its central claim: these failures are fundamental design flaws in multi-agent systems, not LLM limitations that a better base model will fix.

Microsoft Taxonomy of Failure Mode in Agentic AI Systems

Existing AI failures (bias, hallucination, data leakage) amplify in agentic settings.
Autonomy adds new ones: unintended action at scale, tool misuse, mission drift.
The whitepaper works as a risk register for agentic products.

Characterizing Faults in Agentic AI (arXiv:2603.06847)

Failures arise from orchestration, internal state evolution, and environment interaction. They are not just "bad code" or "bad model output."

LLM Agent Hallucinations Survey (arXiv:2509.18970)

The survey names two primary manifestations:

Instruction-following Deviation. The agent doesn't follow the system prompt.
Long-range Contextual Misuse. The agent forgets or misapplies context from earlier turns.

Sub-intention errors break down into Omission (a missed step), Redundancy (a repeated step), and Disorder (steps out of order).

The five industry-recurring modes

Field analyses from Arize, Galileo, and NimbleBrain (2024-2026) converge on five:

Hallucinated actions. The agent invokes a tool that doesn't exist or fabricates arguments.
Scope creep. The agent expands the task beyond the user's ask, creating extra PRs or sending extra emails.
Cascading errors. One wrong call triggers downstream effects. A phantom SKU hallucination triggers four API calls and a multi-system incident.
Context loss. On long-horizon tasks, early-turn constraints get forgotten.
Tool misuse. The agent calls the right tool with wrong arguments, or the wrong tool entirely.

Cascading is the killer. Agents can't distinguish "I failed" from "the task is impossible," so they often hallucinate a success message on a 400 error just to close the loop.

Mitigation: gates at every step

Put automated verification gates at every step of a reasoning chain, checking factual grounding against environment state. Concretely:

Per-step safety classifier (Lesson 21).
Tool-call argument validation (Lesson 06).
Cross-check retrieved content against known facts (Lesson 05, CRITIC).
Detect success hallucination by re-probing state: was the file actually created?

Where failure monitoring goes wrong

Tagging only crashes. Most agent failures produce valid-looking output, so you need content-level checks.
No baseline. Drift detection needs a last-known-good. Without one you can't say "this is getting worse."
Over-alerting. If every failure pages someone, nobody reads the pages. Cluster and rate-limit.

Check: The Concept0/29

1. What is MASFT?Define

2. What is a cascading error in this lesson?Define

3. Which description matches context loss?Define

4. What is tool misuse?Define

5. What does success hallucination mean?Define

6. What is scope creep in an agent run?Define

7. What is instruction-following deviation?Define

8. What are the three sub-intention errors the lesson names?Define

9. Which two manifestations does the LLM Agent Hallucinations Survey list?Define

10. What is a hallucinated action in an agent run?Define

11. What is MASFT's central claim?Explain

12. MASFT reports an inter-annotator Cohen's Kappa of 0.88. What does that say about the taxonomy?Explain

13. Which of these is NOT one of the five industry-recurring failure modes the lesson lists?Explain

14. What does Microsoft's Taxonomy of Failure Mode in Agentic AI Systems whitepaper describe?Explain

15. A chatbot hallucinates a SKU in a text reply. An agent hallucinates the same SKU inside a tool call. What does the lesson say about the agent case?Explain

16. According to the lesson, where do agentic faults arise from?Explain

17. Why do agents often write a success message after a 400 error?Explain

18. What mitigation does the lesson recommend at every step of a reasoning chain?Explain

19. Why is tagging only crashes insufficient?Explain

20. Why does the lesson say drift detection needs a baseline?Explain

21. What does the lesson recommend to fix over-alerting on agent failures?Explain

22. You ask an agent to fix a typo in a README. It fixes the typo, then opens three extra PRs refactoring unrelated files. Which mode is this?Apply

23. A tool returns HTTP 400 while an agent creates a file, yet the agent tells the user it is done. Which gate catches this?Apply

24. A dashboard shows zero crashes this week, yet users report agents quietly doing the wrong thing. What is the likeliest gap?Analyze

25. Every tagged trace pages the on-call engineer, and the team has started ignoring the pages. Which monitoring mistake is this and what is the fix?Analyze

26. An agent's search errors on an unknown SKU, then it invents the SKU and runs four API calls that cause an incident. Which change would have limited the damage most?Analyze

27. Your agent sends customer emails. Which design best guards against success hallucination?Design

28. Your team plans to wait for the next base model release to remove agent failures. What does MASFT's central claim suggest instead?Design

29. Your agent often calls a refund tool with the amount argument missing. Which gate from the lesson do you add first?Design

Build It

code/main.py implements a stdlib failure-mode tagger:

A synthetic trace dataset covering the five modes.
Detector functions per mode, matching signature patterns on tool calls, outputs, and repeated actions.
A tagger that labels each trace and reports the mode distribution.

Run it:

python3 code/main.py

The output is per-trace labels plus an aggregate distribution. It's a cheap reproduction of what Phoenix's trace clustering surfaces.

Check: Build It0/8

1. In main.py, trace t003 (request "list project files") calls a tool named magic_scanner. What does tag() return for it?Apply

2. In main.py, trace t006 calls read_file with args {"file": "/tmp/foo"}. What does tag() return?Apply

3. In main.py, trace t007 (request "create a PR") only runs a search, then claims success with target_state_changed False. What does tag() return?Apply

4. In main.py, detect_cascading_errors fires on t004, where a search errors and three tool calls follow. What is the least number of calls after the first error that makes it fire?Apply

5. In main.py, trace t002 (request "find the config file") ends with a write_file call. Why does detect_scope_creep tag it?Apply

6. In main.py, trace t001 claims success and target_state_changed is False, yet tag() returns clean. What explains it?Analyze

7. In main.py, trace t005 is told "do not modify src/" and then writes src/foo.py, yet tag() returns clean. What explains the miss?Analyze

8. You extend the tagger to catch an agent that sends emails nobody asked for. Following the tagger's style, how do you build the detector?Design

Use It

Phoenix for production drift clustering (Lesson 24).
Langfuse for session replay and annotation.
Custom detectors for domain-specific signatures your observability platform can't catch.

Ship It

outputs/skill-failure-detector.md generates failure-mode detectors tailored to your domain and wired to a trace store.

Author's Note

I treat these failures as design problems in my own harness, not as something to wait out until the next base model. MASFT's central claim is that they are fundamental design flaws and not LLM limitations, and I'd take that at face value: if a better model won't remove them, the fix has to live in the code around the model. My bet is on verification gates at every step, and the state re-probe is the gate I'd start with, because an agent that hallucinates a success message on a 400 error will report done while the target state is unchanged. I'd also set a baseline before trusting any alert. Without a last-known-good, I can't say whether the agent is getting worse.

Exercises

Add a detector for "success hallucination": agent returns success but the target state is unchanged.
Tag 100 real traces from a product you've built. Which mode dominates? What's the cost of fixing it?
Implement a "cascade radius" metric: given a failure at step N, how many downstream steps did it affect?
Read MASFT's 14 failure modes. Pick three that apply to your product. Write detectors.
Wire one detector into a CI job: fail the build if >=5% of traces tag a mode.

Key Terms

Term	What people say	What it actually means
MASFT	"Multi-agent failure taxonomy"	Berkeley 14-mode categorization
Cascading error	"Ripple failure"	One early mistake propagates through N steps
Context loss	"Forgot the constraint"	Long-horizon turn drops early-turn facts
Tool misuse	"Wrong tool / wrong args"	Valid call, wrong invocation
Success hallucination	"Faked completion"	Agent claims success on a 400; state unchanged
Scope creep	"Overreach"	Agent does more than asked
Instruction-following deviation	"Disobedience"	Ignores system prompt or user constraint
Sub-intention errors	"Plan bugs"	Omission, redundancy, disorder in plan execution