Hierarchical Architecture and Its Failure Mode

Once the supervisor pattern clicks, the next question is what happens when the workers are themselves supervisors. Teams have sub-teams and companies have departments of departments, so hierarchical architectures copy that shape.

The catch is that an LLM manager is not a human manager. A human manager carries stable priors about what their reports know. An LLM manager re-reasons the org chart every turn from whatever sits in its context. A small drift in that context can send work to the wrong place all the way down the tree.

The shape

                 Manager
                 ┌─────┐
                 └──┬──┘
           ┌────────┴────────┐
           ▼                 ▼
       Sub-Mgr A         Sub-Mgr B
       ┌─────┐           ┌─────┐
       └──┬──┘           └──┬──┘
         ┌┴──┬──┐          ┌┴──┐
         ▼   ▼  ▼          ▼   ▼
       W1  W2  W3         W4  W5

Every internal node plans, delegates, and synthesizes. The leaves do the actual work.

Where it shines

• Clear org mapping. If the real task is departmental ("legal review the doc, finance review the doc, engineering review the doc, then summarize for exec"), the hierarchy is explicit.
• Local summarization. Each sub-manager synthesizes its team's output before the top manager sees it. Top manager sees three sub-manager summaries, not fifteen worker outputs.

Where it breaks

The 2026 post-mortems on these systems keep finding the same three failure modes:

1. Task assignment error. The manager reads the goal, hallucinates a decomposition, and delegates to the wrong sub-manager. The sub-manager works on whatever it was given, so the error only surfaces at the top synthesis, one level removed from where a human could have caught it.
2. Output misinterpretation. Sub-manager returns "unable to verify claim X." Top manager summarizes as "claim X not confirmed." Meaning drifts a little at each level.
3. Consensus loops. Two sub-managers disagree; top manager asks them to reconcile; they re-delegate down; workers re-run; sub-managers return slightly different answers; loop. CrewAI's Process.hierarchical guards against this with step limits, but then the limit is a hyperparameter you have to tune.

The deciding question

Sequential (linear pipeline) or hierarchical comes down to one question: does the task have independent sub-teams, or is it one linear flow pretending to be a tree? If it is a linear flow, use sequential. If the sub-teams are independent, use hierarchical and budget explicit reconciliation rules.

CrewAI's implementation

Process.hierarchical wires a manager LLM over specialist crews. The manager:

• receives the top-level task,
• assigns subtasks to crews,
• evaluates crew outputs,
• decides whether to accept, re-delegate, or iterate.

Documentation: https://docs.crewai.com/en/introduction (look for "Hierarchical Process" under Core Concepts).

LangGraph's implementation

LangGraph uses nested create_supervisor calls. The inner supervisor has its own graph; the outer supervisor treats the inner graph as an opaque node. This is easier to debug than CrewAI because you can step through each graph separately, but it is harder to express dynamic reshaping of the tree.

Reference: https://reference.langchain.com/python/langgraph-supervisor.

main.py runs a 3-level hierarchy:

• top manager: delegates a task to branches by label (engineering, legal, finance),
• engineering sub-manager: splits into "frontend" and "backend" workers,
• legal and finance sub-managers: one worker each.

The workers return canned answers, so no LLM is involved. The demo contrasts a happy path, where the top manager picks engineering and legal, with a perturbed path where the branch list says finance instead of legal. That label swap is hard-coded in the call to top.run. It stands in for a bad decomposition. The finance sub-manager does finance work, the top synthesis reports finance findings, and the legal question goes unanswered.

Run:

python3 code/main.py

The output prints each path as a tree of sub-managers, leaf questions, and leaf answers, ending in the top synthesis, so you can compare what was asked against what was delivered.

outputs/skill-hierarchy-fitness.md evaluates whether a given task should use hierarchical, sequential, or flat supervisor. It takes a task description, an org structure, and a reconciliation budget, and returns a pattern recommendation along with the failure modes to guard against.

If you ship hierarchical:

• Cap tree depth at 2. Three levels already hide most errors from observability.
• Set an explicit reconciliation budget. Fix the max rounds before the top manager must commit. Usually 2.
• Put provenance on every synthesis. Each node's summary should cite which leaf outputs produced it.
• Alert on decomposition drift. Log the manager's decomposition per step and diff it against the user query. If the decomposition stops covering the query, fire an alert.

1. Run code/main.py and compare happy vs perturbed. How many levels of manager hand-off does it take before the top output fully diverges from the user's question?
2. Add a third level (top → sub → sub-sub → worker). Measure how often the perturbed path corrects itself vs fully diverges as depth grows.
3. Implement a "canary" worker at each sub-manager that is always asked the original user question unchanged. Use the canary answer to detect decomposition drift. How should the manager react when the canary disagrees with the synthesized answer?
4. Read CrewAI's Process.hierarchical docs. Identify one concrete guardrail CrewAI applies (step limit, manager_llm constraint) and describe what failure mode it targets.
5. Compare nested LangGraph supervisors to CrewAI hierarchical. Which makes reconciliation loops cheaper to detect?

• CrewAI introduction: Process.hierarchical: textbook hierarchical with a manager LLM
• LangGraph supervisor reference: nested supervisor via create_supervisor
• Anthropic engineering: Research system: why Anthropic deliberately chose flat supervisor over hierarchical
• Cemri et al.: Why Do Multi-Agent LLM Systems Fail?: MAST taxonomy; section on coordination failures documents decomposition drift