TAI BUI
Back to curriculum
BuildPython

Parallel / Swarm / Networked Architectures

No central decider. Agents read a shared event bus, pick up work asynchronously, and write results back. LangGraph explicitly supports a "Swarm Architecture" for decentralized, dynamic environments, and Matrix (arXiv:2511.21686) carries both control and data flow as serialized messages on distributed queues to remove the orchestrator bottleneck. The tradeoff is explicit: you give up determinism and traceability for scalability. Swarm fits tasks with many independent sub-problems and does not fit tasks that need a single coherent plan.

TIME: ~75 minutesPREREQUISITES: Phase 16 · 05 (Supervisor Pattern), Phase 16 · 04 (Primitive Model)

Learning Objectives

  • Explain why a supervisor becomes a bottleneck at scale and how a shared queue removes the central decider.
  • Decide whether a task fits a swarm or a supervisor from its independence, duration variance, ordering needs, and debuggability needs.
  • Run the three-strategy demo and explain why pull-based scheduling beats a bad fixed assignment on variable-duration tasks.
  • Describe starvation and hot-spotting, and name the mitigations: priority queues with aging, worker specialization, and back-pressure.
  • Apply the Ship It checklist (durable queue, idempotent workers, per-task trace IDs, back-pressure) to a production swarm design.

Problem

A supervisor handles a few workers fine. Hundreds is another matter. Every decision about who does what funnels through one agent, so the supervisor becomes the bottleneck, and one slow planning step stalls the whole system.

Swarm architectures flip the design. Workers pick tasks off a shared queue instead of waiting for a planner to dispatch them, and the coordination lives in the semantics of the event bus. There is no orchestrator, so the system scales until the queue stops scaling.

Warm-up0/3
1. A supervisor agent dispatches work to many workers. Why does it limit scaling as the worker count grows?Warm-up
A supervisor box with six worker boxes below it, each connected to the supervisor by its own line, above a dashed box asking what happens as more workers are added.
2. Which role in the primitive model (agent, handoff, shared state, orchestrator) does a supervisor play?Warm-up
Four boxes labelled Agent, Handoff, Shared State and Orchestrator in a row, a supervisor box below, and a dashed box between them asking which role the supervisor plays.
3. A bank has three tellers and a mix of quick and slow customers. Which setup usually finishes the whole crowd sooner?Warm-up
Three teller boxes and a loose cluster of eight customer squares marked Q for quick or S for slow, above a dashed box asking which setup clears the crowd soonest.
Check: Problem0/1
1. Why does a shared queue remove the supervisor bottleneck?Explain
Two small diagrams side by side: a supervisor box with arrows from three workers on the left, and a queue box with three workers near it on the right, above a dashed box asking why the right side scales better.

Concept

The shape

                ┌──── shared queue ────┐
                │                      │
       ┌────────┼────────┐  ◄──────┬───┘
       ▼        ▼        ▼         │
     Worker  Worker  Worker   Worker
      A       B       C        D
       │        │        │         │
       └────────┴────────┴─────────┘
                 │
                 ▼
            results pool

There is no orchestrator. A worker loops: pull a task, process it, write the result, and optionally enqueue follow-ups.

When swarm fits

  • Many independent tasks. Scraping, transforming, classifying. The tasks do not depend on each other.
  • Variable-duration work. If some tasks take 100ms and others take 10s, a swarm balances load on its own because fast workers pull the next job sooner. A supervisor has to predict durations up front.
  • Throughput over determinism. You care about total completion time, not strict ordering.

When swarm fails

  • Ordered workflows. If step 3 needs step 2's output, a swarm can start step 3 before step 2 is done.
  • Global-plan tasks. Complex research questions need a planner. A swarm of researchers produces independent facts, not a coherent report.
  • Debugging. Without a central log, and with asynchronous work, reproducing a bug is expensive.

Matrix (arXiv:2511.21686)

Matrix is the 2025 paper that takes swarm to its conclusion. Both control flow and data flow are serialized messages on distributed queues, with no central coordinator. Fault tolerance comes from message durability, and scalability becomes the message broker's problem rather than the system's.

Its contribution is a programming model where coordination means asking "what message topic does this agent subscribe to?" instead of "which agent does the supervisor pick next?" The system ends up looking like a pub/sub event mesh.

LangGraph's Swarm Architecture

The LangGraph 2025 docs list "Swarm Architecture" as one of the multi-agent patterns. Agents are nodes, the edges form a directed graph with cycles, and a node can be activated from the pool. A worker picks from the available work by condition, not by supervisor assignment.

Failure mode: starvation and hot-spotting

If workers prefer the fastest available task, long-running tasks can sit in the queue until nothing else is left. That is classic queue starvation.

Three mitigations:

  • Priority queues with explicit aging (increase priority with wait time).
  • Worker specialization: some workers only take "long" tasks.
  • Back-pressure: limit how many fast tasks enter the queue.

The content-based routing link

Swarm pairs well with content-based routing (Lesson 22). Instead of one generic queue, keep a queue per message type and let specialist workers subscribe only to their type. Message-bus architectures that scale to thousands of agents are built on this.

Check: Concept0/18
1. What defines a swarm architecture?Define
A shared queue holding a few tasks, four worker boxes labelled A to D each with an arrow toward the queue, and all four feeding into one results box, with no separate planner box drawn.
2. What is an event bus in a swarm?Define
Three tagged task boxes (resize, email, resize) each with an arrow into a dashed box, which has two dashed arrows out to boxes named destination 1 and destination 2.
3. What is starvation in a task queue?Define
A stack of four new-task boxes above one low-priority task box, with an arrow showing new tasks keep arriving, next to a dashed box asking whether the low-priority task ever gets picked.
4. What does the Matrix framework (arXiv:2511.21686) carry as serialized messages on distributed queues?Define
A box labelled Matrix with two dashed arrows out marked question mark A and question mark B, pointing toward a row of distributed queue boxes with no coordinator box drawn.
5. What is hot-spotting in a swarm?Define
Four worker boxes drawn as stacks of different heights, one much taller than the other three, above a dashed box asking for the name of this pattern.
6. Why does a swarm balance uneven task durations without predicting them?Explain
A queue of alternating short and long task blocks above four worker boxes, with a dashed box asking why no duration prediction is needed.
7. Why can a swarm start step 3 before step 2 has finished in an ordered workflow?Explain
Three step boxes labelled step 1, step 2 and step 3 above four worker boxes, with a dashed box asking whether a worker can pull step 3 before step 2 finishes.
8. Why does a swarm of researchers on a complex question give a poor report?Explain
Four researcher boxes each producing a fact box below it, with a dashed box asking what ties the four facts into one report.
9. Why does preferring the fastest available task cause starvation?Explain
Three time columns, t1 to t3, each showing a quick task arriving above the same long task, with a dashed box asking what happens to the long task over time.
10. How does priority aging stop a long task from starving?Explain
A small line chart with time waiting on one axis and priority on the other, a flat unlabelled line, and a dashed box asking for the shape of the line.
11. Which of these is NOT a listed mitigation for starvation in a swarm queue?Explain
Three boxes listing priority aging, worker specialization and back-pressure, above a dashed box asking whether a fourth unnamed idea belongs on this list.
12. How does LangGraph's Swarm Architecture decide which agent handles work?Explain
Four agent nodes connected by arrows that form cycles between them, with a dashed box asking who acts next.
13. Which workload fits a swarm best?Apply
Four trait boxes named task independence, duration variance, ordering requirements and debuggability need, with a line describing a workload of thousands of independent jobs from 100 milliseconds to 10 seconds and asking swarm or supervisor.
14. A low-importance report task has sat in the queue for hours while a steady stream of quick tasks is processed. What is the failure and the fix?Analyze
A report task box marked waiting for hours next to a row of quick task boxes arriving continuously, above a dashed box asking for the name of the failure and its fix.
15. A swarm runs a five-step pipeline and sometimes fails because step 3 ran before step 2 had written its output. Which assumption was wrong?Analyze
Five step boxes with a note that step 3 started at time 1 and step 2 finished at time 2, above a dashed box asking which assumption about the steps was wrong.
16. Bugs appear only occasionally across hundreds of asynchronous workers and nobody can reproduce them. Which swarm weakness is this?Analyze
Twelve small worker log squares in a grid, standing in for hundreds, next to a dashed box asking why a bug across them is hard to reproduce.
17. Your tasks come in three types and a few specialist workers should only see their own type. Which design does content-based routing suggest?Design
Three task-type boxes with lines converging on a dashed box, which then has dashed lines out to three specialist boxes.
18. A product must scrape 10,000 independent pages and then write one coherent report from the results. How do you split the design?Design
A scrape-10,000-pages box next to a write-one-coherent-report box, above a dashed box asking how to split the design.

Build It

code/main.py runs three scheduling strategies on the same eight tasks: four slow ones (0.4s each) and four fast ones (0.1s each).

  • Sequential baseline: one worker processes the tasks serially.
  • Fixed assignment: each task is pre-assigned to a specific worker, the way a supervisor would do it. The assignment is deliberately bad. Worker 0 gets all four slow tasks plus one fast one, and the other three workers get one fast task each.
  • Swarm: four worker threads pull from a shared queue.Queue until get_nowait raises queue.Empty.

Fixed assignment leaves three workers idle while worker 0 grinds through its slow tasks. In the swarm, the slow tasks sit at the front of the queue, so each worker grabs one, and the fast tasks get picked up as workers free up. Threads are enough here because the tasks are time.sleep calls, which release the GIL.

Run:

python3 code/main.py

The output prints wall-clock time and per-worker task counts for each strategy. From the sleep durations alone, expect roughly 2.0s sequential, 1.7s fixed (worker 0 does 4 x 0.4s + 0.1s), and 0.5s swarm. The swarm counts depend on thread timing, but with this task order they should land near two per worker.

Check: Build It0/6
1. In the Build It demo, why does the swarm finish in about 0.5s while fixed assignment takes about 1.7s?Explain
Two bars labelled fixed assignment at 1.7 seconds and swarm at 0.5 seconds, above a dashed box asking why the swarm bar is shorter.
2. The demo runs eight tasks: four at 0.4s and four at 0.1s. Roughly how long does the sequential baseline take?Apply
Eight small task squares labelled with their durations, four at 0.4 seconds and four at 0.1 seconds, flowing into one worker box, above a dashed box asking for the total wall-clock time.
3. In the fixed assignment, worker 0 gets all four slow tasks plus one fast task and the other three workers get one fast task each. What sets the wall-clock time?Apply
Worker 0 shown with five task squares, four at 0.4 seconds and one at 0.1 seconds, while workers 1 to 3 each hold one 0.1 second task, above a dashed box asking what sets the overall finish time.
4. In the swarm demo the slow tasks sit at the front of the queue and four workers start together. What happens?Apply
A queue of eight task squares with the four 0.4 second tasks ahead of the four 0.1 second tasks, above four ready worker boxes and a dashed box asking what each worker ends up doing.
5. A swarm worker thread calls get_nowait and it raises queue.Empty. What does the worker in the demo do?Apply
A Python code excerpt of a worker loop up to the except queue.Empty line, with the next line left out, next to a dashed box asking for the next line.
6. A teammate reads the demo output and concludes that swarms are always several times faster than supervisors. What is the flaw?Analyze
Three placeholder bars labelled sequential, fixed and swarm with equal heights, and a quote that swarm is always several times faster, above a dashed box asking what the flaw is.

Use It

outputs/skill-swarm-fit.md decides whether a task should use a swarm or a supervisor. It takes four inputs: task independence, duration variance, ordering requirements, and how much debuggability you need.

Ship It

Checklist:

  • Priority queue with aging. Keeps long tasks from starving.
  • Worker idempotency. A task may be pulled more than once if a worker crashes mid-run, so reprocessing it has to produce the same result.
  • Durable queue. Use Kafka, Redis Streams, or a database-backed queue in production. queue.Queue lives in memory only.
  • Observability per task. Give each task a trace ID, and have workers log start and end against it.
  • Back-pressure. If the queue grows faster than workers drain it, slow the producer.
Check: Ship It0/9
1. What is back-pressure?Define
A producer box with an arrow into a mostly full queue bar, next to a dashed box asking what the queue can send back to the producer.
2. What is an idempotent worker?Define
Task T run through a worker to a result box twice: the first time the result box is filled in, the second time it is a dashed box with a question mark.
3. What is a durable queue?Define
Three pending task squares flowing into a crash box, next to a dashed box asking what happens to the tasks after a restart.
4. Why does each task get a trace ID, with workers logging start and end against it?Explain
Four worker log panels, each with its own separate start and end lines, with a dashed box asking how to tie one task's events across the four logs.
5. A worker crashes halfway through a task and the queue hands that task to another worker. What must be true for this to be safe?Apply
A crashed worker 2 box with task T incomplete, a dashed arrow to a worker 4 box that picks up task T, and a dashed box asking what must be true for this handoff to be safe.
6. You are moving the demo to production and a node restart must not lose queued tasks. What do you change first?Apply
A code line constructing queue.Queue with a comment that it lives in this process's memory, next to a dashed box asking whether it survives a restart.
7. Producers are adding tasks faster than workers drain them and the queue keeps growing. Which Ship It item applies?Apply
A line chart of queue length rising steadily over time, above a dashed box asking which Ship It item applies.
8. After a node restart, queued tasks vanished, and after a worker crash some tasks ran twice with duplicate side effects. Which two Ship It items were missing?Analyze
A node-restart box noting queued tasks are gone and a worker-crash box noting duplicate side effects, above a dashed box asking for the two missing Ship It items.
9. You must process 50,000 independent documents whose durations range from very short to very long. The job must survive node crashes and traffic bursts. Which design fits?Design
A box describing 50,000 documents with durations from very short to very long, and two risk boxes, must survive crashes and must survive traffic bursts, above a dashed box asking which design pieces are needed.
Author's Note

I'd run main.py first and read the three wall-clock times before anything else. The workload is eight tasks, four at 0.4s and four at 0.1s, so the sleeps alone put sequential near 2.0s, fixed assignment near 1.7s, and swarm near 0.5s. Numbers far from those are worth a look at the machine's load before you trust the comparison.

My bet is that the fixed-assignment baseline is the part to read closely. It stands in for a supervisor, and it is built to be bad: worker 0 holds all four slow tasks. The speedup says more about that hand-picked split than about swarms in general. A supervisor that knew the durations would not split the work that way, and the swarm's win is that it does not need to know.

I'd also keep the toy in proportion. Workers exit when the queue is empty, nothing enqueues follow-up work, and queue.Queue is plain FIFO, so starvation, priorities, crashes, and idempotency do not come up. Those are the Ship It items. The exercises add priority queues and routing, but crash handling and idempotency are left for you to build.

Exercises

  1. Run code/main.py. How much faster is swarm than sequential on the variable-duration workload? How much faster than fixed assignment?
  2. Add a priority queue variant (use queue.PriorityQueue). Assign priority by task "importance" field. Observe whether low-priority tasks ever starve under continuous load.
  3. Implement a hot-spot detector: log when any worker processes 3× more tasks than the slowest worker. What does that indicate about task-duration distribution?
  4. Read the Matrix paper (arXiv:2511.21686) abstract and Section 3. Identify one specific tradeoff Matrix accepts (scalability gain) and one it gives up (traceability, determinism).
  5. Convert the swarm demo to use a queue.Queue of (task_type, payload) tuples, with workers subscribing only to specific types. What routing rules make sense when tasks are heterogeneous?

Key Terms

TermWhat people sayWhat it actually means
Swarm architecture"Decentralized agents"Workers pull from shared queue; no central orchestrator.
Event bus"Agents subscribe to topics"Message broker that routes tasks to workers by type or content.
Starvation"Task never runs"Low-priority task never gets picked because higher-priority work arrives continuously.
Hot-spotting"One worker drowns"Load imbalance where one worker gets most tasks.
Back-pressure"Slow down the producer"Mechanism that signals upstream to stop producing when the queue fills up.
Idempotent worker"Safe to re-run"A task processed twice produces the same result. Required because workers may crash mid-run.
Durable queue"Survives crashes"Queue backed by disk or replicated storage; tasks are not lost when a worker crashes.
Matrix framework"Full message-passing swarm"Both data and control flow are serialized messages on distributed queues.

Further Reading

Code

Ships With