TAI BUI
Back to curriculum
BuildPython

Handoffs and Routines — Stateless Orchestration

OpenAI's Swarm (October 2024) distilled multi-agent orchestration to two primitives: **routines** (instructions + tools as a system prompt) and **handoffs** (a tool that returns another Agent). There is no state machine and no branching DSL. The LLM routes by calling the right handoff tool. The OpenAI Agents SDK (March 2025) is the production successor, and Swarm stays the cleanest conceptual reference, since its entire source fits in a few hundred lines. The pattern spread because the API is roughly "agent = prompt + tools; handoff = function returning agent." The catch is that it is stateless, so memory is the caller's problem.

TIME: ~60 minutesPREREQUISITES: Phase 16 · 04 (Primitive Model)

Learning Objectives

  • Define a routine (system prompt plus tools) and a handoff (a tool that returns another Agent), and explain why those two primitives are enough to orchestrate several agents.
  • Trace how the run loop detects an Agent-valued return, records a handoff line in the history and re-runs the router with the new active agent.
  • Explain why Swarm is stateless between runs, and say what memory the caller must supply for long or resumed conversations.
  • Choose between handoffs and a GroupChat-style selector by who picks the next speaker, and name the cases where handoffs struggle (long sessions, parallelism, replay).
  • Apply the Ship It checklist (logging, context transfer, authentication, loop detection, fallback agent) to a handoff design.

Problem

Every multi-agent framework wants you to learn its DSL: LangGraph nodes and edges, CrewAI crews and tasks, AutoGen GroupChat and managers. The abstractions are real, but they make orchestration feel heavier than it needs to be.

Swarm goes the other way and uses the tool calling the model already has. A handoff is a tool call. The orchestrator is whichever agent currently holds the conversation. The state machine is implicit in each agent's system prompt.

Warm-up0/3
1. An LLM agent is given a system prompt and a list of functions. What does the model produce when it decides to use one of them?Warm-up
A list of two functions named get_weather and send_email next to a model box, with a dashed question box asking what comes out of the model when it decides to use get_weather.
Typed example with illustrative names, not captured output.
2. Control passes from one agent to another inside a single conversation. Which two things must be settled for the second agent to act?Warm-up
Agent A, currently answering, with an arrow labelled control passes into agent B, about to answer, and a dashed question box asking which two things must be settled for agent B to act.
3. A chat app calls an LLM API once per user message, and the API keeps nothing between calls. How does the app give the model the earlier conversation?Warm-up
A turn 1 message going into an API box that keeps nothing after replying, a turn 2 message below it, and a dashed question box asking what goes into the turn 2 request.
Typed schematic, not captured output.

Concept

Two primitives

Routine. A system prompt that defines an agent's role and available tools. Think of it like a scoped set of instructions: "you are a triage agent; if the user asks about refunds, hand off to the refund agent."

Handoff. A tool the agent can call that returns a new Agent object. The Swarm runtime detects the Agent return value and switches the active agent for the next turn.

Those two pieces carry the whole design.

def transfer_to_refunds():
    return refund_agent  # Swarm sees Agent return → switch active agent

triage_agent = Agent(
    name="triage",
    instructions="Route the user to the right specialist.",
    functions=[transfer_to_refunds, transfer_to_sales, transfer_to_support],
)

The triage agent's system prompt makes it choose the right handoff based on the user message. The LLM's tool-calling does the routing.

Why it is viral

  • Small API. Two concepts to learn.
  • Uses what the model already does. Tool calling is already production-grade across providers.
  • No state-machine burden. You do not describe the graph; the agents' prompts describe who they hand off to.

The stateless trade

Swarm is explicitly stateless between runs. The framework keeps a message history during a run, but it does not persist anything. Memory, continuity, and long-running tasks are the caller's problem.

In production (OpenAI Agents SDK, March 2025) this was one of the main things that changed: the SDK adds built-in session management, guardrails, and tracing while keeping the handoff primitive.

When Swarm/handoffs fit

  • Triage patterns. Front-line agent routes user to a specialist.
  • Skill-based handoffs. "If the task needs code, call the coder; if it needs research, call the researcher."
  • Short, bounded conversations. Customer support, FAQ-to-ticket, simple workflows.

When Swarm struggles

  • Long sessions with shared memory. A handoff swaps in the new agent's prompt and keeps the history. Nothing persists across runs unless the caller manages memory.
  • Parallel execution. Handoff is one-at-a-time: the active agent switches. Parallelism requires the caller orchestrating multiple Swarm runs.
  • Audit and replay. Stateless runs are hard to replay exactly, because the LLM's handoff choice is not deterministic.

OpenAI Agents SDK (March 2025)

The production successor adds:

  • Session state. Persistent thread across runs.
  • Guardrails. Input/output validation hooks.
  • Tracing. Every tool call and handoff is logged.
  • Handoff filters. Control what context transfers on handoff.

The handoff primitive survives; production ergonomics get added around it.

Swarm vs GroupChat

Both use LLM-driven routing, but they differ on who picks next:

  • GroupChat: a selector (function or LLM) picks the next speaker from outside.
  • Swarm: the current agent picks its successor by calling a handoff tool.

In Swarm the agent decides what's next. In GroupChat a manager does. Swarm's decision lives in the active agent's tool call; GroupChat's lives in the GroupChatManager.

Check: Concept0/15
1. Mechanically, what is a handoff in the Swarm model?Define
A function named transfer_to_refunds that returns refund_agent, with an arrow down to a dashed question box asking mechanically what this is called and what happens next.
Code lines copied from main.py's triage_agent_factory.
2. What is a routine in Swarm?Define
The Agent constructor call for triage_agent with its name, instructions and functions list, and a dashed question box asking what the instructions plus tool list together are called.
Code lines copied from main.py's triage_agent_factory.
3. What does 'stateless between runs' mean for Swarm?Define
A box for run_swarm call 1 ending with active switched to refund and a 3 message history, an arrow to a dashed question box asking what run_swarm call 2 has from call 1.
Typed schematic, not captured output.
4. What is the active agent in a Swarm run?Define
A triage box marked active with an arrow labelled handoff to a dashed refund box, and a dashed question box asking what active means after the handoff.
Typed schematic, not captured output.
5. How does the OpenAI Agents SDK relate to Swarm?Define
A Swarm box dated October 2024 with an arrow to an OpenAI Agents SDK box dated March 2025, and a dashed question box asking the relationship between the two.
6. What is a handoff filter in the Agents SDK?Define
An outgoing agent box and an incoming agent box joined by an arrow, with a dashed question box sitting on the arrow asking what hook is named here.
Typed schematic, not captured output.
7. What separates Swarm from GroupChat in who picks the next speaker?Explain
A Swarm box noting the current agent calls a handoff tool and a GroupChat box noting an outside selector, each with a question mark over who picks, and a dashed question box asking who picks the next speaker in each.
Typed schematic, not captured output.
8. Which combination explains why the Swarm pattern spread so quickly?Explain
Three boxes, API surface, tool calling and routing logic, each with a blank question mark, and a dashed question box asking which combination explains why the pattern spread.
Typed schematic, not captured output.
9. Why does Swarm struggle with parallel execution?Explain
An active agent slot box that holds exactly one agent, and three task boxes outlined in warning color, with a dashed question box asking how all three tasks get served at once.
Typed schematic, not captured output.
10. Why is audit and replay hard for a handoff-based system?Explain
The same triage message run twice into two handoff choice boxes, each made by the model, with a dashed question box asking whether the result matches both times.
Typed schematic, not captured output.
11. Which Swarm limitation do the Agents SDK's sessions address?Explain
A run_swarm call A box and a run_swarm call B box with a dashed, unlabeled gap box between them, and a dashed question box asking what fills the gap.
Typed schematic, not captured output.
12. A conversation has to resume tomorrow with the same specialist still active. What does 'memory is the caller's problem' require?Explain
A today box with active set to refund, a dashed box asking what carries over, and a tomorrow box with active left as a question mark, under a dashed question box asking what the caller must supply.
Typed schematic, not captured output.
13. A support bot is built like the Build It demo. A customer is handled by the refund agent, then writes again the next day and the app calls run_swarm(triage_agent, [new_message]). Which agent answers first, and why?Apply
A day 1 box ending with active set to refund, a faint discard mark, and the day 2 run_swarm call passing triage_agent and the new message, with a dashed question box asking which agent answers first and why.
Typed schematic, not captured output.
14. A product must resume a workflow across several days and let reviewers see every tool call and handoff afterwards. Which design fits?Design
A requirements box asking for resuming a workflow across several days and reviewers seeing every tool call and handoff afterward, above four unlabeled dashed candidate boxes, and a dashed question box asking which design fits.
Typed schematic, not captured output.
15. You want one outside component, a function or an LLM, to decide who speaks next instead of each agent deciding. Which design fits?Design
A requirement box asking for one outside component to decide who speaks next instead of each agent deciding, above four unlabeled dashed candidate boxes, and a dashed question box asking which design fits.
Typed schematic, not captured output.

Build It

code/main.py implements Swarm from scratch: an Agent dataclass, a handoff mechanism (tool returns Agent), and a run loop that detects agent switches. A keyword-matching scripted_router stands in for the LLM, so the routing is deterministic.

Demo: a triage agent routes to refund, sales, or support specialists. Each specialist has its own tool. The output prints each handoff as a line in the history.

Run:

python3 code/main.py
Check: Build It0/6
1. In run_swarm, after triage returns an Agent, the router is called again with the same user message. Why?Explain
The run_swarm loop from main.py showing the router called once, a handoff detected and the active agent switched, then the router called again on the same user message, with a dashed question box asking why.
Code copied from main.py's run_swarm.
2. In the Build It demo, what happens when the user sends only 'hello'?Apply
The call run_swarm with the single message hello, next to the triage branch of scripted_router checking for refund, buy and broken keywords and otherwise returning a question mark, with a dashed question box asking what triage does with hello.
Code copied from main.py's scripted_router.
3. With the demo's scripted router, triage receives 'The price is wrong, I want my money back'. Which agent ends up answering?Apply
The message the price is wrong, I want my money back, next to the triage branch of scripted_router showing the refund check runs before the price check, and a dashed question box asking which agent answers.
Code copied from main.py's scripted_router.
4. In the demo, triage hands 'please refund me' to the refund agent. The message contains no digits. What does the refund agent return?Apply
The message please refund me, which has no digits, next to the refund branch of scripted_router that defaults order to 42 and only overwrites it when a word is all digits, and a dashed question box asking what the refund agent returns.
Code copied from main.py's scripted_router.
5. Calling run_swarm(triage_agent, ['I need a refund on order 77', 'order 12 please']) in one run, what does the second message do?Apply
One run_swarm call carrying two messages, a refund request on order 77 and then order 12 please, with a dashed question box asking what the second message does and whether a new handoff line appears.
Code copied in the style of main.py's call.
6. All four demo scenarios print the expected handoffs, but a team sees refund questions reach sales once a real model replaces scripted_router. What did the demo fail to show?Analyze
A scripted_router box that matches keywords with a fixed output, next to a real LLM router box that judges intent with an unknown output, and a dashed question box asking what the four demo scenarios failed to show.
Typed schematic, not captured output.

Use It

outputs/skill-handoff-designer.md designs a handoff topology for a task: which agents exist, which handoffs each can call, and what context transfers.

Ship It

Checklist:

  • Handoff logging. Every handoff writes a trace event with from-agent, to-agent, context snapshot.
  • Context transfer rules. Decide what moves on handoff: full history (expensive), last N messages, or a summary.
  • Guardrail on handoff. A handoff to a specialist with different tool permissions must be authenticated, or prompt injection can force handoffs you did not want.
  • Loop detection. Two agents handing back and forth is a common failure. A simple last-K ring check catches it.
  • Fallback agent. If a handoff target does not exist, fall back to a safe default.
Check: Ship It0/8
1. What does a context transfer policy decide?Define
An outgoing agent's view listing five messages, an arrow into a dashed question box sitting before a faint incoming agent's view box, asking what rule decides what crosses.
Typed schematic, not captured output.
2. What is a handoff loop?Define
Two generic boxes, agent A and agent B, joined by two handoff arrows forming a cycle, with a dashed question box asking the name for this pattern.
Typed schematic, not captured output.
3. Why does Ship It ask you to choose a context transfer rule instead of defaulting to full history?Explain
A context transfer rules checklist box above a cost icon marked full history on every handoff, and a dashed question box asking why Ship It does not default to full history.
Typed schematic, not captured output.
4. A specialist agent holds tools with higher permissions than triage. What does Ship It require before triage may hand off to it?Apply
A triage box with ordinary tools and a specialist box with higher-permission tools, a dashed arrow between them labelled with a warning about an injected message trying to force the handoff, and a dashed question box asking what must be true first.
Typed schematic, not captured output.
5. A triage model calls a handoff to an agent name that is not registered in your system. Which Ship It item covers this, and what should happen?Apply
A call to transfer_to_billing that would return an agent named billing, with a note that no agent by that name is registered, and a dashed question box asking which Ship It item covers this and what should happen.
Scenario typed in the style of main.py's handoff functions; billing is not one of the lesson's four agents.
6. In a design review, the refund agent and the sales agent each carry a handoff tool that returns the other agent, and a vague message ping-pongs between them. Which Ship It control addresses this?Analyze
The refund and sales agents from the lesson's demo joined by two handoff arrows forming a cycle, labelled handoff 1, handoff 2 and a warning-colored handoff 3 and onward, with a dashed question box asking which Ship It control stops this.
Typed schematic, not captured output.
7. An incident review finds only lines like 'triage handed off to refund' in the logs, with nothing about what the refund agent received. Which Ship It item is incomplete?Analyze
A log line reading triage dash greater than refund with nothing else recorded, and a dashed question box asking which Ship It item left this incomplete.
Typed schematic, not captured output.
8. Support chats run for many turns and the refund specialist only needs the order and the complaint. Which context transfer design fits?Design
A support chat so far box listing five-plus turns, an arrow to a refund specialist needs box listing only the order and the complaint, and a dashed question box asking which context transfer design fits.
Typed schematic, not captured output.
Author's Note

I'd run main.py and read the output as a trace, because the whole mechanism is visible in it. Each scenario sends one user message. Triage returns an Agent, the loop appends a (handoff to refund) line to the history, switches active, and runs the router again on that same message so the specialist answers in the same turn.

Keep in mind that scripted_router matches keywords, so this demo shows the loop and not routing quality. Whether a real model picks the right handoff from a system prompt is the part you only learn by trying it. The "hello" scenario is the useful one here: triage finds no match and asks a clarifying question, so no handoff fires.

The statelessness is easy to see in the code. active is a local variable inside run_swarm, so each call begins at whichever start_agent the caller passes (here, always triage) and nothing carries over between calls. There is also no loop detection and no check that a handoff target exists, which is why the Ship It list is not optional for anything real. Exercise 1 needs a second message added to a scenario, since the built-in ones only send one.

Exercises

  1. Run code/main.py, triage to the refund agent. Confirm the second turn's active agent is refund.
  2. Add a loop-detection rule: if the same two agents have handed off 3 times in a row, force an exit. Design the fallback.
  3. Read the OpenAI Agents SDK docs on handoff filters. Implement a "summarize-on-handoff" version: the outgoing agent compresses context to a bullet summary before the incoming agent takes over.
  4. Compare the Swarm handoff to a GroupChatManager selector. Which pattern makes prompt injection worse, and why?
  5. Read the Swarm cookbook (https://developers.openai.com/cookbook/examples/orchestrating_agents). Identify one explicit design decision Swarm makes that OpenAI Agents SDK changed or kept.

Key Terms

TermWhat people sayWhat it actually means
Routine"The agent prompt"System prompt + tool list. Defines role and available handoffs.
Handoff"Transfer to another agent"A tool the active agent can call that returns a new Agent. The runtime switches active agent.
Stateless"No memory between runs"Swarm does not persist anything; memory is the caller's responsibility.
Active agent"Who's speaking now"The agent currently holding the conversation. Handoff changes this.
Context transfer"What moves on handoff"Policy for what history the incoming agent sees: full, last N, or summarized.
Handoff loop"Agents ping-pong"Failure mode where two agents keep handing back to each other.
OpenAI Agents SDK"Production Swarm"March 2025 successor; adds sessions, guardrails, tracing on top of the handoff primitive.
Handoff filter"Gate on transfer"SDK feature to inspect and modify context at the handoff boundary.

Further Reading

Code

Ships With