Tool Use and Function Calling

Function calling means the model emits a structured request (a tool name plus arguments), your runtime validates and executes it, and the result goes back into the conversation. Early work asked whether a model could predict a correct call. Production asks something harder: can it chain tools across 40 steps, with memory and partial observability, recover when a tool fails, and never invent a tool that doesn't exist?

Toolformer set the baseline by showing that models can learn when to call tools with self-supervision. BFCL V4 defines the 2026 target. The distance between the two is where production agents live.

Toolformer (Schick et al., NeurIPS 2023)

Toolformer is a training method where the model annotates its own pretraining corpus with candidate API calls and keeps only the ones that help. For each candidate, you execute the call. If including the result reduces loss on the next token, the annotation stays. Then you fine-tune on the filtered corpus.

The tools covered were a calculator, a QA system, search engines, a translator, and a calendar. The signal is purely whether the tool helps predict text, with no human labels.

The scale result matters: tool use emerges at scale. Smaller models get hurt by tool annotations, larger models gain. That's why 2026 frontier models have strong tool use baked in, while most 7B models need explicit tool-use fine-tuning before they're reliable.

Berkeley Function Calling Leaderboard V4 (Patil et al., ICML 2025)

BFCL is the benchmark that became the 2026 standard for evaluating function calling. V4 is weighted like this:

• Agentic (40%): full agent trajectories covering memory, multi-turn, and dynamic decisions.
• Multi-Turn (30%): interactive conversations with tool chains.
• Live (10%): real prompts submitted by users, a harder distribution.
• Non-Live (10%): synthetic test cases.
• Hallucination (10%): detecting when no tool should be called.

V3 introduced state-based evaluation: after a tool sequence, check the API's actual state (is the file created?) instead of matching the AST of the calls. V4 added web search, memory, and format sensitivity categories.

The headline 2026 finding is that single-turn function calling is nearly solved. Failures concentrate in four places: memory (carrying context across turns), dynamic decision-making (choosing tools based on prior results), long-horizon chains (drift after 20+ steps), and hallucination detection (declining to call when no tool fits).

Tool schema

A tool schema is the description of a tool that you hand to the model: a name, a description, and a JSON Schema for the arguments. Every provider has one, and they differ in details but share the same shape:

name: string
description: string (what it does, when to use it)
input_schema: JSON Schema (properties, required, types, enums)

Anthropic uses input_schema directly. OpenAI uses function.parameters. Both accept JSON Schema. The description is load-bearing because the model reads it to pick the tool, and bad descriptions are the number one root cause of wrong-tool-picked failures.

Argument validation

Trust no tool call. Before executing, check four things:

1. Type coercion. The model may return the string "5" where the schema says int. Coerce if the meaning is unambiguous, reject if not.
2. Enum validation. If the schema says status in {"open", "closed"} and the model emits "in_progress", reject with a descriptive error.
3. Required fields. A missing required field becomes an immediate error observation back to the model, not a crash.
4. Format validation. Dates, emails, URLs: validate with concrete parsers, not regex.

Every failure should come back as a structured observation so the model can retry with the right shape.

Parallel tool calls

Modern providers let the model emit several tool calls in one assistant turn. The loop:

1. The model emits 3 tool calls with distinct tool_use_ids.
2. The runtime executes them, in parallel if they're independent.
3. Each result goes back as a tool_result block correlated by tool_use_id.

Treat correlation IDs as load-bearing. Swap them and results get routed to the wrong calls.

Sandboxing

Tool execution is the sandbox boundary (Lesson 09 covers it in detail). The short version: every tool should declare its read/write surface, network access, timeout, and memory cap. A generic run_shell(cmd) is a red flag. A specific git_status() is much safer.

code/main.py implements a production-shaped tool registry:

• A JSON Schema subset validator (stdlib only).
• Tool registration with description, input schema, timeout, and executor.
• Argument coercion and enum validation.
• Parallel tool dispatch with correlation IDs.
• Error observations as structured strings.

Run it:

python3 code/main.py

The trace shows a mini agent dispatching five calls in one turn. Three succeed (one after coercing the string "4" to an integer), and two are deliberately bad: an enum value outside the schema and a tool that doesn't exist. The registry rejects each with a descriptive error the model can act on.

Every provider (Anthropic, OpenAI, Gemini, Bedrock) has its own tool schema. If you need to support more than one, put a translation layer in front: the OpenAI Agents SDK, the Vercel AI SDK, or a LangChain tool adapter. BFCL is the reference benchmark, so if tool use is central to your product, run it against your agent before you ship.

outputs/skill-tool-registry.md generates a tool catalog, schema, and registry for a given task domain. It includes description-quality checks: does each tool's description tell the model when to use it?

1. Add a "no-op" tool that lets the model explicitly refuse to use any other tool. Measure it on a BFCL-like hallucination test.
2. Implement argument coercion for int-as-string and float-as-string. Where does coercion start to hide real bugs?
3. Add a per-tool timeout and a circuit breaker (refuse the tool for 60s after 3 consecutive failures). What does this change about how the model recovers?
4. Read the BFCL V4 description. Pick one category (say, "multi-turn"), run 10 example prompts through your agent, and report the pass rate.
5. Port the stdlib validator to Pydantic or Zod. What did Pydantic or Zod catch that the toy missed?

• Schick et al., Toolformer (arXiv:2302.04761): self-supervised tool annotation
• Berkeley Function Calling Leaderboard (V4): the 2026 eval benchmark
• Anthropic, Tool use documentation: how tool use works and how to define tools for Claude
• OpenAI Agents SDK docs: function tool type and Guardrails