Skill Libraries and Lifelong Learning (Voyager)

An agent that rebuilds every capability from scratch each session fails in three ways:

1. It wastes tokens. Every task re-elicits the same reasoning.
2. It loses progress. A correction learned in session A never reaches session B.
3. It fails at long-horizon composition. Complex tasks need hierarchies of capabilities, and a one-shot prompt cannot express them.

Voyager's answer is to treat each reusable capability as a named chunk of code in a library. The library retrieves skills by similarity, composes them with other skills, and refines them from execution feedback.

A skill library is a persistent store of named, executable code that an agent can search, compose, and refine. Voyager (arXiv:2305.16291) builds an entire agent around one.

Three components

Voyager structures an agent around three pieces:

1. Automatic curriculum. A curiosity-driven proposer picks the next task based on the agent's current skill set and the environment state. Exploration is bottom-up.
2. Skill library. Each skill is executable code, added when a task succeeds. Retrieval matches the query against skill descriptions by similarity.
3. Iterative prompting mechanism. When a skill fails, the agent gets execution errors, environment feedback, and self-verification output, then refines the skill.

In the Minecraft evaluation (Wang et al., 2024), Voyager got 3.3x more unique items, reached stone tools 8.5x faster and iron tools 6.4x faster, and traversed 2.3x more map than the baselines. Those numbers are specific to Minecraft, but the pattern transfers.

Action space = code

Most agents emit primitive commands. Voyager emits JavaScript functions. A skill looks like this:

async function craftIronPickaxe(bot) {
  await mineIron(bot, 3);
  await mineStick(bot, 2);
  await placeCraftingTable(bot);
  await craft(bot, 'iron_pickaxe');
}

It is composed from sub-skills, stored keyed on its description and embedding, and retrieved as a program rather than a prompt. Voyager's skills are JavaScript; the Build It toy below keeps its code as Python strings next to plain Python callables.

The 2026 Claude Agent SDK skill is the same idea: a named, retrievable chunk of code plus instructions that the agent loads on demand.

Skill retrieval

Say the new task is "make a diamond pickaxe." The agent:

1. Embeds the task description.
2. Queries the skill library for the top-k similar skills.
3. Retrieves craftIronPickaxe, mineDiamond, placeCraftingTable, and so on.
4. Composes the new skill from the retrieved pieces plus whatever new logic it needs.

MCP resources (Phase 13) and Agent SDK skills implement this same pattern: retrieval over a knowledge or code surface, scoped to the current task.

Iterative refinement

The feedback loop runs like this:

1. The agent writes a skill.
2. The skill runs against the environment.
3. One of three signals comes back: success, error (with a stack trace), or self-verification failure.
4. The agent rewrites the skill using that signal as context.
5. The loop repeats until success or a maximum number of rounds.

This is Self-Refine (Lesson 05) applied to code generation, with the environment doing the verifying. CRITIC (Lesson 05) is the same pattern with external tools as the verifier.

Curriculum and exploration

The curriculum module proposes tasks like "build a shelter near the lake" based on what the agent has and what it hasn't done yet. It reads the environment state and the skill inventory, then picks a task just above current capability, which is the exploration sweet spot.

For a production agent, this becomes a "what's missing" operator: given the current library and a domain, which skills aren't covered yet? Teams typically run this by hand as a curriculum review.

Where this pattern goes wrong

• Skill library rot. The same skill gets added ten times under slightly different descriptions. Deduplicate on write so retrieval returns one.
• Composed-skill drift. A parent skill depends on a child that was later refined. Version your skills. A parent pinned to v1 does not pick up v3 on its own.
• Retrieval quality. Vector retrieval over skill descriptions degrades once the library grows past a few hundred skills. Supplement it with tag filters and hard constraints ("only skills with category=tooling").

code/main.py implements a stdlib skill library with these pieces:

• Skill: name, description, code (as a string), a callable fn, version, tags, depends_on, and a history of earlier code.
• SkillLibrary: register, search (token overlap), topo_order and execute (composition by topological sort of dependencies), and refinement via re-registering a name, which bumps the version.
• A scripted agent that registers three primitive skills, composes a fourth, hits a failure, and refines it.

The Concept section describes retrieval by embedding similarity. The toy uses token overlap (Jaccard score on description words) so it stays stdlib-only; Exercise 3 swaps in real embeddings. Two other simplifications: "dedup on write" here means exact name match (a second registration of the same name becomes a version bump, not a new entry), and the toy has no cycle detection or version pinning. Exercises 1 and 2 add those.

Run it:

python3 code/main.py

The trace shows library writes, retrieval, composition, a failed execution, and a v2 refinement. That is Voyager's loop end to end.

• Claude Agent SDK skills (Anthropic): the 2026 reference. Each skill has a description, code, and instructions, and is loaded on demand during an agent session.
• skillkit (npm: skillkit): cross-agent skill management for 32+ AI coding agents.
• Custom skill libraries: domain-specific ones, like SQL skills for data agents or Terraform skills for infra agents. The Voyager pattern scales down well.
• OpenAI Agents SDK tools: the low end of the spectrum, where each tool is a lightweight skill.

outputs/skill-skill-library.md generates a Voyager-shaped skill library for any target runtime, with registration, retrieval, versioning, and refinement wired in.

1. Add a dependency-cycle detector to topo_order(). Today its visited set stops an infinite loop but silently returns an order. What should happen when skill A depends on B and B depends on A? Should that be an error or a warning?
2. Implement per-skill version pinning (depends_on currently holds bare names, so a parent always runs the latest child). When a parent skill composes child crafting@1, a refinement to crafting@2 must not silently upgrade the parent.
3. Replace token-overlap retrieval with sentence-transformers embeddings (or a stdlib BM25 implementation). Measure retrieval@5 on a 50-skill toy library.
4. Add a "curriculum" agent: given the current library and a domain description, propose 5 missing skills. Run it weekly.
5. Read Anthropic's Claude Agent SDK skill docs and port the toy library to the SDK's skill schema. What changes about discoverability?

• Wang et al., Voyager (arXiv:2305.16291): the original skill-library paper
• Claude Agent SDK overview: skills as the 2026 productization
• Anthropic, Building agents with the Claude Agent SDK: skills and subagents in practice
• Madaan et al., Self-Refine (arXiv:2303.17651): the refinement loop underneath Voyager