Hybrid Memory: Vector + Graph + KV (Mem0)

Agents ask three different kinds of questions, and each one is answered well by exactly one kind of store:

• Semantic similarity: "what did we discuss about agent drift last week?" Vector wins. KV and graph miss.
• Fact lookup: "what is the user's phone number?" KV wins. Vector is wasteful and graph is overkill.
• Relationship reasoning: "which customers share the same billing entity?" Graph wins. Vector and KV can't answer it at all.

A production agent issues all three in a single session, so a single-store memory is wrong for two of them, every time. Mem0's contribution is putting all three stores behind one add/search surface, with a scoring function that fuses what comes back.

Three stores in parallel

Hybrid memory is a memory layer that writes every fact to three stores at once (vector, KV, graph) and merges their results when you search. Mem0 (arXiv:2504.19413, April 2025) is the reference implementation. On add(text, user_id, metadata) it does this:

1. Extract candidate facts from the text (an LLM-driven step).
2. Write each fact to the vector store as an embedding, for semantic search.
3. Write each fact to the KV store, keyed on (user_id, fact_type, entity), for O(1) lookup.
4. Write each fact to the graph store (called Mem0g) as typed edges, for relationship queries.

On search(query, user_id), each store answers in its own way:

1. The vector store returns the top-k by embedding cosine.
2. The KV store returns direct hits keyed on the (user_id, type, entity) derived from the query.
3. The graph store returns the subgraph reachable from the entities in the query.
4. A scoring layer fuses the three result sets.

Fusion scoring

score = w_relevance * relevance(q, record)
      + w_importance * importance(record)
      + w_recency * recency(record)

• Relevance is vector cosine, KV exact match, or graph path weight, depending on which store the record came from.
• Importance is tagged at write time or learned. Some facts matter more than others: names, IDs, policies.
• Recency is exponential decay over time since last write or read.

The weights are a per-product decision. Chat agents want a higher w_recency, compliance agents a higher w_importance, retrieval agents a higher w_relevance.

Mem0g and temporal reasoning

The graph store, Mem0g, includes a conflict detector. When a new fact contradicts an existing edge, the old edge is marked invalid but never deleted. A temporal query ("what was the user's city in March?") then traverses the subgraph that was valid at that time.

This is the compliance-grade behavior that Letta's invalidation pattern generalizes.

Benchmark numbers

The Mem0 paper reports (2025):

• LoCoMo (long-form conversation memory): 91.6
• LongMemEval (long-horizon episodic memory): 93.4
• BEAM 1M (1M-token memory benchmark): 64.1

The baselines (full-context 128k LLM, flat vector store, flat KV) all lose by 10+ points. Benchmarks alone shouldn't pick your architecture, since operational shape matters more. But the gap tells you the fusion design is doing real work.

Scope taxonomy

Mem0 splits memory by scope:

• User memory persists across sessions, keyed on user_id.
• Session memory persists within one thread.
• Agent memory is per-agent instance state.

Every write picks exactly one scope, and retrieval can query across scopes with per-scope weights. Mix scopes carelessly and you get the "assistant told Alice about Bob's project" incident.

Where this pattern goes wrong

• Embedding drift. Vector results that look right on the first hundred queries degrade as the corpus grows. Re-embed the top-N most-used records on a schedule.
• KV schema creep. (user_id, type, entity) looks simple until every team invents its own type. Audit the type set quarterly.
• Graph explosion. One noisy extractor can add 50 edges per message. Cap graph writes per add call and drop low-confidence edges.

code/main.py implements the three-store pattern in stdlib:

• VectorStore uses naive token-overlap similarity as an embedding stand-in.
• KVStore is a dict keyed on (user_id, fact_type, entity).
• GraphStore holds typed edges (subject, relation, object, valid).
• Mem0 is the top-level facade: add(), search(), fusion scoring, and scope-aware retrieval.
• A worked trace runs a multi-user, multi-session conversation through all of it.

Run it:

python3 code/main.py

The output shows the three recall paths separately, then the fused top-k. Change the scoring weights at the top of main() and watch the ranking move.

• Mem0 (Apache 2.0) is production-ready. Self-host it with Postgres, Qdrant, and Neo4j, or use the managed cloud.
• Letta uses a three-tier core/recall/archival design. You bring your own vector and graph backends.
• Zep is the commercial alternative, with a temporal knowledge graph and fact extraction.
• Custom builds make sense when you need exact control over the extractor (compliance) or the fusion weights (voice agents, where recency dominates).

outputs/skill-hybrid-memory.md generates a three-store memory scaffold with a fusion scorer, scope taxonomy, and temporal invalidation already wired in.

1. Replace the toy vector similarity with a real embedding model (sentence-transformers, Ollama, OpenAI embeddings). Measure recall@10 on a synthetic long conversation. Does the ranking drift over 1000 writes?
2. Add a temporal query: search(query, as_of=timestamp). Return only records valid at or before that time. Which store needs the most work?
3. Implement a conflict detector: if an incoming fact contradicts a graph edge, invalidate the old edge and log both. Test it on "user lives in Berlin" -> "user lives in Lisbon."
4. Extend the fusion scorer with a user_feedback dimension (thumbs-up on retrieved records). How do you prevent gaming, where the agent only returns records it already liked?
5. Read the Mem0 docs (docs.mem0.ai) and port the toy to mem0 client calls. Compare retrieval quality on the same 20 test queries.

• Chhikara et al., Mem0 (arXiv:2504.19413): the original paper
• Mem0 docs: production API, SDKs, managed cloud
• Packer et al., MemGPT (arXiv:2310.08560): the virtual-context predecessor
• Letta, Memory Blocks blog: the three-tier sibling design