When Agents Disagree About Memory

Most AI agents need memory. Most memory libraries assume there is only one writer, so when two agents disagree, the last write silently wins. Vault is what I built when that assumption stopped being good enough — a TypeScript library (vault-memory) that treats every stored fact as a claim with provenance, keeps compatible facts side by side, and resolves real contradictions with explicit policy instead of last-write-wins.

This post walks through why Vault exists, how it works, the workarounds that keep it practical today, the hurdles I hit along the way, what I learned, and what I would improve next.

---

Table of contents

1. The problem
2. Why last-write-wins fails
3. The solution: claims, not key-value pairs
4. How the pipeline works
5. Architecture at a glance
6. Closing thoughts

---

The problem

AI agents are supposed to remember things from conversation: dietary preferences, timezone, mood, scheduling constraints. Later, when the user asks a follow-up question, the agent should retrieve those facts semantically — not by exact string match, but by meaning.

That part is well understood. Vector search + LLM extraction gets you surprisingly far.

The harder problem appears when more than one agent writes to the same memory store:

• A scheduling agent learns the user is in PST.
• A chat agent casually hears "I'm in EST" in a different conversation.
• A tools agent stores something else entirely on a nearby topic.

Most memory layers treat the store like a cache: write a fact, overwrite if similar, move on. The last write wins. No audit trail. No notion of who wrote what or why one fact should beat another.

That is fine for a single-agent demo. It is not fine for a product where different agents have different trust levels on different domains. A scheduling agent should own user.timezone. A chat agent mentioning a timezone in passing should not silently erase authoritative data.

Vault's pitch is simple:

Every fact is a claim tagged with which agent wrote it. Compatible facts stay. When agents actually contradict each other, we resolve or dispute, instead of blindly overwriting.

---

Why last-write-wins fails

Last-write-wins breaks down in three predictable ways:

Failure mode	What happens
Silent overwrite	A low-trust agent replaces a high-trust fact with no record of the conflict
No provenance	You cannot answer "why does memory say X?" or "who said that?"
Entity drift	Two agents store the same real-world fact under different labels (user.timezone vs user.location) and never trigger cross-agent conflict detection

Vault addresses all three deliberately as core design constraints.

---

The solution: claims, not key-value pairs

The central abstraction in Vault is a claim: one structured fact extracted from conversation.

{
  "entity": "user.timezone",
  "text": "The user is in PST",
  "type": "semantic",
  "sourceAgent": "agent-a",
  "status": "active"
}

Entities group related facts

Entities are dotted-path keys like user.timezone, user.diet, or user.mood. They are not a hardcoded enum by default, the extraction LLM picks a string guided by prompt examples, but they drive everything downstream:

• Cross-agent comparison is entity-scoped (exact string match, loaded from SQL).
• Conflict strategy (recency vs authority) is configured per entity, and only runs after a claim is labeled a contradiction.
• Authority weights in the database are keyed by entity (with wildcard support like user.schedule.*).

Claims have a lifecycle

Status	Meaning
active	Searchable claim. Several can share an entity when they are compatible
superseded	Replaced by a newer claim, kept for audit, removed from vector index
disputed	Challenger parked when authority cannot pick a winner, not indexed

Claims link to history via supersedes. When authority resolution fails, a row lands in the disputes table linking the incoming claim to the incumbent. That table is the audit trail for "why does memory say X?"

Two-pass classification

Instead of one "is this new or an update?" LLM call, Vault runs two passes:

Pass 1 — same agent: Did I already know this? Am I refining my memory, contradicting myself, or stating something genuinely new?

Op	Meaning	Cross-agent pass?
NOOP	Already known	Stop
UPDATE	Refines without contradicting	Skip pass 2
REPLACE	Contradicts own prior claim	Skip pass 2
ADD	Genuinely new for this agent	Run pass 2

Pass 2 — other agents: What do other agents already have on the same entity?

Vault loads every active claim on that entity from other agents, then one LLM call labels each row:

Label	Meaning	What happens
equivalent	Same fact, already stored	Write nothing, unless another row contradicts
compatible	Both can be true	Store the new claim; leave the others active
contradicts	They cannot both be true	Run the entity's strategy on those rows only

Strategy	Behavior when a row is contradicts
recency (default)	Newest claim wins, supersede the contradicting claims
authority	Incoming agent must beat every contradicting agent's weight; else DISPUTE

"The user is vegetarian" and "The user likes Italian food" are compatible, so both stay searchable. "The user eats meat" contradicts the vegetarian claim, so recency or authority applies to that row alone. The Italian-food claim is left alone.

This separation was one of the most important design decisions. Pass 1 handles personal memory hygiene. Pass 2 handles multi-agent policy. Mixing them would have made both harder to reason about and test.

---

How the pipeline works

Every memory.add(messages, conversationId) call runs the same spine:

messages
  → extractFacts          (LLM — latest user turn only)
  → embed                 (per fact text)
  → resolveEntity         (align entity label with similar claims — optional)
  → ensureEntityInRegistry (validate / auto-register — optional)
  → classifyAgainstOwnHistory   (pass 1)
  → if ADD: classifyAgainstOtherAgents (pass 2)
  → apply op              (write SQL row + sync vector index)

Step 1: Extract

An LLM reads the latest conversation turn and returns structured facts:

{
  "facts": [
    {
      "entity": "user.diet",
      "text": "The user is vegetarian",
      "type": "semantic"
    }
  ]
}

Only the latest user turn (plus following assistant messages) is used — not the full conversation history on every write. That keeps extraction focused and token costs bounded per turn.

With useEntityRegistry: true, extraction is constrained to known registry keys or new:user.<domain> for genuinely new subjects.

Step 2: Embed

Each fact's text is embedded once (default: OpenAI text-embedding-3-small, 1536 dimensions). That vector is reused for entity resolution, pass 1, and index sync. Pass 2 does not search by similarity — it reads same-entity rows from SQL.

Step 3: Resolve entity

LLMs drift on entity labels. Agent B might extract user.wrong_label for a fact that is semantically identical to agent A's user.timezone claim. Without alignment, cross-agent conflict never triggers — different entity strings, no match.

resolveEntity() searches all active claims globally and applies a three-band decision:

Similarity	Behavior
≥ 0.85	Auto-align to incumbent entity
0.70 – 0.85	Optional LLM disambiguation
< 0.70	Keep extracted entity

This is what makes the classic PST/EST demo work even when agent B's extractor returns a nonsense label.

Step 4–7: Classify and write

Pass 1 uses vector search over this agent's active claims (top 5) and an LLM chooses ADD, UPDATE, REPLACE, or NOOP. Pass 2, only after ADD, loads other agents' active claims on the exact same entity and labels them. Strategy runs only on contradicts. The write then inserts or updates the SQL row and syncs the vector index in the same logical operation. An equivalent fact writes nothing. A disputed claim is stored and left out of the index.

SQL is canonical. The vector store is an index.

---

Architecture at a glance

┌─────────────────────────────────────────────────────────────┐
│  Agent process (agent-a, agent-b, …)                        │
│  ┌─────────────┐    ┌──────────────────┐                    │
│  │   Memory    │--->│ FaissVectorStore │  (per-process)     │
│  └──────┬──────┘    └────────┬─────────┘                    │
│         │ extract/classify   │ rebuild / upsert             │
└─────────┼────────────────────┼──────────────────────────────┘
          │                    │
          ▼                    ▼
   ┌──────────────────────────────────────┐
   │  Shared DB (SQLite file or Postgres) │
   │  claims · agent_authority · disputes │
   │         · entities (optional)        │
   └──────────────────────────────────────┘

Layer	Technology	Role
Canonical storage	SQLite or Postgres via Drizzle	Claim rows + JSON embeddings
Search index (default)	FAISS (faiss-node, IndexFlatIP)	Per-process exact inner-product search
Legacy opt-out	MemoryVectorStore	Pure JS brute-force cosine — tests/CI
Postgres path	PgVectorStore (vault-memory/pg)	Shared pgvector index across processes

Vault does not ship a database. You bring Drizzle + your driver. The library brings extraction, classification, conflict policy, and vector index sync.

---

Closing thoughts

Vault started as a question: what if agent memory worked more like a claims ledger than a cache? The answer turned into a pipeline — extract, embed, resolve entities, classify twice, write with policy, backed by SQL as source of truth and FAISS as a fast local index.

It is not finished. The library still does not choose the dispute winner. The CLI demo asks the person, and resolveDispute() applies that choice. Consensus strategies are deferred. Entity registry scaling needs scoping work. But the core bet holds: when multiple agents share a store, keeping compatible facts and recording real conflicts is more valuable than pretending every new write replaces the old one.

If you want to explore the code:

• Start at packages/vault-memory/src/memory.ts — the spine of Memory.add()
• Run the CLI demo in apps/cli-agent/ — two terminals, one database, one timezone conflict
• Read the internal docs in docs/

The library is on npm as vault-memory. You bring the database; Vault brings the logic for memory that does not silently overwrite itself.

---

Vault is MIT-licensed. Monorepo: github.com/theMillenniumFalcon/vault.