When Agents Disagree About Memory

Most AI agents need memory. Most memory libraries assume there is only one writer — so when two agents disagree, the last write silently wins. Vault is what I built when that assumption stopped being good enough — a TypeScript library (vault-memory) that treats every stored fact as a claim with provenance, detects contradictions across agents, and resolves conflicts with explicit policy instead of last-write-wins.

This post walks through why Vault exists, how it works, the workarounds that keep it practical today, the hurdles I hit along the way, what I learned, and what I would improve next.

---

Table of contents

1. The problem
2. Why last-write-wins fails
3. The solution: claims, not key-value pairs
4. How the pipeline works
5. Architecture at a glance
6. Workarounds that make it shippable today
7. Hurdles I hit — and how I navigated them
8. What shipped (Phase 1 → Phase 2 → entity work)
9. Validating it end to end
10. Learnings
11. What can be improved
12. Closing thoughts

---

The problem

AI agents are supposed to remember things from conversation: dietary preferences, timezone, mood, scheduling constraints. Later, when the user asks a follow-up question, the agent should retrieve those facts semantically — not by exact string match, but by meaning.

That part is well understood. Vector search + LLM extraction gets you surprisingly far.

The harder problem appears when more than one agent writes to the same memory store:

• A scheduling agent learns the user is in PST.
• A chat agent casually hears "I'm in EST" in a different conversation.
• A tools agent stores something else entirely on a nearby topic.

Most memory layers treat the store like a cache: write a fact, overwrite if similar, move on. The last write wins. No audit trail. No notion of who wrote what or why one fact should beat another.

That is fine for a single-agent demo. It is not fine for a product where different agents have different trust levels on different domains. A scheduling agent should own user.timezone. A chat agent mentioning a timezone in passing should not silently erase authoritative data.

Vault's pitch is simple:

Every fact is a claim tagged with which agent wrote it. When agents disagree, we detect, resolve, or dispute — instead of blindly overwriting.

---

Why last-write-wins fails

Last-write-wins breaks down in three predictable ways:

Failure mode	What happens
Silent overwrite	A low-trust agent replaces a high-trust fact with no record of the conflict
No provenance	You cannot answer "why does memory say X?" or "who said that?"
Entity drift	Two agents store the same real-world fact under different labels (user.timezone vs user.location) and never trigger cross-agent conflict detection

Vault addresses all three deliberately — not as afterthoughts, but as core design constraints.

---

The solution: claims, not key-value pairs

The central abstraction in Vault is a claim: one structured fact extracted from conversation.

{
  "entity": "user.timezone",
  "text": "The user is in PST",
  "type": "semantic",
  "sourceAgent": "agent-a",
  "status": "active"
}

Entities group related facts

Entities are dotted-path keys like user.timezone, user.diet, or user.mood. They are not a hardcoded enum by default — the extraction LLM picks a string guided by prompt examples — but they drive everything downstream:

• Cross-agent conflict search is entity-scoped (exact match).
• Conflict strategy (recency vs authority) is configured per entity.
• Authority weights in the database are keyed by entity (with wildcard support like user.schedule.*).

Claims have a lifecycle

Status	Meaning
active	Current truth — indexed and searchable
superseded	Replaced by a newer claim — kept for audit, removed from vector index
disputed	Challenger parked when authority cannot pick a winner — not indexed

Claims link to history via supersedes. When authority resolution fails, a row lands in the disputes table linking the incoming claim to the incumbent. That table is the audit trail for "why does memory say X?"

Two-pass classification

Instead of one "is this new or an update?" LLM call, Vault runs two passes:

Pass 1 — same agent: Did I already know this? Am I refining my memory, contradicting myself, or stating something genuinely new?

Op	Meaning	Cross-agent pass?
NOOP	Already known	Stop
UPDATE	Refines without contradicting	Skip pass 2
REPLACE	Contradicts own prior claim	Skip pass 2
ADD	Genuinely new for this agent	Run pass 2

Pass 2 — other agents: Does this conflict with what other agents believe on the same entity?

Strategy	Behavior
recency (default)	Newest claim wins — supersede all conflicting active claims
authority	Incoming agent must beat every conflicting agent's weight; else DISPUTE

This separation was one of the most important design decisions. Pass 1 handles personal memory hygiene. Pass 2 handles multi-agent policy. Mixing them would have made both harder to reason about and test.

---

How the pipeline works

Every memory.add(messages, conversationId) call runs the same spine:

messages
  → extractFacts          (LLM — latest user turn only)
  → embed                 (per fact text)
  → resolveEntity         (align entity label with similar claims — optional)
  → ensureEntityInRegistry (validate / auto-register — optional)
  → classifyAgainstOwnHistory   (pass 1)
  → if ADD: classifyAgainstOtherAgents (pass 2)
  → apply op              (write SQL row + sync vector index)

Step 1: Extract

An LLM reads the latest conversation turn and returns structured facts:

{
  "facts": [
    {
      "entity": "user.diet",
      "text": "The user is vegetarian",
      "type": "semantic"
    }
  ]
}

Only the latest user turn (plus following assistant messages) is used — not the full conversation history on every write. That keeps extraction focused and token costs bounded per turn.

With useEntityRegistry: true, extraction is constrained to known registry keys or new:user.<domain> for genuinely new subjects.

Step 2: Embed

Each fact's text is embedded once (default: OpenAI text-embedding-3-small, 1536 dimensions). That vector is reused for entity resolution, classification, and index sync.

Step 3: Resolve entity

LLMs drift on entity labels. Agent B might extract user.wrong_label for a fact that is semantically identical to agent A's user.timezone claim. Without alignment, cross-agent conflict never triggers — different entity strings, no match.

resolveEntity() searches all active claims globally and applies a three-band decision:

Similarity	Behavior
≥ 0.85	Auto-align to incumbent entity
0.70 – 0.85	Optional LLM disambiguation
< 0.70	Keep extracted entity

This is what makes the classic PST/EST demo work even when agent B's extractor returns a nonsense label.

Step 4–7: Classify and write

Pass 1 uses vector search over this agent's active claims. Pass 2 searches other agents' active claims on the exact same entity. Resolution picks a strategy, writes the claim row, and syncs the vector index in the same logical operation.

SQL is canonical. The vector store is an index.

---

Architecture at a glance

┌─────────────────────────────────────────────────────────────┐
│  Agent process (agent-a, agent-b, …)                        │
│  ┌─────────────┐    ┌──────────────────┐                    │
│  │   Memory    │───▶│ FaissVectorStore │  (per-process)    │
│  └──────┬──────┘    └────────┬─────────┘                    │
│         │ extract/classify   │ rebuild / upsert             │
└─────────┼────────────────────┼──────────────────────────────┘
          │                    │
          ▼                    ▼
   ┌──────────────────────────────────────┐
   │  Shared DB (SQLite file or Postgres) │
   │  claims · agent_authority · disputes │
   │         · entities (optional)        │
   └──────────────────────────────────────┘

Layer	Technology	Role
Canonical storage	SQLite or Postgres via Drizzle	Claim rows + JSON embeddings
Search index (default)	FAISS (faiss-node, IndexFlatIP)	Per-process exact inner-product search
Legacy opt-out	MemoryVectorStore	Pure JS brute-force cosine — tests/CI
Postgres path	PgVectorStore (vault-memory/pg)	Shared pgvector index across processes

Vault does not ship a database. You bring Drizzle + your driver. The library brings extraction, classification, conflict policy, and vector index sync.

The monorepo wraps the npm package with:

• packages/vault-memory/ — the product (vault-memory@0.2.0+ on npm)
• apps/cli-agent/ — two-terminal multi-agent demo on one SQLite file
• apps/web/ — marketing landing page (Next.js)

---

Workarounds that make it shippable today

Some parts of the design are principled. Some are pragmatic compromises that work well enough to ship and document honestly.

1. FAISS rebuild for shared SQLite

Each agent process holds its own in-memory FAISS index. When two agents share a SQLite file (the CLI demo pattern), agent B's index does not automatically see agent A's writes.

Workaround: call vectorStore.rebuild(db) before reads and writes when another process may have written claims.

The CLI demo wraps this in syncVectorIndex():

export async function syncVectorIndex(
  vectorStore: VectorStore,
  db: VaultDb
): Promise<void> {
  await vectorStore.rebuild(db);
}

This is O(n) over all active claims on every operation in the demo — acceptable for a harness, not a production pattern at scale. For multi-process production, the intended path is Postgres + PgVectorStore, where the index is shared and rebuild is not required on every op.

2. Pure-JS FAISS shim for CI

Native faiss-node bindings do not always install cleanly in CI or consumer scratch tests. The test suite injects a pure-JS FAISS shim (test/helpers/pure-js-faiss.ts) so conflict harness tests run without native builds.

Consumer verification (bun run verify:consumer) packs the npm tarball, installs it in a temp directory, and uses MemoryVectorStore as a fallback when native bindings fail.

3. Bun + faiss-node trust

On Bun, postinstall for native modules can be blocked. Documented fix:

bun pm trust faiss-node
bun install

This is a toolchain workaround, not a library design choice — but it blocked local development until documented.

4. Authority must be seeded manually

The authority strategy does not infer trust. Without rows in agent_authority, everything silently falls back to recency. That is intentional (explicit policy over magic) but easy to misconfigure.

Workaround for demos: bootstrapSharedStore() seeds EXAMPLE_AUTHORITY_SEED once on first run. Production apps must seed their own weights and set entityStrategies explicitly for entities where domain trust matters.

5. Entity registry loads the full table

When useEntityRegistry: true, listRegistryEntities() runs an unbounded SELECT with no pagination or relevance filter. Registry keys are injected into extraction prompts and borderline LLM disambiguation allow-lists.

This works for tens of entities. At thousands, prompt bloat, token cost, and LLM classification quality degrade — the scaling fix is scoping (similarity filter, cap, pagination), not tuning in-memory Set lookups. More on this in What can be improved.

---

Hurdles I hit — and how I navigated them

Hurdle 1: Multi-agent conflict was the real product — but Phase 1 had to work first

Trying to build extraction, vector search, classification, and cross-agent policy in one shot would have made debugging impossible.

How I navigated it: phased delivery.

• Phase 1: single-agent pipeline — extract, embed, classify against own history (ADD/UPDATE/REPLACE/NOOP), FAISS search, recency decay. Schema was multi-agent-ready from day one (sourceAgent, supersedes, disputed status existed but were unused).
• Phase 2: two-pass classify, agent_authority + disputes tables, recency and authority strategies, synthetic conflict test harness, CLI demo, npm publish at 0.2.0.
• Entity work (P0–P3): resolveEntity() for label drift, optional entity registry, wildcard strategy lookup — additive layers on top of Phase 1/2, not replacements.

The pipeline doc explicitly tracks what entity work changed vs what stayed the same. That discipline kept regressions visible.

Hurdle 2: LLM entity label drift breaks conflict detection

Cross-agent conflict requires an exact entity match. If agent A writes user.timezone and agent B writes user.location for the same fact, pass 2 never fires.

How I navigated it: semantic entity resolution before classify. Search all active claims by embedding similarity, auto-align above 0.85, LLM disambiguate in the borderline band. The PST/EST worked example in the pipeline docs depends on this — agent B's extractor returns user.wrong_label, resolution aligns to user.timezone, authority dispute triggers correctly.

This does not replace a fixed taxonomy (optional registry handles that separately), but it closes the most common drift case when similar claims already exist.

Hurdle 3: Per-process FAISS vs shared SQLite

FAISS is fast and ships as the default, but it is in-process memory. Shared SQLite + multiple agent processes = stale indexes.

How I navigated it: document the rebuild pattern honestly, implement syncVectorIndex() in the CLI demo, and ship PgVectorStore as the multi-process production path. The architecture diagram in the docs always shows SQL as canonical and FAISS as a cache you rebuild when needed.

Hurdle 4: Testing multi-agent behavior without burning API credits

Real LLM calls are non-deterministic and expensive. Conflict resolution needs to be testable in CI.

How I navigated it: a synthetic multi-agent conflict harness with deterministic fake embeddings and scripted LLM responses. Tests cover recency supersede, authority dispute, two-pass skip rules, FAISS index sync, and entity resolution bands — all without OpenAI.

Separate from that: verify:consumer confirms the published tarball works outside the monorepo, including a two-agent conflict script on shared SQLite.

Hurdle 5: Native dependencies vs npm consumer experience

Shipping FAISS as the default is the right performance choice but creates install friction (native bindings, Bun trust, CI variability).

How I navigated it: three backends with a clear default + opt-out story:

Backend	When
FaissVectorStore	Default — local dev, single-machine multi-agent
MemoryVectorStore	Tests, CI, zero native deps
PgVectorStore	Postgres + pgvector, shared across processes

vectorStore is optional on config — omit it and you get FAISS. Pass MemoryVectorStore explicitly when native deps are a problem.

Hurdle 6: Knowing when not to build

consensus and llm-judge resolution strategies are designed but throw if called. resolveDispute() does not exist yet.

How I navigated it: defer until real harness cases prove recency/authority are not enough. Shipping two well-tested strategies beats shipping five half-baked ones. Disputes stay open for the app layer to handle — Vault records what conflicted, not who should adjudicate it in your product UI.

---

What shipped (Phase 1 → Phase 2 → entity work)

Area	Status
Extract → classify → store (ADD/UPDATE/REPLACE/NOOP)	Shipped
FAISS vector search (default)	Shipped
Search-time recency decay	Shipped
Multi-agent two-pass classify	Shipped
recency + authority strategies	Shipped
disputes audit table	Shipped
Entity resolution (similarity + LLM band)	Shipped
Optional entity registry (entities table)	Shipped
Synthetic conflict test harness	Shipped
CLI multi-agent demo (two terminals, one DB)	Shipped
vault-memory@0.2.0 on npm + consumer verification	Shipped
consensus / llm-judge strategies	Not implemented
resolveDispute() API	Not implemented
Approximate FAISS (IVF/HNSW)	Deferred until 10k+ claims

---

Validating it end to end

Automated regression

cd packages/vault-memory
bun run build
bun test
bun run typecheck
bun run verify:consumer

This validates single-agent ops, FAISS sync/rebuild, two-pass classification, both conflict strategies, dispute rows, decay, and tarball consumer smoke tests.

The CLI demo — the "aha" moment

Two terminals, one SQLite file:

# Terminal 1
cd apps/cli-agent && bun run start:agent-a

# Terminal 2
cd apps/cli-agent && bun run start:agent-b

Terminal 1: I'm in PST
Terminal 2: I'm in EST

Expected: agent B's claim is stored as disputed. Search for "what timezone is the user in?" returns agent A's PST claim — because agent A has authority weight 0.9 on user.timezone vs agent B's 0.2.

Then try recency fallback on an unconfigured entity:

Terminal 1: I love Italian food
Terminal 2: I hate Italian food, I only eat Japanese

→ agent B wins on recency, agent A superseded, no dispute row.

That contrast — authority dispute vs recency supersede — is the product in two conversations.

---

Learnings

1. SQL first, vector index second

Treating embeddings in claims.embedding as canonical and FAISS as a rebuildable cache simplified every failure mode. Multi-process drift, crash recovery, manual DB edits — all have the same fix: vectorStore.rebuild(db).

2. Separate "my memory" from "our memory"

Two-pass classification maps cleanly to how humans think about conflicting information: first reconcile with yourself, then reconcile with others. It also makes pass 2 skippable when pass 1 is UPDATE/REPLACE/NOOP — saving LLM calls and avoiding nonsensical cross-agent checks on self-corrections.

3. Explicit policy beats implicit trust

Authority weights are seeded, not inferred. That feels like more setup — and it is — but it prevents silent misconfiguration where you think authority is active but everything is actually recency because you forgot to seed the table.

4. Entity strings are load-bearing

Conflict detection, strategy lookup, and authority weights all key off exact entity match (with wildcard patterns for authority). LLM-chosen entity labels are convenient early and expensive later. Entity resolution and optional registry are the mitigation layers — not optional polish.

5. Test the published artifact, not just the monorepo

verify:consumer caught packaging and export issues that unit tests inside the workspace would miss. For a library meant to ship on npm, that script is as important as the test suite.

6. Document workarounds as first-class architecture

The FAISS rebuild pattern is not a footnote — it is in the architecture doc, vector store doc, onboarding guide, and CLI README. Hiding workarounds creates GitHub issues; naming them creates trust.

7. Additive phases reduce rewrite risk

Entity work (P0–P3) runs before classify and does not replace two-pass logic, superseding, disputes, or FAISS sync. Being explicit about "what is unchanged from Phase 1/2" kept the pipeline doc useful as a canonical reference instead of a historical artifact.

---

What can be improved

Conflict resolution

Gap	Why it matters
consensus / llm-judge strategies	Designed but not built — needed when recency/authority are insufficient
resolveDispute() API	Disputes stay open forever today; app layer must handle resolution manually
Auto-resolve rules	Authority gap + time + repeat disagreements — design exists, not implemented

Scale and performance

Gap	Why it matters
Entity registry scoping	listRegistryEntities() loads the entire table into prompts and allow-lists — O(registry size) per extraction/disambiguation call. Needs similarity filter, cap, or pagination
FAISS rebuild on every op	Fine for CLI demo, not for production multi-process SQLite
Approximate FAISS indexes	Flat index is O(n) — deferred until 10k+ claims, but recall affects classify outcomes
getRegistryEntity full scan	Looks up one key by loading all rows — should be a keyed query

Entity consistency

Gap	Why it matters
No normalization (timezone vs time_zone)	Resolution helps when similar claims exist; cold start still drifts
Wildcard entityStrategies	Authority DB supports wildcards; strategy config lookup is exact-key only today
Registry descriptions unused in disambiguation	Descriptions exist in DB but disambiguation only injects keys — missed opportunity for relevance filtering

Product surface

Gap	Why it matters
Ingestion guardrails	Rate limits, dedup, spam dispute prevention — app-layer today
Hosted vector DB story	FAISS + SQL is deliberate; some teams will want a managed vector service adapter
Dispute UX patterns	Library records conflicts; products need workflows for human or automated adjudication

---

Closing thoughts

Vault started as a question: what if agent memory worked more like a claims ledger than a cache? The answer turned into a pipeline — extract, embed, resolve entities, classify twice, write with policy — backed by SQL as source of truth and FAISS as a fast local index.

It is not finished. Dispute resolution is intentionally left to the app layer. Consensus strategies are deferred. Entity registry scaling needs scoping work. But the core bet holds: when multiple agents share a store, detecting and recording conflict is more valuable than pretending it never happened.

If you want to explore the code:

• Start at packages/vault-memory/src/memory.ts — the spine of Memory.add()
• Run the CLI demo in apps/cli-agent/ — two terminals, one database, one timezone conflict
• Read the internal docs in docs/ — especially pipeline.md and multi-agent-conflict.md

The library is on npm as vault-memory. You bring the database; Vault brings the logic for memory that does not silently overwrite itself.

---

Vault is MIT-licensed. Monorepo: github.com/theMillenniumFalcon/vault.