MemShift: Separating Detection from Repair in LLM-Derived Personal Memory

Introduction

AI assistants increasingly keep a memory of values derived from a person’s records. Suppose an assistant stores the item “my important contacts are Ana, Ben, and Chloé”. A language model, the writer, derived it from calendar records using the person’s own definition: someone with at least $n$ one-off meetings in the last $w$ days. Five shifts can make such an item wrong or put it at risk. A meeting is rescheduled (a source revision) or deleted (a deletion), or the person changes what “important” means (a redefinition). The assistant moves to a new writer model (a writer switch), or the person exports the memory to another product (an export). Practitioners report stale memories ((Mem0 Engineering Team, 2026)), contradictions that pile up when a memory only adds new facts,¹ and memories moved between products by pasting text exports ([5, 52]).

Keeping the item correct takes two steps: detection, noticing that an input changed, and repair, producing the correct new value. A score on the final answer cannot tell which step failed. Consider two memories after the person redefines “important” (Figure 1). The first never notices the change, and 7.5% of the values it serves for the changed definition items are right. The second notices nearly every change and asks its writer to derive the value again, and 6.7% are right. The two get about the same score, but they fail at different steps: the first at detection, the second at repair. Benchmarks such as LoCoMo ((Maharana et al., 2024)), LongMemEval ((Wu et al., 2025)), and MemoryAgentBench ((Hu et al., 2026)) score the final answer after new information arrives, so they cannot separate these memories. They also state every change as new text, but three of the five shifts add no text. A redefinition changes every record unchanged, a writer switch replaces only the model, and an export replaces only the product. Stating a redefinition as a new message would change both the text and the meaning, and the wording of a change alone already moves answers ([74]).

We thus ask: after a change, did a memory fail to notice it or fail to repair it, and what fixes each failure?

We propose a benchmark MemShift that answers this question item by item (§ 3). It forks synthetic worlds into branches that are identical except for the shift. It fixes gold values and their dependencies before any text is written, and it scores answers with a deterministic scorer instead of an LLM judge. For each item whose gold value or status changed, it records whether the memory invalidated the item, whether the re-derived value is right, and what the memory served otherwise. A simple identity links these three terms, missed invalidation $m$, repair correctness $r$, and the served accuracy $a$ of items that were not re-derived, to accuracy after the shift. A memory that reports its invalidations thus gets a diagnosis for every item. Any other memory still gets its carryover: its accuracy on the items that the shift changed and on those it did not.

To validate the measure, we build a reference memory, HoldMem, in which detection and repair are two switches with known effects (§ 4). With $B=1$, each item has a key that invalidates it when one of its inputs changes. With $B=0$, it has none. With $D=0$, the writer re-derives an invalidated item. With $D=1$, the memory recomputes it exactly as gold does (possible for 43 of the 48 items), which gives an upper bound on repair. A third setting runs programs that the writer writes itself.

We test four open-weight models from four families (Qwen3.5-9B, Llama-3.1-8B, gemma-4-12B, and gpt-oss-20b), each as writer and reader, and two Claude models as writers of definition items. The results show three patterns.

1. Detection helps only as much as repair allows (§ 6.1). For every open-weight model, a key raises Update more with exact recomputation than with the writer’s own re-derivation. The difference is from +14.4 pp to +28.5 pp after a redefinition and +5.0 pp to +26.4 pp after a deletion. A strong writer makes the key useful on its own.

2. Repair fails on the items computed by the person’s definitions (§ 6.2). Each open-weight writer re-derives at most 25% of these changed values correctly, while exact recomputation gets at least 91% right. Programs the writer writes over the memory’s typed records close most of this gap, and a Claude writer that sees every record of a definition comes close to exact recomputation.

3. Keys built from the writer’s citations notice every redefinition but miss records that join an item later (§ 6.3).

The interaction between detection and repair also holds on fresh worlds and on held-out definitions, and most items that fail a writer switch are definition items (§ 6.4).

Contributions. (1) MemShift, a benchmark of five shifts with counterfactual branches, gold dependencies, and a deterministic scorer, which separates detection from repair. (2) A reference memory with known detection and repair, which validates the benchmark and measures what keys and programs are worth when a language model re-derives, together with design rules drawn from the results.

Related Work

Evaluating memory. Agent memories page context in and out, reflect, link notes, extract facts, keep temporal graphs, or act as an operating-system resource ([27, 90] (Packer et al. 2023; Park et al. 2023; Rasmussen et al. 2025; Xu et al. 2025)). Many build on RAG ([40, 61, 85]). Benchmarks test recall over long conversations ((Maharana et al., 2024; Wu et al., 2025)), incremental interaction ([71]), hallucination, long-horizon tasks, stale memories, governance, and poisoning ([19, 20, 25, 66, 74] (Ren et al. 2026; Xu et al. 2026; Zhao et al. 2026b)), and personalization ((Salemi et al., 2024; Zhong et al., 2024)). Most of their knowledge-update questions state the change in the history and score only the final answer. A missed detection and a failed repair then look the same. Closest work. Several recent systems pair detection with a repair step, and MemShift can place each of them on $m$, $r$, and $a$. MemoRepair (Zhao et al., 2026a) withdraws everything derived from an invalidated source and rebuilds it by deterministic recomputation, regeneration, or removal. It assumes complete provenance. In our terms, it is a key over complete dependencies with a repair that mixes $D=1$ and $D=0$. The StateMem method of Fan et al. [45] records which records an item was derived from and recomputes the item with a language model, close to our citation key at $D=0$. Its benchmark, StateMemBench, asks whether an answer is current or outdated, but every change arrives as dialogue text, and an LLM judge scores the answer. PLANFENCE [21] checks the stored input versions of a derived plan and reports detection separately from completion, but its repair is always a replan by a language model. User as Code [89] keeps typed state with executable rules, close to the writer’s programs. MemTrace (Long et al., 2026) varies the age of one fixed fact, the question, and the evidence, but never what the gold answer depends on. RECON [7] asks which derived conclusions lose support after an invalidation but does not compare a stale store with its repaired version. Goyal and Ray [55] switch the writer over a fixed history and find that the memory’s representation decides what survives. We split survival into detection and repair, much as studies of model updates separate accuracy from regressions [9] (Echterhoff et al. 2024; Srivastava et al. 2020; Xie et al. 2021; Yan et al. 2021). What MemShift adds is the measurement: detection crossed with repair, in branches that differ only in the shift, with shifts that add no text, and with scoring against gold fixed before any text, as in script-first synthetic worlds (Spencer, 2026; Weston et al., 2016; Xie et al., 2026), and without an LLM judge (Zheng et al., 2023).

Dependency tracking and program-aided reasoning. Build systems and incremental view maintenance detect what a change affects [60] (Mokhov et al. 2018), and they rebuild deterministically. In a memory, a language model rebuilds, so the value of invalidation depends on how well it re-derives. Language models are weak at aggregation over long contexts [68, 81], which definition items require. Program-aided reasoning computes values with programs [24, 47], and MemShift measures what such programs are worth after each shift.

The MemShift Benchmark

MemShift checks every item that a memory derived. After a shift, it asks whether the item is still correct. If it is not, it asks which step failed: detection or repair. This section describes the data, the shifts, the questions we ask, and the measures.

Worlds and Derived Items

A world is twelve months in the life of one synthetic person. It has six sources: e-mail, chat messages, calendar, contacts, tasks, and notes. We first sample the true changes, as in Spencer (2026), and only then render them as text with deterministic templates. Each change inserts, revises, or deletes a record, and each record has an identifier and a revision number. Background traffic, such as newsletters, small talk, and routine mail, adds distracting text. A world has 1,505–1,654 events and about 2.2M tokens.

From a world, a memory derives 48 items of two kinds. The 42 other items are facts that can be read from records: six attributes of the person (employer, manager, home city, time zone, diet, and commute), the employer and city of each of twelve contacts, four recurring series, two appointments, and six task due dates. Five of them are free text, and the rest are fields of structured records. The six definition items are computed with the person’s own definitions: important contacts, close friends, active projects, working hours, VIP senders, and overdue tasks. A definition is a one-sentence rule over structured fields, such as calendar entries, contact cards, tasks, and message headers. It names a source, a condition, and an aggregate, often with a time window and a threshold, as in the opening example, and it has two versions. For every item and time, a deterministic evaluator computes the gold value, the gold status, and the gold dependencies, that is, the records and revisions that the value depends on. No human labels are needed.

Shifts

Table 1 lists the five shifts. A source revision, a deletion, and a redefinition each fork the world into a branch, and each draws from its own random stream. A branch thus differs from the others only in the events of its shift. Three shifts add no text. A redefinition changes what a value means while every record stays the same. The memory is told of the new version, and every question about a definition item names the version in effect, so even a system that stores no definitions learns of it. A writer switch and an export to the Portable Memory format (MacPaw, 2026) change the memory, not the world, so no gold value changes. They test whether what a memory derived survives a new model or a new product.

Shift	What changes	Records	Effect on gold
Changes to the world
source revision	ordinary events revise the records behind about a fifth of the items (a job change, a moved meeting, new messages)	revised	re-evaluated on the revised records
deletion	the record behind half of the items read from a series, task, or contact card is deleted	some deleted	abstain for those items
redefinition	half of the definitions move to their second version, and the memory is told	unchanged	re-evaluated under the new version
Changes to the memory
writer switch	the writer model, at the same tier, one larger, and one smaller	unchanged	none
export	the product: the memory moves to the Portable Memory format, and a second implementation imports it	unchanged	none

Questions and Carryover

A probe is a question about one item at a given time, with a gold value and a gold status. The status is answer, abstain (the record is gone), or blocked (the asking agent’s consent was revoked). We ask probes at the fork and at four checkpoints one, two, four, and eight weeks later. Every system is queried through the same reader, a model that answers each probe from the memory’s evidence within a 4,096-token budget. A deterministic scorer compares dates, times, entities, and sets as typed values. An answer is correct only if both its status and its value match gold. For example, the gold answer for the important contacts is a set, and an answer that names the same people in any order is correct. No LLM judge is used.

Carryover is how much of what a memory derived stays correct after a shift. We count each item once at each checkpoint. An item is changed at a checkpoint if its gold value or status differs from that at the fork. Update is the share of correct answers on the changed items. Stability is the same share on a random sample of the unchanged items, at most as many as the changed items at that checkpoint. Both are percentages, and their differences are in percentage points (pp). A writer switch changes no gold value, so for it we report retention: the share of answers that were correct before the switch and are still correct after it.

Detection, Repair, and What Is Served

Consider the changed items of a memory that reports which items it invalidated. Three numbers describe what happens to them. The missed invalidation $m$ is the share of changed items that the memory did not invalidate after their gold value last changed. The repair correctness $r$ is the share of invalidated and re-derived items that are answered correctly. The served accuracy $a$ is the share of the other changed items, those not invalidated, that are answered correctly. For these items the memory serves either the old value or a new item derived from records that arrived later. By the law of total probability,

Each term belongs to one part of a memory that a builder controls: $m$ to detection, $r$ to repair, and $a$ to what is served without either. Compare two memories with the same repair, one that misses no change ($m=0$) and one that never invalidates ($m=1$). Their Update differs by $r-a$, so this is what detection is worth. Now take two kinds of repair, indexed 0 and 1. The difference between what detection is worth under each,

is the interaction between detection and repair. It is large when detection helps only with a good repair.

Where a change comes from. Update counts every change after the fork. A change is caused by the shift if the gold value at that checkpoint also differs from the gold value of the same world continued without the shift. Otherwise the value would have changed anyway. Some changes come from the clock alone: no record that the value depends on changed and the definition did not change, even when a task becomes overdue or a meeting leaves a time window.

Interface and Diagnostic Tests

A system connects to MemShift through one interface. It ingests events, receives definitions and consent revocations, sets the writer, answers probes, exports, imports, and reports its cost. A system that offers only add and search runs through an adapter. The adapter ingests each event in time order and returns the retrieved context to the shared reader. This is enough to measure carryover. A system that also logs which items it invalidated and re-derived gets $m$, $r$, and $a$ for every item. Four diagnostic tests with fixed thresholds check retention after a writer switch, carryover after an export, accuracy before any shift, and serving cost (Section E).

How MemShift compares. Table Table 2 gives the size of MemShift and compares it with the 15 closest benchmarks, each read from its own paper. None of them has all three properties that the measures need: branches that differ only in the shift, gold dependencies that tell which items a shift affects, and shifts that add no text. MemShift has all three and scores every answer against gold.

worlds (test + replication)	20 + 20
events per world	1,505–1,654
items per world (definition + other)	6 + 42
graded probes	19,562
Changed items (main study)	Revision	Redefinition	Deletion
all	1,168	558	1,767
caused by the shift	787	222	1,370
also without the shift	381	336	397
clock alone	72	69	75
		Benchmarks (of 15)
Property	Yes	Partly	MemShift
gold fixed before text	6	8	✓
gold dependency sets	4	10	✓
redefinition	0	0	✓
writer switch	1	6	✓
export	0	1	∼
counterfactual branches	1	4	✓
no LLM judge	3	2	✓

HoldMem: A Reference Memory

To test a measure of detection and repair, we need a memory whose detection and repair we control. Our reference memory, HoldMem, derives each item from a 6,000-token BM25 selection of the records in the item’s sources. A stored item holds a one-sentence statement and up to six cited records with quotes, and the reader sees only these. Two switches set how the memory detects and repairs. In the terms of Equation (1), $B$ changes $m$ and $D$ changes $r$. Both also change $a$, because a memory that does not invalidate an item may still derive a new one from records that arrived after its last write.

$B$ sets detection. A $B=0$ memory only appends. It calls the writer only when new source records arrive, derives new items from them, and never invalidates an item, so $m=1$. A $B=1$ memory keeps a key for each item. The key records how many events each input of the item has received, the definition version, and the writer. When any of these changes, the memory invalidates the item and re-derives it with the current writer. We test two keys. The source key counts the events of each source of the item. The citation key counts the events of the record groups, such as a message thread, that hold the records the writer cited. Every source receives events between checkpoints, so the source key invalidates 99.7%, 99.3%, and 100.0% of the unchanged items after a source revision, a redefinition, and a deletion. In these worlds it thus acts as a periodic full refresh.

$D$ sets repair. At $D=0$ the writer re-derives an invalidated item from its BM25 selection. At $D=1$ the memory also checks the writer’s value against the records (Section C.1). If the check fails, exact recomputation supplies the value: rules we wrote read the record fields or, for a definition item, apply the codified definition as gold does. For 43 of the 48 items this computes exactly what gold computes, so $D=1$ is an upper bound on repair. For the five free-text items, $D=1$ only checks the writer’s value. A third setting, the writer’s programs ($D_{\mathrm{m}}$), lets the writer itself supply the computation. The writer writes one program per item and definition version, given the one-sentence definition and the field names of the item’s sources. A restricted interpreter runs the program on the memory’s typed records, that is, the live records of those sources as typed fields instead of rendered text.

Cells and controls. We call each setting of $(D,B)$, a cell, written HoldMem$(D,B)$. HoldMem$(1,1)$ uses the source key unless we say otherwise. All cells are built from the same writer outputs, and before any shift every cell gives the reader the same context for every probe. Keys never depend on the repair, so $D$ cannot change detection. After a shift, the $D=1$ repair also differs from the writer’s re-derivation in how the value is presented and in which records it reads. Controls vary one of these differences at a time (Section C.4).

Experimental Setup

Models. By default, Qwen3.5-9B (Qwen Team, 2026c) serves as both writer and reader. It runs at temperature 0 with constrained decoding. To test other model families, Llama-3.1-8B (Grattafiori et al., 2024), gemma-4-12B (Gemma Team, [49]), and gpt-oss-20b (OpenAI, 2025) each replace it as writer and reader on 10 test worlds. To test stronger writers, Claude Sonnet 5.5 and Opus 5.5 (Anthropic, [5]) write the definition items on 10 test worlds.

Baselines and native systems. The main study includes three baselines. Plain RAG retrieves records with dense retrieval. The oracle context gives the reader exactly the records that the gold value depends on. No-store BM25 stores nothing and runs BM25 (Robertson and Zaragoza, 2009) over the live records at every query. We also run 6 existing memory systems, which we call native systems: Mem0 (Chhikara et al., [27]), A-Mem (Xu et al., 2025), MemOS (Li et al., [90]), LightRAG (Guo et al., [58]), HippoRAG 2 (Gutiérrez et al., [62]), and Cognee (Markovic et al., 2025). Each gets three-month worlds through its own interface and answers through the shared reader, so only storage and retrieval differ. We read each system’s $(D,B)$ setting from its documentation and code (Table 10). None checks a value against its records, so all have $D=0$.

System	$D$	$B$	Run natively	Basis
Products
Apple semantic index	partial	0		indirect evidence
ChatGPT memory	0	0		documentation, indirect evidence
Claude API memory tool	0	0		indirect evidence
Claude memory	0	0		documentation, indirect evidence
Gemini memory	0	0		documentation, indirect evidence (low confidence)
Memory libraries
A-Mem (Xu et al., 2025)	0	0	✓	every event becomes a new note, and nothing supersedes or deletes a note
Cognee (Markovic et al., 2025)	partial	$0^{a}$	✓	delete purges the item’s chunks, summaries and the entities and edges only it produced
Graphiti / Zep (Rasmussen et al., 2025)	partial	temporal validity only		code, paper, documentation
HippoRAG 2 [62]	0	1	✓	delete removes the passage and the triples and phrases no other passage supports
Honcho	partial	0		code, documentation
Khoj	0	0		code
Letta (.af) (Packer et al., 2023)	0	0		code, documentation
LightRAG [58]	0	1	✓	delete purges the document’s chunks, vectors and entity and relation contributions
Mem0 (self-hosted) [27]	0	0	✓	the add path with inference emits only new memories, and updates and deletions need explicit calls
MemOS [90]	0	0	✓	add never reads existing memories or their stored source and revision
Supermemory	partial	temporal validity only		indirect evidence
Research systems and data stores
Opal [78]	partial	0		paper
Solid pod (Mansour et al., 2016)	—	—		documentation
User as Code [89]	partial	0		code, paper
Our baselines and cells
plain RAG [85]	—	—		code
no-store BM25	—	—		code
HoldMem(0,0), (0,1), (1,0), (1,1)	as named			code

Experiments. The main experiment runs six HoldMem cells after a source revision, a redefinition, and a deletion on 20 twelve-month test worlds. The cells are the four ($D$, $B$) cells with the source key and two $B=1$ cells with the citation key. The writer-switch experiment has three tiers: from Qwen3.5-9B to MiMo-V2.6-Distill-Qwen-9B (Xiaomi MiMo Team, 2026b) of the same size, from Qwen3.5-9B up to Qwen3.8-27B (Qwen Team, 2026d), and from Qwen3.8-27B down to Qwen3.5-9B. Together with an experiment over world lengths (horizons), these form the main study. Further experiments cover the writer’s programs, refresh cells, held-out definitions, the other writers, native systems, export, and a replication on 20 fresh worlds (Section E).

Statistics. Every estimate has a 95% world-bootstrap interval ([41, 46] (Miller 2024)). Differences between cells are paired by item and checkpoint, and each of 2,000 bootstrap draws resamples the test worlds. The intervals thus cover new worlds from the same generator. For held-out definitions they also resample the definitions.

Results

Unless stated otherwise, the results are from the twelve-month test worlds of the main study, with Qwen3.5-9B as writer and reader, and each contrast is over all changed items. Table 3 gives Update and Stability after the three shifts that change the world. In its block of native systems, all rows use the same worlds, but not all of them are scored on the same items.

Memory	Source revision Update	Source revision Stability	Redefinition Update	Redefinition Stability	Deletion Update	Deletion Stability
HoldMem(0,0)	62.8	72.7	26.9	77.8	9.0	78.3
HoldMem(0,1)	68.3	90.5	27.1	91.8	58.2	84.8
HoldMem(1,0)	80.1	72.8	64.7	77.8	18.8	79.6
HoldMem(1,1)	93.1	93.1	91.6	93.0	73.1	88.4
HoldMem(0,1), citation key	63.2	90.3	21.9	91.8	54.7	84.3
HoldMem(1,1), citation key	79.6	91.2	67.2	91.8	58.1	85.0
HoldMem($D_{\mathrm{m}}$,0)	68.5	69.2	45.0	78.0	16.8	78.0
HoldMem($D_{\mathrm{m}}$,1)	85.0	88.4	70.4	90.7	45.2	82.7
no-store BM25	61.0	69.3	25.3	67.4	71.3	64.8
plain RAG	63.1	66.1	25.6	64.7	69.7	64.4
oracle context	74.1	87.0	45.9	87.3	86.9	87.3
HoldMem(0,0)	57.6	66.6	47.2	69.1	44.0	74.1
HoldMem(0,1), citation key	77.8	98.7	65.5	97.6	70.1	97.2
HoldMem(1,1)	98.2	99.0	97.4	99.0	97.9	99.2
no-store BM25	71.1	70.8	58.3	70.1	60.3	72.1
plain RAG	73.4	66.5	59.1	69.7	58.1	67.6
oracle context	93.8	89.2	91.1	92.0	95.7	91.6
Mem0	70.6	67.5	50.6	69.7	44.7	70.0
A-Mem	69.3	80.2	50.6	81.1	34.2	80.9
MemOS	70.4	68.7	57.4	67.6	42.1	68.6
LightRAG	64.9	85.2	50.6	80.1	31.6	80.9
HippoRAG 2	74.3	82.6	58.3	83.8	47.4	82.7
Cognee	68.8	95.1	56.2	95.0	42.1	95.7

Detection Helps Only as Much as Repair Allows

A key helps when the repair is good. By Equation (1), a key changes $m$ and is worth $r-a$. With Qwen3.5-9B as writer, a key raised Update after a redefinition by only +0.2 pp when the writer re-derived the item. It raised Update by +25.4 pp when the writer’s programs re-derived it, and by +26.9 pp with exact recomputation, the upper bound. The same pattern holds for all four open-weight models (Table 4b). For each model, the key raised Update after a source revision, and $D=1$ raised the accuracy on re-derived items without changing detection. The interaction $B \times D$ was positive after a redefinition and after a deletion, with intervals above zero. After a source revision it was positive for Qwen3.5-9B and gpt-oss-20b. A strong writer, in contrast, makes the key useful on its own. With Claude writers of the definition items, the key alone raised Update on these items at $D=0$ by +30.3 pp to +38.8 pp over the three shifts on 10 worlds.

(a) Shift	Items	Share	Repair with a key % $r_0$	$r_1$	Without a key Update, % $a_0$	$a_1$	$B \times D$ on Update caused by the shift	all
source revision	definition	28.5	16.3	99.7	10.2	65.8	+24.3 [+12.5, +35.7]	+27.3 [+21.6, +32.6]
	other	71.5	89.1	90.7	83.7	85.9	−1.0 [−4.7, +2.0]	−0.6 [−3.8, +2.5]
redefinition	definition	69.4	6.7	100.0	7.5	61.8	+38.7 [+30.4, +46.9]	+38.8 [+32.0, +44.8]
	other	30.6	73.1	73.1	70.8	71.3	—	−0.6 [−4.0, +2.7]
deletion	definition	17.4	11.8	100.0	4.2	56.5	+39.6 [+15.0, +60.0]	+35.4 [+30.4, +40.0]
	other	82.6	68.1	67.5	10.0	10.9	−1.5 [−5.9, +2.5]	−1.4 [−5.4, +2.3]

The interaction comes from the definition items. The source key missed at most 0.6% of the changed items in any class, so the terms in Table 4a explain the interaction well. After a source revision, Qwen3.5-9B re-derived only 16.3% of the changed definition items correctly, while exact recomputation got 99.7% right. All four models show this gap. On changed definition items, each writer re-derived at most 25% correctly after every shift, and exact recomputation at least 91% (Table 16). On the other items, the writer’s re-derivation is close to exact recomputation for every model except Llama-3.1-8B, and the interaction is small. On definition items, the interaction was about the same for changes that the shift caused and for changes that would have happened anyway (Section F.2).

Writer	Class	$S_{\mathrm{src}}$ $r_0$	$r_1$	$B \times D$	$S_{\mathrm{def}}$ $r_0$	$r_1$	$B \times D$	$S_{\mathrm{del}}$ $r_0$	$r_1$	$B \times D$
Qwen3.5-9B	definition	16	99	+27.3 [+21.6, +32.6]	7	100	+38.8 [+32.0, +44.8]	12	99	+35.4 [+30.4, +40.0]
main study	other	89	91	−0.6 [−3.8, +2.5]	73	73	−0.6 [−4.0, +2.7]	68	68	−1.4 [−5.4, +2.3]
	field	90	93	+1.3 [−1.8, +4.4]	71	72	0.0 [−3.7, +3.3]	68	67	−1.5 [−5.5, +2.3]
	free text	85	76	−10.7 [−21.3, −1.3]	86	82	−4.5 [−15.0, 0.0]	86	82	0.0
Qwen3.5-9B	definition	14	100	+33.6 [+28.7, +38.5]	7	99	+38.0 [+31.5, +44.5]	11	100	+40.2 [+35.6, +44.9]
replication	other	90	90	−1.9 [−3.9, +0.2]	84	86	+1.6 [−1.2, +4.8]	66	67	+0.8 [−3.6, +5.2]
	field	93	94	−1.6 [−3.5, +0.3]	84	87	+3.2 [+0.6, +6.6]	66	66	+0.9 [−3.6, +5.4]
	free text	74	66	−3.8 [−12.1, +4.2]	82	82	−7.1 [−17.2, 0.0]	82	82	0.0
gemma-4-12B	definition	19	97	+7.3 [−2.2, +16.1]	16	99	+17.9 [+9.0, +26.3]	19	99	+16.2 [+7.9, +25.5]
	other	95	96	−1.6 [−4.4, +0.7]	89	95	+6.0 [−1.7, +13.9]	79	85	+6.0 [+2.8, +9.2]
	field	97	98	−0.8 [−4.2, +2.4]	89	96	+6.8 [−2.1, +15.2]	79	85	+6.1 [+2.8, +9.3]
	free text	84	84	−6.9 [−12.5, 0.0]	90	90	0.0	90	90	0.0
Llama-3.1-8B	definition	8	98	+12.7 [+5.9, +18.8]	1	92	+21.4 [+15.3, +28.5]	5	94	+20.8 [+11.4, +28.7]
	other	76	86	+0.9 [−9.3, +11.6]	71	93	+19.3 [−1.4, +41.0]	40	64	+27.6 [+21.2, +34.2]
	field	76	92	+6.8 [−3.9, +18.6]	68	95	+20.5 [−3.0, +43.8]	39	63	+27.8 [+21.3, +34.7]
	free text	74	47	−36.2 [−55.4, −12.5]	90	80	+10.0 [0.0, +50.0]	90	80	+10.0 [−18.8, +62.5]
gpt-oss-20b	definition	19	97	+44.8 [+34.7, +55.2]	25	96	+39.3 [+34.3, +44.0]	19	99	+52.6 [+44.6, +61.1]
	other	96	94	+0.2 [−3.3, +4.2]	90	90	+2.4 [−9.7, +12.1]	79	83	+4.5 [+0.7, +7.8]
	field	98	99	+1.4 [−0.5, +3.2]	90	97	+6.8 [−1.7, +15.4]	78	84	+4.9 [+0.8, +8.5]
	free text	81	62	−6.9 [−23.6, +18.3]	90	40	−30.0 [−66.7, 0.0]	90	40	−30.0 [−66.7, 0.0]

The source key works as a full refresh. It re-derives nearly every item at every checkpoint. On changed definition items, a refresh on the same schedule that keeps the old item next to the new one does nearly as well. Removing stale items matters after a deletion, where such a refresh still serves items whose record is gone (Section F.5). A key built from the gold dependencies invalidates far fewer unchanged items and still keeps the interaction on definition items (Table 9).

(a) Shift	Item-times		Key	Unchanged item-times invalidated				Re-derivations
	Changed	Unchanged		All (%)	Def. (%)	Field (%)	Free (%)	All	Def.	Field	Def. share (%)
$S_{\mathrm{src}}$	1,168	2,672	source key	99.7	98.6	99.7	100.0	3,800	470	2,930	97.9
			oracle	16.8	50.3	16.4	1.5	764	377	344	78.5
			scope	—	53.1	—	—	—	382	—	79.6
$S_{\mathrm{def}}$	558	3,282	source key	99.3	95.7	99.3	100.0	3,792	469	2,923	97.7
			oracle	4.3	61.3	2.9	1.1	553	415	124	86.5
			scope	—	64.5	—	—	—	429	—	89.4
$S_{\mathrm{del}}$	1,767	2,073	source key	100.0	100.0	100.0	100.0	3,808	472	2,936	98.3
			oracle	7.0	49.4	3.7	1.1	805	373	418	77.7
			scope	—	52.3	—	—	—	379	—	79.0

Repair Fails on Definition Items, and Programs Fix Most of It

Programs written by the writer keep most of the interaction. Repair ($D$) changes $r$. When the programs of Qwen3.5-9B ran on the typed records with a key, they raised Update on changed definition items after a source revision by +57.4 pp over the writer’s own re-derivation, compared with +82.9 pp for exact recomputation (Table 20). Over all items, the interaction was +25.3 pp after a redefinition and +11.0 pp after a source revision. We also sampled 18 program sets from 6 writers. Over these sets, the interaction on definition items after a redefinition was +23.1 pp [+14.6, +31.2], and it grew with the share of programs that were correct before the shift (Figure 2a). Two further tests show where the programs fail. Returning a failing program’s error message to the writer let more programs run but repaired no more items, so the costly errors are wrong values, not crashes (Table 25). Programs that read fields extracted from model-written prose still gave a positive interaction (Table 25). After a deletion, however, the programs re-derived the other items less accurately than the writer did (−15.1 pp), mostly because they returned values for deleted records (Section G).

Shift	Class	Arm	$B$ effect: $D=0$	$B$ effect: $D=1$	$D$ effect: $B=0$	$D$ effect: $B=1$	$B \times D$
Main study: 20 test worlds
$S_{\mathrm{src}}$	definition	exact	+6.0 [+2.5, +9.6]	+33.3 [+28.2, +38.0]	+55.6 [+50.1, +61.1]	+82.9 [+77.0, +89.1]	+27.3 [+21.6, +32.6]
		programs	+6.0 [+2.5, +9.6]	+33.3 [+28.4, +38.0]	+30.0 [+21.6, +38.2]	+57.4 [+49.0, +65.7]	+27.3 [+21.8, +32.7]
	other	exact	+5.4 [+2.3, +8.7]	+4.8 [+1.6, +8.0]	+2.2 [+0.6, +3.8]	+1.6 [−1.0, +4.2]	−0.6 [−3.8, +2.5]
		programs	+5.4 [+2.3, +8.7]	+9.8 [+6.0, +13.8]	−4.0 [−6.8, −1.2]	+0.5 [−1.9, +3.0]	+4.4 [+0.3, +8.6]
	all	exact	+5.6 [+2.8, +8.3]	+12.9 [+10.1, +15.9]	+17.4 [+15.4, +19.7]	+24.7 [+22.7, +26.7]	+7.4 [+5.0, +9.8]
		programs	+5.6 [+2.8, +8.3]	+16.5 [+13.1, +20.0]	+5.7 [+3.0, +8.3]	+16.7 [+14.3, +19.3]	+11.0 [+8.0, +14.0]
$S_{\mathrm{def}}$	definition	exact	−0.8 [−5.7, +4.2]	+38.0 [+32.6, +43.4]	+54.3 [+48.9, +59.7]	+93.0 [+89.0, +96.7]	+38.8 [+32.0, +44.8]
		programs	−0.8 [−5.7, +4.2]	+34.1 [+27.5, +39.8]	+27.6 [+21.7, +34.1]	+62.5 [+57.5, +67.3]	+34.9 [+27.5, +41.9]
	other	exact	+2.3 [−2.4, +7.4]	+1.8 [−3.4, +7.1]	+0.6 [−1.9, +3.2]	0.0 [−3.4, +2.3]	−0.6 [−4.0, +2.7]
		programs	+2.3 [−2.4, +7.4]	+5.8 [−1.9, +13.9]	−3.5 [−10.6, +2.5]	0.0 [−1.9, +1.4]	+3.5 [−3.3, +10.8]
	all	exact	+0.2 [−2.9, +3.4]	+26.9 [+22.3, +31.6]	+37.8 [+33.6, +42.1]	+64.5 [+59.9, +69.1]	+26.7 [+21.2, +31.8]
		programs	+0.2 [−2.9, +3.4]	+25.4 [+19.1, +31.5]	+18.1 [+13.0, +23.5]	+43.4 [+39.3, +47.8]	+25.3 [+18.8, +31.7]
$S_{\mathrm{del}}$	definition	exact	+7.5 [+3.4, +11.5]	+42.9 [+38.0, +47.8]	+52.3 [+47.0, +57.3]	+87.7 [+83.8, +91.5]	+35.4 [+30.4, +40.0]
		programs	+7.5 [+3.4, +11.5]	+43.2 [+37.1, +49.2]	+30.5 [+24.4, +36.0]	+66.2 [+61.4, +71.0]	+35.7 [+30.1, +40.9]
	other	exact	+58.1 [+52.2, +63.7]	+56.6 [+50.6, +62.3]	+0.9 [−0.5, +2.4]	−0.5 [−4.5, +3.2]	−1.4 [−5.4, +2.3]
		programs	+58.1 [+52.2, +63.7]	+25.3 [+21.0, +29.9]	+3.0 [+1.2, +4.8]	−29.7 [−33.9, −25.2]	−32.8 [−37.5, −27.7]
	all	exact	+49.2 [+44.6, +54.0]	+54.2 [+49.4, +58.8]	+9.8 [+8.1, +11.5]	+14.8 [+11.3, +18.3]	+5.0 [+1.7, +8.1]
		programs	+49.2 [+44.6, +54.0]	+28.4 [+24.9, +32.3]	+7.8 [+6.2, +9.5]	−13.0 [−16.9, −8.8]	−20.8 [−25.1, −16.4]
Replication: 20 fresh worlds
$S_{\mathrm{src}}$	definition	exact	+4.2 [+0.6, +7.6]	+37.8 [+33.4, +42.6]	+51.5 [+46.0, +56.4]	+85.1 [+80.7, +89.8]	+33.6 [+28.7, +38.5]
	other	exact	+4.0 [0.0, +8.1]	+2.0 [−2.2, +6.6]	+1.3 [−0.2, +2.8]	−0.6 [−2.3, +1.1]	−1.9 [−3.9, +0.2]
	all	exact	+4.0 [+0.8, +7.0]	+12.3 [+9.4, +15.7]	+15.7 [+13.5, +17.9]	+24.0 [+22.0, +26.2]	+8.3 [+6.3, +10.4]
$S_{\mathrm{def}}$	definition	exact	−1.5 [−3.9, +0.5]	+36.5 [+29.8, +43.0]	+53.7 [+47.5, +60.4]	+91.8 [+88.2, +94.9]	+38.0 [+31.5, +44.5]
	other	exact	+2.2 [−4.7, +8.9]	+3.8 [−2.8, +10.8]	+0.5 [−1.2, +2.3]	+2.2 [0.0, +4.9]	+1.6 [−1.2, +4.8]
	all	exact	−0.3 [−3.2, +2.4]	+25.9 [+20.5, +31.9]	+36.5 [+32.6, +41.1]	+62.8 [+58.2, +67.9]	+26.3 [+21.5, +31.7]
$S_{\mathrm{del}}$	definition	exact	+4.5 [+0.6, +8.7]	+44.7 [+39.0, +50.4]	+48.9 [+43.4, +54.2]	+89.1 [+84.6, +93.4]	+40.2 [+35.6, +44.9]
	other	exact	+56.3 [+50.2, +62.1]	+57.1 [+50.1, +64.3]	−0.4 [−1.2, +0.5]	+0.5 [−4.2, +5.2]	+0.8 [−3.6, +5.2]
	all	exact	+47.0 [+42.0, +51.8]	+54.9 [+48.7, +61.4]	+8.5 [+7.2, +9.8]	+16.3 [+12.2, +20.5]	+7.9 [+3.7, +12.1]

A Claude writer that sees every record comes close to exact recomputation. The $D=0$ writer sees only a 6,000-token selection. For active projects, overdue tasks, and working hours, this selection always held every record that the definition ranges over. For close friends and VIP senders, it never did. Where the selection was complete, Qwen3.5-9B still re-derived only 36.8% of the changed definition values correctly after a source revision. Claude Sonnet 5.5 and Opus 5.5 re-derived 94.4% and 91.7%, as much as exact recomputation on the same items. Over all selections, the interaction after a redefinition was smaller with either Claude writer, +16.4 pp, but still positive (Figure 2b). When we gave Claude Sonnet 5.5 every live record of the definition’s sources as typed fields, it re-derived 91.8% of the changed definition items correctly. This compares with 50.0% from its selection and 97.3% for exact recomputation, at a cost of up to 44,357 tokens per re-derivation. Qwen3.5-9B reached only 7.1% with every record. This writer thus lacks the ability to compute the value, not the evidence (Table 15).

	Complete (%)	$n$	Selection	Typed	Exact	Typed − selection	Exact − typed
(a) Qwen3.5-9B
important contacts	50.0	67	0.0	0.0	100.0	0.0	+100.0
close friends	0.0	79	0.0	0.0	100.0	0.0	+100.0
working hours	100.0	39	20.5 [0.0, 34.7]	20.5 [0.0, 34.7]	100.0	0.0	+79.5 [+65.3, +100.0]
active projects	100.0	40	42.5 [12.5, 72.7]	12.5 [0.0, 37.5]	100.0	−30.0 [−62.5, 0.0]	+87.5 [+62.5, +100.0]
VIP senders	0.0	72	0.0	0.0	100.0	0.0	+100.0
overdue tasks	100.0	69	7.2 [0.0, 17.8]	18.8 [9.0, 29.7]	98.6 [95.7, 100.0]	+11.6 [+3.0, +22.2]	+79.7 [+69.1, +89.5]
all six	58.3	366	8.2 [3.3, 14.0]	7.1 [4.2, 10.3]	99.7 [99.2, 100.0]	−1.1 [−5.8, +3.2]	+92.6 [+89.3, +95.6]
after $S_{\mathrm{src}}$		165	12.7 [3.8, 21.7]	9.1 [2.6, 15.4]	99.4 [98.2, 100.0]	−3.6 [−10.1, +2.5]	+90.3 [+83.7, +97.1]
after $S_{\mathrm{def}}$		201	4.5 [0.0, 9.0]	5.5 [3.2, 8.0]	100.0	+1.0 [−3.4, +4.9]	+94.5 [+92.0, +96.8]
selection complete		176	17.0 [7.1, 28.5]	14.8 [8.8, 21.0]	99.4 [98.3, 100.0]	−2.3 [−12.0, +6.8]	+84.7 [+78.1, +91.0]
selection incomplete		190	0.0	0.0	100.0	0.0	+100.0
(b) Claude Sonnet 5.5
important contacts	50.0	67	56.7 [35.8, 75.8]	65.7 [53.1, 76.7]	88.1 [75.4, 97.0]	+9.0 [−2.9, +19.5]	+22.4 [+9.0, +35.2]
close friends	0.0	79	1.3 [0.0, 3.8]	97.5 [93.8, 100.0]	100.0	+96.2 [+92.4, +100.0]	+2.5 [0.0, +6.2]
working hours	100.0	39	100.0	100.0	97.4 [90.6, 100.0]	0.0	−2.6 [−9.4, 0.0]
active projects	100.0	40	92.5 [83.3, 100.0]	90.0 [77.3, 100.0]	97.5 [93.2, 100.0]	−2.5 [−6.8, 0.0]	+7.5 [0.0, +16.7]
VIP senders	0.0	72	0.0	98.6 [95.6, 100.0]	100.0	+98.6 [+95.6, +100.0]	+1.4 [0.0, +4.4]
overdue tasks	100.0	69	98.6 [95.6, 100.0]	100.0	100.0	+1.4 [0.0, +4.3]	0.0
all six	58.3	366	50.0 [45.2, 54.5]	91.8 [89.8, 93.7]	97.3 [95.4, 98.9]	+41.8 [+38.0, +46.1]	+5.5 [+2.7, +8.4]
after $S_{\mathrm{src}}$		165	45.5 [36.0, 53.1]	93.3 [90.9, 95.4]	97.0 [95.3, 98.7]	+47.9 [+41.3, +55.6]	+3.6 [+1.1, +6.2]
after $S_{\mathrm{def}}$		201	53.7 [49.5, 57.3]	90.5 [86.7, 93.7]	97.5 [95.3, 99.5]	+36.8 [+34.0, +40.0]	+7.0 [+3.1, +11.8]
selection complete		176	95.5 [91.5, 98.9]	94.9 [91.7, 98.6]	95.5 [91.7, 98.8]	−0.6 [−3.4, +2.4]	+0.6 [−1.7, +3.1]
selection incomplete		190	7.9 [4.5, 10.7]	88.9 [84.9, 93.1]	98.9 [96.8, 100.0]	+81.1 [+75.1, +86.9]	+10.0 [+5.0, +14.6]

(b) Contrast (pp)	Llama-3.1-8B	gemma-4-12B	gpt-oss-20b	Qwen3.5-9B
$B$ effect on Update, source revision	+29.6 [+25.1, +34.1]	+9.5 [+5.5, +13.6]	+12.2 [+8.0, +16.6]	+7.8 [+4.4, +11.7]
$D$ effect on accuracy, source revision	+20.4 [+18.0, +23.1]	+13.1 [+11.6, +14.6]	+12.0 [+10.5, +13.5]	+13.7 [+12.5, +15.2]
$D$ effect on accuracy, redefinition	+40.0 [+36.1, +43.3]	+31.9 [+28.4, +35.0]	+26.5 [+23.7, +29.5]	+33.6 [+31.2, +36.3]
$B \times D$ on Update, source revision	+4.2 [−4.2, +13.1]	+0.8 [−2.1, +3.8]	+12.6 [+9.6, +16.5]	+6.2 [+2.5, +10.3]
$B \times D$ on Update, redefinition	+20.8 [+11.7, +29.4]	+14.4 [+8.3, +20.1]	+28.5 [+23.1, +32.8]	+27.1 [+22.9, +32.1]
$B \times D$ on Update, deletion†	+26.4 [+20.5, +31.9]	+7.8 [+4.9, +10.7]	+13.0 [+9.8, +15.4]	+5.0 [+1.7, +8.1]
No-store BM25 − HoldMem(1,1)	−50.5 [−53.3, −48.4]	−36.3 [−40.1, −32.3]	−30.1 [−33.2, −27.2]	−45.1 [−48.6, −42.2]

Keys Built from Citations Miss Records That Join Later

The citation key notices every redefinition but misses later changes. It missed 16.9% of the changed items after a source revision and 32.4% after a redefinition. The key holds the definition version, so it invalidated all 222 changes that the redefinition caused. It then missed a later change of records or of the clock on 54 of them. After a redefinition, every wrong answer on changed definition items at $D=1$ came from such a miss ($r_1 = 100.0\%$), so this key fails at detection, not at repair. Most misses were records that entered the item’s scope after the write, as in the phantom problem of predicate locks [43], or records that left a time window (Section F.5). Averaged over $D$, the citation key raised Update only after a deletion (+42.5 pp), because the writer’s citations name the deleted record. Forcing the memory to re-derive the definition items it missed after a source revision would have gained +2.6 pp at $D=0$ and +94.9 pp at $D=1$ (Section F.5).

The source key misses changes caused by the clock alone. The key does not track time, so it catches such a change only if an unrelated record of the same source arrives in the same interval. Such stale values caused 5 of the 6 wrong answers of HoldMem(1,1) on changed definition items (Section F.5).

An export that carries the keys keeps detection. We exported HoldMem(1,1) to Portable Memory with its keys and imported it into a second implementation. The import matched the original (0.0 pp). Without the keys, the import detected no later change (Section F.10).

Robustness and Native Systems

Fresh worlds, new definitions, and rewritten text. On 20 fresh worlds, every contrast with the source key replicated (Section F.8). We also replaced the six definitions with 18 held-out definitions from the same grammar on 10 worlds. With intervals that resample both worlds and definitions, the interaction on definition items after a redefinition was +33.0 pp [+22.1, +44.9]. When gemma-4-12B rewrote the message bodies around unchanged gold facts, the effects of $B$ and $D$ and their interaction kept their sign (Section F.8).

Writer switches fail mostly on definition items. At each of the three tiers, at least 97% of the answers that were correct before a switch stayed correct, and 32 of the 41 items that failed were definition items. Over all 8 switches, the new writer’s own accuracy predicted which items stay correct at least as well as the agreement between the old and new writers did (Section F.7).

Stored state pays off over time. Before any shift, no-store BM25 scored −25.8 pp compared with HoldMem(1,1) at twelve months, while the source key used 3.64 times as many tokens (Section F.6).

Documented settings match behavior but do not predict carryover. For each native system, the $B$ setting inferred from how many deleted items it still answered matched its documentation. Yet most of their Update and Stability values fell outside the interval of the HoldMem cell with the same setting (Section F.9). The per-item terms explain more. The tested Mem0 release never updates or deletes a memory, so $m=1$ for every item class, yet the class decides which fix helps. A program run at query time on the live typed records, which Mem0 does not store, raised its Update on changed definition items after a redefinition by +64.4 pp. Invalidation keyed on the source record raised its Update on the other items after a deletion by +35.9 pp.

Discussion

Detection is worth as much as the repair that follows. With a language model as writer, the re-derivation is not exact, and it is worst for values that aggregate many records under a person’s definition. A key helps there mainly with exact recomputation or with a program over typed records. A strong writer that sees every record comes close, but it must read every record at each re-derivation.

A final-answer score hides which step failed. The diagnosis names the term that failed and the item class it failed on, and the class tells which fix helps, as the Mem0 interventions show. On three-month worlds, the ranking of the six HoldMem cells by Update after a source revision nearly fixes their ranking after a redefinition (Spearman $\rho= 0.94$ [0.95, 1.00]), but among the six native systems it does not ($\rho= 0.52$ [−0.09, 0.94]).

Recommendations. We recommend reporting $m$ and $r$ by item class, and Table 5 lists design rules for memories.

Rule	Evidence
1. Key everything that can change after the write, and export the keys	Citation keys miss 16.9% and 32.4% of the changed items after a source revision and a redefinition, mostly records that join later. An import without the keys detects no change.
2. Compute definition items with a program over typed records, one per definition version	Four open-weight writers re-derive at most 25% of the changed definition items correctly, against at least 91% for exact recomputation. The programs of Qwen3.5-9B keep most of the interaction.
3. Remove the items whose record is deleted	After a deletion, the key beats a refresh that keeps old items by +9.9 pp on the other items at $D=1$. Invalidation keyed on the source record raises Mem0’s Update there by +35.9 pp.
4. Check that the record a program names still exists	After a deletion, the programs re-derive the other items less accurately than $D=0$ (−15.1 pp). Of their loss on deleted records, 67.7% is a value returned where the gold status is abstain.
5. Test a new writer on definition items	At the three tiers, 32 of the 41 items that fail a switch are definition items.

Conclusion

We proposed MemShift, a benchmark for evaluating LLMs’ performance in derived personal memory. MemShift scores each item that a shift changes on detection and repair, against gold fixed before any text. Detection helps only as much as repair allows. Repair fails on the items computed by the person’s definitions, and programs over typed records close most of this gap. Keys built from the writer’s citations miss records that join an item later. A memory’s accuracy after a change measures its detection and its repair together. Future work can test memories in which a language model decides invalidation, and real records with vaguer definitions.

Limitations

Synthetic worlds and codified definitions. The worlds are synthetic and rendered from templates, and each definition is a one-sentence rule over structured fields. This lets an evaluator compute gold values and dependencies exactly, without annotators, but real records are messier and real definitions vaguer. Only a small share of the message text was rewritten by a model, field extraction was tested only on model-written prose, and the held-out definitions come from one grammar. Programs written from definitions in the user’s own words still give a positive interaction (Section F.4).

Reference memory and native systems. Every cell with a known $(D,B)$ setting is a HoldMem cell that we designed. $D=1$ runs the same evaluator as gold, so it is an upper bound on repair, not a system under test. The native systems ran on three-month worlds at $D=0$ through our adapters, and their ingestion is noisy. The tested Mem0 release only adds memories, and Graphiti finished no world, so no native run measures invalidation decided by a language model.

Models, readers, and serving format. The main writers and readers are open-weight models with 8–27B parameters. Claude Sonnet 5.5 and Opus 5.5 wrote only definition items, and only Sonnet received every record of a definition. Update also depends on the reader and on how the store is presented. Serving the same $D=1$ store as JSON records changes Update (Table 24), so the signs of the effects generalize better than their absolute levels.

Shift	Class	Main study: $s_k$	Main study: JSON	Main study: Update	Main study: Stability	Replication: $s_k$	Replication: JSON	Replication: Update	Replication: Stability
$S_{\mathrm{src}}$	definition	99	70	$-29.4$ [$-33.6$, $-25.2$]	$-38.7$ [$-50.0$, $-27.6$]	99	67	$-32.1$ [$-35.5$, $-28.7$]	$-36.4$ [$-49.4$, $-22.2$]
	other	91	89	$-1.3$ [$-2.8$, $-0.1$]	$-0.6$ [$-1.5$, $+0.1$]	90	90	$+0.1$ [$-1.3$, $+1.6$]	$-0.4$ [$-1.2$, $+0.4$]
	all	93	84	$-9.3$ [$-10.9$, $-7.7$]	$-2.7$ [$-3.9$, $-1.6$]	92	83	$-9.1$ [$-10.6$, $-7.7$]	$-2.4$ [$-4.0$, $-1.0$]
$S_{\mathrm{def}}$	definition	100	54	$-45.5$ [$-48.9$, $-42.2$]	$-46.2$ [$-80.0$, $-20.0$]	99	56	$-43.4$ [$-48.1$, $-38.8$]	$-30.8$ [$-54.5$, $-8.3$]
	other	73	72	$-1.2$ [$-3.9$, $0.0$]	$0.0$ [$-0.8$, $+0.7$]	86	83	$-2.7$ [$-5.8$, $0.0$]	$-0.2$ [$-0.8$, $+0.4$]
	all	92	60	$-31.9$ [$-35.4$, $-28.6$]	$-1.1$ [$-2.3$, $0.0$]	95	65	$-30.3$ [$-33.5$, $-27.1$]	$-0.9$ [$-1.9$, $0.0$]
$S_{\mathrm{del}}$	definition	99	67	$-32.8$ [$-36.3$, $-29.0$]	$-33.3$ [$-39.2$, $-27.2$]	100	64	$-36.0$ [$-39.9$, $-32.0$]	$-30.2$ [$-37.9$, $-23.1$]
	other	68	16	$-51.4$ [$-56.8$, $-45.1$]	$-0.3$ [$-1.0$, $+0.2$]	67	16	$-50.4$ [$-55.5$, $-45.1$]	$-0.3$ [$-0.7$, $0.0$]
	all	73	25	$-48.2$ [$-52.7$, $-43.0$]	$-2.9$ [$-3.8$, $-2.1$]	73	25	$-47.8$ [$-51.9$, $-43.5$]	$-2.6$ [$-3.5$, $-1.8$]

Statistics. The main-study intervals rest on 20 worlds, which are few clusters for a bootstrap [18]. The replication repeats every contrast on 20 fresh worlds. Intervals are per contrast. Holm and Benjamini–Hochberg corrections over the main contrasts keep 35 and 36 of the 36 intervals that exclude zero (Section E.2).

Ethics Statement

This work concerns systems that store what AI assistants know about individual people. Measuring whether this knowledge stays correct after models, sources, or products change can make assistants more reliable and make it easier for people to take their data with them. The same capabilities, persistent and portable personal profiles, raise privacy risks. All worlds in MemShift are synthetic, no personal data was collected, and no human subjects were involved. Derived profiles also hold facts about third parties, such as the employers and cities of a person’s contacts, and exported provenance reveals which sources and models produced each item. The benchmark scores consent revocation as an outcome, and every stored derived memory we measured still leaks after a revocation (Section F.10). The writer’s programs run only on synthetic records, in a separate process with a time limit and a memory cap. A restricted interpreter accepts only listed syntax and built-in functions (Section C.2).

Reproducibility Statement

Worlds, shifts, and probes are generated deterministically from seeds, and a deterministic scorer checks every answer without an LLM judge. Open-weight models run at temperature 0 with constrained decoding, except for the sampled program sets (temperature 0.7). Claude Sonnet 5.5 and Opus 5.5 ran through Anthropic’s API with the settings in Section C.7. Every run records its code version, world seeds, model revisions, and output hashes in a manifest. The released repository holds the code, generator, worlds, gold labels, all model responses, and run manifests, and it regenerates every number in the paper.

Acknowledgements

This work was supported by JST CREST Grant Number JPMJCR22M2 and JSPS KAKENHI Grant Numbers JP23K17456, JP23K28096, JP25H01117, JP25K21207, and JP26K03246.

References

Extended Related Work

This section describes the closest work of § 2 in more detail. It then groups further work by the question it answers for MemShift.

Memory systems and the closest work. The agent memories in the main text are built for recall. We ask what happens to the values they derived when their inputs change. StateMemBench [45] uses scripted dialogues rendered by an LLM and an LLM judge that scores whether an answer is current or outdated. The scored answers include derived values that are not recomputed after an input changes. It runs eight memory and graph-RAG systems, but every change arrives as dialogue text, and it scores a stale answer, not whether the change was noticed. Its method, StateMem, records derived-from links between memory entries at ingestion, flags the entries that depend on a changed entry without calling a model, and lets an LLM recompute a flagged value at answer time. In our terms, its detection is a key recorded at write time, close to our citation key, and its repair is $D=0$ at query time. Such links miss records that join a value’s scope after it was written (Section F.5). Moreover, Qwen3.5-9B re-derived fewer than half of the changed definition items correctly even from complete evidence (Section F.4). MemTrace (Long et al., 2026) holds one typed fact fixed and varies its age, the question type, and the evidence, with an LLM judge. We instead hold the probes fixed and change what lies behind them. PLANFENCE [21] binds a derived plan to the declared versions of its inputs and checks them before an action. Its repair is always an LLM replan, so it does not vary how a value is recomputed. RECON [7] asks which derived conclusions lose support after an invalidation, but it never compares a stale store with its repaired version. MemoRepair (Zhao et al., 2026a) withdraws everything derived from an invalidated source and rebuilds it by deterministic recomputation, regeneration, or removal. It assumes complete provenance. In our terms, its detection is a key over complete dependencies, like our oracle key built from the gold dependencies (Section C.4). Its deterministic recomputation is $D=1$, and its regeneration is $D=0$. We measure each part when the dependencies are recorded at write time and a language model re-derives. Goyal and Ray (2026) switch the writer or the embedder over a stored state in four idealized formats while the history stays fixed. They find that the representation decides what survives.

Stale values in agent memory. Supersede trains a small model to avoid stale values in a notes memory that the agent rewrites itself (Patel, 2026). RD-Forget suppresses superseded facts through slots that an LLM curator assigns [89]. Reliable post-retrieval assembly uses code to pick the newest version among candidates that an LLM extracted (Reddy and Challaram, 2026). Supersede and the assembly rule replace single facts, so neither derives a value from several records, and RD-Forget’s slots are judged by a model rather than keyed to source versions. Selective QA over conflicting personal sources scores one fixed snapshot (Yang et al., 2026), the setting of our conflict probe. Sleep-time compute precomputes quantities from stored context [91], and carryover asks whether such quantities stay correct when the context changes. Close to our writer switch, agents keep one identity across models and hosts (Zhao and Zhao, 2026), and readers adopt stale memory values (Hu and Ramachandran, 2026). Labels extracted from clinical summaries change with the size of the extraction model (Murin, 2026). Comparisons of memory designs find no universal winner [72] (Zhou et al. 2026a) and find that the backend matters more than the reader’s size (Zhang and Ma, 2026). Checking drafts against stored state repairs implicit stale dependencies (Sun and He, 2026). Temporal question answering [23, 76] (Liška et al. 2022; Vu et al. 2024) and work on knowledge conflicts (Xu et al., 2024) study facts that change over time and model knowledge that goes out of date.

Knowledge editing. Knowledge editing asks whether a model’s answers, including multi-hop answers and ripple effects, follow an edit to its weights [29] (Meng et al. 2022; Meng et al. 2023; Zhong et al. 2023). Memory-based editors [65] (Mitchell et al. 2022) record the edits but not what a stored value was derived from.

Closest benchmarks compared property by property. Table 6 compares MemShift with the 15 benchmarks of our audit, and the counts in the main text come from it. Every entry is read from the benchmark’s paper, and the supporting quotes with their pages are released with the code. StateMemBench, MemTrace, PLANFENCE, and MemoRepair are discussed above. The table uses the following definitions. A benchmark has derived items when its graded items aggregate several records (counts, sets, sums, multi-hop, or temporal aggregation) instead of restating one stored fact. MemShift has them only partly: a writer derives every item, but of a world’s 48 items only the six definition items aggregate records, the two appointments pick the earliest of several, and the rest are attributes or latest values. An implicit conflict is conflicting information without an explicit replacement, which the system must detect. Consent is revoked when the user withdraws consent or asks to forget, and the memory must then stop using or delete the data. A benchmark is counterfactual when it runs the same history with and without the change (MemShift has byte-identical branches). Native systems are the existing memory systems that a paper evaluates, grouped as the paper

Benchmark

World text

Gold fixed before text

Gold dependency sets

Derived items

Source revision

Deletion

Implicit conflict

Redefinition

Writer switch

Export

Consent revoked

Counterfactual branches

No LLM judge

Native systems

Histories

Questions

LoCoMo ((Maharana et al., 2024))

L+H

∼

∼

∼

×

×

×

×

×

×

×

×

✓

0

50

7,512

LongMemEval ((Wu et al., 2025))

L+H

✓

∼

∼

✓

×

∼

×

∼

×

×

×

×

2

500

500

MemoryAgentBench ((Hu et al., 2026))

M

∼

×

∼

✓

×

∼

×

∼

×

×

×

∼

10

130

2,071

HaluMem ([20])

L+H

∼

✓

∼

✓

∼

∼

×

×

×

×

∼

×

6

20

3,467

MemoryArena ([66])

M

∼

∼

✓

∼

×

×

×

×

×

×

×

n/s

5

n/s

701

AMA-Bench ((Zhao et al., 2026b))

M

∼

∼

∼

✓

×

∼

×

∼

×

×

×

×

11

208+

3,696

EvoArena ((Xu et al., 2026))

M

∼

∼

∼

✓

∼

✓

×

∼

×

∼

×

✓

1

10

505

STALE ([19])

L+H

✓

✓

∼

✓

×

✓

×

×

×

×

×

×

5

400

1,200

RECON ([7])

L

✓

✓

∼

∼

✓

✓

×

×

×

×

∼

4

24

1,604

DynamicMem ((Xie et al., 2026))

L

✓

∼

∼

∼

×

✓

×

×

×

×

×

4

10

3,634

Ground Truth First ((Spencer, 2026))

L

✓

∼

∼

✓

×

∼

×

×

×

∼

×

0

20

383

Goyal and Ray ([55])

n/s

✓‡

∼

∼

✓

×

∼

×

✓

∼

×

✓

✓

0

48

7,680

Jiang et al. ([74])

R

×

‡

✓

∼

✓

×

×

×

×

∼

×

0

n/s

2,907

MemSecBench ([25])

L+H

∼

∼

×

∼

∼

∼

×

∼

×

×

∼

×

4

n/s

930

GateMem ((Ren et al., 2026))

L+H

∼

∼

∼

✓

✓

×

×

∼

×

✓

×

×

3

91

2,218

MemShift

T (+L)

✓

✓

∼

✓

✓

∼

✓

✓

∼

✓

✓

✓

6

40

19,562

ends_mid=1 groups them, without long-context and plain RAG baselines, the paper’s own systems, or oracles. Histories (worlds, users, or trajectories) and questions are counted as the paper reports them. For MemoryArena the questions are its tasks, and for EvoArena the counts are those of its PersonaMem-Evo subset.

Checking derived values. LLM extraction and semantic operators build structured views of documents ([6, 63] (Lin 2026; Patel et al. 2025; Shankar et al. 2025; Zhang et al. 2026)). Data-quality systems check and repair such outputs with declarative constraints ([84] (Rekatsinas et al. 2017; Schelter et al. 2018)) or synthesized assertions ((Shankar et al., 2024)) including incremental checks ([75]). ChatDB keeps memory as SQL tables that the model queries and updates ([69]). Our check of a stored value against its records is a per-item constraint. In our results it repairs only when a program also recomputes the value from the memory’s typed records (Section F.4).

Dependency tracking. Our key is standard dependency tracking: the memory compares a stored key with the current one to decide whether to rebuild ((Mokhov et al., 2018)). Self-adjusting computation and incremental view maintenance recompute only what a change affects ([1, 14, 50, 64] (Gupta and Mummick 1995; McSherry et al. 2013)), and

a redefinition corresponds to a view redefinition ([60]). Deletion propagation asks the reverse question, which source tuples to delete so that a given view tuple disappears ([16]). Truth maintenance withdraws beliefs whose justifications fail ([38]), and assumption-based truth maintenance labels each belief with the assumption sets it rests on ([36]). A value defined by a predicate depends on records outside any read set, which is what predicate locks capture ([43]). Build systems and view maintenance gain from invalidation because their rebuild is deterministic. When a language model rebuilds, the result is lossy in a way that depends on the item class and the writer, and a read set or citation list recorded at write time is not a sound dependency set. Dependency-guided rollback repair scores the invalidation of claims separately from recovery ((Wang et al., 2026b)). Database semantics for agent state include versioned resources with isolation ((Mozafari, 2026)), transactional belief commits ([88]), bitemporal operators ((Snodgrass, 1999; Wang, 2026)), and dependency masks ([87]). Temporal memories retire or revoke superseded values ((Yadav, 2026; Zhou et al., 2026b)).

Provenance and freshness. BeliefBank’s constraints relate beliefs to one another ([77]), not a derived value to the source versions it was built from. Provenance models ([15, 17, 26]) Green et al., 2007; W3C, 2013) and provenance systems for agents and LLM processing (Lin et al., 2026; Souza et al., 2025; Wang et al., 2026a) record or infer what a result was derived from. Our key is provenance reduced to what invalidation needs: the event counts of the sources, the definition version, and the writer. View maintenance [30] (Zhuge et al. 1995), session guarantees (Terry et al., 1994), and stream processing [2] (Sotolongo et al. 2025; Tahir et al. 2024) state which input changes a derived result must reflect. For derived items, Update measures how often a read after a shift reflects the shift. Our sync simulators apply the matching contracts of consumer APIs to a personal memory (Section B).

Portability, control, and consent. Tool protocols such as MCP (Model Context Protocol contributors, 2025) and personal agents over a user-controlled repository [13] let different models reach a person’s applications and data. For this reason, the derived memory, not the model, has to carry state across a change of model. Portability is a legal right [44] and an engineering practice [35]. Portable Memory [80] (MacPaw 2026), MIF [3], and memorywire (Munirathinam, 2026) carry provenance, but none records the definition versions that a derivation depended on. Only MIF can record the version of a source document, as an optional field with no invalidation rule. We make such a derivation extension a controlled variable. MIF records the version of the agent that created an item, which matches the writer component of our key. Personal knowledge graphs, personal data stores, local-first software, enclaves, and user-owned agents keep a person’s data under the person’s control [78, 79] (Liu 2026; Mansour et al. 2016; Skjæveland et al. 2024). Usage control, authentication and authorization, information-flow control for agents, and data-use policies enforce consent over records [8, 12, 28, 31, 37, 42] (Pang et al. 2019; Park and Sandhu 2004; Tsai and Bagdasarian 2025; Upadhyaya et al. 2015; Upadhyaya et al. 2018). Our failure-mode probes show that revoking consent over derived items needs the dependency set, which a recorded read set only approximates (Section F.10).

Agent benchmarks. Interactive benchmarks place agents among a person’s apps and tools or with simulated users [10] (Lu et al. 2025; Trivedi et al. 2024; Yao et al. 2025), and they evaluate what an agent does in one episode. Carryover instead evaluates what a memory still knows after its inputs have changed.

Benchmark Construction

This section specifies the worlds, items, gold labels, branches, and scoring of § 3. It follows one item of a released world through a redefinition. It also describes the variants used in the further experiments: held-out definitions, definitions in the user’s own words, unnamed probes, rewritten message bodies, and simulated sync. Generation is deterministic, so a seed and a generator version reproduce a world byte for byte.

A world is a log of versioned source objects. We sample the true changes of a world before any text exists. Each change inserts, updates, or deletes a source object (called a record in the main text). Each source object has an object id and a revision number that only increases, and a deletion leaves a tombstone. Deterministic templates render the objects as text. Background traffic adds distracting text: newsletters that mention the same organizations and cities, small talk, routine work mail, and one-off meetings. A fidelity checker confirms that every gold value can be found in the text of its dependencies. It succeeded on 100.0% of the sampled items and times in the development worlds and on 274,232/274,232 gold facts over all branches of the test worlds. A rendered world that fails the check is not used.

Forty-eight items. Each world has six attributes of the person (employer, manager, home city, time zone, diet, and commute) and an employer and a city for each of twelve contacts. It also has four recurring series, the next dentist and doctor appointments, and six task due dates. The six definition items are important contacts, close friends, active projects, working hours, VIP senders, and overdue tasks.

Gold is computed from the live source objects. For every item and time, an evaluator returns the gold value, the gold status, and the dependency set, that is, the source objects with their revisions. A person attribute follows the world’s script, and a contact attribute is read from the contact card. A series or a due date is read from its object, and an appointment is the earliest upcoming one of its type. A definition item applies the definition version in effect. An item with no current value, because its object is deleted or no appointment lies ahead, has the gold status abstain. Each definition has a version v1 and a version v2 over structured fields of the sources: calendar entries, contact cards, tasks, message and e-mail headers, and a working-hours note. For example, v1 of important contacts counts one-off meetings over a longer window than v2 and requires more of them. No annotator labels an item.

Branches differ only in the shift. A world forks at the end of month nine. Each shift draws from its own random stream, so all branches of a world share every other event byte for byte. A source revision ($S_{\mathrm{src}}$) revises, through ordinary events, the sources behind a sampled share of the items. These events are a job change (with a new manager) or a reorganization, a relocation (with a new time zone), a new diet or commute, an edited contact card, a rescheduled or canceled series, a booked, moved, or canceled appointment, and a new due date. For a definition item, they are new meetings, messages, e-mails, tasks, or a new working-hours note. A redefinition ($S_{\mathrm{def}}$) moves half of the definitions to v2 at the fork and adds no source event, and gold is evaluated under v2. A deletion ($S_{\mathrm{del}}$) deletes the source object behind half of the items read from a series, task, or contact card. A consent revocation ($S_{\mathrm{cons}}$) revokes one scope of one agent. Probes asked by that agent whose dependencies touch the scope have the gold status blocked. The conflict branch of the failure-mode probes adds, on the same day, an update to a contact card and a message from that contact. The two live sources state different values for the contact’s city or employer, and gold is abstain. A writer switch ($S_{\mathrm{wr}}$) and an export ($S_{\mathrm{impl}}$) change the memory, not the world, so they need no branch of their own. The unshifted branch has the same events as the shifted branches up to the fork and no shift after it.

An unstated redefinition limits carryover. If the person does not state a redefinition, no memory can observe it. In the failure-mode probes, carryover after an unstated redefinition is thus at most 6.7% of 15 items, the share whose old answer equals the new gold.

Scoring is typed matching. No LLM judge scores an answer. Dates, times, and weekdays are parsed and compared as values. Entities are compared after normalizing case, punctuation, and whitespace. Diet and commute values use a fixed table of synonyms, and sets are compared as sets. An answer is correct only if its status also equals the gold status.

A worked example. Table 7 follows one definition item of a released test world through a redefinition. It shows the definition in effect, the records that decide the gold value, the item that every HoldMem cell stored before the fork, and the answers of three cells before and after the redefinition.

question	Which projects are the user’s active projects under the user’s current definition? List project names.
	Before the fork, probe on Sep 30, 2026	One week after the redefinition, probe on Oct 8, 2026
definition	v1: An active project is a project with at least one open task.	v2: An active project is a project with at least one task created or updated in the last 14 days (any status).
records	Project	Task	Status	Last updated	Gold v1	Gold v2
	Beacon	Prepare design doc	open	Jun 28	•
	Delta Migration	Present customer survey	open	Aug 17	•
	Horizon Audit	Test design doc	open	Sep 29	•	•
	Horizon Audit	Update hiring plan	open	Oct 5		•
	Before the fork	After the redefinition	What the memory did
gold stored item	Beacon, Delta Migration, Horizon Audit “As of 2026-09-30, the user’s active projects are Beacon, Delta Migration, and Horizon Audit, as these are the only projects with at least one open task.”	Horizon Audit
HoldMem(0,0)	Beacon, Delta Migration, Horizon Audit ✓	abstains ×	kept
HoldMem(0,1)	Beacon, Delta Migration, Horizon Audit ✓	Garnet, Beacon, Delta Migration, Horizon Audit ×	invalidated, re-derived
HoldMem(1,1)	Beacon, Delta Migration, Horizon Audit ✓	Horizon Audit ✓	invalidated, re-derived

Held-out definitions come from a grammar. A grammar samples a definition as the records of one source that meet a condition, an aggregate over them (count, members, or span), a window, and a threshold. Its pairs of source and condition differ from those of the six fixed definitions. The second version changes the window or the threshold, and, as for the six, a redefinition moves half of a world’s definitions. A filter computed from gold alone rejects candidates whose gold value rarely changes or is mostly empty. It would also reject two of the six fixed definitions. The windows and thresholds of the grammar were set from gold statistics alone, and no model output enters the draws. Each of three draws of six definitions (Table 8) replaces the six fixed definitions on ten test worlds.

Definitions in the user’s own words keep the codified content. The program request of the writer’s programs states each definition in a codified form. For this experiment, we restated each of the six definitions as the user would say it, with the same sources, thresholds, windows, and conditions. For important contacts, the first version reads “Count someone as an important contact if they’ve been at five or more of my one-off meetings in the past 90 days. Recurring series don’t count.” The record fields, the item’s sources, and the task text, which form the memory’s own data model, stay as they are.

Unnamed probes leave out the definition. A probe about a definition item ends with a sentence that names the definition in effect. An unnamed probe leaves out this sentence and still asks for the value under the user’s current definition.

Rewritten worlds keep every gold fact verbatim. gemma-4-12B rewrites the bodies of e-mails, chat messages, and notes at temperature 0. Headers and every field read by gold, the evaluators, and the shift scripts keep their template values, so gold is the same as in the template world. Structured records, delete events, and newsletter digests stay templated. The digests hold no gold fact and make up 99.0% to 99.2% of the body text. A rewritten body is accepted only if the reply is complete JSON and every gold fact that gold reads from the record appears verbatim in one of its surface forms. The rewrite must also add no number, month, or weekday name, and no person, company, or city from the generator’s pools. A failing body is retried once and then kept as the template. A rewritten world is used only if the fidelity checker finds every gold value (or as many as in the template world).

Sync simulators follow documented contracts. The sync experiment replaces ideal delivery with the documented incremental-sync contract of a widely used API for each source ([51 c,d]). These are Google Calendar events.list (sync token, full resync on 410), People connections.list (sync token, 7-day expiry), Gmail history.list (history id, full resync on 404), Google Tasks tasks.list (updatedMin), Google Drive changes.list (page token), and Slack conversations.history (oldest). Deletions arrive as tombstones except in chat. WebDAV sync (Daboo and Quillaud, [33]) (used for CalDAV (Daboo et al., [34])), JMAP (Jenkins and Newman, [73]Microsoft, 2026`) follow the same pattern. A client polls every 15 minutes in pages of 50 with a visibility lag $L$, so cursors never expire. The only full resyncs are the calendar’s documented invalidations of its sync token (probability 0.001 per poll).

Gold v1, v2: • when the record is in the dependency set of the gold value under that version. ✓ or ×: status and value match gold or not. Bold: the changed gold value and the answer that matches it.

Worlds are seeded and versioned. In the two closest generators, events are text without object identity (Spencer, 2026), or they come from a language model called at every stage with an unseeded random number generator (Xie et al., 2026). Both judge answers with a language model. Our generator is seeded and versioned, and its events are typed operations on versioned objects, with object revisions, deletions, versioned definitions, and per-agent consent.

Memory Systems, Baselines, and Models

This section describes the reference memory HoldMem and its cells, the baselines, the writer’s programs, the refresh and upsert cells, and the controls. It then describes the native systems, the placement of real memory systems on the $(D,B)$ grid, and the models.

HoldMem Cells

Four cells and two keys. The cells are $(D,B) \in\{0,1\}^{2}$. A $B=1$ key is either a source key or a citation key, which gives the six cells of the main experiment. The source key holds the number of events that each of the item’s sources has received, the definition version, and the writer. The citation key holds the same count for each record group, such as a message thread or a contact card, that contains a record the writer cited, together with the definition version and the writer. An item is the tuple $\{\text{item\_id},\text{statement},\text{key}\mid\varnothing,\text{typed}\mid\varnothing,\text{status},\text{history}\}$. The key is empty at

$B=0$. The typed value with its located spans (and the writer’s program at $D_m$) is empty at $D=0$.

Writing and maintaining an item. To write an item, the writer receives a BM25 selection of the records of the item’s sources (6,000-token budget, in time order). It returns JSON with a typed value, a one-sentence statement, and up to six cited records with exact quotes (Section D). A $D=1$ cell also keeps the typed value with the located spans of its quotes. A checker verifies the value against the live records at write time and at repair time. If the check fails, exact recomputation supplies the value. Exact recomputation has no rule for the five free-text items, so for them $D=1$ only checks, and an item that fails the check is marked unverifiable and not served. During maintenance, a $B=1$ item whose key changed is invalidated and re-derived by the current writer. A $B=0$ cell derives new items from the delta, that is, the records that arrived since its last write. It skips the writer when the delta holds no source record, and it never invalidates. After a writer switch it re-derives nothing and keeps every answer, so we leave it out of the retention test, which requires that at least 97% of the correct answers survive a switch (Section E). With exact recomputation, the stored statement names the definition version it evaluated. The template control measures what this presentation adds (Section F.4).

Every cell starts from the same reader context. All cells are built from the same writer outputs. Before any intervention, the reader context of every probe is hashed. every cell, and a run stops unless all hashes match. After a shift, the cells thus differ only in what the key invalidates and in how an invalidated item is re-derived.

(a) Definition	$S_{\mathrm{src}}$ $B \times D$	$S_{\mathrm{src}}$ $n$	$S_{\mathrm{def}}$ $B \times D$	$S_{\mathrm{def}}$ $n$	$S_{\mathrm{del}}$ $B \times D$	$S_{\mathrm{del}}$ $n$
important contacts	+75.4 [+60.3, +88.1]	61	+60.3 [+48.5, +73.8]	63	+72.4 [+57.4, +85.2]	58
close friends	+35.4 [+26.6, +44.3]	79	+35.4 [+28.2, +42.5]	79	+42.3 [+34.2, +50.0]	78
working hours	0.0	22	+30.9 [+18.6, +43.6]	55	0.0	6
active projects	−20.0 [−55.3, +10.4]	40	+59.1 [+29.5, +82.6]	44	+2.9 [−31.0, +34.2]	35
VIP senders	+4.3 [0.0, +9.4]	70	+2.7 [0.0, +6.7]	74	+2.8 [−2.8, +10.1]	72
overdue tasks	+36.1 [+23.2, +49.3]	61	+54.2 [+37.5, +69.0]	72	+52.5 [+34.5, +69.0]	59
leave one out	+16.5 to +33.8		+34.6 to +47.3		+26.8 to +45.3

(b) Draw	Definition	Source	Condition	Aggregate	v1 → v2	$S_{\mathrm{src}}$	$S_{\mathrm{def}}$	$S_{\mathrm{del}}$
1	priority project	tasks	high priority task	count	≥1 in 30 d → 45 d	+73.3	+107.7	+106.2
	social hours	calendar	social meeting	span	21 d → 14 d	+26.9	+28.6	+29.2
	recently completed task	tasks	done task	members	7 d → 14 d	+33.3	+43.8	+53.1
	evening chat partner	messages	evening message	count	≥1 in 21 d → ≥3	+2.5	+7.5	+2.5
	after hours sender	email	after hours email	count	≥1 in 90 d → ≥2	+4.3	+3.6	0.0
	team regular	calendar	group meeting	count	≥1 in 90 d → ≥2	+13.3	+35.3	+6.7
2	frequent recipient	messages	sent message	count	≥2 in 60 d → ≥4	0.0	+5.4	0.0
	deep dive partner	calendar	long meeting	count	≥2 in 60 d → 90 d	+36.4	+37.9	+39.4
	after hours sender	email	after hours email	count	≥3 in 90 d → ≥2	0.0	0.0	0.0
	productive project	tasks	done task	count	≥1 in 30 d → 60 d	+51.5	+50.0	+80.0
	team regular	calendar	group meeting	count	≥2 in 45 d → ≥1	+40.7	+41.0	+44.4
	priority project	tasks	high priority task	count	≥1 in 45 d → ≥2	+64.3	+68.2	+93.8
3	frequent recipient	messages	sent message	count	≥3 in 21 d → ≥2	+20.5	+10.3	+25.6
	social hours	calendar	social meeting	span	45 d → 14 d	+11.1	+25.0	+6.2
	upcoming task	tasks	due ahead	members	7 d → 21 d	+53.3	+63.2	+65.5
	recently completed task	tasks	done task	members	14 d → 21 d	+30.8	+50.0	+42.1
	hot task	tasks	high priority task	members	7 d → 60 d	+20.0	+29.4	+18.2
	close relative	messages	family message	count	≥8 in 30 d → 60 d	+33.3	+38.5	+45.5

Why the citation key counts only cited record groups. A key over every record group that a derivation read would keep firing on busy groups that read sets include, such as newsletter threads. The citation key thus counts only the groups of the records the writer cited, in both $D$ arms. On the test worlds, the source key invalidated +63.7 pp [+59.9, +67.5] more of the unchanged items than the citation key after a source revision. Section F.5 shows what the citation key misses and why.

Baselines. The main study includes no-store BM25, plain RAG, and the oracle context. No-store BM25 keeps nothing and runs BM25 over the live records at every query. Plain RAG retrieves records with dense embeddings. The oracle context gives the reader the records that the gold label depends on.

depends on. The JSON arm serves the same $D=1$ store as JSON records instead of statements.

The Writer's Programs and the Check-Only Cell

The $D=1$ cell repairs a failed span check with exact recomputation. For definition items, both the check and the repair call the evaluator that computes gold. The $D_m$ cell keeps the checker but takes the replacement value from a program. The writer model writes this program in one extra request per item and definition version (Section D), and the program runs on the typed records of the item’s sources. The cells are $D_m$ with $B=0$, with the source key, and with the citation key, on the 20 test worlds in the three branches. They are built from the writer outputs of the main experiment, and the keys come only from these outputs, so $D$ cannot affect detection. The program request gives the item, the definition in effect with its version, and the typed fields of the item’s sources. It holds no world data, so identical items share one program. A source change runs the program again, and a definition change requests a new program.

Programs run in a separate process with a memory cap and a 2 s limit on CPU time. A restricted interpreter accepts only listed syntax and built-in functions, with no imports, file access, reflection, or wall clock. A program that raises an exception, times out, or returns the wrong type has failed. Definition items are never checked against the gold evaluator. The writer’s value passes if it equals the output of its own program. Otherwise the program’s output replaces it. Other items keep the span check of $D=1$, and if the check fails, the program’s output replaces the value. An item whose program failed when its output was needed is unverifiable. The check-only cell keeps the checker of $D=1$ but derives no replacement value.

Refresh and Upsert Cells

At each checkpoint, a refresh cell re-derives every item whose source key changed, with the request of the $B=1$ cell, and appends the new item. Nothing is invalidated. An upsert cell sends the requests of the append-only $B=0$ cell, and a new item for a slot (its item_id) retires the slot’s older items. Both reuse the writes of the existing cells, and both run at $D=0$, $D=1$, and $D_{\mathrm{m}}$ on the 20 test worlds. The add-only refresh control below is the same refresh at $D=1$ on 10 worlds.

Controls

Six controls vary one confound at a time. The oracle key is built from the gold dependencies. The scope key is built from each definition item’s definition scope, the records its definition ranges over. The add-only refresh adds the new derivation without invalidating the old one. The template control renders the $D=0$ value in the $D=1$ template. The query-time view is no-store BM25 plus the writer’s program, run on the live records at query time. The controls ran on 10 test worlds. The oracle, scope, and source keys are functions of the world, so we audit their detection offline, without a model (Table 9).

The full-access writer. This $D=0$ writer reads every live record of the item’s sources, not only its definition scope, up to 25,000 tokens. When the input is too long, we keep the newest records of the largest source. This cut affects only close friends and VIP senders, in 33.9% [33.3, 34.8] of the re-derivations. For active projects and overdue tasks, whose records the $D=0$ selection already held in full, its request equals the $D=0$ request (Section F.4).

Native Systems

Systems and integration. We ask whether the $(D,B)$ setting of a native memory system, read from its documentation and code, predicts how the system does after a shift. The systems are Mem0 ([27]) and five others: A-Mem (Xu et al., 2025), MemOS (Luo et al., 2025), LightRAG ([58]), HippoRAG 2 ([62]), and Cognee (Markovic et al., 2025). MemOS runs its server-free general_text memory, because tree_text needs a graph server. With its reorganizer off, which is the default, tree_text updates in the same way. Each system uses the main writer, with thinking off, as its LLM. Where a system accepts an embedder, it uses Qwen3-Embedding-0.6B (Zhang et al., 2025). We use only each system’s retrieval context. This context is packed into the 4,096-token evidence region, and the shared reader answers from it, so only storage and retrieval differ. Ingestion failures count as system behavior. Graphiti (Rasmussen et al., 2025), whose LLM invalidates contradicted facts, made 6.1 LLM calls per event with the same model and stopped on invalid JSON after 33 events. Since the tested Mem0 release only adds (see below), no native run has an LLM decide to invalidate a stored value.

Update contracts. The adapter passes each change of the event stream, in time order, through the system’s own API. Document systems (LightRAG, HippoRAG 2, and Cognee) keep one document per source object. A revision replaces the document through a delete and an insert, and a deletion removes it. Mem0, A-Mem, and MemOS receive each revision through their add path and each deletion as tombstone text. Outside the Mem0 invalidation arm (Section F.9), the adapter never calls Mem0’s update or delete API, and the benchmark never invalidates on a system’s behalf. The scores thus describe these integrations.

The tested Mem0 release only adds. Its add path with inference makes one extraction call, which can only add memories. It never runs the step of the Mem0 paper in which the LLM decides to add, update, or delete a memory, and all its history tables hold additions only. Mem0’s $m=1$ thus describes this release, not the design in the Mem0 paper. Mem0 ran without its optional spaCy and fastembed extras, so its search is dense only, without the entity boost.

Worlds, reference cells, and settings. Each of the fifteen three-month test worlds is ingested up to one minute before the fork, and each branch is ingested from that snapshot. Probes are one, two, four, and eight weeks after the fork (one week after a deletion). The reference cells are the four source-key cells on the same worlds plus the citation-key cells (0, 1) and (1, 1), whose granularity is closest to a key per document. Table 10 lists the documented settings together with the audited systems of Section C.6. We infer the setting on one half of the items. A system gets $B=1$ if, after the items’ dependency edges are deleted in the way the adapter delivers a deletion (a delete call or a tombstone), it still gives

Counted since the fork, no key missed a changed item-time (the largest share not invalidated is 0.0%). Re-derivations: invalidations over the four checkpoints, one writer request each, summed over the 20 worlds. Def. share: share of the 480 definition item-times of a shift that were re-derived. —: the scope key is built for definition items only.

(b) Shift	Key	Definition items: $B$ at $D=0$	Definition items: $B$ at $D=1$	Definition items: $B \times D$	All items: $B \times D$
$S_{\mathrm{src}}$	oracle	+6.7 [+1.9, +11.6]	+30.3 [+23.8, +37.2]	+23.6 [+15.0, +33.6]	+6.6 [+2.9, +10.6]
	source	+4.8 [0.0, +10.1]	+30.3 [+23.8, +37.2]	+25.5 [+16.9, +34.6]	+6.2 [+2.5, +10.3]
$S_{\mathrm{def}}$	oracle	−1.5 [−7.4, +4.5]	+37.3 [+31.1, +43.4]	+38.8 [+34.1, +43.7]	+29.9 [+25.0, +34.6]
	source	−1.5 [−7.4, +4.5]	+37.3 [+31.1, +43.4]	+38.8 [+34.1, +43.7]	+27.1 [+22.9, +32.1]
$S_{\mathrm{del}}$	oracle	+2.6 [−1.4, +8.6]	+44.2 [+37.0, +50.9]	+41.6 [+36.4, +46.0]	+10.9 [+6.9, +15.1]
	source	+3.2 [−0.7, +9.0]	+44.2 [+37.0, +50.9]	+40.9 [+36.0, +45.2]	+4.8 [+0.1, +9.9]

the old value with probability at most 0.5. This needs at least 20 valid samples, and otherwise the setting is unidentifiable. $D=1$ would need a structured-repair precision of at least 0.98 and an export with typed values and locatable source spans. On the HoldMem cells, whose settings are known, Cohen’s $\kappa$ between the inferred and the known setting was 1.00 and 1.00 for $B$ (development and three-month worlds) and 0.50 and 0.50 for $D$. Both values for $D$ are below 0.7, so we read $D$ from the export structure instead. Carryover is measured on the other half of the items. Two diagnostic tests (Section E), the utility floor (accuracy before any shift at least that of the oracle context minus 3 pp) and the cost ceiling (a serving cost per query of at most half a full re-read), are computed as in the horizon experiment, with the oracle context of these worlds as reference.

Memory Systems on the (D,B) Grid

Table 10 places fifteen real memory systems and plain RAG on the $(D,B)$ grid, together with the four further systems of the native-system experiment and our own cells. Each placement rests on code, a paper, documentation, or indirect evidence (what the documentation implies without stating it). Documents are as of 2026-09-28, and the settings refer to the releases current on that date. The released configuration pins the exact version of each system we run, and the released audit record gives the reasons and pointers for every placement.

The audit reads $D$ and $B$ by one rubric. Partial $D$ means typed structure or citations without a span-level check. A key that lacks one of its components (source event counts or read set, definition version, or writer) counts as $B=0$, and temporal validity means invalidation only on contradicting information. The native-system experiment instead assigns a system its setting at the document level and infers the setting from measurements (Figure 3), so the two settings can differ, as they do for Cognee. The six systems we run all have $D=0$, because no export keeps a typed value with a source span.

Models and Serving

Table 11 lists every model that the paper uses. Writer and reader are Qwen3.5-9B unless stated otherwise. The writer-switch tiers use MiMo-V2.6-Distill-Qwen-9B and Qwen3.8-27B. Ministral-3-14B is the replacement reader in the failure-mode probes, and bge-m3 replaces Qwen3-Embedding-0.6B in the embedding swap. The other model families serve as new writers, as writer and reader, and as readers. The reasoning of gpt-oss is removed from its reply before parsing. The released run records pin the exact model revisions and software versions.

Model	Size	Context	Roles
Qwen/Qwen3.5-9B ((Qwen Team, 2026a,c))	9B	16,384–65,536	writer, reader, program writer, record rewriter, field extractor, native systems’ LLM
Claude Sonnet 5.5 ([5])	—	1,000,000	reader, definition-item writer, program writer
Claude Opus 5.5 ([5])	—	1,000,000	definition-item writer, program writer
google/gemma-4-12B-it ((Gemma Team, 2026))	12B	16,384	writer, reader, program writer, world rewriter
meta-llama/Llama-3.1-8B-Instruct ((Grattafiori et al., 2024))	8B	16,384	writer, reader, program writer
openai/gpt-oss-20b ((OpenAI, 2025))	20B	16,384	writer, reader, program writer
mistralai/Ministral-3-14B-Instruct-2512-BF16 ((Liu et al., 2026; Mistral AI, 2025))	14B	16,384	writer, reader
Qwen/Qwen3.8-27B ((Qwen Team, 2026b,d))	27B	16,384	writer
XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B ((Xiaomi MiMo Team, 2026a,b))	9B	16,384	writer
Qwen/Qwen3-8B ((Yang et al., 2025))	8B	16,384	writer
Qwen/Qwen3-Embedding-0.6B ((Qwen Team, 2025; Zhang et al., 2025))	0.6B	8,192	embedder
BAAI/bge-m3 ([22])	0.57B	8,192	embedder

Claude Sonnet 5.5 and Claude Opus 5.5 run through Anthropic’s API, and all their responses are released with the code. The API accepts neither a temperature nor a thinking ✓: run natively in the native-system experiment. —: the system stores no derived item. Basis: what the placement rests on, and for a system we run, the property of its code that sets $B$. $^{a}$Measured at document level, as the native-system experiment runs Cognee: (0,1).

budget for these models, so they use its default sampling. As reader, Sonnet returns its answer under a JSON schema. As writers of definition items, both think adaptively at high effort with at most 17,000 output tokens. Their program samples use the default sampling, while the open-weight writers sample at temperature 0.7.

Prompts and Templates

This section gives the writer, program, feedback, and reader requests, and the templates of the stored statements, verbatim from the released code. The other requests (typed input, the rewrites of records and message bodies, and field extraction) are described with their experiments, and every request is released with the code. Angle brackets mark placeholders, which are filled in for each item, time, and world. The examples use the important-contacts item under its first version. Each request below, except the feedback request, consists of a system message and one user message. The open-weight models answer with constrained decoding under a JSON schema.

The writer request. The writer derives one item from its source records. A $B=0$ cell that writes from the delta adds the sentence about the records that arrived since its last write.

System

You maintain a personal knowledge store for one person (called ’the user’). You write one knowledge item at a time from the user’s source records, citing the records you used.

User

Knowledge item: the set of the user’s important contacts (full names)

Open-weight models run under vLLM ([82]) in bfloat16 (gpt-oss-20b in MXFP4) at temperature 0 with JSON-schema decoding. Reasoning is off except in Claude Sonnet 5.5 (effort high for definition items), Claude Opus 5.5 (effort high for definition items, low for programs), and gpt-oss-20b (effort low). The Claude models think adaptively. —: not disclosed.

Value format: a JSON array of full names ([] if
none)
As of: ⟨YYYY-MM-DD HH:MM⟩
Definition in effect (important_contact v1): An
important contact is a person who attended at
least 5 of the user’s one-off calendar meetings
(not recurring series) in the last 90 days.
⟨B=0 delta writes only⟩: The records below are only
those added or changed since ⟨YYYY-MM-DD
HH:MM⟩.

Source records (each starts with its id and
revision):
⟨record id=⟨object id⟩ rev=⟨revision⟩⟩
⟨the record rendered as text⟩
⟨/record⟩
⟨further records, in time order, up to the writer’s token budget⟩
Task: determine the value of the knowledge item
as of the date above, relying on the most
recent applicable evidence. `value` must be
only the value in the format given above (not a
sentence); if the records do not establish a

current value, set value to null. Write statement as one or two self-contained sentences stating the value (name the item and the date). Give up to 6 citations, each with the record id, revision and a short quote (at most 25 words) copied exactly from that record. Reply with JSON only.

The reply follows a schema with three keys: value (a string, a list of strings, or null), statement, and citations (at most six, each with object_id, revision, and quote).

The stored statement. The reader sees only the statement $s_k$ of an item. For a value written by the writer, $s_k$ is rendered from the writer’s statement and citations.

Statement of a written item

[the set of the user’s important contacts (full names) | written ⟨YYYY-MM-DD⟩] ⟨the writer’s statement⟩ Evidence: ⟨object id⟩ r⟨revision⟩: “⟨quote⟩”; ⟨up to six citations⟩

The $D=1$ statement template. When the check fails, exact recomputation (or, in the $D_m$ cell, the writer’s program) supplies the value, and $s_k$ follows a fixed template. For a definition item, the template names the definition version used to compute the value.

Statement of a recomputed item

[the set of the user’s important contacts (full names) | written ⟨YYYY-MM-DD⟩] As of ⟨YYYY-MM-DD⟩, the set of the user’s important contacts (full names) under the definition in effect (important_contact v1: An important contact is a person who attended at least 5 of the user’s one-off calendar meetings (not recurring series) in the last 90 days): ⟨the names, comma-separated, or none⟩. (⟨programmatic extraction, or: computed by the writer’s own program⟩ from ⟨object id⟩ r⟨revision⟩; ⟨up to six source objects⟩)

If there is no current value, the body reads “As of ⟨date⟩, there is no current value for ⟨item⟩ ⟨under the definition in effect⟩.”

The program request. In the $D_m$ cell, the writer also writes a program for each item. The request contains no world data. The experiment with definitions in the user’s own words replaces only the statement after “Definition in effect”.

System

You maintain a personal knowledge store for one person (called ’the user’). You write small, deterministic Python functions that compute one knowledge item from the user’s structured source records.

User

Knowledge item: the set of the user’s important contacts (full names)

Value format: a JSON array of full names ([] if none)

Definition in effect (important_contact v1): An important contact is a person who attended at least 5 of the user’s one-off calendar meetings (not recurring series) in the last 90 days.

Declared sources: calendar

Every record is a dict. Besides the source-specific keys below it has: object_id (str), source (str), revision (int), updated (str "YYYY-MM-DD HH:MM", time of the last change), cluster (str, grouping id) and text (str, the record rendered as plain text).

(the fields of each declared source, here the four kinds of calendar record: settings, series, appointment, and meeting, each with its typed fields)

Task: write a Python function def derive(records, as_of): that computes the value of the knowledge item as of as_of (a string "YYYY-MM-DD HH:MM") from records, the live records of the declared sources at that time (grouped by source in the order listed above, oldest updated first within a source). It must return a Python list of strings in the value format above (an empty list if there are none). (the allowed built-in functions, exception classes, and helpers, and the limits of the interpreter: no imports, no file, or network access, no wall clock, a time limit) Reply with JSON only: {"code": "the Python source of the function and any helpers"}.

The feedback turn. With execution feedback, a program that fails on the records of a write is sent back to the writer. The request repeats the program request and adds the failed reply as the assistant’s turn, followed by this user turn.

User

Your function failed in the restricted interpreter on the live records of the declared sources: (the interpreter’s message, with the returned value if it has the wrong type, at most 600 characters) Write a corrected function. Reply with JSON only, in the same format: {"code": "the Python source of the function and any helpers"}.

The reader request. The shared reader answers every probe of every system from the evidence items that the system returns, within the evidence budget. For Model$_\mathrm{EM}$, these items are the stored statements. An unnamed probe leaves out the last sentence of the question.

System

You answer questions about a person (called ’the user’) using only the evidence provided. Evidence items may disagree or be outdated; prefer the most recent information that applies at the question’s date. If the evidence does not determine the answer, abstain.

User

Evidence:
[1] ⟨evidence item⟩
[2] ⟨evidence item⟩
⟨further items, up to the evidence budget⟩
Question: As of ⟨YYYY-MM-DD HH:MM⟩: Who are the user’s important contacts under the user’s current definition? List full names. Definition in effect: An important contact is a person who attended at least 5 of the user’s one-off calendar meetings (not recurring series) in the last 90 days.
Reply with JSON {"status": "answer" or "abstain", "value": ...}. When answering, give only the value in the requested format: a string, or a JSON array of strings for lists (use [] for an empty list). When abstaining, set value to null.

Setup, Contrasts, and Statistics

This section lists every experiment with the section that reports it (Table 12). All experiments use the same reader template and the same 4,096-token evidence budget. The section then gives the standard report, the four diagnostic tests with their thresholds, the contrasts that separate detection from repair, the statistics, and the compute. Section C.7 gives the models and how they were served.

Experiment	What varies	Cells or systems	Worlds	Section
Main study
main experiment	$S_{\mathrm{src}}$, $S_{\mathrm{def}}$, and $S_{\mathrm{del}}$, probed 1–8 weeks after the fork	six HoldMem cells, check-only cell, JSON arm, no-store BM25, plain RAG, oracle context	20	F.1–F.5
writer switch	$S_{\mathrm{wr}}$ at three tiers	HoldMem cells, calibrated on disjoint worlds	20, and 10 calibration	F.7
horizon	horizons of 1, 3, and 12 months	no-store BM25, plain RAG, HoldMem(1,1), oracle context, with a five-part cost	20	F.6
Where the interaction lies, and the reader
unshifted branch	the same world without the shift	four cells	20	F.2
unnamed probes	definition-item probes that do not name the definition	six cells, $S_{\mathrm{def}}$	20	F.2
held-out definitions	18 definitions from a grammar	HoldMem and $D_{\mathrm{m}}$ cells	10	F.2
Claude reader	Claude Sonnet 5.5 reads the main study’s stores	six cells, JSON arm	20	F.3
What repairs, keys and refresh
writer’s programs	the writer’s program replaces exact recomputation ($D_{\mathrm{m}}$)	$D_{\mathrm{m}}$ with $B=0$, source and citation keys	20	F.4
Claude writers	Claude Sonnet 5.5 and Opus 5.5 write the definition items	four cells and two $D_{\mathrm{m}}$ cells	10	F.4
definitions in the user’s own words	definitions as the user would state them, programs from three writers	two $D_{\mathrm{m}}$ cells	20	F.4
program sets from six writers	three program sets from each of six writers, sampled	two $D_{\mathrm{m}}$ cells	20	F.4
every record as typed fields	the writer re-derives definition items from every live record of their sources, as typed fields, after $S_{\mathrm{src}}$ and $S_{\mathrm{def}}$	$D=0$, $D=1$, and the typed $D=0$ cell with a key, Qwen3.5-9B and Claude Sonnet 5.5 as writers of the definition items	10	F.4
execution feedback	a failing program goes back to the writer with the interpreter’s message, for at most two rounds	$D=0$, $D_{\mathrm{m}}$, and $D_{\mathrm{m}}$ with feedback, each with and without a key	20	F.4
extracted fields	the programs read fields that the writer extracted from prose rewritten from the records, after $S_{\mathrm{src}}$ and $S_{\mathrm{def}}$	$D=0$, $D_{\mathrm{m}}$, and $D_{\mathrm{m}}$ on extracted fields, each with and without a key	10	F.4
controls	six arms that vary one confound at a time	HoldMem and $D_{\mathrm{m}}$ cells	10	F.4, F.5
refresh and upsert	refresh and slot-keyed upsert at $D=0$, $D=1$, and $D_{\mathrm{m}}$		20	F.5
Generality
replication	the main study on other worlds	the cells and baselines of the main study	20	F.8
other writers	five other writers as the new writer (four other families and Qwen3-8B)	writer-switch cells	20	F.7
other families	three other families as writer and reader	main-experiment cells	10	F.8
other readers	three other families as reader	main-experiment cells	5	F.8
rewritten worlds	a model rewrites message bodies around verbatim gold facts	main-experiment cells	10	F.8
Native systems, export, failure modes and sync
native systems	native ingestion, black-box ($D,B$) inference	Mem0 and five further systems, HoldMem cells and baselines on the same worlds	15 three-month	F.9
Mem0 interventions	source-keyed invalidation, query-time view	Mem0	10 three-month	F.9
export	$S_{\mathrm{impl}}$: export to Portable Memory (.mem) with and without the derivation extension	HoldMem(1,1) and a second implementation	20	F.10
failure-mode probes	conflicts, consent, an unstated redefinition, definition items under citation keys, reader swap, embedding swap	HoldMem cells, no-store BM25, plain RAG	5	F.8, F.10
sync	documented incremental-sync contracts instead of ideal delivery	HoldMem ($B \in\{0,1\}$, $D=1$), no-store BM25	5	F.10

The standard report. Every system implements one interface: ingest, define, consent, set writer, query, export and import, and cost. For each shift, the report gives Update and Stability with world-bootstrap intervals. Where a system reports which items are valid, it also gives the missed invalidations, the invalidations of unchanged items (items invalidated although their gold value and status did not change), and the accuracy on re-derived items (the share of re-derived items that the reader answers correctly). Finally, it gives the outcomes of the four diagnostic tests below. A system that offers only add and search runs through an adapter that ingests each event in time order and retrieves context without generating an answer (Section C.5).

Diagnostic tests and their thresholds. The retention test applies to a switch between comparable writers (the three tiers). It requires that the retention, the share of answers correct before the switch that stay correct, is at least 97%. It is a one-sided non-inferiority test with a 90% interval, and we report the net change in accuracy next to it. Switches to other model families use the same threshold. The leaving test applies after an export to a third-party format with a derivation extension that carries the keys, followed by an import into another implementation. It requires that carryover equals that of the original within ±3 pp, tested with two one-sided tests (TOST, Schuirmann (1987)). The utility floor requires an accuracy before any shift of at least that of the oracle context minus 3 pp. The cost ceiling requires a serving cost per query of at most half a full reread. It bounds serving, not maintenance. Three tests pass by construction in some settings: the retention test for cells that never re-derive, the leaving test for two implementations of one data model, and the cost ceiling for a retrieval-based system.

Contrasts and Thresholds

By construction, after a source revision, a redefinition, or a deletion, only $B$ and the granularity of the key can change detection, and only $D$ can change the accuracy on re-derived items. After a writer switch, a $B=1$ memory invalidates every item and a $B=0$ memory none, so only $D$ can change accuracy, and only at $B=1$. After an export, $B$ and $D$ are kept if and only if the derivation extension is exported. Three quantities test this division directly, each against a threshold we set. First, the effect of $B$ on Update after a source revision, averaged over $D$, is compared with +2 pp. This is the gain over flat retrieval that Jiang et al. [74] found for a fine-grained revision ledger on values that were changed and then restored, once the presentation of evidence is matched. Second, $D$’s effect on missed invalidation is compared with a threshold of 2 pp in absolute value in each of six contrasts (two keys and three shifts). Third, writer agreement, the share of items on which two writers are both right or both wrong, should predict survival, that is, whether an item correct before a switch stays correct. We regress $D=0$, $B=1$ survival on the agreement of the two writers. Its calibration slope is compared with 0 (no relation), and its AUROC with 0.60, the weakest discrimination we count as useful, both with 90% intervals.

Statistics and Compute

Statistics. Intervals are 95% world-bootstrap intervals over differences paired by item and checkpoint, with 2,000 draws (Efron and Tibshirani [41]; Field and Welsh [46]). Equivalence tests and writer agreement as a predictor use 90% intervals, and equivalence uses TOST with the margin stated for each test (Lakens [83]; Schuirmann (1987)). We summarize calibration by the logistic calibration slope (Cox [32]; Van Calster et al. (2019)) and discrimination by AUROC. For the held-out definitions and the program sets, the bootstrap draws worlds and definitions (or program sets) independently and takes the product sample. Intervals are per contrast, and we analyze each model family and each native system separately.

Multiple contrasts. The 42 contrasts of $B$, $D$, and $B \times D$ on the three branches that can differ from zero by construction (Section Section E.1) form one family per test set, with one-sided bootstrap p-values. The Holm correction ([67]) keeps 35 of the 36 intervals that exclude zero on the test worlds and 36 of 36 on the replication. Benjamini–Hochberg at $q=0.05 ([11]) keeps 36 and 36. The one interval that Holm drops is the $B \times D$ of the citation key after a redefinition (+7.5 pp [+1.1, +14.0]), which Holm keeps on the replication (+6.8 pp [+2.4, +11.6]). The detection contrasts sit at the lowest p-value the bootstrap can give. Without them, Holm with 10,000 draws over the remaining 30 contrasts keeps 23 of those that exclude zero on the test worlds and 24 on the replication.

Compute. The open-weight models were served on NVIDIA A100 40GB GPUs, which the model servers held for about 140 GPU-hours in total. The native-system runs used 34 of them, all other runs 79, and the servers were idle for the rest. Claude Sonnet 5.5 (claude-sonnet-5-5) and Claude Opus 5.5 (claude-opus-5-5) were called through Anthropic’s API from 2026-10-16 to 2026-10-19, with 278 million input and output tokens in total at a cost of 336 US dollars.

Extended Results

This section gives the full results, ordered by the question they answer. Unless a caption says otherwise, a result comes from the main study: 20 test worlds, Qwen3.5-9B as writer and reader, and the source key for $B=1$. Brackets give 95% world-bootstrap intervals. In a table, an estimate shown without an interval has an interval of zero width at the precision shown, unless the notes of the table call it a point estimate.

Symbols. The shifts are a source revision $S_{\mathrm{src}}$, a redefinition $S_{\mathrm{def}}$, a deletion $S_{\mathrm{del}}$, a writer switch $S_{\mathrm{wr}}$, an export $S_{\mathrm{impl}}$, and, in the failure-mode probes, a consent revocation $S_{\mathrm{cons}}$. $\operatorname{HoldMem}(D,B)$ is a cell of the reference memory. At $B=0$ there is no key and the memory only appends. At $B=1$ a key invalidates an item, and the current writer re-derives it. At $D=0$ the writer re-derives, at $D=1$ exact recomputation repairs, and at $D_{\mathrm{m}}$ the writer’s programs repair. An item-time is one item at one checkpoint, and every count and share below is over item-times. An item-time is changed when its gold value or status differs from that at the fork. On the changed item-times, $m$ is the share not invalidated after the latest change of the gold. An invalidation counts only if it happens after the checkpoint before the interval in which the gold last changed. $r$ is the share of the re-derived item-times that are answered correctly, and $a$ is the share answered correctly among those not invalidated. $r_D$ and $a_D$ are $r$ and $a$ at a given $D$, $r_D$ with a key and $a_D$ without. The accuracy on re-derived items is the share of re-derived item-times, changed or not, that the reader answers correctly (or, where a table says so, whose re-derived value equals gold). Effects are in percentage points (pp). The $B$ effect is $B=1$ minus $B=0$ at a given $D$. The $D$ effect is $D=1$ minus $D=0$ at a given $B$. $B \times D$ is the $B$ effect at $D=1$ (or $D_{\mathrm{m}}$) minus the $B$ effect at $D=0$. A change is caused by the shift when its gold differs from the gold of the same world at the same time without the shift (the unshifted branch). Otherwise, it would also have happened without the shift. If we counted $m$ from the fork, an item-time that was re-derived after an earlier change and is stale after a later one would count as a failed repair. For the source key the two ways of counting differ by at most 0.6% of a class’s changed item-times (1.0% in the replication), and for the citation key on far more (Section F.5). Some analyses count detection from the fork: evidence coverage, typed inputs, the Claude writers’ $r_0$, the controls, the account across writers, the key audit, and the citation key’s misses (Table 9 and 13 to 19 and Figure 2, 4 and 5). There the source key misses no changed item-time, so its $r$ equals its Update.

Which item-times are scored. At each checkpoint after the fork, the reader is asked about every changed item-time. For each world, shift, and checkpoint, it is also asked about a random sample of the unchanged item-times of the same size, drawn with a fixed seed (or all of them if there are fewer). Every $\operatorname{HoldMem}$ cell is asked about the same item-times. Update is computed on the changed item-times and Stability on the sample, while the invalidation shares of the key audit (Table 9) count every item-time. In the main study, each cell is thus scored on 1,168, 558, and 1,767 changed and 1,168, 558, and 1,683 unchanged item-times after a source revision, a redefinition, and a deletion. In the replication, the numbers are 1,170, 575, and 1,739 changed and 1,170, 575, and 1,671 unchanged item-times.

Main Contrasts

The key raises Update, and $D$ does not change detection. Table 18 gives every contrast of the main study next to its estimate in the replication (Section F.8). The effect of $B$ on Update after a source revision, averaged over $D$, was +9.2 pp [+6.7, +11.9], above the threshold of +2 pp. After a redefinition and a deletion it was +13.5 pp [+10.6, +16.5] and +51.7 pp [+47.3, +56.0]. The keys do not depend on $D$, so $D$’s effect on missed invalidation was 0.0 pp for both keys after every shift, and an import without the keys misses every change (Section F.10). By design, the source key’s effects on detection are near 100 pp, apart from the changes that come from the clock alone (Section F.5). With either key, $D$ raised the accuracy on re-derived items after every shift, whether we score the reader’s answer or the re-derived value.

Contrast (pp)	Study	$S_{\mathrm{src}}$	$S_{\mathrm{def}}$	$S_{\mathrm{del}}$
Source key
$B$ effect, detection (either $D$)	main study	+100.0	+100.0	+100.0
	replication	+99.9 [+99.7, +100.0]	+99.8 [+99.5, +100.0]	+100.0
$B$ effect, Update (averaged over $D$)	main study	+9.2 [+6.7, +11.9]	+13.5 [+10.6, +16.5]	+51.7 [+47.3, +56.0]
	replication	+8.2 [+5.3, +11.2]	+12.8 [+9.1, +16.5]	+51.0 [+45.8, +56.2]
$D$ effect, accuracy on re-derived items	main study	+13.6 [+12.4, +14.9]	+33.1 [+30.5, +35.6]	+9.4 [+7.6, +11.3]
	replication	+13.1 [+11.9, +14.6]	+31.6 [+29.3, +34.2]	+9.7 [+7.6, +11.8]
$D$ effect, re-derived value equals gold	main study	+12.2 [+11.6, +12.8]	+14.0 [+13.3, +14.7]	+14.2 [+13.2, +15.2]
	replication	+11.4 [+11.0, +11.9]	+13.0 [+12.4, +13.7]	+13.4 [+12.6, +14.2]
Citation key
$B$ effect, detection (either $D$)	main study	+85.1 [+82.1, +88.3]	+79.2 [+75.1, +83.7]	+92.3 [+90.8, +93.8]
	replication	+86.4 [+84.2, +88.4]	+84.9 [+80.9, +88.7]	+93.2 [+92.4, +93.9]
$B$ effect, Update ($D=1$)	main study	−0.5 [−3.9, +2.9]	+2.5 [−2.7, +7.4]	+39.3 [+33.8, +44.3]
	replication	−0.7 [−3.7, +2.5]	+1.7 [−4.3, +8.0]	+40.2 [+34.5, +46.3]
$B$ effect, Update (averaged over $D$)	main study	0.0 [−3.0, +3.1]	−1.3 [−4.8, +2.0]	+42.5 [+37.8, +47.0]
	replication	−0.9 [−3.9, +2.0]	−1.7 [−5.9, +2.7]	+42.3 [+37.4, +47.4]
$D$ effect, accuracy on re-derived items	main study	+21.8 [+18.7, +25.0]	+54.6 [+49.3, +60.1]	+10.7 [+7.9, +13.6]
	replication	+21.2 [+18.7, +23.9]	+47.5 [+42.7, +53.2]	+9.3 [+7.3, +11.5]
$D$ effect, re-derived value equals gold	main study	+18.1 [+16.5, +20.0]	+25.6 [+22.6, +28.9]	+16.3 [+14.8, +18.0]
	replication	+17.4 [+15.8, +19.1]	+24.1 [+21.6, +26.9]	+16.1 [+14.6, +17.7]
$B \times D$ interaction, Update	main study	−0.9 [−3.6, +1.6]	+7.5 [+1.1, +14.0]	−6.4 [−10.1, −3.0]
	replication	+0.5 [−1.4, +2.7]	+6.8 [+2.4, +11.6]	−4.1 [−7.5, −0.5]

Where the Interaction Lies

The class split. Figure 6 shows the pattern. On definition items, the key helps mainly with exact recomputation (+33.3 pp [+28.2, +38.0] at $D=1$ against +6.0 pp [+2.5, +9.6] at $D=0 after a source revision). On the other items, the interaction is near zero with this reader in both sets of worlds. Table 20 gives every contrast by shift and item class, with exact recomputation in the main study and the replication, and with the writer’s programs.

With exact recomputation, the interaction depends mainly on the item class. Update counts every item-time whose gold differs from its value at the fork. This includes changes after the fork that would have happened without the shift. Table 21 separates the changes that the shift caused from the others. It gives the $B$ effects and $B \times D$ on each part, by item class, with exact recomputation and with the writer’s programs. On definition items, the interaction was $n$: changed item-times. $r_{\mathrm{store}}$: the share of re-derived item-times whose stored state (status and typed value) is the gold. $u$, $l$: accuracy given a right or a wrong store, so $r = r_{\mathrm{store}}u + (1-r_{\mathrm{store}})l$. $u$ is also given by gold status: answer, or abstain where the right state is no value. $a_{\mathrm{store}}$, $a$: the same at $B=0$, where the store is the newest served item. JSON: the $D=1$ store served as JSON records. —: not defined, as every changed definition item-time has a value as its right state and the JSON store has no $B=0$ cell.

Shift	Class	Cause	$n$	$B$ effect $D=0$	$B$ effect $D=1$	$B \times D$ $D=1$	$B \times D$ $D_{\mathrm{m}}$
$S_{\mathrm{src}}$	definition	caused by the shift	111	+11.7 [+2.9, +20.3]	+36.0 [+26.6, +44.0]	+24.3 [+12.5, +35.7]	+23.4 [+11.4, +34.4]
		also without the shift	222	+3.2 [−1.5, +8.1]	+32.0 [+25.5, +37.9]	+28.8 [+20.5, +36.5]	+29.3 [+20.7, +37.2]
	other	caused by the shift	676	+7.2 [+3.9, +10.9]	+6.2 [+2.7, +9.8]	−1.0 [−4.7, +2.0]	+4.9 [+0.6, +9.2]
		also without the shift	159	−2.5 [−8.1, +3.3]	−1.3 [−7.5, +5.0]	+1.3 [−5.4, +8.3]	+2.5 [−2.8, +8.7]
	all	caused by the shift	787	+7.9 [+4.6, +11.5]	+10.4 [+7.2, +13.8]	+2.5 [+0.5, +4.4]	+7.5 [+4.3, +10.4]
		also without the shift	381	+0.8 [−3.0, +4.4]	+18.1 [+12.1, +23.6]	+17.3 [+11.2, +24.0]	+18.1 [+11.8, +24.6]
$S_{\mathrm{def}}$	definition	caused by the shift	222	−6.8 [−12.2, −1.3]	+32.0 [+25.4, +38.5]	+38.7 [+30.4, +46.9]	+32.4 [+23.4, +41.5]
		also without the shift	165	+7.3 [0.0, +14.8]	+46.1 [+35.9, +57.1]	+38.8 [+29.3, +48.4]	+38.2 [+25.3, +49.7]
		new version	235	−7.2 [−12.5, −1.7]	+31.1 [+24.8, +36.9]	+38.3 [+30.5, +46.0]	—
		same version	152	+9.2 [+1.5, +17.1]	+48.7 [+39.1, +59.0]	+39.5 [+30.2, +48.7]	—
	other	also without the shift	171	+2.3 [−2.4, +7.4]	+1.8 [−3.4, +7.1]	−0.6 [−4.0, +2.7]	+3.5 [−3.3, +10.8]
	all	also without the shift	336	+4.8 [+0.9, +8.9]	+23.5 [+17.4, +30.2]	+18.8 [+12.4, +25.1]	+20.5 [+11.8, +29.3]
$S_{\mathrm{del}}$	definition	caused by the shift	48	+18.8 [+8.3, +28.0]	+58.3 [+38.7, +71.1]	+39.6 [+15.0, +60.0]	+39.6 [+15.0, +60.0]
		also without the shift	260	+5.4 [0.0, +10.5]	+40.0 [+34.5, +45.7]	+34.6 [+29.4, +39.3]	+35.0 [+29.0, +40.3]
	other	caused by the shift	1,322	+64.5 [+59.1, +69.5]	+63.0 [+57.0, +68.3]	−1.5 [−5.9, +2.5]	−36.3 [−41.6, −30.7]
		also without the shift	137	−4.4 [−12.6, +3.8]	−5.1 [−14.3, +3.9]	−0.7 [−4.4, +3.0]	+1.5 [−5.6, +10.2]
	all	caused by the shift	1,370	+62.9 [+57.7, +67.8]	+62.8 [+57.1, +67.8]	−0.1 [−4.5, +4.3]	−33.6 [−38.8, −28.5]
		also without the shift	397	+2.0 [−2.0, +5.8]	+24.4 [+18.5, +30.6]	+22.4 [+17.5, +27.2]	+23.4 [+17.6, +29.1]
(b) Shift	Class	$B \times D$, unshifted (pp)	Shifted − unshifted (pp)	Moved (%)
$S_{\mathrm{src}}$	definition	+28.8 [+15.7, +42.7]	−4.5 [−17.5, +9.6]	60.4 [47.7, 72.5]
	other	−1.3 [−3.6, +0.9]	+0.3 [−3.4, +3.5]	1.8 [0.0, 3.9]
$S_{\mathrm{def}}$	definition	+25.2 [+16.2, +33.9]	+13.5 [+1.8, +25.3]	54.5 [45.5, 63.3]
$S_{\mathrm{del}}$	definition	+27.1 [+3.6, +52.3]	+12.5 [0.0, +21.4]	58.3 [40.4, 83.8]
	other	+0.1 [−1.8, +1.9]	−1.6 [−6.7, +2.9]	2.6 [1.3, 4.1]

the same whether the redefinition changed the value or the value would have changed anyway (+38.7 pp and +38.8 pp).

The interaction by cause. Over all items, the interaction after a source revision was +7.4 pp [+5.0, +9.8] on all changes and +2.5 pp [+0.5, +4.4] on the changes the shift caused. After a redefinition the two values were +26.7 pp [+21.2, +31.8] and +38.7 pp [+30.4, +46.9], and after a deletion +5.0 pp [+1.7, +8.1] and −0.1 pp [−4.5, +4.3]. A source revision and a deletion change mostly other items, where the interaction is near zero, so their all-item values on the changes they caused are small. On definition items, the interaction after every shift was about as large on the changes the shift caused as on the others. With the writer’s programs, it was +32.4 pp [+23.4, +41.5] on the changes that a redefinition caused.

Small all-item values on shift-caused changes come from the mix of item classes. The all-item interaction on a set of item-times is a mean, so it equals the sum over classes of each class’s share times its value. On the item-times that a source revision itself changed, definition items make up 14.1% [9.5, 19.0] and contribute +3.4 pp [+1.5, +5.6], and the other items contribute −0.9 pp [−3.8, +1.8]. Together they give the all-item value of +2.5 pp [+0.5, +4.4]. After a deletion, the definition share is 3.5% [1.6, 5.5], and the two contributions are +1.4 pp [+0.3, +2.8] and −1.5 pp [−5.7, +2.4]. If we re-weight to the class mix of all changed item-times, the changes a source revision caused give +6.2 pp [+3.3, +9.2] and the rest give +9.1 pp [+3.8, +15.4] (after a deletion, +5.7 pp and +5.4 pp). The two parts are different item-times, so this comparison is not paired. The unshifted branch gives a paired comparison.

After a source revision, the interaction on definition items also appears without the shift. The unshifted branch has the same events as the shifted branches up to the fork and no shift after it. Four cells answered every item at every checkpoint on it. Table 21 gives the interaction without the shift on the item-times that a shift changed, and the difference from the interaction with the shift. This difference, shifted minus unshifted, was +13.5 pp [+1.8, +25.3] after a redefinition and −4.5 pp [−17.5, +9.6] after a source revision. After a source revision, 60.4% of these item-times also changed without the shift, so the interaction there is not specific to the shift. After a deletion, the difference on definition items was +12.5 pp [0.0, +21.4]. On the other items, no shift changes the interaction measurably.

(a) $n$: the definition item-times after the three shifts. Held: the definition-scope records in the selection. Complete: the item-times with all of them. Cut: the full-access control’s requests cut at 25,000 tokens (10 worlds, $S_{\mathrm{src}}$ and $S_{\mathrm{def}}$). Same request: those equal to the $D=0$ request. Over all six definitions the control re-derived 472 times, 33.9% [31.0, 37.1] with the same request. Typed input: a $D_m$ program’s complete typed input per item-time, as tables without bodies, in Qwen3.5-9B tokens. Fit: the share at or below 31,000 tokens. (b) On the changed definition item-times, $B \times D$ at $D=1$.

The six definitions differ. Table 8 gives the interaction for each definition. It varies widely, so at this level the six definitions are the effective sample. After a source revision it ranges from −20.0 pp (active projects) to +75.4 pp (important contacts). Leaving one definition out moves the class value between +16.5 pp and +33.8 pp after a source revision, and between +34.6 pp and +47.3 pp after a redefinition.

The interaction holds on held-out definitions. Each draw of six held-out definitions (Section B) replaced the six fixed definitions on ten test worlds, with Qwen3.5-9B as writer and reader and every other request unchanged. After the fork, only definition items were asked. The bootstrap draws worlds and definitions independently and takes the product sample. The interaction is positive at both $D=1$ and $D_m$: $B \times D$ was +33.0 pp [+22.1, +44.9] and +27.4 pp [+15.2, +40.7] after a redefinition, +26.7 pp [+16.6, +38.3] and +23.3 pp [+12.9, +36.4] after a source revision, and +33.1 pp [+19.7, +48.6] and +29.0 pp [+16.0, +45.9] after a deletion. It was positive for 17 of the 18 definitions after a redefinition (Table 8).

Naming the definition in the probe changes little. On all 20 worlds after a redefinition, the six cells answered again with unnamed probes (Section B). Items, keys, and writes were unchanged. The effect of $B$ at $D=1$ was +38.0 pp [+32.6, +43.2] without the name and +38.0 pp [+32.6, +43.4] with it (difference 0.0 pp [−1.3, +1.2]). $B \times D$ was +39.5 pp [+33.3, +45.2] without the name, and +36.2 pp [+30.0, +42.0] with the writer’s programs. An append-only memory that holds items under both versions cannot choose between them without the name. However, the key helps mostly where the append-only memory holds no item for the slot derived after the fork (93% of its effect at $D=1$ after a redefinition, Section F.5). There, the name has nothing to choose between.

Stored Value and Reader

With Qwen3.5-9B as reader, the interaction on definition items comes from the stored value. The terms of Equation (1) are based on the reader’s answers, so $r$ mixes two things: whether the re-derivation stored the right value, and whether the reader used it. For each changed item-time, we read the item’s stored state at the probe time (at $B=0$, the newest served item) and judge it with the scorer’s matcher. An item that is not served stores no value. Let $r_{\mathrm{store}}$ be the share of right stores, $u$ the share of right stores that the reader answers correctly, and $l$ the share of wrong stores that it still answers correctly. Then $r=r_{\mathrm{store}}u+(1-r_{\mathrm{store}})l$, and the $B\times D$ interaction splits exactly into a store part and a reader part (Table 13 and Table 22). On definition items, the reader used a right store in 98.1% of cases at $D=0$ and 99.7% at $D=1$ after a source revision. The reader part of the interaction was 0.0 pp [−0.9, +0.9], 0.0 pp, and −0.3 pp [−1.0, 0.0] after the three shifts. A failed re-derivation of these items is thus a wrong stored value, not a reading error.

Shift	Class	Cell	$n$	$r_{\mathrm{store}}$	$u$	$u$, answer	$u$, abstain	$l$	$r$	$a_{\mathrm{store}}$	$a$
$S_{\mathrm{src}}$	definition	$D=0$	333	16.2	98.1	98.1	—	0.4	16.2	10.2	10.2
		$D=1$		99.4	99.7	99.7	—	0.0	99.1	66.1	65.8
		$D=1$, JSON		99.4	70.1	70.1	—	0.0	69.7	—	—
		$D_{\mathrm{m}}$		71.8	99.6	99.6	—	7.4	73.6	38.4	40.2
	other	$D=0$	835	92.0	96.7	99.9	17.2	1.5	89.1	84.2	83.7
		$D=1$		95.4	94.9	99.7	40.9	2.6	90.7	84.9	85.9
		$D=1$, JSON		95.4	93.6	100.0	22.7	0.0	89.3	—	—
		$D_{\mathrm{m}}$		93.5	95.5	100.0	39.7	3.7	89.6	78.7	79.8
$S_{\mathrm{def}}$	definition	$D=0$	387	6.7	100.0	100.0	—	0.0	6.7	8.8	7.5
		$D=1$		99.7	100.0	100.0	—	0.0	99.7	63.0	61.8
		$D=1$, JSON		99.7	54.4	54.4	—	0.0	54.3	—	—
		$D_{\mathrm{m}}$		66.4	100.0	100.0	—	8.5	69.3	33.6	35.1
	other	$D=0$	171	73.7	98.4	100.0	33.3	2.2	73.1	70.2	70.8
		$D=1$		96.5	75.8	100.0	7.0	0.0	73.1	72.5	71.3
		$D=1$, JSON		96.5	74.5	100.0	2.3	0.0	71.9	—	—
		$D_{\mathrm{m}}$		91.8	79.0	100.0	5.7	7.1	73.1	67.3	67.3
$S_{\mathrm{del}}$	definition	$D=0$	308	11.4	100.0	100.0	—	0.4	11.7	4.9	4.2
		$D=1$		99.4	100.0	100.0	—	0.0	99.4	57.1	56.5
		$D=1$, JSON		99.4	67.0	67.0	—	0.0	66.6	—	—
		$D_{\mathrm{m}}$		74.0	100.0	100.0	—	15.0	77.9	31.8	34.7
	other	$D=0$	1,459	73.0	92.4	100.0	91.8	2.3	68.1	6.2	10.0
		$D=1$		81.1	82.9	100.0	81.7	1.4	67.5	7.1	10.9
		$D=1$, JSON		81.1	19.9	100.0	13.9	0.0	16.1	—	—
		$D_{\mathrm{m}}$		52.9	71.8	100.0	68.4	0.7	38.3	6.4	13.0

Shift	Class	Arm	Store part	Qwen3.5-9B Reader part	Claude Sonnet 5.5 Reader part	$B \times D$	Sonnet − Qwen
$S_{\mathrm{src}}$	definition	$D=1$	+27.3 [+21.3, +32.8]	0.0 [−0.9, +0.9]	−8.7 [−13.4, −4.7]	+18.6 [+11.9, +25.1]	−8.7 [−13.4, −4.6]
		$D_{\mathrm{m}}$	+27.3 [+21.3, +32.8]	0.0 [−1.5, +1.4]	−8.7 [−14.2, −4.1]	+18.6 [+11.1, +25.6]	−8.7 [−14.1, −3.8]
	other	$D=1$	+2.8 [+0.6, +4.7]	−3.4 [−6.6, 0.0]	−1.6 [−3.4, +0.5]	+1.2 [−1.1, +3.4]	+1.8 [−1.9, +6.1]
		$D_{\mathrm{m}}$	+7.1 [+4.5, +9.8]	−2.6 [−5.5, +0.3]	−2.0 [−3.7, −0.4]	+5.0 [+2.0, +8.2]	+0.6 [−2.5, +4.2]
	all	$D=1$	+9.8 [+8.0, +11.2]	−2.4 [−4.7, 0.0]	−3.6 [−5.4, −1.9]	+6.2 [+3.8, +8.3]	−1.2 [−3.9, +1.8]
		$D_{\mathrm{m}}$	+12.8 [+10.7, +15.0]	−1.9 [−3.9, 0.0]	−3.9 [−6.0, −2.0]	+8.9 [+6.2, +11.5]	−2.1 [−4.5, +0.9]
$S_{\mathrm{def}}$	definition	$D=1$	+38.8 [+32.0, +44.8]	0.0	−15.2 [−19.1, −11.0]	+23.5 [+18.8, +28.2]	−15.2 [−19.1, −11.0]
		$D_{\mathrm{m}}$	+34.9 [+28.9, +41.0]	0.0 [−3.6, +3.2]	−15.0 [−18.6, −10.9]	+19.9 [+14.9, +24.7]	−15.0 [−20.1, −9.5]
	other	$D=1$	+20.5 [+11.7, +29.1]	−21.1 [−31.7, −10.8]	−10.5 [−16.6, −4.6]	+9.9 [+3.4, +17.3]	+10.5 [+3.1, +18.9]
		$D_{\mathrm{m}}$	+21.1 [+13.8, +29.1]	−17.5 [−28.2, −8.1]	−12.3 [−18.6, −6.2]	+8.8 [−0.6, +18.8]	+5.3 [−4.1, +15.8]
	all	$D=1$	+33.2 [+28.1, +37.9]	−6.5 [−9.8, −3.3]	−13.8 [−17.1, −10.5]	+19.4 [+15.7, +23.1]	−7.3 [−11.4, −3.2]
		$D_{\mathrm{m}}$	+30.6 [+26.0, +35.2]	−5.4 [−10.4, −1.2]	−14.2 [−17.7, −10.6]	+16.5 [+11.7, +21.4]	−8.8 [−14.2, −2.6]
$S_{\mathrm{del}}$	definition	$D=1$	+35.7 [+30.7, +40.4]	−0.3 [−1.0, 0.0]	−10.7 [−15.8, −5.9]	+25.0 [+19.3, +30.4]	−10.4 [−15.5, −5.5]
		$D_{\mathrm{m}}$	+35.7 [+30.7, +40.4]	0.0 [−2.4, +2.3]	−11.4 [−16.3, −6.6]	+24.4 [+18.6, +29.7]	−11.4 [−17.0, −6.0]
	other	$D=1$	+7.2 [+5.0, +9.5]	−8.6 [−11.2, −6.1]	−1.9 [−3.2, −0.6]	+5.3 [+2.3, +8.4]	+6.7 [+3.7, +9.6]
		$D_{\mathrm{m}}$	−20.4 [−24.0, −16.7]	−12.4 [−16.2, −7.9]	−2.3 [−3.7, −0.8]	−22.6 [−26.4, −18.7]	+10.1 [+5.2, +14.3]
	all	$D=1$	+12.2 [+10.0, +14.7]	−7.2 [−9.3, −5.1]	−3.5 [−4.5, −2.5]	+8.7 [+6.2, +11.6]	+3.7 [+1.4, +6.1]
		$D_{\mathrm{m}}$	−10.6 [−14.3, −6.7]	−10.2 [−13.3, −6.6]	−3.8 [−5.0, −2.7]	−14.4 [−18.3, −10.3]	+6.4 [+2.5, +9.6]

On the other items, the reader loses what $D=1$ stores. Here $D=1$ raised the share of right stores, after a redefinition from 73.7% to 96.5%. When the right store was a value, the reader answered correctly in 100.0% of cases. When the right store was “no value”, it answered correctly in only 7.0% [0.0, 20.0] (81.7% [77.2, 85.4] after a deletion). On deleted records, 162 of the 192 reader errors of the $D=1$ cell after a deletion answer “there is no current value” but with the status answer (Table 23). The near-zero interaction on these items is thus a gain in the store that the reader loses, not a re-derivation that already succeeds at $D=0$. When the $D=1$ store is served as JSON records, the stored state is the same by construction. The reader then used a right store on definition items in 70.1%, 54.4%, and 67.0% of cases after the three shifts, so the loss from the JSON format (Table 24) lies entirely in the reader. The writer’s programs store the right definition value less often (71.8% after a source revision), and their interaction there again comes from the store. After a deletion, their interaction on the other items has a negative store part (−20.4 pp [−24.0, −16.7]) and a negative reader part (−12.4 pp [−16.2, −7.9], Section G). Note that $u$ includes finding the statement among all served statements, which the answer logs cannot separate from reading.

Class	$D=0$	$D=1$	$D_{\mathrm{m}}$	Loss (pp)	Share of the loss (%)
correct	66.7	66.2	34.9
value stored	27.3	19.0	48.4	24.7 [20.5, 28.7]	68.9 [61.1, 76.6]
the program’s value				—	67.7 [60.0, 75.3]
reader	5.4	14.1	15.9	10.9 [7.8, 14.1]	30.5 [22.9, 38.2]
failed	0.0	0.0	0.0	0.0	0.0
not re-derived	0.6	0.8	0.8	0.2 [0.0, 0.4]	0.6 [0.0, 1.3]
lost				35.8 [31.2, 39.9]	100.0
gained				4.0 [2.6, 5.5]	—

A stronger reader keeps more of what the store holds. Claude Sonnet 5.5 read every probe of the six cells and the JSON arm in the main study’s stores, with the same reader request and no thinking before the answer. None of its 68,474 answers was unreadable or cut off. With this reader, the interaction on the other items is positive after a redefinition and after a deletion (+9.9 pp [+3.4, +17.3] and +5.3 pp [+2.3, +8.4]). The stores are the same, so the store part of $B \times D$ is unchanged, and the difference between readers lies in the reader part (Table 22 and Figure 7). After a redefinition, the reader part on the other items is −10.5 pp [−16.6, −4.6], compared with −21.1 pp [−31.7, −10.8] for Qwen3.5-9B. Of the 1,210 right “no value” stores on the other items at $D=1$, this reader misread 1, compared with 281 for Qwen3.5-9B. After a source revision, the interaction

on the other items stays near zero (+1.2 pp [−1.1, +3.4]). With the writer’s programs, it is +8.8 pp [−0.6, +18.8] after a redefinition and −22.6 pp [−26.4, −18.7] after a deletion, where the programs’ stored values cause the loss.

On definition items, the gap between readers comes from re-derivation without exact recomputation. On $n$: changed item-times. Missed: not invalidated. Share: missed over the changed item-times of the part (—: no item-time).

(a) Shift	Class	$n$	Missed	Share (%)	Share (%)	Share (%)
$S_{\mathrm{src}}$	definition	333	78	23.4 [18.8, 28.4]	17.1 [7.8, 26.7]	26.6 [19.1, 34.8]
	other	835	96	11.5 [7.5, 15.2]	5.9 [3.3, 8.6]	35.2 [22.2, 49.6]
	all	1,168	174	14.9 [11.7, 17.9]	7.5 [4.7, 10.2]	30.2 [24.1, 36.6]
$S_{\mathrm{def}}$	definition	387	54	14.0 [9.2, 19.1]	0.0	32.7 [21.8, 44.3]
	other	171	62	36.3 [24.3, 50.0]	—	36.3 [24.3, 50.0]
	all	558	116	20.8 [16.3, 24.9]	0.0	34.5 [27.0, 41.9]
$S_{\mathrm{del}}$	definition	308	74	24.0 [19.4, 29.0]	10.4 [0.0, 27.3]	26.5 [21.7, 31.7]
	other	1,459	62	4.2 [2.9, 5.7]	0.0	45.3 [31.5, 59.6]
	all	1,767	136	7.7 [6.2, 9.2]	0.4 [0.0, 1.0]	33.0 [27.8, 37.7]

(b) Shift	Class	Cited	Forced	$B=0$	$D=0$	$D=1$
$S_{\mathrm{src}}$	definition	3.8	98.7	32.1	+2.6 [0.0, +6.2]	+94.9 [+87.3, +100.0]
	other	1.0	62.5	59.4	+63.5 [+48.5, +80.7]	+61.5 [+46.1, +78.7]
$S_{\mathrm{def}}$	definition	5.6	100.0	37.0	+1.9 [0.0, +6.2]	+94.4 [+84.5, +100.0]
	other	0.0	40.3	41.9	+43.5 [+26.2, +65.4]	+40.3 [+25.4, +61.4]
$S_{\mathrm{del}}$	definition	4.1	98.6	33.8	+2.7 [0.0, +6.4]	+94.6 [+86.7, +100.0]
	other	1.6	41.9	43.5	+41.9 [+25.9, +63.4]	+40.3 [+24.4, +61.8]

definition items, the interaction with this reader is lower than with Qwen3.5-9B (−8.7 pp [−13.4, −4.6] after a source revision and −15.2 pp [−19.1, −11.0] after a redefinition). The gap arises at $D=0$ with a key. There, this reader answered 25.5% of the changed item-times correctly after a source revision, although only 16.2% stored a correct value. It recomputed overdue tasks and active projects itself from the records quoted in the stored statement.

What Repairs

Programs over the typed records repair much of what exact recomputation repairs. Four contrasts describe the writer’s programs. On the accuracy on re-derived items, $D_m$ minus $D=0$ (source key) was +7.2 pp [+6.0, +8.4] after a source revision and +21.3 pp [+19.2, +23.4] after a redefinition. $D=1$ minus $D_m$ was +6.4 pp [+5.4, +7.5] and +11.8 pp [+10.5, +13.2], so 47.3% and 35.7% of the

Detection: share of the changed item-times the cell invalidated since the fork. Value equals gold: share of the re-derived item-times whose value equals gold. $D$’s effect on missed invalidation was 0.0 with both keys after every shift in both studies.

Mem0 invalidates nothing ($m=1$ in every row), so its $r$ is undefined and its Update is $a$. $a$, $m$, $r$: point estimates. Effect: Update minus Mem0’s, paired per item-time.

effect of $D$ need exact recomputation. With $D_m$ as the $D=1$ arm, the interaction over all items was +25.3 pp [+18.8, +31.7] after a redefinition and +11.0 pp [+8.0, +14.0] after a source revision. $D_m$’s effect on missed invalidation was 0.0 pp after both shifts. Before any shift, the programs, all written by Qwen3-5.9B to run on typed records, run for 90.6% of 960 items and agree with gold on 60.5%. Two definition programs fail. The program for working hours has a syntax error and never runs (0.0% of items, Section G). The program for VIP senders runs but agrees with gold on 0.0% of items. After a deletion, $D_m$ had a lower accuracy on re-derived items than $D=0$ (−7.8 pp [−10.0, −5.4]), and the interaction over all items becomes negative (−20.8 pp [−25.1, −16.4], Section G).

The writer’s programs keep the interaction on definition items. The class split in Table 20 also uses $D_m$ as the $D=1$ arm. On definition items, the interaction with the writer’s programs was +27.3 pp after a source revision and +34.9 pp after a redefinition. After a redefinition, this is close to the +38.8 pp of exact recomputation, although the level of Update is lower. On changed definition items after a source revision, the programs raise Update with a key over $D=0$ by +57.4 pp [+49.0, +65.7], compared with +82.9 pp [+77.0, +89.1] for exact recomputation. After a deletion at $B=1$, the programs change Update on the other items by −29.7 pp [−33.9, −25.2] compared with $D=0$, and the interaction there is −32.8 pp [−37.5, −27.7]. The definition items rely on at most twelve programs (six definitions with two versions each), shared by both $B$ arms and every world.

Exact: exact recomputation as the $D=1$ arm. Programs: the writer’s programs ($D_m$) as the $D=1$ arm, so in their rows $D=1$ is $D_m$. The $D=0$ cells of the main study serve both of its arms.

Here and in the feedback and extraction cells below, the intervals thus resample worlds, not programs.

Execution feedback lets more programs run but repairs no more items. Two further $D_m$ cells, with and without a key, send a failing program back to the writer (20 test worlds, all three shifts). A program fails when it does not end with a valid typed value. This happens with a reply without code or without a function to call, a syntax error, an exception, a timeout, a sandbox violation, or a value of the wrong type. The writer then receives the program request, its own reply, and the interpreter’s message (at most 600 characters), and it may rewrite the program, for at most two rounds. The message contains no gold value and no expected value. The first program that runs is used and kept for the item’s later writes. If all rounds fail, the outcome is that of $D_m$. Of all 1,860 feedback requests, before and after the fork, 0 contained a string value of the records, so the corrections do not depend on the world.

Table 25 gives the results. Of the program runs for definition items after the fork, 85.8% [84.8, 86.7] ended with a valid value without feedback and 90.4% [89.5, 91.2] with it, with 430 feedback requests over 2,085 runs. Only the working-hours programs failed after the fork (Section G), and feedback made 32.1% of their runs valid. It corrected the program that raised an exception, but not the program with a syntax error, which failed again in both rounds. Repair did not change. On the changed definition item-times, $r$ moved by +0.9 pp [−0.3, +2.5] after a source revision and −0.3 pp [−1.7, +1.0] after a redefinition. Update on definition items after a deletion moved by 0.0 pp [−1.6, +1.6] with a key, and $B \times D$ on definition items by +0.9 pp, 0.0 pp, and 0.0 pp after the three shifts. On the other items, feedback raised the share of valid runs from 92.6% to 96.8%, and $B \times D$ moved by −0.5 pp, −2.3 pp, and −2.1 pp. The errors that cost repair are thus wrong values from programs that run, such as the values returned for deleted records (Section G), which the interpreter does not report.

Programs on fields that the writer extracted from prose keep the interaction. Two further $D_m$ cells, with and without a key, run the same programs on fields that the writer extracted from prose, instead of on the generator’s typed fields (10 test worlds, source revision and redefinition). Qwen3.5-9B at temperature 0 first rewrote every version of every record of the program sources (contacts, calendar, tasks, e-mail, chat messages, and notes) from its typed fields into one to three sentences. Newsletter e-mails, on which no item depends, stay typed. The prompt lists each value as the exact string that the prose must contain, with fixed words for enumerations and booleans (such as “starred” or “done”), and it allows no lists or “field: value” lines. A check requires every value verbatim. A rewrite that misses a value is retried once with the missing values named. If it still misses one, it is kept and counted as missed. The writer then extracts each source’s fields from the prose with a JSON schema for that source, given the field description of the program requests. The programs read the extracted fields together with the record metadata that the store keeps (identifier, source, revision, update time, and cluster). The writer’s own requests, the stores of the other cells, and the reader see the original records, so the contrast changes only what the programs read.

Table 25 gives the results. Of the 10,903 distinct record contents, 82.7% [80.2, 85.0] were extracted with every field equal to the typed field after the scorer’s string normalization (case, punctuation, and whitespace, with attendees as a set). The share was 100.0% for contacts, 91.5% for calendar records, 92.0% for chat messages, 63.1% for notes, 57.3% for e-mails, and 39.8% for tasks, where most errors were in the title. A value was missing from 16.6% of the rewrites. and from 57.8% of the e-mail rewrites. On the definition items, $B \times D$ on Update stayed positive, +18.2 pp [+7.7, +29.7] after a source revision and +25.4 pp [+18.6, +32.4] after a redefinition. The paired differences from typed fields were −6.1 pp and −10.4 pp. Repair correctness on the changed definition item-times fell from 77.0% to 55.8% after a source revision and from 69.2% to 52.7% after a redefinition. Whenever the value of a definition item differed between the two inputs, its write had read a record with an extraction error. On the other items, $B \times D$ did not move (0.0 pp [−1.0, +0.9] and 0.0 pp). The writer’s programs thus keep the interaction when they read fields that the writer itself extracted from rewritten prose.

The interaction holds for definitions in the user’s own words. Nine program sets, written from the statements of Section B, ran in the two $D_{\mathrm{m}}$ cells on all 20 worlds. They are three samples each from Qwen3.5-9B at temperature 0.7 and from Claude Sonnet 5.5 and Opus 5.5 with the settings of Section C.7. After the fork, only definition items were asked. With intervals over worlds and program sets, $B \times D$ on definition items was +32.8 pp [+24.2, +41.3] after a redefinition, +22.4 pp [+14.7, +29.6] after a source revision, and +29.0 pp [+19.8, +37.0] after a deletion. The design rule to compute definition items with a program over typed records (Table 5) thus also applies to a definition that the user states in their own words, given the memory’s data model. We also compared each program set with the same writer’s programs written from the codified statement, using the same samples. The user’s own words changed Update on changed definition items with a key by +1.4 pp [−0.8, +3.7] for Qwen3.5-9B, −7.3 pp [−8.3, −6.2] for Sonnet, and −18.1 pp [−20.5, −15.7] for Opus. They changed agreement with gold before the shift by −4.4 pp, −5.4 pp, and −16.0 pp (Table 25).

Over programs from six writers, the interaction stays positive. Eighteen program sets, written from the codified statements, ran in the two $D_{\mathrm{m}}$ cells on all 20 worlds. There are three samples from each of six writers. Qwen3.5-9B, gemma-4-12B, Llama-3.1-8B, and gpt-oss-20b sampled at temperature 0.7, and Claude Sonnet 5.5 and Opus 5.5 at the API’s default sampling. After the fork, only definition items were asked. With intervals over worlds and program sets, $B \times D$ on definition items was +23.1 pp [+14.6, +31.2] after a redefinition, +15.1 pp [+8.0, +24.4] after a source revision, and +19.4 pp [+10.9, +27.6] after a deletion. After a redefinition, it was +20.4 pp with the programs of Qwen3.5-9B, +38.6 pp with Sonnet, +38.8 pp with Opus, and +30.7 pp with gemma-4-12B, but only +5.4 pp [+0.2, +10.7] with Llama-3.1-8B and +4.7 pp [−0.4, +9.8] with gpt-oss-20b. The writer explains almost all of the variance of this interaction across the 18 program sets ($\eta^2 = 0.99$).

Claude writers re-derive definition items far better than Qwen3.5-9B. On ten test worlds, Claude Sonnet 5.5 wrote the definition items after the shift in all six cells, with the main writer’s request text. It also wrote the programs of the $D_{\mathrm{m}}$ cells. The main study’s Qwen3.5-9B outputs supply every other item and the store before the shift. Claude Opus 5.5 ran the same design with the same settings, except that it wrote programs at low effort, because its thinking cannot be (a) $B \times D$ and $r$ ($n$: changed item-times), and the program runs after the fork that end with a valid typed value ($n$: runs), without and with feedback. (b) $B \times D$ and $r$ with programs on typed and on extracted fields. Values differed between the two inputs on 50 of 198 and 30 of 206 definition item-times after $S_{\mathrm{src}}$ and $S_{\mathrm{def}}$ at $B=0$, and on 58 of 198 and 53 of 206 at $B=1$. Of these, 100.0% had read a record with an extraction error.

(a)	$n$	Without feedback	With feedback	Difference
$B \times D$ on Update (pp), definition items $S_{\mathrm{src}}$	333	$+27.3$ [$+21.8$, $+32.7$]	$+28.2$ [$+22.6$, $+33.9$]	$+0.9$ [$-0.3$, $+2.5$]
$S_{\mathrm{def}}$	387	$+34.9$ [$+27.5$, $+41.9$]	$+34.9$ [$+27.9$, $+41.5$]	$0.0$ [$-2.3$, $+2.0$]
$S_{\mathrm{del}}$	308	$+35.7$ [$+30.1$, $+40.9$]	$+35.7$ [$+29.8$, $+41.3$]	$0.0$ [$-1.6$, $+1.6$]
$r$ (%), definition items $S_{\mathrm{src}}$	333	73.6 [$69.3$, 77.9]	74.5 [$70.3$, 78.8]	$+0.9$ [$-0.3$, $+2.5$]
$S_{\mathrm{def}}$	387	69.3 [$66.6$, 72.0]	69.0 [$66.2$, 71.9]	$-0.3$ [$-1.7$, $+1.0$]
valid runs (%), definition items	2,085	85.8 [$84.8$, 86.7]	90.4 [$89.5$, 91.2]
(b)	$n$	Typed fields	Extracted fields	Difference
$B \times D$ on Update (pp), definition items $S_{\mathrm{src}}$	165	$+24.2$ [$+16.4$, $+32.5$]	$+18.2$ [$+7.7$, $+29.7$]	$-6.1$ [$-9.8$, $-1.4$]
$S_{\mathrm{def}}$	201	$+35.8$ [$+28.0$, $+42.4$]	$+25.4$ [$+18.6$, $+32.4$]	$-10.4$ [$-16.7$, $-2.9$]
$r$ (%), definition items $S_{\mathrm{src}}$	165	77.0 [$71.6$, 82.9]	55.8 [$50.6$, 60.9]	$-21.2$ [$-26.4$, $-16.5$]
$S_{\mathrm{def}}$	201	69.2 [$64.3$, 73.7]	52.7 [$47.3$, 58.3]	$-16.4$ [$-21.8$, $-10.6$]

(c) Writer	$B \times D$ ($S_{\mathrm{src}}$)	$B \times D$ ($S_{\mathrm{def}}$)	$B \times D$ ($S_{\mathrm{del}}$)	Words − codified statement (pp): Update	Words − codified statement (pp): Agreement
Qwen3.5-9B	$+12.7$	$+22.0$	$+16.0$	$+1.4$ [$-0.8$, $+3.7$]	$-4.4$ [$-5.3$, $-3.3$]
Claude Sonnet 5.5	$+27.0$	$+38.2$	$+35.4$	$-7.3$ [$-8.3$, $-6.2$]	$-5.4$ [$-5.6$, $-5.1$]
Claude Opus 5.5	$+27.5$	$+38.2$	$+35.5$	$-18.1$ [$-20.5$, $-15.7$]	$-16.0$ [$-16.4$, $-15.6$]

switched off. None of the 944 distinct writes of either model stopped at the output limit or was unreadable (the longest used 1,953 and 2,118 of 17,000 tokens). $B \times D$ on definition items at $D=1$ was $+16.4$ pp [$+4.8$, $+28.7$], $+16.4$ pp [$+9.7$, $+22.4$], and $+32.5$ pp [$+26.9$, $+38.8$] after the three shifts with Sonnet, and $+23.6$ pp [$+14.2$, $+32.7$], $+16.4$ pp [$+9.8$, $+23.0$], and $+30.5$ pp [$+23.4$, $+38.1$] with Opus. At $D=0$, the key alone raised Update on changed definition items by $+32.7$ pp [$+24.5$, $+39.9$], $+38.8$ pp [$+31.6$, $+46.1$], and $+32.5$ pp [$+25.9$, $+37.8$] with Sonnet, and by $+30.3$ pp [$+21.3$, $+37.6$], $+38.8$ pp [$+30.1$, $+46.6$], and $+32.5$ pp [$+23.8$, $+39.8$] with Opus. With Qwen3.5-9B on 20 worlds, the values were $+6.0$ pp, $-0.8$ pp, and $+7.5$ pp (Table 20). With each writer’s own programs, $B \times D$ after a redefinition was $+12.4$ pp [$+5.7$, $+18.5$] and $+16.4$ pp [$+9.8$, $+23.0$], and Opus’s programs answered as exact recomputation did. After a source revision, $r_0$ on all changed definition item-times was 45.5% [$36.0$, $53.1$] for Sonnet and 43.0% [$32.9$, $51.1$] for Opus, compared with 12.7% [$3.8$, $21.7$] for Qwen3.5-9B on the same item-times, a gain of $+32.7$ pp [$+26.6$, $+38.5$] and +30.3 pp [+22.7, +37.8]. The remaining interaction comes from the item-times whose full definition scope was missing from the writer’s selection, including every close-friends and VIP-sender item-time (Table 14).

(a) Definition	$n$	Held	Complete	Re-derivations	Cut (%)	Same request (%)	Median	Max	Fit (%)
important contacts	240	90.2 [86.3, 93.9]	45.4 [28.3, 62.5]	80	0.0	10.0 [0.0, 25.0]	12,477	15,705	100.0
close friends	240	19.8 [18.6, 20.8]	0.0	80	100.0	0.0	37,314	44,109	0.0
working hours	240	100.0	100.0	80	0.0	0.0	13,370	16,663	100.0
active projects	240	100.0	100.0	76	0.0	100.0	1,870	2,327	100.0
VIP senders	240	23.4 [21.9, 25.0]	0.0	80	100.0	0.0	38,532	47,326	0.0
overdue tasks	240	100.0	100.0	76	0.0	100.0	1,870	2,327	100.0
(b) Shift	Definition scope	$n$	$r_0$ (%)	$r_1$ (%)	$B \times D$ (pp)
Qwen3.5-9B, 20 worlds
$S_{\mathrm{src}}$	complete	144	36.8 [24.8, 47.2]	97.9 [95.5, 100.0]	+22.2 [+8.8, +35.3]
	incomplete	189	0.5 [0.0, 1.6]	100.0	+31.2 [+25.1, +36.6]
$S_{\mathrm{def}}$	complete	197	12.7 [6.0, 19.8]	99.5 [98.3, 100.0]	+50.3 [+37.2, +60.6]
	incomplete	190	0.5 [0.0, 1.6]	100.0	+26.8 [+22.0, +31.7]
$S_{\mathrm{del}}$	complete	121	27.3 [18.6, 36.7]	98.3 [95.8, 100.0]	+39.7 [+24.5, +52.2]
	incomplete	187	1.6 [0.0, 3.4]	100.0	+32.6 [+27.3, +37.2]
Claude Sonnet 5.5
$S_{\mathrm{src}}$	complete	72	94.4 [87.2, 100.0]	94.4 [89.7, 98.5]	−41.7 [−50.9, −30.2]
	incomplete	93	7.5 [4.3, 10.2]	98.9 [96.7, 100.0]	+61.3 [+52.1, +69.2]
$S_{\mathrm{def}}$	complete	104	96.2 [91.5, 100.0]	96.2 [92.3, 100.0]	−21.2 [−31.2, −12.4]
	incomplete	97	8.2 [4.5, 11.4]	99.0 [96.8, 100.0]	+56.7 [+48.0, +65.6]
$S_{\mathrm{del}}$	complete	62	93.5 [82.8, 100.0]	96.8 [92.4, 100.0]	−17.7 [−37.9, 0.0]
	incomplete	92	8.7 [4.7, 12.0]	98.9 [96.6, 100.0]	+66.3 [+57.3, +74.5]
Claude Opus 5.5
$S_{\mathrm{src}}$	complete	72	91.7 [83.7, 97.1]	91.7 [83.6, 97.1]	−23.6 [−31.9, −15.5]
	incomplete	93	5.4 [1.1, 10.0]	95.7 [92.4, 98.9]	+60.2 [+48.9, +69.1]
$S_{\mathrm{def}}$	complete	104	97.1 [92.9, 100.0]	97.1 [94.7, 100.0]	−20.2 [−25.5, −15.5]
	incomplete	97	6.2 [3.1, 9.1]	94.8 [90.4, 98.9]	+55.7 [+45.8, +66.0]
$S_{\mathrm{del}}$	complete	62	95.2 [86.4, 100.0]	91.9 [82.6, 98.4]	−16.1 [−27.3, −7.6]
	incomplete	92	7.6 [3.4, 11.5]	94.6 [90.1, 98.8]	+62.0 [+51.7, +71.6]

The writer’s selection never held the full definition scope of two definitions. An item’s definition scope is the set of records that its definition ranges over. The $D=0$ writer reads a 6,000-token BM25 selection of the records of the item’s sources (Section C.1), while a $D_m$ program reads every typed record of these sources. For each definition, Table 14a gives the share of the definition-scope records that the selection held, and the share of item-times at which it held all of them (complete definition scope), over the definition item-times after the shift. The selection held the full definition scope of working hours, active projects, and overdue tasks at every item-time, and of important contacts at 45.4% [28.3, 62.5] of them. It never held the full definition scope of close friends or VIP senders, and it held 19.8% [18.6, 20.8] and 23.4% [21.9, 25.0] of their definition-scope records.

With the full definition scope, the Claude writers re-derived as well as exact recomputation. Table 14b splits $r_0$, $r_1$, and $B \times D$ on changed definition item-times by complete or incomplete definition scope. Where the scope was complete, Claude Sonnet 5.5 and Opus 5.5 reached an $r_0$ of 96.2% [91.5, 100.0] and 97.1% [92.9, 100.0] after a redefinition, equal to their $r_1$. $B \times D$ was then negative (−21.2 pp [−31.2, −12.4] and −20.2 pp [−25.5, −15.5]). With equal repair, the interaction is $a_0 - a_1$, and the append-only memory serves better values at $D=1$. Where the scope was incomplete, $r_0$ fell to 8.2% [4.5, 11.4] and 6.2% [3.1, 9.1], and $B \times D$ rose to +56.7 pp [+48.0, +65.6] and +55.7 pp [+45.8, +66.0]. Qwen3.5-9B stayed far below $r_1$ even with the complete definition scope (12.7% [6.0, 19.8] after a redefinition and 36.8% [24.8, 47.2] after a source revision). For this writer, the gap at $D=0$ is thus a failure to compute the value. For the Claude writers, the gap lies where the selection was incomplete. Completeness goes with the definition: close friends and VIP senders are never complete, and three other definitions always are. This split thus cannot separate completeness from the definition itself. Giving the writer every record does (Table 15).

The full-access control added uncut records for only two definitions. The full-access writer reads every live record of the item’s sources, up to 25,000 tokens (Section C.4). Table 14a also gives its re-derivations per definition, the share cut at the limit, and the share whose request equaled the $D=0$ request. Every close-friends and VIP-sender request was cut, 33.9% [33.3, 34.8] of all its re-derivations. For active projects and overdue tasks, the request always equaled the $D=0$ request, because the selection already held every record of their sources. The control thus tested more records without a cut only on important contacts and working hours, and only with Qwen3.5-9B as writer.

The complete typed inputs of two definitions exceed the writer’s context. A $D_m$ program receives every typed source record of its item. Table 14 gives the size of this input per definition item-time after the shift, in Qwen3.5-9B tokens, in the most compact of three forms. The first form writes each record as a JSON object. The second leaves out the bodies of e-mails and messages, which no definition reads. The third also leaves out the bodies and writes one table per source, with the column names once and one row per record. The fit column gives the share of item-times at or below 31,000 tokens, which is the writer’s 32,768-token context minus the request and the answer. As tables without bodies, the inputs of close friends and VIP senders reached 44,109 and 47,326 tokens, and none of them fit. On the held-out definitions, 66.7% of the item-times fit in this form.

Given every record as typed fields, Claude Sonnet 5.5 comes close to exact recomputation, and Qwen3.5-9B does not. A further $D=0$ cell with a key re-derives the definition items written after the fork from every live record of the definition’s sources, instead of from the writer’s 6,000-token selection (10 test worlds, source revision and redefinition). The records are the typed fields that the $D_m$ programs read, as one table per source without message bodies (the third form of Table 14). They are uncut and follow the field description of the program request. The rest of the request is the $D=0$ request. Qwen3.5-9B and Claude Sonnet 5.5 each wrote the definition items with the settings of Section C.7, and the main study’s Qwen3.5-9B outputs supply every other item and the store before the shift. The requests had a median of 14,063, a 95th percentile of 40,340, and a maximum of 44,357 Qwen3.5-9B tokens, with medians of 35,446 for close friends and 37,340 for VIP senders. Qwen3.5-9B was thus served with a 65,536-token context here. Of the 472 typed writes of each writer, 1.9% of Qwen3.5-9B’s and 0.0% of Sonnet’s were unreadable, and 1.9% and 0.0% stopped at the output limit.

Table 15 gives $r$ on the changed definition item-times, pooled over the two shifts. Claude Sonnet 5.5 re-derived 91.8% [89.8, 93.7] correctly with every record, compared with 50.0% [45.2, 54.5] from its selection and 97.3% [95.4, 98.9] for exact recomputation (exact minus typed +5.5 pp [+2.7, +8.4]). Where its selection had been complete, every record changed $r$ by −0.6 pp [−3.4, +2.4], so the typed format itself costs no measurable accuracy. Where it had been incomplete, $r$ rose from 7.9% to 88.9%. Close friends and VIP senders, whose records the selection never held in full, reached 97.5% and 98.6%. For this writer, their low $r_0$ thus came from the selection, not from the definitions (Table 14). Important contacts stayed below exact recomputation (65.7% against 88.1%). Qwen3.5-9B re-derived 7.1% [4.2, 10.3] correctly with every record and 8.2% [3.3, 14.0] with its selection, compared with 99.7% [99.2, 100.0] for exact recomputation. Before the fork, its typed derivation equaled gold for 26.7% [21.7, 31.7] of the definition items, and Sonnet’s for 100.0%. What the 9B writer lacks is thus the ability to compute the value, not the evidence. Claude Sonnet 5.5 with every record comes close to exact recomputation, at the cost of reading every record in each re-derivation.

A check alone does not repair. The check-only cell had a lower accuracy on re-derived definition items than $D=0$ (−10.9 pp after a source revision and −4.1 pp after a redefinition). Over all items, adding exact recomputation to the checker raised the accuracy on re-derived items by +15.2 pp and +31.4 pp.

The controls. Each control keeps the main study’s reader context, probe set, and answers before the shift. The scope cell uses the citation key on the other items.

More records did not help Qwen3.5-9B, and the template does not replace exact recomputation. A $D=0$ Qwen3.5-9B writer given every source record gained 0.0 pp and +1.0 pp in accuracy on re-derived definition items after a source revision and a redefinition. Exact recomputation was still ahead by +77.9 pp and +93.6 pp. The control added uncut records only for important contacts and working hours (Table 14). Qwen3.5-9B’s re-derivation also fails where its selection held the full definition scope, so for this writer the advantage of $D$ comes from computation. The control did not run with the Claude writers, but the typed-input cell (Table 15) gives every record uncut to both Qwen3.5-9B and Claude Sonnet 5.5. Rendering the $D=0$ value in the $D=1$ template changed Update on definition items by +0.6 pp [0.0, +1.8], 0.0 pp, and −0.6 pp [−2.0, 0.0] after a source revision, a redefinition, and a deletion. Over all items after a deletion, it changed Update by −35.2 pp [−39.3, −30.3]. The template is applied to every item written after the shift, while $D=1$ applies only to items whose check failed.

On definition items, a program run at query time comes within 10 pp of the stored program. On Update of definition items, the query-time view minus the stored programs was 0.0 pp [−6.4, +5.5] after a source revision and −6.0 pp [−8.3, −4.0] after a redefinition, both within ±10 pp. On the other items, the view fell below no-store BM25 after a source revision (−18.2 pp).

On the other items, the stored programs beat the same programs run at query time. Table 26 compares the view and the stored cell on every item class and on Stability. The differences here are the view minus the stored $D_m$ cell with a key. Pooled over the three shifts, they were −6.6 pp [−9.2, −4.2] on Update and −21.3 pp [−28.0, −14.8] on Stability.

Shift	Class	$n$	View	$D_{\mathrm{m}}$	$B=0$	No-store	View − $D_{\mathrm{m}}$	View − no-store
Update (changed item-times)
$S_{\mathrm{src}}$	definition	165	77	77	48	6	0.0 [−6.4, +5.5]	+70.9 [+65.8, +75.3]
	other	428	70	92	81	88	−22.2 [−28.3, −16.0]	−18.2 [−24.9, −11.0]
	all	593	72	88	72	65	−16.0 [−19.7, −12.6]	+6.6 [+1.6, +12.0]
$S_{\mathrm{def}}$	definition	201	63	69	35	4	−6.0 [−8.3, −4.0]	+59.2 [+54.9, +63.6]
	other	83	69	81	77	81	−12.0 [−22.8, −2.9]	−12.0 [−21.9, −4.9]
	all	284	65	73	47	26	−7.7 [−10.7, −4.8]	+38.4 [+34.1, +43.5]
$S_{\mathrm{del}}$	definition	154	76	79	36	3	−3.2 [−7.6, +1.1]	+73.4 [+67.2, +79.0]
	other	718	43	42	15	88	+1.0 [−5.5, +7.6]	−45.0 [−49.8, −39.7]
	all	872	49	49	18	73	+0.2 [−4.9, +5.4]	−24.1 [−28.4, −19.2]
pooled	definition	520	71	75	39	4	−3.3 [−6.6, −0.5]	+67.1 [+64.0, +70.6]
	other	1,229	54	62	42	88	−8.0 [−12.5, −3.9]	−33.4 [−37.9, −29.0]
	all	1,749	59	66	41	63	−6.6 [−9.2, −4.2]	−3.5 [−7.0, +0.3]
Stability (unchanged item-times)
pooled	definition	100	92	49	72	50	+43.0 [+33.3, +55.6]	+42.0 [+29.5, +53.6]
	other	1,609	62	88	74	70	−25.3 [−32.1, −18.9]	−7.1 [−13.7, +1.0]
	all	1,709	64	85	74	68	−21.3 [−28.0, −14.8]	−4.2 [−11.4, +4.3]

On the other items, they were −22.2 pp and −12.0 pp on Update after a source revision and a redefinition, with a tie after a deletion. On Stability they ranged from −23.6 pp to −28.0 pp after every shift. On definition items, the stored cell also won after a redefinition (−6.0 pp [−8.3, −4.0]), and Update tied after a source revision and a deletion. On the other items, the stored cell keeps the writer’s span-checked value and uses the program’s output only when the check fails, while the view uses the program’s value first. The losses fit a wrong program value that displaces correct live evidence. The view was ahead only on Stability of definition items (+43.0 pp [+33.3, +55.6], 100 item-times), and only because of working hours, whose program never runs (Section G). Without working hours, the difference on Stability of definition items was −3.6 pp [−10.3, 0.0].

The query-time view costs more to serve and less to write. Serving the view cost +246 [+213, +289] reader tokens per query more than serving the stored cell (paired). On the write side, the view requests one program per asked item and definition version, which took 56.7k tokens per world after a source revision. The stored cell writes all 48 items, with 319.7k tokens of ingestion and 1.3M of maintenance. Probes coincide with maintenance, so the two cells are equally fresh, and the comparison is between a stored value and its program.

Across writers, $B \times D$ tracks $r_1-r_0$ on the other items but not on definition items. Figure 4 plots $B \times D$ against $r_1-r_0$ with the source key for the main study, the replication, and the three other model families as writer and reader, and Table 16 gives the terms. Over 24 combinations of four writers, two classes, and three shifts, with the replication held out, the interaction tracks $r_1-r_0$ ($\rho= 0.75$ [0.68, 0.84], least-squares slope 0.30 [0.26, 0.34], and intercept +3.2 pp [+0.6, +5.0]). It also does so within the other items ($\rho= 0.75$ [0.43, 0.97]), but not within definition items ($\rho= -0.21$ [−0.53, 0.31]). There, $r_1-r_0$ is large for every writer, but the interaction after a source revision ranges from +7.3 pp (gemma-4-12B) to +44.8 pp (gpt-oss-20b). The source key missed no changed item-time (with detection counted from the fork), so Equation (2) becomes $B \times D = (r_1-r_0) - (a_1-a_0)$. On definition items, the effect of $D$ at $B=0$, $a_1-a_0$, is large and differs by writer, because writers differ in what the append-only memory derives from the delta. For Qwen3.5-9B it is +55.6 pp, +54.3 pp, and +52.3 pp after the three shifts (Table 20). For Llama-3.1-8B, re-derivation also fails on the other items after a deletion (interaction +27.6 pp). After a redefinition, its interaction on the other items is +19.3 pp [−1.4, +41.0].

Keys and Refresh

The oracle key keeps the effect of $B$ but not Stability. Table 9 audits the source, oracle, and scope keys (Section C.4) View: no-store BM25 plus the writer’s program run on the live records at the probe’s time, with nothing stored. $D_{\mathrm{m}}$: HoldMem with the same programs, stored with the source key. $B=0$: the same without a key. No-store: no-store BM25. $n$: paired item-times. Pooled: the three shifts together. Levels: point estimates, rounded to whole percent.

Update: point estimates. $R$: the share of the effect of $B$ on Update that a $B=0$ variant recovers, $(x-B_0)/(B_1-B_0)$, not evaluated (—) when $B_1-B_0<5$ pp. $B=1$ − refresh: Update of $B=1$ minus that of the refresh at each $D$, paired per item-time.

on the test worlds without a model. The scope key covers definition items only. Offline, the oracle key misses no change since the fork and invalidates only 16.8%, 4.3%, and 7.0% of the unchanged item-times after the three shifts. Over the 20 worlds, it re-derives single-field items 344 times after a source revision, compared with 2,930 for the source key. It re-derives definition items at 77.7% to 86.5% of their item-times, because the time windows of definitions slide, and it invalidates about half of the unchanged definition item-times (49.4% to 61.3%). Run as cells, it keeps, and slightly exceeds, the key’s Update with fewer re-derivations (+2.9 pp [+1.7, +4.1]). This estimate is the mean of per-item paired differences pooled over the six contrasts of $D$ and branch on 10 worlds. Each contrast is thus weighted by its changed item-times, and the two deletion contrasts carry half the weight. On definition items, the oracle key keeps the interaction of the source key (Table 9). Its $B \times D$ was +23.6 pp [+15.0, +33.6], +38.8 pp [+34.1, +43.7], and +41.6 pp [+36.4, +46.0] after the three shifts, compared with +25.5 pp, +38.8 pp, and +40.9 pp for the source key on the same worlds, and its effect at $D=0$ stayed between −1.5 pp and +6.7 pp. Like the source key, it helps mostly through repair. The oracle key loses Stability (−15.2 pp [−27.5, −4.0] on definition items at $D=1$ after a source revision). The source key’s Stability comes from re-deriving unchanged items whose value before the shift the writer got wrong, not from detection.

On definition items, the key works by adding the new derivation. On these items, the scope key matches the source key on Update (0.0 pp after a source revision) and beats the citation key by +29.7 pp [+24.5, +36.8]. On changed definition items at $D=0$ and $D=1$, the key and the refresh give nearly the same Update. The key beats the refresh mainly on the other items after a deletion, where it removes the stale item (Table 27). On the other items after a deletion, the scope cell uses the citation key, so the two cells detect the same changes, and the scope cell minus the citation-key cell on Update is $+3.6$ pp [0.0, $+6.5$].

Shift	$D$	$B=0$	Refresh	Upsert	$B=1$	Refresh	Upsert	definition	other
$S_{\mathrm{src}}$	0	10.2	17.1	10.2	16.2	1.15 [0.86, 1.88]	0.00	−0.9 [−3.6, +0.9]	+1.4 [0.0, +3.2]
	1	65.8	99.1	66.1	99.1	1.00	0.01 [0.00, 0.03]	+0.7 [0.0, +1.6]	0.0
	$D_{\mathrm{m}}$	40.2	71.5	40.5	73.6	0.94 [0.88, 0.98]	0.01 [−0.06, 0.07]	+2.1 [+0.8, +3.8]	+1.2 [0.0, +2.5]
$S_{\mathrm{def}}$	0	7.5	6.7	7.5	6.7	—	—	0.0	+0.6 [0.0, +2.0]
	1	61.8	99.5	61.8	99.7	0.99 [0.98, 1.00]	0.00	+0.3 [0.0, +0.8]	0.0
	$D_{\mathrm{m}}$	35.1	66.4	35.7	69.3	0.92 [0.86, 0.97]	0.02 [−0.07, 0.07]	+2.8 [+1.0, +5.0]	+0.6 [−1.4, +2.5]
$S_{\mathrm{del}}$	0	4.2	11.7	4.5	11.7	1.00 [0.83, 1.13]	0.04 [0.00, 0.13]	0.0 [−0.9, +1.0]	+4.0 [+1.5, +6.7]
	1	56.5	99.4	56.8	99.4	1.00	0.01 [0.00, 0.03]	0.0	+9.9 [+4.9, +14.6]
	$D_{\mathrm{m}}$	34.7	74.4	33.1	77.9	0.92 [0.88, 0.95]	−0.04 [−0.09, 0.01]	+3.6 [+2.0, +5.2]	+0.8 [−3.0, +4.2]

The key compared with a refresh. In these worlds, the source key acts as a periodic full refresh. Some record of an item’s sources arrives between almost any two checkpoints, so the key invalidates nearly every unchanged item-time (Table 9). The add-only refresh (Section C.3}) re-derives the same items with the same request but keeps the old item. Table 27` compares the two on the same item-times. On changed definition items, the key minus the refresh was $-0.9$ pp [$-3.6$, $+0.9$], 0.0 pp, and 0.0 pp [$-0.9$, $+1.0$] at $D=0$ after the three shifts, and 0.0 pp to $+0.3$ pp [0.0, $+0.8$] at $D=1$. With the writer’s programs, the key led by $+2.1$ pp [$+0.8$, $+3.8$] to $+3.6$ pp [$+2.0$, $+5.2$]. With the refresh as the $B$ arm, $B \times D$ on definition items after a redefinition was $+38.5$ pp [$+31.9$, $+44.3$], compared with $+38.8$ pp [$+32.0$, $+44.8$] with a key. Removing the stale item matters most on the other items after a deletion. There, the key led the refresh by $+9.9$ pp [$+4.9$, $+14.6$] at $D=1$ and $+4.0$ pp [$+1.5$, $+6.7$] at $D=0$, because the refresh keeps the item derived from the deleted record next to the new one.

A refresh recovers the effect of $B$, and an upsert does not. The refresh and upsert cells (Section C.3) extend the add-only control to $D=0$, $D_m$, and 20 worlds (Table 27). On definition items, the refresh recovers 1.00 and 0.99 [0.98, 1.00] of the effect of $B$ at $D=1$ after a source revision and a redefinition, and 0.94 [0.88, 0.98] and 0.92 [0.86, 0.97] at $D_m$. At $D=0$, the effect of $B$ after a redefinition is $-0.8$ pp [$-5.7$, $+4.2$], too small for the ratio. After a source revision, the refresh recovers all of it (1.15 [0.86, 1.88]). The upsert recovers 0.01 [$-0.02$, $+0.04$] of the effect of $B$ over the same four combinations, so the effect comes from the key’s re-derivation schedule. After a deletion at $D=1$, the upsert’s Update over all items is 16.0%, compared with 18.8% for the append-only cell, a difference of $-2.8$ pp [$-4.3$, $-1.4$]. Three other model families also served as writer and reader on 10 worlds. Pooled over $D=1$ after a source revision and a redefinition, the upsert recovered 0.07 [$-0.04$, 0.18] of the effect of $B$ on definition items with gemma-4-12B, $-0.16$ [$-0.44$, 0.01] with Llama-3.1-8B, and $-0.03$ [$-0.08$, 0.02] with gpt-oss-20b. In all three families, the effect thus comes from the key’s schedule, as with the main writer.

The key helps where the delta yields no new item, which an upsert cannot reach. The upsert differs from the append-only cell only on slots for which the delta yields a new item. At $D=1$, the delta had yielded one by the checkpoint on 73% and 64% of the changed definition item-times after a source revision and a redefinition, and there the append-only cell already answered 90.1% and 95.6% correctly. On the rest, both $B=0$ cells were right on at most 2.2%, and the $B=1$ cell on 97.8% and 99.3%. These item-times carried 79% and 93% of the effect of $B$ at $D=1$. The key earns this effect through its re-derivation schedule, not by overwriting or removing the stale item. Over the three shifts, the upsert changed 4 answers on definition items and 120 on the other items. After a deletion, the delta yielded no new item on 84% of the changed other item-times, so the upsert could not replace their stale items.

The citation key mostly misses records that could not have been cited. The citation key never invalidated 14.9%, 20.8%, and 7.7% of the changed item-times after the fork, for a source revision, a redefinition, and a deletion. Counted from the latest change, it missed 16.9%, 32.4%, and 8.8%. By item class, counted from the fork, it missed 23.4% of the changed definition item-times and 11.5% of the other changed item-times after a source revision, and 14.0% and 36.3% after a redefinition. Figure 5 classifies every changed item-time that the key missed by the state of the gold dependency set and the writer’s citations at derivation time. Either a record of the set changed that the writer had not cited ($a$), or else a record entered the set ($b$), or else records left a time window or a narrowed scope ($c$). Classes $b$ and $c$, records that entered or left the item’s scope after the item was derived, make up nearly all misses. A miss of class $a$ can also involve a record that entered the set. This was the case for 6.4%, 9.3%, and 10.8% of the missed definition item-times in the main study after the three shifts.

Most missed changes would also have happened without the shift. Table 17 splits the changed item-times that the citation key did not invalidate since the fork by the cause of the change. Because the key holds the definition version, it missed none of the 222 definition item-times that a redefinition changed. After a redefinition, every change it missed would thus also have happened without the shift: it missed 116 of these 336 item-times (34.5% [27.0, 41.9]). Counted from the latest change, it missed 54 of the 222 item-times (24.3% [17.4, 32.0]) whose gold changed again after the redefinition, through records or the clock, and 127 of the 336 others (37.8% [30.5, 45.1]). After a source revision, it missed 7.5% [4.7, 10.2] of the changes caused by the shift and 30.2% [24.1, 36.6] of the others. After a deletion, the shares were 0.4% [0.0, 1.0] and 33.0% [27.8, 37.7]. Nearly all its misses came from records that entered or left an item’s scope after the item was derived, which no citation recorded at write time can contain.

The citation key does not raise Update after a source revision or a redefinition. Its effect on Update, averaged over $D$, was 0.0 pp [−3.0, +3.1] after a source revision and −1.3 pp [−4.8, +2.0] after a redefinition. After a deletion, where the citations name the deleted record, it helped (+42.5 pp [+37.8, +47.0]).

A missed item is served stale. Table 17 gives the accuracies behind the citation key’s effect. On the other items at $D=1$ after a source revision, items that the citation key missed are correct in 1.0% of cases. The append-only cell, which derives a new item from the delta, is correct on 59.4% of the same items. The key’s average effect after a source revision thus hides a split. At $D=1$, its Update over $B=0$ is +6.6 pp [+0.6, +12.2] on definition items and −3.4 pp [−7.4, +0.7] on the other items.

Forcing a re-derivation of the missed items helps only where repair succeeds. On the changed item-times that the citation-key cell did not invalidate, the source-key cell re-derived the same items with the same writer request. Its answers thus show what a forced re-derivation of the missed items would serve (Table 17). At $D=0$, forcing gained little on definition items (+2.6 pp [0.0, +6.2], +1.9 pp [0.0, +6.2], and +2.7 pp [0.0, +6.4] after the three shifts), because the writer’s own re-derivation of these items fails. At $D=1$, it gained +94.4 pp [+84.5, +100.0] to +94.9 pp [+87.3, +100.0]. On the other items, forcing gained at both levels of $D$ (+63.5 pp [+48.5, +80.7] at $D=0$ after a source revision), and the append-only cell served about as many right answers on these item-times (60.4% [48.5, 75.0]).

Changes that come from the clock. The source key counts events per source and does not track time. A change that comes from the clock alone, such as a task that becomes overdue or a meeting that leaves a time window, changes no event count. The key catches it only when an unrelated record of the item’s sources arrives in the same interval between checkpoints. Over the three shifts of the main study, the last gold change came from the clock alone for 99 of the 1,028 changed definition item-times (9.6% [6.1, 13.2]), and the key could not see 8 of them. Of the 6 wrong answers of HoldMem(1,1) on changed definition item-times, 5 are such stale values (7 of 7 in the replication). If missed invalidation were counted from the fork, these errors would count as failed repairs. Counted from the latest change, as in the main text, they are missed detections. The source key then missed 0.6% [0.0, 1.5], 0.3% [0.0, 0.8], and 0.6% [0.0, 1.6] of the changed definition item-times after the three shifts, and $r_1$ stayed at 99.7% [99.1, 100.0] or above. The citation key missed 28.5%, 29.5%, and 28.9% of them, and its $r_1$ was 99.6% or above. With exact recomputation, nearly every wrong answer of this key is thus a missed change. Counted from the fork, its $r_1$ after a redefinition would be 82.3%.

Horizons, Cost, and No-Store BM25

Horizons and cost. Figure 8 gives the accuracy by horizon and the tokens per query. The cost model replays the maintenance schedule at the measured tokens per write. No-store BM25 and plain RAG failed the utility floor. On the three-month worlds, the key at $D=0$ changed Update after a redefinition by +8.7 pp [+3.5, +14.8], compared with +0.2 pp [−2.9, +3.4] on the twelve-month worlds, and $D$ raised repair correctness on changed definition items after a redefinition by +78.5 pp [+73.9, +82.7], compared with +93.7 pp [+89.7, +97.4].

At one and three months, HoldMem(1,1) passes the utility floor because of the definition items. The definition items contribute +8.1 pp, +6.9 pp, and +7.7 pp to the margin of HoldMem(1,1) over the utility floor at one, three, and twelve months. Without them, HoldMem(1,1) passes the floor at 1 of 3 horizons instead of 3 of 3.

Without a shift, stored state pays off over time. At twelve months, the accuracy of no-store BM25 minus that of HoldMem(1,1) was −25.8 pp [−28.5, −23.4]. In a model of its maintenance, the source key cost 3.64 times the tokens of no-store BM25. In the main study, HoldMem(1,1) passed the three diagnostic tests that can fail, with the retention test at the three tiers (Table 28). The fourth, the leaving test, cannot fall between two implementations of one data model (Section E).

Test	Systems or units	Main study: Estimate	Main study: Passes	Replication: Estimate	Replication: Passes
Retention test	HoldMem(1,1), three tiers	−1.4 to −0.4	3/3	−1.9 to −0.1	3/3
	HoldMem(1,1), five other writers	−5.2 to −0.1	2/5	—	—
Leaving test	HoldMem(1,1) with its keys	0.0	1/1	—	—
Utility floor	HoldMem(1,1) · no-store BM25 · plain RAG, three horizons	−22.1 to +12.2	3/3 · 0/3 · 0/3	−23.4 to +13.3	3/3 · 0/3 · 0/3
Cost ceiling	the same systems and horizons	at most 2.1%	9/9	at most 2.0%	9/9
Utility floor	native systems and the two baselines	−21.7 to −1.8	1/8	—	—
Cost ceiling	the same systems	at most 1.1%	8/8	—	—

After a deletion, no-store BM25 ties overall. Table 29 gives Update and Stability of no-store BM25, HoldMem(1,1), and the append-only HoldMem(0,0) by item class after each shift. On Update over all items, no-store BM25 minus HoldMem(1,1) was −32.0 pp [−34.7, −29.5] after a source revision and −66.3 pp [−71.5, −61.3] after a redefinition. After a deletion, no-store BM25 tied HoldMem(1,1) on Update over all items (71.3% against 73.1%, a paired difference of −1.8 pp [−5.6, +2.2]) and won on the other items (85.9% against 67.5%), because no-store BM25 needs no key. HoldMem(1,1) won on Stability and on definition items after each shift of the main experiment. The +51.7 pp that the key gains after a deletion is measured against an append-only store that keeps the stale item: HoldMem(0,0) answered 10.0% of the changed other item-times correctly. The key thus repairs a failure that stored state itself creates. An upsert does not get this benefit, because a deletion yields no new item to replace the stale one.

Shift	Class	No-store	(1,1)	(0,0)	No-store − (1,1) (pp)	No-store	(1,1)	(0,0)
Main study
$S_{\mathrm{src}}$	definition	7	99	10	−91.9 [−96.0, −87.9]	45	100	48
	other	83	91	84	−8.1 [−11.3, −4.9]	71	93	74
	all	61	93	63	−32.0 [−34.7, −29.5]	69	93	73
$S_{\mathrm{def}}$	definition	5	100	7	−94.3 [−97.2, −91.2]	31	92	38
	other	70	73	71	−2.9 [−7.5, +0.9]	68	93	79
	all	25	92	27	−66.3 [−71.5, −61.3]	67	93	78
$S_{\mathrm{del}}$	definition	2	99	4	−97.4 [−99.4, −95.0]	45	100	60
	other	86	68	10	+18.4 [+14.2, +23.3]	66	87	80
	all	71	73	9	−1.8 [−5.6, +2.2]	65	88	78
Replication
$S_{\mathrm{src}}$	definition	7	99	10	−92.0 [−96.3, −87.2]	45	100	59
	other	84	90	86	−6.0 [−8.6, −3.2]	72	92	76
	all	62	92	64	−30.7 [−32.9, −28.5]	70	92	75
$S_{\mathrm{def}}$	definition	7	99	9	−92.0 [−96.0, −87.7]	62	100	77
	other	78	86	82	−7.5 [−14.4, −1.2]	71	92	81
	all	30	95	32	−64.7 [−69.2, −60.5]	71	92	81
$S_{\mathrm{del}}$	definition	3	100	6	−96.5 [−99.0, −93.4]	52	100	63
	other	87	67	10	+20.4 [+14.7, +25.7]	66	88	80
	all	72	73	9	−0.5 [−6.0, +4.8]	65	89	78

The four diagnostic tests. Table 28 gives the results of the retention test, the leaving test, the utility floor, and the cost ceiling for each tier, switch, horizon, and system, in the main study and the replication.

Writer Switches

Design of the eight switches. On the 20 test worlds, the old writer (Qwen3.5-9B, or Qwen3.8-27B for the downgrade) builds the four source-key cells one minute before the fork, with the sources frozen. Each new writer then replaces it in turn and re-derives every $B=1$ item. Retention is $P(\text{correct after} \mid\text{correct before})$. The agreement rate of an item type is the share of items on which both writers are correct or both are wrong, measured on the calibration worlds. For writer agreement as a predictor, we pool the eight switches (the three main-study tiers and the five other writers), regress $D=0$, $B=1$ survival on agreement, and resample by world. In every switch, we also compare the item-paired survival of $D=1$ and $D=0$ items at $B=1$, and the retention test applies to each switch.

Retention stays above 97% at all three tiers. In the worst case, retention changed by −1.4 pp [−1.9, −0.9] (90% interval) relative to full retention, at the same tier. Pooled over the three tiers, writer agreement predicted which of the 2,446 items survive the switches on the test worlds, with a calibration slope of 1.01 [0.90, 1.18] (90% interval) and an AUROC of 0.832. 41 items failed, and 32 of them were definition items.

Survival of a switch exceeds the checker pass rate. At every tier, the $D=1$ survival of a writer switch was higher than the checker pass rate: 98.6% against 79.3% at the same tier, 99.6% against 82.0% for the upgrade, and 99.6% against 77.0% for the downgrade. Exact recomputation recomputes the items that the checker rejects.

The new writer’s own accuracy predicts survival at least as well as agreement. The predictor is the per-type agreement of the old and the new writer on the calibration worlds (Figure 9). Pooled over the three tiers, the new writer’s per-type accuracy reaches an AUROC of 0.887, the old writer’s 0.800, and an indicator of definition items alone

0.869. Outside the definition items, where few items fail, agreement does not predict survival (0.275). Over the eight switches, the new writer’s accuracy reaches 0.849 against 0.827 for agreement (difference +0.022 [−0.001, +0.072], with item types resampled). The old writer’s accuracy (0.607) and the indicator (0.592) fall behind both.

Pooled over the eight switches, agreement predicts survival. In Table 30, the pooled slope was 0.85 [0.79, 0.94] (90% interval) and the AUROC 0.827. For the switch to gemma-4-12B and the switch up to Qwen3-8-27B, almost no item failed, so their slopes are not usable. By point estimate, $D=1$ items survived at least as often as $D=0$ items in 6 of 8 switches, and slightly less often after gemma-412B (−0.1 pp) and gpt-oss-20b (−0.7 pp). Llama-3.1-8B, gpt-oss-20b, and Qwen3-8B did not pass the retention test. The retention test, the utility floor, and the cost ceiling give the same pass counts as in the main study (Table 28).

Writer switch	Slope	AUROC	$D$ surv. (pp)	Retention $\Delta$ (pp)
same tier	1.37 [0.94, 2.19]	0.76	+0.2 [−0.4, +0.7]	−1.4 [−1.9, −0.9]
upgrade (27B)*	−3.52 [−99.29, −0.96]	0.47	0.0 [−0.4, +0.4]	−0.4 [−0.9, 0.0]
downgrade	1.09 [0.95, 1.39]	0.93	+2.7 [+1.7, +3.6]	−0.4 [−0.8, 0.0]
Llama-3.1-8B	0.69 [0.55, 0.85]	0.64	+7.2 [+5.0, +9.2]	−5.2 [−6.4, −4.1]
gemma-4-12B*	—	—	−0.1 [−0.4, 0.0]	−0.1 [−0.4, 0.0]
gpt-oss-20b	0.52 [0.41, 0.66]	0.71	−0.7 [−2.0, +0.5]	−2.4 [−3.2, −1.5]
Ministral-3-14B	0.62 [0.47, 0.85]	0.62	+7.7 [+5.6, +9.9]	−1.7 [−2.5, −1.0]
Qwen3-8B	0.46 [0.26, 0.68]	0.64	+1.0 [−0.2, +2.2]	−3.5 [−4.7, −2.3]
pooled, 8 switches	0.85 [0.79, 0.94]	0.83

After a same-tier switch, retention falls mainly on free text. A same-tier switch makes 91 items of HoldMem(1,1) right and 11 wrong, so net accuracy rises by +8.3 pp. The new writer’s $D=1$ build recomputes items that the old writer had wrong. Retention falls mainly on free text. There, the $D=1$ cell keeps 85.1% compared with 95.5% at $D=0$, because the span check rejects a paraphrase.

Generality

Design of the replication. The replication repeats the main study on 20 fresh test worlds that are disjoint from every other split (Section H). It runs the main experiment with every cell and baseline in the three branches. It also runs the writer-switch experiment with the main study’s calibration agreement rates (not recomputed) and the horizon experiment at one, three, and twelve months. The diagnostic tests apply to each tier, system, and horizon on the fresh worlds as well.

Every contrast with the source key replicates. On the fresh worlds, every contrast with the source key in Table 18 has the main study’s sign and a 95% interval that excludes zero. The exception is $D$’s effect on missed invalidation, which stays at zero as in the main study. With the source key, the effect of $B$ on detection is near 100 pp by design. With the citation key, the effect of $B$ on Update after a source revision and after a redefinition has an interval that includes zero in both studies (averaged over $D$: 0.0 pp [−3.0, +3.1] and −1.3 pp [−4.8, +2.0] in the main study, −0.9 pp [−3.9, +2.0] and −1.7 pp [−5.9, +2.7] on the fresh worlds). Its interaction after a source revision moves from −0.9 pp [−3.6, +1.6] to +0.5 pp [−1.4, +2.7]. On the fresh worlds, the effect of $B$ on Update after a source revision, averaged over $D$, was +8.2 pp [+5.3, +11.2]. $D$’s effect on missed invalidation was 0.0 pp, and writer agreement as a predictor of survival had a slope of 1.05 (90% lower bound 0.93) and an AUROC of 0.79. The retention test, the utility floor, and the cost ceiling give the same pass counts as in the main study (Table 28).

With another family as writer and reader, the main contrasts keep their direction. Other model families (Table 11) replace Qwen3.5-9B in three roles: as the new writer at a switch (Section F.7), as writer and reader of the main experiment, and as reader only. The main writer’s requests are unchanged, so the reader swaps and the writer’s programs reuse the main study’s writer outputs. Each of Llama-3.1-8B, gemma-4-12B, and gpt-oss-20b is writer and reader of the main experiment on 10 test worlds (all branches, cells, and baselines), next to Qwen3.5-9B on the same worlds (Table 4b). In all three families, $D$’s effect on missed invalidation is zero. The following contrasts have 95% intervals that exclude zero in the main study’s direction: the $B$ effect on Update after a source revision, the $D$ effect on the accuracy on re-derived items, $B \times D$ on Update after a redefinition and after a deletion, and the gap of no-store BM25 below HoldMem(1,1). Table 16 splits the terms by item class.

Other readers keep the signs of the main contrasts. Each of the same families also replaces only the reader, on 5 test worlds after a source revision and a redefinition. Every swapped reader keeps the sign of the $B$ effect on Update, the $D$ effect on the accuracy on re-derived items, and $B \times D$ in both branches (Table 31), and $D$’s effect on missed invalidation stays zero.

As reader only (pp)	Llama-3.1-8B	gemma-4-12B	gpt-oss-20b	Qwen3.5-9B
$B$ effect on Update, $S_{\mathrm{src}}$	+26.4 [+18.8, +33.9]	+13.0 [+11.1, +15.2]	+10.1 [+6.7, +14.5]	+9.1 [+5.4, +13.9]
$B$ effect on Update, $S_{\mathrm{def}}$	+17.8 [+11.2, +23.2]	+23.5 [+19.3, +29.3]	+18.1 [+9.4, +24.8]	+17.4 [+10.9, +22.2]
$D$ effect on accuracy on re-derived items, $S_{\mathrm{src}}$	+15.4 [+13.3, +17.7]	+15.2 [+13.4, +16.9]	+13.7 [+11.7, +15.3]	+13.7 [+11.9, +16.2]
$D$ effect on accuracy on re-derived items, $S_{\mathrm{def}}$	+34.1 [+29.7, +38.3]	+35.5 [+31.5, +39.4]	+32.1 [+28.8, +35.4]	+33.4 [+30.1, +37.4]
$B \times D$ on Update, $S_{\mathrm{src}}$	+9.1 [+2.8, +15.9]	+10.4 [+7.0, +14.1]	+10.1 [+3.5, +15.2]	+6.7 [+0.9, +13.4]
$B \times D$ on Update, $S_{\mathrm{def}}$	+30.2 [+23.8, +38.3]	+33.6 [+29.1, +38.1]	+28.2 [+23.6, +31.8]	+30.9 [+24.5, +38.0]
HoldMem(1,1) accuracy, $S_{\mathrm{def}}$ (%)	93.3	96.6	96.3	91.6

Swapping the reader or the embedder changes levels but keeps the signs of the effects of $B$ and $D$. In the experiment with the probes on conflicts and consent (Section F.10), on 5 test worlds, Ministral-3-14B replaces Qwen3.5-9B as reader. Its $B$ effect on Update was +14.3 pp [+10.9, +18.6] and +16.8 pp [+13.4, +20.5] after a source revision and a redefinition (Qwen3.5-9B: +9.1 pp [+5.4, +13.9] and +17.4 pp [+10.9, +22.2]). Its $D$ effect on the accuracy on re-derived items was +16.2 pp [+14.8, +18.3] and +35.1 pp $D$’s effect on missed invalidation was 0.0 in every column. HoldMem(1,1) accuracy: point estimates. Qwen3.5-9B: the main reader, on the same worlds.

[+29.7, +40.0] (Qwen3.5-9B: +13.7 pp [+11.9, +16.2] and +33.4 pp [+30.1, +37.4]). After a redefinition, plain RAG overtook no-store BM25 with this reader (50.7% [45.0, 57.0] against 40.9% [37.1, 45.1], while Qwen3.5-9B gave 43.3% [40.9, 46.1] against 49.3% [45.8, 53.6]). With bge-m3 as the embedding model of plain RAG, its accuracy after the shift moved from 69.7% [66.1, 74.1] to 68.2% [65.7, 70.2] after a source revision and from 43.3% [40.9, 46.1] to 44.6% [40.1, 50.0] after a redefinition.

With rewritten message bodies, the five contrasts keep their direction, and the effect of $D$ on the accuracy on re-derived items shrinks. We repeat the main experiment on worlds in which gemma-4-12B rewrote the bodies of e-mails, chat messages, and notes (Section B). These bodies are 0.9% of the body text, and each keeps its gold surface forms (100.0% found at the worst world and branch). Newsletter digests, which hold no gold fact, stay templated and make up the rest of the body text. On the 10 template worlds of the other model families, the experiment repeats their cells, branches, and probes with the main writer and reader. Both conditions ask identical probes, so each contrast is paired by item (rewritten minus template), with world-bootstrap intervals. On the 10 rewritten worlds, the five contrasts have the same direction as on their template twins (+9.1 pp [+5.3, +13.2], +6.4 pp [+5.2, +7.5], +29.2 pp [+26.7, +32.1], +26.8 pp [+22.4, +31.3], and −40.5 pp [−45.6, −35.8]). The values for the template twins are in the Qwen3.5-9B column of Table 4b. The interaction moves by −0.4 pp [−1.5, +0.7] after a redefinition and by +0.8 pp [−2.7, +3.7] after a source revision. The $D$ effect on the accuracy on re-derived items changes by −7.4 pp [−8.5, −6.4] after a source revision and −4.4 pp [−6.0, −2.9] after a redefinition. Definition items come from structured fields that the rewrite does not touch, so the rewrite can affect only the other items.

Native Systems

Mem0 falls outside the interval of its cell only for Update after a source revision. Mem0 kept 54.9% of the deleted items. By point estimate, this gives its documented setting (0, 0). The interval of this share includes the 50% threshold, so the setting is not identified by the interval (Table 32). Mem0 lies outside the interval of its cell only for Update after a source revision (70.6% [64.2, 76.2] against 57.6% [51.5, 63.4]). Mem0 lies inside the intervals of no-store BM25 and plain RAG (Table 32), as do A-Mem and MemOS, the other systems with its setting. Of its extraction outputs, 398 (3.6%) broke off near Mem0’s default output limit and were dropped, which counts as system behavior. A second ingestion of 3 worlds with identical requests changed 21.5% to 23.6% of the answers after the three shifts and moved Update by +13.2 pp to +20.0 pp. Before the fork, Mem0’s margin over the oracle context was −16.9 pp, so it does not pass the utility floor, like no-store BM25 (−16.8 pp) and plain RAG (−20.7 pp). Serving costs 1.14% of a full re-read. The retention test does not apply to Mem0, and its leaving test was not run.

System, cell	Comparison	Contained by	Native − cell (pp)		Native − no-store (pp)
			Difference	Widened
Mem0 cell (0,0) $P$ 54.9% [49.0, 60.9] inside 5/6	$S_{\mathrm{src}}$	Update	no-store, RAG	+13.1 [−7.4, +18.9]	−0.5 [−5.8, +4.1]
		Stability	(0,0), (1,0), no-store, RAG	+0.8 [−7.8, +8.1]	−5.4 [−11.7, +1.4]
	$S_{\mathrm{def}}$	Update	(0,0)	+3.4 [−4.3, +10.4]	−7.7 [−15.7, −0.4]
		Stability	(0,0), (1,0), no-store, RAG	+0.6 [−6.0, +7.0]	−2.7 [−9.4, +5.6]
	$S_{\mathrm{del}}$	Update°	(0,0)	+13.2 [0.0, +25.0]	−13.2 [−25.6, 0.0]
		Stability°	(0,0), no-store, RAG	−17.0 [−23.9, −8.9]	−6.7 [−11.7, −1.6]
A-Mem cell (0,0) $P$ 66.8% [59.9, 73.3] inside 1/6	$S_{\mathrm{src}}$	Update	no-store, RAG	+11.7 [+6.2, +16.7]	−1.8 [−8.2, +3.7]
		Stability	none	+13.6 [+9.3, +17.9]	+10.5 [+4.5, +16.9]
	$S_{\mathrm{def}}$	Update	(0,0)	+3.4 [−2.0, +8.6]	−7.7 [−14.7, −1.9]
		Stability	none	+12.0 [+5.0, +18.5]	+11.0 [+4.1, +18.3]
	$S_{\mathrm{del}}$	Update°	none	+2.6 [0.0, +7.7]	−23.7 [−33.3, −13.9]
		Stability°	(1,0)	−6.1 [−13.3, +1.1]	+4.3 [−1.9, +10.2]
MemOS cell (0,0) $P$ 54.9% [50.0, 59.9] inside 4/6	$S_{\mathrm{src}}$	Update	no-store, RAG	+12.8 [+7.7, +17.8]	−0.7 [−7.1, +5.3]
		Stability	(0,0), (1,0), no-store, RAG	+2.1 [−4.5, +8.5]	−2.4 [−7.9, +3.0]
	$S_{\mathrm{def}}$	Update	no-store, RAG	+10.2 [+3.0, +16.8]	−0.9 [−6.7, +5.0]
		Stability	(0,0), (1,0), no-store, RAG	−1.5 [−8.6, +5.3]	−2.7 [−9.8, +5.5]
	$S_{\mathrm{del}}$	Update°	(0,0)	+10.5 [0.0, +22.5]	−15.8 [−27.3, −3.0]
		Stability°	(0,0), no-store, RAG	−18.4 [−26.2, −10.6]	−7.7 [−12.5, −2.5]
LightRAG cell (0,1)c $P$ 10.7% [7.3, 14.6] inside 0/6	$S_{\mathrm{src}}$	Update	none	−12.8 [−18.2, −7.6]	−6.2 [−12.5, −0.7]
		Stability	none	−13.5 [−18.4, −8.7]	+16.3 [+8.9, +22.7]
	$S_{\mathrm{def}}$	Update	(0,0)	−14.9 [−20.5, −9.9]	−7.7 [−16.2, −0.8]
		Stability	none	−17.5 [−20.8, −13.9]	+8.3 [+1.8, +15.1]
	$S_{\mathrm{del}}$	Update°	none	−31.6 [−48.6, −17.1]	−26.3 [−39.5, −12.8]
		Stability°	(1,0)	−16.6 [−20.4, −12.5]	+6.2 [+1.4, +11.2]
HippoRAG 2 cell (0,1)c $P$ 9.5% [5.6, 13.7] inside 1/6	$S_{\mathrm{src}}$	Update	(1,0), (0,1)c, no-store, RAG	−3.4 [−9.5, +2.9]	+3.2 [−2.6, +8.5]
		Stability	none	−16.0 [−19.4, −12.6]	+11.8 [+6.6, +17.3]
	$S_{\mathrm{def}}$	Update	no-store, RAG	−7.2 [−12.0, −2.7]	0.0 [−9.5, +8.9]
		Stability	none	−13.8 [−16.8, −10.9]	+13.6 [+5.2, +23.0]
	$S_{\mathrm{del}}$	Update°	(0,0)	−15.8 [−33.3, 0.0]	−10.5 [−22.2, 0.0]
		Stability°	(1,0)	−14.8 [−18.2, −11.7]	+6.7 [+1.5, +12.8]
Cognee cell (0,1)c $P$ 12.3% [8.6, 16.2] inside 1/6	$S_{\mathrm{src}}$	Update	no-store, RAG	−8.9 [−14.1, −4.4]	−2.3 [−8.2, +3.4]
		Stability	none	−3.5 [−6.2, −1.2]	+25.4 [+19.5, +31.0]
	$S_{\mathrm{def}}$	Update	no-store, RAG	−9.4 [−13.7, −5.8]	−2.1 [−8.8, +3.3]
		Stability	oracle	−2.5 [−4.9, −0.4]	+26.1 [+16.3, +37.0]
	$S_{\mathrm{del}}$	Update°	(0,0)	−21.1 [−35.1, −7.9]	−15.8 [−27.0, −3.0]
		Stability°	(0,1)c, (1,1)c	−1.8 [−4.7, +1.1]	+20.2 [+15.6, +24.9]

How the native values are compared. For the five further systems, we compare the setting inferred from behavior with the documented one. We compare each system’s Update and Stability after the three shifts with the 95% interval of the HoldMem cell that has its inferred setting, on the same worlds and branch. For $B=1$, this is the citation-key cell for a document-level system and the source-key cell otherwise. No-store BM25 and plain RAG, which store no derived item, are compared with the same intervals, and we compare mean Stability after a deletion between systems with $B=1$ and $B=0$. A native value says something about its setting only where it also lies outside the intervals of no-store BM25 and plain RAG. Every system ingested 3 of the worlds a second time, and Table 32 widens its intervals by the variance between the two ingestions.

The settings are recovered from behavior, but they do not predict how a system does after a shift. By point estimate, the inferred setting equals the documented one for 5 of 5 systems. 12 of 36 comparisons (33.3%), Mem0 included, fall inside the interval of the cell with the inferred setting (Figure 3), and Table 32 gives the count for each system. Mean Stability after a deletion was 86.4% for $B=1$ systems and 73.2% for $B=0$ systems. $B$ is inferred from the deletion branch, so this last comparison is not independent of the setting. The persistence intervals of Mem0 and MemOS include 0.5, so 4 of 6 settings are identified by interval when Mem0 is counted.

After a source revision, all native systems except LightRAG update like plain RAG, whatever their documented setting. On Update after a source revision, every native system except LightRAG lies inside the intervals of no-store BM25 and plain RAG, and its paired difference from no-store BM25 has a 95% interval that includes zero (Table 32). On the comparisons that were not used to infer the setting, 7 of 24 native values lie inside the interval of the cell with their setting, and 12 of 24 values of plain RAG, which stores

no derived item, lie inside the same intervals. Per event, the five further systems made 1.00 to 2.67 LLM calls and used 0.20 to 0.75 of the prompt and completion tokens of Mem0 (10.7k).

The ranking after a source revision nearly fixes the ranking after a redefinition among the HoldMem cells, but not among the native systems. The rank correlation in Section 7 covers every system measured on these worlds except the JSON arm. It also includes the redefinition and deletion branches of the oracle context, no-store BM25, and plain RAG. Over all 15 systems, the ranking by Update after a source revision correlates with the ranking after a redefinition at Spearman $\rho= 0.96$ [0.85, 0.98] and with the ranking by Stability after a deletion at 0.55 [0.43, 0.66]. The first correlation comes mostly from the HoldMEM cells, which differ widely in Update. Among the 6 cells it is 0.94 [0.75, 1.00], among the 6 native systems 0.52 [−0.09, 0.94], and among the native systems with no-store BM25 and plain RAG 0.76 [0.24, 0.90]. Each interval comes from its own world bootstrap. The native systems lie close together after both shifts, so their ranking after a source revision does not settle their ranking after a redefinition.

Deleting a stale memory does not derive the new value. Mem0 ran as in the native-system experiment on 10 of its three-month worlds, from its stores at the fork, with two interventions. Source-keyed invalidation deletes the memories of a record’s earlier revision before the new revision or the deletion is ingested. The adapter selects these memories by their source record, without gold, and deletes them through Mem0’s own delete call, so Mem0’s LLM decides none of the deletions. A query-time view puts the value of the writer’s program for a definition item, run at query time on the live typed records, first in the evidence for that item. The view thus adds both an evidence path, the typed records that Mem0 does not store, and the computation. Mem0 logged 0 updates or deletions after the fork, so $m=1$ after every shift and in every item class, and Mem0’s Update equals $a$, the accuracy of what it serves without invalidation. The item class, not $m$, tells which intervention helps (Table 19). Mem0 answered 12.2%, 8.9%, and 10.7% of the changed definition item-times correctly after the three shifts. The query-time view raised Update on them by +72.6 pp [+63.1, +81.2], +64.4 pp [+57.4, +71.0], and +71.4 pp [+52.0, +88.6]. Invalidation changed it by +0.6 pp [−3.5, +4.2] to +3.6 pp [0.0, +10.7], with intervals that include zero. Of the definition item-times whose stale memories were deleted by invalidation, Mem0 then answered only 25.0% to 30.8% correctly. On the other items, which the view does not touch, invalidation raised Update by +35.9 pp [+26.1, +44.8] after a deletion, where records lose their source. It changed Update by −8.5 pp [−15.1, −1.1] after a redefinition.

Shift	Class	$n$	Mem0 $a$	Mem0 $m$	Mem0 $r$	Invalidation Effect (pp)	Query-time view Effect (pp)
$S_{\mathrm{src}}$	definition	164	12.2	54.3	28.0	+0.6 [−3.5, +4.2]	+72.6 [+63.1, +81.2]
	other	507	88.0	13.0	87.3	−3.4 [−7.9, +1.4]	0.0
$S_{\mathrm{def}}$	definition	191	8.9	66.5	25.0	+1.6 [−0.6, +4.1]	+64.4 [+57.4, +71.0]
	other	188	90.4	13.3	83.4	−8.5 [−15.1, −1.1]	0.0
$S_{\mathrm{del}}$	definition	28	10.7	53.6	30.8	+3.6 [0.0, +10.7]	+71.4 [+52.0, +88.6]
	other	184	47.3	2.7	83.2	+35.9 [+26.1, +44.8]	0.0

Export, Consent, Conflicts, and Sync

The probes on conflicts and consent ran on 5 test worlds.

Moving to another implementation needs the keys. Exported to Portable Memory with its keys and imported by our second implementation, HoldMEM(1,1) matched the original (0.0 pp) and passed the leaving test. Without the keys, the import detected no later change, and its missed invalidation rose by +100.0 pp. Both outcomes follow from the design (Section E).

A key does not detect conflicts. For 20 contact attributes, we made two live sources disagree, so the gold status is abstain. No HoldMEM cell abstained. The share of these probes answered with abstain was 0.0% for HoldMEM(1,1) and 5.0% [0.0, 15.0] for no-store BM25. A key detects a changed source, not a disagreement, and both values pass the span check.

Consent is applied through the read set. After one agent’s scope was revoked, $B=0$ cells answered 94.8% [90.6, 98.5] of its blocked probes. $B=1$ cells revoke the items whose recorded read set touches the scope. They still answered 63.0% [42.0, 76.3] of the blocked probes and blocked 36.0% [0.0, 79.1] of the answerable ones. No-store BM25 filters the raw sources at query time and leaked 3.2% [0.0, 12.6]. Revoking by read set blocks the right items only when the read set equals the dependency set.

Documented sync contracts cost accuracy with or without a key. With the simulators of Section B, the accuracy on calendar probes changed by −21.2 pp [−27.4, −13.2] with $B=1$, −18.8 pp [−29.4, −12.5] with $B=0$, and −16.2 pp [−20.0, −11.8] for no-store BM25. The sync contracts and the key did not interact measurably on calendar probes (−2.5 pp [−11.2, +8.8], 80 probes on 5 test worlds). Pooled over six sources, the interaction was −3.4 pp [−8.1, +2.4]. The sync worlds sample only events that touch an item, and each event is probed on every item it touches and on equally many items it does not touch.

Error Analysis

This section analyzes two failures: the writer’s programs after a deletion, and the program for working hours.

The programs return a value from another live record where the gold has none. Table 23 sorts every asked item-time after a deletion at $B=1$ by what the cell stored and how the reader answered. On the 1,365 changed item-times whose record was deleted, $D_m$ answered 34.9% [31.2, 38.6] correctly, compared with 66.7% [61.8, 71.5] for $D=0$. It lost 35.8 pp [31.2, 39.9] of these item-times relative to $D=0$. Of this loss, 68.9% [61.1, 76.6] is a value stored where the gold has none, nearly all of it a program value that replaced the writer’s (67.7% [60.0, 75.3]). Most of the rest, 30.5% [22.9, 38.2], comes from the reader. When shown the template “there is no current value”, the reader answers with the status answer about as often at $D=1$ as at $D_m$ ($D_m$ minus $D=1$ on this class: +1.8 pp [−2.3, +5.3]). None of the loss comes from a failed program.

The programs do not react to the deletion. Where the program’s value replaced the writer’s (394 item-times, 28.9% [24.5, 32.7]), exact recomputation stored no value every time. Of these program values, 386 are the same field of another live record. For tasks, it is always the due date of the most recently updated task (189 of 189). For contacts, it is mostly the field of the first contact in the list (122 of 197). On live records that did not change, the programs are wrong about as often as on deleted ones. Task due dates were wrong on 83.3% [71.8, 93.2] of 180 live and 89.8% [81.9, 96.8] of 225 deleted item-times, and contact fields on 26.7% [21.4, 32.6] of 789 and 28.5% [24.2, 32.9] of 936. The recurring-series programs return no value for every deleted record (0.0% of 153 wrong) and are wrong on 11.1% [2.6, 21.7] of 108 live item-times. Before any shift, the task programs agree with gold on 15.0% of items. While the record that the writer cites exists, the writer’s value passes the span check, and the program’s output is only recorded. After a deletion, the writer states no value, the check fails, and the unchecked program value replaces the correct “no value”. We infer which record a program reads from its output.

Failed programs cost Stability instead. Over all re-derived item-times, the difference in accuracy of −7.8 pp [−10.0, −5.4] combines a gain on definition items (+42.5 pp [+39.0, +45.5]) with a loss on the other items (−15.1 pp [−17.3, −12.5]). Failed programs (78 of 960 slots at the last write) account for 68.8% [57.9, 81.2] of what $D_m$ lost on unchanged item-times (−2.2 pp [−4.0, −0.2]).

The program for working hours never runs. The writer’s program for working hours, a single program shared by every world, has a syntax error. It ran in none of the 20 worlds before any shift. The stored item is then unverifiable and not served, so the stored cell failed all 45 unchanged working-hours item-times. The query-time view fell back to no-store BM25, which answered them all. This is the view’s only lead over the stored cell (Section F.4).

Datasheet and License

This section gives the datasheet of the release, following Gebru et al. [48]. Section B specifies the generator.

Motivation. MemShift measures how much of what a memory derived about a person stays correct after the sources, the person’s definitions, the writer model, or the implementation change. Benchmarks that score answers after new history mix detection and repair, and MemShift separates them. It crosses detection with repair in branches that are identical outside the shift, it includes shifts that add no text (a redefinition, a writer switch, and an export), and it scores answers against gold computed before any text exists. The authors created the dataset for this study.

Composition. An instance is a world: twelve months of one synthetic person across six sources (e-mail, calendar, tasks, contacts, notes, and chat). A world has 1,505–1,654 events, about 2.2M tokens, 48 items, and its branches. Probes are questions about one item at a given time, with a gold value and a gold status (answer, abstain, or blocked). The gold labels also record the dependency objects of every item at every time. Splits use disjoint seeds: development worlds 0–9, calibration worlds 900–909, test worlds 1000–1019 (20 worlds), and replication worlds 1020–1039 (20 worlds). Three-month worlds on seeds 1000–1014 serve the native systems. The dataset is a sample from the generator, and further seeds give further worlds. It holds no personal data, no record of a real person, and no human subjects.

Collection process. The data are fully synthetic: a seeded generator writes every world, and no person was recorded, surveyed, or paid. A language model writes world text only in the rewritten worlds, where it rewrites message bodies and keeps every gold fact verbatim (Section B).

Labeling. Deterministic evaluators compute gold from the source objects before any text is rendered. No annotator labels an instance, and no language model scores an answer. The source objects are stored as generated.

Uses. The intended use is to measure the carryover of memories and memory systems under controlled shifts through the seven-method interface. Further uses are to run the four diagnostic tests on a memory and to reproduce the analyses of this paper. The worlds are synthetic evaluation data, so results on them describe synthetic worlds and the models tested, not how a system treats the data of real people. Seeds 1000–1039 are evaluation seeds, and results of a system tuned on them are not comparable.

Distribution and license. Code, generator, worlds, gold labels, and run manifests are released at https://github.com/zichuanxu/memshift as one package with the probes and the analyses. The worlds come with the seeds and the generator version that reproduce them byte for byte. The code is released under the MIT license, and the data (worlds, gold labels, and model outputs) under CC BY 4.0. The open-weight models are not redistributed and keep their own licenses. For Claude Sonnet 5.5 and Opus 5.5, which we accessed through a commercial API, we release their responses.

Maintenance. The authors maintain the package. A change to the generator, a scorer, or a test threshold is a new tagged version. A world is identified by its seed and generator version. Every run manifest records the code version it ran on, so each published result stays tied to the version that produced it. Further seeds extend the dataset without changing existing worlds.