WorksComputer Science
LEDGER: Auditable Agent Memory Under Change
Abstract
Agent memory systems are mostly evaluated on fixed histories, and only their final answers are scored. In real use, facts change, sources differ in reliability, and users ask the agent to forget things. We present LEDGER, a memory layer in which a language model proposes facts and explicit rules decide which value is current, which source to trust, and what to delete. Each fact keeps its source and the period in which it held. LEDGER+EP adds a search over raw conversation turns for questions the ledger does not cover, and scrubs these turns on deletion. We compare both against no memory, full-history replay, retrieval over raw transcripts, and Mem0, with the same answering model and prompt, on a synthetic multi-session coding-agent workload and a 210-question subset of LongMemEval, and we score both the evidence each memory sends to the model and the answer. On the coding workload, LEDGER+EP in fallback mode answers 94.3% of questions correctly on five development seeds and 91.1% on five held-out seeds, against 82.1% and 83.3% for full replay, the strongest baseline. When a deletion request only describes the fact, no deleted value reached our systems' evidence in any evaluated probe. On LongMemEval, LEDGER+EP in augment mode has the same observed accuracy as full replay (74.8%) with 1% of its context. The main limitation is that the coding workload is synthetic and built by us; on LongMemEval, the differences between the strongest systems are within the confidence intervals.
Introduction
Coding agents work on the same repository across many sessions [1], [2]. The repository and its documentation remain available, whereas conversational context from earlier sessions is not automatically carried forward not [3]. Without memory, the agent has to recover these facts again, or the user has to repeat them: which database a service uses, which branch it deploys from, who is on call. A persistent memory carries such facts from one session to the next.
Most of these facts change over time. They appear in user messages, tool output, documents, and code changes. A fact may be updated, restated, or contradicted by a stale document [4], or by the agent’s own earlier erroneous inference [5]. Sometimes the user asks the agent to forget it [6]. A memory that returns an old value next to the current one, trusts a guess over the user, or brings back a deleted value can carry that error into later sessions [6], [7].
Two simple approaches are the standard baselines in recent memory benchmarks [8], [9]. Full-history replay sends the whole past conversation with every question. Nothing is lost, but the context grows with the history, and old, wrong and
deleted values reach the model with the current one. Retrieval over transcripts (transcript RAG) sends only the few past turns most similar to the question. The context stays small, but the search can miss the right turn or return an outdated one. In both cases, the model has to decide on every question which value is current. Dedicated memory systems [3], [10]–[12] reduce what the model reads, but the model still decides when a stored fact is replaced or removed, and their described mechanisms do not generally provide explicit, user-configurable source-reliability rankings.
To the best of our knowledge, no prior benchmark jointly evaluates memory correctness, source trust, deletion compliance, and evidence-level provenance in multi-session coding or conversational agent settings. Existing benchmarks such as LoCoMo and LongMemEval evaluate long-term memory primarily through downstream question answering; LoCoMo additionally reports retrieval recall, but neither benchmark [8], [13] audits whether the evidence ultimately provided to the model reflects the current, trusted, and non-deleted state of the underlying facts.
The objective of this study is to test whether an agent memory can stay correct when facts change, when sources differ in reliability and when users ask to forget, while sending the model far less text than full replay. We ask three questions:
RQ1: How well does the complete system, with LLM extraction and routing, keep stale, contaminated and deleted values out of the evidence it sends to the model?
RQ2: Does such a memory answer better than replay, transcript RAG and an existing memory system under the same answering model and prompt?
RQ3: As the history grows, does it stay correct, and how do its context, storage and model cost grow?
We move some decisions out of the model. In LEDGER, a language model still extracts candidate facts and routes each question to stored facts. Fixed rules then decide which value is current, which source wins a conflict, and what a deletion erases. LEDGER+EP adds a search over raw conversation turns for questions the ledger does not cover. Our contributions are:
Memory design. We propose LEDGER, a fact-centric memory system that maintains current values, validity intervals, provenance, and deletion state under five ingestion rules governing updates, conflicts, and source precedence. LEDGER+EP extends LEDGER by retaining
raw turns, which are scrubbed upon deletion.
Evidence-level evaluation. We introduce an evaluation protocol that audits both the evidence packet supplied to a fixed answering model and the resulting answer, classifying retrieved errors as stale, conflicting, untrusted, or deleted. The protocol reports answer accuracy, retrieval correctness, token use, cost, latency, and storage overhead on a synthetic coding-agent workload with controlled updates, conflicts, deletions, and a simulated 540-day replay.
Controlled comparison. We compare LEDGER and LEDGER+EP with no memory, full replay, transcript RAG, and Mem0 under identical model, prompt, and token-budget conditions. Evaluations cover development and held-out coding-workload seeds and a 210-question LongMemEval subset.
Section. IV-D states which challenge requirements we cover and which, such as a downstream agent task, we do not.
Problem Definition and Scope
Problem setting
An agent works with one or more users over many sessions. Each session produces a stream of events: user and assistant messages, tool calls, tool results, documents and code changes. Every event has an id, a user, a session and a timestamp. Our replay uses these timestamps as a logical clock: sessions are ingested in timestamp order, events with the same timestamp keep their stream order, and an event counts as ingested at its timestamp. Some events state facts: a property of an entity whose value can change, such as the database a service runs on. A memory reads these events. When the agent asks a question, the memory returns a small amount of evidence for an answering model, or abstains.
Facts have two kinds of time. Valid time is the period in which a value was true. Recording time is when the memory learned it; on the replay clock, this is the timestamp of the event that stated it. A question is asked at a time . It can ask about now, or about an earlier time (as of ). The correct answer is the value that was valid at (or at ), based only on what was recorded up to . LEDGER uses a restricted form of this setting, in which the two times are the same (A4).
When a value changes, the old value is not removed: its valid time ends at the change, so it leaves current answers but still answers questions as of an earlier date. Deletion removes a value from both.
Supported operations
Every system, ours and the baselines, is run through the same evaluation adapter. The adapter gives every system the same input, but not every system can act on it.
ingest: add one session’s events, in time order.retrieve: given a question, the user, the time it is asked and optionally an as-of time, return evidence within a token budget, or abstain.delete: forget a fact at the user’s request. The request carries a text description and, when known, the ids of the events that stated the fact. We also test requests that carry only the description.
The runner sends a deletion request right after the user’s “forget” message, which is also ingested as a normal turn, so no system has to detect deletion in the conversation. The transcript baselines and our Mem0 adapter delete only by event id, so a description-only request removes nothing through their delete operation; Mem0 can still act on the “forget” turn through its ordinary update step. This describes the evaluated configuration, not what Mem0 could support with another integration. Only LEDGER routes the request to stored facts.
Desired properties
Table 1 lists seven properties that a correct memory should have. Each one rules out one type of error, and we use these types to label wrong values in the evaluation. The properties describe what we want from any memory. The last column shows under which assumptions LEDGER guarantees each one. These guarantees are per slot: given the right slot, the rules pick the right entry. Reaching the right slot is a separate step, routing (A2).
| Property | Error | LEDGER |
| I1 Deletion: a fact recorded before the user asked to forget it is never returned again, also for questions about the past. The user may state it again later. | resurrected | A1, A2 |
| I2 Supersession: once an accepted update changes a fact, the old value is no longer the current answer. An update is accepted if its source is at least as trusted as the current value. | stale | A1, A3 |
| I3 Temporal: a question as of returns the value valid at , not a later one | future | A1, A4 |
| I4 Deduplication: storing a fact or re-ingesting an event creates no new value entry; a restatement may add provenance | – | A1 |
| I5 Contamination: a less trusted source (a stale README, the agent’s guess) does not override the user | contaminated | A1, A3 |
| I6 Abstention: when nothing relevant is known, the memory says so | no abstention | A2 |
| I7 Isolation: one user’s memory never reaches another user | – | user id |
Table 1. Desired properties of a memory, the error each rules out, and the assumptions under which LEDGER guarantees it.
Assumptions and scope
LEDGER guarantees the properties only under four assumptions:
A1 Correct extraction. For every event that states a fact, the extractor returns a claim with the right value and the same slot key each time the fact is mentioned. It returns no claim for text that does not assert a value, such as a question, a plan or a negation.
A2 Correct routing. Each question and deletion request is mapped to all the slots it is about, and to no other A deletion request whose source events were extracted does not need routing.
A3 Trust order. The source ranking matches how reliable sources are. A document that is newer and correct but conflicts with the user is disputed, not accepted.
A4 Valid time starts when a value is stated. We do not read dates written in the text. Each entry stores only a start time, the timestamp of the event that stated the value, and an end time, the timestamp of the event that replaced it; on the replay clock, the recording time equals the start time. A change reported late, such as “we migrated last Tuesday”, is therefore dated to the message that reports it.
A1 and A2 are idealised conditions. Our extractor and router are LLM calls and do not always meet them; when they fail, the properties that depend on them can fail too, and § 5-F counts how often. I7 needs only that the caller passes the right user id and that every read and deletion goes through the interface, because all state is kept per user.
LEDGER+EP keeps I7, and I1 with one open case; the other properties hold for its ledger facts but not for the raw turns it adds. We study facts with one current value each, in English text, from coding-agent sessions and from LongMemEval chats.
LEDGER Architecture and Mechanisms
Architecture overview
Figure 1 shows the components. An extractor reads each ingested session and proposes claims: this event states that this property has this value. The ledger core applies fixed write rules to each claim, and it alone decides what is current, trusted or deleted. A router maps each question or deletion request to the slots the ledger knows. In our main setup the extractor and the router are calls to a language model. A rule-based extractor serves as a model-free floor.
Figure 1. Components of LEDGER and LEDGER+EP. LEDGER uses the extractor, the router and the ledger core; LEDGER+EP adds the episode store. The model proposes claims and routes questions; the ledger core alone decides what is current, trusted or deleted, and the episode store is scrubbed on every deletion.
We evaluate three variants:
LEDGER: the ledger alone. It returns one fact line per matched slot, or abstains.
LEDGER+EP fallback: adds an episode store of raw turns. It returns the ledger’s facts whenever the ledger has any, and BM25-ranked turns [14] only when it has none.
LEDGER+EP augment: always returns the ledger’s facts first, then fills the rest of the token budget with turns.
The episode store exists because the ledger models only facts with one current value. Some questions are not about such facts: what the assistant recommended, a one-off detail in a long chat, a count across sessions, or a fact whose first statement the extractor missed. Raw turns are not filtered by the rules, so they can carry an old or less trusted value; Sec. III-D states when each mode adds them.
Data model and time
Memory is partitioned by user. A slot is a property of an entity, keyed entity/attribute (e.g. payments-service/database). Each value a slot has held is an entry with a validity interval [15]. The interval starts at the timestamp of the event that states the value (A4) and ends when a newer value supersedes it. A question asked at as of returns the entry that was valid at and recorded at . An entry also keeps the source kind and trust of the event that set it, and the ids of every event that stated it. When a more trusted source restates the value, the entry takes the higher trust, so a later conflicting document is disputed.
Besides entries, the ledger keeps three kinds of records that are never retrieved: disputed values from less trusted sources, kept for audit; unconfirmed values, held until a trusted source states a value; and tombstones, which record that a slot was deleted but not its value.
Ingestion and conflict resolution
Extraction. For each ingested session, the extractor makes one structured-output call and returns claims . It sees the user’s known slots and their current values, and is asked to reuse a known key for the same property. Keys are normalised only in form, so synonyms such as “inventory” and “inventory-service” become two slots. Claims are applied one at a time in the order of their events. Events already ingested are skipped by id (I4). The ledger core never reads free text. Two safeguards check the extractor’s output before any rule applies:
Enumerated event ids. The output schema lists the ids of the batch’s events, so a claim can only cite an event that exists.
Grounding check. A claim is kept only if its value occurs as text in its event, ignoring case and punctuation. This drops invented or copied values, but not a value in a negation or a sentence about another service.
A third safeguard concerns deletion.
Write rules. The core applies one of five rules to each claim, in stream order:
rest ate same value as the current entry, from any source: add the event to its provenance and keep the higher of the two trust levels; no new value entry (I4);
supersede different value from a source at least as trusted as the current entry’s: close the current entry at the claim’s time and open a new one (I2); the closed entry still answers historical questions (I3);
dispute different value from a less trusted source: kept for audit, never retrieved (I5);
introduce no current value, source trust : open an entry;
hold no current value, source trust : keep the value as unconfirmed. If a trusted source later states the same value, it becomes an entry valid from the trusted statement; if it states a different value, the held value becomes a dispute.
Table II traces the rules on one fact.
Source trust. Table III collects the trust policy. Each event’s kind sets its trust. Documents can introduce a value, because they are often a fact’s only first source, but cannot override the user, a tool result or a code change. The assistant’s messages and tool calls can never set or change a value; LEDGER+EP therefore does not send them to the extractor, but keeps them in the episode store.
| Day | Source (trust) | Claim | Rule and effect |
| 1 | user (3) | PostgreSQL 14 | introduce: entry |
| 3 | README (2) | Postgres 12 | dispute: kept for audit only |
| 5 | user (3) | PostgreSQL 14 | restate: event added to provenance |
| 9 | migrate.log (3) | PostgreSQL 16 | supersede: closed; opened |
| 12 | user (3) | “forget the database” | delete: both entries, the dispute and the events erased; tombstone retained |
Table 2. THE WRITE RULES ON ONE SLOT, PAYMENTS-SERVICE/DATABASE.
Asked on day 7: PostgreSQL 14; on day 10 as of day 6: PostgreSQL 14; on day 10: PostgreSQL 16; on day 13: abstain, “deleted at the user’s request”.
| Source | Trust | Introduce | Update | Extracted (LEDGER / LEDGER+EP) |
| User message, tool result, code change | 3 | yes | any value | yes / yes |
| Document | 2 | yes | values of trust | yes / yes |
| Assistant message, tool call | 1 | no (held) | never (disputed) | yes / no |
Table 3. SOURCE TRUST POLICY: WHAT EACH SOURCE MAY DO IN THE LEDGER, AND WHETHER IT IS SENT TO THE EXTRACTOR IN EACH VARIANT.
The ranking rests on assumption A3. We assume the user knows their own setup; tool output and code changes observe the system directly (a migration log, a diff); documents are often right but can be stale; and the assistant’s messages and tool calls are guesses or intentions, not observations. When this assumption is wrong, the policy fails in a known way: a newer and correct document cannot override an older user statement, and a wrong tool result overrides the user.
Retrieval and abstention
The router maps the question to slot keys chosen from a list of the user’s known slots, so it cannot invent a slot. For each matched slot, the ledger returns the one entry valid at the question’s as-of time and recorded by the time it is asked, as a single line with its validity and source. Matched slots without such an entry are left out. When none of them has one, the ledger abstains with a reason: “no known slot matches”, “deleted at the user’s request” or “only an unconfirmed claim” [16]. Under A1 and A2, the evidence therefore never contains a disputed, unconfirmed or deleted value.
In LEDGER+EP, the episode store ranks the user’s stored turns with BM25 and adds the top five, each with its date and source kind, while they fit in the token budget left after the facts. Turns are not filtered by the as-of time. Fallback mode searches only when the ledger returns no fact; augment mode always adds turns after the facts. A question that matches only deleted or empty slots is refused in both modes. If it matches a deleted slot and a live one, augment mode still adds turns, so here only episode scrubbing keeps the deleted value out.
Deletion
Affected slots. If the request carries source event ids, the affected slots are those with any entry, dispute or unconfirmed value from these events. An id cannot single out one fact of an event that states several, so all slots stated by these events are affected; in our coding workload each event states at most one fact. If there are no ids, or the extractor never turned these events into claims, the router maps the description to known slots. As a third safeguard, we also add every slot the description names outright. One routing call can return no slot, and a missed slot is a privacy failure.
Ledger deletion. For each affected slot, the ledger erases every entry, dispute and unconfirmed value recorded up to the request, past values included (I1). Each affected slot keeps a tombstone without its value. The extractor is never shown an erased stored record again; a value the user states after the request opens a new entry, while questions about earlier times still abstain.
Episode scrubbing. LEDGER+EP removes from the episode store every event behind an erased value, every turn that mentions an erased value, every turn up to the request that names a deleted slot outright, and every assistant or tool-call turn up to the request that names an entity the request is about. The last two rules cover what the ledger never extracted. Scrubbing more turns lowers the risk of exposure but can also remove useful details; we did not measure this loss of recall. Every removed event is blocked, so a replay after a crash cannot ingest it again.
Guarantee and limits. After a deletion, no value recorded before the request is returned again, for current or past questions, by LEDGER or by either mode of LEDGER+EP. This holds when the affected slots are found (A2) and the extractor captured the statements of the value (A1). One case remains open in LEDGER+EP: a trusted turn whose value the extractor missed, and which names neither the deleted slot nor any erased value, stays in the episode store and can still be returned. Deletion does not reach earlier snapshots, the response cache or run logs, which a deployment would also have to clear.
Persistence and isolation
Each store writes its full state, including tombstones and blocked ids, to one JSON file and can restore it; LEDGER+EP writes both stores to the same file. In our runs a snapshot is taken after every checkpoint; the file is not written atomically, and we did not test a crash during a write. All state is partitioned by user, and retrieval and deletion only touch the partition of the user id they are given (J7).
| Small s0–4 | Held-out s5–9 | Long s0–2 | LongMem-Eval | |
| Users (histories) | 3 | 3 | 6 | 210 |
| Sessions | 48 | 48 | 360 | ≈48 |
| Events | 184–218 | 184–217 | 1,151–1,203 | ≈493 |
| Tokens | 1.8k–2.3k | 1.9k–2.4k | 11.3k–12.0k | 95k–104k |
| Facts | 22–34 | 20–33 | 60–81 | – |
| Mutations | 63–105 | 64–100 | 242–329 | – |
| Deletions | 4–7 | 5–9 | 20–32 | – |
| Checkpoints | 4 | 4 | 8 | 1 |
| Probes (total) | 512 | 526 | 1,379 | 210 |
Table IV. Workloads. Synthetic columns give the range per seed (one timeline per seed); probes are summed over seeds. LongMemEval values are per history. Tokens: the text of all events of one seed or one history, in model tokens (Sec. IV-C).
Evaluation Design
Coding-agent workload with planted ground truth
A seeded generator produces multi-user, multi-session coding-agent timelines. Each user owns several services; each service has 3–5 facts drawn from nine kinds (database, test command, deploy branch, staging host, runtime, message queue, CI provider, cloud region, on-call owner). A state machine writes each fact’s history as mutations: introduce, restate, update, contradict (from a document or the assistant only) and delete (from the user only). Every mutation is rendered by a template of its source: user message, tool output, code diff, document, or assistant message, alongside chit-chat, tool calls and distractors that name a topic without a value. At checkpoints the generator asks probes of six types: current, post-update, historical (with an as-of time), contamination, post-delete and abstention. The complete generator settings (mutation weights, source mix, probe counts per type) are in the supplement (S1).
Ground truth. Expected answers come from a function of the workload code that reads only each mutation’s kind and time: the answer is the last introduced or updated value at the target time, after the last deletion. It never calls a memory system and never reads event kinds or trust. A text check verifies that no event mentions a value it does not carry, and values are unique per user. We wrote the workload, the oracle and LEDGER together, and the workload encodes the same trust policy: contradictions come only from documents and the assistant. It therefore tests whether a memory follows this policy, not whether the policy is right (A3). Held-out seeds and LongMemEval are our checks against fitting the workload.
We use two presets (Table IV): small for the main comparison and ablations, and long, a simulated year and a half of sessions, for behaviour over time. Seeds 5–9 of the small preset are held out: no design decision was made on them.
LongMemEval
LongMemEval_S [8] has 500 questions, each after a history of about 48 chat sessions. We group the questions into seven categories (six question types and abstention) and draw 30 from each with Python’s random.sample and seed 0 (210 questions; ids in the artifact). Each history is replayed session by session in date order into a fresh store, and the question is asked at its date. Answers are judged with the benchmark’s own prompts, verbatim, on its judge model gpt-4o-2024-08-06 at temperature 0; the only change is that the judge returns yes/no in a JSON field.
Before the main run, a 14-question pilot, drawn in the same way, chose between the fallback and augment modes; we report both. Ten pilot questions are also in the 210, so mode selection saw part of the evaluation set and we do not treat it as untouched test data. Without them, accuracy changes by at most one point per system (augment 75.0%, full replay 74.0%), and no conclusion changes; the ten ids and these scores are in the artifact.
To check the judge, one author audited 60 of its decisions without knowing the system: for full replay, transcript RAG and LEDGER+EP in augment mode, 10 the judge accepted and 10 it rejected, covering all seven categories. The judge agreed with the human on 59 of 60 (98.3%, Cohen’s = 0.97). Mem0, LEDGER and fallback mode were not audited, so this agreement does not validate every system’s score.
Protocol
Systems. No memory; full replay (the whole history, unbounded); transcript RAG (BM25 over events, top 5, within the budget); Mem0 [10] (OSS release 1.0.11, unmodified, with a local vector store, text-embedding-3-small and our extraction model; top 10), and the same with its list of personal fact types replaced by our extractor’s definition of a coding fact, fixed before any run (coding prompt); LEDGER with the LLM extractor and with a rule-based extractor (four hand-written regular expressions, a model-free floor, not a tuned baseline); and LEDGER+EP. Bounded systems share one evidence budget (2,000 tokens on the coding workload, 4,000 on LongMemEval); the runner truncates whole items, but in our runs it never cut any evidence (see below).
Shared answerer. Every system’s evidence goes to the same model (gpt-4.1-mini-2025-04-14, temperature 0, fixed seed) with the same prompt, memory first and question last. The answer is structured (value, abstain flag), so on the coding workload abstention is read from a field and answers are scored deterministically against the planted truth with no LLM judge. An empty or refused answer counts as an abstention.
Two scores. Each probe of the coding workload is scored twice. The evidence verdict scores what memory sent to the model; the answer verdict applies the same rules to the shared model’s answer. A value counts as present when its text, or a fixed alias such as “PG N”, appears as a whole phrase, ignoring case. Each forbidden value is labelled by why it is wrong: resurrected (stated before the last deletion), and after it stale (set before the target time), future (set after the as-of time) or contaminated (from a contradiction). In a historical question, the value valid at the as-of time is the correct answer, not a forbidden one.
Verdicts follow a fixed order. For a probe with an expected value: false abstention if the system abstained; otherwise correct (the value appears, no forbidden value does), polluted (both appear), wrong (only a forbidden value) or missed (neither). For a probe whose truth is abstention: leaked if a forbidden value appears, even if the system also abstained; otherwise correct if it abstained, or no abstention. Deleted-value counts are read from the evidence, whatever the verdict. At the evidence level only an explicit abstention by the memory counts, so systems without an abstention signal (no memory, full replay, transcript RAG) score no abstention on these probes. Only correct counts as a success.
Aggregation and intervals. Evidence and answer accuracy are the share of correct verdicts over all probes of a seed. We report the mean and sample standard deviation over seeds; counts are pooled over seeds, and a probe counts once for each type of forbidden value it contains. On LongMemEval, each system runs once on the same 210 questions; accuracy describes the balanced subset, not the benchmark’s own mix. Its 95% interval is a percentile bootstrap over questions (2,000 resamples), and two systems are compared with a paired bootstrap on the same questions. We do not correct for multiple comparisons, so these comparisons are exploratory.
Cost and efficiency. The runner, not the system, measures latency, counts all injected tokens, and converts model usage to dollars with prices pinned on 2026-10-19 (input charged as uncached, an upper bound). All token counts are model tokens of the answering model’s tokenizer (o200k_base). Our runs counted words and punctuation marks when fitting the budget; the largest bounded evidence set was 205 model tokens on the coding workload and 2,538 on LongMemEval, so neither count cut any evidence, and we recount every logged evidence set with the model tokenizer. Latency is wall-clock time around each system call on one laptop (Apple M5, 16 GB), one call at a time, including model and embedding calls; LLM timings come from uncached repeats, because a cached replay is not a timing. We report write cost per session or history, read and answering cost per question, injected tokens, latency and store size.
Reproducibility. Extraction, routing, Mem0’s writes and answering use gpt-4.1-mini; every logged response reports the snapshot gpt-4.1-mini-2025-04-14. Sampling is pinned (temperature 0 and a fixed seed); every response is cached on disk under a hash of provider, model, prompt and schema, so a run replays at zero cost. Five seeds give variance across workloads; repeated runs with the cache bypassed measure the model’s own noise. Code, workloads, pinned prices, all run logs, sampled ids, per-seed results, the pilot analysis, the cost breakdown and the supplement are available at https://anonymous.4open.science/r/memory-challenge-4482/.
Coverage of the challenge requirements
We measure directly persistence across sessions and restarts, provenance, update, supersession and deletion, bounded evidence and abstention (R1); an established benchmark, required baselines under the same model, prompt and budget, and quality, tokens, cost, ingest time, latency and storage (R2); and checkpoints, typed stale errors, growth, pinned models and repeated runs (R3). We cover four only in part: our traces are heterogeneous but generated from templates; latency growth over time is not reported, because the long-preset replays were cached; the judge audit covers three of seven configurations; and, the main gap, no agent performs a downstream task. A shared model answers questions at checkpoints, so we measure whether memory provides the right facts, not whether an agent fixes a bug or deploys a service better with memory. The full mapping is in the supplement (S2).
Results
Wrong values in the evidence (RQ1)
Trust. With the whole history in context, full replay’s evidence is clean in only 21.9% of probes (Table V). Its answers are right in 82.1% of probes, but 65 of them contain a forbidden value. In 45 of these the model sides with a stale document or with the assistant’s own guess over the user, and it answers only 31 of 67 contamination probes. LEDGER answers 62 of 67, because the less trusted claim never reaches the context. Its evidence is correct in 91.7% of probes. Four of its answers contain a forbidden value, all stale values from updates the extractor missed.
| System | Answer | Evidence | Wrong ans. | Forbidden in evidence: probes (s/f/c) | Deleted value | Tokens | $ |
| No memory | 29.5 ± 1.5 | 0.0 ± 0.0 | 0 | 0 (0/0/0) | 0 / 0 | 0 | 0.05 |
| Full replay | 82.1 ± 3.0 | 21.9 ± 1.8 | 65 | 249 (129/93/93) | 46 / 4 | 1,069 | 0.28 |
| Transcript RAG | 74.8 ± 3.6 | 27.0 ± 5.0 | 88 | 195 (101/69/66) | 39 / 0 | 128 | 0.08 |
| Mem0 | 66.8 ± 1.8 | 34.9 ± 2.7 | 66 | 107 (45/56/13) | 18 / 17 | 141 | 0.37 |
| Mem0 (coding prompt) | 70.6 ± 6.4 | 34.7 ± 2.9 | 82 | 143 (44/76/36) | 16 / 13 | 158 | 0.43 |
| LEDGER (rules) | 39.5 ± 6.9 | 38.7 ± 6.3 | 9 | 9 (9/0/0) | 0 / 0 | 5 | 0.05 |
| LEDGER (LLM) | 90.9 ± 5.5 | 91.7 ± 5.2 | 4 | 4 (4/0/1) | 0 / 0 | 17 | 0.23 |
| LEDGER+EP fallback | 94.3 ± 4.2 | 66.8 ± 4.6 | 10 | 16 (6/8/3) | 0 / 0 | 61 | 0.23 |
| LEDGER+EP augment | 94.4 ± 2.2 | 32.4 ± 7.8 | 15 | 193 (104/70/60) | 0 / 0 | 144 | 0.25 |
Table V. Coding-agent workload, small preset, seeds 0–4 (512 probes). Answer and evidence accuracy (%, mean ± SD over seeds); counts pooled over seeds (Sec. IV-C). Forbidden in evidence: number of probes whose evidence held at least one stale, future or contaminated value, then the probes per type; types overlap, so they do not sum to the first number. Deleted value: evidence/answer, deletion requests with description only. Tokens: mean injected per probe, in model tokens. $: memory plus answering, all seeds.
Model-decided updates. Mem0 lets the model add, update or delete each extracted fact. An update overwrites the memory in place, so the value an earlier date needs is gone: Mem0 answers 27 of 93 historical questions (LEDGER 69, full replay 74), and 27 of its answers give a later value for an earlier date. With the coding prompt it answers more current questions (96 instead of 86 of 112), but its contaminated answers rise from 10 to 27. Mem0 and LEDGER also differ in extraction, routing and retrieval, so this comparison does not isolate one cause.
Deletion. When deletion requests carry only a description, the transcript baselines remove nothing. Full replay sends the deleted value to the model in 46 probes and repeats it to the user in 4; transcript RAG sends it in 39. At answer level the leak is mostly hidden, because the model also sees the “forget it” message. For Mem0 the request is handled only by its ordinary update step, which kept the value: it reached the model in 18 probes and the user in 17 (16 and 13 with the coding prompt). This is a result for the evaluated configuration. LEDGER and LEDGER+EP sent no deleted value, in either mode.
Answer accuracy (RQ2)
Coding workload, development seeds (0–4). We developed the rules and safeguards on these seeds. LEDGER+EP answers 94.3% (fallback) and 94.4% (augment), LEDGER 90.9%, full replay 82.1%, transcript RAG 74.8%, and Mem0 66.8% (70.6% with the coding prompt). LEDGER+EP gains over LEDGER mostly by answering historical questions on which the ledger abstained. In fallback mode, 16 probes have at least one forbidden value in the evidence, against 249 for full replay and 195 for transcript RAG. In augment mode there are 193,
so its correctness depends on the answering model preferring the ledger’s fact line, which comes first.
Coding workload, held-out seeds (5–9). We ran these seeds once, with the final systems (Table VI). LEDGER+EP in fallback mode drops from 94.3% to 91.1%, while full replay rises from 82.1% to 83.3%, so its lead shrinks from 12.2 to 7.8 points. In augment mode the lead shrinks from 12.3 to 8.5 points, and for LEDGER from 8.8 to 4.9. The development seeds therefore flatter our systems. The ranking of the systems holds.
| System | Answer | Evidence | Wrong ans. |
| No memory | 29.4 ± 2.9 | 0.0 ± 0.0 | 0 |
| Full replay | 83.3 ± 2.8 | 21.1 ± 0.7 | 58 |
| Transcript RAG | 78.5 ± 3.1 | 23.8 ± 3.6 | 81 |
| Mem0 | 66.2 ± 7.1 | 34.8 ± 6.2 | 59 |
| Mem0 (coding prompt) | 68.4 ± 4.3 | 36.0 ± 3.1 | 94 |
| LEDGER (rules) | 37.0 ± 2.2 | 37.0 ± 2.2 | 13 |
| LEDGER (LLM) | 88.2 ± 3.2 | 89.9 ± 1.7 | 5 |
| LEDGER+EP fallback | 91.1 ± 5.4 | 64.2 ± 4.8 | 15 |
| LEDGER+EP augment | 91.8 ± 4.1 | 25.9 ± 3.8 | 23 |
Table VI. Held-out seeds 5–9 of the small preset (526 probes), never inspected during development: answer and evidence accuracy (%, mean ± SD over seeds) and answers containing a forbidden value.
LongMemEval. Full replay, now one 108k-token context per question, reaches 74.8% (Table VII). LEDGER+EP (augment) has the same observed accuracy, 74.8%, with 1,087 injected tokens, 1% of the context. The paired 95% interval of the difference (−6.2 to +6.2 points) still allows a real difference of several points either way, so this is not evidence of equivalence. Against transcript RAG it gains 3.3 points (CI 0.0 to +7.1), mainly on knowledge-update (87 vs. 73) and multi-session questions (60 vs. 43). Mem0 reaches 71.0% with 221 tokens; LEDGER+EP (augment) is 3.8 points ahead (CI −3.8 to +11.4), so this sample does not establish a difference. Mem0 leads on preference (70) and temporal questions (80), the personal-chat setting its prompt was written for, but its write cost is three times ours (161 vs. 50 milli-dollars per history). The ledger alone answers only 41.0%: the benchmark mostly asks for one detail buried in unrelated chats, and by design the ledger never answers from the assistant’s statements (SS-A 3%).
| System | Accuracy | SS-U | SS-A | Pref | Multi | KU | Temp | Abs | Tokens | Write | Read | Ans. |
| No memory | 16.2 (11–21) | 0 | 0 | 13 | 0 | 0 | 0 | 100 | 0 | 0.0 | 0.0 | 0.1 |
| Full replay | 74.8 (69–80) | 100 | 97 | 63 | 53 | 90 | 60 | 60 | 108,447 | 0.0 | 0.0 | 43.5 |
| Transcript RAG | 71.4 (65–78) | 100 | 97 | 33 | 43 | 73 | 67 | 87 | 1,072 | 0.0 | 0.0 | 0.6 |
| Mem0 | 71.0 (65–77) | 83 | 40 | 70 | 53 | 83 | 80 | 87 | 221 | 160.9 | 0.0 | 0.2 |
| LEDGER (LLM) | 41.0 (34–48) | 63 | 3 | 13 | 40 | 60 | 13 | 93 | 15 | 49.5 | 1.5 | 0.1 |
| LEDGER+EP fallback | 70.5 (65–76) | 97 | 93 | 30 | 50 | 67 | 67 | 90 | 593 | 49.5 | 1.5 | 0.4 |
| LEDGER+EP augment | 74.8 (69–80) | 97 | 97 | 33 | 60 | 87 | 67 | 83 | 1,087 | 49.5 | 1.5 | 0.6 |
Table VII. LongMemEval_S, stratified subset of 210 questions (30 per category). Accuracy (%) with a 95% bootstrap interval, by category, and mean injected model tokens. Costs in milli-dollars: Write, to ingest one history, paid once; Read (retrieval, including routing) and answer, paid for every question. Each question has its own history here; when several questions share a history, only the write cost is divided among them. Judge: GPT-4o-2024-08-06 with the benchmark’s prompts.
Growth over time (RQ3)
Figure 2 replays 540 simulated days on the long preset, with eight checkpoints. As the history grows from 598 to 4,399 tokens per query, full replay falls from 89.4% to 70.5% and transcript RAG from 88.0% to 66.5%, and the share of their probes with a forbidden value in the evidence grows from 29% to 55% and from 25% to 47%. LEDGER stays between 85.0% and 93.3%, with 1–3% of probes affected, and LEDGER+EP (fallback) between 89.7% and 96.2%, with 3–7%. Mem0 starts at 78.3% and falls to 56–63% from the fourth checkpoint on. Its store stays small (6.5 KiB), so it does not drown in history; it fails on time instead: 312 of its 1,379 probes carry a forbidden value in the evidence, in 170 a future value and in 130 a stale one.
Figure 2. Long preset (6 users, 60 sessions each over 540 simulated days, 8 checkpoints; mean of 3 seeds). Left: answer accuracy per checkpoint. Right: share of probes per checkpoint whose evidence carried a stale, future or contaminated value (104–197 probes per checkpoint).
Endpoint cost. At the last checkpoint, transcript RAG injects about 130 tokens per question, LEDGER 17 and LEDGER+EP 59, against 4,399 for full replay. The ledger stores 1.3 KiB per seed against 43.5 KiB for the transcript store; LEDGER+EP keeps both in 41.1 KiB, because its deletion scrubbing removes 43–80 more turns per seed. Our systems make at most one extraction call per session and one routing call per question throughout, but each call resends the list of known slots: extraction input grows from 418 to 572 tokens per session and routing input from 319 to 508 tokens per question.
Operational reliability
Restarts. We tested five systems: full replay, transcript RAG, LEDGER with both extractors, and LEDGER+EP in augment mode. After every checkpoint of the long preset, we wrote each store to disk, restored it into a fresh instance and continued. For every system, all 1,379 probes got the same verdict and evidence as in the run without restarts. Snapshots stay under 270 KiB and restore in at most 63 ms.
Model change. In LEDGER+EP (augment mode, long preset, seeds 0 and 1), we switched the model behind extraction and routing from gpt-4.1-mini to gpt-4.1 after the fourth of eight checkpoints and kept the store. No migration was needed. Second-half accuracy rises from 92.0% (mini throughout) to 95.4%, within 0.3 points of a run on gpt-4.1 throughout (95.7%), at $1.18 for extraction and routing against $1.89 on gpt-4.1 throughout.
Run-to-run noise. We ran LEDGER and LEDGER+EP (fallback) three times per seed, twice with the response cache bypassed. In 12 and 8 of the 512 probes, the answer verdict differed in at least one run. The standard deviation of accuracy across repeats is 0.7 and 0.4 points per seed, far below the differences between systems.
Overall cost and efficiency
Table VIII gives the cost of the write and read paths on the small preset. The transcript systems retrieve in under a millisecond, but inject more context: 1,069 tokens per probe for full replay and 128 for transcript RAG, against 17 for the ledger and 61 for LEDGER+EP (fallback). Reading and answering cost LEDGER+EP (fallback) 0.30 m$ per probe against 0.55 m$ for full replay, plus 0.27 m$ per session to write; over all seeds, $0.23 against $0.28.
| System | LLM calls/session | Ingest ms | Retrieve ms | Store KiB | Tokens | Write m | Ans m$ | |
| No memory | 0.00 | 0.00 | 0.00 | 0.0 | 0 | 0.00 | 0.00 | 0.10 |
| Full replay | 0.00 | 0.00 | 0.16 | 8.1 | 1,069 | 0.00 | 0.00 | 0.55 |
| Transcript RAG | 0.00 | 0.00 | 0.35 | 8.1 | 128 | 0.00 | 0.00 | 0.16 |
| Mem0 | 1.65 | 2989 | 174 | 1.5 | 141 | 1.06 | 0.00 | 0.16 |
| Mem0 (coding prompt) | 1.83 | 3272 | 202 | 1.9 | 158 | 1.24 | 0.00 | 0.17 |
| LEDGER (rules) | 0.00 | 0.03 | 0.02 | 0.1 | 5 | 0.00 | 0.00 | 0.11 |
| LEDGER (LLM) | 1.00 | 836 | 666 | 0.4 | 17 | 0.31 | 0.18 | 0.11 |
| LEDGER+EP fallback | 0.93 | 817 | 662 | 8.0 | 61 | 0.27 | 0.17 | 0.13 |
| LEDGER+EP augment | 0.93 | 817 | 662 | 8.0 | 144 | 0.27 | 0.17 | 0.16 |
Table VIII. Efficiency on the coding workload (small preset, seeds 0–4): write-time LLM calls per ingested session, median ingest time per session and retrieval time per query, store size at the end per seed, mean injected model tokens per probe, and cost in milli-dollars to write, per ingested session, and to read and answer, per probe. Times of the LLM systems are from uncached repeats (cached replays are not timings); augment mode shares fallback’s write and routing path. Mem0 with the coding prompt is timed from an uncached rerun, since its logged run reused Mem0’s cached embeddings.
What still fails
On the coding workload LEDGER fails 47 of 512 probes. Twenty are routing misses (“no known slot matches”), mostly paraphrases the router does not link to a slot (“where does it publish its events” for a message-queue slot) or slots filed under a short service name. Sixteen are historical questions asked before the first value the extractor caught: when a fact’s first statement is terse or indirect, every earlier as-of question abstains. Four are stale answers from updates stated indirectly (“build #812 reported by Jenkins”), the only probes in which a forbidden value reaches LEDGER’s evidence; two are assistant guesses correctly held but leaving the question unanswered; and in five the evidence was right but the answering model abstained. None is a rule applied wrongly.
LEDGER+EP (fallback) fails 29. Ten answers carry a forbidden value: four are the ledger’s own stale facts above, and six are old values taken from a raw turn. Four answers come where the truth is to abstain, none with a deleted value. In the other
| Change | Metric | Before | After |
| (a) Single-factor trust-rule changes (cached) | |||
| Hold assistant-first values (trust 2)* | wrong answers | 6.0 | 4.0 |
| Hold document-first values (too (trust 3))* | evidence | 91.5 | 89.7 |
| (a′) Complete-configuration comparisons | |||
| Rule-based → LLM extractor | evidence | 38.7 | 91.7 |
| LEDGER → LEDGER+EP fallback | answer | 90.9 | 94.3 |
Table IX. Ablations on the coding workload (small preset, seeds 0–4). Development-time safeguard and prompt changes are in the supplement (S4). Evidence and answer: % correct; other metrics: number of probes.
15 the answering model obtained, mostly because BM25 did not find the right turn. On LongMemEval the dominant failure is recall of a single buried detail: gpt-4.1-mini extracts many incidental “facts” per task chat and misses some personal details, and preference questions (33% for LEDGER+EP) need a summary that neither facts nor five turns give.
Ablations
(a) Single-factor trust-rule changes change one factor and keep everything else. Both rows replay every extraction from cache, so only the rule changes and the difference is exact. Holding the assistant’s first values (trust 2) turns two of six wrong answers into abstentions and changes nothing else.
Holding documents as well (trust 3) costs nine correct answers and fixes none, so documents may still introduce a value.
(a′) Complete-configuration comparisons come from the main runs (Table V): same seeds, answer and budget. They change more than one component. The extractor row changes the whole extraction method. The LEDGER → LEDGER+EP row adds the episode store but also stops extracting low-trust events, so it does not isolate the episode store. Their differences exceed the run-to-run noise, but the same seeds do not remove this confounding.
The safeguards of Sec. III-C and the prompt revisions were each measured once during development, before and after the change; these single runs are in the supplement (S4) and establish no stable, isolated effect. We did not run leave-one-out ablations of the final system, or ablate the trust ranking itself, the episode retriever or the token budget.
Limits of the results
The coding workload is synthetic, built by us, and phrased by templates, and our lead over full replay is about four points smaller on held-out seeds. Its histories stay short (about 4.4k tokens per user at the end), so the long preset tests growth in facts and updates more than in context length. The runs include extraction and routing errors, so they show how close the full system comes to the guarantees of Sec. II, not that they hold. Restarts happen only between checkpoints, and the model change covers one upgrade within one model family, on two seeds.
Discussion
The evidence–answer gap. The answering model hides much of the difference between memories. It often picks the right value from noisy evidence, so full replay answers 82.1% of the coding probes while its evidence is clean in only 21.9%. This repair weakens as history grows, and it never helps privacy: a deleted value that the model does not repeat has still been sent to the provider. In our runs, the largest differences between memories came from deciding what is current, which source wins and what was deleted, more than from finding facts. Memory should therefore be judged on its evidence as well as on its answers.
Configuration trade-offs. The variants differ in who is responsible for correctness. The ledger alone sends only values that passed its rules, but abstains when it has no fact. Fallback mode adds raw turns only on those questions, so it keeps forbidden evidence low (16 probes against 193 for augment) at almost the same accuracy; we recommend it where facts change and sources disagree. Augment mode relies on the answering model to prefer the fact line and suits long chats with many one-off details, as on LongMemEval. Both move cost from reading to writing: with the write cost per history, and the read and answering costs per question, and full replay’s cost per question, LEDGER+EP is cheaper after questions per history, on LongMemEval the second question (, , , m$). The write cost grows with the number of known slots, not with the length of the raw history.
Threats to Validity
Construct validity. We built both the coding workload and the system, so the workload may favour our design. Every system runs on the same timelines with the same answerer, and we also report LongMemEval, which we did not design. Our workload states every change when it happens, so it never tests A4, and how often real traffic breaks A3 or A4 is untested. The LongMemEval judge is a model, audited for three of the seven configurations.
Internal validity. We developed the rules and safeguards while looking at failures on seeds 0–4, and we report results on these seeds. Held-out seeds 5–9, run once, show that the results do not depend on the timelines we inspected, but not that they hold outside the generator, because new seeds use the same templates, fact kinds and mutation rules. The LongMemEval mode choice used a pilot that overlaps the evaluation set.
External validity. The coding workload uses templated phrasings and nine kinds of facts. Real traces are messier, and other fact kinds or phrasings may be harder to extract and route. All extraction, routing and answering use one model family (gpt-4.1-mini, and gpt-4.1 in the model-change run), and all text is English. Other model families may extract differently. Besides the baselines, we compare against one memory product, Mem0; Zep and A-MEM are not evaluated.
Conclusion validity. The coding results use five seeds, and the standard deviation of our systems is up to 5.5 points. The LongMemEval subset is 210 of 500 questions. Differences of a few points, such as between LEDGER+EP and Mem0 or full replay, lie within the paired confidence intervals, which are not corrected for multiple comparisons; small differences should not be read as rankings.
Related Work
Persistent agent memory. Several systems give agents memory that lasts beyond one context window. MemGPT [3] moves text between the context and external storage, and the model edits its own memory through function calls. Mem0 [10] extracts facts from each session and lets the model add, update or delete them. A-MEM [12] stores notes that the model links and rewrites over time. Zep [11] keeps a temporal knowledge graph whose edges record when a fact held, and uses the model to find contradicting edges. Other work stores reflections [16], reusable skills and workflows [17, 18], or a graph index for retrieval [19]. In all of these systems the model decides when a stored fact is replaced or removed; in LEDGER it only proposes claims, and fixed rules decide. Zep is closest to our design, since it also keeps when a fact held, and it also tries to read such dates from the text, which our ledger does not. Mem0 is closest to our pipeline, but an update overwrites the stored text, so the value an earlier date needs is gone.
Temporal and provenance-aware data management. The ideas we use for storage are well established. Temporal databases keep the time during which a fact was true (valid time) apart from the time it was recorded, so a change can be entered after it happened, and they close an interval instead of overwriting a row [15]. Our ledger uses only a restricted form: valid time starts when a value is stated (A4), so it does not recover the real date of a change reported later. Provenance research records which source produced each piece of data [20]. Our contribution is to bring them into agent memory, where the facts come from a language model and not from a trusted application. This requires parts that a database does not have: a trust order over sources that decides which value wins, a lexical check that rejects claims whose value does not appear in their source event, and deletion that also removes matching raw turns and blocks their source events from being written again.
Memory evaluation. LoCoMo [13] tests recall over very long conversations. LongMemEval [8] tests five abilities, including knowledge updates, temporal reasoning and abstention. MemoryAgentBench [9] adds conflict resolution and finds that no current method masters all of its competencies. These benchmarks score the final answer, and their histories contain no deletion requests. Our evaluation also scores the evidence, labels each wrong value by why it is wrong, and tests deletion by description, restarts and a model change.
Conclusion
We presented LEDGER, an agent memory in which a model proposes facts and explicit rules decide which value is current, which source wins and what is deleted, and LEDGER+EP, which adds raw turns. Our strongest supported finding is about evidence: on a coding workload with planted changes, conflicting sources and deletion requests, LEDGER+EP in fallback mode sent a forbidden value in 16 of 512 probes, against 249 for full replay, and no deleted value, while answering more questions correctly on both development and held-out seeds. On LongMemEval it reached the same observed accuracy as full replay with 1% of the context. The main limitation is that the coding workload is synthetic and built by us, and that we measure answers to questions, not an agent’s success on downstream tasks.
References
References
- [1]T. Joshi, S. Chowdhury, and F. Uysal, “SWe-bench-cl: Continual learning for coding agents,” arXiv preprint arXiv:2507.00014, 2025.arxiv.org/abs/2507.00014
- [2]N. C. Rakotonirina, M. Hamdy, J. A. Campos, L. Weber, A. Testoni, M. Fadaee, S. Pezzelle, and M. Del Tredici, “From tools to teammates: Evaluating LLMs in multi-session coding interactions,” in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar, Eds. Vienna, Austria: Association for Computational Linguistics, Jul. 2025, pp. 19 609–19 642. [Online]. Available: https://aclanthology.org/2025.acl-long.964/
- [3]C. Packer et al., “MemGPT: Towards LLMs as operating systems,” arXiv preprint arXiv:2310.08560, 2023.arxiv.org/abs/2310.08560
- [4]Y. Hu, Y. Wang, and J. McAuley, “Evaluating memory in llm agents via incremental multi-turn interactions,” in International Conference on Learning Representations, vol. 2026, 2026, pp. 156 259–156 291.arxiv.org/abs/2507.05257
- [5]M. Zhang, O. Press, W. Merrill, A. Liu, and N. A. Smith, “How language model hallucinations can snowball,” arXiv preprint arXiv:2305.13534, 2023.arxiv.org/abs/2305.13534
- [6]B. Xie, C. Wang, G. Ge, S. Wang, F. Niu, H. Zhang, and W. Wang, “Can agents forget when asked? a benchmark for memory unlearning in llm agents,” arXiv preprint arXiv:2610.18052, 2026.arxiv.org/abs/2610.18052
- [7]Z. Xiong, Y. Lin, W. Xie, P. He, Z. Liu, J. Tang, H. Lakkaraju, and Z. Xiang, “How memory management impacts llm agents: An empirical study of experience-following behavior,” in Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2026, pp. 623–645.
- [8]D. Wu et al., “LongMemEval: Benchmarking chat assistants on long-term interactive memory,” in International Conference on Learning Representations, 2025.arxiv.org/abs/2410.10813
- [9]Y. Hu et al., “Evaluating memory in LLM agents via incremental multi-turn interactions,” arXiv preprint arXiv:2507.05257, 2025.arxiv.org/abs/2507.05257
- [10]P. Chhikara et al., “Mem0: Building production-ready AI agents with scalable long-term memory,” arXiv preprint arXiv:2504.19413, 2025.arxiv.org/abs/2504.19413
- [11]P. Rasmussen et al., “Zep: A temporal knowledge graph architecture for agent memory,” arXiv preprint arXiv:2501.13956, 2025.arxiv.org/abs/2501.13956
- [12]W. Xu et al., “A-MEM: Agentic memory for LLM agents,” arXiv preprint arXiv:2502.12110, 2025.arxiv.org/abs/2502.12110
- [13]A. Maharana et al., “Evaluating very long-term conversational memory of LLM agents,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 2024.
- [14]S. Robertson and H. Zaragoza, “The probabilistic relevance framework: BM25 and beyond,” Foundations and Trends in Information Retrieval, vol. 3, no. 4, pp. 333–389, 2009.
- [15]R. T. Snodgrass, Developing Time-Oriented Database Applications in SQL. Morgan Kaufmann, 1999.
- [16]J. S. Park et al., “Generative agents: Interactive simulacra of human behavior,” in Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, 2023.arxiv.org/abs/2304.03442
- [17]G. Wang et al., “Voyager: An open-ended embodied agent with large language models,” arXiv preprint arXiv:2305.16291, 2023.arxiv.org/abs/2305.16291
- [18]Z. Wang et al., “Agent workflow memory,” arXiv preprint arXiv:2409.07429, 2024.arxiv.org/abs/2409.07429
- [19]B. J. Gutiérrez et al., “HippoRAG: Neurobiologically inspired long-term memory for large language models,” in Advances in Neural Information Processing Systems, 2024.arxiv.org/abs/2405.14831
- [20]P. Buneman, S. Khanna, and W.-C. Tan, “Why and where: A characterization of data provenance,” in Proceedings of the 8th International Conference on Database Theory (ICDT), 2001, pp. 316–330.DOI

