betaFeedback

WorksComputer Science

Instant Weight-Space Consolidation for Long-Term Memory in LLMs

Abstract

Agentic memory systems give LLMs persistent memory by extracting experience into text and retrieving it at query time, but the model itself does not evolve: however many interactions it has had, it starts each query the same. Extraction errors compound, implicit cross-session patterns that no single fragment states cannot be surfaced, and the construction cost grows with every conversation. Fine-tuning on past interactions does change the weights, but requires backpropagation at serving time and degrades on long histories. We introduce CLAM (Carried Latent Adapter Memory), which combines the strengths of both: a recurrent hypernetwork performs parametric consolidation, compiling a stream of session activations into a low-rank adapter in a single forward pass with no gradient steps and no language-model generation calls, while a complementary episodic buffer supplies verbatim detail at inference. A gated latent state carries knowledge across sessions in fixed size, so the parametric memory accumulates without growing. On LoCoMo and LongMemEval-S across three open models (2-7B parameters), CLAM leads all agentic baselines by 1-45 points and all gradient-based parametric methods by 10-45 points, while requiring zero generative LLM calls, zero gradient steps, and 3-151x less wall-clock time than extraction systems to build its memory. The two channels are complementary: the consolidated adapter adds 12-15 accuracy points when the episodic buffer misses the gold evidence, and still adds 4-5 points even with an oracle buffer.

Introduction

Biological memory is not a database that an organism queries; it is a multi-level process in which two complementary systems cooperate. A fast episodic store, seated in the hippocampus, captures specific experiences rapidly and without interference. A slow semantic store, residing in neocortical synapses, integrates those episodes over time into structured, persistent knowledge [33] (Tulving 1972). The bridge between them is consolidation: transient experience becomes a durable change in synaptic weights, and after consolidation the organism is physically different. Complementary Learning Systems (CLS) theory formalizes why both stores are necessary: the fast store gives exact recall of specific episodes without overwriting prior knowledge, the slow store gives generalization and cross-episode integration, and neither alone suffices [24, 18]. They are complementary precisely because they trade off different failure modes: the fast store is lossless but unintegrated, the slow store is integrated but lossy. A system that has only one operates with a fundamental blind spot.

Contemporary LLM memory systems are sophisticated episodic stores. Agentic frameworks extract facts and summaries from conversations, organize them as profiles or knowledge graphs, and retrieve relevant fragments into the context window at query time [5, 28, 25, 26, 20] (Xu et al. 2025; Zhong et al. 2024). This is effective when the needed information was explicitly stated, correctly extracted, and successfully retrieved, but each step introduces error that compounds across the pipeline: extraction misses implicit facts, indexing loses context, and retrieval surfaces the wrong fragment for the query at hand. More fundamentally, the model’s weights never change. Implicit patterns that no single fragment states, cross-session structure that requires integration, and preferences that accumulate gradually over many interactions have no pathway into the model. In CLS terms, these systems have a sophisticated hippocampus and no neocortex: no consolidation pathway turns experience into weight-resident knowledge.

The natural remedy is to write experience into the weights directly. A nascent line of work does so by fine-tuning at test time: generating QA pairs from the conversation and training on them with SGD (Ren et al., [30]), or adapting on unlabelled context with an entropy gate (Hu et al., [14]). This does produce a parametric change, but it requires backpropagation at serving time, which is operationally costly and slow. More fundamentally, as our experiments show, gradient based consolidation degrades on long histories: the gradient signal becomes noisy, the model overfits early sessions, and performance drops as the history grows. It is not the stable, accumulating consolidation that long-term memory requires.

We take a different approach that combines the strengths of both paradigms. A hypernetwork, trained once offline, can map session activations to adapter weights in a single forward pass: the persistence of a weight change at the speed of inference, with no gradient step at test time. We build CLAM (Carried Latent Adapter Memory) on this observation. A recurrent hypernetwork performs parametric consolidation: it ingests each session’s activations from the frozen base LLM and folds them into a compact latent state through a learned, gated recurrence; a single forward pass over the accumulated state emits the low-rank adapter the model wears while answering. A complementary episodic buffer supplies verbatim detail at inference for what a bounded parametric state cannot retain with perfect fidelity. The parametric consolidation channel plays the role of slow, integrated knowledge; the episodic buffer plays the role of fast, lossless recall; and we find they are complementary.

Main results. On LoCoMo and LongMemEval-S across three open base models (Gemma-2-2B (Gemma Team et al., [8]), Qwen3-4B (Yang et al., 2025), Mistral-7B (Jiang et al., [16])), CLAM leads all agentic baselines by 1–45 points and all gradient-based parametric methods by 10–45 points, with 95% bootstrap confidence intervals that exclude zero in 28 of 30 paired comparisons. It does this with zero generative language-model calls, zero gradient steps, and zero generated tokens to build its memory: one hypernetwork forward pass per conversation, 3–151× faster than extraction systems that spend 12–464 generative LLM calls and 3×1043 \times10^{4} to 6.2×1056.2 \times10^{5} generated tokens on the same conversation, and 1.3–16× faster than gradient-based methods that take 18–63 backward passes.

The two channels validate the complementary-systems design: adding parametric consolidation on top of the same episodic buffer is worth +7 to +11 points overall; the adapter contributes +12 to +15 points on the 36% of questions where the buffer fails to surface the gold evidence; and it still adds +4 to +5 points even when the buffer is given oracle access to the gold spans, confirming that the consolidated weights carry information the episodic channel cannot supply rather than a better-indexed copy of it. Contributions.

  • CLAM: a recurrent hypernetwork that performs parametric consolidation of interaction sessions into a low-rank adapter in a single forward pass with a carried latent state, paired with an episodic buffer in a dual-channel architecture inspired by complementary learning systems.

  • A scalable multi-session data pipeline, used to generate up to ∼900\sim900K high-quality, memory-dependent training trajectories.

  • Rigorous evaluation across two benchmarks, three model scales, and twelve baselines spanning both agentic and parametric paradigms.

Related Work

Memory Systems for LLMs

The dominant paradigm stores interaction history externally and retrieves it at query time. Flat retrieval indexes raw conversation chunks and prepends the top matches ([19, 9]). Profile-plus-episodic systems additionally extract structured user facts (Chhikara et al., [5]). Graph-based systems organize memory as a (temporal) knowledge graph for multi-hop traversal ([28, 17]). Agentic architectures manage memory with explicit read/write tools: MemGPT/Letta treats context like paged virtual memory (Packer et al., [25]), and Generative Agents periodically reflect episodic observations into higher-level summaries (Park et al., [26]). Others add forgetting dynamics (Zhong et al., 2024), compression (Fang et al., [7]), self-organizing links (Xu et al., 2025), or intent-aware retrieval over compressed memory units (Liu et al., [20]).

Despite their diversity, these systems share one property: the base model’s parameters never change. Whatever consolidation they perform, summarizing, re-indexing, graph-building, happens in token space and is re-read as context. They are thus non-parametric; most require per-item LLM extraction, so memory is not built instantly; none accumulate in a bounded parametric state that improves with more sessions (their stores grow without bound); and knowledge is not persistent across queries without re-reading history. CLAM is parametric, instant, accumulating, and persistent at once, which motivates our design. A nascent line does modify weights to consolidate memory over time [2]; CLAM shares this goal but writes memory through a single hypernetwork forward pass rather than gradient-based self-modification, and pairs it with episodic recall.

Hypernetworks and Context Distillation

CLAM belongs to a growing line of work that generates adapter weights with a hypernetwork [10], paired with context distillation, which trains a model to behave as if a context were present without reading it [32, 1]; recent work distills context into a latent memory rather than into weights (Zheng et al., 2026). Adapter generators map an input to a low-rank adapter in one forward pass: from a document [4], from a task description [3], an evolving code repository through a recurrent state [12], a reusable skill (Zhao et al., 2026), or video frames (Suri et al., 2026); from an LLM used directly as a weight generator [21]. CLAM reuses this document-to-adapter foundation but differs on three axes. Input: these methods condition

on a single static artifact (one document, skill, or clip); CLAM consumes a stream of interaction sessions. Accumulation: to handle many inputs, most prior work composes one adapter per chunk by rank-concatenation or merging, so the represented set grows with the number of chunks and the merged signal dilutes (roughly 1/N1/\sqrt{N}), which does not scale. Code2LoRA-Evo [12] instead carries a recurrent state across code diffs. CLAM instead keeps a fixed rank and accumulates through a recurrent neural update of a fixed-size latent state, so adapter cost and capacity stay constant as sessions grow. Setting: none target long-term LLM memory or pair adapter generation with a complementary episodic-retrieval channel. We also relate to test-time training [34, 11], which adapts weights per input via gradient steps; our adaptation is a single forward pass with no test-time optimization.

Carried Latent Adapter Memory

A user interacts with an assistant over a stream of sessions S=(S1,…,SN)\mathcal{S} = (S_1,\ldots,S_N); at query time the system must answer using knowledge distributed across the entire stream. CLAM answers through two complementary channels:

y=fθ+ΔW(q,C),(1)y = f_{\theta+\Delta W}(q, C), \tag*{(1)}

where θ\theta are the frozen base-model weights; ΔW=Hϕ(S1:N)\Delta W = H_{\phi}(S_{1:N}) is a low-rank adapter produced by the consolidation channel—a recurrent hypernetwork HϕH_{\phi} that compiles the full session stream into weight-resident knowledge in a single forward pass; and C=Buffer⁡(q,S1:N)C = \operatorname{Buffer}(q,S_{1:N}) is local context supplied by the episodic buffer at query time. The consolidation channel is computed once per stream and cached; it does not depend on qq. The episodic buffer is per query and does not modify θ\theta.

Consolidation Channel

The consolidation channel maps a session stream to an adapter through three stages: read each session into a fixed-size latent workspace, carry the workspace across sessions via a gated recurrence, and decode the final workspace into adapter weights [13].

Read. To encode session StS_t, we pass it through the frozen base model and collect per-layer hidden states Zt∈RL×∣St∣×dZ_t \in\mathbb{R}^{L \times|S_t| \times d}, where LL is the number of transformer blocks and dd the model dimension. A stack of Perceiver cross-attention blocks [15] compresses ZtZ_t into a fixed-size workspace: mm latent queries cross-attend into ZtZ_t, with each slot pooling the session independently of the others.

Carry. Memory requires the workspace to persist and accumulate. We maintain a carried state Qt∈Rm×dQ_t \in\mathbb{R}^{m \times d} that threads through the stream via a gated recurrence (Figure 1b). At each step: (1) the Perceiver encoder reads StS_t using the carried state Qt−1Q_{t-1} as its latent queries, conditioning the read on all prior sessions—each slot pools independently, attending only to the session tokens; (2) a single self-attention layer over the mm slots lets them exchange information and reorganise—this is the only point at which slots communicate; (3) a gated write folds the result into the carried state:

Overview of CLAM and recurrent consolidation cell

Figure 1. (a) CLAM overview: sessions are consolidated into a carried workspace QtQ_t via a recurrent hypernetwork that decodes QNQ_N into a LoRA adapter; the episodic buffer retrieves spans at query time. (b) One consolidation step: Qt−1Q_{t-1} cross-attends into session activations ZtZ_t, slots reorganise via self-attention, and a gate controls the write.

gt=σ(Wg[xt;Qt−1]+bg),Qt=LN⁡((1−gt)Qt−1+gtxt).(2)g_t = \sigma\left(W_g[x_t; Q_{t-1}] + b_g\right), \qquad Q_t = \operatorname{LN}\left((1-g_t)Q_{t-1} + g_t x_t\right). \tag*{(2)}

where xtx_t is the output of the read-and-reorganise steps. The gate produces one scalar per slot, so different slots can retain stable long-lived facts while others are overwritten by new evidence. Layer-Norm after each update prevents magnitude drift over long streams.

Decode. A single Perceiver decoder block maps the final workspace QNQ_N to adapter weights. It uses L×rL \times r output queries—one per (layer, rank-index) pair—each decoded into one rank-1 component:

ΔWℓ=αBℓAℓ,Aℓ∈Rr×dff,Bℓ∈Rd×r,(3)\Delta W_\ell= \alpha B_\ell A_\ell,\qquad A_\ell\in\mathbb{R}^{r \times d_{\mathrm{ff}}},\quad B_\ell\in\mathbb{R}^{d \times r}, \tag*{(3)}

applied to the feed-forward down-projection in every block. The adapter rank is fixed regardless of NN: accumulation happens inside the carried state, not by growing or merging adapters. Consolidating one additional session is an O(1)O(1) update, and the persisted artifact is the same size.

Episodic Buffer

A fixed-size parametric state cannot retain every verbatim detail with perfect fidelity—exact dates, proper names, precise figures require lossless surface form. The episodic buffer provides this complement: it supplies local evidence to a model whose weights have already been shaped by the full interaction history. The adapter encodes cross-session structure—what the user cares about, how entities relate, which topics recur—while the buffer provides the specific tokens the adapted model needs to ground its answer.

We implement the buffer with minimal machinery: each session is embedded with a sentence encoder [29], and at query time we return the top-kk pieces by cosine similarity. Our ablations show the gains are robust to retriever sophistication, and stronger indexing is orthogonal and only helps further.

Inference

At query time: (1) load the cached adapter ΔW=Hϕ(QN)\Delta W = H_\phi(Q_N); (2) retrieve top-kk spans CC; (3) merge θ′=θ+ΔW\theta' = \theta+ \Delta W; (4) generate y=fθ′(q,C)y = f_{\theta'}(q,C). Step 1 is amortised across all queries for a given user and requires no LLM calls or gradient steps; step 2 is an O(N)O(N) embedding comparison; step 3 is a free weight merge; step 4 is standard generation. The parametric channel is O(1)O(1) in the number of past sessions: a constant-size state, a constant-cost update per session, and a constant-size adapter.

Training

CLAM is trained by context distillation: a teacher model answers each query with the full session history in context, and the student—the frozen base wearing only the generated adapter, without that history—must reproduce the teacher’s answer:

L(ϕ)=E(S,q,y)∼D[−log⁡pθ+Hϕ(S)(y∣q)](4)\mathcal{L}(\phi) = \mathbb{E}_{(S,q,y)\sim\mathcal{D}}\left[-\log p_{\theta+H_\phi(S)}(y\mid q)\right] \tag*{(4)}

a cross-entropy loss on teacher-generated answers with a small adapter-magnitude penalty. Only the hypernetwork parameters ϕ\phi are updated; the base model remains frozen. Training uses no retrieval, so the adapter alone must carry the answer—this forces the consolidation channel to be self-sufficient. Training proceeds in two stages. Stage 1 initialises from a released single-document-to-adapter checkpoint (Charakorn et al., [4]); the recurrent machinery (gate, self-attention, positional embedding) is added as no-ops that preserve the pre-trained behaviour. Stage 2 teaches accumulation and gating on multi-session sequences spanning up to several dozen sessions, using truncated backpropagation through time to bound activation memory while still training the recurrence over many steps.

Training Data Generation

No public dataset provides multi-session, memory-dependent training data at scale (LoCoMo has ∼10 conversations, LongMemEval a few hundred questions), so we build one with a fully automated six-stage pipeline (Figure 2), driven throughout by Llama-3.3-70B-Instruct (Dubey et al., [6]). It follows the LongMemEval (Wu et al., 2025) and LoCoMo (Maharana et al., [23]) recipes (planted evidence, distractor sessions, memory taxonomies) but is automated end-to-end.

Synthetic data pipeline

Figure 2. The synthetic data pipeline: personas condition causal event graphs with knowledge-updates; filter questions unanswerable without history.

The pipeline runs in six stages (Figure 2). (1) Personas: thousands of personas are sampled from a balanced, decorrelated attribute grid (industry, style, expertise, region, coherent age/life-stage, interests, plus a “seed twist” and cliche bans). (2) Event graphs: each persona conditions a temporal event graph of about two dozen causally linked events, several of them knowledge-updates that contradict an earlier fact, so a system must resolve to the latest value. (3) Sessions: the events are dramatized as multi-turn conversations that reveal facts naturally, never as lists, across diverse session types. (4) Distractors: inter-persona distractor sessions are interleaved to depths of over a hundred, so evidence is sparse within a long history. (5) Questions: at several history checkpoints we generate a balanced set of six memory-type questions, factual, preference, cross-session synthesis, knowledge-update, temporal, and abstention, with gold answers from a full-context teacher. (6) Memory filter: a must-require-memory test discards any question the base model can already answer with no history (keeping ∼82%), so the training signal is genuinely memory-dependent. An automated LLM-judge quality pass runs between stages.

Experimental Setup

Benchmarks

We evaluate on LoCoMo (Maharana et al., [23]) and LongMemEval-S (Wu et al., 2025), which are standard LLM memory evaluation benchmarks. On LoCoMo we use the 1,540 questions of the four answerable categories (the adversarial category is excluded). For LongMemEval-S we sample 30 questions per type (seed 0) and evaluate on the 173 answerable ones; its 7 abstention items are excluded, as LoCoMo’s adversarial category is. Because the parametric baselines train at test time, they are compared on a 180-question LoCoMo subset and on the same LongMemEval-S questions. Every system is run on the same three open base models: Gemma-2-2B (Gemma Team et al., [8]), Qwen3-4B (Yang et al., 2025) and Mistral-7B (Jiang et al., [16]). On both benchmarks the episodic buffer retrieves the same unit: a session split on turn boundaries into pieces of at most 1,800 characters.

Baselines

Agentic (token-space). Systems that store the history as text and retrieve it at question time: Mem0 (Chhikara et al., [5]), A-Mem (Xu et al., 2025), Generative Agents (Park et al., [26]), MemoryBank (Zhong et al., 2024) and SimpleMem (Liu et al., [20]), each run with its pub-

PropertyValue
Trajectories generated∼900K
Distinct personas23,984
History chunks per trajectory: median / range14 / 2–48
Messages per chunk: median / range7 / 1–25
History length (tokens): median / p905,457 / 13,322
Memory types (balanced)6
vs. LoCoMo (conversations)10
vs. LongMemEval-S (questions)500
SystemSingleMultiTemp.OpenOverallΔ\Delta [95% CI]SSu_uSSa_aPref.M-sessK-upd.Temp.OverallΔ\Delta [95% CI]
Base model: Gemma-2-2B
Mem050.645.419.629.241.8+6.0 [2.1, 10.3]46.420.020.025.934.57.125.6+14.0 [4.7, 23.3]
A-Mem32.234.415.626.028.7+19.0 [14.4, 23.4]34.556.720.011.124.125.028.9+10.4 [1.7, 19.1]
Generative Agents15.416.02.512.512.6+35.1 [31.4, 38.3]37.930.010.00.065.53.624.9+14.5 [5.8, 23.1]
MemoryBank11.912.13.714.610.4+37.3 [32.2, 42.4]51.760.023.311.110.310.728.3+11.0 [2.3, 19.7]
SimpleMem11.915.63.49.410.7+37.1 [33.1, 41.2]0.010.03.30.00.00.02.3+37.0 [29.5, 44.5]
CLAM (ours)58.638.727.449.047.7—62.160.016.725.951.717.939.3—
consolidation-only28.118.85.333.321.9+25.8 [22.9, 29.1]13.816.716.714.817.214.315.6+23.7 [15.6, 31.8]
episodic-only53.531.619.626.040.6+7.1 [5.7, 9.1]55.263.313.33.762.110.735.3+4.0 [−2.9, 11.6]
Base model: Qwen3-4B
Mem061.358.222.426.050.4+1.2 [−1.5, 4.0]75.953.350.040.762.114.349.7−6.4 [−16.2, 3.5]
A-Mem48.040.421.213.538.8+12.8 [8.2, 16.9]13.826.726.70.06.90.012.7+30.6 [22.0, 39.3]
Generative Agents14.911.74.46.211.6+40.0 [36.4, 43.4]24.136.76.73.744.83.620.2+23.1 [14.5, 31.2]
MemoryBank9.510.32.58.38.1+43.5 [40.1, 46.9]48.353.333.314.813.87.128.9+14.5 [5.8, 23.1]
SimpleMem1.825.213.419.89.7+41.9 [36.8, 47.1]34.563.360.07.424.110.734.1+9.2 [0.6, 17.9]
CLAM (ours)61.845.731.846.951.6—48.380.033.325.955.214.343.4—
consolidation-only27.718.85.935.422.0+29.6 [26.4, 33.0]13.813.320.022.210.310.715.0+28.3 [20.2, 36.4]
episodic-only51.631.622.732.340.6+11.0 [9.5, 12.6]51.766.740.011.162.114.341.6+1.7 [−5.2, 8.7]
Base model: Mistral-7B
Mem053.046.517.435.443.2+12.0 [8.1, 16.2]41.446.756.77.427.614.332.9+20.2 [11.0, 29.5]
A-Mem51.340.818.127.140.9+14.4 [8.0, 20.1]10.333.336.70.06.90.015.0+38.2 [29.5, 46.8]
Generative Agents13.515.62.814.611.7+43.6 [38.7, 48.1]24.146.730.00.065.57.129.5+23.7 [15.0, 31.8]
MemoryBank11.611.72.813.59.9+45.4 [40.1, 51.0]48.360.053.37.410.37.131.8+21.4 [12.7, 30.1]
SimpleMem12.819.96.918.813.2+42.1 [29.5, 53.2]10.320.030.00.00.03.611.0+41.9 [33.1, 50.0]
CLAM (ours)64.047.539.655.255.3—69.080.040.033.369.025.053.2—
consolidation-only28.819.910.337.523.8+31.5 [26.1, 36.3]17.213.323.37.417.214.315.6+37.6 [28.9, 46.2]
episodic-only59.044.726.829.247.7+7.5 [5.3, 9.9]62.166.750.011.162.125.046.8+6.4 [0.0, 12.7]

Table 1. Synthetic corpus statistics and scale.

lished pipeline.

Parametric (weight-space). Systems that write the history into parameters, our own setting. We reimplement TMEM Ren et al., [30] in both of its modes, built once per conversation (TMEM-once) (where the model is prompted to generate training data based on conversational history, agnostic of the current query) and rebuilt per question (TMEM-per-question), and TLM Hu et al., [14] (where the model is trained for each new query). We construct four strong baselines: Test-time LoRA, the same rank-8 adapter on the same projection fitted by SGD; FT-Ceiling, a full fine-tune of every weight under one protocol for all bases (AdamW Loschilo[v] and Hutter, [22]),

batch 4, 3% warmup, clipping 1.0, lr 5×10−55 \times10^{-5}, and 8 epochs on LoCoMo, 2 on LongMemEval-S); and D2L-Merge and D2L-Retrieve, the same released hypernetwork Charakorn et al., [4], with the first based on their proposed chunk concatenation mechanism, and the second involving retrieving first to reduce the tokens to be internalized.

Results

Memory at Zero Construction Cost Is Competitive

Claim. CLAM leads every agentic memory system on LoCoMo, and on LongMemEval-S on two of three bases, trailing only Mem0 on Qwen by a non-significant margin, while building its mem-

SystemLLM decode callsLLM tokensgrad. stepswall (s)LLM tokensLLM decode callswall (s)
Mem048.84.9×1054.9 \times10^{5}0200.67781.005.84
A-Mem464.16.2×1056.2 \times10^{5}01,554.51,3961.005.07
Generative Agents34.75.1×1045.1 \times10^{4}034.86881.004.94
MemoryBank28.23.0×1043.0 \times10^{4}059.79281.005.31
SimpleMem12.11.0×1051.0 \times10^{5}0109.216,0165.6212.30
Test-time LoRA🔥0017.714.5501.002.64
TLM🔥0062.625.2651.004.04
TMEM-once🔥5.92.9×1042.9 \times10^{4}31.0175.01141.004.03
FT-Ceiling🔥0047.249.5501.001.86
TMEM-per-question🔥107.55.2×1055.2 \times10^{5}315.0459.0731.004.03
CLAM (ours), consolidation only00010.3701.003.25
CLAM (ours)00010.31,5831.003.25

caption: tbl:4: Cost of each memory, on Qwen3-4B over LoCoMo: build per conversation, answer per question. 🔥 = gradient steps at test time; the red row rebuilds its memory per question, and its build columns give the cost of one rebuild.

:::

ory with no generative language-model calls, no gradient steps, and no generated tokens.

Table Table 2 reports accuracy and Table 4 the corresponding costs. Each row reports CLAM minus that system on paired items, with a 95% bootstrap interval. Figure 3 plots the cost–accuracy trade-off, on which CLAM is the most accurate system at zero construction cost. On LoCoMo CLAM leads the strongest agentic system (Mem0) by 6.0 on Gemma, 1.2 on Qwen, and 12.0 on Mistral; no other agentic system comes within 12 points on any base. On LongMemEval-S CLAM leads every agentic baseline on Gemma (by 10.4 to 37.0) and Mistral (by 20.2 to 41.9) and only trails Mem0 on Qwen (−6.4, not significant). That gap reflects a favourable interaction between Qwen’s strong agentic abilities and Mem0’s extraction pipeline, but it comes at 48.8 generative LLM calls and 4.9×1054.9 \times10^{5} LLM tokens per conversation, a budget CLAM never spends. The sub-category columns show that CLAM’s LoCoMo gains concentrate in temporal and open-domain questions—the categories whose evidence no retriever surfaces verbatim—showing efficacy of the parametric consolidation stream.

Figure 3. Pareto frontier of cost versus quality on both benchmarks (LoCoMo and LongMemEval-S).

Both Channels Are Necessary and Complementary

Claim. Neither channel alone reproduces the full system, and the two are complementary: the adapter carries information retrieval cannot supply.

Single-stream ablation. Block (a) of Table 5 runs each stream on its own. Adding the adapter on top of retrieval adds 7.1, 11.0, and 7.5 points on Gemma, Qwen, and Mistral; adding retrieval on top of the adapter adds 25.8 to 31.5 (all intervals conversation-clustered and clear of zero).

ConfigurationGQM
*aa Single stream vs. both streams
Consolidation only (no retrieval)21.922.023.8
Episodic only (no adapter, ref.)40.640.647.7
Both (CLAM)47.751.655.3
*bb Consolidation gate
Reset state (carried state discarded every session)16.717.224.1
Ungated write (gt≡1g_t \equiv1, overwrite)21.421.222.5
Learned gate (CLAM)21.922.023.8
*cc Is it the adapter, or this adapter?
No adapter (base priors, no memory of any kind)1.13.92.8
Untrained adapter (same shape, random weights)0.00.60.0
CLAM’s adapter20.621.718.3

Table 5. Ablations (LoCoMo, judge accuracy %): (a) one stream versus both, (b) the consolidation gate, (c) whether any adapter would do.

Oracle buffer. Figure 4 varies the buffer from degenerate to ideal. With an oracle buffer, which places the gold evidence in context, redundancy predicts no gain, yet the adapter adds +4.8 [2.2, 7.1] on Gemma and +5.0 [2.8, 7.2] on Qwen (both p<10−3p<10^{-3}); on Mistral, already at 79%, it ties (+0.1). The weights carry information retrieval cannot supply, cross-session integration, implicit patterns, accumulated preferences, rather than a better-indexed copy of it.

Figure 4. The adapter helps at every buffer quality (LoCoMo). Open markers are the buffer alone; stars add the consolidated adapter.

When retrieval fails. Figure 5 shows the converse: on the 36% of questions whose evidence the buffer misses, the adapter adds +12.4, +13.7 and +15.1 points, against +3.9, +9.7 and +3.2 where retrieval succeeds. The adapter’s contribution is largest exactly where the episodic channel breaks down, and it remains positive when the evidence is already in context, including under the oracle buffer: the consolidated weights help the model use the evidence it is given, not only recover what retrieval missed.

Figure 5. The adapter pays when retrieval fails (LoCoMo). Gain from adding the adapter, split by whether the buffer retrieved the gold evidence.

Controls. Discarding the carried state at every session costs 4.8–5.2 points on Gemma and Qwen (Table 5b), and a random adapter of the same shape scores 0.0–0.6% (c).

Parametric Consolidation Scales Where Test-Time Training Does Not

Claim. The full CLAM system leads every weight-space memory; its consolidation channel alone leads every rank-8 method that adapts once per conversation, without a gradient step, and holds its accuracy on long histories.

Table 3 evaluates each baseline as designed, without retrieval. The full system leads every method by 10.6 to 44.6 points on LoCoMo and 28.7 to 44.3 on LongMemEval-S. Like for like, the consolidation channel leads test-time LoRA, TLM, TMEM-once and both D2L variants on both benchmarks; only FT-Ceiling and TMEM-per-question, which rebuilds its memory for every question (315 gradient steps and 107.5 LLM calls per rebuild), lead it, and only on LoCoMo. Given the same episodic buffer, CLAM leads all five weight-space baselines on LoCoMo (+6.9 to +15.7).

SystemrankGQMΔ\Delta vs CLAMΔ\Delta vs consol.GQMΔ\Delta vs CLAMΔ\Delta vs consol.
D2L-Merge8 per session1.14.41.1+44.6 [40.5, 49.0]+18.0 [14.9, 21.2]0.61.21.2+44.3 [39.9, 48.7]+14.5 [11.4, 17.7]
D2L-Retrieve816.115.018.9+30.2 [24.1, 36.1]+3.5 [−0.8, 7.9]12.111.012.7+33.3 [28.5, 38.2]+3.5 [−2.0, 7.1]
Test-time LoRA 🔥81.121.78.9+36.3 [31.7, 40.9]+9.6 [5.5, 13.8]1.76.42.9+41.6 [37.2, 46.1]+11.8 [8.5, 15.0]
TLM🔥81.13.33.3+44.3 [40.1, 48.5]+17.6 [14.4, 21.0]1.27.512.1+38.3 [33.7, 43.0]+8.5 [5.0, 11.9]
TMEM-once🔥84.410.05.6+40.2 [35.9, 45.0]+13.5 [10.1, 17.4]4.69.28.1+38.0 [33.3, 42.6]+8.1 [4.6, 11.8]
TMEM-per-question🔥815.643.317.8+21.3 [16.2, 26.5]−5.4 [−9.3, −0.9]3.536.49.8+28.7 [24.1, 33.3]−1.2 [−5.2, 2.9]
FT-Ceiling🔥full33.338.936.7+10.6 [5.1, 16.2]−16.1 [−19.6, −12.3]5.27.56.4+38.9 [34.3, 43.5]+9.1 [5.8, 12.3]
CLAM consolidation-only820.621.718.3+26.7—15.615.015.6+29.9 [25.0, 34.5]—
CLAM (ours)848.341.750.6——39.343.453.2——

Table 3. Weight-space (parametric) memory, judge accuracy (%) from the adapter alone. 🔥 = trains at test time (takes gradient steps on the conversation before it can answer); unmarked rows are frozen. Δ\Delta is CLAM’s pooled advantage over that row, with a 95% bootstrap interval.

On LongMemEval-S’s 4.5×4.5 \times longer histories, FT-Ceiling falls to 5.2–7.5 while the consolidation channel holds 15.0–15.6 (+9.1 [5.8, 12.3]). Figure 6(a) locates the failure: FT-Ceiling loses 2.2 points per 10 sessions of evidence depth, while CLAM stays flat. In a controlled sweep on our synthetic validation set from 16 to 192 chunks (Figure 6(b)), the consolidation channel stays level, the adapter adds 6 to 11 points over the buffer at every length, and test-time training stays below 5%.

Figure 6. Accuracy as the history grows. (a) By distance between evidence and question (LongMemEval-S, no retrieval). (b) On the same set of questions, with growing history length.

Conclusion

Existing LLM memory systems consolidate experience only in token space. We introduced CLAM, which adds the missing half: a recurrent hypernetwork compiles a stream of sessions into a low-rank adapter in a single forward pass, with no gradient steps and no generation calls, while a gated latent state carries knowledge forward in fixed size and a lightweight retrieval channel supplies verbatim detail at inference. Across two benchmarks and three open base models the two channels prove complementary, and the memory is built at a cost no comparable system approaches. Weight-space consolidation is a practical second memory channel for language agents, and we hope it encourages work on memory that is internalised rather than re-read.

Limitations

CLAM is evaluated on two English conversational-memory benchmarks with base models of 2–7 B parameters; behavior at larger scale, in other languages, and on non-conversational histories such as documents or code remains to be established. The consolidated adapter is a fixed-size store, though better results can be expected from scaling the adapter rank and the hypernetwork’s parameter count. Finally, the hypernetwork is trained once per base model on synthetic multi-session data; transferring a trained hypernetwork to an unseen base model without retraining is outside the scope of this work.

Ethics Statement

This work uses two public benchmarks, LoCoMo and LongMemEval-S, together with a synthetic corpus generated from fictional personas. Both benchmarks, the open base models and the baseline implementations are used in accordance with their respective licences and intended research use; no human subjects were involved and no personal data was collected.

Further Details

Implementation Details

The context encoder is a Perceiver-style resampler of 9 cross-attention blocks with 8 latent slots per layer, a 4×4 \times feed-forward expansion, SiLU activations and RMSNorm. Keys and values are the session activations alone, so slots pool a session independently and exchange no information during a read; the only slot-to-slot mixing is the between-session consolidation block, an 8-head self-attention with pre- and post-LayerNorm and a 4×4 \times expansion, applied once per session boundary. A single-block decoder carries one output query per (layer, rank index) pair — L×rL \times r queries, 208 for a 26-layer base at r=8r=8 — and each decodes into one rank-1 component. The resulting adapter is rank-8 on down_proj in every block, α=r\alpha=r, dropout 0: 2.40 M / 3.54 M / 4.72 M parameters on Gemma / Qwen / Mistral, about 7 MB in half precision, plus a fixed learned bias that does not vary with the conversation. The head is zero-initialised, so an untrained hypernetwork is an exact no-op on the base model. The episodic buffer uses all-MiniLM-L6-v2 over session pieces of at most 1,800 characters, cosine top-kk with k=5k=5 and a 7K-token budget, re-sorted chronologically. At inference the adapter is merged into the base weights and the answer is a single greedy pass capped at 48 new tokens.

Training

Only the hypernetwork is trained, by context distillation against a teacher that answers with the full history in context. Stage 1 initialises from the released document-to-adapter checkpoint with the recurrence inserted as a no-op. Stage 2 trains the carried state on multi-session examples of 3–30 sessions with truncated backpropagation through time: cross-entropy on the gold answer plus a small penalty on adapter magnitude, AdamW at 1×10−51 \times10^{-5}, gradient accumulation 8 for an effective batch of 64, packed context and input lengths of 6144 tokens, and a checkpoint every 1,000 steps. Reported checkpoints are 25K Stage-2 steps for Gemma and 8K for Qwen and Mistral. Training runs on a single A100-80GB and is paid once per base model, never per conversation.

Evaluation

Judge. All reported accuracies use one grader, llama3.3-70b-instruct at temperature 0, with an identical prompt for every system and both benchmarks. The system prompt is:

You are a strict grader for a
question-answering benchmark over long
conversations. You are given a QUESTION,
the reference GOLD answer(s), and a
model PREDICTION. Decide whether the
PREDICTION is factually correct: it must
convey the same key fact(s) as the gold
answer. Ignore differences in phrasing,
extra correct detail, or verbosity. If
the prediction states it cannot answer /
no information, and the gold is a real
answer, it is incorrect. Respond with
EXACTLY one word: Yes or No.

and the user prompt is:

QUESTION: {q} / GOLD
ANSWER(S): {g_1} | {g_2} | ... /
PREDICTION: {p} / Is the prediction
factually correct and consistent with the
GOLD ANSWER(S)? Answer Yes or No.

Judge independence. Because the judge shares a model family with the data generator, we re-grade the same outputs with three other judges under the identical prompt: 2,880 judgments covering 360 items, 8 systems, both benchmarks and all three bases (Table 6, Figure 7). Agreement is high, and no CLAM-versus-baseline comparison changes sign and significance under any judge.

Second judgeAgreeκ\kappaSign keptSig. flips
Llama-3.1-405B (same family, 5.8× larger)94.7%0.86339/420
Qwen3-30B-A3B (independent family)94.1%0.84734/420
Qwen3-8B (independent family)89.7%0.72332/420

Table 6. Agreement with independent judges on the same 2,880 judgments. Sig. flips: CLAM-vs-baseline comparisons whose sign changes with significance.

Figure 7. Per-system accuracy under the paper’s judge vs. each second judge.

Metric. Alongside judge accuracy we report Cover-EM ([27, 31]). Let nrm⁡(⋅)\operatorname{nrm}(\cdot) be the standard answer normalisation (lowercasing, stripping punctuation and articles, collapsing whitespace), pp a prediction and G\mathcal{G} its set of gold answers. Cover-EM scores a prediction 1 if any gold string occurs in it as a substring:

CoverEM⁡(p,G)=max⁡g∈G1[nrm⁡(g)⊆nrm⁡(p)],(5)\operatorname{CoverEM}(p,\mathcal{G}) = \max_{g \in\mathcal{G}} \mathbf{1}\left[\operatorname{nrm}(g) \subseteq\operatorname{nrm}(p)\right], \tag*{(5)}

and the reported figure is its mean over questions. Because the criterion is containment, tokens the prediction adds beyond the gold neither help nor hurt: it is invariant to answer length, unlike token-F1, whose precision term falls as a prediction lengthens. This matters here because mean answer length varies by an order of magnitude across systems.

Further Results

Text-Grounded Evaluation

Tables 7 and 8 repeat the two main results tables under Cover-EM, with identical layout and identical statistical treatment. The ordering is broadly preserved, which indicates the headline result does not depend on the LLM grader.

Full-Context Teacher

Figure 8 compares CLAM with the Llama-3.3-70B teacher, which answers with the entire history in its context. On LoCoMo (≈24\approx24k tokens per history) the teacher scores 68.3; on LongMemEval-S (≈109\approx109k tokens) it falls to 33.5, below CLAM on every base. Five of the 173 LongMemEval-S teacher calls failed and are scored as wrong, so 33.5 is a lower bound (at most 36.4).

Figure 8. CLAM vs. the full-context teacher as the history grows (same 180 questions per benchmark).

Decoding Variance

Table 9 repeats the LoCoMo evaluation with sampling (temperature 0.7, top-pp 0.95, three seeds) on a 399-question stratified subset.

Episodic-Buffer Granularity

The episodic buffer retrieves session-level units: each session, split on turn boundaries into pieces of at most 1,800 characters and carrying its session’s date header. To test whether finer, sub-session episodes help, we vary only the buffer’s unit and keep the consolidation channel, k=5k=5, and the token budget fixed (Gemma-2-2B, LoCoMo, 180 questions). Coarser units work best (Table 10): finer pieces strip the surrounding turns that disambiguate references, and at a fixed kk they carry less evidence into the prompt.

Qualitative Examples

Figures 9–14 show one generated training trajectory for each of the six memory types, drawn verbatim from the training corpus. Each panel gives the persona the history was generated from, the number of sessions in that history, a few turns of the gold-evidence session and one distractor session (two gold-evidence sessions for cross-session synthesis); the remaining sessions are elided.

SystemLoCoMo SingleMultiTemp.OpenOverallΔ\Delta [95% CI]LongMemEval-S SSuSSaPref.M-sessK-upd.Temp.OverallΔ\Delta [95% CI]
Mem024.311.02.88.316.4+5.2 [3.4, 7.3]39.313.30.018.527.67.117.4+10.9 [3.7, 18.2]
A-Mem14.97.15.96.211.1+10.5 [7.9, 13.0]34.526.70.07.413.810.715.6+12.7 [6.1, 19.4]
Generative Agents5.33.51.93.14.1+17.5 [15.5, 19.1]34.516.70.00.051.73.617.9+10.4 [3.2, 17.1]
MemoryBank4.13.50.94.23.3+18.2 [16.1, 20.4]37.926.70.03.710.37.114.5+13.9 [8.4, 20.0]
SimpleMem3.73.51.22.13.1+18.5 [16.3, 20.3]0.00.00.00.00.03.60.6+27.7 [21.6, 34.3]
CLAM (ours)28.815.28.122.921.6—55.236.70.011.141.425.028.3—
consolidation-only4.68.52.815.65.6+16.0 [14.2, 17.7]17.26.70.014.80.017.99.2+19.1 [12.5, 25.8]
episodic-only28.411.32.59.418.6+2.9 [1.8, 4.1]48.350.00.00.044.810.726.0+2.3 [−2.7, 7.3]
Mem026.413.55.013.518.7+2.9 [0.3, 5.6]62.140.00.025.951.717.932.9−2.3 [−10.1, 5.5]
A-Mem26.310.64.75.217.5+4.1 [1.1, 7.4]13.813.30.00.03.417.98.1+22.5 [15.8, 29.1]
Generative Agents6.32.82.23.14.6+17.0 [14.9, 19.2]20.723.30.00.037.90.013.9+16.8 [10.1, 23.5]
MemoryBank3.53.50.62.12.8+18.8 [16.5, 21.2]41.430.00.011.113.87.117.3+13.3 [6.6, 19.9]
SimpleMem0.86.48.16.23.7+17.9 [15.5, 20.2]27.633.30.07.417.27.115.6+15.0 [8.9, 21.7]
CLAM (ours)29.213.19.321.921.6—41.450.00.018.548.325.030.6—
consolidation-only3.36.02.516.74.4+17.1 [15.0, 19.3]3.40.00.022.23.417.97.5+23.1 [16.5, 30.3]
episodic-only28.112.85.613.519.6+2.0 [0.7, 3.1]44.840.00.011.155.221.428.9+1.7 [−3.9, 7.2]
Mem023.610.63.716.716.6+4.0 [1.8, 6.2]34.526.70.03.724.132.120.2+15.6 [8.4, 22.8]
A-Mem26.48.24.010.417.3+3.3 [0.8, 5.3]17.220.00.00.06.910.79.2+26.6 [19.3, 33.8]
Generative Agents5.24.62.87.34.7+15.9 [14.1, 17.8]17.223.30.00.055.210.717.9+17.9 [11.3, 25.1]
MemoryBank4.24.31.610.44.1+16.5 [14.2, 19.4]37.930.00.03.710.310.715.6+20.2 [13.0, 27.5]
SimpleMem3.92.80.94.23.1+17.5 [14.1, 20.9]10.310.00.00.00.00.03.5+32.3 [25.6, 39.0]
CLAM (ours)29.612.45.616.720.6—51.750.00.022.258.632.135.8—
consolidation-only5.16.73.113.55.5+15.1 [12.3, 18.0]6.90.00.03.76.917.95.8+30.1 [23.4, 37.3]
episodic-only28.912.15.615.620.1+0.5 [−1.5, 2.4]58.650.00.07.455.217.931.8+4.0 [−1.0, 9.0]

Table 7. Table 7: Agentic memory under Cover-EM. Identical layout to Table 2, scored with Cover-EM [27, 31] — the normalised gold answer appears in the prediction—instead of the LLM judge. Unlike token-F1 this does not penalise answer length, so systems with different generation styles remain comparable.

SystemrankLoCoMo GQMΔ\Delta vs CLAMΔ\Delta vs consol.LongMemEval-S GQMΔ\Delta vs CLAMΔ\Delta vs consol.
D2L-Merge8 per session1.11.71.0+18.0 [15.3, 20.6]+5.7 [4.3, 7.3]0.62.33.5+29.5 [25.2, 33.7]+5.4 [2.9, 7.9]
D2L-Retrieve85.04.46.1+13.7 [10.7, 16.5]+1.5 [−0.2, 3.1]2.96.45.8+26.6 [22.5, 30.8]+2.5 [0.2, 5.0]
Test-time LoRA80.65.02.8+16.1 [13.9, 18.4]+3.9 [2.2, 5.8]2.32.30.0+30.1 [26.0, 34.1]+6.0 [3.7, 8.3]
TLM80.63.92.2+16.7 [14.3, 19.2]+4.4 [3.0, 6.0]1.73.53.5+28.7 [24.9, 32.8]+4.6 [2.5, 6.9]
TMEM-once82.26.72.2+15.2 [12.7, 17.8]+3.0 [1.6, 4.5]2.95.22.9+27.9 [24.1, 32.0]+3.9 [1.5, 6.4]
TMEM-per-question86.117.810.0+7.6 [5.0, 10.3]−4.6 [−6.9, −2.3]2.329.54.0+19.7 [15.6, 23.7]−4.4 [−7.5, −1.3]
FT-Ceilingfull10.610.612.2+7.8 [5.1, 10.7]−4.4 [−6.5, −2.3]1.73.52.9+28.9 [24.9, 32.9]+4.8 [2.5, 7.1]
CLAM consolidation-only87.26.76.1+12.2—9.27.55.8+24.1 [20.0, 28.3]—
CLAM (ours)820.020.016.7——28.330.635.8——

Table 8. Table 8: Weight-space memory under Cover-EM. Comparison shows delta over CLAM and consolidated channel only.

SystemGreedySampled, mean ± sd
CLAM, Gemma-2-2B47.646.5 ± 1.04
CLAM, Qwen3-4B49.149.7 ± 0.95
CLAM, Mistral-7B50.650.5 ± 0.77

tbl:9: Greedy vs. sampled decoding (LoCoMo, 3 seeds).

Buffer unitUnits / conversationAccuracy (%)
Session pieces, ≤1,800 chars (CLAM)6845.6
1,024-character chunks11840.6
512-character chunks18341.1

Table 10: Episodic-buffer granularity (Gemma-2-2B, LoCoMo, k=5k=5). Only the buffer’s unit changes.

Generated training trajectory: factual recall

Figure 9. Generated training trajectory: factual recall (persona_01276__ckpt14__q01). One gold session and one distractor session are shown; the remaining sessions are elided.

Generated training trajectory: preference inference

Figure 10. Generated training trajectory: preference inference (persona_00553__ckpt30__q02). One gold session and one distractor session are shown; the remaining sessions are elided.

Generated training trajectory: cross-session synthesis

Figure 11. Generated training trajectory: cross-session synthesis (persona_00427__ckpt30__q03). Two of the three gold sessions are shown; the remaining sessions are elided.

Generated training trajectory: knowledge update

Figure 12. Generated training trajectory: knowledge update (persona_02278__ckpt18__q00). One gold session and one distractor are shown; the remaining sessions are elided.

!Temporal reasoning training trajectory panel. Text: Temporal reasoning — 14 sessions in history · gold evidence in session 13. The answer is a date or an ordering, which is carried by the session metadata and by the position of a session in the stream, not by its content. Persona Rukmini Deshpande, 21, Pune, India – Research Assistant in the Department of Linguistics (academic; expertise 1/5); verbose formal register. Working on: Assisting a professor with a research project on the phonetics of Marathi dialects. Interests: Born and raised in a Marathi-speaking family in Pune; Has a pet parakeet named Sunanda... 2024-09-11 · session 13 [gold evidence]. user: That sounds really helpful. I’ll definitely look into it. By the way, I’ve been trying to find some inspiration for my project and I recently attended a workshop on language pedagogy and linguistic theory. It was really enlightening and I got to meet some interesting people. assistant: Workshops can be a great way to learn from others and get new ideas... 2024-01-24 · session 1 [distractor]. user: That’s really helpful, thank you. I’ve recently attended a workshop on corpus linguistics and it’s given me a lot of ideas for how to approach this project... assistant: Co-authoring a paper can be a great way to collaborate with others and share your research with a wider audience... Q When did the user attend a workshop on language pedagogy and linguistic theory? Gold The user attended a workshop on language pedagogy and linguistic theory before September 11, 2024. bbox=0.12,0.28,0.48,0.70

Figure 13. Generated training trajectory: temporal reasoning (persona_02015__ckpt14__q00). One gold session and one distractor session are shown; the remaining sessions are elided.

!Abstention training trajectory panel. Text: Abstention — 10 sessions in history · no gold session (unanswerable). Nothing in the history answers the question. The gold answer is an explicit refusal, so a system that confabulates a plausible answer is penalised. Persona Kasper Vanderstappen, 48, Antwerp, Belgium – Chief Operations Officer (business; expertise 5/5); terse technical register. Working on: Implementing a new supply chain management system across European operations. Interests: Has a collection of 300 antique door knockers from around the world; Competes in the annual Antwerp Bouldering League... 2024-06-15 · session 6 [nearest retrieved session—not evidence]. user: Compliance is key. I’ve been listening to some Belgian jazz while working on my reports, trying to clear my head. Helps me focus. user: ...I’ve been putting in long hours, trying to get everything sorted. Just started playing the saxophone to unwind, but it’s not as easy as it looks. assistant: ...The saxophone is a complex instrument, but with practice, you’ll get the hang of it... 2024-04-22 · session 4 [distractor]. user: Appreciate the help. Used to spend weekends foraging for wild mushrooms in the Belgian Ardennes, but had to cut back to one day a month due to increased work demands. Not much time for hobbies these days. assistant: It can be tough to balance work and personal life... Q What is the name of the user’s saxophone teacher? Gold I don’t have any information about that from our conversations. bbox=0.52,0.29,0.88,0.70

Figure 14. Generated training trajectory: abstention (persona_00144__ckpt10__q11). The nearest retrieved session and one distractor session are shown; the remaining sessions are elided.

References

  1. [1]Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, T. Henighan, Andy Jones, Nicholas Joseph, B. Mann, Nova DasSarma, et al. 2021. A generation assistant as a laboratory for alignment. ArXiv:2112.00861.arxiv.org/abs/2112.00861
  2. [2]Ali Behrouz, Farnoosh Hashemi, Adel Javanmard, and Vahab Mirrokni. 2026. Language models need sleep: Learning to self-modify and consolidate memories. Preprint, arXiv:2606.03979.arxiv.org/abs/2606.03979
  3. [3]Rujikorn Charakorn, Edoardo Cetin, Yujin Tang, and Robert Tjarko Lange. 2025. Text-to-LoRA: Instant transformer adaption. In International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research. PMLR. ArXiv:2506.06105.arxiv.org/abs/2506.06105
  4. [4]Rujikorn Charakorn, Edoardo Cetin, Shinnosuke Uesaka, and Robert Tjarko Lange. 2026. Doc-to-LoRA: Learning to instantly internalize contexts. ArXiv:2602.15902.arxiv.org/abs/2602.15902
  5. [5]Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. 2025. Mem0: Building production-ready AI agents with scalable long-term memory. ArXiv:2504.19413.arxiv.org/abs/2504.19413
  6. [6]Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Ayesha Letman, Akhil Mathur, A. Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. ArXiv:2407.21783.arxiv.org/abs/2407.21783
  7. [7]Jizhan Fang, Xinle Deng, Haoming Xu, Ziyan Jiang, Yuqi Tang, Ziwen Xu, Shumin Deng, Yunzhi Yao, Mengru Wang, Shuofei Qiao, Huajun Chen, and Ningyu Zhang. 2025. LightMem: Lightweight and efficient memory-augmented generation. ArXiv:2510.18866.arxiv.org/abs/2510.18866
  8. [8]Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, B. Shahriari, Alexandre Ramé, Johan Ferret, et al. 2024. Gemma 2: Improving open language models at a practical size. ArXiv:2408.00118.
  9. [9]Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. REALM: Retrieval-augmented language model pre-training. In International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 3929–3938. PMLR. ArXiv:2002.08909.arxiv.org/abs/2002.08909
  10. [10]David Ha, Andrew M. Dai, and Quoc V. Le. 2017. Hypernetworks. In International Conference on Learning Representations. ArXiv:1609.09106.arxiv.org/abs/1609.09106
  11. [11]Moritz Hardt and Yu Sun. 2024. Test-time training on nearest neighbors for large language models. In International Conference on Learning Representations. ArXiv:2305.18466.arxiv.org/abs/2305.18466
  12. [12]Liliana Hotsko, Yinxi Li, Yuntian Deng, and Pengyu Nie. 2026. Code2LoRA: Hypernetwork-generated adapters for code language models under software evolution. Preprint, arXiv:2606.06492.arxiv.org/abs/2606.06492
  13. [13]Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations. ArXiv:2106.09685.
  14. [14]Jinwu Hu, Zhitian Zhang, Guohao Chen, Xutao Wen, Chao Shuai, Wei Luo, Bin Xiao, Yuanqing Li, and Mingkui Tang. 2025. Test-time learning for large language models. In International Conference on Machine Learning.arxiv.org/abs/2505.20633
  15. [15]Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Joao Carreira. 2021. Perceiver: General perception with iterative attention. In International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 4651–4664. PMLR. ArXiv:2103.03206.arxiv.org/abs/2103.03206
  16. [16]Albert Qiaochu Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7B. ArXiv:2310.06825.arxiv.org/abs/2310.06825
  17. [17]Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su. 2024. HippoRAG: Neurobiologically inspired long-term memory for large language models. In Advances in Neural Information Processing Systems 37, pages 59532–59569. Neural Information Processing Systems Foundation, Inc. (NeurIPS). ArXiv:2405.14831.arxiv.org/abs/2405.14831
  18. [18]Dharshan Kumaran, Demis Hassabis, and James L. McClelland. 2016. What learning systems do intelligent agents need? complementary learning systems theory updated. Trends in Cognitive Sciences, 20(7):512–534.DOI
  19. [19]Patrick Lewis, Ethan Perez, Aleksandra Piktus, F. Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Kuttler, M. Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Neural Information Processing Systems, pages 9459–9474. ArXiv:2005.11401.arxiv.org/abs/2005.11401
  20. [20]Jiaqi Liu, Yaofeng Su, Peng Xia, Siwei Han, Zeyu Zheng, Cihang Xie, Mingyu Ding, and Huaxiu Yao. 2026a. SimpleMem: Efficient lifelong memory for LLM agents. ArXiv:2601.02553.arxiv.org/abs/2601.02553
  21. [21]Yewei Liu, Xiyuan Wang, Yansheng Mao, Yoav Gelberg, Haggai Maron, and Muheng Zhang. 2026b. SHINE: A scalable in-context hypernetwork for mapping context to LoRA in a single pass. Preprint, arXiv:2602.06358.arxiv.org/abs/2602.06358
  22. [22]Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations. ArXiv:1711.05101.arxiv.org/abs/1711.05101
  23. [23]Adyasha Maharana, Dong-Won Lee, Sergey Tulaykov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. 2024. Evaluating very long-term conversational memory of LLM agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13851–13870, Bangkok, Thailand. Association for Computational Linguistics. ArXiv:2402.17753.arxiv.org/abs/2402.17753
  24. [24]James L. McClelland, B. McNaughton, and R. O’Reilly. 1995. Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. Psychological Review, 102(3):419–457.
  25. [25]Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph Gonzalez. 2023. MemGPT: Towards LLMs as operating systems. ArXiv:2310.08560.arxiv.org/abs/2310.08560
  26. [26]Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In ACM Symposium on User Interface Software and Technology, pages 1–22. ACM. ArXiv:2304.03442.arxiv.org/abs/2304.03442
  27. [27]Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, and Mike Lewis. 2023. Measuring and narrowing the compositionality gap in language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5687–5711. Association for Computational Linguistics. ArXiv:2210.03350.arxiv.org/abs/2210.03350
  28. [28]Preston Rasmussen, Pavlo Palivchuk, Travis Beauvais, Jack Ryan, and Daniel Chalef. 2025. Zep: A temporal knowledge graph architecture for agent memory. ArXiv:2501.13956.arxiv.org/abs/2501.13956
  29. [29]Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using siamese BERT-networks. In Conference on Empirical Methods in Natural Language Processing, pages 3980–3990. Association for Computational Linguistics. ArXiv:1908.10084.arxiv.org/abs/1908.10084
  30. [30]Tao Ren, Weiyao Luo, Hui Yang, Rongzhi Zhu, Xiang Huang, Yuchuan Wu, Bingxue Chou, Jieping Ye, Jiafeng Liang, Yongbin Li, and Yijie Peng. 2026. Scaling self-evolving agents via parametric memory. ArXiv:2606.04536.arxiv.org/abs/2606.04536
  31. [31]Corby Rosset, Chenyang Xiong, Minh Phan, Xia Song, Paul Bennett, and Saurabh Tiwary. 2020. Knowledge-aware language model pretraining. ArXiv:2007.00655.arxiv.org/abs/2007.00655
  32. [32]Charlie Snell, Dan Klein, and Ruiqi Zhong. 2022. Learning by distilling context. ArXiv:2209.15189.arxiv.org/abs/2209.15189
  33. [33]Larry R. Squire. 1992. Memory and the hippocampus: a synthesis from findings with rats, monkeys, and humans. Psychological Review, 99(2):195–231.
  34. [34]Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei A. Efros, and Moritz Hardt. 2020. Test-time training with self-supervision for generalization under distribution shifts. In International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 9292–9248. PMLR. ArXiv:1909.13231.arxiv.org/abs/1909.13231

Paper details

vlnc:W_zdn3csm3
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents