betaFeedback

WorksComputer Science

From Reflection to Co-Evolution: Self-Evolving Agents with Locally Verified Memory

Abstract

LLM agents have made impressive progress in coding and mathematical reasoning, motivating growing interest in learning from long-horizon interaction. Memory supports this process, but plausible lessons can be redundant or reinforce inappropriate actions. Can a memory's source decision provide a practical test for whether to retain it? We localize consequential trajectory steps, extract candidate memories, and replay each source decision with and without its candidate under the same context. This reveals two properties: most candidates barely shift the probed distribution, and both successful and failed source actions can be reinforced or suppressed. These observations motivate Anchor, which tests shift magnitude and source-role alignment before retention. Admitted memories guide later trajectories, enabling experience-driven improvement under a frozen policy. On held-out ALFWorld and WebShop tasks, Anchor improves success rate and mean score by up to 19.9% and 17.2%, respectively, relative to the strongest baseline for each backbone. With Qwen3.5-27B, it retains 41% fewer ALFWorld cards than unfiltered retention and outperforms all four WebShop cost-comparison baselines while using fewer tokens.

Introduction

Large language model (LLM) applications span coding ([27, 44]), mathematical and logical reasoning ([23, 57 i]), scientific investigation ([50, 8]), and interactive cognitive training ([26]). Beyond individual tasks, recent work asks how agents can improve through long-horizon interaction and reuse experience ([35, 28]). Memory provides a direct mechanism: Reflexion, ExpeL, and ReasoningBank extract reusable lessons from trajectories without updating model parameters ([28, 62, 18]).

Choosing which lessons to retain raises two connected problems. First, textual plausibility and successful retrieval do not establish a memory’s decision value: a lesson may repeat what the policy already knows or reinforce inappropriate action. Aggregate task success also leaves the contribution of an individual memory unresolved ([32]). Second, continual writing increases storage, context, and maintenance costs, so retaining ineffective lessons makes future reuse more expensive ([9]). Recent methods use action changes and trajectory feedback to guide selection ([59, 36]). We seek a candidate-level test that connects retention to behavioral evidence before a memory enters the bank:

💡 Can a memory’s source decision provide a practical test for whether to retain it?

We let the agent localize consequential decisions in completed trajectories and extract a memory from each. Replaying each source decision with and without its candidate under the same context reveals how that memory changes the policy (§ 3).

This replay exposes two properties of self-extracted memory. Memory influence is concentrated: most candidates barely shift the probed distribution, while a smaller subset produces substantial changes (Figure 2). Memory influence is directional: both successful and failed source actions can be reinforced or suppressed after memory injection (Figure 3). A large shift alone therefore does not identify whether the effect agrees with the source role. Conversely, a success memory can strengthen an action without changing the top choice. These observations motivate testing both the magnitude of a memory’s influence and its source-role alignment.

Histogram of action-verb probe JS divergence for 2,785 candidate memories

Figure 2. Shift magnitude. Action-verb probe JS divergence for 2,785 candidate memories. Most shifts cluster near zero; a smaller tail moves the probed distribution more strongly.

Shift direction scatter plot of source credit versus source-action change

Figure 3. Shift direction. Display-normalized source credit ctc_t versus source-action change Δpt(1)\Delta p_t^{(1)}, preserving signs. Role-aligned: success reinforcement (+,+)(+, +) and failure suppression (−,−)(-, -). Misaligned: success suppression (+,−)(+, -) and failure reinforcement (−,+)(-, +).

Figure 1. Memory can change the next action. In this schematic kitchen example, putting a dirty reusable plate in the sink is preferable to the cupboard or trash bin. Anchor screens for a role-aligned change.

Our approach. We introduce Anchor, which tests candidates before retention. Localizer extracts lessons from source decisions; Amplituder measures JS shift; Aspecter checks source-role alignment; and BankMaster stores, retrieves, and maintains admitted cards. Selected memories guide later trajectories, whose feedback supplies new candidates. This feedback loop between behavior and memory is the co-evolution studied here; model parameters remain frozen. Our analysis connects shift magnitude and direction to local action value under explicit conditions.

Empirical findings. On ALFWorld ([29]) and WebShop ([48]), Anchor ranks first across two frozen backbones. With Qwen3.5-27B, it reaches 96.27% ALFWorld success and 73.50 WebShop score, versus 92.54% and 62.74 for the strongest respective baselines. Source-role filtering improves ALFWorld success by 28.6% relative to magnitude-only admission, with 41% fewer ALFWorld cards than unfiltered retention. On WebShop with Qwen3.5-27B, token use is below all four cost-comparison baselines: slightly below MEM1 and approximately 34%, 59%, and 98.5% below ReasoningBank, Reflexion, and AdaMEM, respectively.

Related Work

Learning from interaction. Reflexion and ExpeL extract verbal lessons, Voyager accumulates skills, and CLIN maintains causal abstractions without parameter updates ([28, 62, 35, 17]). ReasoningBank distills strategies, while Evo-Memory and MemEvolve study learning across task streams ([18, 39, 52]). AdaMEM uses trajectory success and action changes to select strategy-training samples; MemRL learns memory utilities from feedback ([59 e]). Anchor instead tests individual candidates at their source decisions before storage.

Memory organization and admission. MemGPT, Mem0, A-MEM, and SimpleMem manage, consolidate, link, or compress persistent information ([19, 7, 43, 15]). LightMem targets consolidation cost ([9]). For selective writes, A-MAC scores candidate attributes, ConsistencyGate checks source support, and Janus compares memories on an evaluation set ([53, 58, 5]). MemGuard uses trajectory-verifier metadata for admission and maintenance ([36]). Anchor adds a paired behavioral test of shift magnitude and source-role alignment.

Decision-aware memory. EPR uses model likelihoods to supervise demonstration retrieval, and CICL selects and compresses context using action-choice effects ([24 Guan et al.,] 2026). Counterfactual auditing of robot memory distinguishes behavioral influence from appropriate guidance [54]. Memento-II formalizes reflective reads and writes as stateful policy improvement [37]. Our focus is persistent admission through source-decision replay; the value analysis connects the resulting distribution shift to local return. Appendix § 9 provides the extended comparison.

Observation: Magnitude and Direction of Memory Effects

Action Shift

A fixed policy πθ\pi_\theta executes task gg, observing oto_t, choosing at∈Ata_t \in\mathcal{A}_t, and receiving feedback rtr_t. Its trajectory and pre-action history are

τ=(g,o0,a0,r0,…,oT−1,aT−1,rT−1,oT),Ht=(g,o0,a0,r0,…,ot).(1)\begin{aligned} \tau&= (g, o_0, a_0, r_0, \ldots, o_{T-1}, a_{T-1}, r_{T-1}, o_T),\\ H_t &= (g, o_0, a_0, r_0, \ldots, o_t). \tag*{(1)} \end{aligned}

After task completion, extractor EE uses the trajectory and feedback to identify consequential steps K(τ)K(\tau) and extract their memories:

M(τ)={mt:mt=E(τ,t), t∈K(τ)}.(2)\mathcal{M}(\tau) = \{m_t : m_t = E(\tau,t),\ t \in K(\tau)\}. \tag*{(2)}

Each memory records reusable guidance together with its source task and state.

Let B\mathcal{B} be the bank before adding these memories, R(B,Ht)\mathcal{R}(\mathcal{B}, H_t) the saved source retrieval, and FF the context builder. The original and injected contexts are

ht=F(Ht,At,R(B,Ht)),htmt=F(Ht,At,R(B,Ht),mt).(3)\begin{aligned} h_t &= F(H_t,\mathcal{A}_t,\mathcal{R}(\mathcal{B},H_t)),\\ h_t^{m_t} &= F(H_t,\mathcal{A}_t,\mathcal{R}(\mathcal{B},H_t),m_t). \tag*{(3)} \end{aligned}

Normalize action log probabilities ℓt(a)\ell_t(a) and ℓtmt(a)\ell_t^{m_t}(a) from the two contexts over the same admissible set:

Pt(a)=exp⁡ℓt(a)∑a′∈Atexp⁡ℓt(a′),Ptmt(a)=exp⁡ℓtmt(a)∑a′∈Atexp⁡ℓtmt(a′).(4)\begin{aligned} P_t(a) &= \frac{\exp\ell_t(a)}{\sum_{a'\in\mathcal{A}_t}\exp\ell_t(a')},\\ P_t^{m_t}(a) &= \frac{\exp\ell_t^{m_t}(a)}{\sum_{a'\in\mathcal{A}_t}\exp\ell_t^{m_t}(a')}. \tag*{(4)} \end{aligned}

We use base-2 Jensen–Shannon (JS) divergence to measure the resulting action-distribution shift:

P‾t=12(Pt+Ptmt),Mt=DJS(Pt,Ptmt)=12DKL(Pt∥P‾t)+12DKL(Ptmt∥P‾t).(5)\begin{aligned} \overline{P}_t &= \frac{1}{2}(P_t + P_t^{m_t}),\\ M_t &= D_{\mathrm{JS}}(P_t,P_t^{m_t})\\ &= \frac{1}{2}D_{\mathrm{KL}}(P_t\Vert\overline{P}_t) + \frac{1}{2}D_{\mathrm{KL}}(P_t^{m_t}\Vert\overline{P}_t). \tag*{(5)} \end{aligned}

Across 2,785 ALFWorld candidates, action-verb probe JS concentrates near zero, with a small tail

of larger shifts (Figure 2). Most candidates barely change action-type preference. This concentration motivates Amplituder (§ 4.3): screen candidates by replay shift to retain memories with a measurable influence on the policy.

Direction

Magnitude leaves open which actions gain probability. Localizer assigns hindsight credit ct∈[−1,1]c_t \in[-1,1] and role yt=sign⁡(ct)∈{+1,−1}y_t = \operatorname{sign}(c_t) \in\{+1,-1\}: success maintain or failure correction. We track the recorded source action:

at(1):=at,a_t^{(1)} := a_t,
Δpt(1)=Ptmt(at(1))−Pt(at(1)).(6)\Delta p_t^{(1)} = P_t^{m_t}(a_t^{(1)}) - P_t(a_t^{(1)}). \tag*{(6)}

Positive Δpt(1)\Delta p_t^{(1)} reinforces the source action; negative values suppress it. Under the same ALFWorld setup, Figure 3 exhibits all four sign combinations.

The pairs (ct,Δpt(1))(c_t,\Delta p_t^{(1)}) distinguish success reinforcement (+,+)(+,+), success suppression (+,−)(+,-), failure suppression (−,−)(-,-), and failure reinforcement (−,+)(-,+). The test ytΔpt(1)>0y_t\Delta p_t^{(1)} > 0 selects the two role-aligned cases. Effects in all four directions motivate Aspector (§ 4.4): check source-role alignment alongside magnitude, using probability changes and rankings to distinguish support for successful actions from correction of failed ones. The plot tracks source-action verbs (L=1L=1).

A Two-Axis View of Memory Effects

We retain PtP_t, PtmtP_t^{m_t}, and MtM_t from § 3.1 and analyze their complete-action form on ata_t. Define the displacement νt(a)=Ptmt(a)−Pt(a)\nu_t(a) = P_t^{m_t}(a) - P_t(a). Whereas Memento-II formalizes memory reads and writes as stateful policy improvement [37], our analysis relates magnitude and direction to the value of changing context at one decision.

Source-action consistency. The bounds track the recorded action at(1)=ata_t^{(1)} = a_t. Interpreting it as the baseline top choice additionally assumes at=arg max⁡a∈Atℓt(a)a_t = \operatorname{arg\,max}_{a\in\mathcal{A}_t}\ell_t(a) at the complete-action level; token-wise greedy decoding alone does not establish this equality.

Magnitude controls effect capacity. Let Qt(a)Q_t(a) be the expected remaining return after action aa under a fixed continuation policy, and write ER[Qt]=∑aR(a)Qt(a)\mathbb{E}_{R}[Q_t] = \sum_a R(a)Q_t(a). The change in expected QQ-value and the action-value range are

EPtmem[Qt]−EPt[Qt]=∑avt(a)Qt(a),\mathbb{E}_{P_t^{\mathrm{mem}}}[Q_t]-\mathbb{E}_{P_t}[Q_t] = \sum_a v_t(a)Q_t(a),
Bt=max⁡a∈AtQt(a)−min⁡a∈AtQt(a).(7)B_t = \max_{a\in\mathcal{A}_t} Q_t(a)-\min_{a\in\mathcal{A}_t} Q_t(a). \tag*{(7)}

Proposition 1 (Radius bound). Let αt=TV⁡(Pt,Ptmem)\alpha_t = \operatorname{TV}(P_t,P_t^{\mathrm{mem}}). For the base-2 JS divergence MtM_t,

∣∑avt(a)Qt(a)∣≤Btαt≤Bt2ln⁡2 Mt.(8)\begin{aligned} \left|\sum_a v_t(a)Q_t(a)\right| \le B_t\alpha_t \\ &\le B_t\sqrt{2\ln2\,M_t}. \tag*{(8)} \end{aligned}

Near-zero complete-action divergence limits the possible expected-value change; the direction of mass transfer determines its sign.

Direction determines the sign. Write vtv_t as probability mass transported from actions that lose mass to actions that gain it:

∑avt(a)Qt(a)=∑a,bμt(a,b)[Qt(b)−Qt(a)].(9)\sum_a v_t(a)Q_t(a)=\sum_{a,b}\mu_t(a,b)[Q_t(b)-Q_t(a)]. \tag*{(9)}

where μt(a,b)\mu_t(a,b) transports mass from aa to bb, with total mass αt\alpha_t (Appendix B.2). Equal JS divergences can yield opposite value changes, depending on where this mass moves.

Using the source action at(1)a_t^{(1)} and role yty_t from § 3.2, define the signed source-role margin

ηt=ytΔpt(1)=ytvt(at(1)).(10)\eta_t = y_t\Delta p_t^{(1)} = y_tv_t(a_t^{(1)}). \tag*{(10)}

Positive ηt\eta_t corresponds to success reinforcement or failure suppression.

Assumption 1 (Value alignment). The source-role margin satisfies ηt≥0\eta_t \ge0. There exists κt≥0\kappa_t \ge0 such that every positive-mass transfer into at(1)a_t^{(1)} when yt=+1y_t=+1, or out of at(1)a_t^{(1)} when yt=−1y_t=-1, satisfies Qt(b)−Qt(a)≥κtQ_t(b)-Q_t(a)\ge\kappa_t.

Proposition 2 (Directional value bound). Under Assumption 1, the change in expected QQ-value satisfies

∑avt(a)Qt(a)≥κtηt⏟aligned gain−Bt(αt−ηt)⏟remaining loss bound.(11)\sum_a v_t(a)Q_t(a)\ge\underbrace{\kappa_t\eta_t}_{\text{aligned gain}}-\underbrace{B_t(\alpha_t-\eta_t)}_{\text{remaining loss bound}}. \tag*{(11)}

Memory injection increases the expected QQ-value whenever κtηt>Bt(αt−ηt)\kappa_t\eta_t>B_t(\alpha_t-\eta_t).

The source-role-aligned transfers have mass ηt\eta_t and gain at least κtηt\kappa_t\eta_t in value; the remaining mass αt−ηt\alpha_t-\eta_t loses at most BtB_t per unit. Thus, local value improves when the aligned gain exceeds the remaining loss bound. Appendix B.3 gives the proof.

These bounds explain magnitude and direction for complete-action distributions. The implementation uses prefix likelihoods and hindsight roles as admission signals (Section 4); it tests probe response, not the unobserved QQ-value conditions. Held-out task performance evaluates the retained memories’ utility.

Transfer to future decisions. Let P=(Pt)P=(P_t) and Pmem=(Ptmem)P^{\mathrm{mem}}=(P_t^{\mathrm{mem}}) denote the baseline and memory-conditioned policies over TT decisions, under the same initial distribution and environment dynamics. Here Ptmem(⋅∣H)P_t^{\mathrm{mem}}(\cdot\mid H) includes memories retrieved at history HH, extending the single-memory distribution PtmemP_t^{\mathrm{mem}}. With QtP(H,a)Q_t^P(H,a) denoting the remaining return after action aa followed by PP, extend the displacement to each history:

vt(H,a)=Ptmem(a∣H)−Pt(a∣H).(12)v_t(H,a)=P_t^{\mathrm{mem}}(a\mid H)-P_t(a\mid H). \tag*{(12)}

Writing JJ for expected task return, the performance-difference identity ((Schulman et al., 2017) gives

J(Pmem)−J(P)=Eτ∼Pmem[∑t=0T−1∑avt(Ht,a)QtP(Ht,a)].(13)J(P^{\mathrm{mem}})-J(P)=\mathbb{E}_{\tau\sim P^{\mathrm{mem}}}\left[\sum_{t=0}^{T-1}\sum_a v_t(H_t,a)Q_t^P(H_t,a)\right]. \tag*{(13)}

When the value-alignment condition yields a nonnegative lower bound in Eq. (11) at every history reached by PmemP^{\mathrm{mem}}, it follows that J(Pmem)≥J(P)J(P^{\mathrm{mem}})\ge J(P) (Appendix § 10.§ 4).

Methodology

Anchor: Write, Test, and Reuse

Following § 3.1–§ 3.2, Anchor tests how much a memory changes the policy and whether the change supports its source role. With πθ\pi_\theta frozen, Localizer extracts candidates, Amplituder and Aspecter test them, and BankMaster stores and reuses admitted cards (Figure 4). Let BiB_i precede construction trajectory ii. Admission decisions Gt∈{0,1}G_t\in\{0,1\} give the update

Figure 4. Anchor overview. Localizer extracts a fixed candidate from a source decision. Amplituder and Aspecter test its shift magnitude and source-role alignment. BankMaster stores admitted memories and retrieves them on later tasks. Kitchen actions and probability bars are schematic.

Mkeep(τ)={mt∈M(τ):Gt=1},\mathcal{M}_{\mathrm{keep}}(\tau)=\{m_t\in\mathcal{M}(\tau):G_t=1\},
Bi+1=Bmaster(Bi,Mkeep(τ),τ).(14)B_{i+1}=\mathcal{B}_{\mathrm{master}}(B_i,\mathcal{M}_{\mathrm{keep}}(\tau),\tau). \tag*{(14)}

Localizer: Memory Extraction

Localizer reflects on the completed trajectory and feedback, selecting consequential source steps with K=5K=5:

K(τ)⊆{0,…,T−1},∣K(τ)∣≤K.(15)K(\tau)\subseteq\{0,\ldots,T-1\},\qquad|K(\tau)|\le K. \tag*{(15)}

Each hint mtm_t receives model-assigned credit ct∈[−1,1]c_t\in[-1,1], whose sign yty_t identifies its source role; zero-credit candidates are excluded. Roles describe local decisions, not episode outcomes. Importance ranks candidates; hints remain fixed during replay and become cards only after admission.

Amplituder: Shift Magnitude

Section 3.1 shows that most candidates barely change the probed distribution. Amplituder therefore filters out negligible shifts before storage. It queries hth_t and htmth_t^{m_t} without executing an environment action, preserving the saved history, task guidance, retrieved cards, and candidate actions. Only the candidate-hint block differs; the resulting distributions yield MtM_t in Eq. (5).

Token-budgeted scoring. We sum log probabilities of the first LL action-content tokens after <action> and normalize over distinct prefixes, yielding P~t\widetilde{P}_t and P~tmt\widetilde{P}_t^{m_t}. ALFWorld uses L=1L=1; WebShop uses L=5L=5. Shared prefixes form one category; admission tracks the source category vt(1)=ϕL(at(1))v_t^{(1)}=\phi_L(a_t^{(1)}). Candidate construction, normalization, and complete-action prompts are detailed in Appendices C.4 and E.

Both roles must cross a magnitude threshold:

At+⟺Mt≥τsucc,A_t^+\Longleftrightarrow M_t\ge\tau_{\mathrm{succ}},
At−⟺Mt≥τfail.(16)A_t^-\Longleftrightarrow M_t\ge\tau_{\mathrm{fail}}. \tag*{(16)}

We use 0.05 and 0.12 bits, respectively. The lower success threshold accommodates source actions already strongly preferred before injection.

Aspecter: Source-Role Alignment

Section 3.2 motivates checking whether memory supports a successful source decision or corrects a failed one. Recall that at(1)a_t^{(1)} is the recorded source action and Δpt(1)\Delta p_t^{(1)} is its probability change after injection (Eq. (6)). The top choices before and after injection are

a^t0=arg⁡max⁡a∈At Pt(a),\widehat{a}_t^0=\underset{a\in\mathcal{A}_t}{\arg\max}\ P_t(a),
a^tmt=arg⁡max⁡a∈At Ptmt(a).(17)\widehat{a}_t^{m_t}=\underset{a\in\mathcal{A}_t}{\arg\max}\ P_t^{m_t}(a). \tag*{(17)}

The predicates below use complete-action notation; the implementation substitutes P~t\widetilde{P}_t, P~tmt\widetilde{P}_t^{m_t}, and source category vt(1)v_t^{(1)} on the fixed prefix set (Appendix D.2).

Success Maintain

The reinforcement condition StgainS_t^{\mathrm{gain}} requires the injected top choice to equal the source action and its probability to increase by at least ϵsucc\epsilon_{\mathrm{succ}}:

Stgain⟺(a^tmt=at(1))S_t^{\mathrm{gain}}\Longleftrightarrow(\widehat{a}_t^{m_t}=a_t^{(1)})
∧(Δpt(1)≥ϵsucc).(18)\land\left(\Delta p_t^{(1)}\ge\epsilon_{\mathrm{succ}}\right). \tag*{(18)}

Failure Corrections

The correction condition FtdirF_t^{\mathrm{dir}} accepts either source suppression by at least ϵdrop\epsilon_{\mathrm{drop}} or an injected preference for an alternative:

Ftdir⟺(Δpt(1)≤−ϵdrop)F_t^{\mathrm{dir}}\Longleftrightarrow\left(\Delta p_t^{(1)}\le-\epsilon_{\mathrm{drop}}\right)
∨(a^tmt≠at(1)).(19)\lor\left(\widehat{a}_t^{m_t}\ne a_t^{(1)}\right). \tag*{(19)}

The ranking branch tests alternative preference, which can hold even without a source-probability decrease; it is distinct from positive signed-margin alignment in Eq. (10).

Combined admission. Let VtV_t denote comparable probe scores and At+,At−A_t^{+}, A_t^{-} the magnitude conditions (Eq. (16)). Admission follows the source role:

yt=+1:Gt=1⟺Vt∧At+∧Stgain,y_t=+1:\quad G_t=1\Longleftrightarrow V_t\wedge A_t^{+}\wedge S_t^{\mathrm{gain}},
yt=−1:Gt=1⟺Vt∧At−∧Ftdir.(20)y_t=-1:\quad G_t=1\Longleftrightarrow V_t\wedge A_t^{-}\wedge F_t^{\mathrm{dir}}. \tag*{(20)}

Otherwise Gt=0G_t=0. Appendix § 11.§ 3 lists threshold values; Algorithm 2 implements the gate.

BankMaster: Retrieval and Maintenance

BankMaster stores each admitted hint verbatim as card guidance, adding only role, applicability, retrieval keys, and source metadata. Card construction preserves the tested text. It retrieves the highest-scoring compatible cards:

BiR(Ht)={m∈Bi:sret(m∣Ht,At)≥τR},\mathcal{B}^{R}_{i}(H_t)=\{m\in\mathcal{B}_i:s_{\mathrm{ret}}(m\mid H_t,A_t)\geq\tau_R\},
R(Bi,Ht)=TopK⁡m∈BiR(Ht)sret(m∣Ht,At).(21)\mathcal{R}(\mathcal{B}_i,H_t)=\operatorname{TopK}_{m\in\mathcal{B}^{R}_{i}(H_t)}s_{\mathrm{ret}}(m\mid H_t,A_t). \tag*{(21)}

Here srets_{\mathrm{ret}} combines task-family and state similarity with action compatibility. Insertion and feedback-driven maintenance update the bank:

Bi+=Insert⁡(Bi,{Card⁡(mt):Gt=1}),\mathcal{B}^{+}_i=\operatorname{Insert}(\mathcal{B}_i,\{\operatorname{Card}(m_t):G_t=1\}),
Bi+1=Maintain⁡(Bi+,Feedback⁡(τ)).(22)\mathcal{B}_{i+1}=\operatorname{Maintain}(\mathcal{B}^{+}_i,\operatorname{Feedback}(\tau)). \tag*{(22)}

Maintenance updates confidence, merges similar same-role cards, decays stale records, and prunes at capacity. Appendix § 11.§ 2 provides details. Held-out evaluation uses read-only retrieval.

Experiments

Our experiments address four research questions:

RQ1: Memory utility. Can the extracted memories improve held-out task performance across environments and backbones while retaining fewer memories than storing every candidate?

RQ2: Admission design. How do shift magnitude and source-role direction contribute to memory quality, and what is gained by combining success memories with failure corrections?

RQ3: Resource efficiency. How does Anchor balance task performance, persistent memory size, and total token use across memory construction and evaluation?

RQ4: Self-evolution. As experience accumulates along a task stream, does selective memory reuse improve the frozen agent’s performance beyond simply expanding its memory bank?

Experimental Settings

Backbones and baselines. We freeze either Qwen2.5-14B-Instruct (Qwen Team, [21]) or Qwen3.5-27B (Qwen Team, [22]) and compare ReAct (Yao et al., [49]), Mem0 (Chhikara et al., [7]), Reflexion (Shinn et al., [28]), MemP (Fang et al., [10]), LangMem (The LangChain Team, [34]), SimpleMem (Liu et al., [15]), ExpeL (Zhao et al., [62]), MEM1 (Zhou et al., 2025), AdaMEM (Zhang et al., [59]), and ReasoningBank (Ouyang et al., [18]). All methods share the evaluation split within each backbone. We use each baseline’s open-source benchmark implementation when available; otherwise, its memory mechanism uses the same environment and action-selection prompt as Anchor. Appendix § 11.§ 4 details adaptation.

Tasks and metrics. Memory construction uses 200 ALFWorld tasks and 400 WebShop tasks. Held-out evaluation uses 402 ALFWorld tasks and 4,055 WebShop instructions from a 10,000-product subset. ALFWorld reports success overall and by family; Overall pools successes across all tasks. WebShop reports mean task score and interaction steps. Evaluation is read-only: outcomes do not update the constructed banks. Appendix § 11.§ 3 gives task counts and implementation settings.

Main Results: Performance on Held-Out Tasks

Table 1 compares held-out performance across two environments, two frozen backbones, and ten baselines. Table 2 and Figure 5 connect these gains to selective retention and bank size.

MethodALFWorld Overall ↑Put ↑Clean ↑Heat ↑Cool ↑Examine ↑Puttwo ↑WebShop Score ↑Steps ↓
Qwen2.5-14B-Instruct
ReAct46.2733.3351.6126.0985.7172.225.8841.778.68
Conventional memory methods
Mem012.690.000.004.350.0088.890.0038.329.11
Reflexion67.1677.7855.9166.6774.6077.7852.9425.248.41
MemP56.7263.8963.4452.1750.7972.2231.3744.6116.21
LangMem36.5712.5032.2626.0971.4383.330.0036.6010.12
SimpleMem32.8423.6112.9046.3830.1662.9635.2945.467.31
ExpeL62.6941.6770.9773.9180.9572.2229.4153.928.71
Self-evolving memory methods
MEM130.3523.6123.6623.1928.5766.6725.4934.469.95
AdaMEM72.6473.6169.8969.5774.6068.5282.3536.759.96
ReasoningBank53.4845.8356.9946.3858.7355.5658.8259.054.84
Anchor (ours)87.0691.6788.1782.6188.8992.5976.4765.886.32
Qwen3.5-27B
ReAct61.1966.6751.6165.2276.1972.2235.2945.9710.64
Conventional memory methods
Mem011.940.000.000.000.0088.890.0052.029.18
Reflexion91.79100.0090.3295.6576.1988.89100.0048.1817.32
MemP57.7198.6181.721.4519.0561.1176.4759.8314.51
LangMem59.7047.2253.7644.9344.44100.0084.3154.858.90
SimpleMem92.5495.8389.2582.6195.24100.0096.0861.687.19
ExpeL82.8475.0090.3273.9195.2477.7882.3555.448.59
Self-evolving memory methods
MEM157.4650.0052.6943.4841.27100.0070.5948.289.47
AdaMEM65.6798.6133.3317.3976.19100.0094.1259.2549.00
ReasoningBank80.3579.1777.4276.8157.14100.00100.0062.746.61
Anchor (ours)96.27100.0090.3291.30100.00100.00100.0073.505.33

Table 1. RQ1 Held-out task performance. ALFWorld success (%) and WebShop mean score and steps. Bold and underline mark the best and second-best results per backbone; task-family ties use Overall.

Admission variantSuccess (%)
ReAct (no memory)61.19
Anchor w/o Amplituder & Aspecter73.38
Anchor w/o Aspecter74.88
Anchor w/o Failure Corrections86.82
Anchor w/o Success Maintain83.83
Anchor (ours)96.27

Table 2. RQ2 Overall ALFWorld admission ablation. Qwen3.5-27B success (%). Underline marks the best ablation.

Success rate, bank size, and token consumption with Qwen3.5-27B

Figure 5. RQ3 and RQ4 Success, bank size, and token consumption with Qwen3.5-27B. (Left) Sample success rate (cumulative; solid) over 200 randomly sampled ALFWorld memory-construction tasks and bank size (dashed) for Anchor and Anchor w/o Amplituder & Aspecter. (Right) WebShop mean score versus total input and output tokens across construction and evaluation (10x10^x scale). Circle area is proportional to mean interaction steps; lower, smaller circles indicate lower cost.

Answer 1. Selected memories improve held-out performance with a smaller bank. Anchor ranks first in all four environment–backbone combinations. On ALFWorld, it retains 41% fewer memories than writing every candidate while raising held-out success from 73.38% to 96.27% (Table 2; Figure 5). Thus, selective retention improves the usefulness of accumulated experience, not just its storage footprint.

ALFWorld task success. With Qwen2.5-14B, Anchor reaches 87.06% success, exceeding AdaMEM (72.64%) by 14.43 points (19.9% relatively). With Qwen3.5-27B, it reaches 96.27%, surpassing SimpleMem (92.54%) by 3.73 points. Differences are calculated before rounding. Gains span task families: Anchor leads five of six with Qwen2.5 and reaches at least 90.32% in every family with Qwen3.5.

WebShop score and interaction efficiency. Anchor scores 65.88 and 73.50 versus ReasoningBank’s 59.05 and 62.74, improving by 11.6% and 17.2% relatively. It also reduces Qwen3.5 mean steps from 6.61 to 5.33 and ranks second in Qwen2.5 step efficiency. Retained guidance improves outcomes without longer interactions.

Contributions of Magnitude and Direction

Table 2 uses Qwen3.5-27B on the same ALFWorld split. Anchor w/o Amplituder & Aspecter writes every candidate; Anchor w/o Aspecter uses magnitude alone. Role ablations keep one role with its tests intact. Appendix Table 3 gives family-level results.

Answer 2. Direction improves magnitude-only admission; retaining both roles performs best. Removing both tests yields 73.38%; magnitude filtering alone reaches 74.88%, a 1.49-point difference. Adding direction raises success to 96.27%, a further 21.39 points (28.6% relative). The larger gain comes from distinguishing source-role effects among candidates that pass magnitude screening. Keeping both roles adds 9.45 points over success memories alone and 12.44 over failure corrections alone. Thus, the strongest tested variant combines behavioral screening with both reinforcement and correction memories.

Resource Use

Answer 3. Selective retention improves the performance–resource trade-off. In Figure 5, Anchor stores 41% fewer ALFWorld memories while achieving higher success. On WebShop with Qwen3.5-27B, it leads the five plotted methods in score, mean steps, and token use (approximately 86.9M). Token use is slightly below MEM1 and approximately 34%, 59%, and 98.5% below ReasoningBank, Reflexion, and AdaMEM, respectively. These comparisons measure bank size, token consumption, and interaction steps. Token totals cover construction and evaluation, including reflection, replay, and maintenance (Appendix § 11.§ 4).

Self-Evolution Along the Task Stream

Answer 4. Selective accumulation improves construction-stream performance and held-out reuse. Figure 5 tracks cumulative success over 200 randomly sampled ALFWorld construction tasks. Anchor and Anchor w/o Amplituder & Aspecter share a frozen backbone but differ in admission. After early fluctuations, Anchor maintains higher cumulative success, finishing approximately 20% higher relatively despite slower bank growth. This curve compares behavior during memory construction; Table 2 tests the resulting banks on held-out tasks. Together, they connect selective experience accumulation to stronger subsequent reuse without parameter updates.

Conclusion

We introduced Anchor, which tests memories at their source decisions before retention. Paired replay measures shift magnitude and source-role alignment under a frozen policy. Across ALFWorld and WebShop, Anchor improves held-out performance; resource comparisons show smaller banks and lower token use. These results highlight memory curation as a practical link between reflection and behavioral improvement. By letting experience shape memory and selected memory guide subsequent behavior, Anchor offers a path toward experience-driven agent co-evolution.

Limitations

The scope of Anchor depends on the resolution of its behavioral probe and access to model probabilities.

Action-space requirements. Our long-horizon benchmarks provide explicit candidate actions. Prefix scoring cannot distinguish actions sharing the scored tokens, including some object-level errors. Open-ended settings require other consistent measures of memory-induced change, such as plan or response differences.

Access to model probabilities. Token log probabilities are required but unavailable through some closed-source interfaces. Model-generated scores, rankings, or repeated-output preferences offer alternatives. Future work can calibrate these measures and extend replay to broader tasks and interfaces.

The Usage of LLM

In accordance with ACL guidelines, we used large language models for writing assistance and language refinement.

References

Extended Related Work

Learning from interaction. LATS combines tree search, value estimation, and reflection for planning (Zhou et al., 2024). Reflexion stores verbal feedback, ExpeL extracts lessons, and Synapse reuses trajectories [28, 62, 63]; Voyager accumulates executable skills [35]. CLIN updates persistent causal abstractions across trials without parameter updates [17]. ReasoningBank distills strategies, Evo-Memory evaluates learning across task streams, and MemEvolve adapts experiential knowledge and its management [18, 39, 52]. AdaMEM combines long-term trajectories with within-episode strategies; its Step-MFT selects strategy-training samples using trajectory success and action changes [59]. MemRL learns memory utilities from execution feedback, while MACE co-evolves agents and memory graphs in multi-agent systems [55, 45]. Anchor uses source decisions and feedback roles to select candidates before storage.

Mem²Evolve couples experience distillation with the creation of tools and expert agents [6]; SkillRL co-evolves a hierarchical skill bank and the policy through reinforcement learning [42]. Anchor selects external lessons for retention under a fixed policy and action interface.

Memory organization and efficient reuse. Surveys characterize agent memory by its functions, representations, and evaluation [61]; CoALA connects modular memory with structured actions and decision-making [31]. Liu et al. [16] show that information position affects performance, motivating careful selection and organization of retrieved experience. MemGPT manages information beyond the context window, Generative Agents combines retrieval with reflection, and MemoryBank adds summarization and forgetting to conversational memory [19, 20] (Zhong et al. 2023). Mem0 consolidates facts, A-MEM links notes, and SimpleMem compresses memories [7, 43, 15]. LightMem addresses consolidation cost, while MemHarness reconstructs experience for the current state [9, 41]. MemBench, LongMemEval, and MemoryAgentBench test the effectiveness of sustained memory use [32, 13] (Wu et al. 2025). Anchor adds behavioral admission to this memory-management pipeline.

Reflective Memory Management combines multi-granularity construction and feedback-driven retrieval for dialogue [33]. Anchor places a paired source-decision test between extraction and storage, evaluating both construction-stream learning and read-only transfer.

Selective and reliable memory updates. Self-Refine iteratively revises outputs using model-generated feedback (Madaan et al., 2023), while CRITIC uses external tools for validation [11]. For persistent memory, A-MAC scores candidate attributes, AdaMem learns preference-conditioned writes, and D-MEM routes selected inputs into a slower evolution path [53, 3, 30]. ConsistencyGate checks source support, TRUSTMEM checks update properties, and Janus compares memories on a compact evaluation set [58, 46, 5]. MemGuard stores trajectory-verifier signals as metadata for admission and later maintenance [36]. Memory poisoning, state contamination, and persistent injection expose risks of unverified writes [4, 38, 47]. Anchor directly measures local action-distribution changes and source-role alignment, extending admission beyond content- and outcome-based criteria.

Repeated updates can degrade useful memories [51], motivating a distinction between initial admission and later maintenance. Anchor addresses admission through source-decision evidence.

Decision-aware memory. In online content evaluation, Zhang et al. [56] identify a gap between AI preference and real user engagement. For memory admission, we likewise prioritize behavioral evidence: paired replay measures how a candidate changes the agent’s decision distribution. Liu et al. [14] retrieve semantically similar examples to improve in-context learning. EPR uses model likelihoods to supervise demonstration retrieval, addressing context selection rather than persistent writes [24]. CICL selects and compresses context using its effects on action choices at retrieval time [12]. Counterfactual memory auditing of robot policies separates a memory’s behavioral influence from whether it guides actions appropriately [54]. Anchor uses paired probes at the source decision and success or failure roles to guide persistent memory admission.

Theoretical Analysis

Proof of Proposition 1

For any constant cc,

∑avt(a)Qt(a)=∑a[Ptmt(a)−Pt(a)][Qt(a)−c],(23)\begin{aligned} \sum_a v_t(a)Q_t(a) &= \sum_a [P_t^{mt}(a)-P_t(a)][Q_t(a)-c], \tag*{(23)} \end{aligned}

because both distributions have unit mass. Taking c=(max⁡aQt(a)+min⁡aQt(a))/2c=(\max_a Q_t(a)+\min_a Q_t(a))/2 gives

∣∑avt(a)Qt(a)∣≤Bt2∑a∣Ptmt(a)−Pt(a)∣=BtTV⁡(Pt,Ptmt)=Btαt.(24)\begin{aligned} \left|\sum_a v_t(a)Q_t(a)\right| &\leq\frac{B_t}{2}\sum_a\left|P_t^{mt}(a)-P_t(a)\right|\\ &=B_t\operatorname{TV}(P_t,P_t^{mt})=B_t\alpha_t. \tag*{(24)} \end{aligned}

Recall Pˉt=(Pt+Ptmt)/2\bar{P}_t=(P_t+P_t^{mt})/2. Pinsker’s inequality in natural logarithms gives

TV⁡(Pt,Pˉt)2≤12DKL(e)(Pt∥Pˉt),(25)\operatorname{TV}(P_t,\bar{P}_t)^2\leq\frac{1}{2}D_{\mathrm{KL}}^{(e)}(P_t\Vert\bar{P}_t), \tag*{(25)}

and the same inequality holds for PtmtP_t^{mt}. Since TV⁡(Pt,Pˉt)=TV⁡(Ptmt,Pˉt)=αt/2\operatorname{TV}(P_t,\bar{P}_t)=\operatorname{TV}(P_t^{mt},\bar{P}_t)=\alpha_t/2, averaging the two inequalities and converting nats to bits yields

αt2≤2ln⁡2 Mt.(26)\alpha_t^2\leq2\ln2\,M_t. \tag*{(26)}

Combining Eqs. (24) and (26) proves Proposition 1.

Probability Transport and Direction

Recall vt(a)=Ptmt(a)−Pt(a)v_t(a)=P_t^{mt}(a)-P_t(a) and αt=TV⁡(Pt,Ptmt)\alpha_t=\operatorname{TV}(P_t,P_t^{mt}). If αt>0\alpha_t>0, define

μt(a,b)=[−vt(a)]+[vt(b)]+αt.(27)\mu_t(a,b)=\frac{[-v_t(a)]_+[v_t(b)]_+}{\alpha_t}. \tag*{(27)}

Its row sums are [−vt(a)]+[-v_t(a)]_+, its column sums are [vt(b)]+[v_t(b)]_+, and its total mass is αt\alpha_t. Therefore

∑a,bμt(a,b)[Qt(b)−Qt(a)]=∑b[vt(b)]+Qt(b)−∑a[−vt(a)]+Qt(a)=∑avt(a)Qt(a),(28)\begin{aligned} \sum_{a,b}\mu_t(a,b)[Q_t(b)-Q_t(a)] &=\sum_b[v_t(b)]_+Q_t(b)\\ &\quad-\sum_a[-v_t(a)]_+Q_t(a)\\ &=\sum_a v_t(a)Q_t(a), \tag*{(28)} \end{aligned}

which proves Eq. (9). For αt=0\alpha_t = 0, set μt=0\mu_t = 0; both sides are zero.

This identity also establishes direction as independent of magnitude. Consider two memories mt+m_t^+ and mt−m_t^- with

Pt=(1/2,1/2),Ptmt+=(3/4,1/4),Ptmt−=(1/4,3/4),(29)\begin{aligned} P_t &= (1/2,1/2),\\ P_t^{m_t^+} &= (3/4,1/4),\\ P_t^{m_t^-} &= (1/4,3/4), \tag*{(29)} \end{aligned}

and Qt=(1,0)Q_t = (1,0). Symmetry gives DJS(Pt,Ptmt+)=DJS(Pt,Ptmt−)D_{\mathrm{JS}}(P_t,P_t^{m_t^+}) = D_{\mathrm{JS}}(P_t,P_t^{m_t^-}), while

EPtmt±[Qt]−EPt[Qt]=±1/4.(30)\mathbb{E}_{P_t^{m_t^\pm}}[Q_t] - \mathbb{E}_{P_t}[Q_t] = \pm1/4. \tag*{(30)}

Two memories can therefore have identical shift magnitude and opposite local value.

Proof of Proposition 2

Under Assumption 1, if yt=+1y_t = +1, then vt(at(1))=ηt≥0v_t(a_t^{(1)}) = \eta_t \ge0, so the transport column ending at at(1)a_t^{(1)} has mass exactly ηt\eta_t. Every positive-mass transfer in this column gains at least κt\kappa_t in value. If yt=−1y_t = -1, then vt(at(1))=−ηt≤0v_t(a_t^{(1)}) = -\eta_t \le0, so the transport row leaving at(1)a_t^{(1)} has mass exactly ηt\eta_t, and each of its positive-mass transfers again gains at least κt\kappa_t.

In either case, these transfers contribute at least κtηt\kappa_t\eta_t. The remaining transport has mass αt−ηt\alpha_t - \eta_t, and each unit can lose at most BtB_t. Applying Eq. (9) gives

∑avt(a)Qt(a)≥κtηt−Bt(αt−ηt),(31)\sum_a v_t(a)Q_t(a) \ge\kappa_t\eta_t - B_t(\alpha_t-\eta_t), \tag*{(31)}

When κtηt>Bt(αt−ηt)\kappa_t\eta_t > B_t(\alpha_t-\eta_t), the lower bound is strictly positive, proving Proposition 2. If every positive-mass transfer is value-aligned, each summand in Eq. (9) is nonnegative, and hence ∑avt(a)Qt(a)≥0\sum_a v_t(a)Q_t(a) \ge0.

From Local Value to Expected Return

Let PP and PmemP^{\mathrm{mem}} be history-dependent policies in the same finite-horizon environment, with bounded rewards, the same initial-history distribution, and terminal value VTP=0V_T^P = 0. Memory construction and retrieval may depend on the history; their resulting action distributions are included in the policies. Define

QtP(H,a)=E[rt+Vt+1P(Ht+1)∣Ht=H,at=a],VtP(H)=∑aPt(a∣H)QtP(H,a).(32)\begin{aligned} Q_t^P(H,a) &= \mathbb{E}\left[r_t + V_{t+1}^P(H_{t+1}) \mid H_t = H, a_t = a\right],\\ V_t^P(H) &= \sum_a P_t(a \mid H)Q_t^P(H,a). \tag*{(32)} \end{aligned}

Conditioning on Ht=HH_t = H under PmemP^{\mathrm{mem}} yields

EPmem[rt+Vt+1P(Ht+1)−VtP(Ht)∣Ht=H]=∑a[Ptmem(a∣H)−Pt(a∣H)]⋅QtP(H,a)=∑avt(H,a)QtP(H,a).(33)\begin{aligned} \mathbb{E}_{P^{\mathrm{mem}}}\left[r_t + V_{t+1}^P(H_{t+1}) - V_t^P(H_t) \mid H_t = H\right] &= \sum_a \left[P_t^{\mathrm{mem}}(a \mid H) - P_t(a \mid H)\right]\\ &\qquad{}\cdot Q_t^P(H,a)\\ &= \sum_a v_t(H,a)Q_t^P(H,a). \tag*{(33)} \end{aligned}

Summing over tt and taking expectations cancels the intermediate value terms. Since E[V0P(H0)]=J(P)\mathbb{E}[V_0^P(H_0)] = J(P), this proves Eq. (13).

At each history reached by PmemP^{\mathrm{mem}}, apply the value-alignment condition to Pt(⋅∣H)P_t(\cdot\mid H) and Ptmem(⋅∣H)P_t^{\mathrm{mem}}(\cdot\mid H), using a designated reference action and role and the continuation values QtP(H,⋅)Q_t^P(H,\cdot). With the corresponding quantities ηt(H)\eta_t(H), αt(H)\alpha_t(H), κt(H)\kappa_t(H), and value range Bt(H)B_t(H), let

bt(H)=κt(H)ηt(H)−Bt(H)[αt(H)−ηt(H)].(34)b_t(H) = \kappa_t(H)\eta_t(H) - B_t(H)[\alpha_t(H)-\eta_t(H)]. \tag*{(34)}

Applying Eq. (11) inside the expectation gives

J(Pmem)−J(P)≥EPmem[∑t=0T−1bt(Ht)].(35)J(P^{\mathrm{mem}})-J(P) \ge\mathbb{E}_{P^{\mathrm{mem}}}\left[\sum_{t=0}^{T-1} b_t(H_t)\right]. \tag*{(35)}

Thus nonnegative bt(H)b_t(H) at every reachable history implies nonnegative expected-return change. The source-state admission test evaluates local memory effects, while this identity connects effects across retrieved trajectories to task return.

Information-Theoretic Interpretation

For the information-theoretic and finite-sample calculations below, abbreviate p=Ptp=P_t and q=Ptmemq=P_t^{\mathrm{mem}} at a fixed step tt, so DJS(p,q)=MtD_{\mathrm{JS}}(p,q)=M_t. The magnitude test has an information interpretation. Draw a fair bit Z∈{0,1}Z \in\{0,1\}, then draw an action AA from pp when Z=0Z=0 and from qq when Z=1Z=1. Writing m=(p+q)/2m=(p+q)/2 and using base-2 entropy gives

I(Z;A)=H(m)−12H(p)−12H(q)=12DKL(p∥m)+12DKL(q∥m)=DJS(p,q).(36)\begin{aligned} I(Z;A) &= H(m)-\frac{1}{2}H(p)-\frac{1}{2}H(q)\\ &= \frac{1}{2}D_{\mathrm{KL}}(p\mathbin{\|}m)+\frac{1}{2}D_{\mathrm{KL}}(q\mathbin{\|}m)\\ &= D_{\mathrm{JS}}(p,q). \tag*{(36)} \end{aligned}

Thus JS measures how distinguishable the two policy conditions are from their actions; it does not say which actions are better. In particular, 0≤DJS(p,q)≤10 \le D_{\mathrm{JS}}(p,q) \le1 bit, irrespective of the action-set size.

Coarsening the action space. For a deterministic map gg from complete actions to action verbs, the Markov chain Z→A→g(A)Z \to A \to g(A) and the chain rule for mutual information yield

DJS(g#p,g#q)=I(Z;g(A))≤I(Z;A)=DJS(p,q).(37)\begin{aligned} D_{\mathrm{JS}}(g_{\#}p,g_{\#}q) &= I(Z;g(A))\\ &\le I(Z;A)=D_{\mathrm{JS}}(p,q). \tag*{(37)} \end{aligned}

The inequality may be strict: a memory can change which object is selected without changing the verb. This statement concerns the pushforward of the same complete-action distributions. The token scorer in Eq. (46) instead constructs its distribution directly from candidate-prefix likelihoods; Eq. (37) characterizes aggregation of complete-action probabilities.

Local sharpness of the radius bound. Take p=(1/2,1/2)p=(1/2,1/2) and q=(1/2+ε,1/2−ε)q=(1/2+\varepsilon,1/2-\varepsilon), with ∣ε∣<1/2|\varepsilon|<1/2. The binary entropy expansion h2(1/2+x)=1−2x2/ln⁡2+O(x4)h_2(1/2+x)=1-2x^2/\ln2+O(x^4) gives

DJS(p,q)=ε22ln⁡2+O(ε4),TV⁡(p,q)=∣ε∣.(38)\begin{aligned} D_{\mathrm{JS}}(p,q) &= \frac{\varepsilon^2}{2\ln2}+O(\varepsilon^4),\\ \operatorname{TV}(p,q) &= |\varepsilon|. \tag*{(38)} \end{aligned}

Consequently, the square-root dependence and leading constant in Eq. (8) are attained asymptotically for this binary perturbation. The bound controls capacity for value change; the transport argument in Appendix § 10.§ 2 is still needed for its sign.

Finite-Sample Resolution of Paired Probes

Suppose each of the two fixed policy conditions produces nn independent action draws on the same d≥2d \ge2 categories, with empirical distributions p^\widehat{p} and q^\widehat{q}. This subsection characterizes a sampling-based alternative to token-score probes; it is independent of the token-budget configuration used by the adapters. Hoeffding’s inequality and a union bound over the 2d2d empirical coordinates show that, with probability at least 1−δ1-\delta,

max⁡x∈{p,q}max⁡a∣x^(a)−x(a)∣≤en:=ln⁡(4d/δ)2n.(39)\max_{x \in\{p,q\}}\max_a \left|\widehat{x}(a)-x(a)\right| \le e_n := \sqrt{\frac{\ln(4d/\delta)}{2n}}. \tag*{(39)}

Hence each empirical distribution is within u=min⁡{1,den/2}u=\min\{1,de_n/2\} in total variation of its population distribution. For the tracked source category at(1)a_t^{(1)} and its role yt∈{+1,−1}y_t \in\{+1,-1\},

∣yt[q^(at(1))−p^(at(1))]−yt[q(at(1))−p(at(1))]∣≤2en.(40)\left|y_t\left[\widehat{q}(a_t^{(1)})-\widehat{p}(a_t^{(1)})\right]-y_t\left[q(a_t^{(1)})-p(a_t^{(1)})\right]\right| \le2e_n. \tag*{(40)}

The classical entropy-continuity inequality (Audenaert, 2007) gives ∣H(r)−H(s)∣≤fd(t)\left|H(r)-H(s)\right| \le f_d(t) when TV⁡(r,s)≤t\operatorname{TV}(r,s)\le t, where

fd(t)={h2(t)+tlog⁡2(d−1),t≤1−1/d,log⁡2d,t>1−1/d.(41)f_d(t)= \begin{cases} h_2(t)+t\log_2(d-1), & t\le1-1/d,\\ \log_2 d, & t>1-1/d. \end{cases} \tag*{(41)}

The mixtures (p+q)/2(p+q)/2 and (p^+q^)/2(\widehat{p}+\widehat{q})/2 are also within uu in total variation. Applying the entropy bound to the three terms of Eq. (36) yields

∣DJS(p^,q^)−DJS(p,q)∣≤2fd(u).(42)\left|D_{\mathrm{JS}}(\widehat{p},\widehat{q})-D_{\mathrm{JS}}(p,q)\right| \le2f_d(u). \tag*{(42)}

This bound certifies a population JS threshold when the measured value exceeds it by more than 2fd(u)2f_d(u); the corresponding signed-margin threshold requires a margin of more than 2en2e_n. These certificates cover the scalar magnitude and signed-margin tests, while source-role correctness and top-action switches use their corresponding criteria. Larger sample sizes tighten both certificates.

Experimental Details

Admission Ablation by Task Family

Table 3 expands the Overall ablation on the same task split. The row at 86.82% removes failure corrections and retains success-maintain memories; the row at 83.83% removes success-maintain memories and retains failure corrections. In both rows, the retained role still undergoes its magnitude and directional tests.

VariantOverallPutCleanHeatCoolExaminePuttwo
ReAct61.1966.6751.6165.2276.1972.2235.29
Anchor w/o Amplituder & Aspecter73.3877.7870.9768.1282.5485.1954.90
Anchor w/o Aspecter74.8881.9469.8973.9180.9583.3358.82
Anchor w/o Failure Corrections86.8291.6786.0279.7190.4892.5980.39
Anchor w/o Success Maintain83.8388.8982.8076.8192.0688.8972.55
Anchor (ours)96.27100.0090.3291.30100.00100.00100.00

Table 3. Full ALFWorld ablation. Qwen3.5-27B success (%) by task family.

BankMaster Details

BankMaster stores each admitted mtm_t verbatim in the card’s memory_guidance field and adds role, applicability conditions, retrieval keys, and source metadata. A success hint specifies an action to reinforce under matching conditions; a failure hint records a pattern to avoid and its immediate correction. Card construction enriches metadata without rewriting, summarizing, or otherwise changing the tested guidance.

The retrieval score in Eq. (21) combines task-family and state similarity with action compatibility. Up to three cards above threshold τR\tau_R enter the policy context; cards whose cues or recommended actions do not match the decision are ignored. Table 4 lists the retrieval threshold, cross-family settings, skip conditions, and card-context budget.

SettingValueSettingValue
Action decodingToken-wise greedy, 256-token limitProbe statisticToken-level action scores
Replay token budget LLALFWorld: 1; WebShop: 5Paired contextFixed within each pair
Failure JS threshold0.12 bitsSuccess JS threshold0.05 bits
Failure probability branchSource decrease ≥0.10\ge0.10Failure ranking branch (OR)Injected top action ≠\ne source
Success rankingInjected top action = sourceSuccess comparisonProbability increase
Success reinforcement≥0.04\ge0.04Localized candidatesK=5K=5 per episode
Maximum cards8 per episodeRetrieval sizeTop 3
Retrieval threshold0.45Cross-family retrievalboost 0.50, penalty 0.15
Retrieval skipconfidence ≥0.50\ge0.50 and entropy ≤1.40\le1.40 or margin ≥0.12\ge0.12Card context budget320 tokens per card
Bank capacity2,000 cardsMerge threshold0.85

Table 4. Implementation settings. Branch-specific thresholds use JS in bits. The measurement interface is described in Appendix § 11.§ 4.

Each card stores confidence, support count, source metadata, and reuse statistics. After an episode, BankMaster updates retrieved-card confidence from feedback, merges similar cards with the same role, decays stale records, and prunes low-confidence cards at capacity, as formalized in Eq. (22). During held-out evaluation, the constructed bank supplies cards without updates.

Implementation Parameters

ALFWorld data. We use the raw TextWorld/PDDL game files in awawa-agi/alfworld-raw, including training and evaluation game specifications. Task counts below refer to the selected experimental tasks.

ALFWorld evaluation set. The held-out ALFWorld set contains 402 tasks: 72 Put, 93 Clean, 69 Heat, 63 Cool, 54 Examine, and 51 Puttwo. Overall success pools all evaluated tasks. For task family ff, let sfs_f and nfn_f denote its success count and evaluation count. We report

Overall=100∑fsf∑fnf,Successf=100sfnf.(43)\begin{aligned} \mathrm{Overall} &= 100\frac{\sum_f s_f}{\sum_f n_f},\\ \mathrm{Success}_f &= 100\frac{s_f}{n_f}. \tag*{(43)} \end{aligned}

Thus Overall is the total number of successes divided by the total evaluation count, rather than an unweighted average of task-family percentages.

WebShop task split. We use the 10,000-product subset of YWZBrandon/webshop-data. Product-ID matching yields 4,455 attributed instructions (400 construction; 4,055 evaluation); 5,545 products have none. Counts refer to product-linked instructions, not unique strings or all product-option combinations.

WebShop evaluation metric. WebShop is evaluated by mean score, not success rate. Let siWSs_i^{\mathrm{WS}} denote the score of evaluation task ii on the reported score scale. Over NWS=4,055N_{\mathrm{WS}}=4{,}055 evaluation tasks, we compute

MeanScoreWS=1NWS∑i=1NWSsiWS.(44)\mathrm{MeanScore}_{\mathrm{WS}}=\frac{1}{N_{\mathrm{WS}}}\sum_{i=1}^{N_{\mathrm{WS}}}s_i^{\mathrm{WS}}. \tag*{(44)}

Unlike ALFWorld’s binary success indicator, this metric averages the task scores directly. Mean interaction steps are reported separately as an efficiency metric.

Admission Configuration and Measurement

Measurement interface. The two adapters share a token-budget parameter LL, counting action-content tokens immediately after <action>. The opening tag is context, not part of the scored span. ALFWorld scores the action head (L=1L = 1); WebShop scores up to five action tokens (L=5L = 5). Appendix § 13 provides one replay template per environment: the complete saved decision context, with its history, task guidance, and retrieved memory. Only the injected condition adds mtm_t. The scoring specification below applies to the paired measurements, including the case studies in Appendix § 14.

Candidate actions. At each source decision, store a finite ordered list CtC_t of legal candidate action strings, including the recorded source action. ALFWorld obtains this list from the TextWorld environment’s admissible_commands. WebShop obtains clickable products, option values, and navigation controls from the current page’s get_available_actions() interface, and serializes each as a complete click[...] command. Thus product selection and option selection are distinct candidates, not a single click category. The environment interfaces are documented in the ALFWorld and WebShop releases. When search is available, the frozen policy generates a finite list of query candidates from the current task and page before the paired test. The adapter serializes these as search[query] actions and stores the list with the source snapshot. Replay reuses this list and never generates new candidates from the injected hint or subsequent feedback. Remove duplicate complete strings while preserving their first occurrence.

Token scores and shared prefixes. To score candidates, tokenize each action continuation using the policy tokenizer and the exact saved prefix ending in <action>. Let z(a)=(z1,…,zn(a))z(a) = (z_1,\ldots,z_{n(a)}) be its action-content tokens, excluding the opening and closing tags, and let k(a)=min⁡{L,n(a)}k(a) = \min\{L,n(a)\}. Define ϕL(a)=z1:k(a)\phi_L(a) = z_{1:k(a)} and form the set of distinct scored sequences

Vt={ϕL(a):a∈Ct}.(45)\mathcal{V}_t = \{\phi_L(a): a \in C_t\}. \tag*{(45)}

Candidates with identical scored sequences share one category. This deduplication prevents a verb from gaining weight merely because it has more object-level completions. For each v=(v1,…,v∣v∣)∈Vtv = (v_1,\ldots,v_{\lvert v\rvert}) \in\mathcal{V}_t, teacher-force its tokens under each saved context h∈{ht,htmt}h \in\{h_t,h_t^{m_t}\} and sum their log probabilities:

ℓL(v∣h)=∑j=1∣v∣log⁡πθ(vj∣h,⟨action⟩,v<j),\ell_L(v \mid h) = \sum_{j=1}^{\lvert v\rvert} \log\pi_\theta(v_j \mid h,\langle\mathrm{action}\rangle,v_{<j}),
P~h(v)=exp⁡ℓL(v∣h)∑u∈Vtexp⁡ℓL(u∣h).(46)\widetilde{P}_h(v) = \frac{\exp\ell_L(v \mid h)}{\sum_{u \in\mathcal{V}_t}\exp\ell_L(u \mid h)}. \tag*{(46)}

We sum token log probabilities, using all content tokens for actions shorter than LL. Leading whitespace and token IDs follow the same saved serialization in both conditions. The resulting distribution is normalized over distinct candidate prefixes: actions sharing their first LL tokens form one category. With L=5L = 5, WebShop click targets occupy different categories when their tokenized arguments differ within the scored span.

The source category is vt(1)=ϕL(at(1))v_t^{(1)} = \phi_L(a_t^{(1)}). Compute JS and the source-category change using P~ht\widetilde{P}_{h_t} and P~htmt\widetilde{P}_{h_t^{m_t}} on exactly Vt\mathcal{V}_t. For example, if take apple 1 and take mug 2 share the same first token, L=1L = 1 assigns both to one category; that category appears only once in the softmax. A valid pair requires the saved context, source candidate, identical tokenization, and finite log probabilities for every scored token in both conditions. Missing top-kk scores invalidate the pair rather than being treated as zero. Resolve score ties by the stored candidate order.

Admission configuration. Both source roles pass a magnitude test and a directional test. We use τsucc=0.05\tau_{\mathrm{succ}} = 0.05 bits for success-maintain memories and τfail=0.12\tau_{\mathrm{fail}} = 0.12 bits for failure corrections. Success-maintain memories require the injected top category to equal the source category and its probability to increase by at least ϵsucc=0.04\epsilon_{\mathrm{succ}} = 0.04. Failure corrections require either a source-category decrease of at least 0.10 or an injected top category different from the failed source category. Algorithm 2 gives the complete predicates.

Baseline adaptation. Each comparison uses the backbone and evaluation split specified in Table 1. For a baseline whose open-source release includes the corresponding benchmark environment, we use that implementation. Otherwise, we adapt its memory mechanism to the same environment and action-selection prompt as Anchor, including the task guidance supplied to the policy. Each baseline retains its own memory extraction, organization, and retrieval procedure. These are complete-system comparisons; Table 2 isolates admission variants within the Anchor pipeline.

Token accounting. The WebShop panel in Figure 5 uses Qwen3.5-27B and counts input and output tokens across all model calls during both memory construction and held-out evaluation. For each system, let QQ be this complete call set. Total consumption is

T(Q)=∑q∈Q(nin(q)+nout(q)).(47)T(Q) = \sum_{q \in Q} \left(n_{\mathrm{in}}(q) + n_{\mathrm{out}}(q)\right). \tag*{(47)}

The total includes action selection, candidate-query generation, reflection and localization, hint writing, both replay conditions, card construction, and model-based maintenance wherever these calls occur in the system. Repeated history and retrieved-card tokens count on each call in which they occur; LL is the scoring span, not the total input length. Environment steps and local, non-model retrieval are not model tokens. Baselines are counted over the same construction-plus-evaluation scope, including their own memory-related model calls. The per-task labels divide this total by 4,055 evaluation tasks, amortizing construction cost across evaluation. Circle area is proportional to the mean evaluation steps reported in Table 1.

Source roles. The Localizer receives the goal, the recorded trajectory, and episode feedback, and autonomously assigns each selected source step a hindsight credit ct∈[−1,1]c_t \in[-1, 1]. Positive credit identifies a decision to reinforce, negative credit identifies a decision to correct, and zero credit denotes an uncertain role. For nonzero credit, the step role is yt=sign⁡(ct)y_t = \operatorname{sign}(c_t). A success-maintain memory identifies a local decision to reinforce; a failure correction identifies a local decision to avoid or correct. The role describes the step, not the outcome of the whole episode. The categorical mapping is yt=+1y_t = +1 for success_maintain and yt=−1y_t = -1 for failure_correction; uncertain candidates are excluded from admission. The gate uses this sign, not the magnitude of ctc_t. The prompt’s importance_score ranks candidates and is separate from signed hindsight credit. Task-specific guidance supplied to the Localizer is reproduced in Appendix § 13.

Evaluation signals. Paired replay measures a fixed hint’s effect on the normalized candidate-prefix distribution. Local verification refers to passing the magnitude and role-specific preference predicates on this distribution. The theoretical bounds instead concern complete-action distributions and condition on Q-value gaps and residual mass transfer. These conditions are analytical assumptions, not quantities measured by the gate. In particular, the failure ranking branch can admit an alternative top category even when the source probability increases; it tests a different preference criterion from positive signed-margin alignment. Held-out task results assess the constructed bank’s utility under the deployed decoding rule.

Mechanistic Metrics

We log candidate roles, JS, signed margins ηt\eta_t, action changes, and rejection reasons, along with episode-level candidate counts, memory costs, probes, and outcomes. Figure 5 reports cumulative success S(N)=100N−1∑i=1N1{episode i succeeds}S(N) = 100N^{-1}\sum_{i=1}^{N} 1\{\text{episode }i\text{ succeeds}\} over 200 randomly sampled ALFWorld construction tasks, for 1≤N≤2001 \le N \le200; dashed curves show bank size. Each point summarizes the task stream up to that episode, with the bank evolving during construction. It is a construction-stream statistic rather than repeated evaluation of bank checkpoints on a fixed task set. Table 1 evaluates the final banks on the separate held-out set.

Algorithms

Algorithm 1 makes the write-and-reuse loop explicit. During memory construction, the agent extracts candidates after observing an episode’s feedback. During held-out evaluation, the constructed bank is read without admitting memories from evaluation outcomes. Local replay scores a saved decision; it does not execute an alternative trajectory.

Formal Admission

For the action distributions in § 4, GtG_t is the branch-specific admission test in Eq. (20). Success-maintain memories require Mt≥τsuccM_t \ge\tau_{\mathrm{succ}}, a^tmt=at(1)\hat{a}_t^{m_t} = a_t^{(1)}, and Δpt(1)≥ϵsucc\Delta p_t^{(1)} \ge\epsilon_{\mathrm{succ}}. Failure corrections require Mt≥τfailM_t \ge\tau_{\mathrm{fail}} and either Δpt(1)≤−ϵdrop\Delta p_t^{(1)} \le-\epsilon_{\mathrm{drop}} or a^tmt≠at(1)\hat{a}_t^{m_t} \ne a_t^{(1)}. The action set, source history, and previously retrieved memory are matched across probes; only the candidate hint changes.

Branch-Specific Implementation

The token-budgeted gate applies Eq. (20) to P~t=P~ht\widetilde{P}_t = \widetilde{P}_{h_t} and P~tmt=P~htmt\widetilde{P}_t^{m_t} = \widetilde{P}_{h_t}^{m_t}. In this algorithm, Mt=DJS(P~t,P~tmt)M_t = D_{\mathrm{JS}}(\widetilde{P}_t,\widetilde{P}_t^{m_t}) and Δpt(1)=P~tmt(vt(1))−P~t(vt(1))\Delta p_t^{(1)} = \widetilde{P}_t^{m_t}(v_t^{(1)}) - \widetilde{P}_t(v_t^{(1)}) are computed on the fixed prefix categories. Define the injected top category btmt=arg⁡max⁡P~tmtb_t^{m_t} = \arg\max\widetilde{P}_t^{m_t}.

Memory evolution

Input: frozen policy πθ\pi_\theta; task stream; initial bank B\mathcal{B}; admission rule GG; candidate budget KK; write flag ww.

  1. for each task gg do

  2. Initialize HH and τ\tau.

  3. while the episode is active do

  4. Observe oo and admissible AA; retrieve M←R(B,H)M \leftarrow R(\mathcal{B}, H).

  5. Build and save h=F(H,A,M)h = F(H,A,M); select and execute aa from πθ\pi_\theta.

  6. Update HH; append the transition and feedback to τ\tau.

  7. end while; record outcome zz.

  8. if w=0w = 0 then continue.

  9. M(τ)←Lloc(τ)\mathcal{M}(\tau) \leftarrow\mathcal{L}_{\mathrm{loc}}(\tau); keep at most KK candidates.

  10. for each mt∈M(τ)m_t \in\mathcal{M}(\tau) do

  11. Load hth_t, AtA_t, at(1)a_t^{(1)}, yty_t; freeze mtm_t.

  12. Add only mtm_t to a copy of hth_t; probe both on fixed candidates.

  13. if probes are invalid: continue.

  14. Compute MtM_t, Δpt(1)\Delta p_t^{(1)} (Eqs. (5) and (6)).

  15. if Gt=1G_t = 1 (Eq. (20)): finalize a card and insert it into B\mathcal{B}.

  16. end for; update confidence from feedback; decay, merge, and prune B\mathcal{B}.

  17. end for; return outcomes and B\mathcal{B}.

Algorithm 1: Memory evolution.

The thresholds are τsucc=0.05\tau_{\mathrm{succ}} = 0.05, τfail=0.12\tau_{\mathrm{fail}} = 0.12, ϵdrop=0.10\epsilon_{\mathrm{drop}} = 0.10, and ϵsucc=0.04\epsilon_{\mathrm{succ}} = 0.04. The lower success threshold allows smaller shifts when the baseline already supports the source action. The failure ranking branch compares the injected top category with the fixed source category, independently of the baseline top category. The probability branch measures suppression; the ranking branch checks which category is preferred after injection.

Prompt Templates

Figure 6 collects the complete prompts for five stages: action selection, memory extraction, hint construction, paired replay, and card construction. Braced fields are filled at runtime. The templates retain all environment-specific instructions, including the ALFWorld heating protocol.

Branch-specific gate

1: if either probe is invalid: return false. 2: if the role is failure correction: 3: &nbsp;&nbsp;&nbsp;&nbsp;return Mt≥τfailM_t \ge\tau_{\mathrm{fail}} and 4: &nbsp;&nbsp;&nbsp;&nbsp;(Δpt(1)≤−ϵdrop or btmt≠vt(1))(\Delta p_t^{(1)} \le-\epsilon_{\mathrm{drop}} \text{ or } b_t^{m_t} \ne v_t^{(1)}). 5: if the role is success-maintain memory: 6: &nbsp;&nbsp;&nbsp;&nbsp;return Mt≥τsuccM_t \ge\tau_{\mathrm{succ}} and 7: &nbsp;&nbsp;&nbsp;&nbsp;btmt=vt(1)b_t^{m_t} = v_t^{(1)} and Δpt(1)≥ϵsucc\Delta p_t^{(1)} \ge\epsilon_{\mathrm{succ}}. 8: return false for an uncertain role.

Algorithm 2: Branch-specific gate.

Information flow. The Localizer uses the trajectory and episode feedback to assign hindsight roles. The hint writer describes a local action under the source decision’s conditions, without presenting future observations as already available or supplying a complete solution. Candidate text remains fixed during replay. After admission, the card builder copies it verbatim into memory_guidance and adds only retrieval and source metadata.

Paired context. Before executing an action, save the rendered messages: system instructions, history selected by FF, task guidance, observation, ordered admissible actions, and retrieved cards. Replay clones this snapshot twice and adds CANDIDATE GUIDANCE immediately after the memory-evidence block in the injected copy. Both calls retain the original card text and order, history clipping, model, tokenizer, candidate actions, output contract, and LL; inserting the hint must not truncate another field. The baseline omits only the candidate paragraph. Existing memory evidence is absent from both calls only when absent from the source prompt. The source action and hindsight role remain checker metadata; subsequent feedback is supplied only to the Localizer and hint writer. A missing snapshot or required score invalidates the pair.

ALFWorld Prompts

Action and replay. Panels (a) and (d) use the same action-only output contract and saved goal, family recipe, phase guidance, observation, admissible actions, history, and cards. Both request a complete admissible action; replay scores its first content token after <action> (L=1L = 1).

Extraction and storage. The Localizer selects at most five consequential decisions, including successful actions and correctable mistakes (b). Long trajectories are compacted to at most 20 selected steps, with observations clipped before serialization. Panel (c) shows the failure-correction hint; the success branch instead reinforces the observed action without adding future steps. The builder stores the admitted hint unchanged, maps its source role to a memory type, and adds retrieval metadata (e).

WebShop Prompts

Action and replay. The policy receives the instruction, task family, page phase, history, current page, available actions, and cards (f). History and page fields are clipped to 1,800 and 5,000 characters when the source prompt is built. Replay reuses those rendered fields and requests the same complete action in both conditions (i). It scores up to L=5L = 5 action-content tokens on fixed candidates, excluding the opening <action> tag.

Extraction and storage. The Localizer selects up to five consequential shopping decisions, including product, option, and checkout errors (g). Each serialized step contains the action, observation, and next observation; the last two fields are clipped to 400 and 300 characters. The writer supplies a local correction or reinforcement, not a full shopping solution (h). The builder preserves the admitted hint verbatim and adds page phase, attribute constraints, retrieval keys, and source metadata in separate fields (j).

Complete prompt templates for ALFWorld action selection and memory extraction

Figure 6. Complete prompt templates: (a) ALFWorld action selection; (b) ALFWorld memory extraction. Braced fields are filled at runtime.

Prompt templates (continued): (c) ALFWorld hint construction; (d) ALFWorld paired replay

Figure 6 (PDF p. 24). Prompt templates (continued): (c) ALFWorld hint construction; (d) ALFWorld paired replay.

Prompt templates (continued): (e) ALFWorld card construction; (f) WebShop action selection.

(e) ALFWorld / 5. Card construction

Role and task

You are ANCHOR MemoryCardBuilder. Return JSON only. Build a reusable step-level card without future-only information or a full solution.

Role mapping

The candidate critical_role defines the card type: success_maintain means success_memory, while failure_correction means fail_memory. This step type is independent of whether the whole episode succeeded; a successful episode can contain a fail_memory and a failed episode can contain a success_memory.

Card fields

Include goal_signature, visible_entities, inventory_state, memory_guidance, task_plan, retrieval_keys, confidence, support_count, uses_future_information, contains_full_solution, copies_verification_hint.

Guidance preservation

Set memory_guidance to VERIFIED HINT verbatim and copies_verification_hint to true. Add only retrieval and source metadata; do not paraphrase, summarize, or extend the guidance.

VERIFIED HINT: {candidate_hint}

FAMILY: {task_family}

GOAL: {goal}

CANDIDATE: {candidate_json}

STATE: {state_json}

POLICY_DIFF: {policy_diff_json}

TASK_PLAN: {task_plan_json}

(f) WebShop / 1. Action selection

TASK: {goal}

TASK FAMILY: {task_family}

PHASE GUIDANCE: {current_phase}

Memory-use rules

Treat each retrieved card as one local page decision, never as a complete shopping plan. Match its task family, page phase, product attributes, and current available action. A matching success card may support the listed action. A failure card describes a pattern to avoid and a local correction; never copy its failed action. Ignore a card when these cues conflict.

MEMORY DECISION EVIDENCE: {retrieved_cards} Use it only when its cue and exact available action match the current page.

Action rules

choose exactly one action, search[<keywords>] or click[<button-or-ASIN>]. Search with precise product keywords; on results select the matching ASIN; on a product page choose required options before Buy Now. Do not click description, features, or reviews. Do not invent a product, option, or button.

RECENT ACTION HISTORY: {recent_history}

Latest observation: {current_page}

Admissible actions: {admissible_actions}

Output

Return exactly one complete action inside <action>...</action>. Output no explanation.

Figure 6 (PDF p. 25). Prompt templates (continued): (e) ALFWorld card construction; (f) WebShop action selection.

Prompt templates (continued): (g) WebShop memory extraction; (h) WebShop hint construction
WebShop / 2. Memory extraction

(a)

WebShop / 3. Hint construction

(b)

Figure 6 (PDF p. 26). Prompt templates (continued): (g) WebShop memory extraction; (h) WebShop hint construction.

Prompt templates (continued): (i) WebShop paired replay; (j) WebShop card construction

(i) WebShop / 4. Paired replay                                                                                                                     User

TASK: {goal}

TASK FAMILY: {task_family}

PHASE GUIDANCE: {current_phase}

Memory-use rules

Treat each retrieved card as one local page decision, never as a complete shopping plan. Match its task family, page phase, product attributes, and current available action. A matching success card may support the listed action. A failure card describes a pattern to avoid and a local correction; never copy its failed action. Ignore a card when these cues conflict.

MEMORY DECISION EVIDENCE: {retrieved_cards} Use it only when its cue and exact available action match the current page.

CANDIDATE GUIDANCE: {candidate_hint} Apply this guidance only when it matches the current page and an available action.

Action rules

choose exactly one action, search[<keywords>] or click[<button-or-ASIN>]. Search with precise product keywords; on results select the matching ASIN; on a product page choose required options before Buy Now. Do not click description, features, or reviews. Do not invent a product, option, or button.

RECENT ACTION HISTORY: {recent_history}

Latest observation: {current_page}

Admissible actions: {admissible_actions}

Output

Return exactly one complete action inside <action>...</action>. Output no explanation.

(j) WebShop / 5. Card construction                                                                                                                     User

Role and task

You are ANCHOR MemoryCardBuilder. Return JSON only. Build a reusable step-level WebShop card from the verified decision, not a complete shopping plan.

Role mapping

Map success_maintain to success_memory and failure_correction to fail_memory; the step role is independent of the whole-episode outcome.

Card content

Store VERIFIED HINT verbatim as memory_guidance. Add shopping phase, product attributes, option constraints, retrieval keys, source metadata, confidence, and support count in separate fields.

Guidance preservation

Do not paraphrase, summarize, or extend memory_guidance. Generalize product-specific names or ASINs only in retrieval metadata, leaving the tested hint unchanged. Exclude future-only information from added metadata.

VERIFIED HINT: {candidate_hint}

FAMILY: {task_family}

GOAL: {goal}

CANDIDATE: {candidate_json}

STATE: {state_json}

POLICY_DIFF: {policy_diff_json}

TASK_PLAN: {task_plan_json}

Figure 6 (PDF p. 27). Prompt templates (continued): (i) WebShop paired replay; (j) WebShop card construction.

Case Studies

The ALFWorld cases illustrate success reinforcement, failure suppression, and memory retrieval during execution. Paired probes compare action-verb probabilities at fixed source states.

Similar Memories, Different Local Effects

Source trajectory. The task is to look at an alarm clock under a desk lamp. At the initial decision, the agent is in the middle of the room and selects go to desk 1. The next observation reveals the alarm clock and lamp on that desk (Figure 7). The recorded source verb go to is also the baseline probe’s top choice, with probability 0.835.

Figure 7. Navigation: the same source state, opposite memory effects. The recorded rollout goes to desk 1, takes the alarm clock, and turns on the lamp to complete the task. Replay tests the initial navigation decision: M1 reinforces go to; M2 suppresses it and favors look. Bars show the two dominant verb categories without renormalization; remaining probability is below 4×10−54 \times10^{-5} in each distribution.

Local comparison. The two records share the initial observation, admissible actions, source action, and baseline distribution. Their hints differ in whether navigation is the action to take or an action already completed (Figure 7).

M1: Support navigation to the target

The action ’go to desk 1’ successfully navigates you to the location containing the target objects. Proceed to interact with the alarmclock as specified in the goal.

M2: Navigation already completed

You have successfully navigated to desk 1. Verify that the alarmclock is present in your current view before proceeding.

Interpretation. M1 reinforces the successful source decision, increasing its verb probability by 0.160. M2 instead describes navigation as completed at a pre-navigation state: it reduces go to by 0.687 and raises look to 0.852. Both shifts exceed the success-role magnitude threshold (Table 5), but only M1 reinforces the source action. The larger shift from M2 therefore fails the directional test.

Hint / source recordSource verbp0→p1p_0 \rightarrow p_1JS (bits)Injected top verbCheck
Navigation hints at the same source state
M1 / eval-00006-000go to.835 → .995.074go toPass
M2 / eval-00007-000go to.835 → .148.375lookReject
Success reinforcement and failure suppression
S1 / train-00012-001take.735 → 1.000.146takePass
F1 / train-00058-000open.935 → .005.806heatPass
p0/p1p_0 / p_1: source-verb probability before / after injection. The source verb is the baseline top choice.

Table 5. Paired source-state probes: navigation hints, success reinforcement, and failure suppression.

A Local Correction for a Failed Heating Trajectory

Source trajectory. The goal is to heat an apple and place it on a dining table. After collecting the apple and reaching the microwave, the recorded agent opens the microwave, moves the apple inside, closes it, and later retrieves and places it. It never executes the environment’s explicit heat action. Subsequent repeated actions and empty outputs exhaust the 50-step limit. Figure 8 shows selected observations and actions in their recorded order.

Figure 8. Heating: correcting a failed source decision. The agent opens the microwave at step 4 but never executes heat, reaching the 50-step limit. Dots omit intervening logged steps; indices are zero-based. The hint suppresses open and reinforces heat, reversing their ranking.

Localizer record. Localizer identifies the step-4 detour in train-00058-000. The record below gives the source action, two relevant admissible actions, and the correction role.

Localized critical step
Record / step: train-00058-000 / 4
Source action: open microwave 1
Admissible actions: open microwave 1; heat apple 1 with microwave 1
Role: failure_correction
Importance score: 1.1
Importance type: heat protocol detour
Why critical: An exact target heat action was available, but the rollout chose another action.
Expected direction: Execute the exact target heat action immediately when it is listed.

Table 7.

Extracted failure-correction hint
Do not open the microwave first; the admissible actions list includes a direct command to heat the apple with the microwave, which should be executed immediately to achieve the goal.

Table 8.

Local comparison. At step 4, the hint reduces the recorded source verb open from 0.935 to 0.005 and raises heat from 0.060 to 0.988 (Figure 8). JS divergence is 0.806 bits, and the recorded checker accepts the correction. This example demonstrates the intended failure-suppression direction at the replay state.

Memory Cards

Figure 9 groups the stored examples by role. Success-maintain memories support target acquisition, state modification, and tool activation; failure corrections address unnecessary appliance interactions, unrelated objects, and spatial-goal errors.

Success-maintain memories

S1: Acquire the target object

Source hint: The alarmclock is currently on desk 1. To proceed with the task, you need to add it to your inventory.

Source action: take alarmclock 1 from desk 1

Task family: look_at_obj

S2: Complete cleaning before placement

When a task requires an object to be in a specific state (e.g., ’clean’) before storage, execute the state-modifying action (e.g., ’clean’) at the appropriate facility (e.g., ’sink’) immediately upon arrival, rather than proceeding to the storage location or performing unrelated navigation.

Source action: clean apple 3 with sinkbasin 1

Task family: pick_clean_then_place

Card ID: self-local-00062-003

S3: Activate the tool before examination

When a task requires examining an object with a specific tool, and the tool is present but inactive, the critical step is to activate the tool using the ’use’ action before attempting the examination. This activation is a prerequisite for the ’examine with’ condition to be satisfied.

Source action: use desklamp 1

Task family: look_at_obj

Card ID: self-local-00048-003

Failure corrections

F1: Avoid opening before direct heating

memory_guidance: Do not open the microwave first; the admissible actions list includes a direct command to heat the apple with the microwave, which should be executed immediately to achieve the goal.

Source action: open microwave 1

Task family: pick_heat_then_place

Card ID: self-local-00058-004

F2: Ignore an unrelated object

When an opened container reveals an object that does not match the target item required for the current heating or placement goal, do not interact with the found object. Instead, close the container and resume the search for the correct target item to avoid unnecessary detours.

Source action: take pot 1 from cabinet 2

Task family: pick_heat_then_place

Card ID: self-local-00022-006

F3: Respect the spatial goal

When a goal specifies an object located relative to another object (e.g., ’under X’), do not interact with instances of that object found elsewhere. Instead, navigate to the reference object and verify the spatial relationship before acting. Avoid taking the object if the goal is merely to observe it in situ.

Source action: take alarmclock 2 from desk 1

Task family: look_at_obj

Card ID: self-local-00096-001

Memory cards grouped by source role

Figure 9. Memory cards grouped by source role. Left: success-maintain memories S1–S3. Right: failure corrections F1–F3. Guidance text and source metadata are shown together; F1 preserves the tested heating hint verbatim.

Inference-Time Memory Injection

Paired source-state probes. Table 5 compares source-verb probabilities at fixed saved contexts. S1 increases take by 0.265 and retains it as the top choice; F1 decreases open by 0.929 and makes heat the top choice. Both satisfy the admission rule.

Source records. Table 5 lists the source records. The navigation probes share the initial state; the heating record links to F1 (self-local-00058-004), which preserves the tested hint. S1 quotes its source hint; S2, S3, F2, and F3 reproduce stored guidance. All probes use the complete saved context, L=1L = 1, and base-2 JS (Appendix § 13). Trajectory excerpts retain the recorded action order. Table 6 identifies the later retrieval episodes and steps.

Episode / stepRetrieved cardExecuted actionFeedback (abridged)
104 / 4self-local-00062-003clean apple 3 with sinkbasin 1Apple 3 is cleaned.
128 / 6self-local-00062-003clean apple 2 with sinkbasin 1Apple 2 is cleaned.
68 / 9self-local-00062-006move apple 1 to fridge 1Apple 1 is placed in the fridge.

Table 6. Stored-card retrievals and executed actions during memory construction.

A later retrieval during execution. At step 4 of training episode 104, the agent is at sinkbasin 1 with apple 3 in inventory. Its single retrieved card, self-local-00062-003, supplies this local instruction:

Retrieved cue and executed action

Injected cue: NEXT ACTION IF MATCHED: clean apple 3 with sinkbasin 1

Available and executed action: clean apple 3 with sinkbasin 1

Next observation: “You clean the apple 3 using the sinkbasin 1.”

Episode outcome: success.

The same card guides another cleaning decision with a different apple; a second card supports final placement. In all three steps in Table 6, NEXT ACTION IF MATCHED is admissible and equals the executed action.

All three episodes are logged as successful. The paired probes isolate a local policy change, and the later traces record retrieval followed by the matching action and observation.

References

  1. [3]Xingyu Chen, Rui Wang, Zhaopeng Tu, and Liefeng Bo. 2026a. Adamem: Learning what to remember with adaptive memory policies for personalized agents. Preprint, arXiv:2606.21144.arxiv.org/abs/2606.21144
  2. [4]Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. 2024. Agentpoison: Red-teaming llm agents via poisoning memory or knowledge bases. Preprint, arXiv:2407.12784.arxiv.org/abs/2407.12784
  3. [5]Zihan Chen, Songwei Dong, Chengshuai Shi, Peng Wang, Song Wang, Cong Shen, and Jundong Li. 2026b. The past is prologue: A plug-in controller for selective updates in sequentially evolving llm memory. Preprint, arXiv:2606.31121.arxiv.org/abs/2606.31121
  4. [6]Zihao Cheng, Zeming Liu, Yingyu Shan, Xinyi Wang, Xiangrong Zhu, Yunpu Ma, Hongru Wang, Yuhang Guo, Wei Lin, and Yunhong Wang. 2026. Mem²evolve: Towards self-evolving agents via co-evolutionary capability expansion and experience distillation. Preprint, arXiv:2604.10923.arxiv.org/abs/2604.10923
  5. [7]Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. 2025. Mem0: Building production-ready ai agents with scalable long-term memory. Preprint, arXiv:2504.19143.
  6. [8]Zhiqing Cui, Xinx iang Yin, Yihong Tang, Xinglang Zhang, Yuanzhe Hu, Siru Zhong, Weidong Tang, Yuxuan Liang, Weijia Li, Ming Jin, Shirui Pan, Yuhao
  7. [9]Jizhan Fang, Xinle Deng, Haoming Xu, Ziyan Jiang, Yuqi Tang, Ziwen Xu, Shumin Deng, Yunzhi Yao, Mengru Wang, Shuofei Qiao, Huajun Chen, and Ningyu Zhang. 2026a. Lightmem: Lightweight and efficient memory-augmented generation. Preprint, arXiv:2510.18866.arxiv.org/abs/2510.18866
  8. [10]Runnan Fang, Yuan Liang, Xiaobin Wang, Jialong Wu, Shuofei Qiao, Pengjun Xie, Fei Huang, Huajun Chen, and Ningyu Zhang. 2026b. Memmp: Exploring agent procedural memory. Preprint, arXiv:2508.06433.arxiv.org/abs/2508.06433
  9. [11]Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. 2024. Critic: Large language models can self-correct with tool-interactive critiquing. Preprint, arXiv:2305.11738.arxiv.org/abs/2305.11738
  10. [12]Xinyu Guan, Qianyang Zhao, and Yuming Deng. 2026. Decision-aware memory cards: Counterfactual-inspired context selection and compression for tool-using llm agents. Preprint, arXiv:2606.08151.arxiv.org/abs/2606.08151
  11. [13]Yuanzhe Hu, Yu Wang, and Julian McAuley. 2026. Evaluating memory in llm agents via incremental multi-turn interactions. Preprint, arXiv:2507.05257.arxiv.org/abs/2507.05257
  12. [14]Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2021. What makes good in-context examples for gpt-3? Preprint, arXiv:2101.06804.arxiv.org/abs/2101.06804
  13. [15]Jiaqi Liu, Yaofeng Su, Peng Xia, Siwei Han, Zeyu Zheng, Cihang Xie, Mingyu Ding, and Huaxiu Yao. 2026. Simplemem: Efficient lifelong memory for llm agents. Preprint, arXiv:2601.02553.arxiv.org/abs/2601.02553
  14. [16]Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023. Lost in the middle: How language models use long contexts. Preprint, arXiv:2307.03172. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Sean Welleck, Bodhisattwa Prasad Majumder, Shashank Gupta, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-refine: Iterative refinement with self-feedback. Preprint, arXiv:2303.17651.
  15. [17]Bodhisattwa Prasad Majumder, Bhavana Dalvi Mishra, Peter Jansen, Oyvind Tafjord, Niket Tandon, Li Zhang, Chris Callison-Burch, and Peter Clark. 2023. Clin: A continually learning language agent for rapid task adaptation and generalization. Preprint, arXiv:2310.10134.arxiv.org/abs/2310.10134
  16. [18]Siru Ouyang, Jun Yan, I-Hung Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T. Le, Samira Daruki, Xiangru Tang, Vishy Tirumalashetty, George Lee, Mahsan Roufouei, Hangfei Lin, Jiawei Han, Chen-Yu Lee, and Tomas Pfister. 2026. Reasoningbank: Scaling agent self-evolving with reasoning memory. Preprint, arXiv:2509.25140.arxiv.org/abs/2509.25140
  17. [19]Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. 2024. Mempit: Towards llms as operating systems. Preprint, arXiv:2310.08560.arxiv.org/abs/2310.08560
  18. [20]Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulation of human behavior. Preprint, arXiv:2304.03442.arxiv.org/abs/2304.03442
  19. [21]Qwen Team. 2024. Qwen2.5: A party of foundation models.
  20. [22]Qwen Team. 2026. Qwen3.5: Towards native multi-modal agents.
  21. [23]Z. Z. Ren, Zhihong Shao, Junxiao Song, Huajian Xin, Haocheng Wang, Wanjia Zhao, Liyue Zhang, Zhe Fu, Qihao Zhu, Dejian Yang, Z. F. Wu, Zhibin Gou, Shirong Ma, Hongxuan Tang, Yuxuan Liu, Wenjun Gao, Daya Guo, and Chong Ruan. 2025. Deepseek-prover-v2: Advancing formal mathematical reasoning via reinforcement learning for subgoal decomposition. Preprint, arXiv:2504.21801.arxiv.org/abs/2504.21801
  22. [24]Ohad Rubin, Jonathan Herzig, and Jonathan Berant. 2022. Learning to retrieve prompts for in-context learning. Preprint, arXiv:2112.08633.arxiv.org/abs/2112.08633
  23. [26]Jingwei Shi, Shengyu Tao, Xinxiang Yin, Chen Huang, Wenqiang Lei, and See-Kiong Ng. 2026a. Letgames: An llm-powered gamified approach to cognitive training for patients with cognitive impairment. Preprint, arXiv:2604.09566.arxiv.org/abs/2604.09566
  24. [27]Jingwei Shi, Xinxiang Yin, Jing Huang, Jinman Zhao, and Shengyu Tao. 2026b. Codehacker: Automated test case generation for detecting vulnerabilities in competitive programming solutions. Preprint, arXiv:2602.20213.arxiv.org/abs/2602.20213
  25. [28]Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. Preprint, arXiv:2303.11366.arxiv.org/abs/2303.11366
  26. [29]Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. 2021. Alfworld: Aligning text and embodied environments for interactive learning. Preprint, arXiv:2010.03768.
  27. [30]Yuru Song and Qi Xin. 2026. D-mem: Dopamine-gated agentic memory via reward prediction error routing. Preprint, arXiv:2603.14597.arxiv.org/abs/2603.14597
  28. [31]Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L. Griffiths. 2024. Cognitive architectures for language agents. Preprint, arXiv:2309.02427.arxiv.org/abs/2309.02427
  29. [32]Haoran Tan, Zeyu Zhang, Chen Ma, Xu Chen, Qianyu Dai, and Zhenhua Dong. 2025a. Membench: Towards more comprehensive evaluation on the memory of llm-based agents. Preprint, arXiv:2506.21605.arxiv.org/abs/2506.21605
  30. [33]Zhen Tan, Jun Yan, I-Hung Hsu, Rujun Han, Zifeng Wang, Long T. Le, Yiwen Song, Yanfei Chen, Hamid Palangi, George Lee, Anand Iyer, Tianlong Chen, Huan Liu, Chen-Yu Lee, and Tomas Pfister. 2025b. In prospect and retrospect: Reflective memory management for long-term personalized dialogue agents. Preprint, arXiv:2503.08026.arxiv.org/abs/2503.08026
  31. [34]The LangChain Team. 2025. LangMem SDK for agent long-term memory. Official project announcement.
  32. [35]Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voyager: An open-ended embodied agent with large language models. Preprint, arXiv:2305.16291.arxiv.org/abs/2305.16291
  33. [36]Haoyu Wang, Guangyuan Dong, He Liang, Zijing Zhang, Jiachen Luo, Chuang Liu, Chao Xue, and Hao Tang. 2026a. Memguard: Persisting verifier signals for llm-agent memory governance. Preprint, arXiv:2608.21867.arxiv.org/abs/2608.21867
  34. [37]Jun Wang. 2026. Memento 2: Learning by stateful reflective memory. Preprint, arXiv:2512.22716.arxiv.org/abs/2512.22716
  35. [38]Yian Wang, Agam Goyal, Yuen Chen, and Hari Sundaram. 2026b. State contamination in memory-augmented llm agents. Preprint, arXiv:2605.16746.arxiv.org/abs/2605.16746
  36. [39]Tianxin Wei, Noveen Sachdeva, Benjamin Coleman, Zhankui He, Yuanchen Bei, Xuying Ning, Mengting Ai, Yunzhe Li, Jingrui He, Ed H. Chi, Chi Wang, Shuo Chen, Fernando Pereira, Wang-Cheng Kang, and Derek Zhiyuan Cheng. 2026. Evo-memory: Benchmarking llm agent test-time learning with self- evolving memory. Preprint, arXiv:2511.20857.
  37. [41]Rong Wu, Daocheng Fu, Licheng Wen, Xuemeng Yang, Shu Zou, Jianbiao Mei, Yuxin Wang, Hairong Zhang, Yu Yang, Tao Hu, Cong Zhang, Botian Shi, and Pin- long Cai. 2026. Memharness: Memory is recon- structed, not replayed. Preprint, arXiv:2607.28272.arxiv.org/abs/2607.28272
  38. [42]Peng Xia, Jianwen Chen, Hanyang Wang, Jiaqi Liu, Kaide Zeng, Yu Wang, Siwei Han, Yiyang Zhou, Xujiang Zhao, Haifeng Chen, Zeyu Zheng, Cihang Xie, and Huaxiu Yao. 2026. Skillrl: Evolving agents via recursive skill-augmented reinforcement learning. Preprint, arXiv:2602.08234.arxiv.org/abs/2602.08234
  39. [43]Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. 2025. A-mem: Agentic memory for llm agents. Preprint, arXiv:2502.12110.arxiv.org/abs/2502.12110
  40. [44]John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. Swe-agent: Agent-computer interfaces enable automated software engineering. Preprint, arXiv:2405.15793.arxiv.org/abs/2405.15793
  41. [45]Kairui Yang, Minghao An, Xunkai Li, Ziheng Yi, Zekai Chen, Guangyuan He, and Rong-Hua Li. 2026a. Mace: Memory-agent co-evolution with adaptive memory graphs for multi-agent systems. Preprint, arXiv:2609.21533.arxiv.org/abs/2609.21533
  42. [46]Tianyu Yang, Sudipta Paul, Vijay Srinivasan, Vivek Kulkarni, and Srinivas Chappidi. 2026b. Trust- mem: Learning trustworthy memory consolidation for llm agents with long-term memory. Preprint, arXiv:2606.25161.arxiv.org/abs/2606.25161
  43. [47]Xianglin Yang, Yufei He, Shuo Ji, Bryan Hooi, and Jin Song Dong. 2026c. Zombie agents: Persis- tent control of self-evolving llm agents via self- reinforcing injections. Preprint, arXiv:2602.15654.arxiv.org/abs/2602.15654
  44. [48]Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. 2023. Webshop: Towards scalable real- world web interaction with grounded language agents. Preprint, arXiv:2207.01206.arxiv.org/abs/2207.01206
  45. [49]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629.arxiv.org/abs/2210.03629
  46. [50]Dechen Zhang, Xuan Tang, Xinxiang Yin, Xingwu Chen, Jian Qian, and Difan Zou. 2026a. Valg: An agentic system for ml theory research. Preprint, arXiv:2608.13060.arxiv.org/abs/2608.13060
  47. [51]Dylan Zhang, Yanshan Lin, Zhengkun Wu, Yihang Sun, Bingxuan Li, Dianqi Li, and Hao Peng. 2026b. Use- ful memories become faulty when continuously up- dated by llms. Preprint, arXiv:2605.12978.arxiv.org/abs/2605.12978
  48. [52]Guibin Zhang, Haotian Ren, Chong Zhang, Zhenhong Zhou, Junhao Wang, He Zhu, Wangchunshu Zhou, and Shuicheng Yan. 2025. Memomevolve: Meta- evolution of agent memory systems. Preprint, arXiv:2512.18746.arxiv.org/abs/2512.18746
  49. [53]Guilin Zhang, Wei Jiang, Xiejiaxhan Wang, Aisha Behr, Kai Zhao, Jeffrey Friedman, Xu Chu, and Amine Anoun. 2026c. Adaptive memory admission control for llm agents. Preprint, arXiv:2603.04549.arxiv.org/abs/2603.04549
  50. [54]Jiajie Zhang, Yankai Xiang, and Changhao Chen. 2026d. Memory that changes action is not memory that guides it: Counterfactual auditing of history-conditioned robot policies. Preprint, arXiv:2609.27247.arxiv.org/abs/2609.27247
  51. [55]Shengtao Zhang, Jiaqian Wang, Ruiwen Zhou, Junwei Liao, Yuchen Feng, Zhuo Li, Yujie Zheng, Weinan Zhang, Ying Wen, Zhiyu Li, Feiyu Xiong, Yutao Qi, Bo Tang, and Muning Wen. 2026e. Memrli: Self- evolving agents via runtime reinforcement learning on episodic memory. Preprint, arXiv:2601.03192.arxiv.org/abs/2601.03192
  52. [56]Xinglang Zhang, Yuanmeng Xiang, Yunyao Zhang, Zeliang Chen, Junqing Yu, and Zikai Song. 2026f. Too good to be real? diagnosing and reducing the gap between ai preference and real user engagement. Preprint, arXiv:2609.18282.arxiv.org/abs/2609.18282
  53. [57]Xinglang Zhang, Yunyao Zhang, Zeliang Chen, Jun- qing Yu, Wei Yang, and Zikai Song. 2026g. Logical phase transitions: Understanding collapse in llm log- ical reasoning. Preprint, arXiv:2601.02902.arxiv.org/abs/2601.02902
  54. [58]Yan Zhang and Shibo Li. 2026. Consistencygate: Preventing memory contamination in llm agents via self-consistency admission control. Preprint, arXiv:2607.22962.arxiv.org/abs/2607.22962
  55. [59]Yunxiang Zhang, Yiheng Li, Ali Payani, and Lu Wang. 2026h. Adamem: Test-time adaptive memory for language agents. Preprint, arXiv:2606.05684.arxiv.org/abs/2606.05684
  56. [61]Zeyu Zhang, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Quanyou Dai, Jieming Zhu, Zhenhua Dong, and Ji- Rong Wen. 2024. A survey on the memory mecha- nism of large language model based agents. Preprint, arXiv:2404.13501.arxiv.org/abs/2404.13501
  57. [62]Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. 2024. Ex- pel: Llm agents are experiential learners. Preprint, arXiv:2308.10144.arxiv.org/abs/2308.10144
  58. [63]Longtao Zheng, Rundong Wang, Xinrun Wang, and Bo An. 2024. Synapse: Trajectory-as-exemplar prompting with memory for computer control. Preprint, arXiv:2306.07863.arxiv.org/abs/2306.07863

Paper details

vlnc:W_esbd4scg
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 1

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

large language models

View statement
Looking for related papers…
Contents