betaFeedback

WorksComputer Science

Agentic Context Management for Multi-Turn Human-AI Conversations

Abstract

LLMs degrade in multi-turn conversation partly because of context pollution: the model's own earlier outputs stay in context and anchor later turns, so an early misreading of the user's intent can persist after the user corrects it. Assistant omission (AO), dropping every assistant message, removes this pollution but fails when interaction is referential: later turns refer back to drafts, decisions, and tool-call results that exist only in assistant turns. We propose Agentic Conversation Context Curation (AC3), a training-free method in which a separate analyzer (1) consolidates the task specification from the user's messages alone, (2) audits the assistant's prior outputs against it, and (3) edits the context, either appending the analysis or discarding the assistant side and rebuilding the context from the specification. A second finding: pollution is contagious. An analyzer that sees the assistant's reasoning while extracting the specification can do worse than no intervention, and instructing it to ignore them did not close the gap, so we exclude assistant turns structurally. On Lost in Conversation (replaying difficulty-selected instances), where AO is an oracle by construction, AC3 closes 55-84% of the gap between full context and AO on three of four tasks and exceeds AO on the fourth, and an LLM that faithfully summarizes the history does not reproduce the gain even with more compute. On real WildChat conversations, a GPT-5-mini judge prefers AC3's next turn over both full context and AO in every configuration of our main table, though this preference depends on the judge; on CollabLLM results are mixed. In stateful tool use (tau2-bench), AO collapses to 0% on every model; AC3 does not collapse, though it falls slightly behind full context. Detector studies show that Reset removes injected pollution but is not span-selective: the gain comes from rebuilding the specification.

Introduction

Multi-turn interaction is the dominant mode of use for large language models, from coding assistants to computer-use agents such as OpenClaw OpenClaw, [10]. Yet LLMs perform substantially worse in multi-turn settings than on equivalent single-turn tasks: Laban et al. [ [9] report an average drop of 39% across 15 models and 6 task domains, even in conversations of only ∼8 turns. They also identify the self-reinforcing dynamic behind the drop: early errors about the user’s intent are recorded in the assistant’s own messages and bias later turns, so errors compound instead of being corrected as new user information arrives. Huang et al. [ [5] call this context pollution, study it in real human-AI conversations, and show that a coarse fix works surprisingly well: assistant omission (AO), which discards all assistant messages from the context.

Preprint.

AO works when the user’s messages so far, taken together, contain enough information to complete the task; we call such turns self-contained. Many turns are instead referential: the correct response depends on content that exists only in earlier assistant turns. In plain chat, users build on what the assistant wrote (“sharpen sentence 2”; “add a column to that query”). In tool use, the dependence is on environment state, a case we call stateful: tool-call results and prior decisions are recorded only in assistant turns. For referential turns, removing assistant messages deletes the content that the next turn depends on.

We propose Agentic Conversation Context Curation (AC3), a training-free, inference-time method that runs inside the agent harness before each assistant turn (Figure 3). A separate LLM, the analyzer, (1) extracts a consolidated task specification from the user’s messages alone, (2) audits the assistant’s prior outputs against that specification, and (3) edits the context with one of three operators: Augment appends the analysis to the conversation, Reset replaces the conversation with the specification and the prior work the analyzer judged correct, and Rewrite has an LLM restructure the conversation around the specification. Augment, which only adds information, fires on every turn after a short warm-up; Reset and Rewrite, which remove content, act only when the analyzer flags an issue. Our default configuration, Gated Reset, additionally skips the first user turns and caps the number of resets. With Reset, the mechanism is simple to state: AC3 detects pollution, discards the assistant side, and rebuilds the specification from the user side, so the assistant continues from what the user asked for rather than from its own invalidated reasoning. In stateful settings the analyzer also records the tool state it has observed, so that a rebuild can carry forward the state the next action needs.

Agentic Conversation Context Curation (AC3)

Figure 3. Agentic Conversation Context Curation (AC3). Before the assistant generates, a separate analyzer runs two analysis subtasks and one intervention step. Subtask 1 (specification extraction) sees only user turns, so it cannot be biased by the assistant’s prior reasoning. Subtask 2 (approach evaluation) compares that specification against the assistant’s full history. An operator (Augment, Reset, or Rewrite; the figure shows a rewritten context) then edits the context; the “retained” content in the rebuilt context is restated by the analyzer, not copied from assistant turns. Structural exclusion in Subtask 1 is essential: when the extractor sees assistant turns, accuracy falls below the full-context baseline on math and code (Table 4).

Unlike pollution-oriented interventions such as AO and ERGO [8], which remove or rewrite context wholesale once triggered, AC3 audits the assistant’s prior work against a user-grounded specification; unlike context management for long-horizon agents [13, 30, 18 inter alia], which compacts a trajectory to fit a token budget, it targets correctness in ongoing dialogue (§ 7). In spirit it is a use of test-time compute [23, 22, 29, 16], spent on curating the input context rather than on searching over outputs.

Building the analyzer exposed an unexpected finding: pollution is contagious. A model tasked with auditing the conversation anchors on the assistant’s prior reasoning if it can see it. When the specification extractor sees assistant turns, accuracy falls below that of no information on two of three LiC tasks, and asking the extractor to reason only from user-stated requirements does not close the gap (§ 5.6; Appendix § 12). Context management must therefore apply to the context manager itself: each analyzer step sees only the conversation slices its role requires, and we enforce this structurally rather than by prompting.

We evaluate on four benchmarks ordered by referentiality (Figure 1). In Lost in Conversation (LiC) [9], each instance is a single-turn question sharded across turns, so the user’s messages together fully specify the task and AO is a design oracle. There, in replay mode on difficulty-selected instances, Gated Reset closes 55–84% of the gap between full context and AO on math, code, and actions and exceeds AO on database (e.g., 15.8%→64.4% on code, a mean of three reruns), and Reset improves over full context in 33 of 36 paired comparisons across three further respondent models on a second difficulty-selected pool. On real human-AI conversations from WildChat [34], a GPT-5-mini judge prefers every AC3 configuration in Table 3 over both AO and full context, although the preference depends on the judge (§ 6). On CollabLLM [26], near the middle of the spectrum, results are mixed: relative to full context, Reset ranges from −0.1 to +7.5pp across respondents and AO from 0.0 to +5.0pp, gaps of at most three problems out of 40. On stateful tool use in tau2-bench [1], AO collapses to 0% on every respondent because tool-call results live only in assistant turns. AC3 remains viable, meaning it does not collapse (Gated Reset scores 7–12pp below full context, not significantly at N=3N=3), but no AC3 operator beats full context there.

Figure 1. The value of omitting assistant turns falls as task referentiality grows. Results for one respondent (gpt-5.4) on the four benchmarks, ordered by how much the next user turn depends on the assistant’s prior turns. Values are taken from Table 2 (LiC, CollabLLM, tau2-bench) and Table 3 (WildChat); the AC3 bar is the best operator for this respondent on each benchmark (on WildChat, best against AO). On LiC, where the user’s messages together fully specify the task, AO is strong and AC3 is higher still. On CollabLLM, AC3-Reset (62.5) is one problem out of 40 ahead of AO (60.0) and three ahead of full context (55.0). On WildChat, a GPT-5-mini judge prefers AC3-Reset’s next turn over both AO and full context (dashed line: parity). On tau2-bench, where tool-call results live only in assistant turns, AO collapses to 0% and AC3 does not collapse, but full context remains best.

StrategyGPT-5-mini vs AOGPT-5-mini vs FCgpt-5.4 vs AOgpt-5.4 vs FCdsv4f vs AOdsv4f vs FCk2.6 vs AOk2.6 vs FC
AC3-Augment——86.078.989.5—82.9—
AC3-Reset83.083.588.677.375.0—71.6—
AC3-Gated-Reset86.183.874.172.4————
AC3-Rewrite82.680.383.372.979.273.691.576.3

Table 3. WildChat: pairwise win rates (%) by respondent. Each cell is the rate at which a GPT-5-mini judge prefers the AC3 response over assistant omission (AO) or full context (FC); 50% is parity. —: not reported. Turn sets and judge checks are in Appendix § 10.§ 6.

Our contributions are:

  • An account of when discarding assistant turns works (self-contained vs. referential turns) that explains AO’s range from design oracle to 0% (§ 2).

  • AC3, in which an analyzer shielded from the assistant’s turns rebuilds the task specification from the user side and edits the context (§ 3), and the finding that pollution is contagious, which makes that shielding a structural requirement (§ 5.6).

  • An evaluation on four benchmarks of increasing referentiality, with matched-compute baselines and a detector evaluation that locates the gain in the rebuild rather than in span-selective editing (§ 5).

Problem setting

Setup and notation

Consider a multi-turn interaction. At turn tt, the conversation context is the message sequence Ct=(u1,a1,u2,a2,…,ut)C_t = (u_1, a_1, u_2, a_2, \ldots, u_t), where uiu_i and aia_i denote user and assistant messages respectively, and the assistant samples at∼pθ(⋅∣Ct)a_t \sim p_\theta(\cdot\mid C_t). Let Ut=(u1,…,ut)U_t = (u_1, \ldots, u_t) denote the user prefix and At−1=(a1,…,at−1)A_{t-1} = (a_1, \ldots, a_{t-1}) the assistant prefix. A system message, when present, is part of both CtC_t and UtU_t.

Context pollution

Let T∗T^* denote the true task specification, which the user reveals incrementally across UtU_t. In principle, UtU_t gives a progressively more complete view of T∗T^*. However, each assistant message aia_i encodes the assistant’s interpretation T~i\widetilde{T}_i of the task at turn ii, and these interpretations may diverge from T∗T^* as new user messages refine or contradict earlier ones. Conditioning on Ct=(Ut,At−1)C_t = (U_t, A_{t-1}) then biases the model toward the stale interpretations, so that, on average over instances,

pθ(correct∣Ut,At−1)<pθ(correct∣Ut).(1)p_\theta(\mathrm{correct} \mid U_t, A_{t-1}) < p_\theta(\mathrm{correct} \mid U_t). \tag*{(1)}

The assistant’s own prior outputs reduce the probability of a correct response. Figure 2 illustrates the dynamic: an early misinterpretation persists in the context window and re-anchors the model’s reasoning even after the user has provided enough information to correct it. Huang et al. [ [5] confirm the inequality empirically: omitting At−1A_{t-1} (assistant omission, AO) recovers much of the lost accuracy.

Illustrative example of context pollution and statefulness

Figure 2. Illustrative example. (a) Context pollution: an early incorrect interpretation by the assistant (a zero prepaid balance) stays in the context and re-anchors a later response even after the user has corrected it. (b) Coarse omission: AO removes the invalidated reasoning but also the state the next action needs (the restart the agent already recommended), so the agent repeats itself. The deficit is behavioral rather than a lack of capability: on LiC, the same models solve the task when it is posed in a single turn [9]. Our method (§ 3, Figure 3) supplies the missing control at the harness level by rebuilding the conversation around a specification drawn from the user’s messages and dropping the invalidated reasoning.

Why omission is not enough: referential vs. self-contained turns

AO assumes that UtU_t alone determines a correct ata_t. We call such turns self-contained; their counterpart is referential, where a correct ata_t depends on content that exists only in At−1A_{t-1}. Referential turns are common. In chat without environment state, users build on assistant-generated content: they edit a draft the assistant wrote, refine code from a previous turn, or ask a follow-up about a specific earlier step [5]. We call a referential turn stateful when the content it depends on is environment state, as in tool use, where tool-call results are recorded in At−1A_{t-1}. Omitting them makes the agent repeat actions at best and lose track of the environment at worst. In every referential case, discarding assistant turns breaks the reference. Sharded LiC [9] sits at the self-contained end of this spectrum and stateful tool use [1] at the other.1

Editing operator desiderata

We seek an editing operator E\mathcal{E} that produces a modified context Ct′=E(Ct)C'_t = \mathcal{E}(C_t) with two properties:

  1. Pollution suppression. Ct′C'_t suppresses the biasing effect of stale interpretations in At−1A_{t-1}, so that pθ(correct∣Ct′)≥pθ(correct∣Ct)p_\theta(\mathrm{correct}\mid C'_t) \ge p_\theta(\mathrm{correct}\mid C_t).

  1. Preservation. Ct′C'_t retains the task-relevant information in UtU_t and At−1A_{t-1}, so that pθ(correct∣Ct′)≥pθ(correct∣Ut)p_\theta(\mathrm{correct}\mid C'_t) \ge p_\theta(\mathrm{correct}\mid U_t) whenever At−1A_{t-1} contains state the next turn needs.

AO satisfies the first property by construction (it replaces At−1A_{t-1} with ∅\varnothing) but violates the second: it cannot retain what it discards. AC3’s operators do not keep assistant spans verbatim; our detector evaluation shows that they are not span-selective (§ 5.7). It is designed to carry task-relevant information forward by restating it: the rebuilt context contains the specification drawn from UtU_t, the analyzer’s record of the prior work it verified, and, in stateful settings, the environment state it has observed. We have not measured how much is carried forward this way, and injected correct facts mostly do not survive a reset (§ 5.7.1).

Methods

We implement the editing operator E\mathcal{E} of § 2 as an analyzer that inspects the conversation before each assistant turn. A model cannot reliably tell its own correct work from its own mistakes when both sit in the same assistant messages, so AC3 externalizes that judgment: the analyzer consolidates what the user asked for from the user’s messages alone, compares the assistant’s work against it, and rebuilds the conversation around that specification rather than around the assistant’s prior reasoning (Algorithm 1, Appendix § 9).

Conversation analysis via subtask decomposition

Identifying pollution requires ground truth, and in multi-turn conversation the user’s messages are that ground truth: an approach is invalidated when later user messages contradict its assumptions. Because new information can look compatible with a polluted approach and users revise earlier requirements, the specification must be consolidated from the union of user messages. Two design requirements follow. First, the specification must be grounded in the user’s requirements rather than in the assistant’s interpretation, matching findings that LLMs cannot reliably self-correct without external feedback [7] and cannot locate their own reasoning errors, although they can correct them once told where they are [20]. Second, the extraction itself must be shielded from the assistant’s reasoning, or it becomes polluted too. We therefore split the analysis into two subtasks (Algorithm 1, lines 2–5; tested in § 5.6):

  • Subtask 1: specification extraction. The model sees only UtU_t (user and system messages), with At−1A_{t-1} structurally excluded, and produces a consolidated task specification T^\hat{T}. The system message grounds the output format (e.g., SQL schema, function signatures). Physically excluding assistant messages implements an attentional control that prompting alone did not achieve in our tests (Appendix § 12).

  • Subtask 2: approach evaluation. The model receives T^\hat{T}, the full context CtC_t (including assistant messages), and optionally a memory cheatsheet MM (§ 3.3). Grounded in T^\hat{T}, it identifies what the assistant got right (aligned) and where it diverged (issues).

Editing operators

The analysis yields (T^,aligned,issues)(\hat{T}, \textit{aligned}, \textit{issues}). Three operators implement E\mathcal{E} with increasing modification of the context. One rule governs when they act: an operator that only adds information acts on every turn after a warm-up, and an operator that removes content acts only when the analysis finds something to remove. The analysis (§ 3.1) is shared by the operators (on WildChat, Gated Reset uses a later revision of the Subtask 2 prompt; § 5.4); the choice of operator is a design knob whose best setting depends on the respondent and the benchmark (Table 2).

StrategyLiC gpt-5.4LiC dsv4fLiC k2.6CollabLLM gpt-5.4CollabLLM dsv4fCollabLLM k2.6tau2-bench gpt-5.4tau2-bench dsv4ftau2-bench k2.6
Baseline60.451.360.455.052.560.068.470.278.9
AO74.569.568.260.052.562.50.00.00.0
AC3-Augment77.067.073.852.557.555.047.450.957.9
AC3-Reset77.268.474.362.555.059.957.9*57.9*71.9*
AC3-Rewrite76.664.274.557.552.557.547.457.966.7

Table 2. Results across respondents on LiC, CollabLLM, and tau2-bench. Benchmark-averaged accuracy (LiC: four tasks on a second difficulty-selected pool, § 5.1; CollabLLM: MATH-Hard and BigCodeBench) or task reward (tau2-bench telecom_small, mean of N=3N=3 seeds). Respondents: gpt-5.4, dsv4f (DeepSeek-V4-Flash), k2.6 (Kimi-K2.6). AC3-Reset has no turn floor or reset cap; *Gated Reset, the only Reset variant run on tau2-bench.

  • Augment. Append the analysis: Ct′=Ct⊕(T^,aligned,issues)C'_t = C_t \oplus(\hat{T}, \textit{aligned}, \textit{issues}). This adds a corrective signal but leaves the polluting content in At−1A_{t-1} visible, where it may still anchor generation.

  • Reset. Replace the conversation with a filled template: Ct′=TEMPLATE⁡(T^,aligned)C'_t = \operatorname{TEMPLATE}(\hat{T}, \textit{aligned}), followed by the latest user message. Because the analysis maps to the template deterministically (no extra LLM call), this is the cheapest operator that removes content, and it does not carry injected pollution forward the way Rewrite can (§ 5.7.1).

  • Rewrite. An additional LLM call generates a restructured context: Ct′∼pθ(⋅∣T^,aligned,issues,Ct)C'_t \sim p_\theta(\cdot\mid\hat{T}, \textit{aligned}, \textit{issues}, C_t). This is the most flexible operator; it trades occasional summarization loss for arbitrary restructuring, and on WildChat it beats Reset for two of four respondents (DeepSeek-V4-Flash and Kimi-K2.6; § 5.4).

  • When operators act. Augment appends its analysis on every turn after a short warm-up, because it only adds information. Reset and Rewrite remove content, so they act only when the analyzer flags substantive issues (Algorithm 1, lines 6–8), which on LiC happens on 94–99% of analyzed turns. On CollabLLM, Augment and Rewrite start at the fourth user turn and Reset at the second. Every LiC, CollabLLM, and WildChat cell of Table 2 and Table 3 follows this rule (Appendix § 10.§ 2). Gated Reset is Reset with a turn floor (it skips the first two user turns; the first on LiC actions, WildChat, and tau2-bench) and a cap of three resets per conversation, which reduce cost and avoid disrupting on-track conversations in agentic settings where a reset can discard tool results that are expensive to re-obtain. Gated Reset is the AC3-Gated-Reset row in our results.

Our default is Gated Reset, the operator we replicate most (N=3N=3 on LiC) and the strongest or tied-strongest AC3 operator on tau2-bench; on text with a strong respondent and no cost constraint, Reset without the floor and cap can do better (§ 5.4). For choosing among operators more generally we offer a heuristic keyed on a property observable before deployment, the strength of the full-context baseline. In stateful settings with a strong baseline, prefer the most conservative operator (Gated Reset), since there is little erroneous reasoning to remove and much tool state to keep (on tau2-bench even Gated Reset does not beat full context); where pollution is high or the respondent is weak, heavier operators have more to remove. We have not validated this heuristic prospectively.

Optional memory

We also test whether the analysis can improve with experience, using the Dynamic Cheatsheet [19]: a single plain-text memory MM, updated globally with no retrieval step, so that retrieval quality cannot confound the effect of memory. After each problem instance, a reflection prompt abstracts transferable takeaways (e.g., common pitfalls, effective consolidation strategies) into MM, capped at 1500 words, and MM is supplied to Subtask 2 on later problems. Memory is an optional add-on: at our sample sizes its gains are not separable from its own run-to-run variance (§ 6; details in Appendix § 14).

Experiments

Unless otherwise noted, GPT-5-mini serves as both the assistant and the analyzer. We call the model that generates the assistant turns the respondent; multi-respondent experiments vary it. The analyzer is GPT-5-mini on tau2-bench and in the gpt-5.4, DeepSeek-V4-Flash, and Kimi-K2.6 columns of Table 3, and each model is its own analyzer in the multi-model LiC study (Table 6). The four benchmarks below differ in how much of the assistant’s prior work the task needs, which lets us read how each intervention responds to increasing referentiality.

ModelStrategyMathCodeDatabaseActionsAvg. Δ
GPT-5-miniBaseline20%10%5%60%
GPT-5-miniReset55%25%40%80%+26pp
GPT-5Baseline40%21%27%85%
GPT-5Reset70%47%65%70%+20pp
DeepSeek V3.2Baseline50%7%10%35%
DeepSeek V3.2Reset80%62%45%70%+39pp
Qwen 3.5 35BBaseline55%21%5%45%
Qwen 3.5 35BReset75%64%75%80%+42pp

Table 6. Multi-model LiC results on a hard subset selected via high baseline failure rate by GPT-5.2. Reset improves accuracy for 15 of 16 model-task pairs across four models from three families.

Lost in Conversation (LiC). LiC [9] shards a fully specified single-turn question into fragments that a simulated user reveals incrementally. We test four domains: math, code, database (SQL), and function calling (“actions”). We compare against AO [5], the Concat User baseline [9], which joins the shards back into a single message, and ERGO [8], which resets via an LLM rewrite of prior user messages. Because each LiC instance is a single-turn question by construction, AO and Concat User are design-oracle baselines. We run LiC in replay mode: all strategies share the same full-context conversation prefix and only the final turn is regenerated, so comparisons measure what an intervention recovers from a fixed polluted history rather than end-to-end multi-turn behavior. Instances are difficulty-selected (Table 1); the protocol and per-task adjustments are in Appendix § 10.§ 1.

CollabLLM. We evaluate on MATH-Hard and BigCodeBench using the CollabLLM multi-turn framework [26]. Unlike LiC, the conversation is not pre-sharded: the simulated user issues an underspecified request (still derived from a single-turn benchmark item) and both sides exchange natural follow-ups, so the assistant’s prior turns contain partial work that may matter later. Evaluation details are in Appendix § 10.§ 5.

WildChat. We evaluate on WildChat-1M [34] following Huang et al. [ [5]. These are real human-AI dialogues in which users reference and build on earlier assistant outputs, so AO can sever the conversational thread and is no longer an oracle. Each strategy generates the next assistant turn for a real conversation prefix, and a GPT-5-mini judge compares it pairwise against AO’s and full context’s responses. Filtering, the two-phase setup, and the judge are in Appendix § 10.§ 6; Appendix § 15 walks through a Maven/Java debugging conversation.

tau2-bench. As a preliminary exploration of the agentic setting, we evaluate on the telecom_small subset of tau2-bench [1], in which a customer-service agent resolves a telecom troubleshooting call through diagnostic tool calls. Its dual-control design lets both the user and the agent change the environment state, so state information is spread across both parties’ messages and actions. We report task reward over three seeds of 19 tasks. The agentic adaptation is in Appendix § 10.§ 7.

One analyzer across benchmarks. All four benchmarks use the same two-subtask analysis of § 3.1, with documented departures, of which two are substantial. The GPT-5-mini CollabLLM result uses an earlier single-pass Rewrite over the full transcript, without structural exclusion (§ 5.3). On tau2-bench, the analysis is re-implemented inside the tau2-bench agent with prompts adapted to tool calls: the analyzer tracks environment state and frames its output as a strategic reflection (Appendix § 10.§ 7). The Rewrite prompt for Table 2 and the gpt-5.4, DeepSeek-V4-Flash, and Kimi-K2.6 columns of Table 3 was selected among five variants on the DeepSeek-V4-Flash LiC pool (Appendix § 11). The operators and their order of intensity are the same everywhere.

Results

Lost in Conversation

AC3 closes most of the gap to the design oracle. On the Gated Reset row (mean of N=3N=3 replay-mode reruns; Table 1), AC3 reaches 80.0% on math, closing 80% of the 25.0pp gap between full context and AO, and 64.4% on code, closing 84% of the 57.9pp gap; on math, code, and actions the closure ranges from 55% (actions, where Gated Reset uses raw n=25n=25 against the other rows’ 23) to 84% (code). On database, both Gated Reset (38.7%) and Reset (48.0%, a single trial) exceed the design oracles (32.0%), possibly because consolidating the shards into a schema-grounded specification helps more than removing assistant turns alone. On math the gain is not separable from the analyzer solving the problem: on the three-respondent matrix 47% of math analyzer outputs contain the answer, and on the rest the paired gain is −2.6pp (p=0.815p=0.815); on code and database (0–1% leakage) it holds on leak-free items (+30.2pp, +26.0pp). The Gated Reset reruns have standard deviations of 5.0–7.2pp, and their means clear the full-context baseline on every LiC task and match or exceed the no-memory Augment operator (Appendix § 10.1.3).

The gain holds on unselected pools. Difficulty selection can inflate gains through regression to the mean, so we also ran LiC database end-to-end on the complete, unselected pool (n=107n=107; assistant gpt-5.4-mini; § 5.2). There, full context scores 56.1%, AO 69.2%, Concat User 63.6%, and Reset 75.7%, above both design oracles; on the unselected code pool (n=100n=100), Reset scores 92.0% against full context’s 83.0% (+9.0pp, exact McNemar p=0.023p=0.023).

Removing polluted content helps beyond the analysis. With the same analysis, Reset and Gated Reset score above Augment on code (+5.3–11.8pp) and database (+6.7–16.0pp). The operators differ only in whether the assistant still sees the polluted history, which is consistent with anchoring on that history being an effect separate from the fragmentation of the task across shards. Except for Gated Reset, these comparisons rest on single trials.

Statistics and scale. On the three-respondent LiC matrix of Table 2 (113–150 conversations per respondent and task), Reset improves over full context in 33 of 36 paired comparisons (3 respondents × 4 tasks × 3 replay prefixes; mean +15.9pp, sign test p<0.0001p < 0.0001). This is a single fixed configuration, not a per-cell maximum, and a perfect nn of a failure-selected pool (its replay prefixes are also weighted toward baseline failures), not a population estimate. At the item level, Reset gains +15.4pp (McNemar p<10−4p < 10^{-4}; 350 item-level wins to 93) with a problem-clustered 95% interval of [+11.5, +19.4]. We report this figure as a descriptive aggregate: the 1,668 items are 191 problems observed under up to 9 correlated replicates each, so they are not independent and the interval comes from a problem-clustered bootstrap rather than from treating items as independent trials.

Generalization across model families. Alongside GPT-5-mini, we evaluate three additional models (GPT-5; DeepSeek V3.2, open-weight with 37B active parameters; and Qwen 3.5 35B-A3B, with 3B active) on a harder subset of 20 instances per task selected from GPT-5.2 logs (Appendix § 10.§ 4). Reset improves accuracy in 15 of 16 model-task pairs (Table 6), by +20 to +42pp on average per model. Table 2 adds three further respondents (gpt-5.4, DeepSeek-V4-Flash, Kimi-K2.6) on the second GPT-5.2-selected pool (50 instances per task). The best AC3 operator beats full context in every LiC cell, and the three operators land within about 4pp of each other; CollabLLM and tau2-bench are discussed in § 5.3 and § 5.5. No single operator dominates everywhere.

Matched-compute baselines: faithful condensation and MT-OSC

A natural objection is that AC3’s gains come from extra compute or from compression per se rather than from adjudicating polluted reasoning. Two baselines on LiC argue against this. First, a per-turn LLM condenser instructed to compress faithfully (to preserve content, not to find errors) does not close the gap on the full, unselected pool. On LiC-database (assistant gpt-5.4-mini, full context 56.1%), the condenser scores 53.3% at one call per turn (−2.8pp, p=0.68p = 0.68) and 47.7% at two calls per turn (−8.4pp, p=0.08p = 0.08); a replicate of the one-call arm also scored 47.7%, so the condenser is neutral to negative at either budget rather than degrading with budget, and a neutral-phrasing variant reproduces the result at 51.4%. AC3-Reset reaches 75.7% (+19.6pp, p=0.0005p = 0.0005) on the same conversations, and the pattern repeats on code (79.0/80.0% for the condenser vs. 83.0% for full context and 92.0% for Reset, +9.0pp, p=0.023p = 0.023). Compute does not explain the difference: the two-call condenser used more than Reset (1.02–1.19× its strategy calls, 1.62–2.14× its tokens) and still lost, while Gated Reset reaches +17.8pp at 0.41× Reset’s calls. The mechanism is visible directly: with the error-finding clause removed, the condenser flags an assistant error in 0 of 340 summaries, so faithful summarization carries invalidated reasoning forward rather than adjudicating it.

Second, we reimplemented MT-OSC [15], a multi-turn one-off sequential condensation method that condenses the chat history in the background (prompt, exemplars, recursion, and hyperparameters verbatim, as no code is released; condenser gpt-5.4-mini instead of Llama-3.3-70B, matching our other arms). At its published window w=4w=4 on LiC-database it is a near no-op: 60.7% vs. 56.1% (+4.7pp, p=0.383p = 0.383). It edits the context in only 6 of 107 conversations, so the gain comes almost entirely from conversations it never touches; forcing engagement at w=2w=2 makes it worse (47.7%). Length-triggered schedules cannot reach the early pollution that LiC produces. We report this as scoped evidence on short conversations, not a verdict on MT-OSC’s long-horizon regime. Both baselines were run on LiC only.

CollabLLM

CollabLLM sits near the middle of the referentiality spectrum, and its results are mixed. The GPT-5-mini run uses an earlier single-pass Rewrite over the full transcript, without structural exclusion; the three-respondent runs of Table 2 use the standard analysis and the operator rule of § 3.2. Averaged over two GPT-5-mini runs, Rewrite is below full context on MATH-Hard (45.0 vs. 50.0) and above it on BigCodeBench (71.2 vs. 67.5, scored by a conversation-aware GPT-5 judge), where AO scores higher still (85.0); one of the two runs alone was above full context on both tasks (Appendix § 10.§ 5). Across the three additional respondents (Table 2), Reset, the one operator that acts in most CollabLLM conversations, scores 62.5 against AO’s 60.0 and full context’s 55.0 on gpt-5.4, 55.0 against 52.5 and 52.5 on DeepSeek-V4-Flash, and 59.9 against 62.5 and 60.0 on Kimi-K2.6. Each of these gaps is at most three problems out of 40. Augment and Rewrite act only from the fourth user turn, which few conversations reach (8 and 12 of 120), so their rows mostly measure full context under fresh sampling; this includes DeepSeek-V4-Flash’s highest row (Augment, 57.5), which acted in 3 of its 40 conversations. The clearest CollabLLM evidence is an execution-scored three-replicate run with DeepSeek-V4-Flash, in which Reset beats full context on BigCodeBench by +15pp in every replicate.

WildChat

Unlike LiC, WildChat contains many referential turns, so AO is no longer an oracle and can degrade quality. Following Huang et al. [ [5], we classify each turn as new_ask, feedback, or no_feedback (the user continues without explicitly referencing the assistant’s response, but often builds on it implicitly; definitions in Appendix § 10.§ 6). Phase 1 of the protocol reproduces their finding: AO helps on new_ask and feedback turns but hurts on referential no_feedback turns, where full context wins the pairwise comparison.

AC3 outperforms both AO and full context. On GPT-5-mini, Gated Reset achieves an 86.1% win rate vs. AO and 83.8% vs. full context while intervening on 72.3% of turns (Table 3); on the no_feedback turns, where AO is weakest, it still wins 85.1% vs. AO and 80.6% vs. full context, and its gate opens less often there (Appendix § 10.§ 6). Every printed cell in Table 3 lies between 71.6% and 91.5%. All of these are preferences of a single GPT-5-mini judge; their sensitivity to the judge is discussed in § 6. With DeepSeek V3.2 as respondent, Reset wins 72.5% vs. AO and 72.5% vs. full context (Table 9). Appendix G walks through a Maven-debugging conversation in which Gated Reset wins all 14 comparisons across 7 turns.

Conditionvs AOvs FCΔ\Delta vs AO
DeepSeek V3.2 (eval split)
Reset72.5%72.5%—
+ memory86.3%80.4%+13.8pp
GPT-5-mini (online)
Reset90.8%86.8%—
+ memory86.7%80.0%−4.1pp

Table 9. WildChat memory ablation. Indented + memory rows add the memory-based learner to the Reset row above. Memory improves DeepSeek’s win rate by +13.8pp when there is headroom; GPT-5-mini’s near-ceiling baseline leaves no room for improvement. For DeepSeek, memory was learned from one-third of WildChat turns and frozen for evaluation; GPT-5-mini uses the online regime. These rows use the evaluation split (DeepSeek) or the online regime (GPT-5-mini), so they are not directly comparable to Table 3.

Reset with and without a floor and cap on a strong respondent. By design, the two Reset variants differ only in the turn floor and the reset cap, since both act only when the analyzer flags an issue. On LiC they coincide: replay regenerates only the final turn, which is past the floor, and the cap never binds. They diverge on WildChat with gpt-5.4: Reset wins 88.6% vs. AO and Gated Reset 74.1%. On the 35 prefixes scored in both arms (of 44 and 58), the gap is −14.3pp but rests on seven discordant turns (6 vs. 1, exact McNemar p=0.125p=0.125), so we read it as a direction. As run on WildChat, however, the two arms also differ in the Subtask 2 prompt revision and the reset template, so the gap cannot be attributed to gating alone.

tau2-bench

AO collapses to 0% on tau2-bench for every respondent we evaluated (Table 2, tau2 block). Stripping assistant messages erases every diagnostic result the agent has fetched, so the agent repeats its lookups until the step budget ends the episode; omission is incompatible with stateful tool use in this setting. The collapse holds at N=3N=3 across 9 cells and 171 rollouts (170 scored) without a single non-zero reward, through a confirmed step-budget (max_steps) termination path, and is significant against full context for every respondent (paired −68.4 / −69.6 / −78.9pp over completed rollouts, p<0.0001p < 0.0001; one DeepSeek-V4-Flash AO rollout errored).

No AC3 operator improves over full context. Across the three-respondent matrix at N=3N=3 (855 rollouts, one of which errored), full context scores 68.4±13.968.4 \pm13.9 (gpt-5.4), 70.2±11.070.2 \pm11.0 (DeepSeek-V4-Flash), and 78.9±0.078.9 \pm0.0 (Kimi-K2.6), and no AC3 operator improves on it. Among the AC3 operators,

Gated Reset is the strongest or tied-strongest on all three respondents (57.9 ± 21.1 / 57.9 ± 10.5 / 71.9 ± 11.0) and stays within roughly 7 to 12pp of full context (all p>0.14p > 0.14), while Augment is significantly worse on all three. Single runs on this benchmark can be unrepresentative: in an earlier single run on gpt-5.4, Augment and Rewrite scored 84.2 and 73.7, against 47.4 for each at N=3N=3, while full context and Gated Reset gave similar values in both. We found no cause (we ruled out model substitution, gating failure, degenerate termination, and rate limits), which is one reason we report only N=3N=3 means.

On GPT-5-mini (Appendix § 10.§ 8), Gated Reset likewise stays within trial noise of full context across 3 seeds. Across all four respondents the load-bearing finding is the same: AO collapses whenever tool-call results live only in assistant turns, while AC3 does not collapse, although it does not exceed full context.

Pollution is contagious

Letting the specification extractor see assistant messages erases most of the analyzer’s gain (Table 4: 80%→40% on math, 53%→11% on code, 32%→8% on database; single trials), to or below the full-context level on math and code. Even a model explicitly tasked as an independent reviewer anchors on the assistant’s flawed reasoning. The polluted single-pass variant then beats the polluted two-step one on math and code (55% vs. 40% on math), consistent with the second subtask treating the biased specification as ground truth. Decomposition helps only when earlier stages produce clean outputs, a structural data-flow constraint that prompting alone does not enforce (Appendix § 13).

ConfigurationMathCodeDatabase
AC3-Augment80%53%32%
− structural exclusion40%11%8%
− subtask decomposition55%21%4%
Full context (no analysis)60%16%4%

Table 4. Progressive ablation of the analyzer (Augment, no memory; single trials on the Table 1 pools). Row 2 lets the specification extractor see assistant messages; row 3 also collapses the two subtasks into a single pass.

The analyzer as a pollution detector

To test whether the analyzer identifies harmful context rather than editing arbitrarily, we evaluate it directly as a detector, with no judge in the label path so that the evaluation cannot be circular. We report two studies in full, including the results that are unfavorable to the method.

Tier A: constructed injection, ground-truth labels

Into 145 LiC database and code conversations we inject two assistant-side spans, one known-false and one known-true, in an identical frame, so labels are correct by construction. Four offline controls (identity editor 0.00 removal; hand-removal oracle 1.00; two delete-everything editors 1.00 removal, 0.00 preservation) show that preservation, not removal, is the metric that cannot be gamed. On the probe-admissible set (n=126n=126 conversations that pass a mechanistic check that the span probe is valid), AC3-Reset removes the injected pollutant 97.6% of the time (123/126) and names it explicitly in its issues output 78.6% of the time (89.7% on the causally harmful subset), at a gate sensitivity of 98.4% (against a 96.8% gate-open rate on the clean arm, so the gate itself barely discriminates); the injected true span is also named as an issue 42.1% of the time. Detection of the pollutant is reliable, but selectivity is absent: preservation is 4.0% and edit precision sits at chance, because Reset discards the assistant side wholesale. Rewrite removes only 27.0% here, so the metric is not saturated by construction. A factorial on the causally validated harmful subset (n=97n=97) shows the accuracy consequence: full context scores 24.7% clean, drops to 9.3% with the injected pollutant present, and recovers to 59.8% under AC3-Reset with that pollutant still in the history.

Tier B: causal span ablation, no LLM in the label path

Constructed spans may be more salient than natural pollution, so we also run a counterfactual study. For 111 spans occurring naturally in 30 LiC conversations (the assistant’s own prose and code, nothing injected), we re-ran the final turn with each span present and with it removed, byte-identical otherwise (3,357 assistant turns, 0 errors); a span is causally harmful if removing it raises accuracy. The label is a measured accuracy difference, with no detector, judge, or LLM anywhere in the label path, and three controls pass in both directions (a contentless span +0.033, n.s.; the injected pollutant +0.368; a full-specification-plus-gold-SQL span −0.447). Natural pollution is real and concentrated: the spread of per-span effects exceeds a replicate-matched null (SD 0.155 vs. 0.125, p=0.0085p = 0.0085; 16 spans with ∣Δ∣≥0.25|\Delta| \ge0.25 where the null predicts 9.3, p=0.017p = 0.017), while the mean span is inert (+0.020). Pollution is carried by a minority of spans rather than spread evenly over the history. AC3 removes all 7 probe-admissible spans labelled harmful (100%), as expected of an operator that discards most assistant content.

We are explicit about what this does not show. Neither operator is span-selective: Reset keeps only 5 of 66 probe-admissible spans and Rewrite 0 of 66, and preservation of causally useful spans is 0%. A label-free test agrees: the mean effect of the spans Reset removed minus those it kept is −0.014 (permutation p=0.85p = 0.85), where a selective editor would score positive. The correct statement of the mechanism is therefore not surgical span preservation but detect, discard the assistant side, and rebuild the specification from the user side; the accuracy gain comes from that rebuild. This also fixes the scope of the method’s safety: structural exclusion is safe exactly where user turns independently specify the task. Where they do not, referents can reach the rebuilt context only through Subtask 2’s aligned record, which does see the assistant’s turns.

Analyzer robustness across model families

Because the analyzer is the one component shared by every operator, we swap it across five models from four families while holding the respondent and operator fixed (DeepSeek-V4-Flash, Reset, LiC code and database). Every analyzer produces a positive, individually significant gain over full context, from +12.9pp (gpt-4o-mini) to +39.9pp (Kimi-K2.6), and the ordering tracks capability rather than provenance: weak analyzers under-detect rather than mis-detect (details in Appendix § 10.§ 3).

Discussion

One analyzer, three operators. The WildChat results (Table 3) show that the best operator depends on the respondent while the analysis feeding the operators is shared. With Kimi-K2.6, Rewrite wins 91.5% vs. AO, ahead of Augment (82.9%) and Reset (71.6%), plausibly because Reset’s template strips conversational structure the next turn refers to; with gpt-5.4 the order reverses (Reset 88.6%, Rewrite 83.3%), possibly because Reset passes the analyzer’s specification through without a second LLM call. On tau2-bench no operator beats full context and Augment is significantly worse than Gated Reset’s parity, so we read no direction for operator intensity from the agentic setting beyond preferring the conservative operator. We therefore present AC3 as decoupling the analysis (one shared analyzer) from the intervention intensity (a per-deployment knob), rather than as one operator that is best everywhere; § 3.2 gives our default.

Memory is a direction, not a measured effect. The optional memory (§ 3.3) is not a measured effect. Its single-trial LiC rows sit above Augment on math, code, and database (Table 1), and on WildChat it improves DeepSeek V3.2’s Reset by +13.8pp but not near-ceiling GPT-5-mini (−4.1pp; Table 9). But a remeasurement of the database cell over four trajectory orderings (gpt-5.4-mini respondent) found memory 8.0pp below no memory on average, with a standard deviation of roughly 6pp. Any gain is not explained by leakage: the learn set does not overlap the LiC evaluation set, and a direct test of the transductive online protocol found no effect (Appendix § 14).

Limitations. Our LiC results replay difficulty-selected instances, so they measure recovery from a fixed polluted history on hard items; the unselected-pool results (§ 5.1) are the exception. The WildChat preference is that of a GPT-5-mini judge: one cross-family check agrees with it on 87.5% of pairs, but in another check three other judges preferred AO over Reset on the same responses, and with GPT-5 as a self-judged respondent AC3 did not beat AO or full context (Appendix § 10.§ 6).

In the fully simulated referential benchmarks (CollabLLM, tau2-bench) AC3 does not reliably beat full context. LiC, CollabLLM, and tau2-bench use simulated users, and most LiC, CollabLLM, and WildChat cells are single trials; our LiC and CollabLLM replicates vary only temperature-1.0 sampling for a fixed history, not the user simulator. AC3 costs two to three extra LLM calls per analyzed turn (Appendix § 10); folding the analysis into the base model by self-distillation [33, 6, 14] is untested. The instantiation differs in documented ways across benchmarks (§ 4), and we have not measured specification extraction on turns whose referent lives only in assistant turns. Long-horizon tasks and broader agentic domains remain untested.

Related work

Multi-turn degradation and context pollution. Laban et al. [ [9] introduced LiC; Huang et al. [ [5] named the failure mode context pollution and showed that AO recovers much of the lost performance; ERGO [8] resets the conversation through an LLM rewrite of prior user turns when output entropy spikes. These interventions decide when to act, but once triggered they remove or rewrite context wholesale, and none audits the assistant’s prior work against a user-grounded specification. Wholesale removal works where turns are self-contained (the LiC setting, where AO is a design oracle); on referential turns omission degrades quality, and in stateful tool use it collapses (§ 5.5).

Agentic context management. Concurrent work on long-horizon tasks (THREAD [13], Recursive Language Models [30], Context Folding [18], U-Fold [17], MemoBrain [12]) targets compaction, fitting more reasoning into a bounded context; we address context pollution, where prior outputs mislead generation even within ample context. System 2 Attention [24] regenerates the context to strip biasing content before answering, in a single call and without auditing prior work against a user-only specification. MT-OSC [15], which condenses multi-turn chat history, is closest in setting, and we compare against it in § 5.2. Detailed comparisons and additional related work are in Appendices § 16–§ 17.

Conclusion

We proposed Agentic Conversation Context Curation (AC3), a training-free method in which an analyzer, shielded from the assistant’s turns, rebuilds the task specification from the user side and edits the context around it. On difficulty-selected LiC instances it closes 55–84% of the gap to the design oracle on three tasks and exceeds it on the fourth; a GPT-5-mini judge prefers it over full context and AO in every configuration of Table 3; CollabLLM is mixed; and on tau2-bench, where AO collapses to 0%, AC3 does not collapse but does not exceed full context. Its operators are not span-selective, so the gain comes from the rebuild.

Two lessons follow. First, pollution is contagious: even a separate reviewer model anchors on the assistant’s reasoning if it can see it, and in our experiments only structural exclusion, not prompting, prevented this. Second, the analyzer is shared but the best operator depends on the respondent, a per-deployment choice. Broader end-to-end, long-horizon, and real-user tests come next.

Algorithm

Algorithm 1 Agentic Conversation Context Curation (AC3)

Require: Conversation context Ct=(u1,a1,…,ut)C_t = (u_1, a_1, \ldots, u_t), operator o∈{Augment,Reset,Rewrite}o \in\{\mathrm{Augment}, \mathrm{Reset}, \mathrm{Rewrite}\}

1: // Subtask 1: Specification extraction 2: Ut←(u1,…,ut)U_t \leftarrow(u_1, \ldots, u_t) ▷ Structurally exclude At−1A_{t-1} 3: T^←ANALYZE⁡(Ut)\widehat{T} \leftarrow\operatorname{ANALYZE}(U_t) ▷ Extract clean task specification 4: // Subtask 2: Approach evaluation 5: (aligned,issues)←EVALUATE⁡(T^,Ct)(\mathrm{aligned}, \mathrm{issues}) \leftarrow\operatorname{EVALUATE}(\widehat{T}, C_t) 6: if o∈{Reset,Rewrite}o \in\{\mathrm{Reset}, \mathrm{Rewrite}\} and issues=∅\mathrm{issues} = \varnothing then 7: &nbsp;&nbsp;&nbsp;&nbsp;return CtC_t ▷ No edit needed (pass-through) 8: end if 9: // Apply editing operator E\mathcal{E} 10: Ct′←INTERVENE⁡(T^,aligned,issues,Ct)C'_t \leftarrow\operatorname{INTERVENE}(\widehat{T}, \mathrm{aligned}, \mathrm{issues}, C_t) 11: return Ct′C'_t

Evaluation details

LiC evaluation adjustments

We make task-specific adjustments to the LiC evaluation protocol to reduce false negatives caused by overly strict extraction and ambiguous formatting, and apply a post-hoc false negative identification procedure across all tasks.

Replay-mode protocol. To isolate intervention effects, we run all LiC strategies in replay mode: every strategy shares the same baseline conversation prefix and only the final turns are regenerated under each strategy. Cell-to-cell comparisons therefore measure what an intervention recovers from a fixed history rather than confounding strategy effects with divergent rollouts.

Sampling parameters. All LLM calls (assistant, user-simulator, system-agent, and analyzer) use temperature =1.0= 1.0 with default top-p. This is the reasoning-model default: GPT-5-family models silently override any temperature=0.0 request to 1.0, so we use 1.0 throughout for consistency. We do not pass a reasoning_effort parameter for analyzer calls (constraining reasoning effort severely degrades analyzer quality on GPT-5-mini in our pilots). The same sampling configuration is used both when generating the original baseline trajectory and when each strategy regenerates the final turn(s) under replay mode.

API cost. All experiments are inference-only and were run via commercial APIs (OpenAI, including Azure OpenAI) and hosted endpoints for open-weight and third-party models (Azure AI Foundry for DeepSeek-V4-Flash and Kimi-K2.6; other providers for DeepSeek V3.2, Qwen, and Llama). We report per-run costs rather than totals so the figures translate directly to reproduction budget. With gpt-5-mini as both assistant and analyzer, a single Gated Reset replay-mode rerun of an LiC task (n=18n=18–2525 samples) costs approximately $0.04–$0.06; tau2-bench is more expensive per task because conversations are longer and tool-call payloads inflate token usage, at roughly $0.70 per trial of the 20-task telecom_small subset (19 scored tasks in the multi-respondent runs), with the Gated Reset variant adding ∼\sim20% over the Baseline due to the analyzer’s 2–3 extra LLM calls per turn. WildChat is scored by GPT-5-mini pairwise judge calls and the CollabLLM GPT-5-mini BigCodeBench results by a GPT-5 conversation judge, over pre-generated assistant responses, so per-experiment cost is dominated by judge tokens rather than full multi-turn simulation; a single judged WildChat condition over n≈175n \approx175 turns costs a few dollars. Despite our scaled intervention adding 2–3 LLM calls per turn, our project-level total is well under the ∼\sim$5,000 envelope reported by Laban et al. [ [9]: Laban et al. ran six task domains across fifteen models with full multi-turn simulation, while our headline tables span four LiC tasks (with replay mode amortizing the baseline trajectories), three CollabLLM/WildChat conditions, one tau2-bench subset, and several model families.

Task-specific prompt and extraction changes

Math. The original LiC math evaluator uses symbolic answer extraction that is overly strict, producing frequent false negatives when the assistant’s final answer is correct but formatted unexpectedly. Since GSM8K problems have integer-only solutions, we simplify extraction by requiring a standardized answer prefix. Our v2 system prompt adds the instruction ANSWER: <number>, making extraction a simple regex match on a known prefix. This eliminates false negatives from formatting variation while remaining faithful to the original task.

Code. The original HumanEval system prompt does not specify how the assistant should structure its code output. In the multi-turn setting, the assistant sometimes omits the function definition entirely (inlining the logic), or assumes input comes from stdin rather than function parameters, because the sharded problem description does not always convey the expected function signature. Our v2 system prompt explicitly requires: (1) exactly one top-level function definition, (2) inputs as function arguments rather than stdin, and (3) a markdown Python code fence for the final answer. We also relaxed the extraction code to handle minor formatting variations (e.g., extra whitespace, trailing comments).

Database. Similar to code, the original database prompt does not specify an output format, making SQL extraction fragile. Our v2 prompt requires the final query in a ```sql markdown code fence, with the constraint that only one such fence should be used for the final answer.

Actions (function calling). This is the only LiC task that expects multiple output artifacts (a list of function calls). Because shards are revealed incrementally, the assistant sometimes reasonably assumes it does not need to repeat previously established function calls in its final response, but evaluation only considers the last assistant message. Our v2 system prompt adds an explicit instruction to output all function calls needed to fulfill the user’s complete request at each turn, ensuring the evaluated response contains the full set.

False negative identification

Through manual trace analysis, we found that the user simulator model (which converts problem shards into natural-language user messages) sometimes rephrases shards in ways that subtly distort the original meaning, introducing ambiguity, altering constraints, or omitting critical details. While LiC controls for the sharding process by verifying that concatenated-shard performance approximates single-turn performance, it does not control for the user simulator’s rephrasing and ordering of those shards.

We account for this with a pool-level pre-filter, computed once and applied identically to every arm. For each instance where the full-context baseline assistant fails, we collate the user-role messages of that baseline trace and compare them against the original single-turn question. We query GPT-5 (Prompt 7, Appendix § 11) to determine whether those messages, taken as a union, provide sufficient information to solve the original problem. The resulting per-task list of instance IDs is stored alongside the replay pool and every strategy is evaluated on the same reduced pool, which is why all rows of Table 1 share the denominators n=20/19/25/23n=20/19/25/23 (math 23–3, code 25–6, database 25–0; the actions column is normalized to a common n=23n=23 samples, except the Get Read row, which uses raw n=25n=25; Appendix § 10.1.3).

We deliberately do not recompute this classifier per run. A reset or rewrite arm stores the edited context, in which the original user turns have been consolidated into a single specification message; the classifier would then see about one user turn per sample on those arms against about five on the full-context baseline, and would exclude their failures at a far higher rate. Computing the filter once on baseline traces removes that asymmetry by construction: it is a property of the instance, not of the arm.

Instances flagged as insufficient are excluded from the evaluation pool for all arms, as the failure is attributable to the evaluation setup rather than to context pollution or the intervention strategy. All accuracies in Table 1 and the main text are raw accuracies on that common pool, with no per-run adjustment (the exploratory soft-attention splits of Table 8 are the one exception, noted in its caption). Manual analysis of the classifier’s outputs finds it to be higher-precision, lower-recall: it conservatively flags only clear cases of information loss or distortion. We consider this acceptable because we prefer to allow some ambiguity to the assistant (counting borderline cases as valid) rather than overly invalidating results.

ConditionMathCodeDatabase
Augment
Soft attention (no memory)40%13%31%
Soft attention + memory56%14%46%
Hard attention80%40%54%
Reset
Soft attention (no memory)42%23%31%
Soft attention + memory42%25%46%
Hard attention67%46%54%

Table 8. Soft- vs. hard-attention results on held-out test splits (n≈7n \approx7–13 per cell). Some math and code cells use a per-run false-negative adjustment that we do not use elsewhere and that favors reset arms, so compare within a row family only. Memory closes up to 65% of the gap to hard attention on database.

Variance of the default operator

We do not report error bars on every cell of Table 1 (see Limitations), but we did spot-check that the headline LiC ordering is not a lucky-run artifact. We re-ran the default operator (Gated Reset) two additional times across all four LiC tasks, giving N=3N=3 per cell when combined with the headline run. We focus the variance budget on a single complete row rather than scattered cells, both so the row is coherent and because Gated Reset is our default operator. Two additional implicit checks come for free from the experiment design: (i) each of AC3’s operator variants (Augment, Reset, Gated Reset, plus their +Memory counterparts) reuses the same upstream analyzer artifact per question, so the four-row ordering across operators is also a check on robustness across different operationalizations of the same analysis step, and Gated Reset clears the full-context baseline on every LiC task in this row, mirroring the variance result; (ii) the multi-model results (Table 6) replay the same protocol across four models from three families and recover the same qualitative ordering in 15 of 16 cells, providing a cross-model robustness check on the analysis step itself.

Protocol. We use replay mode (last-turn regeneration with fixed user-sim trajectory loaded from saved full-context baseline traces). This isolates analyzer + assistant variance (the user simulator’s shard-reveal trajectory is held constant across the three runs) and so likely underestimates full end-to-end variance, which would also include user-sim stochasticity. The configuration matches the headline run for each cell, replaying the same saved baseline traces. On actions, the rebuilt context also restates the system prompt’s instruction to repeat all function calls in the final turn; the other rows receive that instruction only in the system prompt. All runs use gpt-5-mini as both assistant and analyzer at temperature 1.0.

Results. Per-run accuracies and aggregate statistics are below (raw nn, no auxiliary normalization):

TaskHeadlineRun 2Run 3MeanStd
math (n=20n=20)75.080.085.080.05.0
code (n=18n=18–19)72.263.257.964.47.2
database (n=25n=25)44.032.040.038.76.1
actions (n=25n=25)68.056.060.061.36.1

Table 1 (PDF p. 18).

Reading. Std is in the 5–7pp range. Means clear the full-context baseline (60.0/15.8/4.0/34.8) on every task and match or exceed the Augment operator without memory (80.0/52.6/32.0/47.8): 80.0 ±\pm 5.0 vs Augment 80.0 (parity, expected, since math is the easy direct direction); 64.4 ±\pm 7.2 vs Augment 52.6; 38.7 ±\pm 6.1 vs Augment 32.0; 61.3 ±\pm 6.1 vs Augment 47.8. The qualitative ordering (Gated Reset substantially above the full-context baseline; Reset/Gated Reset above Augment on context-removal-sensitive tasks, code/database/actions) is preserved. Specific point-estimate gap-closure percentages cited in the body (e.g., “84% of the gap on code”) should be read as derived from these means.

What this does and does not establish. This is variance under fixed user-sim trajectories with the analyzer and assistant resampled. It does not estimate variance from re-rolling the user simulator (which would require full simulation, not replay). Reruns of the other ours-rows (Augment, Reset, +Memory) are left to future work; the choice to anchor variance evidence on Gated Reset reflects that it is our default operator (§ 3.2).

Notes on Table 1

StrategyMath ↑Code ↑Database ↑Actions ↑
Prior work & baselines
Baseline (full context)60.015.84.034.8
+ Memory55.021.14.034.8
AO [5]⋄85.073.732.082.6
Concat User [9]⋄80.068.432.087.0
ERGO [8]§80.043.912.043.5–52.2
Agentic Conversation Context Curation (ours)
AC3-Augment80.052.632.047.8
+ Memory90.068.444.047.8
AC3-Reset75.057.948.052.2
+ Memory85.068.444.052.2
AC3-Gated-Reset‡80.064.438.761.3

Table 1. Per-task LiC results (GPT-5-mini, accuracy %, replay mode). Instances are difficulty-selected (those the full-context baseline most often failed), so absolute accuracies are low by construction and are not estimates of GPT-5-mini’s multi-turn accuracy. ⋄Design oracles: single-turn by construction. ‡Mean of N=3N=3 runs; other rows are single trials. §Interval; see Appendix § 10.1.4, which also gives pools, denominators, and paired tests.

Pools and denominators. The full LiC pools have 103 (math), 100 (code), 107 (database), and 105 (actions) instances. Difficulty selection keeps instances that the full-context baseline answered incorrectly in at least 60% of five independent baseline runs (four for code) and takes the 25 with the highest error rate (math had only 23 eligible instances, so gap-closure percentages are measured against a same-pool ceiling rather than against 100%). The shared pre-filter (Appendix § 10.1.2) then removes 3, 6, and 0 instances from math, code, and database (n=23n=23–3, 25–6, 25–0). Actions has no filter artifact: its 23 is a legacy “common 23-sample” normalization whose exclusion list is not recoverable, applied to every row except Gated Reset. The Gated Reset row uses raw nn per run (Appendix § 10.1.3): the actions cell uses n=25n=25 rather than the column’s normalized 23, and the three code runs use n=18n=18, 19, and 19. The AO, Augment, and Reset code cells come from runs that scored 18 instances and are printed on n=19n=19 by counting the unscored instance as a failure, the conservative end of each cell’s range; the Augment and Reset code cells are single trials; only the ERGO code cell is a mean over three replicate measurements of the pruned-item split.

ERGO scoring. ERGO is scored on the same pre-filtered pools as every other row (n=20/19/25/23n=20/19/25/23). The actions filter has no recoverable exclusion list, so the ERGO actions cell is printed as the interval that the unknown filter spans. Paired exact sign tests on the same items find no ERGO-vs-AC3 difference in this table statistically distinguishable at n≈20n\approx20 in either direction on code (p=0.375p=0.375) or math (p=1.00p=1.00), the two tasks we tested; at this sample size the table does not settle these orderings.

Design oracles on the unselected pool. The ◇ label describes the LiC construction (each instance is a single-turn question sharded across turns), not a measured ceiling. Measured end-to-end on the complete, unselected database pool (n=107n=107; assistant gpt-5.4-mini, § 5.2), AO reaches 69.2% and Concat User 63.6% against a fully specified single-turn ceiling of 94.4%, with AC3-Reset at 75.7% above both.

Operator activation and serving

§ 3.2 states one rule for when operators act: Augment always fires after its warm-up, and Reset and Rewrite act only when the analyzer flags an issue. Every LiC, CollabLLM, and WildChat cell of Table 2 and Table 3 follows it. Most multi-respondent runs were served through Azure OpenAI and Azure AI Foundry; the remaining instances described below were served through OpenRouter with the same prompts, prefixes, configurations, and judge, with DeepSeek-V4-Flash pinned to a non-thinking endpoint and Kimi-K2.6 to a thinking endpoint to match the Azure deployments. Within a column of Table 2, cells may therefore be served by different providers of the same model.

LiC Rewrite. On 58 of the Rewrite samples in Table 2 (8 for dsv4f, 14 for gpt-5.4, 36 for k2.6) the analyzer reported no issues, so Rewrite leaves the context unchanged and the final turn is generated from the full context (OpenRouter). The gpt-5.4 and k2.6 cells average per-prefix accuracy over the three prefixes with errored samples excluded, and the dsv4f cells pool the prefixes and count errored samples as incorrect; on the Rewrite row the two conventions differ by at most 0.7pp.

WildChat Augment. On 23 of the Augment turns in Table 3 (6 for gpt-5.4, 11 for dsv4f, 6 for k2.6) the analyzer reported no issues. These turns are generated with the analysis appended (OpenRouter) and judged against the same AO and full-context responses with the same judge and prompt.

CollabLLM. CollabLLM conversations end when the simulated user is satisfied, so Augment and Rewrite, which start at the fourth user turn, rarely act, and Reset, which starts at the second, acts in most conversations. Under Azure, the analyzer call failed (a provider content filter or an endpoint parsing error) in every conversation that reached the operator in twelve cells: the Rewrite row, the gpt-5.4 and k2.6 Augment BigCodeBench cells, and the gpt-5.4 and k2.6 Reset cells. These cells were run end to end through OpenRouter with no exclusions, apart from one conversation that failed at its first turn, before any operator could act. All BigCodeBench answers are scored in the same execution sandbox, and a k2.6 AO conversation that the simulated user ended before its first turn is counted. Table 5 gives the per-task results. Across the three respondents, Reset acted in 80 of 119 scored conversations, Augment in 8 of 120, and Rewrite in 12 of 120.

Strategygpt-5.4dsv4fk2.6
Baseline95.0 / 15.095.0 / 10.0100.0 / 20.0
AO95.0 / 25.090.0 / 15.0100.0 / 25.0
AC3-Augment95.0 / 10.0‡100.0 / 15.095.0 / 15.0‡
AC3-Reset100.0‡ / 25.0‡85.0 / 25.094.7‡ / 25.0‡
AC3-Rewrite90.0‡ / 25.0‡85.0‡ / 20.0‡95.0‡ / 20.0‡

Table 5. CollabLLM per-task accuracy (MATH-Hard / BigCodeBench, n=20n=20 per cell except k2.6 Reset MATH-Hard, n=19n=19), with all BigCodeBench cells execution-scored in one sandbox. ‡Served through OpenRouter (Appendix § 10.§ 2); unmarked cells were served through Azure. Baseline is full context. The two-task averages are the CollabLLM block of Table 2.

Analyzer robustness: full results

Because the analyzer is the one component shared by every operator, we test whether AC3 depends on a particular analyzer model. We hold the respondent and operator fixed (DeepSeek-V4-Flash, Reset; LiC code and database, one replay prefix of the GPT-5.2-selected pool, two replicates pooled; n=178n=178 matched pairs, full-context baseline 21.3%, exact McNemar, which treats the pooled replicates as independent and is therefore optimistic) and swap only the analyzer across five models from four families. Every analyzer produces a positive, individually significant gain over full context (all

Table 5: CollabLLM per-task accuracy (MATH-Hard / BigCodeBench, n=20n=20 per cell except k2.6 Reset MATH-Hard, n=19n=19), with all BigCodeBench cells execution-scored in one sandbox. ‡Served through OpenRouter (Appendix § 10.§ 2); unmarked cells were served through Azure. Baseline is full context. The two-task averages are the CollabLLM block of Table 2.

p≤8.2×10−4p \le8.2 \times10^{-4}; per-analyzer tests below): Kimi-K2.6 (Moonshot) +39.9pp, DeepSeek-V4-Flash +28.7pp, gpt-5.4-mini +27.0pp, Llama-3.3-70B (Meta) +18.0pp, and gpt-4o-mini +12.9pp. No configuration falls below baseline on either task in either replicate; the weakest analyzer retains 32% of the best analyzer’s gain and still beats full context by 12.9pp. Degradation is graceful because weak analyzers under-detect rather than mis-detect (gpt-4o-mini declines to edit on a quarter of turns, where strong analyzers edit on ∼97%\sim97\%), and the ordering tracks capability rather than provenance: the best analyzer is non-OpenAI and the weakest is OpenAI.

Exact McNemar tests against full context: Kimi-K2.6 p=2.3×10−17p=2.3 \times10^{-17}, DeepSeek-V4-Flash p=3.0×10−9p=3.0 \times10^{-9}, gpt-5.4-mini p=1.4×10−8p=1.4 \times10^{-8}, Llama-3.3-70B p=5.6×10−6p=5.6 \times10^{-6}, and gpt-4o-mini p=8.2×10−4p=8.2 \times10^{-4}.

Multi-model evaluation protocol

Because stronger models solve easier problems regardless of context pollution, we reran the subset selection protocol using GPT-5.2 LiC logs (instead of GPT-5-mini), selecting the 20 instances per task that GPT-5.2 most often failed to ensure a challenging set for this model class. Each model serves as both assistant (final-turn regeneration) and analyzer. All models replay conversations originally generated by GPT-5.2; the Baseline reflects how model X handles GPT-5.2’s polluted context, while Reset reflects model X given a clean context derived from its own analysis. User and system agents use GPT-4o-mini throughout. We compare Baseline against Reset, one fixed operator chosen before the run. Denominators vary with evaluation errors (13–20 valid samples for code and database); percentages use valid denominators. We additionally evaluate GPT-5, DeepSeek V3.2 (37B active, open-weight), and Qwen 3.5 35B-A3B (3B active, open-weight).

CollabLLM evaluation details

MATH-Hard is scored by an LLM judge. The GPT-5-mini BigCodeBench numbers were scored by a conversation-aware GPT-5 judge that evaluates the assistant’s code against the user’s intent as expressed in the conversation (n=20n=20 per arm). These BigCodeBench numbers are therefore judged functional correctness, not execution pass rates; BigCodeBench’s execution harness (untrusted_check) also scores these conversations once the sandbox’s dependencies are installed, and all multi-respondent BigCodeBench cells (Table 5) use it. GPT-5-mini numbers on MATH-Hard / BigCodeBench averaged over two runs are Baseline 50.0% / 67.5%, Rewrite 45.0% / 71.2%, and AO 50.0% / 85.0%. In one of the two runs, Baseline scored 40.0% / 62.5% and Rewrite 45.0% / 82.5%. This Rewrite is a single pass over the full conversation (no structural exclusion). In a later three-replicate run with execution-based BigCodeBench scoring and DeepSeek-V4-Flash as user, assistant, and analyzer, AC3-Reset scored 21.7 ± 5.8% against full context’s 6.7 ± 5.8% (+15pp in every replicate), and on MATH-Hard, where conversations rarely reach Augment’s fourth-turn start, AC3-Augment matched full context (91.7% each). The multi-respondent CollabLLM averages are in the CollabLLM block of Table 2.

WildChat protocol and turn type definitions

Filtering and protocol. Following Huang et al. [ [5], we filter WildChat-1M to English, technical conversations of 5–10 rounds. The evaluation has two phases: Phase 1 identifies turns where AO degrades quality; Phase 2 evaluates our strategies on all turns using the pairwise LLM judge prompt of Huang et al. [ [5], which they validated against human annotators.

Turn sets in Table 3. The GPT-5-mini column uses the Phase-2 turns; the gpt-5.4, dsv4f, and k2.6 columns use the Phase-1 prefix set (n=44n=44–7676 per cell) with a fixed GPT-5-mini analyzer. Gated Reset was not run for dsv4f and k2.6; the four omitted vs-FC win rates lie between 60.8% and 82.9%.

Turn type definitions. WildChat turns are classified into three types based on the relationship between the current user message and prior conversation context: new_ask (the user introduces a new topic or question unrelated to prior turns), feedback (the user provides feedback on or corrections to the assistant’s prior response), and no_feedback (the user continues the conversation without explicitly referencing the assistant’s response, e.g., providing additional information or following up on their own prior message). AO is most harmful on no_feedback turns, where the assistant’s prior work contains relevant state that should not be discarded. On GPT-5-mini, Gated Reset’s gate opens on 78.1% of new_ask, 78.6% of feedback, and 62.7% of no_feedback turns (72.3% overall).

Judge position bias. The GPT-5-mini judge prefers the second-presented response, but the harness randomizes the A/B order on every call, so the reported win rates are unbiased estimates of the order-balanced quantity in expectation. In a separate seeded GPT-5-mini replication (452 pairs, each judged in both orders, on a turn set distinct from Table 3’s), explicitly order-balancing shifts the Reset and Augment win rates against AO by −2.0 and −0.9pp, to 87.8 ± 2.1% and 91.2 ± 2.1%. In that order-balanced Augment figure, the context is passed through unchanged on the 23 of 227 turns where the analyzer flagged no issues; appending the analysis on those turns, as § 3.2 specifies, gives a single-order win rate against AO of 93.4% (92.1% with pass-through), which we did not re-judge in both orders. A DeepSeek-V4-Flash judge on a shared 160-pair subset agrees with the GPT-5-mini judge on 87.5% of pairs.

Judge dependence. A separate sensitivity check points the other way. Re-judging the same DeepSeek V3.2 responses (evaluation split, n=51n=51, Reset vs. AO), GPT-5-mini preferred Reset, but three other judges (GPT-5, GPT-5.2, and MiniMax-M2.7) preferred AO. With GPT-5 as the respondent (self-judged, n=74n=74), Reset and Rewrite did not beat AO or full context. We therefore read the WildChat results as the preference of the GPT-5-mini judge, corroborated by the cross-family check above but not by every judge we tried; a human evaluation would settle it.

Win rates by turn type. On GPT-5-mini, Gated Reset’s win rate against AO is 87.5% on new_ask, 85.7% on feedback, and 85.1% on no_feedback turns; against full context it is 87.5%, 83.3%, and 80.6%.

tau2-bench agentic adaptation

For tau2-bench, we make three modifications: (1) the analyzer explicitly tracks environment state (account settings, diagnostic results) revealed through tool calls; (2) the analysis is reframed as “strategic reflection,” identifying unexplored diagnostic avenues rather than just flagging errors; and (3) resets are gated and capped at 3 per conversation to prevent infinite loops; Rewrite on tau2-bench uses the same turn floor and cap. The tau2-bench analysis is a re-implementation inside the tau2-bench agent, and its tool-call prompts are not reproduced here.

tau2-bench results and diagnostic analysis

Headline tau2-bench telecom_small results are reported in Table 2 (tau2 block); the per-trial GPT-5-mini breakdown follows (this cell uses all 20 tasks of the subset). Per-trial success rates across seeds 42–44 are: Baseline {45.0, 55.0, 60.0}%, Gated Reset {40.0, 65.0, 40.0}%, AO {0.0}% (single trial; reward is invariant to seed because no diagnostic state is preserved across an AO context, so re-running with a different seed produces the same 0%). Summarized as the mean over the three seeds, Baseline is 53.3±7.6%53.3 \pm7.6\% and Gated Reset 48.3±14.4%48.3 \pm14.4\%, while AO is 0% on every trial because tool-call results live only in assistant turns. Per-trial variance is wide for both Baseline and Gated Reset, and the two are statistically within trial noise of each other: on this stateful regime our method neither helps nor catastrophically hurts the agent’s success rate, at ∼+20%\sim+20\% cost per task. The headroom for a context-editing intervention may be small here: in one author’s single-trial reading of the 11 Baseline failures, without a rubric or a second annotator, only 1 looked pollution-driven (Table 7). We hypothesize tau2-bench’s customer-service policy structure (which lays out high-level diagnostic options for the agent to enumerate) already gives the agent a breadth-first re-entry mechanism around early incorrect paths, partially substituting for the role of an external editor. The decisive contrast in this row is that AO destroys all diagnostic state and fails completely, while AC3 does not collapse. One implementation defect affects every tau2-bench AC3 arm: when the analyzer answers Subtask 1 with a JSON object instead of the expected tag, the raw output is spliced into the agent’s briefing. This happened on 53% of analyzer calls in the multi-respondent sweep (66% on gpt-5.4 Augment); fixing it changed one cell by +2.3pp.

CategoryCountDescription
Task too hard4Agent never tries the right fix
Step budget / Hard persona4Correct approach but runs out of turns
Context pollution1Agent stuck in repetitive loop
User sim issue1User stopped after first message
Premature transfer1Agent transfers instead of troubleshooting

Table 7. Baseline failure-mode breakdown (11 failures; GPT-5-mini, one trial). One author’s reading without a rubric or a second annotator, so a hypothesis rather than a measured taxonomy.

Diagnostic analysis of baseline failures on tau2-bench reveals the distribution of failure modes:

If context pollution is indeed not the dominant failure mode here, the headroom for a pollution-targeted method is small, which is one reason we read tau2-bench as a viability test (does the method avoid the structural collapse that AO suffers?) rather than as an improvement test.

Prompt templates

We provide the full text of key prompts used in our system. Template variables are shown in braces (e.g., {system_message}). Evaluation prompts (the v2 task system prompts and the false-negative check) are in Appendix § 11.§ 4.

Conversation analysis prompts

Prompt 1: Subtask 1, Task specification extraction (hard attention).

Here is the system message that sets the context for this conversation:

<system_message>
{system_message}
</system_message>

Here are the user's messages from the conversation, in order:
{user_messages}

{memory_section}

Construct the complete, up-to-date user task specification from these messages. The
→ system message above defines the task format and constraints (e.g., write SQL, call
→ functions, write code) -- your specification should be grounded in that context.
→ Your goal is to produce a clear, organized specification that consolidates what the
→ user is asking for based on everything they have said so far. Note that the user may
→ continue to refine or extend their requirements in future messages -- this
→ specification reflects the current state of their request, not necessarily the final
→ version.

Guidelines:
- Consolidate and reword freely for clarity. You are writing a clean spec, not
→ transcribing the user's messages.
- Preserve crucial details: specific numbers, formulas, formats, concrete examples, edge
→ cases, constraints, and entity names. These must not be lost or altered.
- If a later message overrides or refines an earlier one, use the latest version only.
- Do not add interpretation or assumptions beyond what the user stated.
- If the user's messages describe multiple distinct deliverables (e.g., multiple
→ function calls, multiple queries, multiple independent outputs), list each as a
→ separate numbered item. Keep single-output tasks as a single specification.

Use this format for your answer:

<task_spec>
[The consolidated user task specification]
</task_spec>

The analyzer sees only the system message and user messages; assistant messages are structurally excluded. This is the structural exclusion tested in § 5.6 (called “hard attention” in Appendix § 12).

Prompt 2: Subtask 2, Approach evaluation.

Here is what the user is asking for:

<task_spec>
{task_spec}
</task_spec>

Here is the conversation so far:

<conversation>
{conversation}
</conversation>

{memory_section}

Compare the assistant's work against the user task specification so far. Be critical --
→ the assistant may have made assumptions early on that contradict requirements the
→ user specified later. Such content is actively harmful: if it remains in context,
→ the assistant will anchor on it and repeat the same mistakes.

If the user task specification contains multiple numbered items (i.e., the user expects
→ multiple distinct outputs), evaluate each item separately. Track which items the
→ assistant has completed correctly, which have errors, and which haven't been
→ attempted. Correctly completed items must be preserved even when other items need to
→ be redone.

Use this format for your answer:

<aligned>
What in the assistant's responses is consistent with the user task specification so far.
→ Be specific -- cite the concrete results the assistant produced that are correct.

If there are multiple items in the specification, list each correctly completed item
→ with the specific result the assistant produced.

</aligned>

<issues> What in the assistant's responses contradicts the user task specification so far, is built on unfounded assumptions, or is unnecessary complexity from earlier wrong attempts. Be specific. If nothing, write "None". If there are multiple items, reference which specific item(s) have problems. </issues>

Subtask 2 receives the clean task specification from Subtask 1 and the full conversation (including assistant messages). It evaluates alignment and identifies issues.

Context rewriting prompts

Prompt 3: Context rewriting, earlier version (Rewrite on GPT-5-mini WildChat and LiC ablations).

You are preparing compacted context for an AI assistant that will continue a multi-turn
conversation. The assistant will see ONLY what you produce -- it will not see the
original conversation history.

An independent reviewer analyzed the conversation and produced structured notes. Your
job is to use the conversation and notes to produce clean, structured context that
preserves every actionable detail.

## Reviewer's Notes (for your eyes only -- do NOT forward these verbatim)

### Task Specification (reviewer's interpretation)
{analysis_user_intent}

### What the Assistant Got Right
{analysis_aligned}

### Issues Identified
{analysis_issues}

## Full Conversation History
{conversation}

## Your Task

Produce compacted context with two sections. Be EXHAUSTIVE -- missing a single parameter
value, constraint, or sub-task will cause the assistant to fail.

### Section 1: Task Specification

Produce a structured, enumerated list of everything the user wants. For each distinct
sub-task or action:
- Number it explicitly (1, 2, 3, ...)
- State the exact function/operation required
- List every parameter with its exact value
- Include all constraints and conditions

Derive this from the user's actual messages. The reviewer's notes are guidance but may
be incomplete. Cross-check against the conversation to ensure nothing is dropped.

### Section 2: Verified Work

List any correct, completed work that should be preserved. Use the reviewer's "what
looks right" as guidance. Present it as structured data (exact values, function
calls, results), not prose descriptions. Write "None" if nothing is salvageable.

Output format -- use these exact tags:

<task_spec>
1) [First sub-task with exact function name and all parameters]
2) [Second sub-task with exact function name and all parameters]

… </task_spec>

<verified_work> [Structured listing of correct completed work, or "None"] </verified_work>

CRITICAL rules:

  • Every sub-task the user requested MUST appear as a numbered item. Count them against the conversation to verify completeness.

  • Preserve exact values: numbers, strings, column names, function signatures, formulas. Do not paraphrase values.

  • Do NOT include the reviewer's error descriptions or issues. The point is clean context.

  • Do NOT suggest asking the user for clarification.

  • Do NOT merge distinct sub-tasks into one item. If the user asked for 3 function calls, list 3 items.

Rewrite uses this prompt to produce the replacement context that the assistant sees in place of the original conversation history. Reset and Gated Reset make no such call: they fill a fixed template with the task specification, the aligned output, and the latest user message. (The WildChat Reset template additionally places the analyzer’s issues in the system prompt and inserts a short assistant acknowledgment.)

Prompt 4: Context rewriting, current version (Rewrite in Table 2 and the gpt-5.4, DeepSeek-V4-Flash, and Kimi-K2.6 columns of Table 3).

You are preparing a compacted context message for an AI assistant that will continue a multi-turn conversation. The assistant will see ONLY the message you produce, plus the system prompt and the most recent user message; it will not see the original conversation history.

An independent reviewer analyzed the conversation and produced structured notes (below). The reviewer's notes are your primary source of truth about what the user wants and what work is on track. The full conversation is also provided so you can preserve exact values, formulas, code, or wording the notes refer to. Trust the reviewer's interpretation rather than re-deriving it from raw messages.

Your role is to prepare context, not to solve the task. The assistant will produce the final response to the user; do not pre-compute the answer or write the assistant's reply on its behalf. Your job is selective attention: keep what helps, remove what distracts.

Your goal: produce a message that gives the assistant the best chance of correctly responding to the user's request(s). Include whatever you think will help; omit anything you think will distract or mislead (incorrect work, dead ends, abandoned approaches). The structure, length, sectioning, and formatting of your message are entirely up to you.

Reviewer's Notes

Task Specification (reviewer's consolidated interpretation)

{analysis_user_intent}

What the Assistant Got Right

{analysis_aligned}

Issues Identified

{analysis_issues}

Full Conversation History

{conversation}

You may think through your approach in free text first if it helps. When you are ready, wrap the final compacted context message in <new_context>...</new_context> tags. Only the content inside those tags will be shown to the assistant.

This version was selected among five candidate prompts (including one found by automatic prompt optimization) by accuracy on the DeepSeek-V4-Flash LiC pool of Table 2, which is also a pool on which it is reported.

Memory-based learning prompts

Prompt 5: Trajectory reflection (Reflect step).

Reflect on this conversation trajectory and produce a bullet list of takeaways for ↪
  improving the conversation analyzer.

## What the Analyzer Does

The analyzer independently reviews multi-turn conversations to identify where the ↪
  assistant's approach has diverged from what the user actually asked for. It ↪
  produces:
↪  - **Task Spec**: Clean specification from user messages only (no assistant assumptions)
- **What Looks Right**: Parts of the assistant's approach that are correctly aligned
- **What Needs to Change**: Issues where the assistant contradicts user requirements

The analysis is then used in one of two ways:
- **S1 (Append)**: Analysis is shown to the assistant as additional context before its ↪
  next response.
↪  - **S2 (Context Edit)**: If issues are found, the old conversation is replaced with ↪
↪    compacted context (task spec + aligned work only), removing the assistant's wrong ↪
↪    reasoning so it can start fresh without anchoring on previous mistakes.

## Evaluation Environment

The user reveals problem information incrementally (sharded disclosure) and does NOT ↪
↪  answer clarifying questions. Asking for clarification wastes turns. Takeaways should ↪
↪  never recommend seeking clarification -- the assistant must work with available ↪
  information and refine as more details arrive.

## How to Read the Trace

The trace shows the conversation with [conversation analysis] blocks interleaved at the ↪
↪  point they occurred. For S2 traces, "Decision: EDIT/NO EDIT" shows whether context ↪
↪  was rewritten, and "NEW CONVERSATION" markers show where the old context was ↪
↪  discarded and replaced with compacted context.

## What Makes Good Takeaways

The analyzer's prompt is intentionally lean -- it defines WHAT to produce but not HOW to ↪
↪  analyze. The cheatsheet fills in the "how":
- **Sequencing**: e.g., "First gather all user-stated constraints, then check each ↪
↪    against the assistant's approach"
- **Diagnostic patterns**: e.g., "When you see X in the conversation, it usually means Y"
- **Pitfalls**: e.g., "The analyzer did X but should have done Y because..."
- **Successful strategies**: What worked well and is worth repeating

<trajectory>
Task: {task_name}
Sample ID: {sample_id}
Outcome: {outcome} (score: {score})
Number of turns: {num_turns}

{conversation}
</trajectory>

Bad takeaways (harm future performance):
- Algorithmic recipes or solution templates
- Problem-specific rules that won't transfer
- Generic advice ("be thorough", "check carefully", "pay attention")

Produce a concise bullet list of takeaways. Each must be a transferable analysis ↪
↪  principle -- something the analyzer can apply to ANY future conversation regardless ↪
↪  of domain.

Takeaways:

After each trajectory, this prompt extracts transferable analysis principles. The output is a bullet list of takeaways grounded in the specific trajectory but generalizable across domains.

Prompt 6: Cheatsheet unification (Unify step).

You are updating a memory cheatsheet by integrating takeaways from recent conversation ↪
  trajectories.

The cheatsheet is used by a conversation analyzer that reviews multi-turn conversations ↪
  to identify where an assistant's approach has diverged from user requirements. The ↪
  analyzer produces a task spec (from user messages only), identifies what's aligned, ↪
  and flags issues that need to change.

The analyzer's prompt defines the input/output format, but the cheatsheet controls the ↪
  "how" -- it can shape the analyzer's internal process:
- **Sequencing**: Specify subgoals and order (e.g., "first gather constraints, then ↪
  evaluate")
- **Diagnostic patterns**: What conversation signals indicate specific error types
- **Pitfalls**: Common analyzer mistakes and how to avoid them
- **Successful strategies**: Approaches that worked well and should be repeated

<current_cheatsheet>
{current_cheatsheet}
</current_cheatsheet>

<takeaways>
{takeaways}
</takeaways>

Integrate these takeaways into an updated cheatsheet.

SIZE LIMIT: The cheatsheet MUST be between 500 and 1500 words. If it would exceed 1500 ↪
  words, aggressively prioritize: keep only the most impactful, most frequently ↪
  relevant lessons. Drop or merge entries that are less likely to change behavior. ↪
  Quality over quantity.

CRITICAL: The cheatsheet must contain TRANSFERABLE ANALYSIS PRINCIPLES -- not solutions, ↪
  templates, or problem-specific recipes. Each entry should describe a pattern the ↪
  analyzer can apply to ANY future conversation, regardless of the domain or problem ↪
  type.

GOOD entries (transferable analysis principles):
- "When the user provides a concrete example, use it to resolve ambiguities definitively ↪
  -- derive the rule from the example rather than listing multiple possible ↪
  interpretations"
- "First build a checklist of all user-stated constraints, then verify each against the ↪
  assistant's current approach -- this prevents missing constraints buried in earlier ↪
  turns"

BAD entries (these cause regressions -- NEVER include):
- Algorithmic recipes or code snippets
- Prescriptive function signatures
- Problem-specific rules that won't transfer
- "Preserve key requirements" / "Remove irrelevant information" (too generic)

Guidelines:
1. Merge takeaways that express the same specific insight.
2. Resolve contradictions -- prefer newer evidence when takeaways conflict.
3. Preserve specificity -- "the analyzer missed a format mismatch across 3 resets" is ↪
  better than "check output format."
4. DROP entries that are algorithmic templates, code snippets, or problem-specific recipes.
5. Every entry should be applicable to a conversation the analyzer has never seen before.
6. Organize by theme (sequencing strategies, diagnostic patterns, pitfalls, etc.).

Updated cheatsheet:

The Unify step integrates individual trajectory takeaways into a single coherent cheatsheet, capped at 1500 words to prevent unbounded growth. The cheatsheet is then injected into the analyzer’s context via the {memory_section} template variable of Prompt 2; Prompt 1 has the same slot, which is used only in the specification-curation experiments of Appendix § 12.

Evaluation prompts

Prompt 7: False-negative check (Appendix § 10.1.2).

You are auditing an AI evaluation system. In this system a "user simulator" reveals a
problem to an assistant piece-by-piece across multiple turns. Your job is to decide
whether the user simulator's messages, taken together (i.e. their UNION), gave a
capable solver ALL the information needed to solve the problem correctly.

## Original Problem (complete, single-turn version)
{full_spec_q}

## Ground Truth Answer
{ground_truth_a}

## System Message (instructions/context given to the assistant)
{system_message}

## User Simulator Messages (all turns, in order)
{user_messages}

## Instructions
Compare the COMBINED content of the system message and all user messages against the
original problem.

IMPORTANT context: It is EXPECTED that information is split across turns. That is by
design, not a flaw. You must evaluate the UNION of all user messages -- treat them
as if you concatenated them into one block of text.

Mark the information as "sufficient" UNLESS one of these is true:
1. A critical detail (number, constraint, relationship, entity) from the original
   problem is completely absent from ALL user messages.
2. A critical detail was CHANGED in meaning (not merely reworded) in a way that makes
   the correct answer unreachable.

Do NOT flag:
- Information arriving late or in a different order than the original.
- Minor rewordings, paraphrasing, or splitting of sentences.
- Information that is inferable from the combination of user messages even if not stated
  in exactly the same words as the original.

Respond with a JSON object:
{
  "sufficient": <true if a capable solver who read ALL the user messages has enough
    info to reach the ground-truth answer>,
  "missing_elements": ["list ONLY details that are truly absent or materially
    distorted; empty list if sufficient"],
  "explanation": "one or two sentence summary"
}

The v2 task system prompts (Appendix § 10.1.1) follow. [[SCHEMA]] and [[FUNCTIONS]] are filled with the instance’s database schema and function list.

Prompt 8: Math system prompt (v2).

As an expert problem solver solve step by step the following mathematical questions.

When you have reached a final numerical answer, highlight it on its own line like this:
**ANSWER: 42**. Write only the number with no units or extra text.

Prompt 9: Code system prompt (v2). You are helping a user write a Python function to solve a programming question. If ↪ something is not clear, you can ask the user to clarify what they need. →

Format:

  • [Standalone] Your answer must contain exactly one top-level function definition. The → ↪ only other top-level statements allowed are import statements -- no helper → ↪ functions, constants, or other top-level code.

  • [Arguments] The function must accept inputs as function arguments, not by reading from → ↪ stdin.

  • [Code fence] Always wrap your final code in a markdown Python code fence: → ↪ python\n<your code here>\n

Prompt 10: Database system prompt (v2).

You are helping a user write SQL queries to a database. If something is not clear, you → ↪ can ask the user to clarify what they need. The schema for the database being → ↪ accessed is the following:

[[SCHEMA]]

When you have a final SQL query to present, always wrap it in a sql markdown code fence → ↪ like this:

SELECT ...

This makes it easy to identify your answer. Only use one ```sql fence for your final query.

Prompt 11: Actions system prompt (v2).

You are an expert in composing functions. You are given a question and a set of possible → ↪ functions. Based on the question, you will need to make one or more function/tool → ↪ calls to achieve the purpose. If none of the functions can be used, point it out. If the given question lacks the → ↪ parameters required by the function, also point it out. You should only return the function calls in your response.

If you decide to invoke any of the function(s), you MUST put it in the format of → ↪ [func_name1(params_name1=params_value1, params_name2=params_value2...), → ↪ func_name2(params)] You SHOULD NOT include any other text in the response.

At each turn, you should try your best to complete the tasks requested by the user → ↪ within the current turn. Continue to output functions to call until you have → ↪ fulfilled the user's request to the best of your ability. Once you have no more → ↪ functions to call, the system will consider the current turn complete and proceed to → ↪ the next turn or task.

Here is a list of functions in JSON format that you can invoke.

[[FUNCTIONS]]

Soft vs. hard attention

Structural exclusion of assistant messages (hard attention) exploits a property of LiC: user messages alone fully specify the task. In realistic settings, the task may depend on prior assistant work. For example, when a user says “sharpen the phrasing in sentence 2,” the referent exists only in the assistant’s output. Huang et al. [ [5] observe similar cases where user messages alone are insufficient for understanding the task.

We introduce a soft-attention variant where Subtask 1 sees the full conversation but explicitly reasons through source attribution: listing user-stated requirements with provenance, flagging assistant-originated interpretations as hypotheses, and writing the specification from user-grounded requirements only. We call this “soft attention” because the model must rely on transformer attention to selectively weight user-grounded information, rather than having polluting content physically removed from its context.

Hard attention (structural exclusion) substantially outperforms soft attention across all tasks (Table 8), indicating that prompting alone does not substitute for structural exclusion. Reset also outperforms Augment with polluted specifications (code: 23% vs. 13%), suggesting that context pollution and specification pollution are separable problems. These splits differ from Table 1’s pools, so absolute values are not comparable across the two tables.

Memory-based depollution (using hard-attention specifications as targets from which the Dynamic Cheatsheet learns principles for removing pollution) shows its strongest effect on database (+15pp), closing 65% of the gap to hard attention. The learned principles converge on structured heuristics: “build a user-only spec first, then overlay assistant suggestions as hypotheses.” Code remains resistant (+1–2pp), suggesting more structurally diverse contamination patterns that a minimal memory approach cannot address.

Structural-exclusion ablation (Table 4)

The headline numbers in the main paper used structural exclusion: Subtask 1 is given only user messages, and physically cannot see assistant turns. We progressively remove design choices to test whether this is load-bearing or whether prompt-level instructions (“please reason only from user turns”) could substitute. First, we drop hard attention (let Subtask 1 see assistant messages while keeping subtask decomposition); second, we additionally collapse the two-step analysis into a single pass.

Assistant messages pollute the analyzer. Letting the specification extractor see assistant messages drops performance below baseline on math and code (database is the exception, staying slightly above). Even a model explicitly instructed to act as an independent reviewer anchors on the assistant’s reasoning, biasing the extracted task specification.

A related ablation agrees with the rebuild account: Subtask 1 alone with structural exclusion (the specification, without the approach evaluation) scores above the full two-subtask Augment on code and database (63% and 40% in the same ablation run, vs. the two-subtask 53% and 32% of Table 4).

Pollution compounds across stages. On math and code, the single-pass variant outperforms the polluted two-step version. When Subtask 1 is polluted, Subtask 2 treats the biased specification as ground truth and amplifies the upstream error. Decomposition into stages is only beneficial when earlier stages produce clean outputs; otherwise, fewer stages may be safer than more. The takeaway transfers to any multi-stage LLM pipeline in which one model evaluates another’s output: structural data-flow control may be needed when prompting alone is insufficient for selective attention.

Memory-based learning details

We evaluate two learning regimes. Online: reflections update the cheatsheet incrementally as problems are solved in batches, so later problems benefit from accumulated principles, a form of continual test-time learning. Offline: a set of training trajectories is used to initialize the cheatsheet via a Reflect-then-Unify pipeline (individual reflections followed by a consolidation pass), and the frozen cheatsheet is applied at evaluation time. The offline regime provides a controlled comparison where the only variable is the presence of the cheatsheet. On LiC, we use online learning. On WildChat, we use the offline regime with a train/eval split (one-third training, two-thirds evaluation) to avoid confounding memory quality with ordering effects and to test cross-domain transfer. On LiC the online regime is transductive: the cheatsheet applied to a given instance is distilled from other instances of the same evaluation set, including their reference answers. We measured the effect of that exposure directly by comparing the same instance evaluated with an empty cheatsheet against the same instance evaluated with a cheatsheet distilled from 5–20 other evaluation instances: the difference is 0.0pp on both database (n=15n=15) and math (n=14n=14). The designated memory learn set also has zero exact and zero near-duplicate overlap with the canonical LiC evaluation set. The roughly 6pp run-to-run standard deviation quoted in the main text comes from four trajectory orderings of the LiC database cell with a gpt-5.4-mini respondent (28.0 ± 6.5% with memory vs. 36.0% without; relearning at a fixed ordering gives a comparable spread).

Trajectory example: WildChat Maven debugging

We present an illustrative (favorable) WildChat conversation in which Gated Reset (GR) outperforms both full context (FC) and assistant omission (AO) on every evaluated turn. The conversation (300289e3… , 8 rounds, originally GPT-3.5-turbo) involves iterative debugging of a Maven/Java build in GitHub Codespaces.

Conversation summary

The user runs mvn clean install and encounters a sequence of errors. Early turns establish the environment (GitHub Codespaces) and initial diagnosis (JRE vs. JDK). By Turn 8, the user reports “Still getting: Fatal error compiling: error: invalid target release: 17” after following the assistant’s prior suggestion, a classic no-feedback turn where the user implicitly references the prior assistant response without providing explicit corrective feedback.

Turn 8: three conditions compared

AO (assistant omission). With all prior assistant messages replaced by [Response provided] placeholders, AO re-suggests apt-get install and checking java -version, repeating steps the user has already performed. AO cannot connect “still getting” to what was already tried. FC (full context). FC sees the full history and includes update-alternatives commands, but still partially repeats earlier suggestions. The model anchors on the same diagnostic approach (checking java -version) rather than recognizing that the user already tried the basic install without resolution.

GR (Gated Reset). The analyzer identifies that the prior assistant’s diagnostic reasoning was flawed (relying on java -version text parsing rather than checking javac directly) and that the conversation was stuck repeating an insufficient fix. The rebuilt context restates the correct diagnosis (Maven needs a JDK) without the flawed reasoning. GR’s response:

  • Correctly diagnoses why the error persists: Maven may use a different Java than what java -version reports

  • Provides non-redundant steps (does not re-suggest things already tried)

  • Includes solutions FC and AO both missed: devcontainer.json configuration, the <release>17</release> tag in pom.xml

Analyzer output (abbreviated).

Issue: The assistant said that seeing “OpenJDK Runtime Environment” in java -version means the system has a JRE rather than a JDK. That is not reliable: java -version output often contains “Runtime Environment” even when a JDK is installed. The correct checks are presence of javac and the value of JAVA_HOME.

Aligned: Maven’s “No compiler is provided” message means Maven cannot find javac and needs a JDK. Installing a JDK inside Codespaces is a valid remediation path.

Judge evaluations

All three pairwise comparisons were judged by GPT-5-mini:

  • FC vs. AO: FC wins. “Both responses correctly diagnose the problem. [FC] is slightly clearer and more prescriptive.”

  • AO vs. GR: GR wins. “[GR] is substantially more complete and actionable.”

  • FC vs. GR: GR wins. “[GR] is significantly more complete and actionable. It gives precise diagnostic commands, exact installation commands, environment variable exports, pom.xml changes, devcontainer recommendations, and verification steps.”

Conversation-level results

Across all 7 evaluated turns, GR wins every pairwise comparison against both baselines (Table 10). On turns where AO beats FC (turns 2, 10, 12, 14), context pollution hurts more than context loss, yet GR still wins: its rebuilt context drops the flawed reasoning and restates what the user already tried.

TurnUser messageFC vs AOGR vs AOGR vs FC
2“Why do I keep getting error. . . ”AOGRGR
4“I am using Codespaces”FCGRGR
6“Now I get: [new error]”FCGRGR
8“Still getting [same error]”FCGRGR
10“I do not have a bashrc file”AOGRGR
12[Shares GitHub Actions YAML]AOGRGR
14“No WAR file generated” + pom.xmlAOGRGR

Table 10. Per-turn winners for the WildChat Maven debugging conversation. GR wins all 14 pairwise comparisons. On context-dependent turns (4, 6, 8), AO fails because it cannot connect the user’s references to prior discussion.

Additional related work

Multi-turn degradation and context pollution (extended). Laban et al. [ [9] introduced the Lost in Conversation (LiC) framework, showing that multi-turn performance drops 39% on average across 15 LLMs, driven by increased unreliability (+112%) in addition to reduced capability (−16%). Huang et al. [ [5] termed this context pollution and showed that simply omitting all assistant messages recovers much of the lost performance, showing that the assistant’s own outputs are a major source of the degradation. Khalid et al. [ [8] proposed ERGO, which monitors token-level entropy and triggers context resets via an LLM rewrite of prior user messages. Once triggered, both approaches remove or rewrite context wholesale. This is effective for self-contained tasks, where the model can re-derive answers from user messages alone, which is the setting LiC was designed to test (each instance is a single-turn question sharded across turns). On referential tasks (WildChat) omission degrades quality, and on stateful agentic tasks (tau2-bench) it collapses from near-oracle to zero, because assistant turns carry content the task needs. We build on their diagnostic finding: AC3 rebuilds the specification from the user side, closes most of the LiC gap, and does not collapse in the agentic setting. Note that ERGO is not a design oracle: its LLM rewrite step can paraphrase or compress the user’s exact phrasing, which is why it underperforms simple concatenation on database and actions (Table 1).

Test-time scaling. Methods such as chain-of-thought [23], self-consistency [22], and structured search [29] scale inference compute within a single generation. Our work extends test-time scaling to the multi-turn setting: rather than searching over possible outputs, we edit the input context presented to the model across turns, investing additional LLM calls to improve the quality of information the model reasons over.

Memory-based self-improvement. A growing line of work uses persistent memory to improve LLM capabilities without parameter updates. General-purpose memory systems store and retrieve knowledge for personalization and continuity [28, 35]. Agentic memory systems go further, accumulating procedural knowledge and task-specific heuristics that improve agent performance over time: the Dynamic Cheatsheet [19] maintains a mutable text document updated across problem instances; Voyager [21] builds a skill library for open-ended exploration; and several recent systems learn reusable reasoning strategies [11, 4, 3, 25, 32]. A separate thread optimizes the memory system itself rather than the downstream task [31, 27]. We adapt the Dynamic Cheatsheet for AC3, targeting accumulated analysis principles at the analyzer’s executive function rather than the task-solving assistant.

Extended agentic context management discussion

THREAD [13] spawns sub-agents that report summarized results back to a main thread, reducing context accumulation. Recursive Language Models [30] treat context as a variable manipulated via code operations in a REPL, enabling a controller LLM to programmatically decompose and recursively query sub-calls. Context Folding [18] uses reinforcement learning (FoldGRPO) to train agents that branch into sub-trajectories and collapse intermediate steps into concise summaries, maintaining active context 10×10 \times smaller than ReAct baselines.

These methods share a common goal, compaction: fitting more reasoning into a bounded context window. Our work addresses a fundamentally different problem: context pollution, in which the assistant’s own prior outputs mislead future generation even well within the context window. Compaction methods preserve information while reducing tokens; we instead remove the assistant’s invalidated reasoning and rebuild the task specification from the user side. The two goals are complementary but require different analytical capabilities.

U-Fold [17] targets multi-turn conversation with a two-stage pipeline: intent summarization (creating a TODO-list from conversation history) and data extraction (retaining only relevant tool responses). While structurally similar to our two-subtask decomposition, U-Fold addresses redundancy in tool responses, not self-induced pollution from the assistant’s own reasoning. MemoBrain [12] formalizes “executive memory” as an online mechanism that constructs a dependency-aware graph of reasoning trajectories and exerts cognitive control over working context. Like our work, it draws on the executive function framework and controls information flow to the reasoning agent. However, MemoBrain’s executive control is triggered by approaching budget limits and its operations (folding sub-trajectories into thought units, flushing low-salience nodes) serve long-horizon compaction. Our executive function targets a different deficit: the inability to disregard prior reasoning that has been invalidated, even when the model has ample context budget.

Executive function as a design pattern

Our approach instantiates a general pattern: externalized executive function via an agent that schedules LLM calls with custom context windows. Different subtasks of self-reflection require different context: specification extraction must be shielded from pollution, approach evaluation needs both clean intent and the full history, strategic reflection identifies unexplored alternatives. This pattern generalizes: any setting where an LLM reasons about potentially misleading context (code review, fact-checking) could benefit from structured context control. The executive function framing provides the design vocabulary: what should each reasoning step attend to, and what should it be shielded from?

We do not claim the LLM is self-aware in any phenomenological sense. Rather, the system designer’s knowledge of context pollution as a systematic failure mode is encoded into the agent architecture, analogous to using an external checklist to compensate for known cognitive biases.

References

  1. [1]Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik Narasimhan. τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment, 2025. URL https://arxiv.org/abs/2506.07982.
  2. [2]Eunsol Choi, Jennimaria Palomaki, Matthew Lamm, Tom Kwiatkowski, Dipanjan Das, and Michael Collins. Decontextualization: Making sentences stand-alone. Transactions of the Association for Computational Linguistics, 9:447–461, 2021. doi: 10.1162/tacl_a_00377. URL https://aclanthology.org/2021.tacl-1.27/.DOI
  3. [3]Runnan Fang, Yuan Liang, Xiaobin Wang, Jialong Wu, Shuofei Qiao, Pengjun Xie, Fei Huang, Huajun Chen, and Ningyu Zhang. Memp: Exploring Agent Procedural Memory, 2026. URL https://arxiv.org/abs/2508.06433.
  4. [4]Matthew Ho, Chen Si, Zhaoxiang Feng, Fangxu Yu, Yichi Yang, Zhijian Liu, Zhiting Hu, and Lianhui Qin. ArcMemo: Abstract Reasoning Composition with Lifelong LLM Memory, 2025. URL https://arxiv.org/abs/2509.04439.
  5. [5]Jenny Y. Huang, Leshem Choshen, Ramon Astudillo, Tamara Broderick, and Jacob Andreas. Do LLMs Benefit from Their Own Words?, 2026. URL https://arxiv.org/abs/2602.24287.
  6. [6]Jonas Hübötter, Frederike Lübeck, Lejs Behric, Anton Baumann, Marco Bagatella, Daniel Marta, Ido Hakimi, Idan Shenfeld, Thomas Kleine Buening, Carlos Guestrin, and Andreas Krause. Reinforcement Learning via Self-Distillation, 2026. URL https://arxiv.org/abs/2601.20802.
  7. [7]Ryo Kamoi, Yusen Zhang, Nan Zhang, Jiawei Han, and Rui Zhang. When can LLMs actually correct their own mistakes? a critical survey of self-correction of LLMs. Transactions of the Association for Computational Linguistics, 12:1417–1440, 2024. doi: 10.1162/tacl_a_00713. URL https://aclanthology.org/2024.tacl-1.78/.
  8. [8]Haziq Mohammad Khalid, Athikash Jeyaganthan, Timothy Do, Yicheng Fu, Sean O’Brien, Vasu Sharma, and Kevin Zhu. ERGO: Entropy-guided Resetting for Generation Optimization in Multi-turn Language Models, 2025. URL https://arxiv.org/abs/2510.14077.
  9. [9]Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. LLMs Get Lost In Multi-Turn Conversation, 2025. URL https://arxiv.org/abs/2505.06120.
  10. [10]OpenClaw. OpenClaw: open-source personal AI assistant. https://github.com/openclaw/openclaw, 2026. Version 2026.3.8, accessed 2026-03-09.
  11. [11]Siru Ouyang, Jun Yan, I-Hung Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T. Le, Samira Daruki, Xiangru Tang, Vishy Tirumalashetty, George Lee, Mahsan Roufoei, Hangfei Lin, Jiawei Han, Chen-Yu Lee, and Tomas Pfister. ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory, 2026. URL https://arxiv.org/abs/2509.25140.
  12. [12]Hongjin Qian, Zhao Cao, and Zheng Liu. MemoBrain: Executive Memory as an Agentic Brain for Reasoning, 2026. URL https://arxiv.org/abs/2601.08079.
  13. [13]Philip Schroeder, Nathaniel Morgan, Hongyin Luo, and James Glass. THREAD: Thinking Deeper with Recursive Spawning, 2025. URL https://arxiv.org/abs/2405.17402.
  14. [14]Idan Shenfeld, Mehul Damani, Jonas Hübötter, and Pulkit Agrawal. Self-Distillation Enables Continual Learning, 2026. URL https://arxiv.org/abs/2601.19897.
  15. [15]Jyotika Singh, Fang Tu, Miguel Ballesteros, Weiyi Sun, Sandip Ghoshal, Michelle Yuan, Yassine Benajiba, Sujith Ravi, and Dan Roth. MT-OSC: Path for LLMs that get lost in multi-turn conversation, 2026. URL https://arxiv.org/abs/2604.08782.
  16. [16]Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters, 2024. URL https://arxiv.org/abs/2408.03314.
  17. [17]Jin Su, Runnan Fang, Yeqiu Li, Xiaobin Wang, Shihiao Cai, Pengjun Xie, Ningyu Zhang, and Fajie Yuan. U-Fold: Dynamic Intent-Aware Context Folding for User-Centric Agents, 2026. URL https://arxiv.org/abs/2601.18285.
  18. [18]Weiwei Sun, Miao Lu, Zhan Ling, Kang Liu, Xuesong Yao, Yiming Yang, and Jiecao Chen. Scaling Long-Horizon LLM Agent via Context-Folding, 2025. URL https://arxiv.org/abs/2510.11967.
  19. [19]Mirac Suzgun, Mert Yuksekgonul, Federico Bianchi, Dan Jurafsky, and James Zou. Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory, 2025. URL https://arxiv.org/abs/2504.07952.
  20. [20]Gladys Tyen, Hassan Mansoor, Victor Carbune, Peter Chen, and Tony Mak. LLMs cannot find reasoning errors, but can correct them given the error location. In Findings of the Association for Computational Linguistics: ACL 2024, 2024.
  21. [21]Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An Open-Ended Embodied Agent with Large Language Models, 2023a. URL https://arxiv.org/abs/2305.16291.
  22. [22]Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-Consistency Improves Chain of Thought Reasoning in Language Models, 2023b. URL https://arxiv.org/abs/2203.11171.
  23. [23]Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, 2023. URL https://arxiv.org/abs/2201.11903.
  24. [24]Jason Weston and Sainbayar Sukhbaatar. System 2 Attention (is something you might need too), 2023. URL https://arxiv.org/abs/2311.11829.
  25. [25]Rong Wu, Xiaoman Wang, Jianbiao Mei, Pinlong Cai, Daocheng Fu, Cheng Yang, Licheng Wen, Xuemeng Yang, Yufan Shen, Yuxin Wang, and Botian Shi. Retriever: Self-Evolving LLM Agents through an Experience-Driven Lifecycle, 2025a. URL https://arxiv.org/abs/2510.16079.
  26. [26]Shirley Wu, Michel Galley, Baolin Peng, Hao Cheng, Gavin Li, Yao Dou, Weixin Cai, James Zou, Jure Leskovec, and Jianfeng Gao. CollabLLM: From Passive Responders to Active Collaborators, 2025b. URL https://arxiv.org/abs/2502.00640.
  27. [27]Yiming Xiong, Shengran Hu, and Jeff Clune. Learning to Continually Learn via Meta-learning Agentic Memory Designs, 2026. URL https://arxiv.org/abs/2602.07755.
  28. [28]Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. A-MEM: Agentic Memory for LLM Agents, 2025. URL https://arxiv.org/abs/2502.12110.
  29. [29]Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of Thoughts: Deliberate Problem Solving with Large Language Models, 2023. URL https://arxiv.org/abs/2305.10601.
  30. [30]Alex L. Zhang, Tim Kraska, and Omar Khattab. Recursive Language Models, 2026a. URL https://arxiv.org/abs/2512.24601.
  31. [31]Guibin Zhang, Haotian Ren, Chong Zhan, Zhenhong Zhou, Junhao Wang, He Zhu, Wangchunshu Zhou, and Shicheng Yan. MemEvolve: Meta-Evolution of Agent Memory Systems, 2025. URL https://arxiv.org/abs/2512.18746.
  32. [32]Qizheng Zhang, Changran Hu, Shubhangi Upasani, Boyuan Ma, Fenglu Hong, Vamsidhar Kamanuru, Jay Rainton, Chen Wu, Mengmeng Ji, Hanchen Li, Urmish Thakker, James Zou, and Kunle Olukotun. Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models, 2026b. URL https://arxiv.org/abs/2510.04618.
  33. [33]Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models, 2026. URL https://arxiv.org/abs/2601.18734.
  34. [34]Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. WildChat: 1M ChatGPT Interaction Logs in the Wild, 2024. URL https://arxiv.org/abs/2405.01470.
  35. [35]Zijian Zhou, Ao Qu, Zhaoxuan Wu, Sunghwan Kim, Alok Prakash, Daniela Rus, Jinhua Zhao, Bryan Kian Hsiang Low, and Paul Pu Liang. MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents, 2025. URL https://arxiv.org/abs/2506.15841.

Paper details

vlnc:W_c3rzueak
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents