betaFeedback

WorksComputer Science

Forget Freely, Grep Later: Lossless, Index-Free Memory for Long-Horizon LLM Agents

Abstract

Long-horizon LLM agents often exceed their context windows, forcing them to discard information that may be needed later. Existing context management strategies rely on summarization, masking, or structured memory, requiring decisions about what to preserve or how to retrieve it before future needs are known. However, these methods can introduce lossy compression, potentially discarding details needed for future tasks. To address this limitation, we introduce GrepMem, a simple, lossless, and index-free memory mechanism built on two components: an append-only plain-text log of the agent's complete history and a regular-expression search tool for on-demand recall. Unlike structured memory systems, GrepMem requires no embeddings, retrieval index, memory extraction, or additional models. We train Qwen3.5-9B with reinforcement learning under five context management strategies and evaluate on multi-question research sessions, long-stream recall, and agent-trajectory question answering. GrepMem achieves 95-97% accuracy on ContextBench, versus at most 21% for lossy baselines, and outperforms SUPO's summarization by 8.8 percentage points on BCP-Multi. GrepMem scales effectively to increasingly long input streams while remaining on the accuracy-token-cost Pareto frontier across all five evaluation tasks. Our results show that a plain-text log and grep provide a simple but competitive approach to long-horizon agent memory, without requiring complex structured memory infrastructure.

Code   Interactive Demo

Introduction

Long-horizon LLM agents accumulate interaction histories that eventually exceed their context windows, forcing them to make room for new observations while retaining useful information. In deep research, for example, an agent may inspect many

documents before combining its findings into a final answer [1, 3] (Wei et al. 2025). An identifier or fact encountered early may become relevant only after several later searches, making it difficult to decide which details to retain when reducing context. Keeping the entire trajectory in context is expensive and eventually exceeds the window; even within that window, models do not reliably use all available information (Liu et al., 2024). Therefore, the challenge is not only to manage the working context but also to preserve the ability to recover previously observed information on demand.

Existing approaches address this challenge through compression or external memory but can incur additional computational overhead while risking the loss of important information needed for future tasks. Yet unlike human memory, digital storage allows agents to preserve complete interaction histories beyond the limited context window at relatively low cost. Recent studies further suggest that simple lexical search over raw histories can rival structured memory systems [9] (Sen et al. 2026), offering a lightweight way to recover past information on demand. Taking this idea to its logical extreme, we revisit the boundary between short-term context management and long-term memory in LLM agents: can a complete digital record and a simple search tool provide effective agent memory without a structured store or retrieval index?

We introduce GREPMEM, which separates the agent’s bounded working context from its persistent history (Figure 2). We build on the summarization loop of SUPO (Lu et al., 2026), a reinforcement learning (RL) algorithm that jointly optimizes tool use and context summarization. When the context reaches a length threshold, the agent summarizes its interaction history and resumes from the initial prompt and the summary. GREPMEM extends this loop by automatically appending every message and tool output to a persistent plain-text log. After the first compaction, the agent can retrieve earlier observations through a single tool, grep_history, which searches the log using regular expressions and supports both retrieving and counting matching lines. In GREPMEM, summaries provide context for ongoing reasoning, while the lossless log preserves all details omitted during compaction. No embeddings, retrieval index, memory extraction, or additional models are required.

Figure 2. GREPMEM Overview. (a) The basic agent loop generates responses and tool calls from its active context, while GREPMEM additionally appends these interactions and tool results to a persistent history log. (b) Context compaction generates a summary when the working context reaches the compaction limit, allowing the agent to resume execution with a compacted context. GREPMEM builds on SUPO’s summarization loop. Extending this framework, GREPMEM enables (c) on-demand retrieval through grep_history and (d) a lossless, append-only full history log containing user messages, tool calls, tool results, summaries, and agent responses. By combining context summarization with a log preserving every message and tool output verbatim, GREPMEM enables grep_history to retrieve matching log lines after the first compaction without embeddings, a retrieval index, or an additional model.

We train Qwen3.5-9B (Qwen Team, 2026) under five context management strategies: ReAct (Yao et al., 2023), observation masking (Lindenbauer et al., 2025), SUPO (Lu et al., 2026), MemGPT-style memory (Packer et al., 2024), and GREPMEM (Table 1) using the same RL recipe, training data, and task tools (§ 4). We evaluate each strategy before and after RL training on multi-question research with BCP-Multi, constructed from BrowseComp-Plus question triples [1]; long-stream recall on ContextBench (Shao et al., 2026); and agent-trajectory question answering on AMA-Bench (Zhao et al., 2026). Together, these settings test both retaining findings for known questions and recovering details for questions revealed later.

MethodHow it worksScales past the windowKeeps everythingRecallNo index
ReAct (Yao et al., 2023)keeps all, stops when full✗✓✗✓
Masking (Lindenbauer et al., 2025)hides old tool outputs✓✗✗✓
SUPO (Lu et al., 2026)summarizes and restarts✓✗✗✓
MemGPT (Packer et al., 2024)summarizes, manages memory✓✓✓✗
GREPMEM (ours)summarizes, greps raw log✓✓✓✓

Table 1. Comparison of Context Management Strategies. We compare how five strategies manage context, preserve interaction history, and support retrieval. GREPMEM augments SUPO with an automatically appended plain-text history log and a grep tool, preserving the full history for retrieval without an index.

Overall, GREPMEM improves accuracy while remaining on the accuracy–token-cost Pareto frontier for all five evaluation tasks (§ 5.4, Figure 1 and Figure 5). Its greatest advantage emerges when exact details must remain accessible beyond the active context window. GREPMEM achieves 95–97% accuracy on ContextBench, compared with at most 21% for SUPO’s summarization (Lu et al., 2026) and observation masking (Lindenbauer et al., 2025). On BCP-Multi, it improves accuracy by 8.8 percentage points over SUPO and 7.0 over MemGPT. These gains come at competitive token costs: on ContextBench, GREPMEM uses only 29% as many tokens as MemGPT while achieving substantially higher accuracy on Log Triage.

Figure 1. Mean Accuracy vs. Token Cost After RL. We compare four context management strategies by their mean accuracy and token usage across five evaluation tasks. SUPO compresses context through summarization, while Masking discards older tool outputs to free context space. MemGPT maintains editable working memory, archival memory, and searchable message history. Our method, GREPMEM, combines summarization with a lossless, searchable history log, enabling on-demand retrieval through grep without a retrieval index. Token usage includes prompt and generated tokens across all model calls. Figure 5 reports per-task results.

Figure 5. Per-Task Accuracy vs. Token Cost After RL. We compare the accuracy and token usage of RL-trained strategies on each evaluation task. Cost is the number of tokens processed per episode (prompt and generated tokens over all LLM calls; ContextBench: per stream), on a log scale. Dashed: the Pareto frontier.

Our contributions are threefold:

  • Lossless memory without an index. We introduce GREPMEM, which pairs lossy context summarization with a lossless trajectory log preserving the full interaction history, enabling the agent to recover omitted details through a single search tool, grep_history (Figure 2).

  • Pareto-optimal accuracy–token-cost trade-offs. We compare five context management strategies before and after RL training across three complementary settings. GREPMEM lies on the accuracy–token-cost Pareto frontier across all five evaluation tasks, achieving higher accuracy than summarization-based baselines at competitive token costs (Figure 1 and Figure 5, Table 3).

  • Scalable and accurate recall over long histories. GREPMEM maintains high accuracy on ContextBench’s KV Store and Log Triage tasks (Table 2) even as input streams grow far beyond the model’s context window, enabling accurate and scalable long-horizon recall without complex memory infrastructure (Figure 4).

TaskQuestionsTiming
BCP-Multi828before
AMA-Bench file2,496before
AMA-Bench stream2,268after
ContextBench KV576after
ContextBench Log480after

Table 2. Evaluation Task Sizes. Questions are totals per evaluation run. Timing indicates whether questions are revealed before or after the relevant input.

StrategyBCP-MultiAMA-Bench fileAMA-Bench streamContextBench KVContextBench LogAvg.
After RL
ReAct0.238.326.60.00.013.0
SUPO38.145.545.213.96.629.8
Masking34.844.544.920.76.630.3
MemGPT39.946.445.499.950.056.3
GREPMEM46.946.346.197.495.366.4
Before RL
ReAct0.022.426.20.00.09.7
SUPO45.245.541.710.96.630.0
Masking18.642.342.113.35.524.4
MemGPT34.445.446.195.848.954.1
GREPMEM40.946.945.893.975.060.5

Table 3. Accuracy Before and After RL Training. We compare the accuracy (%) of five context management strategies using Qwen3.5-9B before and after training with the same RL recipe (§ 4). Within each block, bold marks the best value in each column, and underlining marks the second best. For ContextBench, we report exact-value accuracy on KV Store and exact-answer accuracy on Log Triage. Avg. denotes the mean accuracy across five tasks.

Figure 4. ContextBench Performance Across Stream Lengths. We evaluate how stream length affects ContextBench performance before (dashed) and after RL (solid) training.

Related Work

Compressing the working context. Context management strategies extend agent trajectories by selecting what remains in the active context. Observation masking (Lindenbauer et al., 2025) replaces older tool outputs with placeholders while preserving the agent’s reasoning and actions, retaining recent observations within a fixed window, while ReSum (Wu et al., 2026) and SUPO (Lu et al., 2026) use summaries to continue beyond the window. Learned alternatives update bounded memory states (MemAgent and MEM1; Yu et al., 2026; Zhou et al., 2026), compress sub-trajectories (Context Folding and AgentFold; Sun et al., 2026; Ye et al., 2026), or edit the context directly (Context Language Models; Shao et al., 2026). These mechanisms determine which details remain available in context. Following SUPO’s summarization loop, GREPMEM separates working context compression from access to past experience: summaries maintain a compact working context, while the original history remains searchable so that omitted details can be recovered.

Organizing external memory.

External stores preserve information beyond the working context, but differ in representation and access. MemGPT (Packer et al., 2024) combines searchable message history with working and archival memory; Generative Agents (Park et al., 2023) retrieve recorded experiences by recency, importance, and relevance. Other systems organize linked notes (A-MEM; Xu et al., 2025), extracted facts (Mem0; Chhikara et al., [2]), or knowledge graphs (Zep and HippoRAG; Rasmussen et al., 2025; Gutiérrez et al., [4]). These approaches introduce decisions about memory organization and retrieval in addition to solving the task (Yang et al., 2026). LongMemEval (Wu et al., 2025a) and MemoryAgentBench (Hu et al., [5]) also evaluate complementary aspects of long-term memory. Unlike prior approaches, GREPMEM retrieves information directly from a plain-text history log through a single index-free search tool.

Searching history and learning to recall.

Recurrent Language Models (Zhang et al., 2026) expose long inputs to programmatic inspection, and coding agents interact with files and environments through code and specialized tools (Yang et al., 2024; Song et al., 2026). Closer to our setting, Sen et al. (2026) compare grep with vector retrieval over chat archives, while ReFind (Li et al., [9]) uses Algorithm 1 GREPMEM: one research episode

  1. Inputs: policy πθ\pi_\theta; task; interaction limits

  2. Initialize: context cc and history log with the system prompt and task; disable grep

  3. while interaction budget remains do // Context compaction

  4. if context approaches the window limit then

  5. Generate summary s∼πθ(⋅∣c,summarize)s \sim\pi_\theta(\cdot\mid c,\text{summarize})

  6. Append ss to the history log

  7. Reset cc to the system prompt, task and ss

  8. Enable grep_history

  9. end if // Act or recall

  10. Sample reply a∼πθ(⋅∣c)a \sim\pi_\theta(\cdot\mid c)

  11. Append aa to context cc and the history log

  12. if aa contains a final answer then

  13. return the answer in aa

  14. end if

  15. Execute the requested tools; grep reads the log

  16. Append tool outputs to cc and the history log

  17. Update interaction budgets

  18. end while

  19. Ask πθ\pi_\theta for a final answer from cc, without tools

  20. return the answer

agent-controlled lexical search over unmodified conversations. These studies motivate raw-history retrieval but do not train agents to search their own growing trajectories. RL has also been used to train agents for tool-based research [6, 8, 3] (Wu et al. 2025b; Shi et al. 2026a). Related work improves RL training efficiency through adaptive curricula (Shi et al., 2026b) and agent generalization through reusable skills extracted from successful trajectories (Xu et al., 2026). We bring these directions together by training recall alongside task actions and summarization, and comparing five memory mechanisms.

GrepMem

Bounded Context, Persistent History

As discussed in the introduction, memory is essential for long-horizon agents, yet existing approaches face a trade-off: context compression risks losing information beyond the context window, while long-term memory systems often introduce additional complexity in memory retrieval. These limitations highlight the potential of combining context compaction with simple, long-term access to the complete interaction history to address both information loss and retrieval complexity.

GREPMEM revisits the relationship between short-term context management and long-term memory by maintaining a bounded working context alongside a lossless interaction history. Algorithm 1 summarizes the overall procedure in detail.

Building on the summarization loop of SUPO (Lu et al., 2026), our context compaction mechanism generates a summary when the working context approaches its limit, allowing the agent to resume from the original prompt and summary. Meanwhile, an append-only log preserves all interactions, including messages, tool calls, outputs, and summaries, making the interaction history lossless: nothing is discarded, even after the working context is compacted. Together with on-demand grep retrieval, this gives the agent a compact working context and a complete, searchable history (Figure 2). During an episode, the agent iteratively generates responses and invokes task-specific tools to interact with its environment. Each tool call returns an observation to the working context, while the harness appends the agent’s messages, tool calls, and outputs to the persistent history log. These two representations serve complementary purposes: the working context supports ongoing reasoning through summarization, while the lossless history enables on-demand recovery of details omitted during compaction. Appendix § 8 describes how summarization input fits within the context window.

On-Demand Recall with Grep

After the first compaction, grep_history becomes available during subsequent task execution. The agent decides when to invoke it, subject to the interaction budget. The tool searches the persistent history log using case-insensitive regular expressions. By default, it returns matching lines in chronological order, with surrounding context and their original locations in the history, enabling recovery of details omitted during compaction.

To limit response size, results are paginated, and the agent can request additional pages to access the remaining matches. The tool also supports a count-only mode that reports the total number of matching lines without returning their contents. Because retrieval relies on regular-expression matching rather than semantic similarity, the agent must formulate search patterns that match relevant text in the history log. These patterns may be simple keywords, such as names or identifiers, or more complex regular expressions. Appendices § 7 and § 8 give the tool schema, output limits, and examples.

Learning with Reinforcement Learning

Since GREPMEM preserves complete histories and enables on-demand retrieval, effective memory use requires the agent to decide when and how to retrieve relevant information. We therefore use reinforcement learning to teach agents these capabilities by jointly optimizing task actions, context summarization, and memory retrieval through a shared episode-level reward, without separate supervision for summaries or grep queries. We evaluate GREPMEM both before and after RL training, with details provided in Section 4 and Table 3.

Experimental Setup

We evaluate GREPMEM with four baselines across five tasks spanning multi-question research, recorded-trajectory analysis, and recall over long input streams. GREPMEM achieves the highest average accuracy both before and after RL training (Table 3) and lies on the accuracy–token-cost Pareto frontier across all five tasks (§ 5.4).

Context Management Strategies

We compare five context management strategies, summarized in Table 1. ReAct (Yao et al., 2023) retains full history until the context limit is reached. Masking (Lindenbauer et al., 2025) replaces older tool observations with placeholders. SUPO (Lu et al., 2026) summarizes and resumes the context. MemGPT-style memory (Packer et al., 2024) combines a notepad with searchable message history and archival memory. GREPMEM augments SUPO with a lossless, append-only history log and an index-free grep interface as described in § 3. Implementation details appear in Appendices § 8 and § 9.

Training and Shared Settings

To ensure a fair comparison, we separately train Qwen3.5-9B (Qwen Team, 2026) with thinking enabled under each context management strategy using the same GRPO-based RL recipe (Shao et al., 2024). All strategies use a 32,768-token context window and the same interaction budgets.

Our training set contains 4,000 episodes: 70% single-question OpenResearcher tasks (Li et al., [10]), 15% three-question sessions constructed from the OpenResearcher training split, and 15% synthetic ContextBench-style streams (Shao et al., 2026). This mixture targets complementary capabilities: research and tool use in single-question tasks, retention of findings across questions in multi-question sessions, and precise lookup and counting over long histories in synthetic streams. OpenResearcher tasks use binary correctness rewards, averaged across questions in multi-question sessions. Details on optimization, data construction, and interaction budgets appear in Appendix § 10.

Evaluation Tasks

We evaluate GREPMEM on BCP-Multi, AMA-Bench file and stream (Zhao et al., 2026), and ContextBench (Shao et al., 2026), as summarized in Table 2. These tasks are distinguished by whether questions are revealed before or after the relevant input. When questions are known in advance, agents can use them to guide retention and search. This setting tests whether agents can retain findings while investigating multiple research questions (BCP-Multi) or search recorded trajectories with a shared search tool (AMA-Bench file). When questions are revealed after the input, agents must process the input without knowing future needs, so answering depends on whether relevant details remain available in the working context or, for strategies supporting retrieval, through memory search (AMA-Bench stream and ContextBench).

Questions provided before the input. BCP-Multi extends single-question evaluation to test retention across research questions. We group three questions into one episode so that findings from earlier searches must remain accessible while the agent investigates other questions and prepares a joint final answer. We construct it from BrowseComp-Plus (Chen et al., [1]) by shuffling its 830 BrowseComp questions (Wei et al., 2025) with a fixed seed into 276 disjoint triples, omitting the remaining two questions. All three questions are provided together in the initial prompt and are answered together within 60 turns over the original 100K-document corpus. The agent retains its context across all three questions in each episode. AMA-Bench file uses recorded agent trajectories and associated question–answer pairs provided by the benchmark (Zhao et al., 2026), with the source trajectories kept outside the model’s working context and accessible through grep_trajectory. We evaluate 208 trajectories, with the agent answering all 12 questions for each trajectory in a single session with a shared budget of 60 turns. We provide all context management strategies with grep_trajectory, a tool that searches recorded actions and observations using regular expressions and returns matching lines grouped by step. Unlike grep_history, which searches the agent’s own interaction history, grep_trajectory searches the benchmark-provided fixed source trajectory.

Questions revealed after the input. ContextBench (Shao et al., 2026) evaluates whether agents can recall information from long sequences of key–value records or service logs. The agent receives these records incrementally, then answers 24 questions about them. KV Store requires retrieving exact values, while Log Triage requires finding specific log entries and counting entries that satisfy given conditions. We evaluate input sequences ranging from 33K to 968K tokens across 11 length levels, with four random seeds per level. AMA-Bench stream evaluates question answering over recorded agent trajectories presented incrementally. Unlike AMA-Bench file, which stores the source trajectory as a searchable file, AMA-Bench stream feeds each trajectory into the agent’s context in successive chunks of approximately 12K characters. We evaluate 189 trajectories, each containing at most 128K tokens. After the agent has processed an entire trajectory, we save its final context and memory state. Each of the trajectory’s 12 questions is then answered independently, starting from an independent copy of the final context and memory state. Both settings hide questions during input processing, preventing question-guided retention.

Metrics and Reporting

We evaluate the model before and after RL training. OpenResearcher, BCP-Multi, and AMA-Bench answers are graded by gpt-5.6-sol (OpenAI, 2026) against references; ContextBench uses its official programmatic graders. We report 95% confidence intervals for accuracy. These intervals reflect variation across test items, not across training runs. We report total token cost across all model calls, including summarization and recall (Appendix § 11).

Results and Analysis

We evaluate the effectiveness of different context management strategies in terms of accuracy, token cost, and how their performance changes with RL training. Table 3 compares accuracy before and after RL, while Figure 3 tracks performance throughout training. Building on these results, we further examine when GrepMem improves accuracy, how RL affects different context management strategies, how performance varies with stream length, and the trade-off between accuracy and token cost.

Figure 3. Evaluation Performance of Five Context Management Strategies During RL Training. Scores are averaged over the evaluation samples available at each checkpoint. Evaluation details are provided in Appendix § 11.

Where Does GrepMem Improve Accuracy?

Questions known before the input. On BCP-Multi, the RL-trained GrepMem achieves 46.9% accuracy, outperforming SUPO by 8.8 percentage points and MemGPT by 7.0 points. Although all three questions are provided in the initial prompt, the agent must retain findings from earlier searches while investigating subsequent questions. SUPO and GrepMem average 25–27 context compactions per episode, testing whether these findings remain accessible after repeated summarization. GrepMem can recover findings omitted during compaction by searching its interaction history. On AMA-Bench file, by contrast, all strategies can revisit the original trajectory through a shared search tool, while all strategies except ReAct perform similarly, suggesting that searching the agent’s own history provides the greatest benefit when the original information is no longer directly accessible.

Questions revealed after the input. The largest gains over lossy baselines occur on ContextBench, where KV Store and Log Triage require exact values or individual log entries that summarization can discard, making them well suited to lossless recall. After RL, GrepMem achieves 97.4% accuracy on KV Store and 95.3% on Log Triage, compared with just 6.6–20.7% for SUPO and Masking. However, revealing questions after the input does not by itself make lossless recall beneficial. On AMA-Bench stream, all strategies except ReAct differ in accuracy by at most 1.2 percentage points after RL, and GrepMem’s lead over SUPO narrows from 4.1 points before RL to 0.9 points after RL. Even on trajectories exceeding the context window, the gain is not statistically significant. Unlike ContextBench, many AMA-Bench stream questions concern causes or state changes that summaries can preserve, rather than exact values or log entries. These results suggest that the value of lossless recall depends more on whether a question requires details omitted by summarization than on when the question is revealed.

How Does RL Affect Context Management Strategies?

As shown in Table 3, RL improves the average accuracy of most context management strategies, with GREPMEM gaining 5.9 percentage points to achieve the highest post-RL average accuracy of 66.4%. The largest improvement occurs on ContextBench Log Triage, where accuracy increases by 20.3 points since it requires lookup and counting over service logs. RL may improve query formulation and evidence aggregation, enabling GREPMEM to retrieve and utilize historical information more effectively. On ContextBench KV Store, accuracy increases by 3.5 points. Since this task requires exact-value retrieval for a given key, RL may help GREPMEM formulate more effective search queries. On BCP-Multi, accuracy improves by 6.0 points. As agents must retain and reuse findings across multiple questions in a shared session, RL may improve search timing, query formulation, and evidence reuse. In contrast, AMA-Bench file and stream show almost no improvement after RL (-0.6 and +0.3 points, respectively). These tasks require reasoning about state transitions and causal relationships across agent trajectories, which retrieval optimization alone may not sufficiently address.

These results suggest that RL gains depend not only on whether historical information remains accessible, but also on whether the task’s required operations can be improved through optimizing the agent’s tool-use and evidence-utilization behavior.

How Does Performance Scale with Stream Length?

As shown in Figure 4, GREPMEM maintains strong performance on both KV Store and Log Triage as stream length increases, demonstrating its ability to scale to longer input streams. In contrast, SUPO and masking exhibit substantial performance degradation as stream length grows, with accuracy approaching zero in some settings. MemGPT maintains relatively stable performance on Log Triage but remains substantially below GREPMEM, with accuracy around 50%. These widening gaps suggest that preserving searchable history becomes increasingly valuable as input length grows. Furthermore, RL improves GREPMEM’s performance on Log Triage at longer stream lengths, while its KV Store accuracy remains high both before and after training. These results suggest that GREPMEM’s searchable history enables effective scaling to longer input streams by preserving access to relevant information, while RL further improves evidence utilization. Such capabilities are valuable for long-running agents, where historical information accumulates beyond the model’s context window.

What Does Grep Cost?

GREPMEM lies on the accuracy–token-cost Pareto frontier across all five benchmark settings: no evaluated strategy is both cheaper and more accurate (Figure 1 and Figure 5). Compared with SUPO, it uses only 6% more tokens on BCP-Multi and 5% fewer on AMA-Bench file while achieving higher accuracy. On ContextBench, it substantially outperforms SUPO with only 19% additional token usage and consumes only 29% as many tokens as MemGPT. The notable exception is AMA-Bench stream, where GREPMEM uses 2.3× as many tokens as SUPO for a small improvement in accuracy and calls grep roughly 19 times per trajectory. These results suggest that recall incurs additional retrieval cost, but it is worthwhile and most cost-effective when the retrieved evidence materially improves the final answer.

Conclusion

We introduce GREPMEM, a simple, lossless, and index-free memory mechanism that combines context summarization with a complete, searchable interaction history. Our results demonstrate that GREPMEM supports accurate long-horizon recall, scales effectively to increasingly long input streams, and lies on the accuracy–token-cost Pareto frontier across all five evaluation tasks. Comparisons before and after RL training further show that RL can improve how agents retrieve and utilize historical information, with benefits varying across tasks and context management strategies.

Our work reexamines the boundary between short-term context management and long-term memory in LLM agents. What should an agent forget when it can remember everything, and how much complexity does remembering truly require? By preserving complete histories alongside compact working contexts, GREPMEM suggests that lossless preservation and lossy compaction can serve complementary purposes. Looking ahead, we envision GREPMEM as a simple but competitive baseline for evaluating future methods, inspiring principled memory designs for reliable long-running agents.

Limitations

Our evaluation focuses on controlled benchmarks and a single model, leaving open how GREPMEM generalizes to broader agent settings. However, these benchmarks cover diverse memory requirements, from multi-question research to exact recall over long histories, providing evidence for the benefits of lossless memory. GREPMEM also relies on lexical search, which may be less effective when relevant information cannot be identified through explicit search patterns. Nevertheless, our goal is not to replace more sophisticated memory systems in every setting, but to demonstrate that preserving raw history and retrieving it with a simple tool can already provide strong performance without structured memory infrastructure. Finally, the benefits of lossless recall depend on the task: when summaries retain sufficient information, additional retrieval may introduce unnecessary overhead. This suggests that an important future direction is to determine when simple, lossless retrieval is most valuable and how to use it selectively.

Ethics Statement

Our experiments use established benchmarks, fixed offline corpora, and publicly available agent trajectories, without involving human subjects or collecting new personal data. GREPMEM preserves complete interaction histories, which may contain sensitive information in real-world deployments. This concern is shared by other memory systems that retain historical interactions, rather than being unique to our approach. As with any persistent memory mechanism, appropriate access controls and data retention policies should be applied according to the deployment setting. Our work focuses on improving long-horizon agent reliability through a simple and transparent memory mechanism. To support transparency and reproducibility, we will release our code, prompts, and trained model checkpoints.

References

Prompts

All prompts are copied from the harness. The placeholder {extra_tools} is empty until the first compaction and then holds the grep_history line below.

System Prompt (Deep Research)

You are a deep-research agent. Answer the user's question by researching a fixed document corpus with the tools provided.

Tools:

  • search(query): dense retrieval over the corpus; returns the top 10 documents (docid, url, first ~512 tokens).

  • get_document(docid): returns the full text of a document (up to 8192 tokens).

{extra_tools}

Guidelines:

  • Questions often require combining several clues; break them down, search iteratively, and verify candidates against the evidence.

  • Prefer short, specific search queries. Open documents when snippets are insufficient.

  • When you are confident, stop calling tools and give your final answer as a short phrase inside \boxed{}, e.g. \boxed{Marie Curie}.

  • You have at most 30 turns (assistant replies). The number of turns left is shown after each tool result; give your final answer before it runs out.

ReAct omits the last guideline. GREPMEM adds “Replies that only call grep_history do not count as turns (up to 10 such replies).” MemGPT adds an equivalent instruction for its memory tools.

grep_history Description (After First Compaction)

  • grep_history(pattern, max_matches, offset, context_lines, count): regex search over your full, uncompressed interaction history (all earlier tool outputs and messages), including parts removed from your context by summarization. Each match is labeled with the turn and tool call it came from.

The tool’s JSON schema documents the arguments (max_matches: default 10, at most 30; offset: matches to skip; context_lines: default 1, at most 3; count: only count matching lines). It recommends patterns made of previously observed exact strings (names, numbers, dates, IDs).

Compaction Prompt (SUPO and GREPMEM)

Your context window is full and will be reset. Write a summary that lets you continue the research from scratch with only the original question and this summary. Include:

  1. Key findings so far, with the docids/urls that support them.

  2. Candidate answers and how well each is verified.

  3. Clues already resolved and approaches/queries that did not work.

  4. Concrete next steps.

Do not call any tools. Respond only with the summary.

Post-Compaction Message

{question}


Your earlier context was compacted.

Summary of your progress so far:

{summary}

(The full raw history of the earlier context is searchable with grep_history.)

[{n} of 30 turns left]

Continue the research.

The note in parentheses appears only for GREPMEM.

Masking and MemGPT. Masking (Lindenbauer et al., 2025) adds to the system prompt: “Your context window is limited: when it fills up, the outputs of your oldest tool calls are replaced by a short placeholder (your own messages and tool calls are kept).” A masked output reads [Old tool output removed from the context (n characters).]. MemGPT (Packer et al., 2024) adds a description of its context (“when it fills up, the oldest messages are removed from it and replaced by a summary (you are warned shortly before). Removed messages remain searchable with conversation_search; keep what you need in your working context or archival memory.”) and of its memory tools:

MemGPT Memory Tools

- working_context_append(content) /
working_context_replace(old_content,
new_content): edit your working
context, a notepad of up to 4000
characters shown at the end of this
system prompt; it is never removed
from your context.
- archival_memory_insert(content) /
archival_memory_search(query, page):
store passages in your unlimited
archival memory and search them
(keyword search).
- conversation_search(query, page):
case-insensitive text search over
every earlier message and tool output,
including those removed from your
context.

MemGPT Memory Warning

[System message] Memory pressure: your
context is about {pct}% full. When it
is full, the oldest messages will be
removed from it and replaced by a
summary. Save anything you will need
later to your working context or
archival memory now.
Your context window is full. The
messages above, after the task, will
be removed from your context (they
remain searchable with
conversation_search); the most recent
messages are kept. Write a summary of
the messages that will be removed, so
that you can continue the task with
your working context, this summary and
the kept messages. If the messages
above begin with an earlier summary,
merge it into the new one. Keep the
key facts, findings and progress,
copied exactly where later steps may
need them.
Do not call any tools. Respond only
with the summary.

Turn budget and forced answer. After each tool result, strategies other than ReAct see [{n} of 30 turns left]; before the last turn the note reads “this is your last turn. You must give your final answer in \boxed{} now; do not call any more tools.” An episode that runs out of turns (or whose final reply is cut off) gets one call without tools:

Forced-Answer Prompt

You have used all your turns. Do not
call any more tools. Based on what you
have found so far, give your final
answer now as a short phrase inside
\boxed{}; give your best guess if you
are unsure.

Three-question sessions ask for one numbered \boxed{} line per question instead. ReAct ends unanswered when its window is full, since there is no room for another call.

Implementation Details

Log format and grep output. Each log record is a header line (e.g. [turn 4 tool get_document(...)]) followed by its text, and grep_history matches the text line by line; records of earlier grep_history outputs are skipped, since they only repeat older matches. The output starts with the number of matching lines by source, then lists the matches grouped under their record’s header, with line numbers within the record (: marks a match, - a context line, – a gap).

Matches are returned oldest first, ten per page by default (at most 30), with one surrounding line by default (at most three). Each response is capped at 6K characters, ends at a whole match and gives the offset for continuing the listing. Lines are clipped to 300 characters around the match, an invalid pattern returns the compiler’s error, and a search that takes more than 10 s is stopped with an error. An example on a small log:

Example of History Search and Counting

grep_history(pattern="doc_8812|born in 19\d\d", context_lines=1)

4 matching line(s) in 3 history entries (3 in tool outputs, 1 in summaries).
== [turn 3 tool search({"query": "Georgian composer opera 1960s"})]
  1: [1] docid: doc_8812
  2- url:
     https://example.org/composers/a
== [turn 4 tool get_document({"docid": "doc_8812"})]
  1: docid: doc_8812
  2- url:
     https://example.org/composers/a
  4- Early life
  5: The composer was born in 1931 in Tbilisi and
  6- studied at the Tbilisi State Conservatory until 1954.
== [compaction 1 summary]
  1: Candidate composer found in doc_8812; birth year and conservatory still to verify.
(showing matches 1–4 of 4)

grep_history(pattern="doc_8812", count=true)

3 matching line(s) in 3 history entries (2 in tool outputs, 1 in summaries).

Fitting the compaction call. Several large tool outputs in one turn can push the context past τ\tau before the agent can compact, but the compaction prompt must still fit in WW. We estimate tokens from characters, calibrated on the last call’s exact prompt-token count, and drop the oldest turns (always keeping the system prompt, the task and the latest turn), then shorten the latest tool outputs if needed, until the prompt fits. If the model answers the compaction request with a tool call instead of text, its final reasoning is used as the summary.

Baselines

Masking. Lindenbauer et al. (2025) keep the last MM observations. With a fixed context window, we mask observations according to a token budget. When the next model call would exceed τ\tau, the oldest unmasked observations are replaced by the placeholder until the context is below τ/2\tau/ 2. In stream tasks, these observations are the incoming user messages. Since Qwen3.5 (Qwen Team, 2026) retains the reasoning from every turn in the current episode, masking observations alone may not free enough space. In that case, we remove reasoning from the oldest turns while preserving their actions. We drop entire turns only if the prompt still does not fit within the context window. As with compaction, masking in batches keeps the prompt prefix unchanged between masking events.

MemGPT. The main context is the system prompt with the working-context block, followed by a message queue headed by the task and a recursive summary. At 70% of the agent is warned. At τ\tau, the queue is flushed: the oldest messages are evicted until about half the window remains, and the model writes a new recursive summary of the old summary and the evicted messages, in a call without tools. Like MemGPT’s summarizer, tools are disabled during this call. When tools are available, the base model returned a memory-tool call instead of a summary in about one-third of context flushes. conversation_search finds the messages (whole tool outputs or streamed batches) that contain a case-insensitive substring and reports their number, five per page. Archival memory ranks passages with BM25 (Robertson and Zaragoza, 2009) instead of MemGPT’s embeddings and also returns five per page. On ContextBench, each streamed operation allows up to 8 memory-tool replies, and the chain of replies within one operation is trimmed if it would overflow the window.

Training Details

Data. We train on 4,000 episodes comprising three task types: 70% single-question OpenResearcher episodes, 15% three-question OpenResearcher sessions, and 15% synthetic ContextBench-style streams. OpenResearcher [10] questions are English questions with answers of at most ten words (3,013 questions), searched over the OpenResearcher corpus of 14.9M web documents with Qwen3-Embedding-8B (Zhang et al., 2025); search returns the top 10 documents with their id, URL and first ∼512\sim512 tokens. get_document returns up to 8,192 tokens of a selected document. To construct three-question training sessions, we shuffle the OpenResearcher training split with seed 2, partition it into triples and take the first 600 triples. Each triple is presented as three numbered questions in one initial prompt and answered together as numbered \boxed{} lines. The 60-turn budget is shared across the three questions, not enforced as separate 20-turn sessions. A question may appear both in a single-question training episode and in a three-question session. ContextBench training streams use seeds from 1000 on (evaluation uses 42–45), with KV Store at 1,000 or 2,000 records and Log Triage at 1,600 or 3,200 lines (1.5–6×\times the window), and 12 queries each.

Turns as training sequences. Each model call is one step. Consecutive turns that extend the same context are merged into one training sequence. A context compaction or a new user turn in a stream starts a new sequence whose prompt is the full context available at that point. Compaction and memory calls are sequences of their own, so summaries are trained too. All training sequences within an episode share the same episode-level advantage, allowing RL to assign credit across context compactions without separate rewards for summaries or individual grep calls.

Hyperparameters and compute. Table 4 lists the hyperparameters, which are shared by all strategies. We train with rLLM (Tan et al., 2025) on verl (Sheng et al., 2025), with vLLM [7] for rollouts. Each run uses 24 H100 GPUs (8 for the FSDP trainer, 16 for rollouts; ReAct uses 8 GPUs for rollouts because its rollouts are shorter), and a dense-retrieval server on 3 more GPUs is shared by all runs. In total, training used about 6K H100-hours; MemGPT’s runs were the slowest, since its ContextBench rollouts make several hundred calls.

Base modelQwen3.5-9B (thinking)
AlgorithmGRPO, std-normalized advantages
Prompts per step / rollouts per prompt8 / 8
Steps80
Learning rate / weight decay10−610^{-6} / 0.1
Clip range (low / high)0.2 / 0.28
KL penaltylow-variance KL, β=0.01\beta= 0.01
Loss aggregationtoken mean
Dynamic samplingdrop uniform-reward groups
Off-policy correctiontoken-level TIS, cap 2
Async staleness threshold0.5, partial rollouts
Temperature / top-pp0.7 / 0.95
Window WW / generation GG / threshold τ\tau32,768 / 4,096 / 28,672
Turn budget TT30 per question
Free grep / memory-tool repliesup to 10 per episode
RewardLLM judge (questions), grader (streams)
RetrieverQwen3-Embedding-8B, top 10
Hardware24×\timesH100 per run (8 train, 16 rollout)

Table 4. Training Configuration. We report the model, optimization settings, interaction budgets, and hardware used for training. Training hyperparameters are shared across all context management strategies.

Rewards. OpenResearcher tasks use binary correctness rewards, averaged across questions in three-question sessions. ContextBench uses its official programmatic grader. Neither summarization nor individual grep calls receive separate rewards.

Evaluation Details

Benchmarks. BCP-Multi uses the same tools and retriever as training, pointed at the BrowseComp-Plus [1] corpus. We shuffle its 830 questions with Python’s random.Random(1).shuffle, take consecutive non-overlapping triples and discard the final two questions. This produces 276 episodes containing 828 questions, grouped identically across strategies and checkpoints. Each initial prompt presents three questions and requests numbered final answers. The agent retains its context and memory as it works through the questions. ContextBench (Shao et al., 2026) uses six KV Store levels (R520–R16000) and five Log Triage levels (L640–L12800), with four random seeds per level. Operations are delivered one per turn, using the benchmark’s official 32K-token budget and graders. KV values end with a hash that the grader requires. AMA-Bench (Zhao et al., 2026) stream feeds each trajectory in ∼12\sim12K-character chunks (189 trajectories of at most 128K tokens, 2,268 questions) and asks each question on a copy of the final state, following the benchmark’s per-query protocol. AMA-Bench file gives every strategy grep_trajectory, the same tool as grep_history applied to the trajectory’s steps. Each session answers all 12 questions for one trajectory (60 turns; 208 trajectories; 2,496 questions). We evaluate BCP-Multi and AMA-Bench answers against references using gpt-5.6-sol (OpenAI, 2026). For BCP-Multi, we follow the BrowseComp grading rubric (Wei et al., 2025).

Samples and re-evaluation. Evaluations are conducted every ten training steps at a sampling temperature of 0.7. ReAct, masking and MemGPT have two samples per checkpoint. Our first evaluation harness discarded a compaction summary whenever the model answered the compaction request with a tool call and wrote the summary in its reasoning. Training was unaffected because it used a different serving path. We fixed the evaluation harness and re-evaluated SUPO and GREPMEM at every checkpoint. For GREPMEM on all benchmarks and SUPO on BCP-Multi and AMA-Bench file, we report only these re-evaluation results. SUPO’s compactions on ContextBench and AMA-Bench stream offer no tools, so its two samples per checkpoint there are unaffected. Masking, MemGPT and ReAct (which never compacts) were evaluated with the fixed harness. For the reported RL-trained models, we drew one more sample for every strategy except ReAct, whose selected checkpoint was not retained, and two more for SUPO and GREPMEM on the re-evaluated benchmarks, so each item is averaged over three samples (ReAct: two). Costs after RL come from one of these samples, run for every strategy with the same settings (eight GPUs, up to 256 concurrent episodes); MemGPT’s third sample was split across several servers, so its costs are the mean of its two regular evaluations, which used the same settings. Confidence intervals come from 10,000 paired bootstrap resamples of the evaluation items (questions, episodes, streams or trajectories). For AMA-Bench question types we resample trajectories.

Additional Results

Single checkpoints. Single checkpoints fluctuate by several points (Figure 3): SUPO on BCP-Multi drops from 42.9 at the preceding evaluation to 38.1 for the reported RL-trained model, and GREPMEM on ContextBench Log Triage drops from 97.3 to 88.1 between two earlier evaluations.

AMA-Bench stream by length and question type

Table 5 breaks AMA-Bench stream down after RL. GREPMEM is within 2.3 points of SUPO on both length buckets and all four question types, with no statistically significant differences. Its largest gain is on trajectories longer than the window (+2.3 [−0.2, 4.9]).

SubsetSUPOMask.MemGPTGREPMEM
≤ 32K tokens (124)48.248.347.948.3
32K–128K tokens (65)39.438.340.641.7
A: recall48.848.349.250.0
B: causal inference42.244.145.143.1
C: state updating51.749.449.853.6
D: state abstraction31.932.031.130.4

Table 5. AMA-Bench Stream Accuracy by Length and Question Type. We break down accuracy (%) after RL training by trajectory length and question type. Length groups are defined by whether trajectories fit within the 32K-token context window, with trajectory counts shown in parentheses. Question types cover recall, causal inference, state updating, and state abstraction. Bold text marks the best value in each row.

Cost. Table 6 lists the costs behind Figure 1 and Figure 5, and Figure 6 plots accuracy against wall-clock time per episode instead of tokens. GREPMEM also lies on the accuracy–runtime Pareto frontier for every task. On ContextBench, it is faster than SUPO, taking 38 versus 61 minutes per KV Store stream and 56 versus 84 minutes per Log Triage stream, as it generates shorter summaries. We could not measure ReAct’s cost because its selected checkpoint was not retained. ReAct stops when its context window is full.

Figure 6. Per-Task Accuracy vs. Runtime After RL. We compare the accuracy and wall-clock runtime of RL-trained strategies on each evaluation task. Runtime is measured in minutes per episode, or per stream for ContextBench, and is shown on a logarithmic scale. Dashed lines indicate the Pareto frontier. Runtime measurements are approximate because episodes ran concurrently on shared inference and retrieval servers.

BenchmarkSUPOMaskingMemGPTGrepMEM
BCP-Multi1.15M / 84949K / 511.97M / 1011.22M / 88
AMA-Bench file590K / 41490K / 34674K / 38560K / 38
AMA-Bench stream377K / 24404K / 231.30M / 67869K / 56
ContextBench2.11M / 1142.41M / 1128.60M / 3912.51M / 154

Table 6. Token Usage and LLM Calls After RL. We report the average token usage and number of LLM calls per episode for each strategy and benchmark. Each entry shows processed tokens / LLM calls, with tokens including prompt and generated tokens summed over all calls. ContextBench costs are measured per stream. ReAct is omitted because its selected checkpoint was not retained.

References

  1. [1]Zijian Chen, Xueguang Ma, Shengyao Zhuang, Ping Nie, Kai Zou, Sahel Sharifymoghaddam, Andrew Liu, Joshua Green, Keshama Patel, Ruoxi Meng, Mingyi Su, Yanxi Li, Haoran Hong, Xinyu Shi, Xuye Liu, Hosna Oyarhosseini, Nandan Thakur, Crystina Zhang, Luyu Gao, and 2 others. 2026. BrowseComp-Plus: A Fair and Disentangled Evaluation Benchmark for Deep Search Agents. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 22349–22370, San Diego, California, United States. Association for Computational Linguistics.
  2. [2]Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. 2025. Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory. In ECAI 2025. IOS Press.arxiv.org/abs/2504.19413
  3. [3]Jiaxuan Gao, Wei Fu, Minyang Xie, Shusheng Xu, Chuyi He, Zhiyu Mei, Banghua Zhu, and Yi Wu. 2026. Unlocking Long-Horizon Agentic Search with Large-Scale End-to-End RL. In The Fourteenth International Conference on Learning Representations.arxiv.org/abs/2508.07976
  4. [4]Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su. 2024. HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models. In Advances in Neural Information Processing Systems, volume 37, pages 59532–59569. Curran Associates, Inc.arxiv.org/abs/2405.14831
  5. [5]Yuanzhe Hu, Yu Wang, and Julian McAuley. 2026. Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions. In The Fourteenth International Conference on Learning Representations.arxiv.org/abs/2507.05257
  6. [6]Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan O Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning. In Second Conference on Language Modeling.arxiv.org/abs/2503.09516
  7. [7]Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles, SOSP ’23, pages 611–626. ACM.arxiv.org/abs/2309.06180
  8. [8]Kuan Li, Zhongwang Zhang, Huifeng Yin, Liwen Zhang, Litu Ou, Jialong Wu, Wenbiao Yin, Baixuan Li, Zhengwei Tao, Xinyu Wang, Weizhou Shen, Junkai Zhang, Dingchu Zhang, Xixi Wu, Yong Jiang, Ming Yan, Pengjun Xie, Fei Huang, and Jingren Zhou. 2025. WebSailor: Navigating Super-human Reasoning for Web Agent. Preprint, arXiv:2507.02592.arxiv.org/abs/2507.02592
  9. [9]Ruizhe Li, Licheng Zhang, Benfeng Xu, Mingxuan Du, Zheren Fu, and Weidong Chen. 2026a. When Your Agent Opens the Chat App: Agent-Controlled Search over Raw Chat Logs Rivals Structured Memory. Preprint, arXiv:2608.12888.arxiv.org/abs/2608.12888
  10. [10]Zhuofeng Li, Dongfu Jiang, Xueguang Ma, Haoxiang Zhang, Ping Nie, Yuyu Zhang, Kai Zou, Jianwen Xie, Yu Zhang, and Wenhu Chen. 2026b. OpenResearcher: A Fully Open Pipeline for Long-Horizon Deep Research Trajectory Synthesis. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 908–937, Budapest, Hungary. Association for Computational Linguistics.arxiv.org/abs/2603.20278

Paper details

vlnc:W_ntjvfnyn
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents