betaFeedback

WorksComputer Science

Multi-Agent Memory Inconsistency: Where Factual Conflicts Disappear in LLM Agents

Abstract

Multi-agent delegation lets LLM systems distribute evidence collection across workers with separate contexts. This creates a consistency challenge: facts retrieved by workers may be missing from the evidence available to the lead for a joint conclusion. We introduce a benchmark and evidence-flow profiler for studying multi-agent memory inconsistency through controlled factual conflicts in Harvey LAB document-review tasks. Pooled across three models with comparable generation spending, requiring delegation increases retrieval of both conflicting claims from 50.2% to 68.7% relative to single-agent execution, yet conflict detection falls from 16.2% to 13.2%. Within this setting, workers report both claims for only 35.2% of retrieved pairs, and pooled matched comparisons show higher detection than single-agent execution when one worker reads both claims but lower detection when the claims are split across workers. Worker-to-lead handoffs are a central target for improving conflict detection in multi-agent LLMs.

Introduction

Multi-agent LLM systems scale complex workflows by distributing work across agents with separate contexts. In lead–worker (NOT SURE the definition is widely used. See my comment) architectures, a lead assigns subtasks to specialized workers and combines their findings (Fourney et al., [10]). Research systems use this structure to explore multiple information sources in parallel and report gains over single-agent execution (Anthropic, [4]). Controlled evaluations nevertheless show that these gains depend on task structure and coordination choices (Kim et al., [21]).

Delegation, despite its parallel exploration gains, reshapes agent working memory. In single-agent systems, working memory tracks task state across successive model calls (Sumers et al., 2024); under delegation, task information is distributed across agents’ contexts, messages, and shared artifacts, creating a coordination challenge (Yu et al., 2026c). To analyze this gap, we study multi-agent memory inconsistency: whether task-relevant evidence remains available across local and shared states, where a joint conclusion requires it. As Figure 1 illustrates, a worker can accurately summarize its assigned documents yet omit a claim needed to detect a conflict with another worker’s findings. We examine this problem through factual conflict detection in document-review tasks, comparing single-agent execution (None), discretionary delegation (Optional), and mandatory delegation (Required). Required reserves source inspection for workers assigned to document partitions. Across three models and two runs per model under common per-task dollar allowances, Required increases retrieval of both conflicting claims (Source Reads Pair Recall) from 50.2% to 68.7% relative to None, yet conflict detection (Judge Pass) falls from 16.2% to 13.2%. Optional achieves the highest pooled detection rate, at 24.2%.

Figure 1. Multi-agent memory inconsistency. A and B give conflicting CorpX FY2025 revenue figures. (a) The single agent retrieves A, B, and A′; two workers also retrieve B′, completing a second fact pair. (b) Focusing on A/B, both facts occupy one single-agent context or separate worker contexts. (c) B is read but omitted from a worker summary, leaving the lead with only A and the conflict undetected.

Tracing the conflicting claims leads to the largest drop in evidence preservation at the worker-to-lead handoff. Under Required, Pair Recall falls from 68.7% at Source Reads to 24.4% at Lead Evidence. Workers report both claims for only 35.2% of the pairs they collectively retrieve, leaving the lead without the complete pair its workers retrieved.

As the first stage-wise empirical study of multi-agent memory inconsistency, this paper makes three key contributions:

  • Benchmark and profiler (§ 3). We construct 73 controlled numerical conflicts across 11 Harvey LAB document-review tasks ([12]), with a profiler tracing both claims through Source Reads, Worker Reports, Lead Evidence, and the Final Report. Pair Recall measures evidence coverage separately from semantic conflict detection under the three delegation policies.

  • Evidence loss at the worker-to-lead handoff (§ 4.3). Pooled across models and runs under comparable generation spending, Required retrieves more complete evidence pairs than None but detects fewer conflicts. The largest measured loss occurs at the worker-to-lead handoff, where workers omit claims.

  • Different outcomes for Same and Split evidence (§ 4.4). In pooled cohorts defined by Required’s Source Reads, Required detects more conflicts than None on identical cases when one worker reads both claims (34.2% versus 22.5%), but fewer when the claims are split across workers (10.0% versus 18.9%).

Background and Related Work

Scaling and Delegation in Agent Systems

LLM agents interleave reasoning traces with actions on external environments ((Yao et al., 2023)) and retain information across steps and trials through memory and reflection ([37, 32] (Wang et al. 2024a)). Multi-agent systems divide tasks among communicating agents with assigned roles ([23, 33, 13]). In the orchestrator–worker pattern, a lead plans and synthesizes while workers execute subtasks ([1, 4]); Magentic-One similarly separates orchestration from specialized execution ([10]). AutoGen supports configurable conversations among agents and tools ((Wu et al., 2024)), while MetaGPT structures collaboration through specialized roles and standard operating procedures ([15]).

Coding agents have followed the same path. SWE-bench evaluates the resolution of real GitHub issues by editing a repository ([19]). SWE-agent addresses it through a purpose-built agent–computer interface ((Yang et al., 2024)), OpenHands provides an open platform for developer agents that supports coordination among multiple agents ((Wang et al., 2025)), and MAGIS divides planning and coding among four role-specific agents ((Tao et al., 2024)). Current coding harnesses provide subagents that work in their own context windows or threads and return summaries to the main agent ([5, 30]). For long-form writing, STORM gathers material through simulated conversations in which writers with different perspectives question a source-grounded expert ([35]), and DeepResearch Bench evaluates the reports and citations of deep-research agents ([8]).

Multi-agent debate was proposed to improve factuality and reasoning ([9]), yet a single agent with strong prompts nearly matches the best discussion methods ((Wang et al., 2024b)), and debate systems do not reliably outperform self-consistency or ensembling without careful tuning ([38]). The benefit of multi-agent over single-agent systems diminishes as model capabilities improve ([11]), and single agents match or outperform multi-agent systems on multi-hop reasoning when reasoning tokens are held constant ((Tran and Kiela, 2026)). Controlled comparisons show that agent-system performance depends on task structure [21]. Agentic systems can be designed automatically [17], although automatically designed multi-agent systems can incur greater cost without outperforming strong single-agent baselines [20].

Context policies govern which evidence agents receive and retain (Yu et al., 2026b); history edits can also increase execution cost when they disrupt prefix caching, even with fewer prompt tokens (Yu et al., 2026a). For coding agents, masking old observations halves cost relative to an unmanaged history while matching the solve rate of LLM summarization [26], and compaction can silently remove task constraints [34]. Iterative context rewriting can likewise erode details over time (Zhang et al., 2026b). For delegated workflows, we track which retrieved facts reach the lead and final report, together with conflict detection and generation spending.

Distributed Evidence and Multi-Agent Failures

MAST categorizes failures in multi-agent traces across frameworks [6], and Agent-Debug diagnoses errors in agent trajectories and supplies corrective feedback (Zhu et al., 2025). Failure attribution methods aim to identify the agent and step responsible for a failed trajectory (Zhang et al., 2025b, 2026a). HiddenBench tests collective reasoning and communication by supplying complete information to single agents and distributing facts across multi-agent groups [25]. In our setting, both single-agent and multi-agent systems must first retrieve evidence from document collections and then produce long-form reports, exposing what happens before and after workers select findings to report.

A practitioner account attributes conflicting sub-agent outputs to context that the subagents do not share and recommends sharing full agent traces (Yan, 2025), whereas recommended subagent designs return condensed summaries to the lead (Anthropic, 2025b,a). An information-theoretic analysis treats a model that compresses context for another model as a noisy channel and finds that the mutual information between a context and its compression predicts downstream performance ([14]). We measure this handoff for specified claims.

Tracing tools provide a complementary basis for inspecting these workflows. AgentSight correlates LLM traffic with system events (Zheng et al., 2025), and PROV-AGENT connects agent interactions to workflow provenance ([39]). Our profiler uses captured execution records to track specified conflicting claims: which agents read them, which contributors report them, and whether the final report identifies their discrepancy.

Agent Memory and Conflicting Evidence

MemGPT manages memory tiers to extend an agent’s limited context window ([31]), and surveys organize the design and evaluation of agent memory ([18] (Zhang et al. 2025d)). Benchmarks test long-term memory across conversation sessions ([29] (Wu et al. 2025)). G-Memory organizes multi-agent collaboration trajectories in a three-tier graph hierarchy (Zhang et al., 2025a). An architectural account of multi-agent memory separates update visibility from conflict resolution over shared or distributed state (Yu et al., 2026c). StateFuse designs memory interfaces that preserve conflicting claims (Volkov et al., 2026), while Facts Without Rules traces privacy-related usage constraints lost during handoffs (Wang et al., 2026). Inconsistency also arises among the context views served to a single coding agent: CodeNib compiles lexical, dense, and structural views of one commit so that their results can be cross-checked, and reports that incrementally maintained views can diverge from a full rebuild even when every update completes without error (Yu et al., 2026b). We trace known factual comparisons through source inspection and worker reporting in document analysis, measuring whether both claims remain available for a joint conclusion.

Knowledge-conflict studies examine how models respond when supplied passages contradict parametric knowledge or one another ([28, 7] (Xie et al. 2024; Xu et al. 2024)). WikiContradict and ContraDoc evaluate models on contradictory passages and self-contradictory documents ([16, 24]). Model performance also degrades when relevant information sits in the middle of a long context ([27]), and summarizing the insights repeated across many documents remains an open challenge for long-context LLMs and RAG systems ([22]). These evaluations give a single system the evidence to be reconciled, whereas our setting requires separate agents to retrieve and report both claims before any model can compare them.

FEVER evaluates supporting-evidence recovery alongside claim verification (Thorne et al., 2018), and HotpotQA evaluates supporting facts alongside multi-hop answers (Yang et al., 2018). Adapting the complete-evidence-group principle to the two claims defining a planted conflict lets us measure evidence recovery at each stage separately from semantic detection.

Benchmark and Measurement

We evaluate whether agents flag planted numerical conflicts in their reports, using the document edits, delegation policies, and profiling pipeline summarized in Figure 2.

Figure 2. Benchmark construction and evidence measurement. (a) Controlled edits introduce numerical conflicts into Harvey LAB document collections; agents generate a red-flags report, which is evaluated against the specified conflicts. (b) None, Optional, and Required control delegation and lead source access. (c) The profiler correlates captured traffic, agent/tool events, and artifacts to measure Pair Recall across four stages and a separate Judge Pass outcome. Same/Split separates pairs supplied by one individual from pairs supplied only across individuals; at the lead, provenance refers to immediate contributors.

Conflict Injection in Harvey LAB

We build on 11 Harvey LAB due-diligence tasks in which agents review document directories and write red-flags reports ([12]). Each task retains its original instructions and document collection, with controlled factual edits and conflict-specific grading criteria.

To plant conflict ii, we replace selected appearances of a recurring numerical value AiA_i with BiB_i in designated directories, leaving AiA_i unchanged elsewhere. Each conflict specification records the values, source locations, permitted equivalent representations, edits, and grading criteria. Five to seven conflicts appear per task, totaling 73; all conflicts in a task are evaluated against the same report.

Agents receive the ordinary task prompt and edited documents. The evaluator keeps the conflict specifications and grading rubrics separate from the agent inputs. A successful report flags the discrepancy, states the need for reconciliation, and accurately treats the quantitative comparison.

Delegation Policies

Every policy receives the same task and edited corpus. Under None, one agent inspects documents and writes the report. Under Optional, the lead may inspect sources itself, create workers, and assign overlapping reviews. Under Required, workers inspect assigned document partitions while the lead coordinates and synthesizes their findings without reading source documents. Workers can also contribute to a shared report draft. Optional and Required permit up to ten concurrent workers, with additional instances possible across continuation phases.

Optional measures whether models choose to delegate, whereas Required evaluates execution under mandatory delegation. It adapts the separation of coordination and execution used by Magentic-One, which routes tool use through specialized agents (Fourney et al., [10]), and the original AgentOrchestra planner, which delegates environment interaction to sub-agents (Zhang et al., 2025c). In our document tasks, this policy combines mandatory delegation, partitioned source assignments, and restricted lead access.

Evidence-Flow Profiling

Our profiler captures model requests and responses through a local proxy and correlates them with SDK events, native traces, tool outputs, and report artifacts. Agent and tool-call identifiers attribute evidence to its reader, sender, or recipient within the trajectory prefix that produced the graded report. Source evidence is counted from returned content, including search excerpts.

We follow four measurement stages. Source Reads captures endpoints visible in source-tool outputs; Worker Reports captures outgoing worker messages and completion reports, including peer-directed messages. Lead Evidence combines incoming evidence with the lead’s permitted source reads. Final Report contains endpoints in the selected, graded deliverable. Figures and tables abbreviate these as Source, Worker, Lead, and Final.

A uniform matcher identifies endpoints through fact-specific anchors and verified equivalent repre­ sentations. We track outgoing messages and incoming receipts separately, using plaintext collaboration traces for GPT-6 delegated executions. Source Reads, Worker Reports, and Lead Evidence accumulate observations through the graded checkpoint; Final Report measures the selected deliverable.

Pair Recall and Conflict Detection

For conflict observation ii, let Gi={Ai,Bi}G_i = \{A_i, B_i\} be its planted pair and let EitE_i^t contain the endpoints observed at stage tt, pooled across eligible agents or artifacts. Adapting FEVER’s complete-evidence-group evaluation (Thorne et al., 2018; Sheffield NLP, [36]), we define Pair Recall as

Rpairt=1N∑i=1N1[Gi⊆Eit].(1)R_{\mathrm{pair}}^t = \frac{1}{N}\sum_{i=1}^{N} \mathbf{1}\left[G_i \subseteq E_i^t\right]. \tag*{(1)}

A conflict earns pair credit when both endpoints are present, regardless of unrelated evidence. Table 1 labels this quantity Total at the first three stages and Both at Final Report.

ModelDelegationSource Reads TotalSource Reads SameSource Reads SplitWorker Reports TotalWorker Reports SameWorker Reports SplitLead Evidence TotalLead Evidence SameLead Evidence SplitFinal Report BothJudge PassCost (× budget)
GPT-5.6 LunaNone69.969.90.0———69.969.90.021.215.80.96
Optional67.165.12.10.70.70.063.063.00.026.023.30.95
Required78.832.945.924.018.55.524.018.55.511.610.31.02
GPT-6 LunaNone59.659.60.0———59.659.60.017.817.10.95
Optional65.163.71.40.70.70.063.763.70.019.919.20.95
Required61.626.035.618.513.05.518.513.05.512.311.60.96
Claude Haiku 5.5None21.221.20.0———21.221.20.015.815.80.95
Optional56.224.032.237.024.712.337.728.88.931.530.10.98
Required65.817.148.630.817.813.030.817.813.023.317.80.97
PooledNone50.250.20.0———50.250.20.018.316.20.95
Optional62.850.911.912.88.74.154.851.83.025.824.20.96
Required68.725.343.424.416.48.024.416.48.015.813.20.98

Table 1. Pair Recall and conflict detection under common generation allowances. All entries except Cost are percentages. Two runs per model yield n=146n = 146 conflict observations per model/policy (n=438n = 438 per pooled policy). Total and Both give Pair Recall; at the first three stages, Total is divided into Same and Split (gray) by reader or contributor (§ 3.4). A dash denotes no worker channel. Cost is mean generation spending divided by the common $59.60 per-round allowance, excluding judging. Cell shading is proportional to the value on a common 0–80% scale. Bold marks the highest Pair Recall and Judge Pass rate within each model or the pooled rows.

At the first three stages, we separate Same, where one individual supplies both endpoints, from Split, where only the contributors’ union contains both. Same takes precedence when multiple routes exist, giving Rpairt=Rsamet+RsplittR_{\mathrm{pair}}^t = R_{\mathrm{same}}^t + R_{\mathrm{split}}^t. These labels describe cumulative provenance across the trajectory, identifying readers at Source Reads, sending workers at Worker Reports, and immediate contributors at Lead Evidence. For example, under Optional, a lead-read AiA_i and worker-supplied BiB_i form Split, whereas one worker supplying both forms Same. Because Final Report is a single artifact, it uses Both without a Same/Split breakdown.

An LLM judge (claude-sonnet-5) grades the reports, using model-based evaluation (Zheng et al., 2023). Our rubric has two criteria per conflict: identifying the discrepancy and accurately treating its quantitative comparison. Judge Pass requires both criteria, including an explicit flag for the specified conflict; its rate measures conflict detection. We evaluate semantic grades alongside endpoint presence. Judge Pass evaluates each planted conflict individually.

Experimental Results

We first examine models’ delegation choices and worker contributions (§ 4.2). We then compare evidence collection with conflict detection and trace where Pair Recall declines (§ 4.3). Finally, we compare Required and None on matched cases grouped by whether one worker reads both conflicting claims or the claims are split across workers (§ 4.4).

Experimental Setup

We evaluate GPT-5.6 Luna and GPT-6 Luna through the Codex SDK, and Claude Haiku 5.5 through the Claude Code SDK. Each model runs each policy twice on all 11 tasks, yielding 198 task executions and 1,314 conflict observations. This gives 146 observations per model/policy and 438 per pooled policy, with repeated conflicts clustered within tasks.

Generation uses common per-task dollar allowances totaling $59.60 per round. When spending remains below a configured fraction of the allowance, the system continues its native session to review and revise the report, retaining its history and cumulative spending. Accounting covers all lead and worker calls, including revisions, compaction, and paid interruptions; judging is excluded. Because a phase finishes before the controller checks whether to stop, spending can exceed the target budget, with mean spending per model/policy ranging from 0.95 to 1.02 times the allowance.

Table 1 reports Pair Recall, evidence provenance, Judge Pass, and spending for each model/policy. Figure 3 summarizes the main patterns in these results.

Figure 3. Summary of observed patterns in delegation, evidence preservation, and conflict detection. (a) Model-dependent Optional behavior motivates measuring Required to separate delegation choice from delegated execution. (b) Required’s greater evidence collection translates poorly into Judge Pass. (c) Required has larger Pair Recall drops for all three models, with most of its loss between Source Reads and Lead Evidence. Worker Reports uses all 146 conflict observations per model/policy, including executions without workers; None has zero worker contribution. Percentage-point changes use unrounded rates.

Delegation Behavior and Worker Contributions

Under Optional, Haiku creates workers in all 22 executions, whereas GPT-5.6 does so in 9/22 and GPT-6 in 10/22 (Figure 3a). Workers also contribute different amounts of evidence. Across all 146 conflict observations per model, both GPT models’ Worker Reports achieve Pair Recall of only 0.7% (1/146) under Optional, compared with 24.0% under Required for GPT-5.6 and 18.5% for GPT-6. Haiku’s workers achieve 37.0% under Optional and 30.8% under Required.

Required lets us compare evidence preservation under mandatory delegation across models with different tendencies to delegate.

From Evidence Collection to Conflict Detection

Pooled across models and runs, Required increases Source Reads Pair Recall to 68.7%, compared with 50.2% under None, but Judge Pass falls from 16.2% to 13.2% (Figure 3b). Optional reaches 62.8% Source Pair Recall and 24.2% detection. Collecting more evidence at the source stage therefore does not guarantee higher conflict detection in the final report.

The gap is visible within each model. Relative

to None, Required raises Source Reads Pair Recall by 8.9 percentage points (pp) for GPT-5.6 and 2.1 pp for GPT-6, while detection falls by 5.5 pp for each. Haiku gains 44.5 pp in Source Pair Recall but only 2.1 pp in detection. Optional has the highest detection rate for all three models. fig:3c shows the difference between Source Reads and Final Report Pair Recall, divided into the decline before and after Lead Evidence. Required has a larger total decline than None for every model, with most of its decline occurring before the lead. Under None, source reads enter the lead’s evidence, so there is no drop between those two stages. Each bar measures percentage-point changes over all 146 conflict observations for that model and policy.

Across all 438 Required observations, Source Reads contain both claims in 301 cases (68.7%), compared with 107 (24.4%) at Worker Reports and Lead Evidence. Of the 301 pairs present at Source Reads, 106 also appear in Worker Reports (35.2%). Worker Reports and Lead Evidence have identical labels, locating the observed handoff gap between source inspection and worker reporting. Collective retrieval therefore often fails to provide the lead with both claims needed to detect a conflict.

Evidence Distribution in Matched Cases

Figure 4 examines whether conflict detection differs when one agent supplies both claims and when the claims are split across agents. Same and Split cohorts are defined by Required’s observed evidence distribution, with None evaluated on the identical model, run, task, and conflict instances. The upper group conditions on Source Reads and the lower group on Lead Evidence. Each cohort’s size nn applies to both policies, with percentages calculated within that cohort.

Figure 4. Examination of how multi-agent memory consistency relates to conflict detection via conditioned Sankey diagrams comparing matched cases. Pooled across runs, Required has higher Judge Pass than matched None in the Same cohorts, but lower Judge Pass in the Split cohorts. This pattern holds for both Source Reads and Lead Evidence groupings.

Pooled across runs, Required achieves Judge Pass on 38/111 observations (34.2%) in the Source Reads Same group, compared with 25/111 (22.5%) under None. In the Split group, it detects 19/190 (10.0%), compared with 36/190 (18.9%). Conditioning on Lead Evidence gives the same direction: 72.2% versus 33.3% for Same, and 17.1% versus 28.6% for Split. Required’s advantage over None is concentrated in cases where one agent supplies both claims.

Shared-draft reads and arithmetic reconstruction can change a pair from Split at Source Reads to Same at Lead Evidence. The two groupings therefore describe evidence contributors at their respective stages.

Conclusion

Answering where factual conflicts disappear in multi-agent LLM systems requires separating retrieval coverage from distributed memory consistency. While mandatory delegation (Required) broadens collective source retrieval over single-agent execution (None), net conflict detection drops because evidence fractures across two stages. First, at the worker-to-lead handoff, workers summarizing isolated partitions lack the cross-document context needed to recognize figures that appear unproblematic in isolation as conflicting across documents, omitting one or both claims in 73.2% of split-evidence cases compared with 50.5% when a single worker reads both. Second, even when both conflicting claims reach the lead, conflict resolution depends heavily on provenance: leads flag 72.2% of conflicts supplied by a single contributor (Same), yet only 17.1% when claims arrive divided across separate worker reports (Split).

For system designers, these findings caution against rigid, non-overlapping context partitioning on cross-document verification tasks; permitting overlapping source access and lead source inspection (Optional) consistently outperforms both single-agent and strictly partitioned execution. More broadly, end-to-end task accuracy masks opposing retrieval gains and coordination losses. Our benchmark and evidence-flow profiler allow practitioners to separate source discovery, handoff retention, and lead synthesis, providing a testbed for structured reporting protocols and conflict-preserving multi-agent memory.

Limitations

Our evidence comes from 11 Harvey LAB tasks with planted numerical conflicts and two runs per model. Broader document domains, non-numerical conflicts, and more repetitions would test how widely the observed patterns hold, with uncertainty estimates accounting for shared tasks. The profiler measures cumulative evidence presence; tracing when both claims become jointly available would extend this analysis. Manual inspection identified three saved Judge Pass labels that credited neighboring comparisons rather than the planted conflict; broader human review could refine endpoint matching and semantic grading.

Required combines mandatory delegation, partitioned source assignments, and restricted lead access, while model families use different SDKs. The matched groups also reflect agents’ observed retrieval and reporting choices. Supplying identical evidence while varying which agents receive it would help separate the effect of distribution from retrieval difficulty. Reporting interventions could then test whether preserving paired claims improves detection. Budgets were calibrated from observed spending, and GPT-5.6 Required checkpoints were selected retrospectively; future evaluations should fix budgets and checkpoint rules in advance.

References

  1. [1]Anthropic. 2024. Building effective agents. Engineering report, December 19.
  2. [4]Anthropic. 2025c. How we built our multi-agent research system. Engineering report, June 13.
  3. [5]Anthropic. 2026. Create custom subagents. Claude Code documentation, accessed November 16, 2026.
  4. [6]Mert Cemri, Melissa Z. Pan, Shuyi Yang, Lakshya A. Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandran, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. 2025. Why do multi-agent LLM systems fail? In Advances in Neural Information Processing Systems, Datasets and Benchmarks Track.arxiv.org/abs/2503.13657
  5. [7]Hung-Ting Chen, Michael Zhang, and Eunsol Choi. 2022. Rich knowledge sources bring complex knowledge conflicts: Recalibrating models to reflect conflicting evidence. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2292–2307. Association for Computational Linguistics.arxiv.org/abs/2210.13701
  6. [8]Mingxuan Du, Benfeng Xu, Chiwei Zhu, Licheng Zhang, Xiaorui Wang, and Zhendong Mao. 2026. DeepResearch Bench: A comprehensive benchmark for deep research agents. In International Conference on Learning Representations.arxiv.org/abs/2506.11763
  7. [9]Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. 2024. Improving factuality and reasoning in language models through multiagent debate. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 11733–11763. PMLR.arxiv.org/abs/2305.14325
  8. [10]Adam Fourney, Gagan Bansal, Hussein Mozannar, Cheng Tan, Eduardo Salinas, Erkang Zhu, Friederike Niedtner, Grace Proebsting, Griffin Bassman, Jack Gerrits, Jacob Alber, Peter Chang, Ricky Loynd, Robert West, Victor Dibia, Ahmed Awadallah, Ece Kamar, Rafah Hosn, and Saleema Amershi. 2024. Magentic-one: A generalist multi-agent system for solving complex tasks. ArXiv:2411.04468.arxiv.org/abs/2411.04468
  9. [11]Mingyan Gao, Yanzi Li, Banruo Liu, Yifan Yu, Phillip Wang, Ching-Yu Lin, and Fan Lai. 2025. Single-agent or multi-agent systems? Why not both? ArXiv:2505.18286.arxiv.org/abs/2505.18286
  10. [12]Niko Grupen, Gabe Pereyra, and Julio Pereyra. 2026. Open-sourcing harvey’s long horizon legal agent benchmark. Benchmark release, May 6.
  11. [13]Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang. 2024. Large language model based multi-agents: A survey of progress and challenges. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 8048–8057.
  12. [14]Shizhe He, Avanika Narayan, Ishan Khare, Scott Linderman, Christopher Ré, and Dan Biderman. 2026. An information theoretic perspective on agentic system design. In International Conference on Learning Representations.arxiv.org/abs/2512.21720
  13. [15]Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmid­huber. 2024. MetaGPT: Meta programming for a multi-agent collaborative framework. In International Conference on Learning Representations.arxiv.org/abs/2308.00352
  14. [16]Yufang Hou, Alessandra Pascale, Javier Carnerero-Cano, Tigran Tchkr­achian, Radu Marinescu, Elizabeth Daly, Inkit Padhi, and Prasanna Sat­tigeri. 2024. WikiContradict: A benchmark for evaluating LLMs on real-world knowledge conflicts from Wikipedia. In Advances in Neural Information Processing Systems, volume 37, pages 109701–109747.
  15. [17]Shengran Hu, Cong Lu, and Jeff Clune. 2025a. Automated design of agentic systems. In International Conference on Learning Representations.arxiv.org/abs/2408.08435
  16. [18]Yuyang Hu, Shichun Liu, Yanwei Yue, Guibin Zhang, Boyang Liu, Fangyi Zhu, Jiah­ang Lin, Honglin Guo, Shihan Dou, Zhiheng Xi, Senjie Jin, Jie­jun Tan, Yanbin Yin, Jongnan Liu, Zeyu Zhang, Zhongxiang Sun, Yutao Zhu, Hao Sun, Boci Peng, and 28 others. 2025b. Memory in the age of AI agents. ArXiv:2512.13564.arxiv.org/abs/2512.13564
  17. [19]Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2024. SWE-bench: Can language models resolve real-world GitHub issues? In International Conference on Learning Representations, pages 54107–54157.arxiv.org/abs/2310.06770
  18. [20]Prathyusha Jwalapuram, Hehai Lin, Chuyuan Li, Fangkai Jiao, Sudong Wang, Yifei Ming, Zixuan Ke, Chengwei Qin, Giuseppe Carenini, and Shafiq Joty. 2026. The illusion of multi-agent advantage. ArXiv:2606.13003, version 2.arxiv.org/abs/2606.13003
  19. [21]Yubin Kim, Ken Gu, Chanwoo Park, Chunjong Park, Samuel Schmidgall, A. Ali Heydari, Yao Yan, Zhihan Zhang, Yuchen Zhuang, Yun Liu, Mark Malhotra, Paul Pu Liang, Hae Won Park, Yuzhe Yang, Xuhai Xu, Yulin Du, Shwetak Patel, Tim Althoff, Daniel McDuff, and Xin Liu. 2026. Towards a science of scaling agent systems. ArXiv:2512.08296, version 3.arxiv.org/abs/2512.08296
  20. [22]Philippe Laban, Alexander Fabbri, Caiming Xiong, and Chien-Sheng Wu. 2024. Summary of a haystack: A challenge to long-context LLMs and RAG systems. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 9885–9903. Association for Computational Linguistics.
  21. [23]Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dimitrii Khizbullin, and Bernard Ghanem. 2023. CAMEL: Communicative agents for “mind” exploration of large language model society. In Advances in Neural Information Processing Systems, volume 36, pages 51991–52008.arxiv.org/abs/2303.17760
  22. [24]Jierui Li, Vipul Raheja, and Dhruv Kumar. 2024. ContraDoc: Understanding self-contradictions in documents with large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 6509–6523. Association for Computational Linguistics.
  23. [25]Yuxuan Li, Aoi Naito, and Hirokazu Shirado. 2026. Systematic failures in collective reasoning under distributed information in multi-agent LLMs. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pages 69250–69268. PMLR.arxiv.org/abs/2505.11556
  24. [26]Tobias Lindenbauer, Igor Slinko, Ludwig Felder, Egor Bogomolov, and Yaroslav Zharov. 2025. The complexity trap: Simple observation masking is as efficient as LLM summarization for agent context management. ArXiv:2508.21433.arxiv.org/abs/2508.21433
  25. [27]Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173.arxiv.org/abs/2307.03172
  26. [28]Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. 2021. Entity-based knowledge conflicts in question answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7052–7063. Association for Computational Linguistics.
  27. [29]Adyasha Maharana, Dong-Ho Lee, Sergey Tulaykov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. 2024. Evaluating very long-term conversational memory of LLM agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13851–13870. Association for Computational Linguistics.
  28. [30]OpenAI. 2026. Subagents. Codex documentation, accessed November 16, 2026.
  29. [31]Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. 2023. MemGPT: Towards LLMs as operating systems. ArXiv:2310.08560.arxiv.org/abs/2310.08560
  30. [32]Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, pages 1–22. ACM.arxiv.org/abs/2304.03442
  31. [33]Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahai Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2024. ChatDev: Communicative agents for software development. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15174–15186. Association for Computational Linguistics.arxiv.org/abs/2307.07924
  32. [34]Jeanne Salle, Hardik Bhatnagar, Matthias Bethge, Sahar Abdelnabi, and Maksym Andriushchenko. 2026. Benchmarks are surprisingly compaction-friendly. ArXiv:2610.31778.arxiv.org/abs/2610.31778
  33. [35]Yijia Shao, Yucheng Jiang, Theodore A. Kanell, Peter Xu, Omar Khattab, and Monica S. Lam. 2024. Assisting in writing Wikipedia-like articles from scratch with large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 6252–6278. Association for Computational Linguistics.
  34. [36]Sheffield NLP. 2018. FEVER scorer. Official evaluation implementation; accessed November 6, 2026.
  35. [37]Noah Shinn, Federico Cassano, Ashwin Gopinath, Kartik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, volume 36, pages 8634–8652.arxiv.org/abs/2303.11366
  36. [38]Andries Petrus Smit, Nathan Grinsztajn, Paul Duckworth, Thomas D. Barrett, and Arnu Pretorius. 2024. Should we be going MAD? A look at multi-agent debate strategies for LLMs. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 45883–45905. PMLR.arxiv.org/abs/2311.17371
  37. [39]Renan Souza, Amal Gueroudji, Stephen DeWitt, Daniel Rosendo, Tirthankar Ghosal, Robert Ross, Prasanna Balaprakash, and Rafael Ferreira da Silva. 2025. PROV-AGENT: Unified provenance for tracking AI agent interactions in agentic workflows. In 2025 IEEE 21st International Conference on e-Science, pages 467–473. IEEE.

Paper details

vlnc:W_83matekm
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents