WorksComputer Science
Cleaning an Agent's History Hurts When the World Still Holds the Error
Abstract
Tool-using language-model agents accumulate histories of their own mistakes, and a system that compacts, repairs or hands off such a history decides what it says about those mistakes while their effects stay in the world. Prior work cleans the history at a fixed world, where showing a corrected step and saying it is done move together. We ask whether the same correct content helps or hurts depending on how the history states it and what the world holds. We freeze an agent mid-task with its exact world state, edit the history or the world, and let a receiver model continue. On AgentCE-Bench, after the agent commits a wrong value, we hold the world fixed and add the correct value by two natural edits: a truthful note beside the preserved error or the agent's own completed action. The note raises recovery, the share of remaining targets the receiver gets right, from 51% under a matched neutral note to 66%. The completed action lowers it below the unedited history, from 50% to 44%, so the value's presence alone does not predict the effect. On AppWorld, after a database write, we vary whether its receipt reports success or failure and whether its effect is present. A success receipt lowers whole-task success from 45% to 28% while the effect is absent and raises it from 38% to 49% once present. The two effects differ by 27 points pooled over two receivers, a difference positive in six of eight receivers at 97.5% and seven at 95%. A history edit's effect thus depends on the world it describes, and a history made to look clean while the world still holds the error can leave an agent worse off than no edit at all.
CCS Concepts
• Computing methodologies → Intelligent agents; Natural language processing; • General and reference → Empirical studies.
Introduction
Systems that manage the history of a tool-using agent compact a long context [17] [40], repair it after a failed step [20], or hand a task to another agent [8, 18]. That history, the agent’s record, holds its actions, the receipts the environment returned, and its reasoning [45], mistakes included. Every such system decides what it says about them. An answer these systems lean toward, to clean the record, is the one we find hardest to support. A recorded error can mislead the agent, but it is also evidence that the world has changed, and deleting it does not undo the change [10] [46]. Prior interventions edit the record at a fixed world. Rewrite a failed step as a success and the agent both sees the correct value and is told the step is done; prior work cannot tell which drives the change. We separate the two with two designs on two benchmarks, AgentCE-Bench [44] and AppWorld [35], each letting a live receiver, the model that takes over, continue an exactly restored trajectory (Figure 1). The first varies the record over a fixed world, the second the world under a fixed record. We ask whether the same correct content helps or hurts, depending on its context and the world it describes.
Figure 1. The two designs, record above the line and world below. Left, AgentCE-Bench: a world holding a committed error under two records carrying the same correct value, on recovery. Right, AppWorld: one original success receipt over two worlds, on whole-task success. Each mark is an effect pooled over gpt-5.5 and claude-sonnet-5, with its 95% interval against the comparator it names; blue is a gain and orange a loss, in every figure. In each design the effect’s sign changes with the half varied.
In the first design, on AgentCE-Bench, at 319 frozen states where an agent has committed a wrong value to persistent state, we place the same verified-correct value into the record by two natural edits. Recovery is the share of remaining look-ahead targets the agent gets right. Appended as a truthful note beside the preserved error, the value raises recovery from 51% to 66% against a neutral note matched in role, placement, turn count and length. Recorded instead as the agent’s own completed action, it lowers recovery below the unedited record, from 50% to 44%; this loss is the half that bears on compaction and repair (§ 5.1). The gap between them, 22 points, is the quantity fixed before collection, positive in both receivers.
Why would the correct value, standing in the agent’s own call, hurt? The two edits disclose the same value but differ in more than truth (§ 4.2, § 6). A copying account, in which the agent re-emits a value it can see, predicts the falsified record comes out ahead; it comes out behind, so the answer’s presence alone does not explain the result (§ 5.2). Two readings stay open: the record as a map of remaining work (§ 5.3) and a role effect favouring the note (§ 6).
In the second design, on AppWorld, we freeze an agent after a database write and induce the disagreement ourselves (§ 4.3). We keep the write’s original success receipt or swap in a failure receipt, and independently set the database to its state before or after the write, so in each world exactly one receipt is true; the two receipts also differ in content. A receiver, gpt-5.5 or claude-sonnet-5, then continues. While the effect is absent, the success receipt lowers whole-task success from 45% to 28%, the AppWorld form of a clean-looking record over an unfixed world. Once the effect is present, it raises success from 38% to 49%. The interaction, 27 points pooled over the two receivers, is positive in each. Over eight receivers from four providers it is positive in six at 97.5%, the level it is read at, and in seven at 95%; gpt-4o-mini is unresolved at either level (§ 5.5).
Our contribution is a controlled study of how a history edit acts on a tool-using agent, in three parts: 1) two controls that separate what a record asserts from the value it discloses, the matched neutral append and the falsified record, asserting that the agent set a correct value the world lacks, a control that to our knowledge no prior restorable-state intervention carries (§ 2, § 4.2); 2) a sign reversal at a fixed world on AgentCE-Bench: the same correct value raises recovery as an appended note and lowers it as the agent’s own completed action (§ 5.1 and § 5.2); and 3) a sign reversal at a fixed record on AppWorld, over eight receivers: a success receipt’s effect depends on the world, positive in six of the eight at 97.5% and seven at 95% (§ 5.4 and § 5.5).
For history management the consequence is practical. On AgentCE-Bench, a record made to look clean while the world still held the error left agents worse off than no edit at all; on AppWorld, a success receipt over an absent effect cost 17 points against the failure receipt. A compaction, repair or handoff policy has to be judged against the world. Exact restoration can score it on the task, at a chosen frozen state, before deployment.
Related Work
To our knowledge, no prior work varies a record’s truth over a fixed world, or the world under a fixed record.
Phenomenon. Success falls once the context holds the model’s own earlier mistakes, a self-conditioning effect that larger models do not escape [33], on long tasks whose success already falls with horizon [39] and whose failed runs are often decided by one early error [49]. Self-conditioning is established on injected histories with no environment, so record and world stay entangled. We restore the world beneath the record.
History management. Compaction rewrites a record’s length and content, compressing observations and history for task success [17] or silently dropping the constraints a session set [40]. Repair reverts record and world together, by rollback that replays an earlier action prefix [20] or by rewinding aligned checkpoints of context and environment [51], so the two are never put into disagreement. Handoff varies what a successor is shown over a frozen repository, every view truthful [18], checks a successor’s completion against live receipts [8], or varies the continuing model [9]. Self-correction work varies the record alone, with no world: a critique-and-revise loop that does not help unaided [15], the error’s apparent author [36] or message role [4], or its location by truncation [37].
Restorable-state edits. Editing a record at a restorable state is our setting, not our contribution. The handoff [18] and the rewind [51] above, and a replay intervening on one step [30], restore an executable state and vary what the agent sees. Wu et al. Wu et al. [42] edit the record instead, splicing fabricated receipts into a frozen prefix, but execute nothing. Indirect prompt injection also enters through tool output [6, 11, 47], but plants third-party instructions, not a false receipt for the agent’s own action. To our knowledge, none of these holds what the record asserts apart from what the world discloses. A 360-passage survey found no such passage among the 53 that vary what an agent is shown at a fixed executable state (Appendix § 16).
Controls. Three papers, none at a restorable state, separate what a record discloses from what it asserts, [13] and [26] fully and [14] in part, since in [14] the value is disclosed through the tool, not the memory. No paper we read carries a control isolating the act of appending, matched on role, apparent author, placement, turn count and length; fifteen discuss such a control and each says it lacks one. That absence overstates the distance, since our own control matches four of the five, not apparent author (§ 4.2). Self-preference, found with authorship shown [26] [29] and without it [41], bears on that one unmatched attribute. In small code models repairing failed programs, a content-free notice matches genuine execution feedback [38], and a length-matched placebo span serves the same role elsewhere [16]. Position effects [22] are why our append is matched on placement. We therefore run the neutral append as an arm rather than assume appending is inert.
Nearest work on the falsified half. The work most similar to our falsified record is Gumaan [13], in models of 135M to 1.7B parameters. It keeps one failing action string byte-identical and finds that the agent re-emits the failed call more often once it is recorded with its error, while swapping only the observation moves little. Like us, it holds what the record asserts apart from the value it discloses, and it comes closer to the append control than any other work we read, though its no-append condition drops the failed call as well as the note. Its counterfactual probe scores the next action without running the agent, and its executed rollouts vary how the failure is written into the context rather than the observation. There, deleting the failed step and retrying from a clean context repeats the failed call most and does not raise task success. We execute every contrast as a continuation over a restored world and add the matched neutral append.
Nearest work on the truthful half. The work most similar to our truthful note is Lin et al. [21], which injects diagnostic annotations into an agent’s own record. It separates content, correctness and placement: correction accuracy 78.7% with a correct hint, 46.3% with a placebo and 35.6% with no hint. Its closest falsified condition is a contradictory hint, a false guidance note rather than a falsified record of what happened, so it cannot compare an asserted value against a disclosed one at the same world. Its placebo lies above no hint; our neutral append is unresolved on AgentCE-Bench (§ 5.1) and not inert for one AppWorld receiver (§ 5.6). Our truthful note bundles the content, correctness and placement that [21] separates (§ 4.2, § 6).
Crossed designs. We claim no novelty for a design or its interaction: Xiao et al. [43] cross memory against the next question, the route by which a wrong answer shapes the simulated user’s next question. Ours crosses the record against an executable consequence that persists on its own. Record–world disagreement is also classical in agent theory: execution monitoring [5] [27], epistemic planning [2], diagnosis of unrecorded actions [24], BDI effector reports [3] and reconsideration [19], trust [28], belief revision [32] and sensing that corrects a believed success [7] (Appendix § 16). Those accounts prescribe how an agent should revise. We measure what a language-model receiver does against two of them: an agent trusting its receipts and one re-checking the world (§ 5.4).
Problem Setting
We define the objects the two designs share and the two quantities they estimate. The first design holds the world fixed and estimates ; the second crosses the record with the world and estimates .
Record and world. An agent’s frozen state has two parts. The record , the agent’s history, is everything the agent is shown: the task, its own past actions, the receipts the environment returned for those actions, and its reasoning in between. The world is the executable state those actions produced: database rows, committed values, files on disk. describes , and nothing enforces that the description is correct. After a mistaken action, carries the consequence while carries an account that may be complete, partial, or false.
Frozen state. A frozen state is one pair stored so that it can be re-entered any number of times, as a packet (§ 4.1); every comparison is within one state.
Edit. An edit maps a record to at a frozen state and leaves untouched. It reveals what the agent can read off but not off , and conceals what it could read off and now cannot. It asserts the account gives of what happened: who acted, what it achieved, and what is left to do. Revealing and asserting come apart. At a state where the agent has committed a wrong value and the correct one is known, that value can be placed as a third party’s note or as the agent’s own completed action. The two edits disclose one string and assert opposite things about , though they differ in more (§ 4.2, § 6). Every edit here rewrites one call or receipt, or appends one note; the rest of is unchanged but for padding (§ 4.2).
Continuation. A continuation restores , hands control to a receiver model, and lets it act under a fixed step budget. Its endpoint scores the terminal world against the task’s own specification: on AgentCE-Bench, recovery, the share of the look-ahead targets (§ 4.1) holding the correct value when the continuation ends; on AppWorld, the environment’s own whole-task success. No endpoint scores the record.
Estimand 1, the record edit at a fixed world. At a frozen state where the agent has committed a wrong value to persistent state, four arms restore the same (§ 4.2). is the original record, padded for length (§ 4.2). is falsified so that the agent’s own completed action asserts the correct value was set while still holds the wrong one. appends to a truthful note naming the mistaken action and giving the correct value. appends a content-free note matched to . Writing each letter for the mean endpoint under that arm, is what the truthful note adds over a matched empty note, and is what the falsification adds over the untouched record. The estimand compares the two edits, each against its own control; a positive says the truthful note helps more than the falsified record does. We call this design’s four conditions arms and the second design’s four cells.
Estimand 2, the record crossed with the world. On AppWorld (§ 4.3) we cross two factors at each frozen state, which we take at a database write we call the focal step. The record factor sets the focal step’s receipt to assert that the step succeeded (succ) or failed (fail) (§ 4.3). The world factor sets the database to sit before (pre) or after (post) that step. In each world exactly one receipt is true: fail in pre, succ in post. Writing each cell for its mean endpoint, the estimand is . It measures how far a success receipt’s advantage over a failure receipt depends on the world.
Scope. In both designs we choose the state, the error and the edit, so neither estimates how often records and worlds disagree in use; they measure what an edit does once they do. The disagreement does arise unprompted. Of the 1,350 AgentCE-Bench source episodes’ §4.2’s states come from, 1,074 called done, and in 393 of those the world held a wrong value the agent’s own receipt had confirmed as set. On AppWorld, by contrast, all 72 successful writes in its 75 source attempts had their effect in the database, so there we introduce it.
Method
Benchmarks, restoration and receivers
Benchmarks. AgentCE-Bench [44], our primary benchmark, generates tool-using environments of controllable difficulty. The agent fills a grid: some squares are hidden slots, and the agent queries tools to find each slot’s value and commits a candidate to it, for example a meal or a venue for one square of a weekly plan. frozen state a slot is outstanding if it does not hold its true value, a wrongly committed slot included. The look-ahead targets, fixed before any arm is applied, are the first three outstanding slots in the task’s order (two where only two remain). A wrong commitment is scored against ground truth rather than by an LLM judge, since no judge tops 70% precision [23]. AppWorld [35], the second, exposes everyday applications as executable APIs over a populated database; the agent writes Python to satisfy a request, and the environment’s evaluator scores the result. Nothing is pooled across the two, and their endpoints (§ 3) compare in sign, not magnitude.
Restoration. Restoration re-enters a packet in a fresh process. We certify it as replay audits do [48]: under a scripted policy, five restored clones and one live continuation of each certification state must end in one identical final state and grader result. AgentCE-Bench passed 200 of 200 (Clopper–Pearson 95% lower bound 0.9817 over states, 0.9762 over its 153 task instances); AppWorld passed 20 of 20 (0.832). The experimental states are disjoint from the certification states, except one AppWorld task family (§ 4.3); each of the 319 AgentCE-Bench packets was checked against its frozen snapshot, and all 65 AppWorld checkpoints were restore-verified.
Receivers. gpt-5.5 and claude-sonnet-5 run on both designs. The AppWorld design adds six: gpt-5, gpt-5-mini, gpt-4o and gpt-4o-mini (OpenAI), gemini-3.1-pro-preview (Google) and grok-4.6 (xAI), for eight receivers from four providers. Each runs under its benchmark’s own scaffold and system prompt with no sampling temperature set; Appendix A.2 gives routes, decoding and ids, and Appendix J the verbatim arm texts.
Calibration. We calibrate difficulty per receiver and benchmark to a 25 to 75% success band on a disjoint calibration pool, because an effect worth a few points cannot show through a floor or a ceiling. On AppWorld’s pool claude-sonnet-5 sat below the band and gpt-5.5 passed inside it (47.5% [34.3, 60.9] of finished episodes); we kept claude-sonnet-5 as a named below-band receiver and treat every comparison with it as confounded with operating point. In the collected cells claude-sonnet-5’s means run from 16.2% to 28.5% and gpt-5.5’s from 40.8% to 69.2%. The added receivers ran without recalibration, and grok-4.6 sits at the ceiling in two cells (90.5% and 92.1%).
The four-arm design on AgentCE-Bench
States. The 320 frozen states come unmodified from an earlier study of ours, each taken two-thirds of the way through a task, after a wrong value was committed to persistent state. A rule-governed builder verifies each candidate’s wrong action, its linked successful receipt and the snapshot’s own error detector; one state is dropped because its agent self-corrected, leaving 319. The trajectories came from gemini-3.7-flash or gpt-5.6-luna (149 and 170 of the 319), neither a receiver here, so every continuation inherits another model’s history (§ 6). The erroneous slot is a look-ahead target in 272 of them. The earlier study measured other contrasts on these states, never .
Arms. The four arms of §3 are built as follows. Agents produce the record–world mismatch on their own when they falsely claim success [1]. F induces it by replacing the selected call’s identifier and receipt with the correct ones. T’s user-role message names the erroneous action and its receipt, the correct identifier and the still-wrong state. N’s user-role message says it holds no task guidance and no detail of any earlier step, and a one-token filler pads the packet to T’s exact cl100k token count. O and F are padded to each other’s length with a declared state-neutral filler. O’s only change (Appendix J.2).
Held fixed. The world is fixed across all four arms, and the disclosed value across T and F. Those two differ in channel and apparent author (a user note against the agent’s own call), position, whether the wrong value stays visible, whether work is said to remain, and truth. With no false note and no truthful in-place edit, compares two natural edits, not cells of a factorial (§6). T and N match on role, placement, turn count and cl100k token count only; they do not match on information, authority wording or naturalness, nor on apparent author.
Collection. Each receiver runs one fresh continuation per arm and state on a 24-step budget, giving 638 complete four-arm blocks and 2,552 continuations. Block order is randomised and arm permutations balanced. All 638 blocks completed on their first attempt. The budget binds unevenly: 71.4% of continuations use all of it, from 60.3% under T to 80.1% under F, so part of could be steps the budget cut off (§6).
The record × world design on AppWorld
Source episodes. A source episode runs unmanipulated under gpt-5.5, within a 120-step ceiling, until the first API call that changes application data, succeeds, is not an authentication step and leaves the task unfinished. The episode is frozen at that call, the focal step. We induce the record–world disagreement, as an aborted transaction [25], a stale receipt or a corrupted tool output [50] would; on AgentCE-Bench the agent’s own mistake causes it. Every receiver but gpt-5.5 continues another model’s trajectory, a cross-model handoff [9]; gpt-5.5 continues its own (§6).
Receipts. The record factor (§3) touches nothing in the record but the focal receipt. The failure receipt is one fixed string in the environment’s own error format, “Execution failed. Traceback: RuntimeError: The operation did not take effect; no application data was changed.” The success receipt is the API’s own return, a confirmation message sometimes carrying an identifier such as a sent email’s id. The receipts thus differ in content as well as truth, and only the failure receipt states explicitly that no data changed.
Cells. Succe{p}, the original receipt over the completed write, is the one cell untouched beyond the shared handoff. In succe{p} the record claims a write the world does not hold; in fail{post} it denies one the world holds. Restoring reinstates the database and transcript but not the source agent’s interpreter state, so every cell is a handoff into a fresh interpreter, with no uninterrupted baseline.
Frame. The frame, the task families we estimate on, comes from a census of the 70 AppWorld test families none of our earlier work had used, one hash-selected instance each. Under a criterion fixed ahead of any outcome, at least 50 had to validate, the source episode reaching an eligible focal step. The 65 that validated form the frame; the other five failed before any continuation ran, for reasons our logs did not capture, so the frame is the families whose hash-selected instance reached a qualifying write (Appendix C.1).
Collection. Each family’s checkpoint receives four cells × two receivers × two fresh repetitions: 16 continuations per family, 1,040 in 260 four-cell blocks. Every continuation runs on a 30-step budget, the calibration probes’ cap; it stops only 4 of 1,040, against 71.4% on AgentCE-Bench. Rescoring every stored final database state with the environment’s evaluator reproduces all 1,040 endpoint labels. Rescoring checks the scorer, not the run, so we also re-executed each continuation’s recorded code from its restored start state: all 1,040 labels recur, and 1,037 final database states match exactly, the other three differing only in a randomly drawn delivery time.
Estimation
AgentCE-Bench. Each design has one frozen analysis contract, the written specification of unit, estimand, interval and seed. The prewritten estimand (§ 3) carries an unadjusted 95% interval from 10,000 task-cluster bootstrap draws, the level set when it was the only confirmatory quantity planned. The four-arm design’s prewritten script ran as written and produces every figure in § 5.1; its slot decompositions and the N–O row are descriptive.
AppWorld. The AppWorld contract produces every figure on this benchmark with zero model calls. A unit is one restored continuation; intervals come from 10,000 percentile-bootstrap replicates that resample task families. The prewritten primary quantity is the interaction , read at 97.5% because the design reserved two 97.5% intervals across two confirmatory primaries, and the targeted control’s; and the two record effects it is built from are read together as one prewritten conjunction, not as three primary findings. and come from separate collections, and nothing is adjusted between them. Every other contrast carries an unadjusted 95% interval. Beyond the one receiver difference the contract prewrote (§ 5.4), we estimated no between-receiver contrast: on AppWorld an agent’s main effect is smaller than its agent-by-task interaction [34].
Verdicts and missing data. A prewritten rule returns one of three verdicts: resolved away from zero in a stated direction; ruled out, when an interval excludes the effect a reading needs; or unresolved. Unresolved is a verdict: we never round a bound to the threshold it failed to clear, nor read it as a weak positive or as evidence of absence. A block may be re-attempted whole, and only its first complete attempt enters an estimate; nothing is stitched or imputed. Ordinary and budget-exhausted zeros are outcomes; model, restore and spending-ceiling stops are missing data. The spending ceiling (§ 6 per continuation on AgentCE-Bench, $4 elsewhere) stopped no unit in either primary design or the targeted control. It stopped 36 grok-4–6 units, in AppWorld attempts that never completed (§ 5.5). The stops fell near-equally on the success and failure receipts.
Deviation on AppWorld. The preregistered script runs only after a sealed final-state replay, whose closure file the collector cannot write (Appendix § 15.§ 1). The script has therefore never run on the collected data; every AppWorld figure comes from a reimplementation of the same contract with the replay gate removed. A second implementation, written from the contract without importing the first, agrees on 16 of 16 contrasts and 12 of 12 cell means, and the re-execution of § 4.3 does what the replay was for (a deviation; § 6).
What was fixed in advance. was fixed before its own pilot ran (12 states, 96 continuations). was fixed as a confirmation after its pilot had been read: 12 training-split families and 96 continuations, with a positive but unresolved interaction, +16.67 pp [−8.33, 37.50] at 97.5%. No state was selected on any effect size and no pilot continuation is pooled.
The targeted control. The targeted control (§ 5.3; Appendix § 12) has its own frozen contract, but its preregistered estimator could not run because the collection stopped short (§ 6), so its analysis frames and companion endpoint, which scores the slot a claim names, were chosen after collection stopped. Two later implementations, the second written without importing the first, agree on 50 of 50 contrasts; that guards against coding error, not against choosing frames after seeing the data. The preregistered frame, restricted to complete blocks, gives +4.627 pp [2.990, 6.294] at 97.5% against the post-hoc +4.628 pp, with every verdict unchanged. Its primary contrast, specificity (§ 5.3), with its 3 pp margin rule, and Appendix § 12.§ 4’s step contrasts are read at 97.5%, the rest at 95%; re-execution reproduced every endpoint (Appendix § 14).
Results
The sign reversal on AgentCE-Bench
Figure 2 reports the four-arm design of § 4.2. Of ’s two halves, F–O, the completed action against the unedited record, bears on compaction and repair: what changed the record costs while the world still holds the error. Recovery is 50.13% [46.32, 53.97] under the unedited record and 50.57% [46.76, 54.39] under the matched neutral note; the neutral note’s own effect, +0.44 pp [−1.36, 2.32], is unresolved. The truthful note beside the preserved error raises recovery to 65.78% [62.54, 68.97], +15.20 pp [12.46, 18.00] over the neutral note. The same value recorded as the agent’s own completed action lowers it to 43.70% [40.18, 47.15], −6.43 pp [−8.88, −3.97] against the unedited record. The gap between the two effects, the prewritten primary quantity = +21.63 pp [18.21, 25.08], is positive in both receivers. Contrasts are read as nothing: the note beats the unedited record by +15.65 pp [13.06, 18.31], the completed action falls below the neutral note by −6.87 pp [−9.46, −4.41], and even the neutral note’s upper bound is far below the gap, so neither control drives it.
Figure 2. The sign reversal at a fixed world, on AgentCE-Bench, pooled over gpt-5.5 and claude-sonnet-5 (top) and per receiver. T−N (blue): the truthful note against a matched neutral note. F−O (orange): the same value as the agent’s own completed action, against the unedited record. The grey band spans the two marks; the black mark is their gap, D = (T−N) − (F−O). The T−O and F−N rows compare each edit against the other edit’s comparator; the bottom row is the control N−O. Hollow marks an interval that includes zero. Recovery on the look-ahead targets; 95% task-cluster bootstrap intervals.
needs a verified correct value, which a deployed history manager may lack, so T–N, the truthful note against the neutral note, is diagnostic. Two receivers do not make it model-independent, and it bundles information, wording and placement (§ 5.2–§ 5.3).
The gain is local (Table 1). Of T–N’s +15.20 pp, +11.57 pp falls at the slot the note names and +3.63 pp at the other look-ahead slots. On the 47 states whose committed error lies outside the look-ahead targets, the pooled contrasts are unresolved and T–N is negative for clause–sonnet–5 and positive for gpt–5.5 (Table 1, last two rows).
Presence of the answer does not predict the sign
The falsified record shows the correct value in the agent’s own call and the unedited record shows the wrong one. A simple copying account, in which the agent re-emits a value it can see, therefore predicts the falsified record should come out ahead. It comes out 6.43 pp behind, so the answer’s presence alone does not explain it.
| Contrast (look-ahead endpoint) | pp [95%] |
| T−N, all 319 states | +15.20 [12.46, 18.00] |
| part at the slot the note names | +11.57 [9.85, 13.43] |
| part at the other look-ahead slots | +3.63 [1.67, 5.64] |
| unmentioned slots, renormalised | +5.75 [2.56, 8.93] |
| F−O, all 319 states | −6.43 [−8.88, −3.97] |
| part at the slot the edit names | −2.30 [−3.68, −0.97] |
| part at the other look-ahead slots | −4.13 [−5.98, −2.27] |
| T−N, 47 error-outside states | −1.42 [−6.74, 3.19] |
| D, same 47 states | +1.77 [−5.32, 8.16] |
| T−N, same 47, claude-sonnet-5 | −10.64 [−19.15, −2.13] |
| T−N, same 47, gpt-5.5 | +7.80 [0.71, 14.89] |
Table 1. Where the gain and the loss fall on AgentCE-Bench, on the three-slot look-ahead endpoint of §4.1; unadjusted 95% task-cluster bootstrap intervals. Each 319-state total, T−N then F−O, is split between the slot the edit names and the other look-ahead slots; “renormalised” is T−N as a rate over only the slots the note does not name. The last four rows restrict to the 47 states whose committed error lies outside the look-ahead targets. All but the two totals are descriptive.
Two accounts in which the agent uses what it sees do predict the direction; both are disclosure-as-used (§5.3). A repair account has the agent notice the wrong value and fix it, which models do poorly unaided [15] and better when regeneration restarts at an oracle-located error [37]. A map account has the agent read the record as a map of what remains, in which a slot marked done is not revisited. The map account predicts the falsified arm’s loss at the slot it marks done, but falsifying one call costs more in total on the other look-ahead slots, −4.13 pp, than at the named one, −2.30 pp, and on the 47 states where the named slot is not scored D is unresolved (Table 1). We lack a mechanism for the loss on unnamed slots.
Neither disclosure-as-used reading is closed: the truthful note says work remains and the falsified record says none does, so either effect might turn on what is left to do, not truth; §5.3 narrows this.
A targeted control on disclosure-as-used
A nine-arm control on the same benchmark and receivers, not a replication (Appendix § 12), asks whether a false success claim steers the receiver by where it points rather than by what it reveals. Its primary quantity, specificity, is the gain on slots no arm named and no world repaired when the false claim matches the repaired slot, against when it does not; it is +4.628 pp [2.989, 6.317] at 97.5%. No arm hands the slot a correct value, so the strong form of disclosure-as-used, that the receiver was handed an answer, is ruled out by construction; the thin form, that the record tells the receiver where work remains, survives. The reading is limited three ways (§6): the collection is incomplete and unsealed; the prewritten 3 pp margin is unresolved at a lower bound of 2.989 pp; and the pooled figure rests on the later of two focal slots, so a positional account, or effort moved between slots under the binding step budget, fits as well, though the gain does not come from taking more steps (Appendix § 12.§ 4).
The record and the world interact on AppWorld
So far the record has varied over a fixed world; the AppWorld design turns this around and varies the world under a fixed record. Figure 3 reports the record × world design of §4.3. With the focal step’s effect absent, pooled whole-task success is 28.46% under a success receipt and 45.38% under a failure receipt; with it present, 48.85% and 38.46%. The two cells where record and world agree, fail and succ, differ by only 3.47 pp. A success receipt changes success by −16.92 pp [−24.62, −9.62] while the effect is absent and by +10.38 pp [3.46, 17.69] once present, both at 95%. The interaction, the arrow in Figure 3, is +27.31 pp [14.23, 41.54] at 97.5%, positive in each receiver (Figure 4), with wide intervals, read for sign. An agent trusting its receipts predicts this sign, one re-checking the world none.
Figure 3. A success receipt’s effect flips sign with the world on AppWorld. Each point is the success receipt minus the failure receipt on whole-task success, with the database step’s effect independently absent or present; the arrow is their interaction, read at 97.5%. 95% task-family bootstrap intervals on the two effects; 1,040 continuations over 65 task families, pooled over gpt-5.5 and claude-sonnet-5. Per receiver: Figure 4; cell means: Figure S1.
Figure 4. Figure 3’s interaction for each of eight receivers, grouped by provider. Left: unadjusted 95% (thick) and 97.5% (thin) task-family bootstrap intervals; † marks the six of eight excluding zero at 97.5%; seven exclude it at 95%. Black: the reported frame, 65 task families for §5.4’s two receivers and the 63 all six added receivers completed for the rest. Grey: an added receiver’s own 65-family frame (none for grok-4.6). Right: each receiver’s two record effects on whole-task success, effect absent (squares) and present (circles), at 95%; orange a loss, blue a gain; both resolve with opposite signs in five receivers. Hollow: the 95% interval includes zero.
Re-issuing the focal step’s request, a prespecified diagnostic, tracks the world: under a success receipt the pooled re-issue rate falls from 52.69% to 10.38% once the effect is present, by 42.31 pp [32.69, 51.92]. These are descriptive counts, not a mediation estimate. The receipt moves that rate for claude-sonnet-5; for gpt-5.5 its effect is unresolved (Appendix C.§ 3). In the 40 of 65 families where the focal step inserted a database row, under the false failure receipt 51.25% of claude-sonnet-5’s continuations write the row again, against 3.75% under the true success receipt. The failure receipt’s claim that no data changed (§ 4.3) is an open rival account.
The interaction is 23.85 pp smaller for claude-sonnet-5 than for gpt-5.5 [−41.54, −6.92]. Their base success rates differ (§ 4.1), so we claim only that both interactions are positive.
How far it reaches
Figure 4 reports the interaction for eight receivers on § 5.4’s frozen packets and estimator, each on the frame its caption gives. At 97.5%, the level the pooled interaction is read at, six of eight resolve positive; at 95%, seven of eight, gpt-5 joining them and gpt-4o-mini alone unresolved. Among the added receivers that resolve, the interaction runs from +16.67 pp [0.79, 32.54] for gpt-5 to +61.90 pp [46.03, 77.78] for gemini-3.1-pro-preview and +61.90 pp [42.86, 80.95] for gpt-4o. We added gpt-4o after gpt-4o-mini failed to resolve, to test a generation-only explanation: both predate the models trained to reason at length before acting, and gpt-4o resolves. The full flip, both record effects resolving with opposite signs, holds pooled; on each receiver’s own frame it holds for gpt-5.5, gemini-3.1-pro-preview, grok-4.6, gpt-5-mini and gpt-4o (Appendix § 11.§ 2). Nothing is pooled over all eight or ordered by effect; grok-4.6’s own frame is 63 families, the spending ceiling having stopped two of its blocks.
On the added receivers’ 63-family frame gpt-4o-mini is unresolved, at +2.38 pp [−6.35, 11.11]. Rescoring all 520 of its stored final states (65 families) reproduces every label, ruling out a scoring error. It re-issues the focal request in over half its continuations in every cell; the receipt shifts that rate, never below half (Appendix § 11.§ 3).
Further collections
Three separate collections, each with its own continuations, bear on the result (Appendix § 13). {ellipsis}
Channel decomposition. On AgentCE-Bench, Its prewritten primary, a bare error flag that names the wrong action and value but gives no replacement, is unresolved. A note carrying only the correct value gains +11.70 pp [9.20, 14.29] over the unedited record, against +16.46 pp [13.79, 19.25] for the full note in this collection (+15.65 pp for the same contrast in § 5.1), so the value alone keeps most of the full note’s gain. Record × world factorial, on the same 319 states. It asks § 5.4’s question and gets a weaker answer: an interaction of +3.90 pp [0.56, 7.18], excluding zero but not its prewritten 3 pp margin and resolving for claude-sonnet-5 alone. Appends on AppWorld. A note that only acknowledges the failure is unresolved, and a matched neutral note is not inert for claude-sonnet-5, +6.67 pp [2.42, 12.10].
Overall, a record edit’s effect depends on the world it describes and on how it describes it (§ 5.1, § 5.4).
Limitations
Eight limits follow, gravest first, detailed in Appendix § 15.§ 2.
Truth and channel are not crossed (§ 4.2). Models predating those trained to reason at length before acting catch an error presented as the user’s more often than as their own [4, ?], a role effect that alone would favour the note; the note bundles the answer (§ 5.1). A false note and a truthful in-place edit would separate truth from channel.
The AppWorld receipts differ in content as well as truth (§ 4.3), which may explain part of the interaction and § 5.4’s duplicate writes; content-matched receipts would remove it.
The budget binds unevenly on AgentCE-Bench (§ 4.2). emerges before it binds: at 6 steps, with over 92% of each arm running, is +26.15 pp [23.35, 29.00] (Appendix § 15.§ 2), narrowing to +21.63 pp as arms finish. In the targeted control the budget confounds displaced effort with specificity (§ 5.3); a non-binding one would not.
Few receivers carry the primary result. Two do, at different operating points (§ 4.1). Most continue another model’s trajectory (§ 4.2–§ 4.3), a factor one study found inert [12]. “Six of eight” treats receivers as fixed, not sampled, with unadjusted intervals. gpt-4o is a follow-up, added after gpt-4o-mini failed to resolve (§ 5.5); the seven named before it give 5 of 7 at 97.5% and 6 of 7 at 95%. All are closed API models whose version can change. A third four-arm receiver and recalibrated AppWorld receivers would widen this.
The edits and states are ours. One failure-receipt string, a note wording varied once, which moved the gain (§ 5.6), and states we selected (§ 3). We tested one flipped receipt and one appended note, not a compaction or handoff policy; nothing here bounds an adversary. Running such a policy at the same states is the test.
The gain is local. Most of T−N sits at the named slot, and on the 47 error-outside states the receivers disagree in sign (Table 1); more such states would settle it. AgentCE-Bench’s generator is unaudited for score-inflating regularities [31], which an audit or a second benchmark would address.
Two preregistered analyses never ran. The sealed AppWorld script and the targeted control’s estimator refuse (§ 4.4), so their figures come from reimplementations, each checked against a second. The AppWorld script needs its closure file repaired.
The targeted control is incomplete. It holds 4,068 of 4,320 planned units, its specificity is unresolved against the prewritten 3 pp margin and rests on one focal slot, and it has no truthful arm. It cannot be completed under its sealed plan: a provider refused 45 of its 252 missing units, which may not be retried. The remedy is a fresh, completable one with a truthful arm. A policy should be tested against the world; exact restoration can score it on the task, at a chosen frozen state, before deployment.
Conclusion
An agent’s record and the world it describes are coupled: the same content helps or hurts depending on how the record states it and what the world holds. We showed this in two designs where a receiver continues from a restored frozen state. At a fixed world, the same correct value raised recovery as a truthful note beside the preserved error, from 51% to 66% against a matched neutral note, and lowered it as the agent’s own completed action, from 50% to 44% against the unedited record; the gap, 22 points pooled, is positive in each receiver (§ 5.1). The presence of the answer, on its own, does not predict an edit’s effect (§ 5.2). At a fixed record, a success receipt lowered task success by 17 points while the step’s effect was absent and raised it by 10 once present, an interaction of 27 points pooled over two receivers (§ 5.4). Per receiver, the interaction is positive in six of eight at 97.5% and seven at 95% (§ 5.5).
One compatible explanation is the account the record gives of the world: a value set, or a slot still outstanding. That account may work through its truth or only through implied work. The targeted control rules out the strong form of the second reading, that the receiver was handed an answer, but not the thin form, that the record marks where work remains (§ 5.3). The next runs are § 6’s remedies, chiefly a false note and a truthful in-place edit, and an append matched on apparent author. On AgentCE-Bench, a record made to look clean while the world still held the error left agents worse off than no edit at all, so a compaction, repair or handoff
Reproducibility. AgentCE-Bench runs at pinned source commit 22ae16b7, AppWorld at package version 0.1.3.post1. AgentCE-Bench’s packets are reused unmodified from the earlier study of §4.2. Zero-model-call scripts recoupte every readout from the frozen contracts (§4.4 says which choices were prespecified). Code, contracts and per-unit readouts are released with this preprint as arXiv ancillary files (Appendix A.4); the exact tests are in Appendix J.
Use of AI assistance. The experiments, analyses and this manuscript were developed with the assistance of AI tools: LLM-based coding agents (Anthropic Claude through Claude Code, and OpenAI GPT-6) assisted with experimental design, implementation, analysis and drafting, and Google Gemini in two review calls. The authors set the research direction and scope, decided what the paper reports, and take full responsibility for its content. Appendix K lists the exact model identifiers recorded in the retained review prompts; the authoring sessions’ prompts were not retained, and we do not reconstruct them.
Conventions and reproducibility
Effects are in percentage points (pp). AppWorld endpoints are whole-task success; AgentCE-Bench endpoints are the graded look-ahead score unless a table says otherwise. Every interval carries a resolution: positive, negative, unresolved, or equivalent to zero. “Unresolved” is a verdict, not evidence of absence, and a bound is never rounded to its threshold; a displayed bound of +0.00 is a two-decimal display, and resolutions come from the bounds at full precision. Intervals are 95% unless marked 97.5%, the level of each two-comparison primary family. On AppWorld, pre is the database before the focal step (its effect absent) and post the database after it (effect present); a world effect is post − pre. Every estimate below is recomputed by the bundle’s scripts (§A.4); the contrasts are in the paper’s number table, NUMBERS.tsv, and the process-measure rates are in their own readout.
Benchmarks and versions
AgentCE-Bench runs from our fork at source commit 22ae16b7, stamped into every snapshot and bank record; the fork is a third-party checkout we may not redistribute. AppWorld runs at package version 0.1.3.post1, pinned with its API documentation, datasets and task files; its dependency snapshot is in the bundle.
Receivers
Decoding and scaffold. No sampling temperature or top-p is set for any receiver. The shared harness transport caps output at 8,192 completion tokens and turns tool calling on with tool_choice auto and no parallel tool calls. It adds to each benchmark’s own system prompt one sentence asking for a one-to-two-sentence belief statement before each action. claude-sonnet-5 runs through the native Messages API with extended thinking disabled. Routes differ by collection. On AgentCE-Bench (the four-arm design) both receivers ran through an internal API gateway. On AppWorld (the record world crossing and the appends collection), gpt-5.5 and gpt-5.6-sol ran through a direct enterprise route at reasoning effort “medium”, where the 8,192-token cap is requested but not enforced. In the channel-decomposition collection, gpt-5.5 moved from the gateway to that route during its first tranche.
| Design | Benchmark | Receivers |
| Four-arm design | AgentCE-Bench | gpt-5.5, claude-sonnet-5 |
| Targeted control | AgentCE-Bench | gpt-5.5, claude-sonnet-5 |
| Channel decomposition | AgentCE-Bench | gpt-5.5, claude-sonnet-5 |
| Record world factorial | AgentCE-Bench | gpt-5.5, claude-sonnet-5, gpt-5.6-sol |
| Record world crossing | AppWorld | gpt-5.5, claude-sonnet-5 |
| Receiver expansion | AppWorld | adds gemini-3.1-pro-preview, grok-4.6, gpt-5, gpt-5-mini, gpt-4o-mini, gpt-4o: eight receivers in four families |
| Appends | AppWorld | gpt-5.5, claude-sonnet-5, gpt-5.6-sol |
Table 3.
Unit counts
| Design | Delivered / planned | Blocks | Frame |
| Four-arm design | 638 complete blocks over 319 states, four continuations each | — | complete |
| Targeted control | 4,068 of 4,320 units | 452 of 480 | 189 never collected; 63 permanently held (§E.2) |
| Record world crossing | 1,040 of 1,040 units | 260 of 260 | complete; 65 families 2 receivers 2 repetitions 4 arms |
| Expansion: gemini-3.1-pro-preview | 520 of 520 | 130 of 130 | own frame 65 families |
| Expansion: grok-4.6 | 504 analysed / 512 delivered / 520 planned | 128 of 130 | 63 of 65 families; own frame = shared frame |
| Expansion: gpt-5, gpt-5-mini | 1,040 of 1,040 (two receivers) | 260 of 260 | 65 families each |
| Expansion: gpt-4o-mini | 520 of 520 | 130 of 130 | 65 families |
| Expansion: gpt-4o | 520 of 520 | 130 of 130 | 65 families (§D.4) |
| Appends | 60 instances | — | 54 clusters |
| Record world factorial | 319 of 319 states, hash-matched to the four-arm design | — | complete |
Table 4.
grok-4.6’s shortfall is ours, not the receiver’s: our own per-unit spending ceiling refused two of its 130 blocks. On the four-arm design, pilot states are in the frame while no pilot continuation is pooled into any estimate.
Released bundle
The code and data bundle is distributed with this preprint as arXiv ancillary files, listed on the abstract page and stored under anc/. Every readout in the paper is recomputed from frozen records by scripts that make zero model calls; no number is transcribed from a model’s output. The bundle holds:
sources/: the analysis contracts and design records of each collection; the recomputing estimators and their independent reimplementations (for example reproduce_e36_confirmation.py, estimate_e39.py, recompute_e35.py); the endpoint re-derivation and re-execution scripts; collection plans, sealed plans and frame-verification records; and the number table manuscript-v2/evidence/release/NUMBERS.tsv;
readouts/: per-unit readouts of the record × world crossing and the receiver expansion, and the endpoint re-derivation records;
packets/: the arm packet corpus for both benchmarks (§J);
positioning/: every judged passage of the positioning survey (§I);
MANIFEST.tsv: each member with its origin and both hashes. Absolute paths in the records are rewritten to bundle-relative tokens; the recorded file_sha256 values bind each packet to its plan entry across the rewrite.
The four-arm design on AgentCE-Bench
Construction
A unit is one restored AgentCE-Bench continuation from a frozen mid-task state; the cluster is the task. Four arms over the same 319 states, two receivers, 638 complete blocks:
| Arm | Packet name | What it holds |
| O | own__self | the agent’s own history, unedited; the world carries the error’s consequence |
| F | history_healed__world_consequence__matched | the history rewritten so the error never appears, against the unchanged wrong world; the world still carries the consequence |
| T | e29-truthful__self | the history preserved byte-for-byte, plus one trailing truthful retrospective correction |
| N | e29-neutral__self | the history preserved byte-for-byte, plus one trailing content-neutral note, length-matched to T |
Table 1 (PDF p. 12).
The primary contrast is a difference of contrasts, : the gain from a truthful annotation over a matched neutral note, minus the gain from silently healing the record. A block’s four arms co-started, so no arm ran under conditions its blockmates did not share.
Packet matching (read from the arm packets of one sampled state). Arms are matched on token count within a pair: O and F both carry 391 messages and 114,321 cl100k tokens; T and N both carry 392 messages and 114,449 cl100k tokens, with N’s packet recording matched_to: e29-truthful__self and matched_to_cl100k: 114449. O and F are padded to equality with an explicitly state-neutral filler; N holds the turn with a declared placebo note (both quoted in §J.2). Each packet records its content level, wrapper, dose, error classes, token counts, and an intervention block naming the source arm and the oracle used; the healing arm’s block records the substituted identifier, that the history was rewritten, and that the world was not repaired.
Primary contrasts
| Contrast | Scope | Estimate | 95% interval | Resolution |
| pooled | +15.20 pp | [+12.46, +18.00] | positive | |
| gpt-5.5 | +15.94 pp | [+12.75, +19.17] | positive | |
| claude-sonnet-5 | +14.47 pp | [+10.82, +18.18] | positive | |
| pooled | −6.43 pp | [−8.88, −3.97] | negative | |
| gpt-5.5 | −8.10 pp | [−11.44, −4.65] | negative | |
| claude-sonnet-5 | −4.75 pp | [−8.05, −1.52] | negative | |
| pooled | +21.63 pp | [+18.21, +25.08] | positive | |
| Contrast | Scope | Estimate | 95% interval | Resolution |
| gpt-5.5 | +24.03 pp | [+19.59, +28.47] | positive | |
| claude-sonnet-5 | +19.23 pp | [+14.63, +23.88] | positive | |
| raw T − F | pooled | +22.07 pp | [+19.25, +24.92] | positive |
| gpt-5.5 | +23.77 pp | [+20.11, +27.32] | positive | |
| claude-sonnet-5 | +20.38 pp | [+16.82, +23.93] | positive |
Table 2.
319 tasks, 319 states, 638 pairs pooled. Intervals are task-cluster bootstrap percentile, 10,000 resamples, unadjusted 95% per the design document. D and the raw T − F difference are different quantities (+21.63 and +22.07); the headline figure is D.
Secondary components
| Contrast | Estimate | 95% interval | Resolution |
| N − O | +0.44 pp | [−1.36, +2.32] | unresolved |
| T − O | +15.65 pp | [+13.06, +18.31] | positive |
Table 5.
Both are descriptive unadjusted 95% intervals, outside the primary family. The split of T − N between the slot the note names and the other look-ahead slots is the main text’s Table 1: +11.57 pp [9.85, 13.43] and +3.63 pp [1.67, 5.64], summing to the +15.20 pp total by construction. That split is on the look-ahead endpoint. On the global-progress endpoint, which scores the share of all outstanding slots rather than of the look-ahead targets, T − N is +14.34 pp [11.77, 16.86], of which +12.24 pp [10.57, 13.93] falls at the named slot and +2.09 pp [0.24, 3.97] at the other outstanding slots. The two endpoints have different denominators, so neither split may be quoted against the other’s total; every figure in the main text is on the look-ahead endpoint.
A matched neutral note may not be assumed to be a no-op: N − O is unresolved here, but on AppWorld the same kind of note is not inert for one receiver (§F.1).
The record world crossing on AppWorld
Frame and cells
A unit is one restored AppWorld continuation from a frozen mid-episode arm packet. 65 independent task families, two receivers, two repetitions, four arms: 1,040 units in 260 blocks, complete. The focal step is the first eligible successful single-request non-terminal database mutation in a source episode. The frame was verified before any continuation was launched: 260 packet restores checked, exact semantic restore on each, every source goal unsolved in both worlds, zero model calls during verification.
The census took all 70 test families untouched by our earlier work, 10 test_normal and 60 test_challenge, one hash-selected instance each, and validated them in frozen hash order before any was frozen, under a criterion fixed ahead of any outcome: at least 50 validated families. The 65 that validated became the frame. The five that did not, one test_normal and four test_challenge, failed before any continuation ran, and no artifact characterises why. The design does not measure spontaneous recovery in an uninterrupted state, nor any rollback made without an oracle.
| Cell | Record the agent sees | World it acts in |
| S_pre | receipt says the mutation succeeded | effect absent (database before the step) |
| S_post | receipt says the mutation succeeded | effect present (database after the step) |
| F_pre | receipt says the mutation failed | effect absent (database before the step) |
| F_post | receipt says the mutation failed | effect present (database after the step) |
Table 6.
The four packets of a unit differ only in the focal receipt and in which world checkpoint is restored (§J.1 gives the packet structure and quotes both receipts).
Cell means and contrasts
| Scope | S_pre | S_post | F_pre | F_post |
| pooled | 28.46 | 48.85 | 45.38 | 38.46 |
| gpt-5.5 | 40.77 | 69.23 | 66.92 | 56.15 |
| claude-sonnet-5 | 16.15 | 28.46 | 23.85 | 20.77 |
Table 7.
Cell means in %. The pooled interaction is consistent with them: . The interaction is an identity, the difference of the two world effects, not a third fact.
| Contrast | Scope | Estimate | Interval | Level | Resolution |
| interaction | pooled | +27.31 pp | [+14.23, +41.54] | 97.5% | positive |
| gpt-5.5 | +39.23 pp | [+23.08, +56.15] | 95% | positive | |
| claude-sonnet-5 | +15.38 pp | [+3.85, +27.69] | 95% | positive | |
| record effect, world pre | pooled | −16.92 pp | [−24.62, −9.62] | 95% | negative |
| gpt-5.5 | −26.15 pp | [−38.46, −13.85] | 95% | negative | |
| claude-sonnet-5 | −7.69 pp | [−14.62, −1.54] | 95% | negative | |
| record effect, world post | pooled | +10.38 pp | [+3.46, +17.69] | 95% | positive |
| gpt-5.5 | +13.08 pp | [+3.85, +23.08] | 95% | positive | |
| claude-sonnet-5 | +7.69 pp | [−0.77, +16.15] | 95% | unresolved | |
| world effect under a success record | pooled | +20.38 pp | [+12.69, +28.46] | 95% | positive |
| gpt-5.5 | +28.46 pp | [+16.15, +40.77] | 95% | positive | |
| claude-sonnet-5 | +12.31 pp | [+4.62, +20.00] | 95% | positive | |
| world effect under a failure record | pooled | −6.92 pp | [−13.46, −0.77] | 95% | negative |
| gpt-5.5 | −10.77 pp | [−20.00, −2.31] | 95% | negative | |
| claude-sonnet-5 | −3.08 pp | [−10.00, +3.85] | 95% | unresolved | |
| receiver difference (sonnet − gpt) | descriptive | −23.85 pp | [−41.54, −6.92] | 95% | negative |
Table 8 (PDF p. 14).
The negative world effect under a failure record means success falls once the effect is present while the record still reports failure. Figure S1 carries the per-receiver panels of this crossing. The two receivers differ by −23.85 pp [−41.54, −6.92], but that gap is confounded with operating point: claude-sonnet-5’s four cell means run 16.2–28.5%, nondegenerate but below the 25–75% calibration band and compressed relative to gpt-5.5’s 40.8–69.2%. It is not evidence that one model is less susceptible. No between-receiver contrast was estimated beyond the one the contract prewrote, and the only claim made across the two is that the sign is the same.
Figure S1. Cell means of the record × world crossing on AppWorld, per receiver: whole-task success under the success and the failure receipt, with the focal step’s effect absent or present (1,040 continuations, 65 task families). The receivers sit at different operating points, so the panels are not a comparison of susceptibility (Appendix C.§ 2).
Process measures
The design prespecified a descriptive diagnostic before treatment collection, whether a continuation issues the exact normalised focal request again, and never reported it. It is computed here with the design’s own matcher, imported unchanged, which reproduces all 12 per-arm diagnostic means the pilot’s readout stored over its 96 units. Rates in %, 95% family-cluster bootstrap (10,000 draws), descriptive and unadjusted; the design receivers on the published 65-family frame, the added receivers on the shared 63-family frame.
| Re-issued the focal request | Frame | S_pre | S_post | F_pre | F_post |
| gpt-5.5 | 65 fam. | 50.8 [39.2, 62.3] | 16.2 [9.2, 23.8] | 46.2 [36.2, 56.2] | 16.2 [9.2, 23.8] |
| claude-sonnet-5 | 65 fam. | 54.6 [43.8, 64.6] | 4.6 [0.8, 9.2] | 90.8 [84.6, 96.2] | 53.1 [41.5, 64.6] |
| gemini-3.1-pro-preview | shared 63 | 30.2 [19.8, 40.5] | 9.5 [4.0, 15.9] | 61.1 [50.8, 70.6] | 27.0 [17.5, 37.3] |
| grok-4.6 | shared 63 | 65.9 [54.8, 77.0] | 6.3 [2.4, 10.3] | 86.5 [78.6, 93.7] | 53.2 [42.1, 64.3] |
| gpt-5 | shared 63 | 65.9 [55.6, 76.2] | 31.0 [19.8, 42.1] | 58.7 [47.6, 69.0] | 26.2 [16.7, 35.7] |
| gpt-5-mini | shared 63 | 72.2 [61.9, 81.7] | 43.7 [32.5, 54.8] | 75.4 [66.7, 83.3] | 42.1 [31.7, 52.4] |
| gpt-4o-mini | shared 63 | 68.3 [57.9, 77.8] | 55.6 [46.0, 65.1] | 81.0 [72.2, 88.9] | 61.1 [50.8, 71.4] |
Table 9 (PDF p. 14).
A second measure is defined for the 40 of 65 states whose focal step inserted a record: whether the continuation’s own appended database statements contain an exact copy of that record (same table and column values, ignoring only the server-generated id and record_hash), a redo where the effect is absent and a second copy where it is present (80 continuations per cell).
| Wrote an exact copy of the focal record | ||||
| gpt-5.5 | 53.8 [38.6, 68.8] | 11.2 [3.8, 20.2] | 70.0 [56.4, 82.4] | 18.8 [8.6, 30.4] |
| claude-sonnet-5 | 55.0 [40.0, 69.5] | 3.8 [0.0, 10.0] | 91.2 [82.6, 97.8] | 51.2 [35.9, 66.7] |
Contrasts, pp: the world effect on re-issuing under a success receipt is +34.62 [+23.08, +46.15] for gpt-5.5, +50.00 [+38.46, +60.77] for claude-sonnet-5, +42.31 [+32.69, +51.92] pooled, and under a failure receipt +30.00, +37.69 and +33.85; the record effect on re-issuing, failure minus success, is +36.15 [+25.38, +46.92] (effect absent) and +48.46 [+36.92, +60.00] (present) for claude-sonnet-5 but −4.62 [−19.23, +10.00] and +0.00 [−9.23, +9.23] for gpt-5.5; and the second-copy rate, minus , is +47.50 [+32.50, +62.82] for claude-sonnet-5 against +7.50 [−5.26, +20.65] for gpt-5.5.
Reading. The world moves re-issuing for every receiver, so none is a pure receipt-truster. The receipt moves it strongly for claude-sonnet-5, grok-4.6 and gemini-3.1-pro-preview, and not detectably for gpt-5.5. Where a receiver re-issues over a completed write, it writes a second copy of the record; that is the route by which a false failure receipt lowers success after the step took effect. gpt-4o-mini re-issues heavily in every cell (§D.3).
Limits. These count request attempts and database statements; neither is a successful mutation, and nothing here is a mediation estimate. The exact-copy measure says nothing about the 25 update or delete effects. A first version of it counted any insert into the focal table and was withdrawn before use, because tasks that insert several rows into that table made it count other work as copies.
The receiver expansion on AppWorld
The expansion adds receivers and model families to the same benchmark with the same frozen packets and the same contract. There is no pooled expansion figure: the pooled scope is defined for exactly two receivers. Rows on the 65-family frame and on the shared 63-family frame (the 63 families all six added receivers completed) are not comparable with each other, and no table here is a ranking: no receiver-by-interaction contrast was estimated beyond the one prewritten receiver difference of §5.4.
Interaction by receiver
| Receiver | Family | Frame | Interaction | 95% interval | Verdict |
| gpt-5.5 | OpenAI | 65 families | pp | positive | |
| claude-sonnet-5 | Anthropic | 65 families | pp | positive | |
| gemini-3.1-pro-preview | 63 shared | pp | positive | ||
| grok-4.6 | xAI | 63 shared | pp | positive | |
| gpt-5-mini | OpenAI | 63 shared | pp | positive | |
| gpt-5 | OpenAI | 63 shared | pp | positive | |
| gpt-4o-mini | OpenAI | 63 shared | pp | unresolved | |
| gpt-4o | OpenAI | 63 shared | pp | positive |
Table 10.
These are the frames of the main text’s Figure 4.
Own frames
Each added receiver on the families it completed itself, all 95%:
| Receiver | Units | Interaction | Record effect, pre | Record effect, post | World effect, success record | World effect, failure record |
| gemini-3.1-pro-preview | 520, 65 fam. | |||||
| grok-4.6 | 504, 63 fam. | |||||
| gpt-5-mini | 520, 65 fam. | unresolved | ||||
| gpt-5 | 520, 65 fam. | unresolved | unresolved | |||
| gpt-4o | 520, 65 fam. | |||||
| gpt-4o-mini | 520, 65 fam. | unresolved | unresolved | unresolved | unresolved | |
| gpt-5.5 (validation) | 520, 65 fam. | |||||
| claude-sonnet-5 (validation) | 520, 65 fam. | unresolved | unresolved |
Table 11.
The two validation rows are the design receivers re-estimated by the expansion’s own estimator. They reproduce the crossing’s figures exactly over 1,040 units and 65 families, which licenses reading the added receivers on the same scale. grok-4.6’s own frame and the shared frame coincide, so its row is identical on both.
The unresolved receiver
On the 63-family frame, gpt-4o-mini is unresolved at pp . It predates the models trained to reason at length before acting; so does gpt-4o, which resolves at pp on the same frame, so generation alone does not account for it, nor do we attribute it to capability. Unresolved does not mean nothing moved. With the effect present rather than absent, success changes by pp under a failure receipt and by pp , unresolved, under a success receipt. On its own 65-family frame its unmanipulated cell, S_post, scores below S_pre, 26.92% against 36.92%. Re-scoring all 520 of its stored final states reproduces every label (160 successes, 360 failures) and all four cell means, so scoring is not the cause. Restoration is unlikely to be: every packet was restore-verified, and the same packets serve the other seven receivers. It re-issues the focal request in over half its continuations in every cell (§C.3), which is consistent with its not using the receipt, though an unresolved interaction is not evidence of absence. A step-cap detector on this receiver reports 30 units stopped by status against 34 reaching the cap by step count, a small internal discrepancy.
The follow-up receiver, gpt-4o
gpt-4o was added after gpt-4o-mini failed to resolve. Its collection closed complete, 130 of 130 planned blocks and 520 units over 65 families, and it was estimated with the same frozen contract and estimator as every other receiver. Both record effects resolve with opposite signs. Its own-frame cell means are 11.54%, 57.69%, 53.08%, 39.23%. Its shared-frame interaction equals gemini-3.1-pro-preview's to two decimals by coincidence; the two receivers' cells and intervals differ.
It was collected across several invocations: 67 blocks were carried from earlier runs, cut short by a host memory guard and a DNS outage, neither a receiver behaviour. No block is stitched across attempt numbers, but in 15 blocks the four arms were delivered by different invocations, up to 71.6 h apart, because a resume keeps delivered arms. Dropping the 14 families involved leaves 51, and the interaction reads +62.75 pp [43.14, 82.35]: the result does not rest on those blocks. All 520 units were restored from their saved final states and re-scored with zero model calls, and all 520 agree with the recorded labels.
The targeted control on AgentCE-Bench
Overview
The targeted control (§ 5.3) asks whether a false success claim steers the receiver by where it points rather than by what it reveals. It is a nine-arm crossing on the same benchmark and receivers, not a replication. Specificity, the gain on slots no arm named and no world repaired when the false claim matches the repaired slot rather than not, is +4.628 pp [2.989, 6.317] at 97.5%, positive in each receiver: +5.800 pp [3.814, 7.865] for gpt-5.5 and +3.456 pp [1.312, 5.477] for claude-sonnet-5 (Figure S2). No arm hands the scored slots a correct value, verified in all 4,068 units. The design is in §E.2, the per-slot estimates in §E.3, the step attribution in §E.4, and the three weaknesses that limit the reading in §H.2.
Figure S2. The targeted control on AgentCE-Bench: specificity, the gain on slots no arm named and no world repaired when the false claim matches the repaired slot, pooled at 97.5% and split by focal slot at 95% (paired frame, 223 checkpoints, 4,014 units; Appendix E.3).
Design and delivery
A record factor (original / A-claimed-successful / B-claimed-successful) crosses a world factor (neither repaired / repair A / repair B), giving arms O_O O_A O_B F_A F_A F_B F_B A_F A_A A_B F_AA F_BB F_AB F_BA F_BB over 240 independent checkpoints × 2 receivers (gpt-5.5, claude-sonnet-5), planned at 4,320 continuations in 480 blocks. Checkpoints were selected 12 per domain and then a global remainder under a SHA-256 seed, with zero model calls during construction; achieved domain counts are course 38, meal 45, pc_build 42, shopping 40, travel 32, workforce 43.
| Quantity | Value |
| Planned | 4,320 units in 480 blocks, 240 checkpoints |
| Analysed | 4,068 units in 452 blocks |
| Never collected | 189 units in 21 blocks |
Table 1 (PDF p. 17).
| Quantity | Value |
| Permanently held | 63 units in 7 blocks |
The seven held blocks are five provider-policy refusals, one model error that was not a policy refusal, and one block terminated by a forced reboot. Nothing was zero-filled, imputed, retried, stitched, or described as completion. The collection can never be completed under its sealed plan, because 45 units in 5 blocks are arms the provider refused outright. Collecting the 189 would buy 10 further paired checkpoints, and the sealed estimator would still refuse.
The preregistered estimator therefore never ran. Every figure below comes from an independent reimplementation of the frozen contract, and two checks stand in for the missing verification: a second reimplementation, written without importing the first, agrees on 50 of 50 contrasts over 4,068 units; and a forward state replay rebuilt each delivered unit’s initial world from the sealed plan and arm packet and re-executed its recorded trajectory, with all 4,068 units passing every check and zero model calls. The replay establishes that each stored final state is the one its own trajectory produces; it does not re-generate the continuations.
Estimates
Two post-hoc frames are never mixed: paired_both_receivers (223 checkpoints, 4,014 units, both receivers) and all_available_claude-sonnet-5 (229 checkpoints, 2,061 units). The primary endpoint is the graded score.
| Contrast | Scope | Estimate | Interval | Level | Resolution |
| specificity (primary) | pooled | +4.628 pp | [+2.989, +6.317] | 97.5% | positive |
| gpt-5.5 | +5.800 pp | [+3.814, +7.865] | 95% | positive | |
| claude-sonnet-5 | +3.456 pp | [+1.312, +5.477] | 95% | positive | |
| corresponding attenuation | pooled | +3.282 pp | [+0.938, +5.684] | 95% | positive |
| gpt-5.5 | +3.039 pp | [+0.336, +5.675] | 95% | positive | |
| claude-sonnet-5 | +3.525 pp | [+0.028, +7.092] | 95% | positive | |
| noncorresponding attenuation | pooled | −1.346 pp | [−3.307, +0.651] | 95% | unresolved |
| gpt-5.5 | −2.761 pp | [−5.445, −0.104] | 95% | negative | |
| claude-sonnet-5 | +0.069 pp | [−3.124, +3.408] | 95% | unresolved | |
| unrepaired rewrite effect | pooled | −3.642 pp | [−5.555, −1.642] | 95% | negative |
| gpt-5.5 | −4.282 pp | [−6.749, −1.748] | 95% | negative | |
| claude-sonnet-5 | −3.002 pp | [−5.605, −0.395] | 95% | negative | |
| focal cell A | pooled | +0.624 pp | [−1.657, +2.947] | 95% | unresolved |
| gpt-5.5 | +1.752 pp | [−1.050, +4.596] | 95% | unresolved | |
| claude-sonnet-5 | −0.504 pp | [−4.200, +3.062] | 95% | unresolved | |
| focal cell B | pooled | +8.632 pp | [+5.660, +11.542] | 95% | positive |
| gpt-5.5 | +9.848 pp | [+6.075, +13.585] | 95% | positive | |
| claude-sonnet-5 | +7.416 pp | [+3.029, +11.708] | 95% | positive | |
| receiver difference (sonnet − gpt) | descriptive | −2.344 pp | [−5.231, +0.534] | 95% | unresolved |
Table 1 (PDF p. 18).
Paired frame. Against the 3 pp practical margin the primary contrast was written to clear, the verdict is unresolved, with a lower bound of 2.989 pp. Primary contrast averages two focal-cell halves whose intervals do not overlap, focal A (the shown usually earlier in the recorded solution, 227 of 240 states) unresolved and focal B positive, so it is not one homogeneous effect. In the untreated arm the receiver already attends far more to the later focal cell than to the earlier one (§E.4). A positional account of which cell carries the effect therefore fits these data as well as a specificity account does. No arm varies position, so we offer it as a conjecture.
The contract’s prewritten conjunction, pooled specificity 97.5% lower bound > 0, pooled unrepaired rewrite 95% upper bound < 0, and pooled corresponding attenuation 95% lower bound > 0, holds (+2.989, −1.642, +0.938); these are three bounds of one prespecified conjunction, not three findings. On the single-receiver frame specificity is +3.482 pp [+1.391, +5.491], and the focal split survives (focal A −0.714 pp [−4.367, +2.804], focal B +7.679 pp [+3.395, +11.831]). On the companion endpoint, which scores the slot a claim names and was chosen after the stop, specificity is +1.850 pp [+0.112, +3.643] at 97.5%, but the receivers disagree (gpt-5.5 +4.821 pp [+3.027, +6.614]; claude-sonnet-5 −1.121 pp [−3.251, +1.009], unresolved); it is a diagnostic, never the primary reading.
Step attribution
| Measure | Estimate | Interval | Level | Resolution |
| steps used | −0.297 steps | [−0.502, −0.108] | 97.5% | negative |
| hit horizon | −0.026 proportion | [−0.046, −0.008] | 97.5% | negative |
| touches the common cell | +1.667 steps | [+1.223, +2.127] | 97.5% | positive |
223 clusters, 4,014 units. These answer the objection that the gain is disengagement or extra persistence, and exclude it, because the effort contrasts run opposite to the endpoint gain. No displacement fraction exists or is computable: step allocation is post-treatment, so any “share of the effect” number would be invented. In the untreated arm the receiver spends 4.92 steps on the later focal cell against 1.02 on the earlier, a factor of about 4.8; this is the basis of the positional conjecture above, not a finding. The budget binds: 78.69% of continuations hit the 24-step horizon, median 24 steps, mean 21.31, with 12.51% at or below 12 steps, over 4,068 units.
Further collections
The three collections
Each collection has its own continuations, so a contrast that recurs is an independent estimate, not a re-reading of the four-arm design. The channel decomposition (§F.2) and the record world factorial (§F.3) reuse the four-arm design’s 319 AgentCE-Bench states; the appends collection, below, has no table.
Appends, on AppWorld. A separate collection of 60 instances adds a third receiver, gpt-5.6-sol, predeclared as a descriptive transfer check. It also adds a record that only acknowledges the failure; acknowledging it is unresolved, −2.50 pp [−5.08, 0.00]. A matched neutral note is not inert for claude-sonnet-5, +6.67 pp [2.42, 12.10], one of the three receivers. The instances were drawn from existing calibration episode stores, one checkpoint per task, so a receiver is not always continuing its own error. Every unresolved result here is underpowered rather than informative about absence.
Channel decomposition
319 tasks, two receivers, descriptive unadjusted 95% intervals except where noted. The flag arm names the wrong action and the current wrong value but gives no replacement (“That assignment is incorrect. This note does not provide a replacement id.”); the answer arm carries only the correct value. The full note’s +16.46 pp here is an independent estimate of §5.1’s +15.65.
| Contrast | Scope | Estimate | Interval | Resolution |
| flag − neutral (primary, Bonferroni two-comparison) | pooled | +1.62 pp | [−1.04, +4.36] at 97.5% | unresolved |
| gpt-5.5 | +3.50 pp | [+0.47, +6.53] | positive | |
| claude-sonnet-5 | −0.26 pp | [−3.50, +2.82] | unresolved | |
| full − answer | pooled | +4.75 pp | [+2.72, +6.79] | positive |
| gpt-5.5 | −0.21 pp | [−2.40, +1.99] | unresolved | |
| claude-sonnet-5 | +9.72 pp | [+6.58, +12.96] | positive | |
| flag − original | pooled | +0.50 pp | [−1.80, +2.77] | unresolved |
| full − original | pooled | +16.46 pp | [+13.79, +19.25] | positive |
| answer − original | pooled | +11.70 pp | [+9.20, +14.29] | positive |
Knowing an error occurred, without the corrected value, does not detectably help. The per-receiver cells carry no multiplicity adjustment, and no between-receiver interaction was estimated.
Record world factorial
319 states, hash-matched to the four-arm design’s 319, with no missing data. Three receivers here, not the four-arm design’s two; “pooled” averages claude-sonnet-5 and gpt-5.5 with equal weight, and gpt-5.6-sol is reported separately, never inside that scope.
| Contrast | pooled | gpt-5.5 | claude-sonnet-5 | gpt-5.6-sol |
| rewrite − original, world unrepaired | −5.54 [−8.08, −3.15] | −6.02 [−9.51, −2.55] | −5.06 [−8.71, −1.48] | −11.58 [−15.55, −7.65] |
| rewrite − original, world repaired | −1.64 [−4.25, +0.97] unres. | −5.47 [−9.39, −1.76] | +2.20 [−1.44, +5.87] unres. | −7.46 [−11.00, −3.84] |
| Contrast | pooled | gpt-5.5 | claude-sonnet-5 | gpt-5.6-sol |
| world repair under original history | +3.08 [+0.77, +5.39] | +2.41 [−0.61, +5.47] unres. | +3.74 [+0.22, +7.32] | +2.45 [−0.90, +5.84] unres. |
| world repair under rewritten history | +6.98 [+4.31, +9.70] | +2.96 [−0.70, +6.65] unres. | +11.00 [+6.87, +15.13] | +6.56 [+2.64, +10.58] |
The design’s prewritten primary quantity is the difference between the last two rows, +3.90 pp [0.56, 7.18]: its interval excludes zero but not its own 3 pp margin, and it resolves for claude-sonnet-5 alone. The second row’s pooled unresolved is an average over receivers running in opposite directions, not a null. A provider-token residual diagnostic shows token count is differenced out of the interaction rather than driving it.
Restoration and endpoint checks
Restoration. The certification of the restoration machinery is in § 4.1. The certification states are disjoint from the experimental states except for one AppWorld task family (§ 4.3).
Endpoints. The endpoint labels of the record world crossing and of every expansion receiver were independently re-derived. Each unit’s saved final state was restored into a fresh process and rescored by the environment’s own evaluator, with zero model calls.
| Collection | Units re-derived | Agreements | Label split (success / failure) |
| Record world crossing | 1,040 | 1,040 | 419 / 621 |
| Expansion: gemini-3.1-pro-preview | 520 | 520 | 326 / 194 |
| Expansion: grok-4.6 | 512 | 512 | 413 / 99 |
| Expansion: gpt-5 | 520 | 520 | 228 / 292 |
| Expansion: gpt-5-mini | 520 | 520 | 225 / 295 |
| Expansion: gpt-4o | 520 | 520 | 210 / 310 |
| Expansion: gpt-4o-mini | 520 | 520 | 160 / 360 |
Table 12.
Rescoring checks the scorer, not the run, and it is not the closure audit the sealed AppWorld script requires (§H.1). A separate re-execution therefore rebuilt every record world crossing final state from its restored start state by running the receiver’s own recorded code, with zero model calls. All 1,040 labels agree, and 1,037 of 1,040 final database states match exactly; the other three differ only in a delivery time AppWorld draws at random. The targeted control’s 4,068 units were verified by the forward state replay of §E.2.
Deviations and limits
Deviations from the preregistered analyses
AppWorld. The preregistered script runs only after a sealed final-state replay, which in turn requires the collector’s closure file. That file cannot be written: the collection was run by a separate resumable collection script whose runs the collector’s inventory does not see, so the collector would re-buy all 260 blocks, and each collection process refuses to overwrite a delivered unit. The script has therefore never run on the collected data. Every AppWorld figure comes from a reimplementation of the same contract (same unit, estimand, bootstrap and seed) with the replay gate removed, and a second implementation written from the contract without importing the first agrees on 16 of 16 contrasts and 12 of 12 cell means.
Targeted control. Its preregistered estimator refuses on a partial frame and never ran (§E.2). Its figures come from two reimplementations that agree on 50 of 50 contrasts. On the preregistered frame restricted to complete blocks, with its planned weights, specificity is +4.627 pp [2.990, 6.294] at 97.5% against the post-hoc +4.628 pp, and every verdict is unchanged.
In neither case was a completion, verification or provenance artifact hand-authored, an assertion relaxed, or an output root repointed to make the refusal go away.
The eight limits in full
The main text states each limit in one or two sentences; the full reading is here. Our evidence is two benchmarks: 319 frozen states on AgentCE-Bench continued by two receivers under a 24-step budget, and 65 AppWorld task families continued by eight receivers under a 30-step budget, with every edit written by us at a state we chose and every receiver a closed API model.
Truth and channel are not crossed. On AgentCE-Bench the falsified record and the truthful note differ in channel, apparent author, position, whether the error stays in view and whether work is said to remain, as well as in truth (§ 4.2), so compares two natural edits. Models predating those trained to reason at length before acting catch the same error far more often when it is presented as the user’s than as their own [4, ?], a role effect that alone would favour T’s note over F’s edit. The truthful note also bundles the content, correctness and placement of the fix (§ 2), and it needs a verified correct value that a deployed history manager may lack (§ 5.1). A false note and a truthful in-place edit on the same 319 states are the two cells that would separate them, and the first we would add.
The AppWorld receipts differ in content as well as truth. Only the failure receipt states that no data changed, while the success receipt is the API’s own confirmation (§ 4.3), so part of the interaction may be what each receipt says rather than whether it is true; the 51.25% duplicate writes under the false failure receipt (§ 5.4) are the behaviour that content would predict. Content-matched receipts on the same 65 packets would remove this.
The budget binds, and unevenly. On AgentCE-Bench 71.4% of four-arm continuations use all 24 steps, from 60.3% under T to 80.1% under F (§ 4.2), so is read at a horizon the arms exhaust at different rates. Reading the endpoint at shorter budgets from the stored per-step states shows that emerges before the budget binds: at 6 steps, while over 92% of every arm is still running, is already +26.15 pp [23.35, 29.00], and it shrinks to +21.63 pp by step 24. In the targeted control the budget binds in 78.69% of continuations, so displaced effort and specificity leave the same signature (§ 5.3); a rerun under a budget that does not bind would tell them apart.
Few receivers carry the primary result, and the receivers are fixed, not sampled. Two receivers carry the four-arm result, calibrated to different operating points; on AppWorld only gpt-5.5 met the 25–75% calibration band (47.5% [34.3, 60.9] completed-only), while grok-4.6 sits at the ceiling in two cells (§ 4.1). Every receiver except gpt-5.5 on AppWorld continues another model’s trajectory, so receiver differences are confounded with operating point and with the cross-model handoff, a factor one study found inert [12]. “Six of eight” tallies unadjusted intervals at 97.5% (seven of eight at 95%) over receivers treated as fixed, not sampled. One focal step in each of 65 families leaves per-receiver intervals wide. gpt-4o is a follow-up, added after gpt-4o-mini failed to resolve (§ 5.5); the seven receivers named before it give 5 of 7 at 97.5% and 6 of 7 at 95%. The receivers are closed API models whose versions can change; ids and routes are in §A.2. A third receiver on the four-arm design, a receiver continuing its own frozen trajectory there, and recalibrated AppWorld receivers would widen this.
The edits and states are ours. We wrote one failure-receipt string and one note wording, at states we selected, so the design does not estimate how often records and worlds disagree in use (§ 3). The one time the note wording was varied, a note carrying only the correct value gained +11.70 pp against +16.46 pp for the full note in a separate collection (§ 5.6), so T − N depends on wording. We tested one flipped receipt and one appended note, not one corruption or handoff policy that rewrites a whole record; continuing such a policy at the same frozen states is the test our conclusion calls for. We, not an adversary, made every manipulation, for a single agent continuing alone, so nothing here bounds an adversary choosing where to falsify, or agents that check one another’s records.
The gain is local. Of T − N’s +15.20 pp on the look-ahead endpoint, +11.57 pp sits at the slot the note names (Table 1). On the 47 states whose error lies outside the look-ahead targets, pooled T − N and D are unresolved, and the receivers disagree in sign, claude-sonnet-5 −10.64 pp [-19.15, −2.13] and gpt-5.5 +7.80 pp [0.71, 14.89]. Whether a truthful note helps beyond the slot it names needs more states of that kind than 47. AgentCE-Bench is a generator we have not audited for the regularities by which a generated task can let a score overstate a capability [31]; an audit or a second benchmark would address this. On the record × world question was asked only in § 5.6’s factorial, whose interaction, +3.90 pp [0.56, 7.18], excludes zero but not its prewritten 3 pp margin.
Two preregistered analyses never ran. On AppWorld the sealed script refuses because its closure file cannot be written (§H.1), so every figure comes from a reimplementation of the same contract, which a second, independent implementation reproduces. Re-executing every continuation from its start state reproduces all 1,040 labels, but this is not the sealed replay. Every AppWorld cell is also a handoff into a fresh interpreter, so no uninterrupted baseline anchors the cell means (§ 4.3). Running the sealed script requires repairing the collector’s closure path. The targeted control’s estimator refuses on a partial frame.
The targeted control is incomplete. It holds 4,068 of 4,320 planned continuations: 189 were never collected and 63 are permanently held in seven blocks. Its specificity contrast is unresolved against the 3 pp margin (§ 5.3), it has no truthful arm, so it separates lies by where they land, not lies from truth, and its pooled figure rests on one of its two focal slots. It cannot be completed under its sealed plan, because a provider refused 45 of the 252 missing units outright and those may not be retried; the remedy is a fresh, completable control with a truthful arm at the same focal slots.
Literature positioning
Classical precedents. Record–world disagreement is classical. Execution monitoring re-senses the world [5, ?]; epistemic planning separates information state from world [2]; diagnosis conjectures unrecorded actions [24]; BDI agents act on effector reports [3] and ration reconsideration [19]. Trust models weight others’ reports [?]; belief revision accepts its input [?]; and sensing corrects an agent that believes a failed action succeeded [7], the case of our falsified receipt, arising there from misexecution and here from a record edit.
The positioning survey. We assembled 360 positioning passages, one per paper, from a targeted literature search over eight question groups (preserving, rewriting or annotating an error; oracle assistance and answer disclosure; restart, summary, compaction or continuation; rollback, repair and persistent consequences; factor identification; location specificity; record × operation-state handoff; counterfactual and measurement validity). We checked it against anchor works’ reference lists and a live spot-check. A language-model reader judged each passage against six written dimensions, so the counts below are positioning evidence over our own collection, not a measurement of the literature; the novelty claim rests on the nearest works the main text names. The search promoted 363 papers; three had no retrievable copy, so every count is over the 360 judged.
| Dimension | Count |
| a: the record and the world treated as two separable objects | 91 of 360 |
| b: what the agent is shown varied while the underlying world state is held fixed | 53 of 360 |
| a ∧ b: both | 23 of 360 |
| c: the matched append control (five attributes) | 0 of 360 |
| d: what a record reveals held apart from what it asserts | 3 of 360, one only partially [13, 14, 26] |
| a ∧ b ∧ (c ∨ d): the conjunction the positioning argument rests on | 0 of 360 |
| b ∧ d: reveal held apart from assert at a held-fixed executable state | 0 of 53 |
Table 13.
The zero for c is an upper bound on the gap. Dimension c requires all five attributes to be matched: role or channel, apparent author, position, turn count and tokenised length. Our own control matches four (not apparent author), so a paper controlling for appending exactly as we do would also score zero. Fifteen passages discuss such a control, and each says it lacks one.
The 23 papers that do both a and b, by citation key, each with its passage in the bundle’s positioning/ archive: aborted-not-forgotten-kv-rollback, acrfence, agentrewind, beyond-final-scores, chronomem, crab-checkpoint-restore, dart-semantic-recoverability, deltabox-stateful-agent-checkpointing, dependency-guided-rollback-repair, generator-assistant-stepwise-rollback, goal-drift, handoff-debt, inconsistent-tool-responses, inside-scaffold, osworld-2, recoverability-system-primitive, reliable-commit-gates, repair-or-resample, residual-completion-handoffs, resume-means-resume, rollback-induced-reflection, safe-to-resume-rollback, toward-reliable-context-compression-long. None carries c or d.
Prompts and arm packet texts
Arm packets are frozen artifacts, each carrying its own SHA-256 and referenced from its collection plan by hash. Packet schema p001-surface-packet-v1: schema, surface, name, source_state_id, messages, world_state, sha256, plus per-design fields for content level, wrapper, dose, error classes, token counts and the intervention block. The bundle carries both corpora complete, 638 AgentCE-Bench packets and 260 AppWorld arm packets, under packets/.
AppWorld: the receipt is the manipulation
Within a unit, the four packets’ message lists are identical except at one position, the receipt returned at the focal step, and the pre/post pair have byte-identical message lists, differing only in which checkpoint of the world is restored.
The failure receipt is a fixed template, identical across units (verbatim):
Execution failed. Traceback:
RuntimeError: The operation did not take effect; no application data was changed.
The success receipt is the environment’s own task-specific response, for example, for a draft-creating step:
{"message": "Draft created.", "draft_id": 518}
Every packet ends with the same fixed continuation preamble (verbatim):
A new Python interpreter is being used for this continuation. Variables from earlier cells are not
defined here. Continue the same task using the documented app APIs; obtain any values you need again.
AgentCE-Bench: the four arms
The truthful arm’s trailing note, in a sampled state (identifiers are the state’s own):
Retrospective correction (oracle-verified; this note was authored by the pipeline from detector ground
truth, not generated by the agent as self-reflection). At message index 264 the assistant issued
set_slot(row=2, col=4, id=C1795); the tool response at message index 265 accepted it. That assignment is
incorrect: the verified correct id for (row 2, col 4) is C4B01. At this checkpoint the current solution
record is id C1795 at (row 2, col 4), so its consequence remains in effect until the assignment is
corrected.
The matched neutral arm’s trailing note, same state (the padding is literal and is how length matching is achieved):
Neutral process note. This message intentionally holds no task guidance and no details about any earlier
step, its outcome, or the present state of the solution. It exists only to hold the turn and to match
length: a a a a a a a a a . . .
The baseline (O) and healing (F) arms are padded to equal length with a declared state-neutral filler:
State-neutral padding: this note adds no candidate, constraint, plan, conclusion, or instruction and must
not affect task execution.
The truthful packet’s intervention block records “error-preserving truthful correction; history preserved byte-for-byte” and names the oracle as detector ground truth. That is the same ground truth the healing arm uses, which makes the two arms comparable. Both arms place the same verified value before the agent, so the strong form of disclosure-as-used, that the truthful arm wins by supplying the answer, is excluded by construction.
The targeted control’s nine arms are constructed from the same bank under its sealed plan, and each delivered unit’s packet was rebuilt from that plan during the forward state replay (§E.2).
K Use of AI assistance
The main text’s statement on the use of AI assistance applies throughout. Models used as AI assistance, as recorded in the retained review records (66 prompt files and 65 input manifests; each prompt record carries the model id and the date of the call):
| Model id | Manifests naming it (of 65) |
| claude-opus-5-5 | 59 |
| claude-fable-5-1 | 58 |
| gpt-6-sol | 58 |
| claude-opus-4-8 | 58 |
| gemini-3.1-pro-preview | 2 |
Table 1 (PDF p. 23).
The models studied in the experiments are data rather than assistance and are listed with their routes and decoding settings in §A.2. The prompts and transcripts of the AI-assisted authoring sessions (harness, runners, analysis scripts and manuscript prose) were not kept, and none is reconstructed. One further model id appears in prose records of that work but in none of the 65 manifests, and is not asserted as verified. One review record of the 66 has no input manifest, so its inputs are not enumerated.
References
- [1]Laksh Advani. 2026. From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents. Workshop paper, Failure Modes in Agentic AI (FAGEN), ICML 2026. arXiv:2606.09863 [cs.LG]. https://arxiv.org/abs/2606.09863
- [2]Thomas Bolander and Mikkel Birkegaard Andersen. 2011. Epistemic planning for single- and multi-agent systems. Journal of Applied Non-Classical Logics 21, 1 (2011), 9–34. doi:10.3166/jancl.21.9-34DOI
- [3]Rafael H. Bordini and Jomi Fred Hübner. 2010. Semantics for the Jason Variant of AgentSpeak (Plan Failure and some Internal Actions). In ECAI 2010 – 19th European Conference on Artificial Intelligence (Frontiers in Artificial Intelligence and Applications, Vol. 215), Helder Coelho, Rudi Studer, and Michael Wooldridge (Eds.). IOS Press, 635–640. doi:10.3233/978-1-60750-606-5-635DOI
- [4]Kuan-Yen Chen, Fang-Yi Su, Shih-Yen Lin, Bao Li, and Jung-Hsien Chiang. 2026. The Self-Correction Illusion: Role Relabeling Gates Explicit Error Flagging in Large Language Models. arXiv:2606.05976 [cs.AI] https://arxiv.org/abs/2606.05976
- [5]Giuseppe De Giacomo, Ray Reiter, and Mikhail Soutchanski. 1998. Execution Monitoring of High-Level Robot Programs. In Proceedings of the Sixth International Conference on Principles of Knowledge Representation and Reasoning (KR’98), Anthony G. Cohn, Lenhart K. Schubert, and Stuart C. Shapiro (Eds.). Morgan Kaufmann, 453–465.
- [6]Edoardo Debenedetti, Jie Zhang, Mislav Balunovic, Luca Beurer-Kellner, Marc Fischer, and Florian Tramer. 2024. AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents. In Advances in Neural Information Processing Systems 37 (NeurIPS 2024), Datasets and Benchmarks Track. Curran Associates, Inc., 82895–82920. arXiv:2406.13352 [cs.CR] doi:10.52202/079017-2636DOI
- [7]James P. Delgrande and Hector J. Levesque. 2012. Belief Revision with Sensing and Fallible Actions. In Proceedings of the Thirteenth International Conference on Principles of Knowledge Representation and Reasoning (KR 2012), Gerhard Brewka, Thomas Eiter, and Sheila A. McIlraith (Eds.). AAAI Press, 148–157.
- [8]Runzhi Deng, Yiming Zhong, Fang Zhao, and Pan Zhou. 2026. Do Not Restart: Residual Completion for Stateful Agent Handoffs. arXiv:2609.13800 [cs.AI] https://arxiv.org/abs/2609.13800
- [9]Ashritha Gonuguntla. 2026. The Replay Gap: Static Evaluation of Model Switching in LLM Agents Across the Wrong World. Workshop paper, Efficient Reasoning Workshop, COLM 2026. arXiv:2608.08239 [cs.LG] https://arxiv.org/abs/2608.08239
- [10]Jim Gray. 1981. The Transaction Concept: Virtues and Limitations. In Proceedings of the International Conference on Very Large Data Bases (VLDB). 144–154. doi:10.5555/1286831.1286846
- [11]Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. 2023. Not What You’ve Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security (AISec ’23). ACM, New York, NY, USA, 79–90. arXiv:2302.12173 [cs.CR] doi:10.1145/3605746.3623985
- [12]William Guey and Pierrick Bougault. 2026. Self-Preference Is Weak or Absent in Verifiable Instruction-Following Revision: A Four-Model Test Under Genuine Authorship. arXiv:2606.20093 [cs.CL] https://arxiv.org/abs/2606.20093
- [13]Esmail Gumaan. 2026. Feedback That Backfires: Why Small Language Model Agents Repeat the Call They Just Watched Fail. arXiv:2608.23651 [cs.SE] https://arxiv.org/abs/2608.23651
- [14]Jundong Hu and Shekar Ramachandran. 2026. The Memory Trust Gap: Capability-Dependent Failures in Persistent-Memory Agents. arXiv:2609.01852 [cs.AI] https://arxiv.org/abs/2609.01852
- [15]Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2024. Large Language Models Cannot Self-Correct Reasoning Yet. arXiv:2310.01798 [cs.CL] https://arxiv.org/abs/2310.01798
- [16]Anik Jha. 2026. Does Fault Localization Beat a Fresh Attempt? A Placebo-Controlled Study of Test-Guided Code Repair. arXiv:2609.00854 [cs.SE] https://arxiv.org/abs/2609.00854
- [17]Minki Kang, Wei-Ning Chen, Dongge Han, Huseyin A. Inan, Lukas Wutschitz, Yanzhi Chen, Robert Sim, and Saravan Rajmohan. 2026. ACON: Optimizing Context Compression for Long-Horizon LLM Agents. In Proceedings of the 43rd International Conference on Machine Learning (ICML 2026) (PMLR, Vol. 306). arXiv:2510.00615 [cs.AI] https://arxiv.org/abs/2510.00615
- [18]Dipesh KC and Anjila Budathoki. 2026. Handoff Debt: The Rediscovery Cost When Coding Agents Take Over Interrupted Tasks. arXiv:2606.02875 [cs.AI] https://arxiv.org/abs/2606.02875
- [19]David N. Kim and Michael P. Georgeff. 1991. Commitment and Effectiveness of Situated Agents. In Proceedings of the 12th International Joint Conference on Artificial Intelligence (IJCAI 1991), John Mylopoulos and Raymond Reiter (Eds.). Morgan Kaufmann, 82–88.
- [20]Xingzuo Li, Kehai Chen, Yunfei Long, Xuefeng Bai, Yong Xu, and Min Zhang. 2025. Generator-Resistant Stepwise Rollback Framework for Large Language Model Agent. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 17683–17700. arXiv:2503.02519 doi:10.18653/v1/2025.emnlp-main.892
- [21]Xiaofeng Lin, Yunxi Wang, Tung Sum Thomas Kwok, Daniel Guo, Sahil Arun Nale, Charles Fleming, and Guang Cheng. 2026. REFLECT: Intervention-Supported Error Attribution for Silent Failures in LLM Agent Traces. arXiv:2606.09071 [cs.AI] https://arxiv.org/abs/2606.09071
- [22]Nelson F. Liu, Kyle Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023. Lost in the Middle: How Language Models Use Long Contexts. arXiv:2307.03172 [cs.CL] https://arxiv.org/abs/2307.03172
- [23]Xing Han Lù, Amirhossein Kazemnejad, Nicholas Meade, Arkil Patel, Dongchan Shin, Alejandro Zambrano, Karolina Stańczak, Peter Shaw, Christopher J. Pal, and Siva Reddy. 2025. AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories. arXiv:2504.08942 [cs.LG] https://arxiv.org/abs/2504.08942
- [24]Sheila A. McIlraith. 1998. Explanatory Diagnosis: Conjecturing actions to explain observations. In Proceedings of the Sixth International Conference on Principles of Knowledge Representation and Reasoning (KR’98), Anthony G. Cohn, Lenhart K. Schubert, and Stuart C. Shapiro (Eds.). Morgan Kaufmann, 167–179.
- [25]Bardia Mohammadi, Néarchos Potamitis, Lars Klein, Akhil Arora, and Laurent Bindschaedler. 2026. Atomix: Timely, Transactional Tool Use for Reliable Agentic Workflows. arXiv:2602.14849 [cs.LG] https://arxiv.org/abs/2602.14849
- [26]Arjun Panickssery, Samuel R. Bowman, and Shing Feng. 2024. LLM Evaluators Recognize and Favor Their Own Generations. In Advances in Neural Information Processing abs/2604.23283
- [47]Qiuzi Zhan, Zhixiang Liang, Zifan Ying, and Daniel Kang. 2024. InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents. In Findings of the Association for Computational Linguistics: ACL 2024. Association for Computational Linguistics, Bangkok, Thailand, 10471–10506. arXiv:2403.02691 [cs.CL] doi:10.18653/v1/2024.findings-acl.624DOI
- [48]Haiyue Zhang. 2026. Credit Without Ground Truth: Auditing Step-Level Credit Assignment in LLM Agents Against Executed Replay. arXiv:2608.19760 [cs.LG] https://arxiv.org/abs/2608.19760
- [49]Xiangxin Zhao, Han Li, Shuaiti Li, Tianyi Zhao, Earl T. Barr, Federica Sarro, and He Ye. 2026. Failure as a Process: An Anatomy of CLI Coding Agent Trajectories. arXiv:2607.09510 [cs.SE] https://arxiv.org/abs/2607.09510
- [50]Dongsheng Zhu, Xuchen Ma, Yucheng Shen, Xiang Li, Yukun Zhao, Shuaiqiang Wang, Lingyong Yan, and Dawei Yin. 2026. When Tools Fail: Benchmarking Dynamic Replanning and Anomaly Recovery in LLM Agents. arXiv:2606.05806 [cs.AI] https://arxiv.org/abs/2606.05806
- [51]Yu Zhuang, Kefei Chen, Yitong Duan, Shuxin Zheng, Jian Li, and Xu-Yao Zhang. 2026. AgentRewind: Recoverable Execution for Long-Horizon LLM Agents. arXiv:2608.14380 [cs.AI] https://arxiv.org/abs/2608.14380





