WorksComputer Science
When Should an Agent Refresh? Adaptive Index Reuse for Efficient Long-Horizon Reasoning
Abstract
Language agents read code, tool output, and dialogue in contexts that grow at every step. Dynamic sparse attention lets a small indexer choose the earlier tokens each query attends to, but the indexer still scores the whole prefix at every layer. Cross-layer reuse methods such as IndexCache rerun the indexer at only some layers, with one fixed pattern for every input. In an agent, the relevant evidence can change abruptly when a traceback or test result arrives, so fresh selections matter more at some steps than at others. We propose Adaptive Index Reuse (AIR), which keeps IndexCache's execution and learns, for each segment of an agent episode, how many layers refresh their selection. A small controller reads signals from completed computation and the latest tool event, chooses a pattern from a nested library, and learns from terminal task reward under an expected compute budget. On Qwen3-4B-Instruct-2507 with a 128K context, AIR improves success over the strongest static schedule at matched cost on SWE-bench Verified (27.0% to 32.4%), Terminal-Bench 2.0, and three -bench domains, and it lowers time per successful task on all five. Across 50 backbone, context, and benchmark settings, it matches or exceeds that schedule in 44.
Introduction
Tool-using language agents work over contexts that keep growing. A coding agent reads an issue, searches files, reads a traceback, edits code, and runs tests; a customer-service agent alternates between dialogue, policy lookup, and API calls. Every generated token attends over a key-value cache that holds this whole history, so long trajectories make each decision expensive.
Dynamic sparse attention lowers this cost by letting each query attend to a small, query-dependent set of positions (Yuan et al., [54]; DeepSeek-AI, 2025). In DeepSeek Sparse Attention (DSA), a lightweight indexer at every layer scores all earlier positions and passes the top to core attention (DeepSeek-AI, 2025). Core attention then reads positions, while the indexer still scores the whole prefix at every layer, and this selection work becomes the cost that grows with context. IndexCache observes that adjacent DSA layers select largely overlapping positions (Bai et al., [5]). It runs the indexer only at some Full layers and lets each Shared layer reuse the positions chosen by the nearest preceding Full layer; on a 30B DSA model its authors report removing 75% of indexer computation, with up to 1.82× prefill and 1.48× decode speedups at 200K tokens. A greedy search on calibration loss fixes the pattern once, and the same pattern serves every input and every point of a generation.
We call recomputing the selected positions at a layer a refresh. An agent’s need for refreshes changes during an episode. During routine generation the relevant evidence moves slowly and tolerates aggressive reuse; a new traceback, file, test result, or user request can change it abruptly and make stale selections costly. Static reuse decides where in depth a model refreshes. This paper asks when, along a trajectory, a larger refresh budget is worth its cost, and how to learn that decision from task outcomes.
We introduce Adaptive Index Reuse (AIR), which learns to allocate refresh computation from observable agent state (Figure 1). AIR keeps IndexCache’s Full/Shared execution and its greedy pattern search, and it stores one pattern per refresh budget from the search path as a nested runtime library. Before each segment of the agent’s work, a small controller reads signals that already exist, such as the type of the latest tool output and statistics of finished computation, and chooses a library entry for the next segment. Every eligible layer keeps storing index keys while it is Shared, so any entry can run at any segment, and the cost model charges this upkeep. The controller is trained from terminal task reward under an expected budget on indexing and controller compute, and validation data decide whether a trained controller or a static fallback is deployed.

Figure 1. AIR’s decision loop. Before segment computes, the controller maps state from finished work to a budget; the segment runs under that budget’s nested mask. Full layers select position IDs; Shared layers reuse them. Every layer keeps its own KV cache, and the next observation feeds decision .

Figure 2. Offline preparation and training. Each stage uses its own owner-disjoint fitting partition and is frozen before the next; validation selects a learned controller or a static fallback before any final run.
On SWE-bench Verified, Terminal-Bench 2.0, and three -bench domains, AIR raises success over the strongest static schedule at matched cost by 2.2 to 5.4 points and cuts time per successful task by 6.8 to 18.4% (Table 1). Controls show that the gain needs both state information and credit for delayed effects. The gain is uneven: it shrinks, ties, or reverses in 13 of 50 settings, all-refresh execution still scores higher on two benchmarks, and tools take enough of each episode that whole-agent speedups over all-refresh are 1.05 to 1.11. Our contributions are:
| Benchmark | Success (%) ↑ Static | Success (%) ↑ AIR | Success (%) ↑ | Min/success ↓ Static | Min/success ↓ AIR |
| SWE-bench Verified | 27.0 | 32.4 | +5.4 | 64.6 | 52.7 |
| Terminal-Bench 2.0 | 22.5 | 24.7 | +2.2 | 61.9 | 56.4 |
| retail | 53.3 | 57.2 | +3.9 | 14.1 | 13.1 |
| airline | 42.5 | 46.5 | +4.0 | 14.1 | 12.9 |
| telecom | 42.5 | 46.5 | +3.9 | 19.5 | 17.9 |
| Differences are calculated before rounding. |
Table 1. AIR raises success and lowers time per successful task on every benchmark. Qwen3-4B-Instruct-2507, 128K cap; Static is the strongest feasible static schedule () at matched expected cost.
a formulation of refresh allocation in agents as constrained control from causal observations, with the nested library as a hierarchical action
a cost model that charges index-key maintenance, selection, and controller overhead;
a training and deployment procedure that combines a replay-based warm start with standard Lagrangian policy gradient and selects the deployed policy on held-out validation data;
a controlled evaluation on five backbones, two context caps, and five agent benchmarks, with controls that separate layer placement, timing, state information, and delayed credit.
Related Work
Cheaper selection. Many methods make one sparse selection cheaper: token, block, or page selection ([19, 38, 54]), a lightweight learned indexer (DeepSeek-AI, 2025), routing each query to a few indexer heads ([58]), block-then-token search Xu et al. [48], or a compressed index-key representation shared across layers with layer-specific selection Wang et al. [42]. AIR leaves the cost of each indexer call unchanged and decides which layers make a call, so these methods are both competitors and candidates for composition.
Cross-layer reuse. TidalDecode, Kascade, IndexCache, and LongCat Sparse Attention reuse selections across layers with a structure fixed per model [51, 12, 5, 55]; some architectures also share key-value tensors across layers or are pretrained with sparse attention [8, 36, 37, 43, 22] (Gao et al. 2026). AIR inherits IndexCache’s Full/Shared execution, its nearest-Full reuse rule, and its greedy search. Its additions are the per-segment choice among nested patterns, the observations that drive it, index-key maintenance at every eligible layer, and training from task reward under a compute budget.
Online refresh decisions. HShare fixes a sharing configuration per input batch after prefill Wu et al. [46], Slow-Fast Inference refreshes a reused token set near sentence boundaries Xie et al. [47], ReTopK reuses the selections of similar earlier queries with an exact fallback Yao et al. [53], and a model can be prompted to declare its own attention scope Ho et al. [17]. Mixture-of-Depths and CALM allocate computation across depth adaptively [34, 35]. In AIR the decision variable is a complete depth pattern of indexer calls for each agent segment, learned from terminal reward. The learning procedure uses standard Lagrangian actor-critic tools [45, 7, 39].
Serving systems. Inference engines such as vLLM and SGLang [21, 57] manage cache memory and batching, and several compared methods are released as patches to them; our MiniMax-M2 runs attach the indexer through an attention adapter in a serving engine. Speculative decoding Leviathan et al. [23] and KV-cache eviction Zhang et al. [56] reduce other costs; we do not combine them with AIR, and speculative decoding is disabled in our MiniMax-M2 runs. Systems that tier model state across memories Niko1221 [28] compete with AIR’s index keys for accelerator memory (Appendix § 23).
Adaptive Index Reuse
Sparse selection and cross-layer reuse
At layer and query position , a DSA-style indexer scores every candidate position with small heads,
where and are index-query and index-key projections of the layer input and is a softmax gate over heads. The layer keeps the highest-scoring candidates, , and core attention reads them together with a few sink positions and a recent window from the layer’s own key-value cache. Candidates exclude the sinks and the window, so for a fixed number of attended positions the core-attention work is the same whichever layer chose them (Appendix § 8).
IndexCache assigns each eligible layer a role, Full or Shared. A Full layer runs the indexer; a Shared layer reuses the positions of the nearest preceding eligible Full layer,
and the first eligible layer is always Full. Reuse shares only position IDs, which travel through a buffer that lives for one query position. With the pattern F, S, S, F, S, the first layer writes its selection, the next two attend over it, the fourth overwrites it, and the fifth attends over the fourth layer’s selection; each layer gathers its own keys and values at the shared positions. In AIR, every eligible layer also projects and stores the index key of each new token while it is Shared. Its index-key cache then covers the whole prefix, so a Shared layer can become Full at a later segment without replaying past states.
Decisions, budgets, and the library
An agent episode alternates between environment observations and model responses. The controller decides at segment boundaries: a segment starts at every model invocation and after every decoded tokens, and it covers the prefill of newly appended tokens and the decoding that follows. The budget for segment is chosen before any of its tokens is computed, because it fixes which layers run the indexer for all of them. Each budget names one complete depth pattern from a library built offline. Following training-free IndexCache, we score a pattern by the next-token loss of agent-generated calibration sequences run under it. Greedy elimination starts from the all-Full pattern and repeatedly converts to Shared the layer whose conversion raises this loss least. Each budget stores the pattern reached on this single path when its number of Full layers is reached, so the library is nested: a larger budget adds Full layers without moving existing ones. The primary backbone, with 36 eligible layers, uses ; other backbones scale these counts to their number of eligible layers (Appendix § 12).
Observations. Before segment the controller reads , a fixed-length summary of the history computed only from finished computation. It combines the position in the trajectory (context length, segment index, tool steps), the latest tool event (its type, length, and traceback or test-summary flags), the previous budget and cumulative cost, and model and index statistics of the completed segment, such as predictive entropy and the overlap between recent selections. A low budget therefore never receives information that only a refresh would have produced (Appendix § 13).
Cost model
The controller’s cost signal counts, analytically from tensor shapes, every operation whose amount the controller changes or adds. Index-key maintenance is paid at every eligible layer for every token under every budget; selection, comprising the index query, scoring, and top-, is paid only at Full layers; and runtime and controller overhead covers buffer traffic, pattern switches, feature extraction, and the controller’s inference. Each operation costs its floating-point operations plus its bytes moved times a fixed rate (the accelerator’s peak BF16 throughput divided by its memory bandwidth). The indexing cost of segment is
over the segment’s query positions , and the segment cost adds the overhead terms. Core attention is excluded, because its work per query is the same under every budget. Episode costs are normalized by , the mean all-refresh episode cost on controller-training data, and the budget bounds the expected normalized cost. Measured latency, full-model compute, and memory are reported separately (Appendix § 14).
Learning and deployment
The controller is a two-layer MLP network that outputs a categorical distribution over budgets, while the backbone, indexer, and library stay frozen. Each episode ends with a verifier score . Learning solves
where and are the expected score and expected normalized cost, through the Lagrangian with a compute price $\lambda \ge 0 ({xref}Figure 2`).
Warm start. Terminal reward is sparse, so training starts from a supervised estimate. We replay logged all-refresh continuations under every library pattern with teacher forcing, which yields each budget’s increase in token loss and its normalized cost on the same text. Cross-fitted regressors predict these labels from the observation, and the controller is fitted to a soft target that trades predicted loss against predicted cost at the smallest price meeting (Appendix § 16).
Constrained policy gradient. Replay labels cannot show how a budget changes later text, tool calls, or success. The second stage therefore trains on-policy, crediting each decision with the terminal score minus the priced cost still to come,
with a learned value baseline (Williams, [45]). The price rises when a batch overspends the budget and falls otherwise, on a slower timescale than the policy (Borkar, [7]; Tessler et al., [39]). During training, every sampled budget runs as sampled, so the gradient credits actions that actually executed (Appendix § 17).
Algorithm 1 summarizes training. EXECUTE-SEGMENT(, ) runs segment under mask : the queued prefill and up to decoded tokens. Each query position passes through every layer and appends its key and value, and at eligible layers its index key, exactly once; Full layers write selected position IDs to , and Shared layers read them and attend with their own key-value tensors. The helper returns the segment cost , including controller overhead (Algorithm 2 gives the query- and layer-level procedure).
Algorithm 1 AIR training (overview).
Input: frozen backbone, indexer, nested library ; warm start ; budget
1: ; initialize critic and price 2: repeat 3: for each episode in a batch do 4: for each segment until ends do 5: ▷ completed computation only 6: sample ▷ runs as sampled 7: 8: if the response is complete and calls a tool, run it and queue its output 9: end for 10: record the verifier reward 11: end for 12: compute the returns (Eq. (5)) at the current 13: one actor step along 14: regress the critic onto 15: update from the batch’s overspend every batches 16: save the checkpoint if its position is predeclared 17: until the rollout budget is exhausted
Deployment. The constraint holds only in expectation, so feasibility is decided on held-out validation tasks from repository owners that appear in no other partition. Every saved checkpoint and every static library entry receives simultaneous bounds on expected cost and success. The deployed controller is the feasible learned checkpoint with the largest success bound; if none is feasible, the best feasible static entry is deployed as a static fallback. This choice is frozen before final evaluation. A controller that always returns one budget recovers a static IndexCache schedule.
Experimental Setup
Backbones. Every backbone is dense in attention as released and receives an injected indexer distilled from its own attention (, ), its own library, and its own controller. Qwen3-4B-Instruct-2507 is the primary model ([30]); Qwen3-1.7B, Qwen3-8B, the original Qwen3-4B, Gemma-3-4B-it, and the mixture-of-experts MiniMax-M2 extend the study ([49, 15, 9]). All backbones run at a 32K cap, and those that support it also at 128K (Appendix § 18.§ 1).
Benchmarks and metrics. SWE-bench Verified (500 tasks) carries the primary comparison ([20, 10]); Terminal-Bench 2.0 (89) ([25]) and the retail (114), airline (50), and telecom (114) domains of -bench ([6]) test transfer. Success is the percentage of attempts solved; for , each task runs four times and success is pass@1, the mean over trials, while pass⁴ is the four-trial reliability, the share of tasks solved in all four trials. Time is mean end-to-end minutes per attempt, counting model, tool, environment, and user-simulator time for failed attempts too; minutes per successful task divides total time by the number of successes, and speedup divides the all-refresh time by a method’s time.
Fitting data. Controllers are fitted on 37,968 repository-repair tasks from SWE-Gym and SWE-smith ([29, 50]), after removing every task whose repository owner also owns a SWE-bench Verified repository; owners are split into disjoint partitions, one per fitting stage (Appendix § 19.§ 3). Each backbone’s repair controller is evaluated unchanged on all five benchmarks; separate in-domain studies train on subsets of the target benchmarks and report held-out subsets.
Comparators and cost matching. Native dense attention and all-refresh sparse execution bound what the injected sparse attention keeps. Static IndexCache runs one library pattern for the whole episode in a specialized runtime that omits index keys at permanently Shared layers. Three static selections have distinct roles: a development reference fixed before controller training; the deployed policy, chosen on validation as above; and the analysis comparator, the static library entry with the highest success on the final sample among those within the cost tolerance. Because the analysis comparator sees the final sample, it favors the static side. The budget is with tolerance 0.53. Controls replace AIR’s ingredients one at a time (placement, state-free timing, rules, local learning, restricted state), and external selectors change how a selection is computed or stored (Appendix § 20).
Results
Main comparison
Higher success at matched cost. On the primary model at 128K, validation deploys a learned controller (Appendix § 24.§ 1). On SWE-bench Verified it solves 32.4% of tasks, against 27.0% for the analysis comparator, the static schedule , at a normalized indexing and control cost of 0.524 against 0.502; both lie within the 0.53 tolerance. Across the five benchmarks, success rises by 2.2 to 5.4 points, and minutes per successful task fall on all five, from 64.6 to 52.7 on SWE-bench Verified (Table 1). Time per attempt changes little: it is 2.0% lower on SWE-bench Verified (1,025 against 1,046 seconds), so the gain per successful task comes mainly from solving more tasks.
Strong comparators remain visible. Table 2 adds external selectors. AIR has the highest success (32.4%) and the lowest time per success (52.7 minutes) of the frozen-backbone methods on SWE-bench Verified, and HISA with AIR leads telecom on both measures (46.9%, 17.0 minutes) and airline on time per success (11.9 minutes). On Terminal-Bench 2.0, four settings, including all-refresh and LongCat cross-layer indexing, reach 27.0% against AIR’s 24.7%; LongCat cross-layer indexing leads retail, LatentIndex airline, and HISA with AIR telecom. All-refresh beats AIR on airline as well (48.5 against 46.5%) while taking 11.2% longer per attempt there, and 4.6% longer on Terminal-Bench 2.0. All-refresh stays within 1.5 points of native dense attention, which takes 1.27 to 1.43 times as long per attempt.
| Method | SWE-bench V. Succ.↑ | SWE-bench V. Min/s.↓ | Terminal-Bench Succ.↑ | Terminal-Bench Min/s.↓ | retail Succ.↑ | retail Min/s.↓ | airline Succ.↑ | airline Min/s.↓ | telecom Succ.↑ | telecom Min/s.↓ |
| References | ||||||||||
| Native attention | 32.0 | 71.5 | 28.1 | 67.3 | 57.7 | 20.8 | 50.0 | 19.1 | 46.5 | 27.8 |
| All-refresh | 31.6 | 56.8 | 27.0 | 54.1 | 57.0 | 14.6 | 48.5 | 13.8 | 45.4 | 20.2 |
| Static allocation | ||||||||||
| Static () | 27.0 | 64.6 | 22.5 | 61.9 | 53.3 | 14.1 | 42.5 | 14.1 | 42.5 | 19.5 |
| Alternative selectors | ||||||||||
| Training-aware IndexCache | 30.4 | 56.3 | 23.6 | 58.8 | 56.4 | 13.3 | 49.0 | 12.2 | 45.0 | 18.4 |
| LatentIndex | 31.6 | 55.1 | 27.0 | 52.3 | 56.4 | 14.1 | 49.5 | 12.8 | 45.2 | 19.4 |
| LongCat CLI | 31.4 | 53.8 | 27.0 | 50.6 | 58.6 | 12.7 | 49.0 | 12.2 | 45.0 | 18.5 |
| ReTopK (per head) | 29.8 | 60.0 | 27.0 | 52.6 | 56.6 | 14.0 | 47.5 | 13.4 | 46.1 | 19.0 |
| AIR-based methods | ||||||||||
| HISA + AIR | 30.6 | 55.5 | 24.7 | 55.2 | 56.1 | 12.9 | 48.5 | 11.9 | 46.9 | 17.0 |
| AIR | 32.4 | 52.7 | 24.7 | 56.4 | 57.2 | 13.1 | 46.5 | 12.9 | 46.5 | 17.9 |
Table 2. AIR-based methods have the highest success on two benchmarks and the lowest time per successful task on three, and AIR improves over the matched static schedule on all five. Qwen3-4B-Instruct-2507, 128K; bold: best frozen-backbone value (ties included); tint: AIR-based; gray: native attention, unranked.
long per attempt.
Against the selector frontier. Figure 3a places AIR among 24 settings on SWE-bench Verified that vary the refresh count of the flat, HISA, and LongCat hierarchical selectors or the number of active MISA heads. Static settings trade success for time along their curves; AIR reaches 32.4%, above every one of them, at 17.1 minutes per attempt. The Terminal-Bench 2.0 and airline exceptions above hold for this frontier as well (Appendix § 24.§ 5).
Figure 3. AIR on SWE-bench Verified and in training (Qwen3-4B-Instruct-2507, 128K). (a) Success against minutes per attempt for 24 selector settings. (b) Validation success against normalized cost for three seeds; dashed: the 0.50 budget. (c) All-refresh time divided by each method’s time, per attempt. (d) Success on long-history continuations at prefix occupancy .
Learning, per-attempt gains, and long histories. The deployed policy comes from a consistent training process: every seed of the primary controller moves from high cost to below the budget while its validation success rises, and it ends with a mixed allocation over the menu (Figure 3b). The time saving holds attempt by attempt: the median paired speedup over all-refresh is 1.06 on SWE-bench Verified, while the indexer adaptation of ReTopK is slower than all-refresh on most attempts (Figure 3c). In continuations that resume from shared snapshots once the prefix fills 60, 75, or 90% of usable context, AIR solves more of the remaining SWE-bench Verified work than the static comparator and all-refresh in every band (Figure 3d); across all five benchmarks, all-refresh scores higher in 5 of the 15 bands, including all three airline bands (Appendix § 24.§ 6).
What the gain depends on
Table 3 removes one ingredient at a time. Placement matters at a fixed refresh count: seeded staggered masks lose 1.8 to 3.0 points on SWE-bench Verified to the greedy mask with the same . Varying the budget without reading the state adds nothing: a state-independent mixture scores 27.0% and a fixed temporal schedule 27.2%. Rules that read the state recover part of the gap (29.4% for an event trigger, 30.0% for lagged staleness), a supervised router reaches 30.4%, and a contextual bandit trained on immediate feedback 30.6%, so credit for delayed effects adds 1.8 to 2.0 points over local learning. Restricting the controller’s state costs 1.0 to 4.2 points, with event metadata alone losing most. Airline is the exception: five of six restricted or locally trained variants score 1.0 to 3.0 points above AIR there. Across 96 factorial configurations, the budget has the largest effect, and shorter decision intervals and the warm start both raise success (Figure 4); Figure 24 in Appendix § 24.§ 3 gives every factor with its time per attempt.
| Variant | SWE | Term. | Retail | Airl. | Tel. |
| Static and state-free | |||||
| Greedy mask, | 27.0 | 22.5 | 53.3 | 42.5 | 42.5 |
| Staggered mask, | 24.0 | 21.3 | 50.4 | 40.5 | 40.1 |
| State-free mixture | 27.0 | 22.5 | 52.9 | 42.5 | 42.5 |
| Rules and local learning | |||||
| Event trigger | 29.4 | 23.6 | 53.3 | 48.0 | 41.7 |
| Lagged staleness | 30.0 | 22.5 | 55.0 | 47.5 | 44.7 |
| Supervised router | 30.4 | 23.6 | 56.6 | 49.5 | 46.1 |
| Contextual bandit | 30.6 | 23.6 | 56.8 | 46.0 | 45.0 |
| Restricted state | |||||
| Event metadata only | 28.2 | 23.6 | 56.6 | 48.0 | 45.0 |
| Model + index stats | 31.4 | 24.7 | 56.6 | 47.5 | 44.5 |
| AIR (terminal reward) | 32.4 | 24.7 | 57.2 | 46.5 | 46.5 |
Table 3. Among the variants shown, AIR leads on SWE-bench Verified, retail, and telecom, ties the best Terminal-Bench 2.0 score, and the supervised router leads on airline. Success (%), Qwen3-4B-Instruct-2507 at 128K, ; bold: best per column, ties included.
Figure 4. Decision-interval and warm-start ablations on SWE-bench Verified (Qwen3-4B-Instruct-2507, 128K). Points are mean success over the 96 factorial configurations; gray marks show min–max ranges in (a) and the 48 matched pairs in (b).
Figure 24. Budget, decision interval, and library each move SWE-bench Verified success by up to 4 points, and the warm start adds 1.4 points on average. Dots: mean over the 96 controllers at each level; bars: range; in tokens (call: call-only); libraries G greedy, U uniform, S staggered, I independent.
An allocation example. In one SWE-bench Verified repair, a visible test returns a new reversed-range diagnostic. The controller sees the test-result flag on the latest tool event and raises the budget from to at the next model call, while fixed IndexCache keeps ; both then read the same branch to the same depth. The example shows where each method spends index work; the success rates come from the aggregate results (Appendix § 25 gives this event and full traces, including one in which AIR ends worse).
Across backbones and context caps
Across the 50 backbone, cap, and benchmark settings, AIR’s success exceeds the comparator’s in 37, ties in 7, and falls below it in 6 (Figure 5). Four losses are on Gemma-3-4B-it, where only five of 34 layers are eligible: three are its 32K settings, where no learned controller met the validation cost bound and the static fallback is deployed, and one is 128K telecom (). The other two are 32K airline on the primary model () and MiniMax-M2 (). The largest gain is on Qwen3-8B at 32K on Terminal-Bench 2.0 (+5.6). Time per attempt stays between 3.9% longer and 7.0% shorter than the comparator’s. Training behaves alike on every backbone: normalized cost falls from 0.79 below the 0.50 budget within about 4,000 rollouts while validation success rises by 6.2 to 7.6 points at 128K, except Gemma-3-4B-it at 32K, whose cost stays at 0.59 (Appendix § 24.7).
Figure 5. AIR’s success minus the analysis comparator’s (percentage points) for every backbone, cap, and benchmark; blue favors AIR. Gemma-3-4B-it at 32K deploys the static fallback .
Transfer and reliability. The repair controller transfers to terminal and customer-service tasks with the gains above, and training in the target domain adds 1.3 to 5.0 points on held-out subsets at equal time per attempt. On , AIR’s four-trial reliability pass4 exceeds the comparator’s in five of six domain and cap settings and is lower on 32K airline (Appendix § 24.§ 6).
Where the time goes
AIR halves the indexing work of all-refresh execution, and the agent loop dilutes the saving (Figure 6). Tools, the environment, and the user simulator take 52 to 55% of AIR’s time on SWE-bench Verified and Terminal-Bench 2.0 and 27 to 31% on , so whole-agent speedups over all-refresh are 1.05 and 1.10 to 1.11. Model-side gains are larger: across backbones and caps on SWE-bench Verified, AIR runs prefill 1.14 to 1.30 times and decoding 1.05 to 1.17 times as fast as all-refresh. The speedup grows from 0.97–1.00 at 4,096 tokens to 1.20–1.33 at 126,976 tokens, because selection work grows with the prefix while key maintenance and controller work stay constant per query. The design’s own costs stay small: keeping every eligible layer ready for promotion costs up to 2.3% more time and up to 12 times the index-cache memory at small refresh counts, and controller cost barely moves end-to-end time (Appendix § 24.§ 4).
Figure 6. Tools and the environment bound the whole-agent speedup, which grows with the prefix (Qwen3-4B-Instruct-2507, 128K). (a) Time per attempt relative to all-refresh; (b) AIR’s speedup against prefix length.
Composition. With the flat indexer, learned allocation beats the selector’s static schedule by 2.2 to 5.4 points; with HISA, four-head MISA, or LongCat hierarchical indexing, the change ranges from −2.5 to +3.4 points, so the benefit of learned allocation depends on the selector (Appendix § 24.§ 5).
Conclusion
AIR starts from an execution scheme that already works, IndexCache’s Full/Shared reuse, and adds a decision about when to spend fresh selections. A small controller that reads completed computation and tool events, chooses among nested patterns, and learns from terminal reward under an expected compute budget improves success over the strongest static schedule at matched cost on every benchmark of the primary model and lowers time per successful task. The settings where a static schedule ties or wins, a backbone with few eligible layers and benchmarks where all-refresh still scores higher, mark where state-dependent allocation has the least to offer.
Limitations
The gains depend on the backbone and the benchmark: on Gemma-3-4B-it, where five of 34 layers are eligible, validation deploys a static fallback at 32K, and all-refresh sparse attention still scores higher than AIR on Terminal-Bench 2.0 and airline. The indexer is injected into dense checkpoints and may lose quality that a natively sparse model would keep. Index keys at every eligible layer cost memory that static IndexCache avoids. The finite library cannot reach patterns outside it, and the expected-cost constraint lets individual episodes exceed the budget. Tool and environment time bound the whole-agent speedup. Small benchmarks (50 airline tasks, 20 to 40 tasks in the in-domain subsets) limit precision, SWE-bench Verified concentrates nearly half of its instances in one repository, and each setting reports one deployed policy. Results use fixed benchmark releases and one user simulator (Appendix § 23).
Ethical Considerations
AIR reduces indexing compute for agents and so slightly lowers the energy needed per successful task; it does not change what an agent is allowed to do. The benchmarks use public repositories, terminal tasks, and simulated customers; no human participants or personal data were involved. The user simulator is a commercial model used under its terms. Faster agents can be deployed more widely, raising the usual concerns about automated code changes and customer-facing actions; the method inherits those risks from the underlying agent.
Extended Background: Sparse Selection and Cross-Layer Reuse
AIR builds on DSA-style dynamic sparse attention and on IndexCache’s cross-layer reuse. For the objects the two methods share we adopt IndexCache’s notation [5]. is the number of transformer layers, the sequence length, the layer index, and the position of the current query token. For that query, layer ’s indexer produces a score vector with one entry per candidate position, and holds the IDs of the highest-scoring candidates. The count refers to these selected candidates only; the sink and recent-window positions introduced below are attended in addition.
Query positions and candidate positions. Positions are numbered from 1, and the query at position is the token whose layer outputs are being computed: at each layer, its hidden state supplies the attention query vector, and the earlier positions supply keys and values from the layer’s cache. During prefill, a block of newly appended tokens is processed together, and under the causal mask each of them is a query that sees only the positions up to itself. During incremental decoding, one token is generated at a time, and the query is the most recently appended token. Each query’s own key and value are appended before attention, so the query at position sees exactly the positions , itself included, and equals the context length at that query.
Two small groups of positions are always attended without being scored. The sink set is a fixed set of positions, chosen per backbone, that every query attends regardless of the ranking; it keeps a stable set of anchor positions in the attended support. A query can attend only positions that already exist, so at position the sinks in use are . The recent window holds the last positions up to , or all of when , and supplies the immediate local context. The indexer ranks the remaining positions, the candidate positions . Candidate positions are tokens of the prefix; Section A.1 defines which transformer layers take part in index reuse.
Indexer scores. At layer the indexer compares the current query with every candidate position through small score heads. Each head projects the normalized layer input into an index query for the query position, , and into an index key for every earlier position , ; tilded bold vectors denote these index-head projections, and the index keys are cached as tokens arrive. A head’s score for position is the scaled dot product of its query and key, passed through a ReLU so that only positive matches contribute. A query-dependent softmax gate over heads then weights the per-head scores and sums them into one ranking value per candidate position:
This score adapts the DSA lightning indexer; Appendix § 11 lists the differences. A layer that selects keeps the highest-ranked candidates,
where the operator ranges over : it returns the best positions, or all of when fewer than exist, so it keeps positions, and it breaks ties toward the smaller position. Core attention then reads the attended set , the selected candidates together with the available sinks and the recent window, from the layer’s own key-value cache. A position that is both a sink and in the window is attended once, so . For a fixed number of attended positions, the core-attention work per query is the same whichever layer chose them. Figure 7 shows the two steps at one Full layer: scoring and top- selection, then the gather of the layer’s own keys and values at the selected, sink, and recent positions.

Figure 7. Dynamic sparse attention at a Full layer: (a) the indexer scores the prefix and keeps the top- positions plus sink and local positions; (b) core attention gathers the layer’s own keys and values there.
IndexCache execution semantics
Running the indexer at every layer repeats much of the same ranking work, because adjacent layers of DSA models select largely overlapping positions. IndexCache therefore lets only some layers rank. Backbones differ in which layers can use this kind of attention, so we first name the layers that take part in index reuse. The eligible layers form an ordered set of size . For the Qwen backbones every layer is eligible, ; for a model that interleaves sliding-window and global attention, only the global layers are eligible (Section K.1). A layer pattern assigns a role to each eligible layer. A Full layer runs the indexer and selects positions with Equation (7); a Shared layer reuses the positions of the nearest preceding eligible Full layer,
As in IndexCache, the first eligible layer is always Full, so every query receives at least one fresh selection; we write the maximization index as because denotes agent steps here.
The position buffer and the persistent caches. Two kinds of state are involved, and they have different lifetimes. The selected positions travel through a temporary buffer that lives for one query position: each Full layer overwrites it, each Shared layer reads it, and it is released after the query’s last layer. With five eligible layers and the pattern F, S, S, F, S, layer writes , layers and attend over , layer overwrites the buffer with , and layer attends over . The next query starts again at , so every generated token makes its own selections. Layers outside keep their native attention and neither read nor overwrite the buffer. The persistent caches grow at every token and are never recomputed when the pattern changes: each layer keeps its own key-value cache, and each eligible layer keeps a cache of index keys. Reuse therefore shares only position IDs. Each layer gathers its own keys and values at the shared positions, so a Shared layer’s output still depends on its own key-value tensors (Figure 8).

(a) Nearest-Full index buffer . Full layers score the prefix; Shared layers read the latest position IDs.

(b) Shared positions, private KV tensors. Layers share position IDs and keep their own persistent KV state.
Figure 8. IndexCache execution inherited by AIR: Full layers overwrite the position buffer , Shared layers reuse it, and AIR appends index keys at every eligible layer so any of them can later become Full.
Shared execution and later promotion. A Shared layer skips the query-time part of the indexer: the index query, the scoring of candidates, and the top- selection. In AIR every eligible layer still projects and stores its index key for each new token, whatever its current role. This upkeep lets a layer that is Shared now become Full at a later segment: its index-key cache already covers the whole prefix, so it can score past positions without replaying past hidden states. Each cached entry was computed from the layer input when its token was processed, and that input depended on the selections made under the pattern then in force; this is the usual causal semantics of incremental decoding. A specialized static runtime whose pattern never changes can omit index keys at the layers it keeps Shared. AIR keeps them at every eligible layer, and its cost model charges this upkeep (§ 3.3). Table 4 sets out what AIR shares with IndexCache and what it adds.
| Axis | IndexCache | AIR |
| Layer action | Full or Shared | Full or Shared |
| Patterns | One deployed pattern | One pattern per budget, nested |
| Decision state | None (global) | Current agent segment |
| Pattern selection | Calibration LM loss | Calibration LM loss |
| Controller training | None | NLL-regret warm start, then terminal reward and cost |
| Compute target | Fixed retention ratio | Expected trajectory budget |
| Index keys at Shared layers | Not stored | Stored at every eligible layer |
| Controller overhead | None | Included in the compute cost |
| Relation | Static schedule | Constant policy recovers it |
Table 4. What AIR shares with IndexCache and what it adds. Both methods execute the same Full and Shared layer roles.
Method Details
This appendix adds to § 3 the details that the main text states only in outline.
Segments. An agent episode alternates between environment observations and model responses , where indexes agent steps. Suppose a test run returns a 1,200-token report and the model answers with 1,300 tokens. The new invocation opens segment , which covers the prefill of the 1,200 report tokens and the first 512 decoded tokens; segments and cover the next 512 and the remaining 276 decoded tokens. Every prefilled or decoded token is a query position that passes through all layers , so the decision operates on three nested timescales (Figure 10).

Figure 10. Three execution timescales: one library mask per segment of the agent loop, cache appends per query within a segment, and the buffer per query across depth.
Menus. Each budget names a library pattern with Full eligible layers (Figure 9). The primary backbone, with eligible layers, uses ; other backbones scale these counts with

Figure 9. Offline library construction: one greedy elimination path from all-refresh to aggressive reuse supplies a nested mask for each refresh count in the menu.
taken as a set, so duplicate counts collapse before the controller’s output size is fixed. The rule keeps all-refresh as the largest entry; Table 8 reports the realized fractions .
Table 8: Backbones. “Supported context” follows each model card; caps, menus, and settings are our choices (Hugging Face repositories pinned to fixed revisions). : decoder layers; : eligible layers; menus follow Equation (9).
| Checkpoint (role) | Params (total/active) | Attention | / | Menu (fractions) | Supported context | Context caps | Reasoning mode | Precision |
| Qwen3-4B-Instruct-2507 (primary) | 4.0B | GQA, 32 Q / 8 KV heads | 36 / 36 | 3, 5, 9, 18, 36 (.08, .14, .25, .50, 1) | 262,144 native | 32K, 128K | non-thinking only | BF16 |
| Qwen3-1.7B (smaller) | 1.7B | GQA, 16 Q / 8 KV | 28 / 28 | 3, 4, 7, 14, 28 (.11, .14, .25, .50, 1) | 32,768 | 32K | hybrid; thinking off | BF16 |
| Qwen3-8B (larger) | 8.2B | GQA, 32 Q / 8 KV | 36 / 36 | 3, 5, 9, 18, 36 (.08, .14, .25, .50, 1) | 32,768; 131,072 with YaRN | 32K; 128K with YaRN | hybrid; thinking off | BF16 |
| Gemma-3-4B-it (second family) | 4B | 5 local (1,024) : 1 global; 8 Q / 4 KV | 34 / 5 | 1, 2, 3, 5 (.20, .40, .60, 1) | 128K (131,072) | 32K, 128K | none | BF16 |
| MiniMax-M2 (MoE extension) | 230B / 10B | GQA, 48 Q / 8 KV; 256 experts, 8 active | 62 / 62 | 6, 9, 16, 31, 62 (.10, .15, .26, .50, 1) | 196,608 | 32K, 128K | interleaved thinking, retained weights | FP8 |
| Qwen3-4B (release control) | 4.0B | GQA, 32 Q / 8 KV | 36 / 36 | as primary | 32,768; 131,072 with YaRN | 32K | hybrid; thinking off | BF16 |
Table 8.
Paid probe. Statistics of the newly appended tokens, such as the indexer’s score entropy on them, would require selector work. A paid-probe ablation buys them: before the decision, it runs the frozen indexer of the first eligible layer , whose input does not depend on the budget, on the new tokens in temporary buffers, keeps three summary statistics, leaves the persistent caches untouched, and charges all of its computation and memory traffic to the controller (Appendix § 13).
Controller size. The actor has a hidden layer of width on the -dimensional observation and one output per budget, so with biases it has
parameters, where : five budgets for the Qwen backbones and MiniMax-M2 and four for Gemma. With , the actor has 17,157 parameters (17,028 for Gemma), and a critic of the same width with one scalar output has 16,641. The critic and the two warm-start predictors are used during training only; the frozen feature projections run at deployment and are charged to .
The controller’s size does not grow with context length, but the indexer state does: each eligible layer stores index-key values per token, of the layer’s core key-value cache, or one quarter for and .
Policy-gradient estimator. The controller samples budgets at a fixed temperature , and the critic serves only as a baseline. With the backbone and indexer frozen, only the controller’s sampling probabilities depend on , so the likelihood-ratio identity gives an unbiased gradient
over a batch of on-policy episodes, with from Equation (5). Costs incurred before decision cannot depend on , which justifies summing cost from . The actor takes one step per batch, the critic then regresses onto , and every batches the price takes the projected step
so cost weighs more in after an overspending batch. The runtime executes every sampled budget as sampled, because the estimator would otherwise credit an action that never ran. If no static policy passes the validation bounds either, the operating point is declared infeasible and no policy is deployed at that budget (Appendix § 17). Algorithm 2 summarizes execution and training.
Notation
Please check Table 5 for full details.
| Symbol | Meaning |
| Number of decoder layers ( for the primary backbone) | |
| Ordered set of eligible layers, its size, and its first layer, which is always Full | |
| Sequence length | |
| Indices for layers, agent steps, controller decisions, query positions, and key positions | |
| Selected positions per query, and | |
| Fixed sink positions, and those available at query : | |
| Recent window: the last positions up to | |
| Candidate positions: | |
| Indexer score vector at layer for query position | |
| Selected top- positions (at most ) used by layer for query position | |
| Index query and index key of head (tilded bold: index-head projections) | |
| Attended set at layer : , of size | |
| Temporary index buffer, overwritten by each Full layer and released after the last layer | |
| Persistent index-key cache at layer | |
| Full/Shared layer pattern; is the nearest preceding Full layer | |
| Budget menu of backbone , library pattern for budget , and its number of Full eligible layers ( for the primary backbone) | |
| Realized refresh fraction | |
| Controller observation before decision , the selected budget, and the pattern in force | |
| Aggregated teacher attention distribution and indexer output distribution over | |
| Agent steps and controller decisions in episode ; | |
| Fixed FLOP-per-byte conversion in the cost model | |
| Compute cost of segment and of the episode | |
| Indexing cost: all-layer index-key maintenance and Full-only selection work | |
| Cost of traffic and pattern switches for segment | |
| Controller overhead: features, probe, and policy inference for segment | |
| Immutable terminal verifier score | |
| Expected episode compute budget, and the frozen mean all-refresh training cost | |
| Episode cost and budget normalized by | |
| Nonnegative dual price for compute | |
| Independent RL training runs, and controller seeds per final task |
Table 5. Core notation for agentic adaptive index reuse.
Execution Details
Indexer specification
The score of Equation (1) adapts the DSA lightning indexer, which sums ReLU-activated query-key products weighted by query-dependent head weights (DeepSeek-AI, 2025). Our specification differs from DSA in two respects: each score head has its own key projection, whereas DSA shares one index key across heads, and the head weights are softmax-normalized gates, whereas DSA uses an unnormalized projection. The scale is equivalent to the constant scaling that the DSA reference implementation folds into its head weights, because ReLU commutes with positive scaling. Per-head keys make the index-key cache times larger than a shared key of the same dimension would.
Index-key maintenance and cache history
Section A.1 explained why every eligible layer keeps its index keys while it is Shared: a layer promoted at a segment boundary finds keys for the whole prefix in its own cache and can score past positions without replaying past hidden states. This subsection records what that upkeep costs and what the cached values mean. Each eligible layer appends its core keys and values and its index keys at every token, whatever its current role. The upkeep costs stored values per token and eligible layer, plus the index-key projection at eligible layers that are currently Shared. For comparison, the core cache holds key and value entries per token and layer for every backbone in Table 8. The alternative, reconstructing keys only at promotion time, would require either stored layer inputs, which take values per token and layer, or a replay of the prefix; storing the keys is the cheaper option. Every cached entry keeps the value computed when its token was processed. That value depended on the layer input at the time, and the layer input depended on the selections made under the pattern then in force, so two runs that chose different budgets earlier can hold different cache contents for the same prefix. This is the usual causal semantics of incremental decoding, and it is the reason the cost and feature definitions refer to the actual execution history. Figure 11 follows one decision and one query through this execution on the Gemma-3-4B-it layout, where sliding-window layers bypass the position buffer.

Figure 11. One AIR decision and one query on the Gemma-3-4B-it layout: (A) the mask is fixed from completed history; (B) Full layers write position IDs that Shared layers read, while sliding-window layers bypass the buffer; (C) IDs are released per query and caches persist.
Relation to IndexCache and the scope of the policy-class guarantee
AIR inherits three elements from IndexCache: the observation that cross-layer top- overlap makes consecutive indexer calls partly redundant, the Full/Shared branch that skips those calls while every layer keeps its own KV tensors, and greedy loss-based search over patterns (Bai et al., [5]). What changes is the policy class and its objective. IndexCache fixes one pattern from calibration loss; AIR chooses among several precomputed patterns from the evolving agent state and learns that choice from terminal task feedback under a compute constraint. The setting differs too: IndexCache runs on models whose indexers were trained with sparse attention, whereas we inject an indexer into a dense checkpoint (Section H). In this paper, “static IndexCache” therefore means IndexCache’s training-free search applied to our injected indexer.
What the policy class guarantees. A constant policy that always picks one library entry reproduces that static schedule exactly, including its selected positions and outputs, and a policy that ignores the state reduces to a fixed mixture of static schedules; both serve as controls (Section K.3). Their execution costs match AIR only in the dynamic-ready runtime, which maintains index keys at every eligible layer. A specialized static runtime omits those keys at permanently Shared layers and is the stronger systems comparator. The policy class contains every constant policy over the declared library. Under the same budget , execution in the dynamic-ready runtime, and the same cost model, in which a policy of the class pays its feature-extraction and controller-inference costs even when its output is constant, the best feasible policy in the class is therefore at least as good as the best feasible constant library policy. The argument covers neither a static schedule run without a controller, which avoids , nor the specialized static runtime, which also avoids the index keys at permanently Shared layers. It also guarantees nothing empirical: training may reach a different policy, and the argument says nothing about held-out quality or latency. An empirical claim requires AIR to beat the strongest feasible static schedule at matched realized cost (Section K.5).
Schedule Library Construction
The library separates where refresh happens from when it happens. It is built once per backbone, offline, before any controller exists, and it decides which layers are Full at each budget; the controller later decides only which budget to use.
Search objective. For each refresh count in the menu, we look for the pattern with Full eligible layers, among them, that gives the lowest next-token loss on agent-generated calibration sequences when the whole sequence, prefix included, runs under that static pattern. The loss is a token-weighted average whose weights balance environment-event categories and repositories. The calibration set is collected once by all-refresh execution, and every candidate pattern is scored on this same reference corpus. This model-side surrogate decides only where to refresh; terminal reward enters later, at the outer level.
Greedy elimination. Exhaustive search over patterns is combinatorial, so we follow training-free IndexCache and remove layers greedily from the all-Full pattern [5]. Each step tries every remaining Full eligible layer except as Shared, evaluates the calibration loss on the same cached sequences, and commits the removal with the lowest loss. A path from Full layers down to the smallest menu entry takes steps and candidate evaluations: for the primary model, 33 steps and 627 evaluations from 36 to 3; for MiniMax-M2, 56 steps and 1,876 evaluations from 62 to 6; for Gemma’s global layers, 4 steps and 10 evaluations. Because every
Algorithm 2 AIR: segment-level budget selection inside constrained policy-gradient training
Input: backbone with eligible layers , frozen indexer, nested library , warm start , Output: predeclared controller checkpoints for validation-based selection
procedure FORWARD ▷ one query position under pattern
for do
Append layer ’s key and value for
if then
Run the layer’s native attention ▷ untouched
else
Append layer ’s index key for
if then ▷ always at
Select (Eqs. (1)–(7));
else
▷ from layer
end if
Attend over with layer ’s own keys and values
end if
end for
Release
end procedure
; initialize the critic and the price
repeat
for each episode of a batch do
for each segment until the episode ends do
Build from completed computation; sample
FORWARD for each query position of the segment
Add the segment cost (§ 3.3); run the tool call if the response ended in one
end for
Record the verifier score
end for
Compute ; take one actor step along (Eq. (5)); fit the critic to
Update every batches (Eq. (12)); save checkpoints at predeclared positions
until the rollout budget is exhausted
budget’s pattern is read off this single path, the library is nested. A held-out schedule-validation subset, drawn by owner group from the schedule-calibration partition with a fixed fraction and seed before the search, checks the stability of the path and selects nothing.
Controller Observations and Feature Requirements
The controller must decide before it pays for a refresh, so everything it observes comes from computation that has already finished. We keep three kinds of signal apart: lagged statistics that the default controller reads at no extra selector cost, paid probes that an ablation buys explicitly, and offline diagnostics that are never controller inputs.
Features by source. The controller receives a partial observation of the causal history , and every default feature exists before any model or selector computation governed by . The observation concatenates five groups of features with missing-value indicators. Position features record context length, environment step, segment index, and tokens since the last tool return. Environment features derive from raw tool type, output length, traceback and test-result parser flags, file transitions, and elapsed tool steps, and a compact embedding of the latest tool output is the mean of the backbone’s input embeddings over its tokens. Cost features record the normalized cumulative cost and its gap from the target. The remaining two groups are lagged: they summarize computation from the completed segment . Lagged model features summarize the final token of that segment, including predictive entropy and a low-dimensional projection of its hidden state. Lagged index features include the previous mask, the layerwise distance to the most recent Full layer, and overlap, score entropy, and selection margin computed from queries that have already completed. The hidden-state and tool-output projections are principal-component maps fitted on controller-training states before reinforcement learning and frozen thereafter. Missing-value indicators distinguish the first segment, which has no completed predecessor, from observed zeros.
Timing and cost requirements. Table 6 gives the requirements that each feature group meets. For every feature we record its source token, source layer, timestamp, cache version, formula, missingvalue rule, dependence on earlier actions, runtime, memory traffic, and any duplicated selector work. Future tokens, terminal reward, counterfactual loss, repository identity, and any statistic produced after is chosen are excluded. Paid probes. Score entropy, top- margin, or index overlap for the new query would require selector work, so the default controller never receives these current-query values. A paid-probe ablation runs one frozen, action-independent probe on the newly appended tokens before is chosen. The probe uses the frozen indexer of the first eligible layer , which is Full under every budget and whose input depends only on the embeddings and on any native, noneligible layers below it, and it reports score entropy, top- margin, and overlap with the most recent completed selection at . The probe runs on temporary state. It computes the embeddings, the native layers below , and the index queries and keys of for the newly appended tokens in scratch buffers, scores them against the cached index keys of the prefix and against one another under the causal mask, and keeps only the three statistics; nothing is appended to the persistent caches, and the selected positions are discarded. The segment’s forward pass then computes these quantities again and appends every key and value, and every index key, exactly once per position, as Algorithm 2 specifies. Within the paid-probe comparison, the runtime executes the probe once per segment under every policy and adds all of its compute and memory traffic to ; each probe operation is counted once, separately from the forward pass that follows (Table 7). This rule prevents a low-budget action from receiving refresh-level information for free.
| Block | Example fields | Requirement and cost treatment |
| Trajectory position | Segment index, context length, elapsed tool steps | Read from history at the boundary and normalize with training-only statistics |
| Interface event | Traceback, test result, file read, edit, API reply | Derive from the latest completed tool event, with no verifier or future outcome |
| Budget history | Previous budget, cumulative normalized cost, target gap | Compute only from executed actions and the frozen cost model |
| Cache state | Previous mask, distance to last Full layer, completed index age | Record source token, layer, timestamp, cache version, and action-history dependence |
| Lagged uncertainty | Completed-segment entropy and logit margin | Read retained logits from segment , with a missing flag at |
| Lagged index state | Completed-query overlap, score entropy, selection margin | Reuse statistics already produced in segment , with no new prefix scoring |
| Lagged representation | Projected hidden state of the last completed token | Freeze source layer, PCA map, normalization, precision, and retention cost |
| Observation content | PCA-projected mean input embedding of the latest tool output | Remove raw owner and repository identity; fit no projection on final tasks |
| Paid current-prefix probe | Action-independent entropy, overlap, or drift proxy | Run once before , discard selected IDs, and include all compute and memory traffic in the cost |
Table 6. Controller observation groups and the timing and cost requirements each meets. Every value exists before the segment’s budget is chosen.
Offline structural diagnostic. For offline analysis only, we quantify structural change around an environment event . Let and denote the selected IDs on matched diagnostic queries immediately before and after the event, restricted to their common prefix. Their Jaccard distance, one minus the ratio of intersection to union, computed for each layer, tests whether agent events alter retrieval structure across depth (Figure 12). Because it uses post-event selector output, the diagnostic is never a controller input.
Controller architecture. The controller is a two-affine-layer network (hidden width , a pointwise nonlinearity, and outputs) whose logits, divided by a fixed temperature , define a categorical distribution over the backbone’s budgets; Equation (10) gives its parameter count, and and are fixed per backbone. We choose on controller-training data before reinforcement learning and keep it fixed afterward, including at deployment, so the deployed policy is the policy that was trained. Deterministic argmax routing is evaluated only as a secondary variant with its own feasibility check. A value network with the same hidden width, a scalar output, and no shared parameters serves as the baseline during policy-gradient training and is discarded afterward. The controller is kept small to limit its latency and its capacity to overfit; its per-segment inference, FLOPs plus activations, and the feature computation that precedes it are charged to (Table 7). Two ablations restrict the state: the model-only controller removes interface features, and the phase-only baseline keeps deterministic event metadata and drops model and index signals.
| Term | Floating-point operations | Bytes |
| for key projections, plus input normalization | written, plus amortized weight reads | |
| for query and gate projections; for dot products; for scaling, ReLU, and gating; for top- selection | of index keys read, plus amortized weight reads | |
| None | written at a Full layer or read at a Shared layer | |
| , per segment | None | for the new mask when |
| , per segment | Entropy and margin over the retained final-token logits; PCA projections; overlap statistics | Logits, stored position sets, and projection weights read |
| , per segment (paid-probe comparison only) | For the appended tokens: embeddings and native layers below (standard per-token counts); at ; at ; entropy, margin, and overlap statistics | Weight and embedding reads; temporary writes and reads of layer inputs and index keys; prefix index keys of read; previous selection read |
| , per segment | Two-layer MLP: plus activations | Controller weights and read |
Table 7. Cost breakdown by operation, per query position and layer unless stated otherwise.
Cost Model and Counting Rule
Two different quantities are called cost in this paper. The optimization cost is the analytic indexing and control cost of § 3.3: it is the axis of the constraint, the dual update, and the decision rules, and it is computed identically for every policy from tensor shapes. The resource measurements, such as full-model compute, wall-clock time, and memory, describe what a deployment would experience and are reported separately.
What the optimization cost leaves out. Core attention is excluded from the optimization cost because, for a given query, every budget attends to the same number of positions: selected positions plus the available sink and window positions, each counted once. Its total, like that of projections and feed-forward layers, still varies with episode length, so full-model FLOP equivalents and end-to-end latency are reported separately; a full-model saving cannot exceed the indexer’s share of per-token work. A policy can also lower by ending episodes early, which shows up as lower success when it harms the task, so we also report cost per processed token, generated tokens, tool calls, and episode length. Resource reports further include index-key and core KV-cache bytes, peak memory, and cache-allocation latency.
Normalization and the static reference. Raw counts would give a poorly scaled dual problem next to a terminal score in . For each suite and context condition, we therefore divide costs by , the mean episode cost of all-refresh execution on the controller-training partition, frozen before controller fitting; segment costs are normalized the same way, . A preregistered ratio sets the normalized budget . On a separate static-selection partition, the static reference is the library entry with the highest success rate among those whose mean normalized cost stays within , measured in the specialized static runtime, with ties broken toward lower cost and then fewer Full layers. Its constant policy is frozen before routing labels are built. If no library entry stays within budget, the operating point is invalid.
Counting rule. Table 7 lists the analytic counts behind Equation (3) and the buffer and controller terms of § 3.3. For query position and layer , let , let be the number of positions actually selected, let be the bytes per index-key element, and let position IDs use 4 bytes. Weight reads are counted once per forward step and divided among the query tokens processed in that step. Each entry is converted to a FLOP equivalent (§ 3.3). We fix per backbone whether the scoring kernel materializes scores in accelerator memory; if it does, their writes and reads are added to the byte count.
Full-model and wall-clock measurements. Full-model FLOP equivalents, which are reported but excluded from the optimization cost, add the standard per-token counts for projections, feed-forward layers, and core attention over positions.
Sparse Attention and Indexer Preparation
Every schedule in this paper runs on an injected indexer, so each backbone needs a working sparse-attention implementation before any library or controller can be built. Preparation attaches the indexer, distills it in two stages, and accepts or rejects it; Gemma-3-4B-it and MiniMax-M2 need further adaptations, and the training-aware IndexCache comparison adapts the indexers to one fixed pattern.
Attaching the indexer
Every backbone in Table 8 is dense in attention as released and contains no indexer, so AIR runs on an injected selector. For each backbone we attach the indexer of Equation (1) to every eligible layer and replace that layer’s dense core attention with top- sparse attention through the model’s attention interface, which keeps its rotary embeddings, query-key normalization, and grouped-query layout. Layers outside are not modified. All backbone parameters stay frozen. The indexer uses score heads and selected positions, excluding sinks and the local window, on every backbone. is a shared experimental choice: the score heads have their own projections, separate from the backbone’s key-value groups, of which the backbones have 8 (Qwen, MiniMax-M2) or 4 (Gemma-3-4B-it). The index dimension , the sink set, the local-window width , and the index-key precision are fixed per backbone. Each backbone stores index-key values per token and eligible layer on top of its core cache of values per token and layer (, for Qwen and MiniMax-M2; , for Gemma).
DSA and the models evaluated by IndexCache train their indexers jointly with sparse attention ([5] (DeepSeek-AI 2025)). Our backbones were trained with dense attention only, so all-refresh sparse execution can lose quality relative to the native model. The sparse-attention check measures this loss on fitting validation before any controller is trained.
Distillation
Each backbone’s indexer is distilled from that backbone’s own dense attention, as in the DSA warm-up stage (DeepSeek-AI, 2025). For a query with a nonempty candidate set, the target sums the dense attention weights of all query heads of layer (, and 48 for Qwen3-1.7B, the 4B and 8B Qwen checkpoints, Gemma-3-4B-it, and MiniMax-M2) and renormalizes them over , and the indexer’s output distribution is the softmax of over the same set. Only indexer parameters are trained, with

Figure 12. Offline structural-change diagnostic: selected positions of matched queries before and after event are compared on their common prefix by Jaccard distance.
where the expectation runs over eligible layers and over query positions with nonempty candidate sets in agent-generated prefixes from the backbone’s indexer-training partition. Dense teacher passes first cache and the layer inputs, and an initialization stage trains each indexer on these dense states. A second stage executes the all-refresh sparse student, recomputes the indexer inputs from the student’s hidden states, stops gradients through those states, and keeps the same dense targets on the full candidate set; in DSA’s sparse stage, the KL term covers only the selected set. Top- selection stays outside the gradient path, and backbone and KV-projection gradients remain disabled throughout.
The target sums the head attentions. Each head’s dense attention at a global or full-attention layer is a distribution over the same causal positions , so the sum and the average differ only by the factor , which cancels after restriction to and renormalization. The restriction removes the sink and local positions, which the indexer never ranks, whereas IndexCache defines its aggregated distribution by averaging over heads without restriction [5]. Plain bold p and q denote distributions, while tilded bold vectors are index-head projections.
Acceptance
Held-out acceptance tests on each backbone’s indexer-validation subset, drawn by owner group from the indexer-training partition with a fixed fraction and seed before distillation, cover next-token loss, selected teacher mass, short-prefix execution, deterministic top- ties, incremental decoding, absolute position offsets, and equality of offline and online feature extraction. The acceptance decision precedes schedule search. After acceptance, the indexer is frozen, and every static schedule, router, bandit, and controller on that backbone shares its hash. A rejected indexer fails the sparse-attention check for every cell of that backbone.
Backbone-specific adaptations
Gemma-3-4B-it. The checkpoint is multimodal; we use only its language model. The layout is taken from Google DeepMind’s reference implementation (gemma/gm/nn/_gemma.py and _modules.py at commit 0f7e742) [16]: the 4B model has 34 layers whose attention pattern repeats five local sliding-window layers and one global layer, with a 1,024-token window, 8 query and 4 key-value heads of dimension 256, query-key RMS normalization before rotary embeddings, and rotary bases of for local and (scaled by 8) for global layers. The global layers are therefore 1-indexed layers 6, 12, 18, 24, and 30, and only these are eligible. The reference code establishes the architectural layout, and agreement with our pinned checkpoint is checked separately. The configuration, tokenizer, and chat template of the pinned Hugging Face revision (093f9f38) match the reference layout, including the attention pattern, window size, and head geometry. The sliding-window layers never consume a selection made by a global layer and never overwrite . The first global layer always refreshes, so every query receives one fresh selection; the menu therefore contains the anchor-only pattern. Because global and local layers use different rotary bases and the injected indexer reads the global layer’s own input, the index keys are computed in the global layers’ representation only.
MiniMax-M2. All 62 layers use full grouped-query softmax attention with 48 query and 8 key-value heads, partial rotary embeddings, and per-layer query-key normalization, and each block routes tokens to 8 of 256 experts. The released weights are FP8 with block scaling. The indexer is attached through an attention adapter in the serving engine, which computes index keys from each layer’s input, keeps expert routing and expert weights unchanged, and runs with the tensor and expert parallel layout fixed for the backbone. The configuration lists multi-token-prediction modules, but the released checkpoint contains no weights for them; speculative and multi-token decoding are disabled for every method, and any later study of speculative decoding would be a separate condition. The adapter reproduces the unmodified engine’s dense outputs when the indexer is bypassed, and a reference implementation’s all-refresh sparse outputs, on fixed prompts within a fixed tolerance, as part of the infrastructure check. Hosted access to the model cannot implement these changes. Pre-serving the interleaved thinking history means that earlier <think> blocks remain in the prefix and are attended, selected, and costed like any other tokens.
Training-aware IndexCache
Training-aware IndexCache trains the indexers for one fixed pattern with a multi-layer distillation loss, and its sparse training phase also updates the remaining weights with the language-modeling loss [5]. We compare AIR against a frozen-backbone variant that keeps only the indexer loss (Appendix § 20). For a pattern , the group served by Full layer is , and multi-layer distillation minimizes
Because each budget defines different served groups, an indexer adapted for one pattern is fitted to that pattern’s groups only. AIR therefore keeps one frozen indexer bank shared by every library entry, and training-aware IndexCache is evaluated as a separate comparison with one adapted bank per evaluated pattern.
Supervised Warm Start
The warm start turns logged agent behavior into an initial controller whose predicted cost meets the budget, in three steps: replay produces counterfactual labels on fixed text, regressors predict those labels from the observation, and a priced soft target over budgets defines the policy that the controller imitates.
Replay labels. All-refresh execution collects agent segments on controller-training environments. For each logged pre-action history , we keep the continuation the behavior run produced and replay it with teacher forcing under every library pattern, from the same prefix and caches. Replay yields, for each budget , the mean negative log-likelihood of the continuation’s assistant tokens and its normalized compute cost . These are counterfactual labels on fixed text: they do not show how another budget would have changed later text, tool calls, or terminal success.
Predictors and the soft target. Local regret measures the loss increase relative to the static reference. Two action-conditioned regressors, for regret and a nonnegative for normalized cost, are fitted with a Huber loss averaged over all actions in the menu. Their predictions are cross-fitted, so the prediction for a record comes from models that never saw that record’s split group (Section K.2). For a compute price , the predictions define a soft target over budgets,
which prefers budgets with low predicted regret and, as grows, budgets with low predicted cost. At every state the expected predicted cost under is nonincreasing in , because its derivative is minus the variance of under , divided by . We therefore find by bisection the smallest price at which the expected predicted episode cost over the logged trajectories meets ; if no price suffices, the operating point is infeasible. The warm-start parameters minimize the KL divergence from to , averaged over logged states.
Because those states come from all-refresh execution, the warm policy’s own cost can differ from . The reinforcement-learning stage adjusts the price on its own trajectories, and the validation bounds of Appendix § 17 decide whether any trained policy meets the budget. The frozen warm policy also serves as the supervised baseline.
Constrained Reinforcement Learning Details
Gradient estimator and update schedule. The policy gradient of Equation (5) is an unbiased estimate of at fixed when episodes come from the current policy, have bounded length and bounded score and cost, is positive and differentiable, and the baseline depends only on and on parameters fixed before the batch. Costs incurred before decision cannot depend on , which justifies summing cost from , and the constant has no gradient. An inaccurate critic, including one fitted under an earlier , adds variance but no bias. Episodes end at submission, environment termination, or a token, tool-step, or wall-time limit, so they contain at most decisions. During training the runtime executes every sampled budget as sampled, so the gradient uses the logged budget that actually ran; work forced by the frozen cache-rebuild rule after a changed prefix is counted in the cost of the segment in which it occurs. The actor takes exactly one step per batch, the critic takes squared-error steps on the same batch, and the price is updated every batches with a smaller step size, so it moves on a slower timescale ([7, 39]). We run independent training runs from with different controller seeds.
Validation bounds and deployment. § 3.4 explains why feasibility is tested on held-out data. Before training, we freeze a cost tolerance , a confidence level , and a finite candidate set containing every predeclared checkpoint of every run and every static library policy. On the controller-validation partition, a grouped bootstrap gives each candidate a one-sided upper bound on expected normalized cost and a lower bound on expected success, each at level for candidates, so by the Bonferroni inequality all bounds hold jointly with probability at least , up to bootstrap approximation error. Among learned checkpoints whose cost bound is at most , we deploy the one with the largest success bound. If none qualifies, we deploy the qualifying static policy with the largest success bound, and if no static policy qualifies either, the operating point is declared infeasible. The deployed controller samples from the trained policy with the same , using a counter-based random stream keyed by task and decision index.
Figure 13 sketches the training loop. Algorithm 3 gives the full online execution of one episode, Algorithm 4 the offline preparation and warm start, and Algorithm 5 the training loop with policy selection.

Figure 13. Primal-dual training: each batch of episodes gives one actor step and several critic steps, the price moves on a slower timescale, and validation bounds select the deployed policy.
Extended Experimental Design
Backbones and scale
Each backbone answers a different question about where adaptive allocation helps (Table 8). Qwen3-4B-Instruct-2507 is the primary model ([30]): it is small enough for repeated agent rollouts with reinforcement learning, and it supports long contexts natively. Qwen3-1.7B and Qwen3-8B form the size comparison within the Qwen3 family ([49, 14 d]). Gemma-3-4B-it adds a second attention family, in which global attention layers are interleaved with sliding-window layers ([15, 14]). MiniMax-M2 is the mixture-of-experts extension, with about 230B total and 10B active parameters, and tests whether the method carries over to a large sparse model served by an inference engine ([9, 27]). Qwen3-4B-Instruct-2507 differs from Qwen3-1.7B and Qwen3-8B in release, context length, and reasoning mode as well as in size, so a difference among the three cannot be attributed to parameter count alone. The original Qwen3-4B ([15]), which has the primary model’s size and the release of the other two, serves as a supplementary control for this size comparison.
Every checkpoint is dense in attention as released: none contains an indexer, so AIR always runs on an injected selector. For the Qwen checkpoints every layer is eligible. Gemma-3-4B-it repeats five sliding-window layers followed by one global layer, according to Google DeepMind’s reference implementation ([16]); AIR is applied only to the five global layers (1-indexed layers 6, 12, 18, 24, and 30), and the sliding-window layers keep their native 1,024-token window and never touch . The in-
Algorithm 3 Online execution of one episode under a controller (deployment, rollouts, and validation)
Input: backbone with frozen indexer and nested library, feature map , policy , limits, controller seed Output: trajectory log, terminal score , compute cost
Empty all caches; queue the system and task prompt for prefill;
while the episode has not terminated do
from completed computation only; sample from the stream keyed by (task, )
FORWARD(, ) (Algorithm 2) for each query position of segment : queued prefill, then up to decoded tokens
Add (Section 3.3)
If the response is complete, parse its action; if it is a tool call, execute it and queue the observation for prefill
extended with and the segment’s outcome;
end while
return the log, , and
Algorithm 4 Offline preparation and supervised warm start. Every data collection and the measurement of use all-refresh execution.
Input: backbone, grouped partitions (Section K.2), menu , ratio , temperature Output: frozen indexer, nested library, , static reference , and warm start ; or an infeasibility verdict
Train the indexer with Eq. 13; run its acceptance tests; freeze it
Collect ; build the library by greedy elimination (Section E); freeze it
Measure and set ; select and freeze ; run the capability check (Sections 3.3 and K.2)
Collect
for each logged pre-action history in and every do
Replay the fixed continuation under ; record and
end for
Cross-fit the regret predictor and the cost predictor by split group
By bisection, find the smallest at which the predicted cost of (Eq. 15) meets ; if none exists, return operating point infeasible
Fit by minimizing the KL divergence from to
Algorithm 5 Constrained policy-gradient training and policy selection
Input: outputs of Algorithm 4, , , step sizes, , , , rollout budget, , Output: one selected learned or static policy, or an infeasibility verdict
for run do
; initialize ;
repeat
Run a batch of episodes with Algorithm 3 under
Compute (Eq. 5) for every decision with the current
Take one actor step along (Eq. 5), then critic steps regressing onto
Update (Eq. 12) if the batch count is a multiple of
Save the checkpoint if its position is predeclared
until the rollout budget is exhausted
end for
Compute simultaneous cost and success bounds on controller validation for every candidate
candidates whose cost bound is at most
if contains a learned checkpoint then
return the learned checkpoint in with the largest success bound
else if then
return the static policy in with the largest success bound
else
return operating point infeasible
end if
frastructure check confirms this layout against the pinned checkpoint’s own configuration. All 62 layers of MiniMax-M2 use full grouped-query softmax attention, so all are eligible. Equation (9) then gives the menus in Table 8: five budgets for the Qwen checkpoints and MiniMax-M2, and four for Gemma, whose deduplicated menu includes the anchor-only budget .
Contexts and decoding. Two context conditions are defined by their total-context cap, which bounds prompt and generation together. All cross-model comparisons use a common 32K cap, which every checkpoint supports natively. The 128K cap tests allocation when the candidate set is four times larger, and it is run only where the checkpoint supports it: natively for Qwen3-4B-Instruct-2507, Gemma-3-4B-it, and MiniMax-M2, and for Qwen3-8B only through the YaRN extension its model card describes, which we identify as an extension and apply only at 128K. Qwen3-1.7B and the original Qwen3-4B are evaluated at 32K only. Every cap reserves a fixed allowance for generation. Full episodes start from the original task state and run under the cap; continuations of long agent histories isolate scheduling under long state (Appendix § 19.§ 2). Each checkpoint uses its own chat template and tool-call format, and its reasoning and decoding settings are fixed across all methods compared on it. The hybrid Qwen3 checkpoints run with thinking disabled, matching the primary model’s non-thinking mode. MiniMax-M2 runs in its native interleaved-thinking mode and keeps every earlier <think> block in the history, as its model card requires, with the card’s sampling parameters and its own history format. The Qwen and Gemma checkpoints decode greedily.
Per-backbone sparse attention. Each backbone receives its own injected indexer ( score heads, separate from the backbone’s key-value heads, and ), acceptance tests, schedule library, normalizations, and controller; nothing trained on one backbone is reused on another. MiniMax-M2 runs through an attention adapter in its serving engine with speculative and multi-token decoding disabled (Appendix § 15.§ 4). A hosted API cannot be used, because AIR changes attention internally.
Evaluation populations and fitting data
Table 9 lists the five final evaluation populations. SWE-bench Verified carries the primary claim ([20, 10]). Terminal-Bench 2.0 ([25]) and the retail, airline, and telecom domains of -bench ([6]) test whether a controller fitted on repository repair transfers unchanged to terminal work and to dual-control customer service. The benchmark releases, task lists, and scoring definitions are fixed to those in Table 9, and every reported score names its release, because later releases change tasks and scoring under the same names: -bench v1.0.0, for example, keeps the database check for retail but adds LLM-judged natural-language assertions to 112 of its 114 retail tasks, drops the required-output check there, and edits the gold actions or user scenarios of some retail and airline tasks with unchanged identifiers. Comparisons and pooling stay within one release (Appendix § 19.§ 5).
Table 9: Final evaluation populations. Counts are the published size and the size of the pinned task list; the pinned count is authoritative for analysis. Releases and scoring definitions are fixed.
| Population | Tasks (published / pinned) | Release, harness, and scorer | Endpoint | Role |
| SWE-bench Verified | 500 / 500 | Pinned dataset revision; official swebench harness (version and image digests fixed) | Official resolved rate: all FAIL_TO_PASS and PASS_TO_PASS tests pass | Primary claim; repair controller |
| Terminal-Bench 2.0 | 89 / 89 | Harbor, dataset [email protected] | Official task-verifier success | Transfer |
| retail | 115 / 114 | tau2-bench v0.2.0, base tasks | pass@1 and passover 4 trials | Transfer |
| airline | 50 / 50 | tau2-bench v0.2.0, base tasks | pass@1 and passover 4 trials | Transfer |
| telecom | 114 / 114 | tau2-bench v0.2.0, base tasks, default dual control | pass@1 and passover 4 trials | Transfer |
Table 9.
trials. In -bench an LLM-simulated user converses with the agent; in telecom the user also has its own tools and acts on a shared environment, so solving a task requires coordinating actions on both sides ([6]). We keep this dual-control setting and fix the user-simulator checkpoint and configuration across all compared methods. Each task is run four times with the same deployed policy. pass@1 is the mean one-trial success over the four trials. pass, the four-trial reliability, requires success in all four trials of that policy; it is stricter than the conventional pass@4, which counts a task as solved if any of four attempts succeeds. The randomness of a trial has four separate sources, each with its own fixed seed stream: the controller-training seed of the selected policy, the controller’s sampling at evaluation, the agent’s own decoding when it samples, and the user simulator.
Fitting pipeline. For each backbone, fitting proceeds through indexer preparation, indexer acceptance, schedule calibration, selection of the static reference, warm-start fitting, constrained reinforcement learning, held-out controller selection, and one frozen final evaluation (Figure 14). In the main protocol, which evaluates a repair-trained policy for transfer on the full populations, every stage uses fitting data, and the five final populations are used only for the final evaluation. The in-domain studies (Section K.4) are the one exception: they train and select their controllers on designated training and validation subsets of Terminal-Bench 2.0 and of the domains, and they are evaluated only on the held-out subsets.

Figure 14. Experimental pipeline per backbone: separate fitting partitions train the indexer, calibrate the masks, and fit and select the controller, each frozen before the next stage.
The repair fitting pool combines SWE-Gym and SWE-smith ([29, 50]). After mapping every task to its canonical upstream repository and owner, we remove every task whose owner, or a former name of that owner, owns a SWE-bench Verified repository, keep Python tasks with a nonempty issue statement, and remove SWE-smith tasks duplicated in SWE-Gym. The resulting list of eligible tasks has 37,968 tasks from 122 repositories and 115 owners, and the owners serve as split groups for five partitions (Appendix § 19.§ 3, Table 12). Disjointness from Verified rests on this owner-level filter; SWE-bench’s own training split is not used.
| Partition | Target | All tasks: Tasks | All tasks: Repos | All tasks: Owners | SWE-Gym: Tasks | SWE-Gym: Share | SWE-smith: Tasks | SWE-smith: Share |
| Indexer training | 20% | 7,592 | 24 | 23 | 484 | 19.9% | 7,108 | 20.0% |
| Schedule calibration | 10% | 3,788 | 13 | 12 | 228 | 9.4% | 3,560 | 10.0% |
| Static selection | 10% | 3,801 | 12 | 12 | 251 | 10.3% | 3,550 | 10.0% |
| Controller training | 45% | 17,087 | 52 | 51 | 1,101 | 45.2% | 15,986 | 45.0% |
| Controller validation | 15% | 5,700 | 21 | 17 | 374 | 15.3% | 5,326 | 15.0% |
| Total | 37,968 | 122 | 115 | 2,438 | 35,530 |
Table 12. Repair-pool partitions by owner group (seed 20261007). Counts are eligible tasks. Shares are of the whole pool and of each source.
The repair pool supplies fitting data for repository repair only. Terminal and tool-use cells also need fitting data for their pre-evaluation checks and for method qualification. These data come from a source disjoint from Terminal-Bench 2.0 and from the base tasks, which remain reserved, in full, for transfer evaluation. In the transfer protocol, final benchmark data never influence indexer training, schedule search, feature or cost normalization, heuristic thresholds, controller fitting, check out-
comes, or checkpoint selection.
Checks before evaluation. A comparison of refresh schedules means little if the injected sparse attention has already lost most of the backbone’s ability, or if a task is so easy or so hard that every schedule scores the same. Three checks on fitting data therefore decide, before any final run, which cells of the backbone-by-population matrix are evaluated. The sparse-attention check requires that the backbone’s indexer pass its acceptance tests and that all-refresh sparse execution stay within a fixed margin of native dense execution on fitting validation. The capability check requires that the static reference’s success on fitting validation lie within a fixed interval, so that a floor or ceiling does not decide the comparison. The infrastructure check requires that the chat template, tool-call parsing, harness, serving backend, and memory at the context cap be qualified. Every checkpoint in Table 8 is evaluated on all five populations at each of its caps. A cell that fails the sparse-attention or infrastructure check is not evaluated. A cell that fails only the capability check is evaluated and reported descriptively, outside the confirmatory and secondary tests. Every failed cell stays in the reported matrix, marked with the check it failed.
Comparisons
The comparisons are arranged so that each one isolates one ingredient of AIR. Appendix M is the authoritative inventory of every arm, with the components it shares, its calibration, its added costs, and the question it answers (Tables 14, 15, 16, and 17); Table 10 summarizes the tracks.
Native dense execution runs the unmodified backbone and measures how much task quality the injected sparse attention keeps. All other arms share the injected indexer, , the sink and local sets, and the cost model. Static IndexCache runs one library pattern per episode in two runtimes: a dynamic-ready runtime that maintains index keys at every eligible layer, as AIR does, and a specialized runtime that omits them at permanently Shared layers; the strongest feasible specialized policy is the primary comparator. The controls for placement, state-free timing, rules, local learning, and restricted state, the alternative selectors, the composition experiments, and the retrained-backbone track are defined in Appendix M.
Qualification before evaluation. Every method follows the same sequence. It is first implemented on the primary model and checked for correctness and compatibility with fitting data from repository repair, terminal, and tool-use tasks, the latter two from the disjoint fitting source of Section K.2. The checks of Section K.2 then run on fitting data, and the extension of each qualified method to the other backbones is decided from its compatibility with the backbone’s attention layout and its resource cost (Appendix M.5). Only after these steps is each method evaluated, once, on SWE-bench Verified, Terminal-Bench 2.0, and the three domains. Within a comparison, the agent scaffold, checkpoint, decoding, population, and resource limits are held fixed. A method that changes the attended support or the model parameters is compared on end-to-end success and resource use, with the change reported, because the indexing cost of Equation (3) covers only the refresh decision.
Transfer and in-domain studies
Repository repair is the only domain with a large fitting pool, so the main question about other domains is transfer. The main transfer experiment selects one policy per backbone on repair validation tasks and evaluates it unchanged on Terminal-Bench and on all three domains. Transfer evaluation uses the cost normalization and the budget fixed on repair-training data, so a transfer result measures how well a repair-trained allocation rule carries over to terminal work and to dual-control customer service. Separate in-domain studies then ask how much of any gap comes from the training domain. They train terminal and tool-use controllers on training and validation tasks drawn from the benchmarks themselves (Appendix L.4). The pinned benchmarks provide no training split, so these studies partition the benchmark tasks by group and report only the held-out subset, with its size. Their results are labelled as in-domain results on held-out subsets, and they are reported apart from unchanged-policy transfer, which covers the full populations.
Endpoints and the primary comparison
Endpoints. Quality is the population’s official success measure (Table 9), reported as an absolute rate and as a percentage-point difference. Efficiency is reported at four levels that are kept apart: the indexing and control work per attempt from § 3.3; full-model work and model-only inference time; end-to-end time per attempt, with tool, environment, and user-simulator time shown separately; and prefill latency, decode throughput, peak accelerator memory, and cache memory. Input and output tokens, tool calls, and episode length are reported alongside. Cost per success divides the total measured cost of all attempts, failed ones included, by the number of successful attempts; when a method has no success, the ratio is reported as undefined. Reductions in index work are reported in their own units, separately from measured wall-clock time.
| Track | Methods | Extension rule | What must be controlled |
| Shared indexer | Anchor, static, periodic, staggered; mixture, temporal, event, staleness, supervised, bandit, AIR | Every accepted backbone | Frozen backbone and indexer; paired budget and state controls |
| Alternative indexer | Training-aware IndexCache; LatentIndex; MISA; HISA; LongCat CLI / HI | Every compatible injected indexer | Separate calibration; key, pooling, routing and training costs |
| Temporal adaptation | ReTopK with injected indexer | Every compatible injected indexer | Exact prefill; decode reuse; cache invalidation on prefix rebuild |
| Native-score methods | Published ReTopK; HShare; SFI; TidalDecode; Quest; Kascade | Qwen family; other families after adapter check | Different selector; own scores, caches and complete runtime charged |
| AIR composition | AIR with HISA, MISA-4 or LongCat HI | Where both components qualify | Own all-refresh reference, static library and fitted controller |
| Retrained backbone | Full LongCat Sparse Attention | Primary model only | Continual training and changed support; upfront cost disclosed |
| Related architecture | CLA; YOCO; You Only Index Once; HySparse / HySparse2; MiniMax Sparse Attention | Related work only | Different pretrained checkpoint; no frozen-backbone score assigned |
Table 10. Comparison tracks: each answers its own question and holds different factors fixed; read numerical rows within their track.
Three static selections. Three selections involve static policies, and each has its own role. The development static reference is chosen on the static-selection partition before any routing label is built; it anchors the warm-start labels and the capability check, and it gives a secondary contrast. The deployed controller is chosen on controller validation by the procedure of § 3.4 and frozen before final evaluation; when no learned checkpoint is feasible, it is the static fallback. The analysis comparator is chosen on the final sample itself: the static library entry with the highest success among those within the cost tolerance, run in the specialized runtime, and reselected inside every bootstrap replicate. Because it sees the final sample, the analysis comparator favors the static side, which makes the primary test conservative. Table 11 lists the three selections for every backbone and cap.
| Backbone | Cap | Dev. % | Dev. cost | Deployment | Final R values | |
| Qwen3-1.7B | 32K | R=14 | 9.4 | 0.485 | Learned | 14 |
| Qwen3-4B-Instruct-2507 | 32K | R=18 | 23.0 | 0.485 | Learned | 18, 36 |
| Qwen3-4B-Instruct-2507 | 128K | R=18 | 23.0 | 0.485 | Learned | 18 |
| Qwen3-8B | 32K | R=18 | 30.6 | 0.485 | Learned | 18 |
| Qwen3-8B | 128K | R=18 | 30.6 | 0.485 | Learned | 18 |
| Gemma-3-4B-it | 32K | R=2 | 12.2 | 0.388 | Static R=2 | 2, 3 |
| Gemma-3-4B-it | 128K | R=2 | 12.2 | 0.388 | Learned | 2, 3 |
| MiniMax-M2 | 32K | R=31 | 57.0 | 0.485 | Learned | 31 |
| MiniMax-M2 | 128K | R=31 | 57.0 | 0.485 | Learned | 31 |
| Qwen3-4B (original) | 32K | R=18 | 18.1 | 0.485 | Learned | 18 |
Table 11. The three static selections come from different data: the development reference (static-selection partition), the deployed policy (validation), and the analysis comparator (each final population; last column).
The primary comparison. The primary claim concerns Qwen3-4B-Instruct-2507 on SWE-bench Verified at the 128K cap and compares the deployed controller with the analysis comparator. AIR can succeed in two ways: it can reach higher success at matched cost, or it can match success, within a noninferiority margin, while spending clearly less. Let and be the differences in expected success and normalized cost between the deployed controller and the analysis comparator, with one-sided confidence bounds (lower, on ) and (upper, on ) at level . The adaptive claim holds if either preregistered rule holds,
where rule (i) is higher success at matched cost and rule (ii) is noninferior success with a minimum saving; splitting between the rules bounds the familywise error by . Tasks are the resampling unit; repeated trials and controller seeds stay inside their task’s cluster. The same comparison is computed for every other evaluated cell as a secondary analysis, with Holm adjustment within each predeclared family (populations for one backbone; backbones for one population). Training curves of verifier reward, validation success, normalized cost, constraint violation, the dual price, budget frequencies, and controller overhead are reported per backbone and labelled as repair training or in-domain training (Appendix § 19).
Experimental Protocol Details
Per-backbone settings
Contexts. A context cap bounds prompt plus generation for one episode and includes a fixed generation reserve . The common cross-model cap is . The 128K cap, , is used for Qwen3-4B-Instruct-2507 (262,144 native), Gemma-3-4B-it (131,072 native), and MiniMax-M2 (196,608 native), and for Qwen3-8B only with the YaRN rope scaling of factor 4 described on its model card; the YaRN setting is labelled as a context extension and applied only in that model’s 128K runs, so its 32K and 128K results come from two configurations of the same weights. Qwen3-1.7B, whose card states a 32,768-token context and no extension, and the original Qwen3-4B release control run at 32K only. Each backbone’s candidate-set size grows with up to its cap, so at 32K the fixed covers a larger fraction of the prefix than at 128K.
Templates, tools, and reasoning. Each checkpoint uses the chat template shipped with its pinned revision. All four Qwen3 templates emit tool calls as JSON objects inside <tool_call> tags and receive tool results as <tool_response> blocks in a user turn; the templates of Qwen3-1.7B, Qwen3-4B, and Qwen3-8B are identical, and that of Qwen3-4B-Instruct-2507 differs only in how it handles thinking. We parse this format with the Hermes-style parser that the Qwen documentation names, since the model cards name none. The hybrid Qwen3 checkpoints run with thinking disabled through the template. This matches the primary model, which supports only non-thinking mode, and follows their cards’ instruction to leave thinking content out of the history. MiniMax-M2’s template emits XML tool calls inside <minimax:tool_call> tags and returns tool results in dedicated tool turns; it runs with the model’s own reasoning and tool-call parsers, keeps every earlier <think> block in the history in its original form, as its model card requires, and samples with the card’s temperature (1.0), top-p (0.95), and top-k (40). Gemma-3-4B-it has no thinking mode, and its chat template, as reported by public mirrors of the gated repository, has no tool role and requires strict alternation of user and model turns; Google’s documentation describes tool use through prompting. Its tool calls therefore use a fixed text protocol: tool schemas are described in the first user turn, the model answers with a Python-style function call in a fenced block, which a deterministic parser extracts, and tool results are returned in the next user turn. The parse-failure rate of this protocol is part of the infrastructure check. The Qwen and Gemma checkpoints decode greedily. Within a checkpoint, every compared method uses the same template, parser, reasoning setting, and decoding configuration.
Full episodes and long-history continuations
Full episodes. A full episode starts from the original task state and runs under the cap of its context condition until the task ends or a limit is reached. Full episodes measure end-to-end success and resource use, and they are the unit of the primary comparison.
Long-history continuations. Most full episodes spend little time near the context cap, so continuations of long agent histories are evaluated as well, to isolate scheduling under long agent-generated state. A cap condition has the total-context cap that its 32K or 128K label names. A continuation must leave backbone ’s fixed generation reserve free, so its prefix holds at most tokens, counted with ’s tokenizer. The long-history band for is , where the fraction is fixed before construction. A frozen behavior agent runs each eligible fitting or evaluation task with all-refresh sparse execution under the declared tools and limits and the cap of condition , and we retain the first nonterminal controller boundary whose serialized causal prefix length lies in the band. A trajectory that terminates first, or whose prefix grows past the band between two boundaries, yields no snapshot, and the number of such trajectories is reported for each condition. The rule is defined when and ; the band is then nonempty, because for every positive integer . Both and are set on fitting data before any evaluation history is constructed. Snapshots are built once, by the frozen behavior agent, and the same snapshots serve every compared method. The snapshot stores the transcript, sandbox state, remaining tool and token budgets, environment and user-simulator random state, and split-group ID. Population construction reads only prefix length and termination; verifier output, candidate losses, and later task outcomes play no role.
Every compared policy starts from the same snapshot. An all-refresh burn-in rebuilds the core key-value cache and the index-key cache over the common prefix, and is empty before the first evaluated query. Burn-in work is identical across policies, reported separately, and excluded from the post-branch cost contrast. From the branch point onward, each policy controls every model-call and -token boundary until termination. The runtime inserts no filler tokens, duplicates no tool output, and preserves the original event order. A condition with fewer eligible trajectories than the fixed minimum fails this population check and is not evaluated; the construction rule stays fixed. All snapshots from one source task inherit its split group, so a shorter branch cannot enter training while a longer branch of the same task enters evaluation.
Fitting data and partitions
Sources and eligibility. The pool is built from SWE-Gym (Hugging Face revision bb94ed9e, 2,438 tasks from 11 repositories) and SWE-smith (revision ea6d7173, 59,136 tasks from 222 repositories) ([29, 50]); SWE-Gym-Lite is a subset of SWE-Gym and adds no tasks. Every task is first mapped to its canonical upstream owner/repository. SWE-smith stores tasks under its own mirror namespace (swesmith/owner__repo.commit); we recover the upstream owner and repository from each mirror name, confirm them against a GitHub snapshot, which also resolves renamed repositories, and group tasks by that upstream owner. The SWE-bench Verified revision c104f840 contains twelve repositories under twelve owners (astropy, django, matplotlib, mwaskom, pallets, psf, pydata, pylint-dev, pytest-dev, scikit-learn, sphinx-doc, sympy).
Each task receives at most one exclusion reason, assigned in the order below, and SWE-Gym loses no task under any rule. Owner-level matching is case-insensitive and removes 4,449 SWE-smith tasks from eight repositories owned by Verified owners (django/channels, django/daphne, pallets/click, pallets/jinja, pallets/markupsafe, pydata/patsy, pylint-dev/astroid, pytest-dev/iniconfig); no training repository coincides with a Verified repository itself. Two historical namespaces are excluded as whole owner groups because one repository in each moved to a Verified owner: PyCQA/pylint became pylint-dev/pylint, which the GitHub API confirms by redirect, and kennethreitz/requests became psf/requests, which the API no longer confirms because the old name now belongs to a 2021 fork of psf/requests. Neither moved repository appears in the sources; the exclusion removes 488 tasks from the other repositories in those namespaces, PyCQA/flake8 and kennethreitz/records. Because SWE-smith also contains other languages, we keep repositories whose tasks mostly modify Python files, as in Verified, which removes 8,228 tasks. SWE-smith tasks that match a SWE-Gym task by repository and pull-request number, identical issue text, or identical patch are duplicates; 347 are dropped in favor of the SWE-Gym copies. Finally, 10,094 SWE-smith tasks whose issue statement is empty or whitespace-only are dropped, because an agent cannot be posed such a task.
The eligible pool contains 37,968 tasks: 2,438 from SWE-Gym (11 repositories, 11 owners) and 35,530 from SWE-smith (120 repositories, 113 owners), together 122 repositories under 115 owners, since some repositories appear in both sources. Eligibility depends on task metadata only.
Container qualification. Before a partition is used, each task must also pass a container check under the pinned harnesses. The task’s environment image must be available, the reference fix must make every FAIL_TO_PASS and PASS_TO_PASS test pass, and an inert patch, which leaves the bug in place, must leave at least one FAIL_TO_PASS test failing. Each task’s check runs once. A task is run a second time, under a lighter load, when its reference-fix run leaves no parsed test results, including a timeout at the harness’s 90-second limit for a test run, or when that run passes every FAIL_TO_PASS test and fails only PASS_TO_PASS tests; the second outcome is final. One property of the pinned SWE-smith harness excludes tasks for a reason that lies outside the task. After applying a patch, the harness restores the files named by the task’s test identifiers to their original version, and for a doctest that name is a source module. When the reference fix edits such a module, the harness undoes the fix before the tests run, and the task fails its reference check. A static check of the frozen data finds 896 such candidates among the 35,530 eligible SWE-smith tasks. Candidates that fail their reference check are excluded. Those tasks are excluded like every other task that fails the check, which keeps the fitting pool consistent with the harness that will score it, and the summary reports them as a separate failure class next to the candidate count. Tasks that fail are dropped without reassigning owners, and retained counts are kept per partition and source, apart from the eligibility counts.
Partitions. The canonical owner is the split group. This paper fixes the assignment: complete owner groups go to five partitions with target task shares of 20% indexer training, 10% schedule calibration, 10% static selection, 45% controller training, and 15% controller validation, applied both to the whole pool and within each source. Each owner is described by its pair of SWE-Gym and SWE-smith task counts. Owners are placed in decreasing order of size, with ties broken by a permutation drawn with seed 20261007. Each owner goes to the partition that minimizes the sum over partitions of the squared deviations of the overall share and of each source’s share from the target, each divided by the target. A deterministic local search over single-owner moves and pairwise swaps then lowers this objective from 0.0029 to 0.0006. Table 12 gives the result. An earlier rule assigned each owner
by its majority source alone; it placed pandas-dev (737 SWE-Gym and 1,716 SWE-smith tasks) by its SWE-smith count and left controller training and validation with 18% and 6% of SWE-Gym. The vector rule brings every partition within 0.7 percentage points of its target in each source.
Whole-owner grouping still limits source diversity. SWE-Gym has only eleven owners, so each partition’s SWE-Gym tasks come from one to three of them, and the 374 SWE-Gym tasks in controller validation all come from Project-MONAI/MONAI. Controller validation therefore reports success and normalized cost separately for each source alongside the pooled values. The pooled bounds of Appendix J remain the selection criterion, and the SWE-Gym portion of validation is reported as a single-repository diagnostic. No owner or repository appears in more than one partition, and validation tasks never share an owner with training tasks. These guarantees concern the repair pool and SWE-bench Verified; the terminal and tool-use benchmarks are separated from fitting data by the requirement stated in Section K.2. An MBPP-derived repair suite remains a development suite, used only for instrumentation and leakage tests ([4]).
Per-backbone partitions. The same group assignment is used for every backbone, but each backbone collects its own behavior trajectories, fits its own indexer, library, feature and cost normalization, and controller, and selects its own checkpoint. The cost normalization of a backbone is measured on its repair controller-training partition and is then used unchanged for that backbone’s transfer evaluation. Every mutation, environment seed, replay row, and continuation snapshot inherits its source group.
Transfer and in-domain studies
Transfer. Each backbone’s transfer policy is the repair policy selected on repair controller validation. It is evaluated unchanged on Terminal-Bench 2.0 and on the three domains, with the repair cost normalization and the repair budget , and with no tuning on the target population.
In-domain studies. Two further studies ask what training in the target domain adds, and their results are reported apart from transfer. The terminal study partitions the 89 Terminal-Bench 2.0 tasks into training, validation, and held-out groups, stratified by task category, with a fixed seed. Results are reported on the held-out subset, labelled with its size. The tool-use study trains a separate controller for each domain. The pinned release has no training split. Release v1.0.0 publishes one (74/40 for retail and telecom, 30/20 for airline) whose identifiers all occur in the pinned v0.2.0 base tasks and together cover them exactly. We therefore apply these identifier lists to the pinned v0.2.0 task definitions and scorer, carve validation tasks from the training identifiers by group, and report results on the 40, 20, and 40 held-out tasks under the v0.2.0 scorer. For telecom, the pinned release also contains a larger generated task set beyond the base tasks, which may supplement training after its disjointness from the held-out identifiers is verified. Each in-domain study measures its own cost normalization on its training partition, and each in-domain policy is evaluated only on its held-out tasks; the transfer results cover the full populations.
-bench release and scoring
All results use release v0.2.0 (tag commit f8de30c, 2025-10-06) and its base task sets of 114 retail, 50 airline, and 114 telecom tasks; the paper’s table lists 115 retail tasks, and the pinned task list is authoritative. In v0.2.0, retail and airline tasks carry no explicit reward basis, so the default product of a database check (the final database must equal the one produced by the gold actions) and a required-output check applies, and telecom tasks are scored by environment assertions (94 tasks) or environment assertions plus actions (20 tasks). In v1.0.0 (tag 17e07b1, 2026-03-18), 112 retail tasks are scored by the database check together with natural-language assertions judged by an LLM, of which only 40 tasks carry nonempty assertions, and the remaining 2 retail tasks by the database check alone; airline keeps the database and required-output checks, and telecom’s base-task scoring is unchanged. The same release also edits gold actions or user scenarios for some retail and airline tasks without changing their identifiers.
Because these changes alter both what is scored and some tasks themselves, results are labelled with their release, and comparisons stay within one release. The v0.2.0 default user model is the versioned alias gpt-4.1 at temperature 0; we pass the dated checkpoint gpt-4.1-2025-04-14 at temperature 0 explicitly, which is the simulator used in the benchmark paper and pinned by later releases, and keep it fixed for every method.
Checks before evaluation
The three checks are run per backbone and population on fitting data, before any final run, and their outcomes are recorded in the evaluation matrix. The sparse-attention check requires that the backbone’s indexer pass the acceptance tests of Appendix H and that all-refresh sparse execution stay within a fixed success margin of the native model on fitting validation from the corresponding domain. The capability check requires that the static reference’s success on fitting validation lie within a fixed interval ; a cell outside the interval is reported descriptively, without a confirmatory contrast. The infrastructure check requires a qualified chat template and tool-call parser with a fixed maximum parse-failure rate, a harness that reproduces the benchmark’s reference scoring on its provided reference solutions where they exist, a serving backend that reproduces reference outputs (for MiniMax-M2, the attention-adapter parity tests of Appendix H: dense bypass against the unmodified engine and all-refresh sparse execution against the sparse reference implementation), and enough memory to run the backbone at its cap. If the primary cell itself fails a check, the failure is reported and the primary claim moves to the predeclared fallback, the primary model’s in-domain retail study, evaluated on its held-out tasks.
Endpoints and reporting
Success. SWE-bench Verified counts an instance as resolved when all of its FAIL_TO_PASS and PASS_TO_PASS tests pass under the official harness. Terminal-Bench 2.0 uses the task verifier’s binary outcome. In , a trial succeeds when the pinned scorer’s reward, the product of the components listed for the task, equals one. For a task with trials of one trained policy and successes, is estimated by and averaged over tasks, so the case is the mean success rate, which we write pass@1, and , the all-four success, is the fraction of tasks solved in every trial. The four trials of a task use four controller-evaluation seeds and four user-simulator seeds and always the same controller checkpoint, so measures success in all four trials of one trained policy. When results are reported for several controller-training runs, is first computed for each run and then averaged across runs. Scores are compared only within the pinned benchmark release. Success is reported as an absolute rate and as a percentage-point difference from each comparator.
Resource use. For every attempt we record the indexing and control work of § 3.3; full-model work; model-only inference time; end-to-end time, split into model time and tool, environment, and user-simulator time; prefill latency and decode throughput; peak accelerator memory and the sizes of the core key-value and index-key caches; input and output tokens; tool calls; and episode length. Cost per success and index-work reductions follow the definitions of Section K.5.
Primary comparator and decision rules. After final evaluation, we choose , the static library entry with the highest success on the final sample among those whose mean normalized cost is at most , run in the specialized static runtime, with ties broken toward lower cost and then fewer Full layers. Because this choice sees the final sample, it favors the static baseline; every bootstrap replicate repeats it. The analogous differences against , the reference fixed during development, form a secondary preregistered contrast. Before final evaluation, we freeze , , , and ; each decision rule in Equation (16) is an intersection of two one-sided conditions, and allotting to each rule keeps the familywise error of their disjunction at most . Both contrasts are repeated with measured end-to-end time as the cost axis as secondary analyses. Every endpoint includes controller and probe overhead.
Statistical aggregation
Task-level comparisons pair environment images, initial states, decoding configuration, limits, and task order, and they keep the four random streams of Appendix § 21 separate. On SWE-bench Verified and Terminal-Bench 2.0 each task is run once, with one controller-evaluation seed; in each task has four trials of the deployed policy, which first yield the task’s pass@1 and pass contributions. Reward and cost are then averaged within a task before resampling, so the task is the resampling unit and repeated trials and seeds stay in their task’s cluster.
On SWE-bench Verified, the bootstrap resamples tasks. Verified has only twelve repositories, one of which contributes 46% of the instances, so repository-clustered resampling rests on a coarse cluster structure and serves only as a sensitivity analysis. On Terminal-Bench 2.0 tasks are resampled directly, and in each domain tasks are resampled with their four trials. Validation bounds on the fitting pool use the owner-clustered bootstrap. Policy fitting, normalization, schedule search, dual tuning, and controller runs happen once, before resampling, and every replicate reuses their results. Secondary cells use Holm adjustment within two predeclared families: the five populations for one backbone, and the backbones evaluated on one population. Variation across the training runs is reported for the validation candidates; the final contrast concerns the single selected policy.
Training diagnostics
For every backbone and training domain we report, against rollout count, the terminal verifier reward of training episodes, success on controlled validation, the mean normalized cost and its excess over , the dual price , the frequency of each budget, and the controller’s overhead as a share of indexing cost. Curves are labelled by training domain: the repair curves belong to the policies evaluated for transfer, and the terminal and tool-use curves belong to the separate in-domain studies.
Evaluation matrix
All five checkpoints and the release control are evaluated on all five populations at the 32K cap, and the three 128K-capable checkpoints plus Qwen3-8B with YaRN at the 128K cap. Each cell records the outcomes of the three pre-evaluation checks before final evaluation. A cell enters the confirmatory and secondary analyses only if all three checks pass; a cell that fails only the capability check is evaluated and reported descriptively, a cell that fails the sparse-attention or infrastructure check is not evaluated, and every failed cell is reported with the check it failed. Table 13 lists the questions isolated by each comparison.
| Comparison | Held fixed | Question |
| Native vs. all-refresh sparse | Backbone, prompts, tools, limits | Does the injected sparse attention preserve agent capability? |
| All-refresh vs. static IndexCache | Backbone, indexer, , runtime | How much indexer work does static reuse remove, and at what quality? |
| Dynamic-ready vs. specialized static runtime | Masks and model outputs | What does keeping all index keys cost? |
| Anchor-only, periodic, staggered vs. greedy static | Refresh count | Does layer placement matter within a budget? |
| Static vs. mixture and temporal schedule | Action frequencies or time sequence | Does varying the budget without state help? |
| Heuristics vs. supervised router | State, menu | Do simple rules capture the allocation signal? |
| Supervised router vs. bandit vs. AIR | State, menu, budget | What do chosen-action feedback and delayed credit add? |
| AIR vs. strongest feasible static | Everything except the scheduler | Primary contrast (Equation (16)) |
| AIR vs. alternative indexers and caches | Backbone, harness, population | End-to-end success and resource frontiers |
| Flat/HISA × static/AIR | Final attended budget | Does allocation help after cheaper selection? |
| Repair-trained policy on terminal and | Policy, normalization, budget | Does allocation transfer unchanged? |
| In-domain vs. transferred policy (held-out subsets) | Held-out tasks | What does in-domain training add? |
| Across backbones at 32K | Cap, populations, protocol | Does the result depend on scale, family, or MoE? |
| Qwen3-4B vs. Qwen3-4B-Instruct-2507 | Size and architecture | How much of a Qwen scale difference is release? |
| 32K vs. 128K cap | Backbone, selector, menu | Does lower selected-key density change refresh value? |
Table 13. Comparisons and the question each isolates. Rows apply to every evaluated backbone and population unless stated otherwise.
Comparison Protocols
The comparisons fall into four groups, which differ in what they hold fixed and therefore in how a difference can be interpreted: refresh controls on the shared indexer (Appendix § 20.§ 1), alternative indexers and cache representations (Appendix § 20.§ 2), composition experiments (Appendix § 20.§ 3), and architectural and retrained-backbone comparisons (Appendix § 20.§ 4). For each method outside the first group, we describe its mechanism, how we adapt it to our injected indexer or backbones, the state and cost it adds, and the qualification test it must pass. Source versions, code revisions, and differences between papers and their released code are collected in Appendix § 20.§ 6.
Refresh controls on the shared indexer
Table 14 lists every controlled comparison and Table 15 every ablation. Unless a table says otherwise, a method shares the backbone, injected indexer, , sink and local sets, agent harness, prompts, tool schemas, decoding configuration, limits, and cost model of its backbone, and differs only in how many and which eligible layers refresh. Calibration and training use the fitting partitions of Appendix § 19.§ 3; in the transfer protocol nothing is tuned on a final population, and in an in-domain study only that study’s designated training and validation subsets are used. Every controller and heuristic is charged for its own feature extraction and for any probe it runs. These controls answer the paper’s central question at matched indexer cost, and only they enter the decision rules of Equation (16).
| Method | What changes | Fixed | Calibration or training | Question |
| Native backbone | Unmodified attention, no indexer | Harness, decoding | None | Does the injected sparse attention preserve agent success? |
| All-refresh sparse | Every eligible layer Full | Indexer, | None | Upper end of refresh cost |
| Anchor-only () | Only Full | Indexer, | None | Is any depth refresh needed? |
| Periodic depth mask | Full layers at a fixed interval, same | Refresh count | None | Does greedy placement matter? |
| Staggered depth mask | Full layers at seeded random positions, same | Refresh count | Preregistered mask seeds | Is the greedy pattern better than chance? |
| Static IndexCache | One greedy library pattern for the episode | Library | Greedy calibration search | Static reuse at each budget |
| Static runtime variants | Dynamic-ready vs. specialized runtime | Pattern, outputs | None | Cost of keeping all index keys |
| Strongest feasible static | Best specialized static entry within tolerance | Selection rule | Reselected on the final sample and in every bootstrap replicate | Primary comparator |
| State-independent mixture | Budgets sampled from fixed frequencies | Marginal action rates | Frequencies frozen on controller validation | Does state matter beyond the action mix? |
| Fixed temporal schedule | Budget follows a fixed time sequence | No state input | Sequence fixed on fitting data | Does timing without state help? |
| Event-triggered refresh | Budget raised after tool events | Thresholds | Thresholds set on fitting data | Do simple event rules suffice? |
| Lagged-staleness threshold | Budget raised when completed-query overlap drops | Threshold, probe cost | Threshold set on fitting data | Does a cheap staleness signal suffice? |
| Supervised router | Warm-start policy, frozen labels | Features, menu | Counterfactual replay | Value of local prediction alone |
| Contextual bandit | Policy trained on immediate replay feedback | Features, menu | Replay rewards, priced cost | Value of chosen-action feedback |
| AIR | On-policy constrained policy gradient | Features, menu | Terminal reward under the budget | Value of delayed terminal credit |
Table 14. Native reference and refresh controls on the shared indexer. “Fixed” lists what is held constant beyond the shared setup described in the text.
| Method | What changes | Fixed | Calibration or training | Question |
| Phase / model / index / model+index / full state | Controller inputs | Training recipe | As AIR | Which signals carry allocation value? |
| Decision interval | and model-call-only decisions | Menu | As AIR | Responsiveness vs. overhead |
| Library construction | Uniform, staggered, or independently searched patterns | Refresh counts | As AIR | Outer policy vs. inner schedule |
| Budget sweep | Ratio | Everything else | As AIR | Shape of the trade-off |
| Overhead | Feature blocks added one at a time | Everything else | As AIR | Does added overhead remove the saving? |
Table 15. Controller and library ablations. Each varies one factor of AIR and keeps the rest of its recipe.
Several entries need more detail. Anchor-only execution keeps only Full, so each query receives one fresh selection and no refresh deeper in the network; it is the pattern and bounds the indexer cost from below. The periodic depth mask places Full layers at a fixed interval within a query, and the fixed temporal schedule varies the budget over time, cycling through a fixed sequence of budgets at segment boundaries without reading the state. The lagged-staleness heuristic raises the budget when the overlap between the two most recent completed Full queries falls below a threshold; because it uses only completed computation, it needs no extra selector work, and a variant with a paid probe on the new tokens counts the probe in its cost. For Gemma, whose menu has four entries on five global layers, all five static refresh counts also run as static controls. The deployed controller’s static fallback (Appendix § 17) and the strongest feasible static comparator of the primary analysis are separate policies, selected on different data (Section K.5).
The phase-only ablation reads the phase from observable event metadata at the segment boundary: the tool name, whether the last tool output contained a traceback or a test summary, the output length, and whether a new file was opened. The model-only, index-only, model-plus-index, and full-state ablations add the feature blocks of Appendix § 13 in that order. The remaining ablations vary the decision interval , replace the greedy nested library with uniform, staggered, or independently searched patterns, sweep the budget ratio , and add feature blocks one at a time with their counted overhead.
The adaptive claim fails if the selected policy satisfies neither decision rule. Every endpoint includes controller and probe overhead, so overhead that removes the saving makes the rule fail. Whether one repository or task family accounts for the difference is a sensitivity analysis, reported with leave-one-repository-out and per-family contrasts, and it adds no rejection criterion beyond the decision rules. If supervised routing matches AIR, an adaptive result can stand while the delayed-credit claim is rejected.
Alternative indexers and cache representations
These methods keep the frozen backbone and change how a selection is computed, what is cached, or which positions are attended. Each runs under the same agent harness, prompts, decoding, limits, and final populations as AIR. Because they change the selector or the attended support, the indexer-only matched-cost rule does not apply to them; it is reserved for comparisons on the shared indexer. We compare them with AIR on end-to-end success, the complete indexing cost including their own state maintenance, full-model compute, model-only and end-to-end latency, index-cache bytes, and peak memory. Prefill and decode are measured separately for every method, and any context length at which a method starts to pay off comes from our own profiling at the 32K and 128K caps on our hardware and backbones.
Every adaptation passes a qualification test on the primary model. Each reimplementation reproduces the published algorithm’s selections on reference inputs, or, where the published method runs on a different selector, the documented relation to our flat indexer (for example, exact equality of selected sets below a length threshold). All calibration uses fitting data only.
Training-aware IndexCache. This comparison tests whether adapting the indexers to one fixed pattern matches a learned allocation over unadapted indexers. Starting from the frozen backbone and the accepted indexer, multi-layer distillation (Equation (14)) adapts one indexer bank per evaluated pattern on indexer-training data [5]. This frozen-backbone variant is ours; the original method also updates the remaining weights with the language-modeling loss. Because it changes indexer parameters, we report it separately from the shared-indexer controls, together with its adaptation compute and the memory of keeping one
bank per pattern. We reimplement the loss on our injected indexer.
LatentIndex. This comparison tests whether sharing a compressed index-key representation across layers, while keeping a separate selection at every layer, gives a better success-to-cost trade-off than refreshing selections only at some layers. For each group of consecutive layers, LatentIndex caches one latent per token computed from the group’s first layer, folds each layer’s key decoder into that layer’s query, and keeps layer-specific head weights and top- selection [42]. Its training-free variant fits the latent maps in closed form by reduced-rank ridge regression onto each layer’s native indexer keys, and its hierarchical variant lets the group’s first layer keep a candidate set from which the other layers select. Our backbones have no native indexer, so the regression targets are the index keys of our accepted injected indexer, and calibration uses our fitting data; the paper calibrates on general-text corpora. Groups are formed over consecutive eligible layers; for Gemma, the five global layers form one group of four and one of one. Because the method changes the indexer representation, we compare success, complete indexing cost, latency, index-cache bytes, and peak memory. Our reimplementation reproduces the paper’s closed-form fit and its layer-specific selections on reference inputs.
Head routing (MISA). This comparison tests whether activating fewer indexer heads per query reduces the cost of each selection enough to rival refreshing fewer layers. MISA scores each indexer head with block-pooled index keys and keeps only the highest-scoring heads per query [58]. Our indexer has heads, so we preregister active-head settings , with eight active heads as the control that reproduces the flat indexer. The router’s pooling and scoring and the maintenance of pooled keys are counted. Our implementation follows the paper’s router, which scores each head by its gate-weighted mean affinity to block-pooled index keys (Eq. 7 of the paper), in both prefill and decode, and it keeps pooled keys as a persistent per-block cache; the released decode path differs from the paper (Appendix § 20.§ 6). Savings reported for native DSA indexers were measured with 32 or 64 heads, so we measure the savings for our eight-head indexer directly.
Hierarchical selection (HISA). This comparison measures how much a cheaper block-then-token search changes success and cost by itself. HISA ranks blocks of mean-pooled index keys, keeps a fixed number of blocks, always including the first and last, and runs token-level scoring only inside them [48]. With and the paper’s defaults (block size 128, 64 kept blocks), the result is dense for , equals flat selection for , and approximates it beyond. We follow version 4 of the paper, which describes HISA as approximating DSA and reports per-layer overlap with flat selection of about 93–95% that falls to about 79% near 18K positions, and we report selection overlap with the flat indexer by context length. The pooled block keys are a persistent per-layer cache appended block by block, and their maintenance is counted. We reimplement HISA over our indexer’s per-head keys, and the reimplementation reproduces flat selection exactly for .
Similarity-guided temporal reuse (ReTopK). This comparison tests whether reusing supports across decoding steps when queries are similar can match AIR’s trade-off, which reuses supports only across depth. ReTopK keeps, for each attention head, a small FIFO cache of recent normalized queries and their selected supports. A new query retrieves its most similar cached queries, merges their supports with the most recent tokens, and reranks the union with exact query-key scores; it falls back to exact top- when the best similarity is below a threshold, and it refreshes exactly every fixed number of decode steps [53]. We run two versions. The published version works per attention head on the backbone’s own query-key scores, as in the paper. The adapted version replaces exact query-key scoring with our injected indexer so that it shares AIR’s selector, and it is labelled as an adaptation. Both keep candidate reranking, the similarity fallback, and the periodic refresh, with the paper’s default settings fixed before any run. The method is defined for decoding. During prefill of a tool output, both ReTopK versions run exact selection, and the newly appended positions enter later supports through the recent window, reranking, and refresh; AIR and the other schedules keep their own prefill execution, in which the segment’s mask decides which layers select. The query-support cache stays valid while the prefix is append-only, because cached supports refer to fixed positions; when the frozen cache-rebuild rule changes or truncates the prefix, the cache is cleared and reseeded. Costs include similarity lookup, cache storage, candidate scoring, reranking, fallbacks, and periodic refreshes. We reimplement both versions and qualify them before use.
LongCat Sparse Attention components. LongCat Sparse Attention combines three components that require different amounts of training [55]. Cross-layer indexing shares one layer’s indexing result with the next layers of a fixed group and relies on cross-layer distillation of the indexer. As a targeted comparison, we apply it to our injected indexer with fixed groups of two eligible layers and distill the indexer while keeping the backbone frozen; this tests whether a uniform, trained sharing structure matches learned allocation over a frozen indexer. Hierarchical indexing is training-free: it ranks pages of tokens by sub-block mean keys before token-level scoring, and it is evaluated alone and in composition with AIR where its page structure is compatible with our cache layout. Streaming-aware indexing changes the attended support to a fixed sink and sliding window plus selected middle positions and needs training of the model; it belongs to the full method, which we treat as a retrained-backbone comparison (Appendix § 20.§ 4). Public code targets LongCat’s own DSA indexer, so on our backbones every component runs through an adapter to our injected indexer, and we implement the distillation and hierarchical indexing ourselves (Appendix § 20.§ 6).
Other adapted methods. HShare shares critical-token indices across layers, heads, and consecutive decoding steps with a configuration chosen per input batch after prefill [46]; Slow-Fast Inference refreshes a reused token set near sentence boundaries and after a fixed refresh budget [47]; TidalDecode selects tokens with exact scores at a few fixed layers [51]; Quest selects key-value pages by query-aware bounds [38]; and Kascade reuses exact top- indices from anchor layers chosen on a development set [12]. These methods select with the backbone’s own attention scores, and each runs in its published form where its released code supports the backbone, with an adapter otherwise (Appendix § 20.§ 6). Kascade’s anchor selection and head mapping are recomputed on our fitting data, and HShare’s sharing ratio and SFI’s maximum refresh interval are fixed before final evaluation.
Composition experiments
The composition experiments test whether learned refresh allocation remains useful after each selection has become cheaper. The main composition crosses two selectors, the flat injected indexer and HISA, with two schedulers, the strongest feasible static schedule and AIR. For each selector we also run all-refresh execution as a reference, so the experiment has six arms per backbone and population. The final number of attended positions, plus the sink and local positions, is the same in every arm. Each selector receives its own static library, calibrated on fitting data with its own costs, and its own AIR controller, trained and selected under that selector’s cost model; the HISA controller therefore learns how much a refresh costs when selection is hierarchical. AIR with HISA is our proposed combination, and neither paper reports it. Compositions of AIR with MISA head routing and with LongCat hierarchical indexing follow the same design where the qualification tests pass.
Architectural and retrained-backbone comparisons
Some methods cannot run on a frozen checkpoint. Full LongCat Sparse Attention combines its three components with a changed attended support and requires training of the model [55]. We treat it as a retrained-backbone comparison on the primary model only, run after continual training whose data, compute, and changes to the attended support are disclosed, and compare it on end-to-end success and resource use. Architectures that share key-value tensors across layers or are pretrained with sparse attention, such as Cross-Layer Attention, YOCO, You Only Index Once, HySparse and HySparse2, and MiniMax Sparse Attention [8, 36, 37, 43, 22] (Gao et al. 2026), require different pretrained checkpoints and are discussed as related work only.
Qualification and extension to other backbones
Every method in this appendix is first qualified on Qwen3-4B-Instruct-2507 with fitting data from repository repair, terminal, and tool-use tasks, the latter two from the fitting source of Section K.2, which is disjoint from the transfer populations. A fixed rule then decides whether the method extends to the other backbones, and the method is evaluated on SWE-bench Verified, Terminal-Bench 2.0, and the three domains. Shared-indexer controls run on every backbone whose sparse-attention check passes. Methods that operate inside our injected indexer (training-aware IndexCache, LatentIndex, head routing, HISA, and the LongCat indexing components) extend to every such backbone, on Gemma restricted to its global layers, and on MiniMax-M2 within its compute allocation. Methods that score with the backbone’s own attention (ReTopK in its published form, HShare, Slow-Fast Inference, TidalDecode, Quest, and Kascade) extend to Qwen3-1.7B and Qwen3-8B, which share the primary model’s attention layout, and to Gemma and MiniMax-M2 through adapters for their attention modules. Full LongCat Sparse Attention runs on the primary model only.
Source versions and code availability
Table 16 records, for each alternative method, the source version, the code we start from, and whether we reproduce the published method or adapt it; Table 17 records its calibration or training, its persistent state and invalidation rule, and the extra cost it is charged. Where released code is missing, covers only part of a method, or differs from the paper, the details are as follows.
| Method | Source | Code we start from | Reproduction |
| Training-aware IndexCache | arXiv 2603.12201v1 | THUDM/IndexCache 08d22d69 (DSA models only) | Ours, on injected indexer |
| LatentIndex | arXiv 2610.04635v1 | None in the linked TransArch repository | Ours; targets are our indexer’s keys |
| MISA (head routing) | arXiv 2605.07363v1 | TransArch, directory MISA_preprint (decode router differs from paper) | Ours, paper’s router |
| HISA | arXiv 2603.28458v4 | TileLang example dsa_hisa; SGLang branch hisa_pr (DSA models) | Ours, on injected indexer |
| ReTopK | arXiv 2607.27692v1 | No public code | Published per-head version and indexer-based adaptation, both ours |
| LongCat cross-layer indexing | arXiv 2608.01662v2 | Inference reuse in SGLang main 9ddbba5; distillation code unpublished | Adapter to our indexer and distillation, ours |
| LongCat hierarchical indexing | arXiv 2608.01662v2 | No public code | Ours |
| HShare | ICLR 2025 | wuhuaijin/HShare 523bea0c (Llama, Mistral) | Adapter for Qwen3 |
| Slow-Fast Inference | arXiv 2603.12038 | LV-NUS/SFI 1415706f (vLLM, Qwen3) | Published code on Qwen3 |
| TidalDecode | ICLR 2025 | DerrickYLJ/TidalDecode 81878af2 (Llama-3) | Adapter for Qwen3 |
| Quest | ICML 2024 | mit-han-lab/Quest (Llama family) | Adapter for Qwen3 |
| Kascade | arXiv 2512.16391v2 | microsoft/kascade d4463fcb (Qwen3-8B mappings) | Published code; mappings refit |
Table 16. Alternative-method documentation. “Ours” marks a reimplementation or adaptation. Table 17 continues with calibration, persistent state, and counted cost.
| Method | Calibration or training | Persistent state and invalidation | Extra cost counted |
| Training-aware IndexCache | Multi-layer distillation per pattern on indexer-training data | One indexer bank per pattern; caches as flat indexer | Adaptation compute; bank memory |
| LatentIndex | Closed-form reduced-rank ridge fit on fitting data | One latent per token per layer group; append-only | Latent projections, latent cache, per-layer selection, candidate rescoring (hierarchical) |
| MISA (head routing) | None; active heads preregistered | Block-pooled keys, appended per block | Router scoring and pooled-key maintenance |
| HISA | None | Block-pooled keys, appended per block | Block scoring, pooled-key maintenance |
| ReTopK | None; defaults fixed in advance | Per-head FIFO of queries and supports; cleared on prefix change | Similarity lookup, cache storage, candidate scoring, fallbacks, refreshes |
| LongCat cross-layer indexing | Cross-layer distillation of the indexer | As flat indexer | Distillation compute |
| LongCat hierarchical indexing | None | Page and sub-block mean keys, appended | Page scoring and pooled-key maintenance |
| HShare | Sharing configuration per input batch after prefill; ratio fixed in advance | Shared critical-token indices across layers, heads, steps; full KV kept | Configuration search, approximate scoring |
| Slow-Fast Inference | None; interval fixed in advance | Reused token set refreshed at boundaries | Dense refresh steps, support construction |
| TidalDecode | Selection layers fixed on fitting data | Token set reused across later layers per step | Exact scoring at selection layers |
| Quest | None | Per-page min/max key metadata | Page bounds and scoring |
| Kascade | Anchors and head mapping on fitting data | Anchor indices reused across layers | Exact anchor attention |
Table 17. Alternative-method documentation, continued from Table 16: calibration or training, persistent state and its invalidation, and the extra cost each method is charged.
Training-aware IndexCache. The released code patches SGLang and vLLM for natively sparse DSA models only [40], so the multilayer loss is our reimplementation.
LatentIndex. The linked MuLabPKU/TransArch repository contains no LatentIndex code, so the method is our reimplementation.
MISA. In the released vLLM patch (TransArch, directory MISA_preprint, commit ec9e6cb), prefill recomputes block means on every call, and the one-stage decode path has pooled routing commented out and routes heads by the current token’s own index key. We use the released code only to check prefill behavior.
HISA. The paper’s claim changed across revisions: version 1 stated that HISA preserves the identical token-level top- pattern, whereas version 4 states that it preserves the top- index interface; the arXiv abstract page still carries the earlier wording. The released kernels and serving integration target DSA models [48].
ReTopK. The paper gives no code link. LongCat Sparse Attention. Public code covers the components unevenly across the released model meituan-longcat/LongCat-Flash-Lite-Sparse (revision 35de83a), SGLang pull request #32918, and SGLang main at 9ddbba5. For model inference, SGLang main contains the LongCat sparse-attention paths, and pull request #32918 adds the fixes the Lite-Sparse checkpoint needs; we implement the three components on our injected indexer. For streaming-aware indexing, SGLang main contains the selection rule that forces the first 16 and last 1,024 positions into the selected set; we implement the fused sparse and sliding-window attention kernel and the training code ourselves. For cross-layer indexing, SGLang main contains inference-time reuse (every second layer recomputes, set by cli_factor=2), and we implement the cross-layer distillation and training ourselves. We implement hierarchical indexing from the paper’s description. All of this code targets LongCat’s own DSA indexer.
Native-score methods. Released code for Slow-Fast Inference supports the Qwen3 family, including Qwen3-4B-Instruct-2507, and released code for Kascade provides head mappings for Qwen3-8B; the released code for HShare targets Llama and Mistral checkpoints, and that for TidalDecode and Quest targets Llama-family checkpoints; each needs an adapter for Qwen3’s attention modules [24, 26, 18, 41].
Agent Runtime, Logging, and Random Streams
Schedules can be compared only if every arm faces the same task under the same conditions and leaves a record that can be replayed. The runtime, the logged record, and the random streams below are shared by every method.
Environment and agent loop. Each task starts from a versioned environment image and a fixed initial state. The agent follows a ReAct-style loop that interleaves reasoning with tool actions [52]. Coding tasks expose file search, file read, patch editing, shell execution, and test tools inside a versioned sandbox; terminal tasks expose the benchmark’s shell interface; tool-use tasks expose only the benchmark’s domain APIs and policy documents. Every method receives the same system prompt, task prompt, tool schema, context policy, decoding rule, token limits, tool-step limit, wall-time limit, and environment seed; model decoding follows each backbone’s fixed rule (Appendix § 19.§ 1). In -bench, a simulated user with a fixed checkpoint and configuration converses with the agent; in telecom the user also acts through its own tools on a shared environment, and the default dual-control setting is kept for every method [6].
Cache reuse at decision boundaries. At a new model call, the runtime checks the prefix hash against the existing core KV and indexer-key caches. A matching append-only prefix keeps both caches. The runtime prefills only the newly appended system, user, or tool-observation tokens under the newly selected mask and then decodes the first tokens under the same mask. A -token boundary inside a response keeps both caches and selects a new mask for the later decode queries. A changed or truncated prefix follows one frozen cache-rebuild rule for every method, and resets for every query position.
Logging and replay. Every run records the model revision and weight hash, indexer hash, prompt hash, tool-schema hash, benchmark revision, container digest, decoding configuration, controller seed, timeout, and cost-profiler revision in a run configuration record that is frozen before controller fitting, so replay can separate model-side schedule changes from changes in the task runtime. Every model call logs the hash of the full causal prefix, the incoming cache version, the segment boundary, the chosen budget, the exact layer mask, the selected-index lineage, the generated tokens, the parsed tool action, the environment response, the verifier output, the controller counter, and the per-segment cost breakdown. Hidden tests and future environment state never enter controller features. A replay tool reproduces model-side computation from the prefix hash and checkpoint hashes.
Random streams and pairing. Four random streams are kept separate and recorded: controller-training seeds, which index the training runs; controller-evaluation seeds, which drive the deployed controller’s sampling through a counter-based stream keyed by task, trial, and decision index; agent-sampling seeds, used only by checkpoints that sample; and user-simulator seeds in . Because budget sampling has its own stream, it leaves model decoding unchanged. Paired com-
parisons share task order, decoding configuration, environment seed, timeout policy, and step limit. The trajectory depends on actions, so generated text generally differs across methods, and common random numbers align only the exogenous streams. The final analysis treats each task as the paired unit.
Reproducibility and Leakage Tests
Three failure modes could produce an apparent benefit of adaptive allocation without one existing: an execution error that makes schedules differ in ways the cost model does not see, a feature that leaks information from after the decision, and overlap between fitting and evaluation data. The tests below address each in turn.
Execution tests. Deterministic unit tests check incremental decoding against full-prefix evaluation under every schedule, as well as absolute offsets, short-prefix selection with fewer than candidate positions, causal masks, index-buffer overwrite, and Shared-to-Full switches. A switch test confirms byte-identical indexer keys under continuous execution and frozen-prefix replay. Candidate actions at one state share the same prefix and differ only in the selected schedule.
Feature leakage tests. Feature tests perturb future observations, terminal reward, the candidate action, the repository path, and the task identity. Valid pre-action features must stay unchanged under future or candidate-action perturbations, and identity perturbations must not reveal an owner or repository through raw paths or text keys.
Split audits and artifact hashes. Split audits require zero split-group overlap across indexer training, schedule calibration, static selection, controller training, controller validation, and final evaluation, and zero repository or owner overlap between the repository-repair fitting pool and SWE-bench Verified. Artifact hashes cover the backbone, indexer, schedule library, feature definitions, policy weights, dual variable, prompts, tool schemas, benchmark version, and analysis code. The final evaluator writes one immutable result after all shards pass the population audit.
Extended Limitations
These caveats expand the Limitations section without adding results.
Injected sparse attention. The indexer is injected into dense pretrained backbones that were never trained with sparse attention, so the sparse attention may lose quality that a natively sparse model would keep; a native sparse checkpoint with production kernels would give stronger systems evidence.
Expressiveness and cost. Index keys at every eligible layer let any layer be promoted and cost memory that static IndexCache avoids, which can reduce the feasible batch size at long contexts; the specialized static runtime omits that memory. The finite library keeps every mask valid by construction but cannot reach patterns outside it.
Learning. The expected-cost constraint lets individual episodes exceed the budget, while safety-sensitive serving may require hard per-request limits. Terminal reward has high variance; value baselines, the supervised warm start, and grouped batches reduce it, and the method uses no reward shaping. The controller reads tool-interface metadata, which ties the policy to one tool interface; the restricted-state ablations measure this dependence.
Evaluation. Each setting reports one deployed policy, and variation across the three controller-training seeds enters only through validation selection. Populations of 50 to 114 tasks limit the precision of the and Terminal-Bench 2.0 comparisons, continuations cover only the snapshots that reach each occupancy band, and in-domain results cover held-out subsets of 20 to 40 tasks. SWE-bench Verified concentrates nearly half of its instances in one repository, so repository-level conclusions rest on few clusters, and the MBPP-derived development suite is too short for the long-context conditions. Each backbone has its own indexer, library, normalization, and controller, so cross-backbone differences mix the injected sparse attention with allocation. Qwen3-8B reaches 128K only through a context extension, MiniMax-M2 runs through an adapter with FP8 weights, and Gemma-3-4B-it offers only five eligible layers. Results use fixed benchmark releases and one user simulator. Serving-side memory tiering, such as the expert cache and tiered KV storage of Strata [28], lies outside the protocol; combining it with AIR would require measuring where the persistent index-key cache lives.
Complete Results
These tables and figures hold the evidence behind § 5; each subsection follows the matching results subsection. Unless a caption says otherwise, displays use Qwen3-4B-Instruct-2507 at the 128K cap. Success is the percentage of attempts solved, with the four trials of each task averaged; time is mean end-to-end time per attempt, counting model, tool, environment, and user-simulator time for failed and successful attempts alike; speedup divides the all-refresh mean time by the method’s mean time on the same backbone, cap, and population; and successful tasks per hour is 3,600 times all successful attempts divided by all elapsed seconds. The comparator is the analysis comparator of Appendix § 18.§ 5, reselected for each backbone, cap, and population on the final sample. Displays report point estimates for one deployed policy per setting. Heatmap scales are centered at 0 for differences and at 1 for ratios, with blue for gains and red for losses. Training curves show the three seeds (11, 29, 47) as faint lines and seed means, smoothed with a causal exponential moving average (), as solid lines. In tables, bold marks the best value in the stated comparison set (ties at the reported precision included), a tint marks AIR-based rows, and gray marks unranked references.
Primary model at 128K
Validation deploys the learned controller of seed 47 after 12,288 rollouts (Table 19). Its normalized indexing and control cost is 0.524, against 0.502 for the comparator ( in the specialized runtime); both lie within the tolerance , and the controller itself accounts for 0.023 of AIR’s cost. On SWE-bench Verified, the predeclared test of rule (i) in Equation (16) supports higher success at matched cost (Table 18). The development reference is also on this backbone and cap (Table 11), so the secondary contrast against it has the same values. Paired by task and trial, AIR solves attempts that the static comparator fails and spends a smaller share of its time on failed attempts; against the other three comparators of Figure 15, these shares differ by at most 2.4 points, and on airline all three spend less than AIR. Raising the budget from to 0.70 raises success on four benchmarks, leaves SWE-bench Verified at 32.4%, and lengthens every episode by 1.8 to 4.2%; lowering it to 0.35 costs 0.4 to 2.6 points. At each budget, AIR reaches higher success than static schedules with similar time per attempt (Figure 16).
Table 18: AIR’s success gain over the analysis comparator is positive on every population at a cost difference of at most 0.022. : success difference (pp); : normalized cost difference; differences are calculated before rounding.
| Population | Tasks | Comparator | (pp) | |
| SWE-bench Verified | 500 | +5.4 | +0.022 | |
| Terminal-Bench 2.0 | 89 | +2.2 | +0.020 | |
| retail | 114 | +3.9 | +0.017 | |
| airline | 50 | +4.0 | +0.013 | |
| telecom | 114 | +3.9 | +0.017 |
Table 18.
| Backbone | Cap | Deployed type | Candidate | Success LB % | Cost UB |
| Qwen3-1.7B | 32K | Learned | s47_n12288 | 6.2 | 0.477 |
| Qwen3-4B-Instruct-2507 | 32K | Learned | s11_n12288 | 22.0 | 0.481 |
| Qwen3-4B-Instruct-2507 | 128K | Learned | s47_n12288 | 22.6 | 0.473 |
| Qwen3-8B | 32K | Learned | s11_n12288 | 27.2 | 0.476 |
| Qwen3-8B | 128K | Learned | s11_n6144 | 29.0 | 0.488 |
| Gemma-3-4B-it | 32K | Static fallback | static_r2 | 5.7 | 0.412 |
| Gemma-3-4B-it | 128K | Learned | s11_n12288 | 14.2 | 0.503 |
| MiniMax-M2 | 32K | Learned | s11_n12288 | 55.9 | 0.490 |
| MiniMax-M2 | 128K | Learned | s47_n12288 | 55.4 | 0.477 |
| Qwen3-4B (original) | 32K | Learned | s47_n9216 | 17.9 | 0.493 |
Table 19. Validation deploys a learned controller in nine of ten configurations and the static fallback (shaded) for Gemma-3-4B-it at 32K. Feasible: cost upper bound at most 0.53; candidate names give seed and rollout count.
Figure 15. Paired by task and trial, AIR solves attempts that the strongest static schedule fails and spends a smaller share of its time on failed attempts. Top: outcome shares against each comparator; bottom: share of time spent on failed attempts.
Figure 16. At similar time per attempt, AIR reaches higher success than static schedules on all five benchmarks (Qwen3-4B-Instruct-2507, 128K). Blue: static , comparator open; red: AIR at ; dashed: all-refresh; arrows: AIR minus the static comparator at the deployed budget (pp).
Across all methods (Table 20, Figure 17), AIR gains 0.2 to 1.1 points over all-refresh on SWE-bench Verified, retail, and telecom with speedups of 1.05 to 1.11, and loses 2.2 and 2.0 points on Terminal-Bench 2.0 and airline; on SWE-bench Verified only AIR and LatentIndex are both faster than all-refresh and at least as successful. HISA and LongCat hierarchical settings are the fastest on SWE-bench Verified and Terminal-Bench 2.0, one-head MISA and HISA with AIR are among the fastest on , and native attention is the slowest everywhere. Across five workload strata, defined by quintiles of the all-refresh episode’s tool-call count, AIR’s ratio of mean times stays at 1.03 to 1.05 on SWE-bench Verified and Terminal-Bench 2.0 and 1.09 to 1.12 on (Figure 18). Prefill accounts for about 80% of model time for every method, so selection work during prefill drives most of the index cost.
| Method | SWE | Terminal | Retail | Airline | Telecom |
| References | |||||
| Native attention | 22.9 (0.78) | 18.9 (0.77) | 12.0 (0.70) | 9.6 (0.70) | 12.9 (0.71) |
| All-refresh sparse | 17.9 (1.00) | 14.6 (1.00) | 8.3 (1.00) | 6.7 (1.00) | 9.2 (1.00) |
| Strongest feasible static | 17.4 (1.03) | 13.9 (1.05) | 7.5 (1.11) | 6.0 (1.11) | 8.3 (1.10) |
| Alternative injected selectors | |||||
| Training-aware IndexCache | 17.1 (1.05) | 13.9 (1.05) | 7.5 (1.12) | 6.0 (1.12) | 8.3 (1.11) |
| LatentIndex | 17.4 (1.03) | 14.1 (1.03) | 7.9 (1.05) | 6.3 (1.06) | 8.7 (1.05) |
| LatentIndex + HS | 17.2 (1.05) | 13.7 (1.06) | 7.5 (1.12) | 6.0 (1.12) | 8.3 (1.11) |
| MISA, 1 head | 17.3 (1.04) | 14.0 (1.04) | 7.2 (1.16) | 5.7 (1.17) | 8.0 (1.15) |
| MISA, 2 heads | 17.3 (1.04) | 13.9 (1.05) | 7.3 (1.14) | 5.8 (1.14) | 8.1 (1.13) |
| MISA, 4 heads | 17.4 (1.03) | 14.0 (1.04) | 7.7 (1.09) | 6.1 (1.09) | 8.5 (1.08) |
| MISA, 8 heads | 18.0 (1.00) | 14.6 (1.00) | 8.4 (1.00) | 6.7 (1.00) | 9.2 (1.00) |
| HISA, all refresh | 16.9 (1.06) | 13.7 (1.07) | 7.5 (1.11) | 6.0 (1.11) | 8.3 (1.10) |
| HISA, static schedule | 16.6 (1.08) | 13.7 (1.07) | 7.5 (1.11) | 6.0 (1.11) | 8.3 (1.10) |
| MISA 4, static schedule | 17.1 (1.05) | 14.0 (1.04) | 7.7 (1.09) | 6.1 (1.09) | 8.5 (1.08) |
| LongCat CLI | 16.9 (1.06) | 13.7 (1.07) | 7.4 (1.12) | 6.0 (1.12) | 8.3 (1.11) |
| LongCat HI | 17.3 (1.04) | 13.8 (1.06) | 7.7 (1.09) | 6.1 (1.09) | 8.5 (1.08) |
| LongCat HI, static schedule | 16.9 (1.07) | 13.6 (1.08) | 7.7 (1.09) | 6.1 (1.09) | 8.0 (1.15) |
| ReTopK, indexer adapted | 18.3 (0.98) | 15.0 (0.97) | 8.3 (1.01) | 6.6 (1.01) | 9.1 (1.01) |
| Native-score methods | |||||
| ReTopK, per head | 17.9 (1.00) | 14.2 (1.03) | 7.9 (1.05) | 6.4 (1.05) | 8.7 (1.05) |
| HShare | 17.4 (1.03) | 13.8 (1.05) | 7.4 (1.12) | 6.0 (1.12) | 8.2 (1.12) |
| Slow-Fast Inference | 17.1 (1.05) | 14.0 (1.04) | 7.6 (1.10) | 6.1 (1.10) | 8.4 (1.09) |
| TidalDecode | 17.5 (1.02) | 14.0 (1.04) | 7.4 (1.13) | 5.9 (1.13) | 8.2 (1.12) |
| Quest | 17.0 (1.05) | 13.9 (1.05) | 7.4 (1.12) | 6.0 (1.12) | 8.2 (1.12) |
| Kascade | 17.1 (1.05) | 13.9 (1.05) | 7.3 (1.15) | 5.8 (1.15) | 8.1 (1.13) |
| AIR-based methods | |||||
| HISA + AIR | 17.0 (1.06) | 13.7 (1.07) | 7.2 (1.16) | 5.8 (1.16) | 8.0 (1.15) |
| MISA 4 + AIR | 17.3 (1.04) | 14.1 (1.03) | 7.3 (1.15) | 5.8 (1.15) | 8.1 (1.13) |
| LongCat HI + AIR | 17.1 (1.05) | 13.6 (1.07) | 7.3 (1.15) | 5.8 (1.14) | 8.1 (1.13) |
| AIR | 17.1 (1.05) | 13.9 (1.05) | 7.5 (1.11) | 6.0 (1.11) | 8.3 (1.10) |
Table 20. Mean minutes per attempt (speedup over all-refresh) for every method; bold marks the fastest per column. AIR is 4 to 5% faster than all-refresh on SWE-bench Verified and Terminal-Bench 2.0 and 9 to 10% faster on . Key: bold marks the best value in each column of the stated comparison set, ties at the reported precision included; AIR-based rows; gray: the all-refresh reference.
Figure 17. Few settings are both faster than all-refresh and at least as successful; on SWE-bench Verified only AIR and LatentIndex are. Left: success change (pp); right: speedup over all-refresh.
Figure 18. AIR’s speedup over all-refresh changes little with episode length: across five workload strata it stays at 1.03 to 1.05 on SWE-bench Verified and Terminal-Bench 2.0 and 1.09 to 1.12 on . Ribbon: interquartile range of per-attempt ratios.
Backbones and caps
AIR’s success exceeds the comparator’s in 37 of 50 backbone, cap, and benchmark settings, ties in 7, and falls below it in 6 (Figure 5). Validation selects a learned controller in nine of ten backbone and cap configurations; for Gemma-3-4B-it at 32K no learned candidate meets the cost bound and the static fallback is deployed (Table 19). AIR’s time per attempt ranges from 3.9% longer than the comparator’s (Qwen3-8B at 128K on SWE-bench Verified, a tie in success) to 7.0% shorter (Qwen3-4B-Instruct-2507 at 32K on airline). In successful tasks per hour at 32K, AIR matches or exceeds the comparator in 21 of 25 settings, by up to 23% on Qwen3-8B on Terminal-Bench 2.0; its lower-yield settings are Gemma-3-4B-it on the three domains, where it is the static fallback, and MiniMax-M2 on airline (Figure 19).
Figure 19. At 32K, AIR matches or exceeds the comparator in successful tasks per hour in 21 of 25 cells, by up to 23%. Left: success difference (pp); right: ratio of successful tasks per hour.
Table 21 compares AIR with all-refresh on each backbone. Model-side speedups are larger than whole-agent ones: on SWE-bench Verified, AIR runs the reference prefill (measured on an isolated reference workload) 1.14 to 1.30 times and model-only decoding 1.05 to 1.17 times as fast as all-refresh on every backbone and cap, while its whole-agent speedup there ranges from 0.96 (Gemma-3-4B-it at 32K) to 1.05 (Figure 20). On the success-yield plane at 32K, AIR lies on the boundary of settings that no other displayed setting beats on both axes in none of the five benchmarks for
Figure 20. AIR’s whole-agent speedup on SWE-bench Verified is 0.96 to 1.05, smaller than its prefill and decode gains because tools and the environment take about half of each episode.
Qwen3-1.7B and Gemma-3-4B-it, one for Qwen3-4B-Instruct-2507 and MiniMax-M2, and two for Qwen3-8B (Figure 21 shows Qwen3-8B).
Figure 21. On Qwen3-8B at 32K, AIR lies on the success-yield boundary (dotted: settings no other displayed setting beats on both axes) in two of the five benchmarks. Blue: static menu.
Ablations, decision interval, and the paid probe
Each ingredient of AIR is removed in turn on the primary model (Table 22, Figure 22). Periodic masks come within 1.2 points of the greedy mask with the same on SWE-bench Verified, and seeded staggered masks lose 1.8 to 3.0 points. Across all eight combinations of event, model, and index state, the full state is best on every benchmark, tied on Terminal-Bench 2.0 and airline with model plus index state, and removing all three groups costs 2.2 to 4.5 points (Figure 23).
Figure 22. Every state ablation and both locally trained alternatives lower SWE-bench Verified success relative to AIR, by 1.0 to 4.2 points; on airline five of the six score higher. Left: success difference (pp); right: time change (%), positive is slower.
Figure 23. The full controller state gives the highest success on all five populations (tied on Terminal-Bench 2.0 and airline with model plus index state); removing all three groups costs 2.2 to 4.5 points. Rows: the other seven combinations of event (), model (), and index () state.
| Method | SWE | Terminal | Retail | Airline | Telecom | Speedup |
| All-refresh reference | 31.6 | 27.0 | 57.0 | 48.5 | 45.4 | 1.00 |
| Placement and state-free variation | ||||||
| Strongest feasible static () | 27.0 | 22.5 | 53.3 | 42.5 | 42.5 | 1.03 |
| Staggered mask, | 24.0 | 21.3 | 50.4 | 40.5 | 40.1 | 1.00 |
| State-independent mixture | 27.0 | 22.5 | 52.9 | 42.5 | 42.5 | 1.03 |
| Rules and local learning | ||||||
| Event-triggered rule | 29.4 | 23.6 | 53.3 | 48.0 | 41.7 | 1.04 |
| Lagged-staleness rule | 30.0 | 22.5 | 55.0 | 47.5 | 44.7 | 1.04 |
| Supervised router | 30.4 | 23.6 | 56.6 | 49.5 | 46.1 | 1.04 |
| Contextual bandit | 30.6 | 23.6 | 56.8 | 46.0 | 45.0 | 1.04 |
| Restricted controller state | ||||||
| Event metadata only | 28.2 | 23.6 | 56.6 | 48.0 | 45.0 | 1.02 |
| Model and index statistics | 31.4 | 24.7 | 56.6 | 47.5 | 44.5 | 1.04 |
| AIR budgets ( and unranked) | ||||||
| 29.8 | 23.6 | 56.8 | 45.5 | 44.1 | 1.04 | |
| , deployed | 32.4 | 24.7 | 57.2 | 46.5 | 46.5 | 1.05 |
| 32.4 | 28.1 | 59.0 | 49.0 | 47.4 | 1.02 |
Table 22. Among the variants, AIR has the highest success on SWE-bench Verified, retail, and telecom and ties the best Terminal-Bench 2.0 score; on airline five locally trained or restricted-state variants score higher, the supervised router most. The larger budget is a separate experiment and scores higher on four populations. Success (%); last column: SWE-bench Verified speedup over all-refresh. Key: bold marks the best value in each column of the stated comparison set, ties at the reported precision included; AIR-based rows; gray: reference, not ranked; other budgets are not ranked.
The factorial study fits 96 controllers on SWE-bench Verified, crossing three budgets, four decision intervals, four mask libraries, and warm start on or off (Figure 24). The budget has the largest effect: configurations at average 30.2% success against 26.2% at , and adds no success (30.1%) while taking 32 seconds longer per attempt. Mean success falls from 30.1% at to 27.2% with call-only decisions, and the greedy and independently searched libraries beat uniform and staggered ones. The supervised warm start raises success in all 48 matched pairs, by 0.6 to 2.6 points, and shortens time per attempt by 1.0 to 1.9%; across matched cells, interactions between factors lie between −1.5 and +1.1 points of success and between −7.4 and +12.9 seconds per attempt.
The paid probe raises or matches success at every decision interval, by up to 2.3 points, and adds 1 to 5 seconds per attempt; with the probe off, call-only decisions give the lowest or tied-lowest success on every benchmark (Figure 25). The probe adds about 0.02 to index and control cost, and 0.03 at , where the probe-on candidates exceed the cost tolerance; its cost includes embeddings, native-prefix processing, and indexer work, and is largest on Gemma-3-4B-it (Table 23). Stochastic deployment scores higher than argmax deployment in 36 of 40 matched settings and ties in the rest, by 1.0 point on average; the deployed controller samples with the probe off at . Removing the controller’s cost shortens episodes by 0.3 to 0.6%, and making it eight times larger lengthens them by 2.4 to 3.9% (Figure 26).
Figure 25. The paid probe raises or matches success at every decision interval, by up to 2.3 points, and adds 1 to 5 seconds per attempt. Curves join , 512, 2048, and call-only decisions with the probe off (red) or on (cyan); open markers fail the cost check.
Figure 26. Controller cost barely moves end-to-end time: removing it shortens episodes by 0.3 to 0.6%, and an eight-fold cost lengthens them by 2.4 to 3.9%. Left: time change; right: normalized index and control cost.
| Backbone | Cap | Success change (pp) | Added cost | Time change (%) |
| Qwen3-1.7B | 32K | 0.0 to +2.0 | 0.019 to 0.031 | 0.0 to +0.4 |
| Qwen3-4B-Instruct-2507 | 32K | +0.4 to +2.0 | 0.013 to 0.024 | 0.0 to +0.4 |
| Qwen3-4B-Instruct-2507 | 128K | +0.7 to +2.2 | 0.014 to 0.024 | +0.1 to +0.6 |
| Qwen3-8B | 32K | 0.0 to +1.5 | 0.014 to 0.024 | 0.0 to +0.4 |
| Qwen3-8B | 128K | +1.0 to +1.6 | 0.014 to 0.024 | +0.3 to +0.6 |
| Gemma-3-4B-it | 32K | +2.0 to +3.9 | 0.217 to 0.352 | -1.5 to +3.6 |
| Gemma-3-4B-it | 128K | 0.0 to +2.0 | 0.139 to 0.225 | +1.3 to +3.0 |
| MiniMax-M2 | 32K | +0.7 to +2.2 | 0.009 to 0.015 | 0.0 to +0.4 |
| MiniMax-M2 | 128K | 0.0 to +1.8 | 0.008 to 0.015 | +0.1 to +0.5 |
Table 23. At the paid probe raises or matches success everywhere; its cost is small except on Gemma-3-4B-it, whose probe runs five native layers. Entries: ranges over populations of probe-on minus probe-off.
Where the time goes
Keeping a static mask dynamic-ready, with index keys at every eligible layer, leaves success unchanged and costs up to 2.3% more time, up to 5.6% lower decode throughput, and up to 12 times the index-cache memory at small refresh counts; at the specialized and dynamic-ready run-
| Backbone | Caps | Speedup | Success change (pp) |
| Qwen3-4B-Instruct-2507 | 32K, 128K | 0.99–1.11 | −2.2 to +1.1 |
| Qwen3-8B | 32K, 128K‡ | 1.00–1.13 | −1.5 to +0.9 |
| Gemma-3-4B-it | 32K†, 128K | 0.96–1.03 | −5.0 to −1.3 |
| MiniMax-M2 | 32K, 128K | 1.01–1.12 | −2.6 to +1.1 |
| Qwen3-4B (original) | 32K | 0.99–1.08 | −2.0 to +0.2 |
Table 21. AIR relative to all-refresh on each backbone: ranges of whole-agent speedup and success change over all evaluated caps and benchmarks. † static fallback ; ‡ YaRN extension.
times coincide (Figure 27). AIR finishes faster than all-refresh on most attempts, with median paired speedups of 1.06 on SWE-bench Verified, 1.05 on Terminal-Bench 2.0, and 1.10 to 1.11 on ; the indexer adaptation of ReTopK is the one method slower than all-refresh on most SWE-bench Verified and Terminal-Bench 2.0 attempts (Figure 28). In a prefix study that holds each workload fixed over eleven prefix lengths, with the 4,096-token generation reserve kept, AIR’s speedup over all-refresh rises from 0.97 to 1.00 at 4,096 tokens to 1.20 to 1.33 at 126,976 tokens, most on , where model work dominates the episode; on SWE-bench Verified it rises from 0.98 to 1.20 (Figure 29). Selection work grows with the prefix while key maintenance and controller work stay constant per query.
Figure 27. Keeping a static mask dynamic-ready leaves success unchanged and costs up to 2.3% more time, 5.6% lower decode throughput, and 12 times the index-cache memory at small (SWE-bench Verified; blue 32K, red 128K).
Figure 28. AIR shortens most attempts relative to all-refresh: median paired speedups are 1.06 on SWE-bench Verified, 1.05 on Terminal-Bench 2.0, and 1.10 to 1.11 on , and 96 to 100% of attempts finish faster. Curves: distributions of per-attempt time ratios.
Figure 29. The prefix effect holds on all five populations: AIR’s speedup over all-refresh rises from 0.97–1.00 at 4,096 tokens to 1.20–1.33 at 126,976 tokens, most on .
Other selectors, composition, and retrained backbones
No alternative selector dominates AIR on every benchmark (Table 24). On SWE-bench Verified, AIR has the highest success of 24 selector settings; on Terminal-Bench 2.0 and airline, settings with more Full layers or a different selector score up to 2.2 and 2.5 points higher (Figure 30). Routing fewer of the eight indexer heads, in the style of MISA, cuts index cost by about 70% at one head and shortens episodes by 14 to 15%, at a cost of 5.0 to 7.0 points of success (Figure 31). In successful tasks per hour, AIR at leads 17 settings on SWE-bench Verified and ranks second on telecom (Figure 32).
Table 24: Among the frozen-backbone methods shown, AIR has the highest success on SWE-bench Verified, HISA with AIR on telecom, and LongCat CLI on retail, and HISA with AIR is the fastest or tied-fastest on every population; Table 20 lists configurations that are faster on some populations. Gray: native attention, unranked.
| Method | Success (%) ↑ | Speedup over all-refresh ↑ | ||||||||
| SWE | Term. | Retail | Airl. | Tel. | SWE | Term. | Retail | Airl. | Tel. | |
| References | ||||||||||
| Native attention | 32.0 | 28.1 | 57.7 | 50.0 | 46.5 | 0.78 | 0.77 | 0.70 | 0.70 | 0.71 |
| All-refresh sparse | 31.6 | 27.0 | 57.0 | 48.5 | 45.4 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| Strongest feasible static | 27.0 | 22.5 | 53.3 | 42.5 | 42.5 | 1.03 | 1.05 | 1.11 | 1.11 | 1.10 |
| Alternative injected selectors | ||||||||||
| Training-aware IndexCache | 30.4 | 23.6 | 56.4 | 49.0 | 45.0 | 1.05 | 1.05 | 1.12 | 1.12 | 1.11 |
| LatentIndex | 31.6 | 27.0 | 56.4 | 49.5 | 45.2 | 1.03 | 1.03 | 1.05 | 1.06 | 1.05 |
| LatentIndex + HS | 30.0 | 24.7 | 54.6 | 48.5 | 44.5 | 1.05 | 1.06 | 1.12 | 1.12 | 1.11 |
| MISA, 4 heads | 28.8 | 23.6 | 55.3 | 47.5 | 43.6 | 1.03 | 1.04 | 1.09 | 1.09 | 1.08 |
| HISA, all refresh | 30.8 | 25.8 | 56.8 | 48.5 | 44.7 | 1.06 | 1.07 | 1.11 | 1.11 | 1.10 |
| LongCat CLI | 31.4 | 27.0 | 58.6 | 49.0 | 45.0 | 1.06 | 1.07 | 1.12 | 1.12 | 1.11 |
| LongCat HI | 30.0 | 27.0 | 55.7 | 48.0 | 42.5 | 1.04 | 1.06 | 1.09 | 1.09 | 1.08 |
| ReTopK, indexer adapted | 30.4 | 22.5 | 55.3 | 47.5 | 44.5 | 0.98 | 0.97 | 1.01 | 1.01 | 1.01 |
| Native-score methods | ||||||||||
| ReTopK, per head | 29.8 | 27.0 | 56.6 | 47.5 | 46.1 | 1.00 | 1.03 | 1.05 | 1.05 | 1.05 |
| HShare | 28.0 | 23.6 | 55.7 | 46.0 | 44.5 | 1.03 | 1.05 | 1.12 | 1.12 | 1.12 |
| Slow-Fast Inference | 30.8 | 23.6 | 56.4 | 49.0 | 45.0 | 1.05 | 1.04 | 1.10 | 1.10 | 1.09 |
| TidalDecode | 26.8 | 22.5 | 54.4 | 47.0 | 43.9 | 1.02 | 1.04 | 1.13 | 1.13 | 1.12 |
| Quest | 30.4 | 23.6 | 56.6 | 48.5 | 45.6 | 1.05 | 1.05 | 1.12 | 1.12 | 1.12 |
| Kascade | 28.0 | 22.5 | 53.3 | 46.0 | 41.9 | 1.05 | 1.05 | 1.15 | 1.15 | 1.13 |
| AIR-based methods | ||||||||||
| HISA + AIR | 30.6 | 24.7 | 56.1 | 48.5 | 46.9 | 1.06 | 1.07 | 1.16 | 1.16 | 1.15 |
| AIR | 32.4 | 24.7 | 57.2 | 46.5 | 46.5 | 1.05 | 1.05 | 1.11 | 1.11 | 1.10 |
Table 24. Key: bold marks the best value in each column of the stated comparison set, ties at the reported precision included; AIR-based rows; gray: unranked reference.
Figure 30. AIR has the highest success of 24 selector settings on SWE-bench Verified, retail, and telecom; on Terminal-Bench 2.0 and airline other settings score up to 2.2 and 2.5 points higher. Solid: sweeps per selector; dashed: MISA head sweep.
Figure 31. Routing fewer of the eight indexer heads (MISA) cuts index cost by about 70% at one head and costs 5.0 to 7.0 points of success.
Figure 32. AIR at gives the most successful tasks per hour of 17 settings on SWE-bench Verified and ranks second on telecom; other settings yield more on Terminal-Bench 2.0 and airline. Dotted: settings no other setting beats on both axes.
With the flat indexer, AIR scheduling beats the selector’s static schedule on every benchmark by 2.2 to 5.4 points; with HISA, four-head MISA, and LongCat hierarchical indexing the change ranges from −2.5 to +3.4 points (Figure 33). Speedups
Figure 33. Learned allocation helps most with the flat indexer; with cheaper selectors its success gain depends on the population. (a) AIR minus the selector’s static schedule (pp); (b) speedup over the selector’s all-refresh run.
over each selector’s own all-refresh run are 1.05 to 1.11 with the flat selector and up to 1.06 with the others; in 10 of the 15 settings for the three cheaper selectors, the selected static schedule is the selector’s own all-refresh (). Full LongCat Sparse Attention retrains the backbone in a separate track: it costs 180 GPU-hours of adaptation (192 with AIR), changes success by −1.2 to +1.3 points against the unadapted all-refresh model, and needs 6,403 to 12,858 attempts to repay its preparation through faster serving, counting model-serving GPU time only; adding AIR raises its success on four of five benchmarks (Table 25) by 5.0 points on Terminal-Bench 2.0 and airline (20 tasks each) and by 1.3 points on retail and telecom (40 each), with time per attempt within 0.5% (Table 26); the two benchmarks with the largest gain are those on which all-refresh beats the transferred controller. These subset scores come from controllers trained and validated on other tasks of the same benchmark and are reported apart from the full-population transfer results.
Table 25: Full LongCat Sparse Attention keeps success within 0.8 points of the unadapted all-refresh model at 1.08 to 1.17 times its speed, and adding AIR raises success on four of five populations; repaying 180 GPU-hours (192 with AIR) takes 6,403 to 12,858 attempts.
| Population | Success (%) | Speedup | Break-even attempts | ||||
| Unadapted | LongCat | + AIR | LongCat | + AIR | LongCat | + AIR | |
| SWE | 31.6 | 30.8 | 31.8 | 1.09 | 1.07 | 6,403 | 7,290 |
| Terminal | 27.0 | 27.0 | 25.8 | 1.10 | 1.08 | 7,369 | 8,390 |
| Retail | 57.0 | 56.6 | 57.9 | 1.17 | 1.15 | 8,987 | 10,231 |
| Airline | 48.5 | 48.5 | 49.5 | 1.17 | 1.15 | 11,294 | 12,858 |
| Telecom | 45.4 | 45.0 | 46.7 | 1.15 | 1.14 | 8,652 | 9,851 |
Table 25. Key: bold marks the best value in each column of the stated comparison set, ties at the reported precision included; AIR-based rows.
Table 26. In-domain training raises held-out success over the transferred repair controller by 1.3 to 5.0 points at equal time per attempt (minutes). Both controllers run on the same held-out tasks; differences are calculated before rounding.
| Subset | Tasks | Held-out success (%) | Minutes per attempt | ||||
| Transfer | In-domain | (pp) | Transfer | In-domain | |||
| Terminal-Bench 2.0 | 20 | 15.0 | 20.0 | +5.0 | 15.5 | 15.5 | |
| retail | 40 | 56.9 | 58.1 | +1.3 | 7.5 | 7.5 | |
| airline | 20 | 35.0 | 40.0 | +5.0 | 6.5 | 6.5 | |
| telecom | 40 | 44.4 | 45.6 | +1.3 | 9.0 | 9.0 |
Transfer, in-domain training, reliability, and long histories
The repair controller transfers to the other benchmarks with the gains of Table 18. On held-out subsets of the target benchmarks, controllers trained in-domain raise success over the transferred controller
AIR keeps four-trial reliability close to native attention and all-refresh: at 128K its pass1 is 21.1% on retail against 15.8% for the comparator, and the comparator is 2.0 and 2.6 points lower on airline and telecom; on 32K airline the comparator is higher (16.0 against 12.0%) (Table 27). In long-history continuations, every method resumes from the same all-refresh snapshot once the prefix fills a fraction of usable context, with the comparator frozen from the full-episode comparison; a task is eligible at its first boundary
Table 27. AIR’s (all-four success, %) exceeds the comparator’s in five of six cells; on 32K airline the comparator is higher (16.0 against 12.0).
| Method | 32K | 128K | ||||
| Retail | Airline | Telecom | Retail | Airline | Telecom | |
| Native attention | 16.7 | 16.0 | 10.5 | 21.1 | 10.0 | 10.5 |
| All-refresh sparse | 16.7 | 16.0 | 10.5 | 21.1 | 10.0 | 10.5 |
| Strongest feasible static | 13.2 | 16.0 | 8.8 | 15.8 | 8.0 | 7.9 |
| Training-aware IndexCache | 16.7 | 14.0 | 10.5 | 21.1 | 10.0 | 10.5 |
| LatentIndex + HS | 14.0 | 14.0 | 11.4 | 18.4 | 10.0 | 10.5 |
| HISA + AIR | 15.8 | 14.0 | 9.6 | 20.2 | 10.0 | 10.5 |
| AIR | 14.9 | 12.0 | 10.5 | 21.1 | 10.0 | 10.5 |
Key: bold marks the best value in each column of the stated comparison set, ties at the reported precision included; ■ AIR-based rows.
whose prefix lies in , where is the cap minus the 4,096-token generation reserve (Table 28). AIR’s success exceeds the comparator’s in 12 of 15 benchmark and occupancy bands and ties it in 3, all-refresh scores higher in 5 bands, including all three airline bands (Figure 34), and AIR finishes the remaining work 4 to 10% faster than all-refresh in every band (Figure 35). On SWE-bench Verified, AIR solves 31.1, 20.3, and 31.4% of the remaining work at , 0.75, and 0.90, against 25.3, 17.0, and 26.2% for the static comparator and 29.6, 19.9, and 29.7% for all-refresh (Figure 3d).
Table 28. Eligible long-history snapshots per occupancy band; fewer tasks reach the higher bands (SWE-bench Verified: 392, 271, and 172 of 500).
| Population | Tasks | |||
| SWE | 500 | 392 | 271 | 172 |
| Terminal | 89 | 71 | 50 | 32 |
| Retail | 114 | 96 | 75 | 52 |
| Airline | 50 | 41 | 31 | 18 |
| Telecom | 114 | 99 | 77 | 49 |
Figure 34. In long-history continuations, AIR’s success exceeds the comparator’s in 12 of 15 population and occupancy bands and ties it in 3; all-refresh scores higher in 5 bands. Success covers only the continuation from a shared snapshot.
Figure 35. In long-history continuations, AIR finishes the remaining work 4 to 10% faster than all-refresh in every band and within 2.2% of the comparator. Snapshots, bands, and methods as in Figure 34.
Training dynamics
Constrained training behaves the same way on every backbone at 128K (Figure 36). Normalized cost starts at 0.79 and crosses the 0.50 budget after about 3,700 to 4,000 rollouts, as the dual multiplier peaks near 0.12; once the constraint is slack, the multiplier decays toward zero and cost settles at 0.44 to 0.46, while validation success rises by 6.2 to 7.6 points. The primary controller moves most of its probability from (36% at the start, 3% at the end) to and , which together receive about two thirds of its choices, and it still uses and for a third of its decisions. In every seed of the primary controller, action entropy rises to about 1.55 nats in the first 3,000 to 3,700 rollouts and then falls to 1.40 to 1.44, the expected Full-layer fraction falls from 0.57 to 0.31 or 0.32, and seed 29 keeps a positive dual price longest, at 0.04 after the last update (Figure 37).
Figure 36. Training brings every 128K backbone below the 0.50 budget (dashed; dotted: tolerance) within about 4,000 rollouts while validation success rises by 6.2 to 7.6 points. (d) Budget mix of the primary model.
Figure 37. Each seed of the primary controller moves toward the budget while raising validation success, explores before it commits, and keeps a mixed allocation. (a) success against cost, with markers at 0, 6,144, and 12,288 rollouts; (b) action entropy; (c) expected Full-layer fraction; (d) dual updates.
At 32K, five of six backbones reach the budget and their constraint violations fall to zero; Gemma-3-4B-it, with five eligible layers, ends at a cost of 0.59 with a violation of 0.09 and its dual multiplier still rising, at 0.53, which is why validation deploys its static fallback (Figure 38). Seed-mean validation success at 32K rises by 5.6 to 7.2 points and terminal reward by 5.4 to 9.0 points for every backbone over 12,288 rollouts (Figure 39). Every 32K controller still uses at least three refresh options more than 10% of the time at the end of training; the Qwen and MiniMax-M2 controllers put about two thirds of their choices on their third and fourth options, and Gemma-3-4B-it almost all on its three smallest (Figure 40). Controllers trained separately in each of the five fitting domains converge to a cost of 0.44 to 0.46, place about two thirds of their choices on and , and raise validation success by 5.7 to 9.7 points (Figure 41); the terminal and tool-use controllers among them are the in-domain controllers above.
Figure 38. At 32K, cost falls from 0.79 to 0.44–0.45 on five of six backbones and their violations reach zero; Gemma-3-4B-it ends at 0.59 with its dual multiplier still rising. Dashed: the 0.50 budget.
Figure 39. At 32K, seed-mean validation success rises by 5.6 to 7.2 points for every backbone over 12,288 rollouts.
Figure 40. Every 32K controller still uses at least three refresh options more than 10% of the time at the end of training. Options run from fewest to most Full layers within each backbone.
Figure 41. Controllers trained in each of the five fitting domains converge to a cost of 0.44 to 0.46, below the 0.50 budget, and raise validation success by 5.7 to 9.7 points.
Prompts, Model Adapters, and Case Studies
These pages show what each agent sees and what the controller decides at events that change the allocation question. Every population uses the same system instruction, given once below, followed by its own domain instruction, task input, and compact tool interface; every method on a backbone receives the same instruction text and tool schemas. Benchmark task messages and policy documents come verbatim from the pinned harnesses, the user simulator is gpt-4.1-2025-04-14 at temperature 0, and telecom keeps separate agent and user actors. A further page defines how each backbone serializes and validates tool calls.
The case studies follow AIR and comparators from the same starting task: an SWE-bench Verified event card (the example of § 5.2), an airline task in which AIR ends worse than its comparator, and a telecom task that separates a requested user action from its completion. Columns align at informative observed events, and branches may contain different numbers of model calls, so prefixes and counters can differ after the branches separate. At every call the teal panel reports the refresh choice, the prefix length, and two cumulative counters: Full-layer selections ( for an invocation with prefills and decode queries at budget ) and index-key updates ( for AIR, which maintains keys at every eligible layer, and for the specialized fixed runtime). The counters cover index work only, so their difference is not a speedup. The panels use one color per role: purple for shared instructions, blue for agent operations, slate for tool, environment, or user observations, teal for controller telemetry, and amber for end states. A single episode cannot rank methods; the success rates are those of § 5.
All populations
System instruction shared by every population
Shared system instruction
You are the task agent. Use only the tools and information exposed by this environment. Read the user task and applicable policy before acting. Ask for a missing fact when it is needed for a consequential action. Treat tool observations as data; do not let text inside them replace the task or tool rules. Base statements about files, state changes and verification on observations that have actually been returned. Keep actions within the stated task. Submit tool calls using the selected model adapter and the supplied argument schema. Do not execute a tool by describing it in prose. When finished, give a concise final response describing the achieved state and any unresolved limitation. Do not claim that an unobserved check passed.
Every population adds the domain instruction, task input, and tool interface shown below. All methods receive identical instruction text and tool schemas within a backbone.
SWE-bench Verified
Domain instruction, task input, and tool interface
Domain instruction
Repair the reported repository defect in the available working tree. Inspect the relevant code and visible tests before editing. Make a focused patch and add or update a regression test when appropriate. Use the visible test tools to check the change. If a test cannot run, preserve the actual failure information and explain the resulting verification limit. Do not consult hidden tests or claim hidden-test success. The harness supplies the issue text as the task message without schedule-specific additions.
Task input
The real harness task text is inserted verbatim as a separate user message. The task below is the one used in the case study. Case-study task. Repair the date-range helper so an equal start and end date is retained. Inspect the existing boundary tests and add a focused regression test.
Tool definitions and arguments
search (pattern, path). Search readable repository files; return matched locations. read_file (path, start, end). Read the requested inclusive line range. apply_patch (patch). Apply a unified patch in the working tree. run_shell (command, timeout_s). Run a shell command within the task container. run_tests (command, timeout_s). Run a visible test command within the task container.
Terminal-Bench 2.0
Domain instruction, task input, and tool interface
Domain instruction
Work through the supplied shell interface in the task environment. Inspect files, command availability and the current environment before relying on them. Keep the original task objective and allowed output location in view. A long command output is an observation; use targeted inspection when only a small part is needed. Preserve the meaning of command failures, exit status and changed working directories. Report the resulting state and any remaining failure without inventing execution output.
Task input
The real harness task text is inserted verbatim as a separate user message. The task below is the one used in the case study. Case-study task. Build the local wheel into dist/ and confirm that the expected entry point is present in its metadata.
Tool definitions and arguments
shell (command, timeout_s). Run one shell command through the benchmark terminal interface.
retail
Domain instruction, task input, and tool interface
Domain instruction
Help the customer using only the exposed retail APIs and the injected retail policy. Read the relevant account or order information and the policy conditions before a state-changing request. Ask for the identification, choice or confirmation that the policy requires. Do not infer eligibility from a short summary when a tool can provide the applicable rule. State-changing calls require the required explicit consent. Explain a restriction using the policy that is actually visible. Report the result returned by the API.
Task input
The real harness task text is inserted verbatim as a separate user message. The task below is the one used in the case study. Case-study task. I would like to return the headphones in order R-order because they do not fit comfortably.
Tool definitions and arguments
get_order (order_id). Return order details visible to the authenticated customer. get_return_rules (order_id). Return the policy applicable to the specified order. request_return (order_id, item_id, reason, confirmed). Request a return after the relevant policy checks and user confirmation.
airline
Domain instruction, task input, and tool interface
Domain instruction
Help the customer with the existing booking using the exposed airline APIs and injected airline policy. Check current itinerary, fare conditions, availability and the complete quoted cost before requesting a change. Resolve missing preferences with the customer. Explain a newly exposed restriction or fee before asking for confirmation. Commit only the change the customer has explicitly approved and that the applicable policy allows. Describe the returned booking state accurately.
Task input
The real harness task text is inserted verbatim as a separate user message. The task below is the one used in the case study. Case-study task. Please move my B-booking to a later departure on the same date. I need to keep my checked bag.
Tool definitions and arguments
get_booking (booking_id). Return the authenticated customer booking and its fare conditions. search_options (origin, destination, date). Return currently offered itinerary alternatives. quote_change (booking_id, option_id). Return eligibility, price and restrictions for a proposed change. commit_change (booking_id, option_id, confirmed). Submit a quoted change after explicit confirmation.
telecom
Domain instruction, task input, and tool interface
Domain instruction
Help the customer diagnose the service issue under the injected telecom policy and actor-specific tools. The agent controls only its permitted backend APIs. The user controls the permitted device tools. Distinguish a request for a user action from an action already completed. Check the observable shared state after backend or device changes. Obtain required consent before a backend change. Do not assume that a radio reset or provisioning request restored service; ask for or read the allowed follow-up observation.
Task input
The real harness task text is inserted verbatim as a separate user message. The task below is the one used in the case study. Case-study task. Mobile data is not working on line T-line. I can help with checks on my phone.
Tool definitions and arguments
get_line_status (line_id). Read the customer line state visible to the service agent. refresh_provisioning (line_id, confirmed). Request an allowed backend provisioning refresh after confirmation. check_device (check). Inspect one user-device property through the user actor. User actor only. toggle_airplane_mode (seconds). Perform the allowed user-device radio reset. User actor only.
Model adapters and action validation
One instruction set; model-specific serialization
Chat templates
Qwen3. Use the pinned checkpoint template with Hermes-style JSON tool parsing. Hybrid checkpoints have thinking disabled; Instruct-2507 is non-thinking. Tool results enter the template’s user-side response blocks. Qwen decoding is greedy.
Gemma 3. Put schemas in the first user turn. Extract a single fenced Python-style function call and return the tool observation in the next user turn. Preserve the shipped user/model alternation. Gemma decoding is greedy.
MiniMax-M2. Keep the native XML tool and reasoning parsers, interleaved thinking and earlier reasoning blocks in history. Use temperature 1.0, top-p 0.95 and top-k 40. No hidden reasoning is reproduced in these pages.
Error-handling rule
Decode exactly one action envelope. Validate tool availability, actor, required fields, argument types, positive timeout/range fields and explicit confirmation for the state-changing examples. Reject unknown tools, extra fields, duplicate arguments, Python evaluation and calls belonging to another actor.
An invalid action executes zero tools. Return the parser’s short error message as a new observation and allow another response within the existing episode limits. Do not repair the call silently or grant an extra step budget. Environment authorization and policy checks still apply after syntax validation. A final natural-language response is handled by the harness’s final-response path, outside this action parser.
Action-envelope examples
Qwen-style JSON
<tool_call>
{"name":"shell","arguments":
{"command":"pwd","timeout_s":30}}
</tool_call>
Gemma-style fenced call
shell(command="pwd", timeout_s=30)
MiniMax-style XML format
One invoke element names the tool; named parameter elements carry JSON-literal values inside a minimax:tool_call wrapper. The complete valid XML string and parser example are retained in the source.
Parser scope
The included parser validates action envelopes and executes no tools. The JSON and XML envelopes are examples of the two formats; compatibility with each pinned native template is checked during adapter qualification. The MiniMax example defines JSON literals inside XML parameters so its type checks are reproducible. The tool names form a compact interface for these pages, and the benchmarks’ own APIs may name tools differently.
One completed observation, two refresh choices
When should a new visible test diagnostic trigger a higher refresh budget?
Shared task and available history
Repair the date-range helper so an equal start and end date is retained. Inspect the existing boundary tests and add a focused regression test. The same working tree and task text are presented to every method. A boundary edit and visible test request have completed; the test returned a new reversed-range diagnostic.
Event 3: Read the newly relevant branch
Available before decision. The returned test diagnostic includes a traceback/test-result flag about the reversed-range branch.
AIR
read_file path="range_utils.py" start=35 end=65
Controller / selector
; causal event features. Prefix after call: 2,706 tokens. Selections: 22,926; key updates: 97,416.
Fixed IndexCache R9
read_file path="range_utils.py" start=35 end=65
Controller / selector
; fixed mask. Prefix after call: 2,712 tokens. Selections: 24,408; key updates: 24,408.
Returned after the action. The read exposes the newly relevant branch and the code that needs a follow-up revision.
End state and interpretation
AIR. The newly relevant branch is identified for a follow-up revision. Fixed IndexCache R9. The same branch is identified with a constant budget. AIR raises its refresh budget at the model call after the test diagnostic appears, while fixed IndexCache keeps and ReTopK reuses decode selections. All methods identify the same next repair step, so the case shows when each spends index work.
The displayed boundary is Event 3 of the complete SWE gallery. Cumulative counters include its earlier read, edit and test-request invocations.
AIR maintains all 36 eligible-layer keys.

Figure 42. SWE-bench Verified event card for Section 5.2: after a new test diagnostic, AIR raises its budget from to at the next model call, while fixed IndexCache keeps . Counts are cumulative Full-layer selections and index-key updates.
A late constraint can expose a scheduling miss
What does an unfavorable allocation response look like in an event-aligned comparison?
Shared initial task
Please move my B-booking to a later departure on the same date. I need to keep my checked bag. The same booking, user preference and initial policy are available. The booking interface exposes later-option as a candidate whose conditions still need a quote.
Figures 43–45 align the two branches at three events.
Visible final messages
AIR. “I still need to reconcile the checked-bag condition before changing the booking.” Fixed IndexCache R9. “The later fare adds a $35 checked-bag charge. Do you approve that charge before I change the flight?”
End state and interpretation
AIR. The change is still unresolved at the end of the episode.
Fixed IndexCache R9. The necessary customer clarification has been requested; no unconfirmed change is committed.
AIR’s allocation response is unfavorable here: it needs an additional recovery call and leaves the change unresolved, while fixed IndexCache requests the customer clarification first. The case identifies a failure mode to test for; its cause and frequency need the aggregate results.
AIR maintains all 36 eligible-layer keys.
Dual control separates a request from its completion
Does the controller react to observed shared-state changes without treating a requested user action as completed?
Shared initial task
Mobile data is not working on line T-line. I can help with checks on my phone. Agent-side tools and user-device tools remain assigned to their respective actors.
Figures 46–48 align the three methods at three events.
Visible final messages
AIR. “Your device reports mobile data connected, and the line status now shows active provisioning.” Fixed IndexCache R9. “Your device reports mobile data connected, and the line status now shows active provisioning.” ReTopK (indexer-adapted). “Your device reports mobile data connected, and the line status now shows active provisioning.”
End state and interpretation
AIR. The end state records matching backend and user observations.
Fixed IndexCache R9. The same observations are obtained.
ReTopK (indexer-adapted). The same observations are obtained with exact tool-output prefill.
The controller updates its features only after the backend or the user reports the shared state, so a requested user action counts as done only once it is observed. All three methods reach the same observed end state; the case shows actor separation and feature timing.
AIR maintains all 36 eligible-layer keys.

Figure 43. Airline case, Event 1: both methods read the current booking, AIR at and fixed IndexCache at .

Figure 44. Airline case, Event 2: both methods request the same quote, AIR at ; the quote adds a $35 bag charge that needs the customer’s confirmation.

Figure 45. Airline case, Event 3: after the bag condition appears, AIR answers at and needs a second call at , while fixed IndexCache stays at .

Figure 46. Event 1: Read the line state. Available before decision. The customer reports missing mobile data. No backend or device inspection has completed. AIR: get_line_status; line_id="T-line"; Controller / selector; ; causal event features. Prefix after call: 1,306 tokens. Selections: 6,530; key updates: 47,016. Fixed IndexCache R9: get_line_status; line_id="T-line"; Controller / selector; ; fixed mask. Prefix after call: 1,306 tokens. Selections: 11,754; key updates: 11,754. ReTopK (indexer-adapted): get_line_status; line_id="T-line"; Controller / selector; Exact tool-output prefill; decode-selection reuse. Prefix after call: 1,306 tokens. Returned after the action. The line-status response reports stale provisioning. The agent requests permission for a backend refresh. Figure 46: Telecom case, Event 1: all three methods read the line state, which reports stale provisioning; AIR runs at .

Figure 47. Event 2: Apply an authorized backend change. Available before decision. The user has explicitly confirmed the backend refresh. No restoration of connectivity has been observed. AIR: refresh_provisioning; line_id="T-line"; confirmed=true; Controller / selector; ; causal event features. Prefix after call: 1,892 tokens. Selections: 17,078; key updates: 68,112. Fixed IndexCache R9: refresh_provisioning; line_id="T-line"; confirmed=true; Controller / selector; ; fixed mask. Prefix after call: 1,892 tokens. Selections: 17,028; key updates: 17,028. ReTopK (indexer-adapted): refresh_provisioning; line_id="T-line"; confirmed=true; Controller / selector; Exact tool-output prefill; decode-selection reuse. Prefix after call: 1,896 tokens. Returned after the action. The API acknowledges the refresh request; this acknowledgment does not establish device connectivity. Figure 47: Telecom case, Event 2: after the user confirms the backend refresh, AIR raises its budget to ; the API acknowledges the request, which does not restore connectivity.

Figure 48. Event 3: Observe the user-side action. Available before decision. The user performs toggle_airplane_mode(seconds=10), then check_device(check="mobile_data"). The returned user observation says mobile data is connected. These user-side actions are complete before the next agent call. AIR: get_line_status; line_id="T-line"; Controller / selector; ; causal event features. Prefix after call: 2,396 tokens. Selections: 21,614; key updates: 86,256. Fixed IndexCache R9: get_line_status; line_id="T-line"; Controller / selector; ; fixed mask. Prefix after call: 2,396 tokens. Selections: 21,564; key updates: 21,564. ReTopK (indexer-adapted): get_line_status; line_id="T-line"; Controller / selector; Exact tool-output prefill; decode-selection reuse. Prefix after call: 2,404 tokens. Returned after the action. The line-status response now reports active provisioning, consistent with the completed user observation. Figure 48: Telecom case, Event 3: once the user-side actions are observed, all three methods read the line status again, which now shows active provisioning; AIR runs at .
References
- [4]Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. 2021. Program synthesis with large language models. arXiv preprint arXiv:2108.07732.arxiv.org/abs/2108.07732
- [5]Bai, Y., Dong, Q., Jiang, T., Lv, X., Du, Z., Zeng, A., Tang, J., & Li, J. (2026). IndexCache: Accelerating sparse attention via cross-layer index reuse. In arXiv preprint.arXiv
- [6]Barres, V., Dong, H., Ray, S., Si, X., & Narasimhan, K. (2025). $\tau$-bench: Evaluating conversational agents in a dual-control environment.arXiv
- [7]V. S. Borkar. 2005. An actor-critic algorithm for constrained Markov decision processes. Systems & Control Letters, 54(3):207–213.DOI
- [8]Brandon, W., Mishra, M., Nrusimha, A., Panda, R., & Ragan-Kelley, J. (2024). Reducing transformer key-value cache size with cross-layer attention. Advances in Neural Information Processing Systems, 37.arXiv
- [9]Chen, A., Li, A., Zhou, B., Gong, B., Jiang, B., Dan, B., & al., . et . (2026). The MiniMax-M2 series: Mini activations unleashing max real-world intelligence.arXiv
- [10]Chowdhury, N., Aung, J., Shern, C. J., Jaffe, O., Sherburn, D., Starace, G., Mays, E., Dias, R., Aljubeh, M., Glaese, M., Jimenez, C. E., Yang, J., Ho, L., Patwardhan, T., Liu, K., & Madry, A. (2024). Introducing SWE-bench Verified. OpenAI Blog. <a href="https://doi.org/https://openai.com/index/introducing-swe-bench-verified/" rel="noopener noreferrer" target="_blank">https://doi.org/https://openai.com/index/introducing-swe-bench-verified/</a>
- [12]Dhruv Deshmukh, Saurabh Goyal, Nipun Kwatra, and Ramachandran Ramjee. 2025. Kascade: A practical sparse attention method for long-context LLM inference. arXiv preprint arXiv:2512.16391.arxiv.org/abs/2512.16391
- [14]Team, G. (2025). gemma-3-4b-it. Hugging Face model card (gated). <a href="https://doi.org/https://huggingface.co/google/gemma-3-4b-it" rel="noopener noreferrer" target="_blank">https://doi.org/https://huggingface.co/google/gemma-3-4b-it</a>
- [15]Team, G., Kamath, A., Ferret, J., Pathak, S., Vieillard, N., Merhej, R., Perrin, S., Matejovicova, T., Ramé, A., Riviere, M., & al., . et . (2025). Gemma 3 technical report.arXiv
- [16]DeepMind, G. (2026). Gemma: reference implementation. <a href="https://doi.org/https://github.com/google-deepmind/gemma" rel="noopener noreferrer" target="_blank">https://doi.org/https://github.com/google-deepmind/gemma</a>
- [17]Namgyu Ho, Huzama Ahmad, Woosung Koh, Se-Young Yun, Tal Schuster, and Cicero Nogueira dos Santos. 2026. Language models can control their own attention. arXiv preprint arXiv:2609.02737.arxiv.org/abs/2609.02737
- [19]Huiqiang Jiang, Yucheng Li, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Zhenhua Han, Amir H. Abdi, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2024. MInference 1.0: Accelerating pre-filling for long-context LLMs via dynamic sparse attention. In Advances in Neural Information Processing Systems, volume 37, pages 52481–52515.arxiv.org/abs/2407.02490
- [20]Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2024. SWE-bench: Can language models resolve real-world GitHub issues? In International Conference on Learning Representations.arxiv.org/abs/2310.06770
- [21]Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gon, J. E., Zalez, H., & Zhang, . and I. S. (2023). Efficient memory management for large language model serving with PagedAttention. Proceedings of the 29th Symposium on Operating Systems Principles, 611–626.
- [22]Xunhao Lai, Weiqi Xu, Yufeng Yang, Qiaorui Chen, Yang Xu, Lunbin Zeng, Xiaolong Li, Haohai Sun, Haichao Zhu, Vito Zhang, Jinkai Hu, Jiayao Li, Rui Gao, Zekun Li, Songquan Zhu, Jingkai Zhou, and Pengyu Zhao. 2026. MiniMax sparse attention. Preprint, arXiv:2606.13392.arXiv
- [23]Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023. Fast inference from transformers via speculative decoding. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 19274–19286. PMLR.arxiv.org/abs/2211.17192
- [24]LV-NUS. (2026). SFI: Slow-fast inference. <a href="https://doi.org/https://github.com/LV-NUS/SFI" rel="noopener noreferrer" target="_blank">https://doi.org/https://github.com/LV-NUS/SFI</a>
- [25]Merrill, M. A., Shaw, A. G., Carlini, N., Li, B., Raj, H., Bercovich, I., Shi, L., Shin, J. Y., Walshe, T., Buchanan, E. K., & al., . et . (2026). Terminal-Bench: Benchmarking agents on hard, realistic tasks in command line interfaces. International Conference on Learning Representations. 2601.11868
- [26]Microsoft. (2026). Kascade. <a href="https://doi.org/https://github.com/microsoft/kascade" rel="noopener noreferrer" target="_blank">https://doi.org/https://github.com/microsoft/kascade</a>
- [27]MiniMax. (2025). MiniMax-M2. Hugging Face model card. <a href="https://doi.org/https://huggingface.co/MiniMaxAI/MiniMax-M2" rel="noopener noreferrer" target="_blank">https://doi.org/https://huggingface.co/MiniMaxAI/MiniMax-M2</a>
- [28]Niko1221. (2026). Strata: Qwen3.8-Flash-Next on consumer hardware. <a href="https://doi.org/https://github.com/Niko1221/Strata" rel="noopener noreferrer" target="_blank">https://doi.org/https://github.com/Niko1221/Strata</a>
- [29]Jiayi Pan, Xingyao Wang, Graham Neubig, Navdeep Jaitly, Heng Ji, Alane Suhr, and Yizhe Zhang. 2025. Training software engineering agents and verifiers with SWE-Gym. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 47717–47737. PMLR.arxiv.org/abs/2412.21139
- [30]Team, Q. (n.d.). Qwen3-1.7B. Hugging Face model card. <a href="https://doi.org/https://huggingface.co/Qwen/Qwen3-1.7B" rel="noopener noreferrer" target="_blank">https://doi.org/https://huggingface.co/Qwen/Qwen3-1.7B</a>
- [34]David Raposo, Sam Ritter, Blake Richards, Timothy Lillicrap, Peter Conway Humphreys, and Adam Santoro. 2024. Mixture-of-depths: Dynamically allocating compute in transformer-based language models. arXiv preprint arXiv:2404.02258.arxiv.org/abs/2404.02258
- [35]Schuster, T., Fisch, A., Gupta, J., Dehghani, M., Bahri, D., Tran, V. Q., Tay, Y., & Metzler, D. (2022). Confident adaptive language modeling. Advances in Neural Information Processing Systems, 35, 17456–17472.
- [36]Sun, Y., Dong, L., Zhu, Y., Huang, S., Wang, W., Mag, S., Zhang, Q., Wang, J., & Wei, F. (2024). You only cache once: Decoder-decoder architectures for language models. Advances in Neural Information Processing Systems, 37. 2405.05254
- [37]Yutao Sun, Yanqi Zhang, Li Dong, Jianyong Wang, and Furu Wei. 2026. You only index once: Cross-layer sparse attention with shared routing. arXiv preprint arXiv:2606.06467.arxiv.org/abs/2606.06467
- [38]Jiaming Tang, Yilong Zhao, Kan Zhu, Guangxuan Xiao, Baris Kasikci, and Song Han. 2024. Quest: Query-aware sparsity for efficient long-context LLM inference. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 47901–47911. PMLR.arxiv.org/abs/2406.10774
- [39]Tessler, C., Mankowitz, D. J., & Mannor, S. (2019). Reward constrained policy optimization. International Conference on Learning Representations. 1805.11074
- [40]THUDM. (2026). IndexCache. <a href="https://doi.org/https://github.com/THUDM/IndexCache" rel="noopener noreferrer" target="_blank">https://doi.org/https://github.com/THUDM/IndexCache</a>
- [41]authors, T. (2025). TidalDecode. <a href="https://doi.org/https://github.com/DerrickYLJ/TidalDecode" rel="noopener noreferrer" target="_blank">https://doi.org/https://github.com/DerrickYLJ/TidalDecode</a>
- [42]Zhaohui Wang, Zhixin Pan, Fanxu Meng, and Muhan Zhang. 2026. LatentIndex: Cross-layer sharing with layer-specific selection for sparse attention. arXiv preprint arXiv:2610.04635.arxiv.org/abs/2610.04635
- [43]Jianyu Wei, Yizhao Gao, Qihao Zhang, Shimao Chen, Zhengyu Tang, Yu Cheng, Shengjie Zhou, Zihan Jiang, Yifan Song, Hailin Zhang, Liang Zhao, Bo Yang, Gang Wang, Shijie Cao, and Fuli Luo. 2026.
- [45]Williams, R. J. (1992). Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(3–4), 229–256.
- [46]Wu, H., Li, L., Huang, H., Tu, Y., Zhang, J., Yu, M., & Yan, J. (2025). HShare: Fast LLM decoding by hierarchical key-value sharing. International Conference on Learning Representations.
- [47]Xingyu Xie, Zhaochen Yu, Yue Liao, Tao Wang, Kim-Chuan Toh, and Shuicheng Yan. 2026. Slow-fast inference: Training-free inference acceleration via within-sentence support stability. arXiv preprint arXiv:2603.12038.arXiv
- [48]Yufei Xu, Fanxu Meng, Fan Jiang, Yuxuan Wang, Ruijie Zhou, Zhaohui Wang, Jiexi Wu, Zhixin Pan, Xiaojuan Tang, Wenjie Pei, Tongxuan Liu, Di Yin, Xing Sun, and Muhan Zhang. 2026. HISA: Efficient hierarchical indexing for fine-grained sparse attention. In Conference on Language Modeling. ArXiv:2603.28458.arXiv
- [49]Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., & al., . et . (n.d.). Qwen3 Technical Report. In arXiv. Retrieved arXiv:2505.09388
- [50]John Yang, Kilian Lieret, Carlos E. Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, and Diyi Yang. 2025b. SWE-smith: Scaling data for software engineering agents. In Advances in Neural Information Processing Systems (Datasets and Benchmarks Track), volume 38. ArXiv:2504.21798.arXiv
- [51]Lijie Yang, Zhihao Zhang, Zhuofu Chen, Zikun Li, and Zhihao Jia. 2025c. TidalDecode: Fast and accurate LLM decoding with position persistent sparse attention. In International Conference on Learning Representations. ArXiv:2410.05076.arXiv
- [52]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations.arxiv.org/abs/2210.03629
- [53]Wenshuai Yao, Wenyong Zhou, Hanyong Shao, Yizhe Chen, Zhiyuan Ning, Yuannuo Feng, Ru Huang, and Kechao Tang. 2026. Recall before you rank: Similarity-guided top-K reuse for efficient long-context attention. arXiv preprint arXiv:2607.27692.arXiv
- [54]Jingyang Yuan, Huazuo Gao, Damai Dai, Junyu Luo, Liang Zhao, Zhengyan Zhang, Zhenda Xie, Yuxing Wei, Lean Wang, Zhiping Xiao, Yuqing Wang, Chong Ruan, Ming Zhang, Wenfeng Liang, and Wangding Zeng. 2025. Native sparse attention: Hardware-aligned and natively trainable sparse attention. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 23078–23097, Vienna, Austria. Association for Computational Linguistics.arxiv.org/abs/2502.11089
- [55]Zan, W., Zhang, J., Tan, J., Liu, H., Wang, C., Li, X., Ma, D., Wu, G., Liu, Y., Li, F., Lin, Y., Sun, P., Pei, G., Xie, Y., & Cai, X. (2026). LongCat Sparse Attention: Taming the lightning via streaming-aware hierarchical cross-layer indexing. In arXiv. arXiv:2608.01662
- [56]Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, Zhangyang “Atlas” Wang, and Beidi Chen. 2023. H2O: Heavy-hitter oracle for efficient generative inference of large language models. In Advances in Neural Information Processing Systems, volume 36, pages 34661–34710. Curran Associates, Inc.arxiv.org/abs/2306.14048
- [57]Zheng, L., Yin, L., Xie, Z., Sun, C., Huang, J., Yu, C. H., Chai, S., Kozyrakis, C., Stoica, I., Gonzalez, J. E., Barrett, C., & Sheng, Y. (2024). SGLang: Efficient execution of structured language model programs. Advances in Neural Information Processing Systems, 37, 62557–62583.
- [58]Ruijie Zhou, Fanxu Meng, Yufei Xu, Tongxuan Liu, Guangming Lu, Muhan Zhang, and Wenjie Pei. 2026. MISA: Mixture of indexer sparse attention for long-context LLM inference. arXiv preprint arXiv:2605.07363.arXiv






























