betaFeedback

WorksComputer Science

RISA: Recoverable Sparse Attention for Block-Diffusion Language Models

Abstract

Block-wise diffusion language models (DLMs) refine multiple tokens in parallel, offering an alternative to token-by-token autoregressive decoding. Their long-context inference, however, remains bottlenecked by memory-bound attention: every denoising step repeatedly reads a growing historical key–value (KV) cache. Directly applying sparse attention does not fully solve this problem, because different queries can select different pages, inflating the block's collective read set, while repeated support construction adds overhead and permanent eviction can discard information needed later. We present RISA, a recoverable sparse-attention method with shared read plans for block-wise DLMs. RISA exploits the fact that historical KV remains fixed within a block while queries evolve across denoising steps. Newly generated history is organized into exact priority and recovery tiers; seed queries then build a protected, bounded read plan whose indices are reused across the block while attention is recomputed with fresh queries and current-block KV. This separates history preservation from repeated reading and amortizes sparse-support construction without changing the backbone or decoder. On DreamReasoner-8B, RISA retains comparable task performance. At 32K, the default configuration speeds up attention by 2.67×2.67\times and fixed-trajectory requests by 1.36×1.36\times over dense, and attention speedup reaches 5.53×5.53\times at 128K. Across eight settings on three models, RISA decodes faster than the tested MAGE, QUEST, and LoSA budgets at matched KL.

Introduction

Diffusion language models (DLMs) generate text by iteratively refining multiple token positions in parallel, offering an alternative to token-by-token autoregressive decoding [2, 14, 18]. Large-scale models such as LLaDA and Dream demonstrate the potential of this paradigm for language modeling, instruction following, and reasoning [16] (Ye et al. 2025). Block-wise DLMs combine parallel refinement within each block with autoregressive progression across blocks, allowing completed blocks to serve as cached context for subsequent generation [1, 23]. This parallelism, however, does not remove the cost of accessing history. Caching avoids recomputing committed key–value (KV) states, but every denoising step still reads the growing prefix to compute attention. For long prompts and extended reasoning traces, these repeated KV accesses become increasingly memory-bound and limit the decoding speedup that block-wise parallelism can deliver [11, 24].

Sparse attention is a natural way to reduce this cost by reading only a subset of the historical KV [20, 25] (Zhang et al. 2023). Yet sparsity designed for ordinary autoregressive decoding does not directly translate into efficient block diffusion. A block contains many evolving queries, and independently selecting a small support for each query can produce a much larger union of accessed pages—the KV inflation problem identified by Xi et al. [24]. Reconstructing these supports across denoising steps further introduces repeated selection overhead. An alternative is to permanently evict low-priority KV, but doing so can remove information that later blocks need. Recent diffusion-specific methods therefore amortize attention computation through pattern, index, or output reuse [21, 11, 24]. The resulting design challenge is to bound repeated KV reads without permanently discarding historical information that later blocks may still need under different queries.

Block diffusion provides a useful structure for addressing this question. During the denoising of one block, historical KV remains fixed while the queries associated with the unfinished block continue to evolve. This creates a natural block-level scope for selecting historical addresses once and reusing them across subsequent denoising steps, while still recomputing attention with fresh queries. At the same time, the repeated queries issued before a generated block ages out provide multiple observations of how its tokens are used. These observations can inform longer-term priority without forcing an irreversible decision immediately after generation, giving the block a natural scope for read selection and history management without assuming static attention.

We introduce RISA, a sparse-attention method that separates preserving historical KV from deciding which history a block reads (Figure 1). Newly generated KV first remains in a grace window, where repeated denoising queries provide observations before it is assigned to a compact priority tier or an addressable recovery tier; both retain exact KV values on the GPU. For each new block, seed queries jointly select a protected, bounded page plan across the available history. Subsequent denoising steps reuse these indices while recomputing attention with their current queries and current-block KV, and later blocks may recover history omitted from an earlier plan. The backbone, native decoder, commit rule, and full-prefix KV finalization remain unchanged throughout generation.

Figure 1. Dense rereads history; RISA reuses one bounded read plan. (a) Dense attention revisits the historical prefix at every denoising step. (b) Per-step top-kk reduces each attention support but repeats scoring and selection. (c) RISA uses the seed forward to observe available history, admits aging KV, constructs one protected read plan, and reuses its historical indices for later steps while queries continue to evolve; finalization still reads the full legal prefix.

Our main contributions are:

  • We introduce a recoverable history organization that uses repeated denoising observations to prioritize generated KV while keeping lower-priority history available for later blocks.

  • We develop a block-static sparse-attention executor that bounds older-history reads and reuses one page plan across denoising steps while recomputing attention with fresh queries.

  • We provide a cross-model quality and system evaluation across reasoning, code, retrieval, and long-context settings. On DreamReasoner-8B, RISA retains comparable task performance. At 32K, the default configuration speeds up attention by 2.67×2.67\times and fixed-trajectory requests by 1.36×1.36\times. At 128K on H200, it reaches 5.53×5.53\times attention speedup. Across the eight in-window settings, it decodes faster than the tested MAGE, QUEST, and LoSA budgets at matched KL.

Related Work

Diffusion language models. Discrete and masked diffusion generate text by reversing token corruption ([2, 14, 18]; LLaDA and Dream scale this approach to language modeling ([16] (Ye et al. 2025)). Block Diffusion combines within-block denoising with autoregressive progression and prefix caching ([1]), while DreamReasoner develops block-size curricula for reasoning ([23]). We target the repeated attention reads that remain after historical representations have been cached.

KV compression and sparse attention. FlashAttention optimizes exact attention through IO-aware execution and work partitioning ([7, 6]). KV compression uses observed attention ([13, 12] (Zhang et al. 2023)), sinks and recency ([26]), or head specialization ([26]). Retrieval methods preserve a larger history and select query-relevant KV ([20, 17, 25]). TidalDecode reuses indices across layers ((Yang et al., 2024)); HiSparse combines host storage with bounded GPU residency ((Xie et al., 2026)). RISA instead keeps both tiers on the GPU: priority determines access preference, not deletion, and the objective is reduced read traffic rather than KV-memory compression.

Efficient diffusion inference. Fast-dLLM introduces approximate block-wise KV caching for bidirectional DLMs and confidence-aware parallel decoding ([22]); dKV-Cache likewise targets redundant representation computation ([15]). Caching reduces what must be recomputed, while parallel decoding changes which tokens are committed together. Cached KV can nevertheless be read in full at every step. RISA instead reduces these reads through sparse attention while preserving the backbone and native commit and stopping rules. It addresses a complementary bottleneck rather than replacing the cache or decoder.

Within sparse diffusion inference, SparseD reuses head-specific patterns ([21]), MAGE derives token indices from exact all-mask attention and reuses them within a block ([11]), and Sparse-dLLM combines eviction with block-boundary refresh ([19]). LoSA instead reuses prefix-attention outputs for stable tokens ([24]). RISA recomputes attention for every current-block query and selects jointly from page summaries. Its distinction is not index reuse alone, but combining a protected, bounded read plan with observation-guided, recoverable generated history that remains addressable across later blocks.

Motivation: Preserve History, Avoid Repeated Work

Sparse attention is useful only when the cost it removes is not replaced by irreversible cache decisions or repeated support construction. We therefore ask two questions: should low-priority history remain recoverable, and must a block rebuild its support at every denoising step? The two decisions operate at different time scales: storage priority persists across blocks, while a read plan serves only the block currently being refined. Diagnostic alternatives reuse the same sampled queries, KV values, and masks, so they isolate these mechanisms without changing the generated trajectory; protocol details appear in the appendix.

Low Priority Does Not Mean No Future Use

Attention-based eviction estimates importance from observations available so far ([12] (Zhang et al. 2023)). Such evidence can prioritize what should be read frequently, but it is weaker evidence for permanent deletion because later blocks issue different queries. We therefore treat priority as a scheduling decision rather than a claim that omitted history has become irrelevant.

Figure 2a shows that demoted cohorts continue to receive attention from later blocks. Moreover, redirecting the same extra read budget to additional HOT pages leaves a larger approximation error than retrieving COLD pages (Figure 2b). Recent use is therefore informative for prioritization, but not a reliable certificate for irreversible eviction; recoverable storage preserves this distinction.

Figure 2. Low-priority history can remain useful. (a) Full-domain attention continues to revisit tokens after they have been demoted from the priority tier. (b) At a comparable read budget, retrieving from the recovery tier yields lower attention-output error than spending the same budget only on additional priority pages. Error bars in (a) resample requests.

Bound Reads and Amortize Selection

Query-aware retrieval reduces KV reads but introduces scoring, ranking, and dispatch ([20]). In block diffusion these costs can repeat even though historical KV is unchanged: attention weights evolve, but one historical address set can often serve several denoising queries.

Figure 3 illustrates the two savings. Sparse support bounds the historical rows consumed by attention, whereas plan reuse removes repeated selection without reusing stale attention outputs. The controlled ablation in § 5.3 shows the same separation in latency. Sparsity and reuse therefore address complementary costs: what is read, and how often those reads must be selected.

Reuse also requires a bounded read set: a shared plan does not scale if it retains a fixed fraction of an ever-growing prefix. We therefore protect recent or otherwise mandatory context separately and cap eligible older HOT reads. This separates two complementary sources of savings within the same block: sparse support limits recurring KV reads, while plan reuse amortizes the work of finding that support. The main ablation confirms that the cap lowers attention cost where it binds (Table 3). The resulting plan bounds recurring reads while leaving lower-priority history recoverable for later blocks and longer prefixes, rather than turning a read-priority decision into permanent eviction.

VariantDreamReasoner Attn. (ms/step)DreamReasoner KV reads (%)DreamReasoner KL (10−310^{-3})SDAR Attn. (ms/step)SDAR KV reads (%)SDAR KL (10−310^{-3})
Recovery and tiering
No COLD recovery2.545646.71.684844.3
COLD reads spent on HOT2.625945.11.755645.1
Flat retrieval2.92605.211.77542.60
COLD quota of 16 pages2.655714.51.714913.9
Admission
Recency admission2.83604.101.76541.70
Random admission2.74604.141.76541.72
Mass + learned correction2.81604.081.85541.76
Plan reuse and HOT cap
Per-step reselection3.69603.333.14541.66
No HOT cap2.85604.202.16980.52
RISA (final)2.86604.211.77541.74

Table 3. Ablations on DreamReasoner and SDAR. Each variant changes one component of the final configuration. The workload uses a 16K prompt and 4,096 output tokens. Attention latency includes selection and amortized block work, profiled on one document per model (dense: 4.38 and 3.86 ms). KV reads are relative to dense and, with KL to dense, are measured on six documents using dense’s trajectories.

Figure 3. Read sparsity and index reuse remove different work. (a) Attention mass changes across denoising, while a block-static execution can keep the same historical support. (b) Sparse execution reduces requested KV rows, and reusing the plan avoids repeatedly rebuilding that support. Counts include seed and mandatory work and should not be interpreted as direct HBM-traffic measurements.

RISA

RISA separates the lifetime of stored history from the lifetime of its read plan (Figure 4). The three panels expose this separation directly: observations determine storage priority as blocks age, seed queries compose a bounded plan, and later steps reuse only its historical addresses while recomputing attention. Thus, storage state persists across blocks, whereas the sparse plan is temporary state for the block being denoised. Each model uses one fixed budget, with no change to its backbone or decoder.

Figure 4. Storage priority, read-plan composition, and execution. (a) Grace-window observations determine whether generated KV enters compact HOT or addressable COLD storage; both keep exact values. (b) Seed queries score page summaries, then protected HOT, selected HOT, recovered COLD, and mandatory recent rows form one read plan. (c) Later steps reuse the historical plan with new queries and current recent KV, combining all contributions under one softmax.

Observe Recent History Before Prioritizing It

Newly committed blocks enter a fully readable grace window. Repeated queries supply observations before aging entries enter compact HOT storage or addressable COLD storage. Both preserve their stored KV values on the GPU; prompt KV stays in HOT but remains subject to read selection. Future plans can read COLD entries directly without promotion or deletion.

For each token and KV head, a fixed priority rule maps accumulated attention observations to a priority score. A running quantile of prior scores targets eight retained HOT entries per 32-token source block. Observations come from the executed attention distribution, with no additional teacher forward. This score controls long-term storage priority rather than the current block’s read set. We score each token by its grace-window attention mass (Equation (2)). Mass-only admission closely matches the learned correction on AIME24 (Section B).

With 32-token blocks, 16-slot pages, and G=8G = 8, the seed observes the outgoing eighth block before admission. Later steps read the seven more recent blocks, when present, and the active block. Short prefixes below 512 tokens retain the carrier’s original dense-attention path.

Storage state versus read state. HOT/COLD assignment persists across blocks and determines how historical KV is organized, whereas the read plan is temporary state for the block currently being denoised. A page omitted from one block’s plan is therefore not deleted: it remains addressable and can be selected again by a later block. Conversely, retaining a page in storage does not force every step to read it. This separation bounds recurring reads without turning sparsification into irreversible KV eviction, so omitted pages remain eligible for future block plans.

Select One Protected, Bounded Plan

The seed reads full HOT, mandatory rows, and up to q=128q = 128 query-selected COLD pages, producing logits and saved queries. HOT eligibility and protection are prepared before this forward. After observations are updated and aging entries are admitted, the native rule commits seed proposals. If denoising continues, saved queries build the reuse plan. Newly admitted HOT rows remain mandatory for this block and join the ranked candidate set when the next block begins.

Each layer and KV head has one plan shared by its positions and mapped query heads. For each query, we take the larger dot product with a page’s minimum or maximum key vector, then sum these scores over the block before selecting pages. This yields one bounded support rather than a union of per-query sets. The max-after-dot heuristic uses page-wise key extrema [20], but differs from QUEST’s coordinate-wise bound in how relevance is aggregated across the page.

Let HbprotH_b^{\mathrm{prot}} contain the first four and last eight sealed HOT pages, protecting initial and recent resident context (Xiao et al., 2024b). From the remaining eligible pages EbE_b, we select a fraction ρ\rho, subject to a 512-page cap; ρ=0.5\rho= 0.5 for DreamReasoner and LLaDA2.2-mini and 1.0 for SDAR. The budget kbk_b rounds ρ∣Eb∣\rho|E_b|, clips it to the cap and candidate count, and selects at least one page when available. Using pooled page scores uu, we combine mandatory and selected rows into the block’s read support:

Sb=Rb∪rows⁡(Hbprot∪Top⁡kb(u,Eb)∪Top⁡q(u,Cb)),(1)S_b = R_b \cup\operatorname{rows}\left(H_b^{\mathrm{prot}} \cup\operatorname{Top}_{k_b}(u,E_b) \cup\operatorname{Top}_{q}(u,C_b)\right), \tag*{(1)}

where CbC_b contains COLD candidates and RbR_b contains mandatory recent/current, partial-page, and newly admitted HOT rows. The cap excludes protected pages, COLD, and mandatory rows. Each valid row appears once; unselected HOT pages stay available for subsequent block plans.

Reuse Addresses, Not Attention Outputs

Later steps reuse SbS_b with fresh queries, current-block KV, and attention outputs. Historical and recent/current partials share one softmax normalization; they are not independently normalized outputs that are averaged. Persistent device buffers hold indices and valid lengths for CUDA-graph replay; a block completed by its seed skips plan construction entirely.

Reuse addresses, recompute attention. The shared plan fixes only which historical rows are gathered. Each denoising step still recomputes query–key scores, normalization, and value aggregation using its current queries and current-block state. Thus the optimization does not assume that attention probabilities are static across denoising; it amortizes support construction while preserving step-specific attention computation. Mandatory recent/current rows remain outside the eligible-HOT cap so that newly evolving context is not frozen by the seed plan.

After commitment, the native full-prefix finalization pass reads all legal KV to write the block’s representations, independently of SbS_b. Seed attention, maintenance, and finalization therefore remain context-dependent costs. Recent protection and later recovery mitigate omitted-support error without making attention lossless. Exact storage does not imply dense-equivalent trajectories.

Evaluation

Experimental setup. We evaluate DreamReasoner-8B (Wu et al. [23]), SDAR-8B-Chat (Cheng et al. [4]), and LLaDA2.2-mini (inclusionAI, 2026), a mixture-of-experts model with a native 128K window, in BF16 on NVIDIA H200 GPUs, using 32-token blocks and one request per device. RISA uses G=8G = 8, Kmax⁡=512K_{\max} = 512, and q=128q = 128, with a fixed read fraction per model: ρ=0.5\rho= 0.5 on DreamReasoner and LLaDA2.2-mini and 1.0 on SDAR. Efficiency is measured at 16K, 32K, 64K, and 128K prefixes. DreamReasoner and SDAR are configured for contexts up to 32K; we use the longer prefixes to characterize long-context scalability, while task-quality evaluation remains within the configured range. LLaDA2.2-mini covers all four contexts natively.

Metrics. Besides task accuracy, we measure fidelity as the KL divergence from dense on dense’s own denoising states, and two latencies: attention latency, the attention path per denoising step including selection and amortized per-block work, and per-request latency, the wall time of a complete decode. Per-request budget sweeps replay dense’s trajectory for 512 output tokens, so every method runs the same denoising steps (Section A).

Baselines. Dense uses each carrier’s native full-attention path. QUEST performs query-dependent sparse retrieval, while MAGE forms a block-level sparse pattern from its all-mask observation and reuses that pattern during later denoising steps. As no official MAGE implementation was publicly available at the time of our evaluation, we reimplement it following the algorithm and configuration described in the paper (Kwon et al. [11]), overlapping its selection with the forward pass on a separate CUDA stream and keeping its first two layers exact. LoSA (Xi et al. [24]) recomputes only the queries that changed most and caches the rest; wewise implement it from the paper. Dense, QUEST, MAGE, LoSA, and RISA share the same model weights, tokenizer, prompts, native decoder, and block size, paged attention kernel, and CUDA graphs. QUEST and MAGE use nominal 1,024-token budgets in the task benchmarks. RISA’s cap applies to eligible HOT pages, with protected HOT, recovered COLD, and mandatory recent/current rows tracked separately. The budget sweeps record realized reads and compare latency at matched fidelity.

End-to-End Accuracy

Benchmarks. DreamReasoner-8B is evaluated on GSM8K, MATH-500, AIME25, LiveCodeBench v6, and HotpotQA; SDAR-8B-Chat is evaluated on GSM8K, MATH-500, and HotpotQA. All methods use identical prompts, decoding settings, and generation limits within each model and benchmark. Mathematics uses answer accuracy, LiveCodeBench uses execution pass@1, and HotpotQA uses token F1. GSM8K, MATH-500, LiveCodeBench, and HotpotQA use greedy decoding; AIME25 uses temperature 0.6 with prompt and output together limited to 32,768 positions. Output caps are 24,576 tokens for mathematics/code and 1,024 for HotpotQA. Benchmark scores and the DreamReasoner and SDAR attention profiles (Figure 6) use q=16q = 16, the learned admission correction, and ρ=0.75\rho= 0.75 on SDAR.

Figure 6. Efficiency across prefix lengths on both models. (a, c) Attention speedup over dense for DreamReasoner-8B and SDAR-8B-Chat (QUEST and MAGE at 1,024 tokens), from kernel profiles of 512-token requests with per-block work amortized over 15.6 or 10.3 steps per block. (b, d) Per-request speedup over dense, the values of Table 2: complete 512-token decodes on dense’s fixed trajectory (MAGE and QUEST at 8,192 tokens, LoSA at 4,096). Dotted: dense (1×1\times). Gray: beyond the configured 32K window, execution only.

DreamReasoner-8B. Table 1 reports competitive scores across reasoning, code, and retrieval. RISA obtains 86.0% on GSM8K and 88.8% on MATH-500, compared with dense’s 85.0% and 88.2%. Its 50.0% LiveCodeBench accuracy matches dense and exceeds QUEST and MAGE by 2.5 and 7.5 points. On HotpotQA, it is 0.3 F1 below dense but 1.0 and 4.7 above the sparse alternatives. The ranking is task-dependent: RISA ties QUEST on GSM8K, while MAGE is 0.2 points higher on MATH-500. On AIME25, RISA scores 60.4%, versus 58.3% for dense, 59.4% for QUEST, and 59.2% for MAGE. Overall, RISA remains close to dense while matching or exceeding the reported sparse alternatives on four of the five DreamReasoner tasks.

MethodDreamReasoner-8B GSM8KDreamReasoner-8B MATH-500DreamReasoner-8B AIME25DreamReasoner-8B LCBDreamReasoner-8B HotpotQASDAR-8B-Chat GSM8KSDAR-8B-Chat MATH-500SDAR-8B-Chat HotpotQA
Dense85.088.258.350.040.288.565.453.6
QUEST86.088.259.447.538.987.566.252.3
MAGE85.589.059.242.535.288.066.251.4
RISA86.088.860.450.039.989.566.853.4

Table 1. Benchmark accuracy with 32-token blocks (original configuration). Mathematics uses answer accuracy, LCB execution pass@1, and HotpotQA token F1; bold marks the best sparse method.

SDAR-8B-Chat. We use the same history-management design on SDAR with the fixed configuration stated above (Table 1). RISA reaches 89.5% on GSM8K and 66.8% on MATH-500, versus dense’s 88.5% and 65.4%, and is 0.2 F1 below dense on HotpotQA. It leads the sparse baselines on all three tasks under identical prompts, benchmark settings, and generation limits.

Efficiency Evaluation

Attention latency. Figure 5 sweeps the budget of every method and plots attention speedup against KL to dense within each model and context; at the default budgets, RISA has the highest attention speedup in all eight in-window settings. Attention latency covers GPU work for attention, selection and indices, normalization/merge, and required KV-cache operations, including attention in full-prefix finalization. Per-block work uses a common dense-derived amortization horizon of 15.6 steps per block for DreamReasoner and 10.3 for SDAR. RISA has the lowest measured attention cost among the evaluated implementations at every length on both models (Figure 6). On DreamReasoner, speedup over dense increases from 1.50×1.50\times at 16K to 2.32×2.32\times, 3.51×3.51\times, and 5.53×5.53\times at 32K, 64K, and 128K. Over this eightfold increase in prefix length, dense cost rises from 3.201 to 18.763 ms, whereas RISA rises from 2.135 to 3.394 ms. The cap constrains repeated eligible-HOT reads, but seed, metadata, and finalization work still scale with history. The result is slower cost growth, not constant-time attention. This widening gap reflects the bounded read plan: as the prefix grows, dense attention repeatedly revisits more history, whereas RISA limits the historical support consumed by later denoising steps. The same trend holds on SDAR: RISA reaches 1.24×1.24\times, 1.84×1.84\times, 2.76×2.76\times, and 4.58×4.58\times speedup as the prefix grows.

Quality–speed frontiers for DreamReasoner-8B, SDAR-8B-Chat, and LLaDA2.2-mini

Figure 5. Quality–speed frontiers. Attention-latency speedup over dense (ms per denoising step on natively decoded 512-token requests, per-block work spread over dense’s steps per block) against KL to dense on dense’s states; lower right is better. Dashed line: dense (1×). The two y regions use separate log scales. Color: method; shape: context; dotted lines: each method’s frontier; ring: RISA’s default; no point is clipped. At LLaDA2.2-mini 16K, MAGE’s 8K and RISA’s ρ=1.0\rho= 1.0 budgets are slightly slower than dense.

From attention savings to model execution. Attention savings matter only if they survive the rest of the forward pass. Table 2 times complete 512-token decodes on dense’s fixed trajectory at each method’s default budget. Per-request speedup grows with context on DreamReasoner and SDAR; on LLaDA2.2-mini, where attention is a smaller share of each step, it is smaller but present at every context. RISA is also faster per request than every sparse baseline at each measured length. Across budgets on the same trajectory, each tested MAGE, QUEST, and LoSA budget has a faster RISA point with no higher KL (Figure 9 and Table 7). When each method follows its own trajectory on book-prefixed MATH-500 at 32K, RISA has the lowest profiled attention latency and mean per-request latency on DreamReasoner and SDAR and matches dense accuracy on the same 48 problems per model (Table 9); there, per-request speedup reflects both attention savings and changes in decoding length. Projection, feed-forward, decoding, and synchronization remain outside the sparse-attention savings and determine how much of the attention-path improvement appears in per-request latency.

MethodDreamReasoner-8B 16KDreamReasoner-8B 32KDreamReasoner-8B 64K{sup}†`DreamReasoner-8B 128K{sup}†`SDAR-8B-Chat 16KSDAR-8B-Chat 32KSDAR-8B-Chat 64K{sup}†`SDAR-8B-Chat 128K{sup}†`LLaDA2.2-mini 16KLLaDA2.2-mini 32KLLaDA2.2-mini 64KLLaDA2.2-mini 128K
RISA1.151.361.762.671.171.401.762.671.071.061.161.25
MAGE 8K0.991.231.432.031.071.301.472.070.951.031.001.14
QUEST 8K0.921.081.352.040.911.111.362.021.011.001.101.19
LoSA 4K0.841.011.181.640.861.031.191.640.890.940.951.09

Table 2. Per-request speedup over dense (decode wall time; dense = 1.00×1.00\times) for complete 512-token requests, with every method replaying dense’s denoising trajectory on the same documents; RISA’s default, MAGE and QUEST at 8,192 tokens, LoSA at 4,096. †Beyond DreamReasoner’s and SDAR’s configured 32K window (YaRN); execution only.

Figure 9. Quality–latency Pareto frontiers for complete requests. Every budget of every method, placed by per-request speedup over dense and KL to dense; lower right is better. Speedup is total dense wall time over total method wall time for complete 512-token decodes replayed on dense’s denoising trajectory, and KL is measured on the same requests; the star at (1, 0) is dense. Lines join each method’s Pareto budgets within the panel (no other budget of the same method is both faster and lower in KL), faded markers are budgets beaten by another budget of the same method, and the filled circle is RISA’s default. Points whose KL exceeds a panel’s range are drawn on its top edge at their measured speedup.

Model (dense s)Baseline BudgetBaseline sBaseline KLBaseline ReadMatched RISA BudgetMatched RISA sMatched RISA KLMatched RISA Read
DreamReasoner-8B (4.09 s)MAGE 1,0243.1843.90.144,096, ρ=0.5\rho=0.52.9528.40.19
MAGE 2,0483.2226.60.178,192, ρ=0.5\rho=0.53.0014.40.31
MAGE 4,0963.2814.20.2316,384, ρ=1.0\rho=1.03.105.60.55
MAGE 8,1923.396.80.3416,384, ρ=1.0\rho=1.03.105.60.55
QUEST 1,0243.44326.50.091,024, ρ=0.5\rho=0.52.9179.30.10
QUEST 4,0963.70125.50.181,024, ρ=0.5\rho=0.52.9179.30.10
QUEST 8,1923.7672.60.292,048, ρ=0.5\rho=0.52.9349.50.13
LoSA 1,0243.92149.00.141,024, ρ=0.5\rho=0.52.9179.30.10
LoSA 4,0964.06140.10.341,024, ρ=0.5\rho=0.52.9179.30.10
SDAR-8B-Chat (7.13 s)MAGE 5125.1380.20.101,024, ρ=1.0\rho=1.04.8244.40.08
MAGE 1,0245.1841.80.112,048, ρ=1.0\rho=1.04.8528.00.11
MAGE 2,0485.2219.80.144,096, ρ=1.0\rho=1.04.9315.30.17
MAGE 4,0965.319.50.208,192, ρ=1.0\rho=1.05.026.50.29
MAGE 8,1925.424.00.3212,288, ρ=1.0\rho=1.05.123.30.41
QUEST 1,0245.88351.70.091,024, ρ=1.0\rho=1.04.8244.40.08
QUEST 2,0485.95218.00.121,024, ρ=1.0\rho=1.04.8244.40.08
QUEST 4,0966.34149.10.171,024, ρ=1.0\rho=1.04.8244.40.08
QUEST 8,1926.4793.20.291,024, ρ=1.0\rho=1.04.8244.40.08
LoSA 1,0246.65161.10.121,024, ρ=1.0\rho=1.04.8244.40.08
LoSA 4,0966.90156.50.321,024, ρ=1.0\rho=1.04.8244.40.08
LLaDA2.2-mini (2.95 s)MAGE 1,0242.8438.30.224,096, ρ=0.5\rho=0.52.7933.50.23
MAGE 4,0962.8720.10.308,192, ρ=0.5\rho=0.52.8119.30.34
MAGE 8,1922.8811.50.4016,384, ρ=0.75\rho=0.752.8511.30.57
QUEST 1,0242.88129.50.131,024, ρ=0.5\rho=0.52.7876.80.14
QUEST 4,0962.9530.40.218,192, ρ=0.5\rho=0.52.8119.30.34
QUEST 8,1922.9715.00.3312,288, ρ=0.75\rho=0.752.8314.10.45
LoSA 1,0243.10137.50.191,024, ρ=0.5\rho=0.52.7876.80.14
LoSA 4,0963.15128.60.371,024, ρ=0.5\rho=0.52.7876.80.14

Table 7. Table 7. Absolute per-request latency at matched fidelity, 32K. Every method follows dense’s fixed denoising trajectory on the same documents. Seconds are decode wall time for 512 tokens, excluding the warm-up document. KL (×10−3\times10^{-3}) and the KV read ratio include all paired documents. For each baseline budget, the matched RISA budget is the fastest one whose KL is not higher (HOT cap in tokens, read fraction ρ\rho). Dense’s seconds per request are given beside each model.

Model, contextDenseRISAMAGEQUESTLoSA
DreamReasoner, 32K1.00 / 1.00<br>5.8 / 77.1 / 7952.69 / 1.55<br>3.7 / 77.1 / 7711.50 / 1.27<br>4.6 / 79.2 / 7781.36 / 1.16<br>5.0 / 81.2 / 7581.24 / 0.89<br>6.5 / 75.0 / 882
SDAR, 32K1.00 / 1.00<br>6.9 / 62.5 / 5873.02 / 1.47<br>4.7 / 62.5 / 5851.95 / 1.30<br>5.3 / 62.5 / 6041.37 / 1.07<br>6.5 / 60.4 / 5991.26 / 0.92<br>7.5 / 56.2 / 657

Table 9. Native decoding on book-prefixed MATH-500. Each setting uses 48 paired problems and a 2,048-token output cap. Each method follows its own trajectory. Cells report attention and per-request speedup over dense (top), then mean decode seconds, answer accuracy (%), and mean output tokens (bottom). Attention speedups use one profiled request per method and setting. MAGE and QUEST use 8,192 tokens and LoSA uses 4,096.

Where the attention path spends time. Figure 7 separates prefix attention from Index, which covers scoring and baseline selection, and Top-k, the fused RISA selector. Merge combines attention partials and observations, and Cache covers KV movement and layout maintenance. MAGE constructs its scores from the all-mask block observation, an expense that grows with the prefix even at a fixed selection budget. QUEST scores each query and attends the resulting union per KV head. LoSA reuses outputs for queries that change little, and its Index time exceeds its Attention time in all twelve settings. RISA scores page summaries and reuses the bounded read plan across steps. It has the lowest summed kernel time among sparse methods at every model and prefix, with LoSA’s totals 2.12.1–3.0×3.0\times those of RISA. The per-request measurements capture how these savings translate to full model execution.

Figure 7. Attention latency breakdown. Attention-path kernel time per denoising step on an H200 NVL for a 512-token decode on a common prompt per model and prefix. Block work is amortized over dense’s 22.2, 12.3, and 15.2 steps for DreamReasoner, SDAR, and LLaDA2.2-mini. D/Q/M/L/R denote Dense, QUEST, MAGE, LoSA, and RISA. QUEST and MAGE use 8,192 tokens and LoSA uses 4,096. Index includes baseline selection and Top-k is RISA’s selector. Dense is the total. Gray marks execution with YaRN.

Scalability to Blackwell GPUs. We additionally measure amortized attention-path latency for SDAR-8B on an NVIDIA RTX PRO 6000 Blackwell Server Edition at a 64K-token prefix. Using SDAR-8B’s measured average of 10.30 denoising steps per block, dense requires 10.43 ms per step, compared with 3.48 ms for RISA (2.99× speedup), 7.18 ms for MAGE (1.45×), and 5.39 ms for QUEST (1.94×). The attention-path advantage therefore persists on a newer GPU generation.

Memory footprint. Recoverability trades storage for fewer reads. Peak allocation is 29.38 versus dense’s 27.62 GiB at 16K and 44.20 versus 39.27 GiB at 64K. Packed KV, metadata, and scratch add storage in exchange for fewer reads during denoising.

Ablation Study

Table 3 isolates recoverable history, block-static plan reuse, and bounded HOT reads, changing one component at a time on a workload in which generated history ages into COLD. Spending the recovery quota on HOT pages raises KL by more than an order of magnitude at comparable reads. Disabling recovery also raises KL. Separate tiers also beat a single ranking of all history at nearly identical reads, and a smaller COLD quota saves little time but raises KL. Plan reuse is what keeps attention cheap: rebuilding the plan at every step reads the same history and gains a little fidelity, but takes longer. On SDAR, lifting the HOT cap reads nearly the whole prefix and costs more attention time; on DreamReasoner, removing the cap barely changed realized reads in this workload. These ablations separate the roles of the three components: recovery preserves access to older history, plan reuse amortizes support construction, and the HOT cap bounds recurring reads.

Admission policies yield similar KL on the 16K ablation. On AIME24, where more generated history ages out of the grace window, mass-only admission lowers KL relative to recency and random admission at similar read volumes. It also approaches the fidelity of the learned correction (Table 4), which motivates using the mass score directly.

On LLaDA2.2-mini, COLD recovery lowers KL at comparable reads on both the 16K and 64K prompts (Figure 10). The needle test probes a different capability: retrieving distant evidence from the prompt while the HOT cap binds. From 32K to 128K, recall stays within six percentage points of dense while RISA reads 32–48% as much history. At 128K it matches dense recall while reading about a third as much.

Figure 10. Recovery and retrieval under bounded reads. (a) Generated-history recovery: KL to dense (×10−3\times10^{-3}) after 4,096 generated tokens (six documents per model and context) for RISA, flat retrieval (one ranking over HOT and COLD), HOT-only (recovery reads spent on HOT pages), and no recovery. The first three use comparable read volumes where measured. (b) Prompt retrieval of three needles among distractors (n=32n = 32; 12 on LLaDA2.2-mini): per-needle recall for dense and RISA (top, with gaps in percentage points) and RISA’s KV reads relative to dense (bottom; dark: HOT cap active in at least half of sparse blocks).

Lowering the read fraction reduces realized reads at 16K on DreamReasoner. At 32K, the same reduction leaves the read volume nearly constant because both settings propose more eligible pages than the cap allows. Lowering the cap itself reduces reads further and raises KL. The read fraction thus controls the budget at shorter contexts, while the HOT cap sets the limit as the eligible history grows (Section E).

Block size. With 64-token DreamReasoner blocks, RISA still lowers both attention and per-request latency at 16K and 32K (Table 8). Each block size uses its own dense trajectory, and we hold the page size, grace window, and admission reservoir fixed in tokens. Changing the block size can change how many denoising steps share a plan, so we measure latency and KL against dense at the same block size. The cap binds at 32K for both block sizes, where RISA retains the latency benefit while reading roughly 30% of history. The measured speedup is lower at 64 tokens, even as the read ratio remains close at each prefix. At 32K, the 64-token configuration cuts attention time by more than half and still speeds up the complete request.

BlockContextAttention (ms / step) DenseAttention (ms / step) RISAAttention (ms / step) SpeedupRequest (s) DenseRequest (s) RISARequest (s) SpeedupKL (×10−3\times10^{-3})Reads (%)Cap (%)
3216K3.191.841.73×3.783.281.15×5.3540
3232K5.952.062.89×4.143.061.36×14.431100
6416K3.362.601.30×4.193.841.09×6.9520
6432K5.812.762.11×4.313.381.28×16.129100

Table 8. Table 8. Block-size sweep on DreamReasoner-8B. An execution test of dense and RISA only, on DreamReasoner-8B at 16K and 32K with 12 documents per cell. DreamReasoner was trained and evaluated at block sizes of 4–32 tokens, and 64-token blocks extend the range tested here. Request latency (decode wall time, s), KL, reads, and cap activity use 512-token decodes replayed on each block size’s own dense trajectory, with KL referenced to the dense run at that block size. Attention profiles cover output positions 320–511 and amortize block work over dense’s steps per block. Both block sizes use 16-token pages, an 8,192-token HOT cap, a 256-token grace window, a 128-page COLD quota, ρ=0.5\rho=0.5, and mass-only admission; the allocator preserves the admission rate and the reservoir size in tokens. Cap activity is the share of RISA’s sparse blocks whose reused plan hits the HOT cap.

Conclusion

RISA reduces repeated KV reads in block diffusion by keeping exact history recoverable and reusing a bounded read plan across denoising steps. Across eight in-window settings, it decodes faster than the tested MAGE, QUEST, and LoSA configurations at matched KL. On book-prefixed MATH-500 at 32K, it lowers per-request latency under native decoding on DreamReasoner and SDAR and matches dense answer accuracy over 48 paired problems per model. The 128K H200 scaling results and the 64K SDAR Blackwell study show that the attention-path advantage persists as context and hardware change. Keeping exact history resident trades KV-cache storage for fewer repeated reads.

AI USE STATEMENT

Generative AI tools were used to edit author-written text for grammar, clarity, and readability. The authors reviewed all AI-assisted edits and take full responsibility for the final content of the paper.

REPRODUCIBILITY STATEMENT

The paper specifies the inference policy, decoding settings, and measurement scope. Static figures and directly editable tables accompany the source. Primary and supplementary studies use the measurement definitions and benchmark protocols documented in the paper and source.

Protocol and Metrics

Fidelity. Dense decodes each request with its native decoder. At every denoising step the method under test receives exactly dense’s state (committed tokens and remaining masks), and we record KL⁡(pdense∥pmethod)\operatorname{KL}(p_{\mathrm{dense}} \parallel p_{\mathrm{method}}) at every masked position. We use KL rather than excess negative log-likelihood, which is not monotone in the number of reads.

Latency. We report two latency metrics. Attention latency is the time of the attention path per denoising step, in ms: attention, selection, indices, merging, required KV-cache work, and attention-related block work (seed step, planning, MAGE’s all-mask scoring, finalization) spread over dense’s steps per block, averaged over every step of the request. It is timed per request as the method’s step time minus dense’s non-attention time, with dense’s attention time from kernel profiles; kernel profiles also apportion it into components (Figure 7), counting kernel time only. Per-request latency is the measured wall time of a complete decode after prefill, including seed steps, planning, denoising, and finalization; we report it as total dense time over total method time across documents. Under the fixed trajectory every method replays dense’s denoising schedule on dense’s states, so all methods run the same blocks and steps; under native decoding (Tables 6, 9 and 10) each method follows its own trajectory. Fixed-length timing uses 512 output tokens with end-of-sequence disabled, one request per GPU, and CUDA graphs. Timing excludes the warm-up document when applicable. The KV read ratio counts every historical KV row read, including protected, grace-window, and recovered pages, relative to dense.

Context and block lengths. DreamReasoner-8B and SDAR-8B-Chat are configured for 32,768 positions. The 32K timing setting reserves 512 positions for decoding; their 64K and 128K settings use YaRN and measure execution only. The primary evaluation uses 32-token blocks, a reported operating point for DreamReasoner-8B and the native block size for SDAR-8B-Chat-b32 and LLaDA2.2-mini. Table 8 studies 32- and 64-token blocks.

Admission and History Selection

Mass score. The priority of a generated token ii, computed separately for each layer and KV head, is the attention mass it received during its grace window:

πi=log⁡(1+wi),wi=∑b∈G(i)∑t∑r∈b, g∈hpr,g(t)(i).(2)\pi_i = \log(1+w_i), \qquad w_i = \sum_{b\in\mathcal{G}(i)} \sum_t \sum_{r\in b,\ g\in h} p_{r,g}^{(t)}(i). \tag*{(2)}

summed over the blocks G(i)\mathcal{G}(i) that read ii in the window, their denoising steps tt, query rows rr, and the query heads gg mapped to KV head hh. Because pp is the probability of the sparse attention that already ran, scoring needs no extra forward pass.

Allocator. When a source block leaves the grace window, each layer and KV head scores its 32 tokens with Equation (2). Without score history it admits the top eight. Otherwise it admits the scores above the 1−8/321 - 8/32 quantile of the previous min⁡(n,24)×32\min(n, 24) \times32 scores, keeping between two and 24 tokens, and all 32 scores join the reservoir. Admitted exact KV enters HOT and the rest enters COLD; both remain stored. The last partial prompt block stays in HOT.

Learned correction. The learned admission variant adds a bounded term, πi=log⁡(1+wi)+4tanh⁡(MLP⁡(xi,eℓ))\pi_i = \log(1 + w_i) + 4 \tanh(\operatorname{MLP}(x_i, e_\ell)), with six observation features xix_i (grace-window mass, mass at the admitting block, step-zero mass, top-eight hit count, key norm, and position in the block) and an eight-dimensional layer embedding eℓe_\ell, 801 parameters in total. Training used 10,332 block–layer samples from 12 development prompts. The sparse model visits the states and a frozen dense teacher provides the target, the log of the mass a token receives from two blocks later onward.

Admission on AIME24. At nearly identical read volumes, mass-only admission approaches the fidelity of the learned correction and improves on recency and random admission (Table 4). Ranking HOT and COLD jointly also increases KL at similar reads.

RuleKL (×10−3\times10^{-3})KV reads (%)
Mass + learned correction8.6075.9
Recency admission10.0876.2
Random admission10.3076.0
Flat retrieval11.1576.6
Mass only (RISA)8.7276.0

Table 4. Admission and retrieval on AIME24. Seven DreamReasoner problems with a 16,384-token output cap. Each variant uses a 128-page COLD quota. KL to dense is measured on dense’s denoising states. Reads are realized historical KV reads relative to dense.

COLD quota. The default COLD quota is 128 pages per layer and KV head. Table 3 compares it with a quota of 16 pages.

Context-adaptive HOT cap (RISA-K). RISA-K sets the HOT cap to max⁡(512,⌈f∣Eb∣⌉)\max(512, \lceil f|E_b| \rceil) pages per layer and KV head, where EbE_b is the eligible HOT set. On tuning books, we choose the smallest tested ff whose KL at 128K is at most the fixed cap’s KL at 32K. This selects f=0.5f = 0.5 (KL 0.0061 against a target of 0.0085). On twelve LLaDA2.2-mini evaluation documents, KL for the fixed and adaptive caps is 0.019 and 0.012 at 32K, 0.023 and 0.008 at 64K, and 0.055 and 0.012 at 128K on dense trajectories. On these fixed trajectories, RISA-K is 1.04–1.17× faster than dense per request.

LLaDA2.2-mini Adaptation

Only the attention core is replaced: the checkpoint’s fused projection, query/key normalization, rotary embedding, and output projection run unchanged, and dense, RISA, and every baseline use the same attention kernel. The mixture-of-experts layers run as grouped matrix multiplications over the same routed experts, which reproduce the checkpoint’s expert loop exactly and allow CUDA-graph capture. Tokens are committed by the checkpoint’s confidence threshold with its token-editing step disabled for every method, so blocks keep 32 positions and finished blocks are immutable. BF16 rounding alone changes expert routing in this model, so every method’s KL includes a small numerical floor.

Additional Efficiency Results

Table 5 reports attention-latency and per-request speedup for every model, context, and method, Figure 9 repeats the budget sweep of Figure 5 with per-request latency on the fixed trajectory, and Table 6 reports per-request speedup under native decoding, where each method follows its own decoding trajectory.

Attention-latency and per-request speedup over dense at the default budgets (RISA’s default, MAGE and QUEST at 8,192 tokens, LoSA at 4,096), for every measured model, context, and method. Attention speedup is dense’s attention latency (ms per denoising step; the Dense attn. column) over the method’s, timed on 512-token requests in which each method decodes natively, with each method’s per-block work (seed step, planning, MAGE’s all-mask scoring, finalization) spread over dense’s steps per block in the same session. Per-request speedup is total dense wall time over total method wall time for complete 512-token requests replayed on dense’s denoising trajectory. Read is RISA’s KV read ratio. Bold marks the fastest method. †Beyond the configured window (YaRN); execution only.

ModelContextDense attn. (ms/step)Attention-latency speedup RISAMAGE 8KQUEST 8KLoSA 4KPer-request speedup (fixed trajectory) RISAMAGE 8KQUEST 8KLoSA 4KRead
DreamReasoner-8B16K3.271.591.100.670.621.150.990.920.840.54
32K5.952.671.621.151.021.361.231.081.010.31
64K†9.903.952.301.771.431.761.431.351.180.17
128K†20.318.883.493.522.272.672.032.041.640.10
SDAR-8B-Chat16K3.171.721.380.690.641.171.070.910.860.54
32K5.793.022.091.201.051.401.301.111.030.29
64K†9.744.162.511.791.441.761.471.361.190.16
128K†20.428.633.703.472.272.672.072.021.640.10
LLaDA2.2-mini16K1.241.100.910.480.481.070.951.010.890.56
32K2.001.701.280.750.711.061.031.000.940.34
64K3.362.331.471.110.991.161.001.100.950.23
128K6.293.371.761.791.441.251.141.191.090.17

Table 5.

MethodDreamReasoner-8B 16KDreamReasoner-8B 32KDreamReasoner-8B 64K{sup}†DreamReasoner-8B 128K{sup}†SDAR-8B-Chat 16KSDAR-8B-Chat 32KSDAR-8B-Chat 64K{sup}†SDAR-8B-Chat 128K{sup}†LLaDA2.2-mini 16KLLaDA2.2-mini 32KLLaDA2.2-mini 64KLLaDA2.2-mini 128K
RISA1.201.512.103.051.161.471.872.960.981.121.111.20
MAGE 8K0.891.261.542.171.041.321.512.200.771.040.941.06
QUEST 8K0.791.011.392.240.881.081.392.170.861.061.011.03
LoSA 4K0.720.951.141.620.800.991.201.710.750.780.861.00

Table 6. Per-request speedup under native decoding (decode wall time; dense = 1.00×1.00 \times). Each method, dense included, decodes its own 512-token trajectory on the same documents (DreamReasoner: 30 at 16K and 32K, six at 64K and 128K; SDAR: six; LLaDA2.2-mini: 12); speedup is total dense time over total method time. RISA’s default, MAGE and QUEST at 8,192 tokens, LoSA at 4,096. Bold: fastest sparse method per column. †Beyond DreamReasoner’s and SDAR’s configured 32K window (YaRN); execution only.

Figure 8 reports both latency metrics on LLaDA2.2-mini: attention latency (a), measured as in Table 5, and per-request speedup on the fixed trajectory (b). The per-request speedup is smaller because attention is a modest share of each step on this mixture-of-experts model. The kernel-profile components in Figure 7 count kernel time only, so their totals are lower than the timed attention latency. RISA is the fastest method on both metrics at every length.

Figure 8. LLaDA2.2-mini at 16K–128K. (a) Attention latency in ms per denoising step for dense (dashed) and each method, measured as in Table 5 on 512-token requests with per-block work spread over dense’s steps per block; RISA’s attention-latency speedup over dense is printed above each length. (b) Per-request speedup over dense (dashed, 1×) for complete 512-token requests replayed on dense’s denoising trajectory. RISA’s default, MAGE and QUEST at 8,192 tokens, and LoSA at 4,096 tokens.

Absolute latency at matched fidelity. Table 7 reports measured seconds per request at 32K for every MAGE, QUEST, and LoSA budget and, for each, the fastest RISA budget with no higher KL, on paired documents with dense’s fixed trajectory. Each of the 28 baseline operating points has a faster RISA counterpart with no higher KL.

Block size. RISA reduces both attention and request latency at 32 and 64 tokens on DreamReasoner-8B (Table 8). At 64 tokens the request speedup is 1.09× at 16K and 1.28× at 32K, with the HOT cap active in every measured 32K block. The model was trained and evaluated at block sizes 4, 8, 16, and 32 ([23]), so 64 tokens is an execution stress test. Each measured block size uses its own dense trajectory and therefore its own KL reference.

Complete requests on a natural long task. Table 9 evaluates native decoding on book-prefixed MATH-500 at 32K. Every method decodes the same 48 prompts per model. RISA leads in attention and per-request speedup on DreamReasoner and SDAR and matches dense answer accuracy on both.

Additional Quality Results

We report one-factor sensitivity, multi-needle retrieval, and fidelity on benchmark prompts. On DreamReasoner at 16K, lowering ρ\rho from 0.5 to 0.25 nearly triples KL as fewer HOT pages are read. At 32K, the cap holds reads near 31% over the same change in ρ\rho. Lowering the cap itself to 2,048 tokens at 32K raises KL about fourfold (Table 10).

VariantDreamReasoner 16KDreamReasoner 32KSDAR 16KSDAR 32K
cap Kmax⁡=128K_{\max}=128 pages (2,048 tokens)1.26×1.26\times / 0.02921.52×1.52\times / 0.05751.21×1.21\times / 0.01381.51×1.51\times / 0.0167
cap Kmax⁡=256K_{\max}=256 pages0.96×0.96\times / 0.01271.46×1.46\times / 0.03051.20×1.20\times / 0.00541.50×1.50\times / 0.0077
cap Kmax⁡=512K_{\max}=512 pages (default)1.14×1.14\times / 0.00451.61×1.61\times / 0.01351.17×1.17\times / 0.00121.47×1.47\times / 0.0028
cap Kmax⁡=1,024K_{\max}=1{,}024 pages1.14×1.14\times / 0.00451.40×1.40\times / 0.00521.12×1.12\times / 0.00041.39×1.39\times / 0.0008
no cap1.14×1.14\times / 0.00451.40×1.40\times / 0.00521.12×1.12\times / 0.00041.30×1.30\times / 0.0004
ρ=0.25\rho=0.250.99×0.99\times / 0.01321.74×1.74\times / 0.01391.20×1.20\times / 0.00541.47×1.47\times / 0.0029
ρ=0.5\rho=0.51.14×1.14\times / 0.00451.61×1.61\times / 0.01351.18×1.18\times / 0.00121.46×1.46\times / 0.0028
ρ=0.75\rho=0.750.96×0.96\times / 0.00461.59×1.59\times / 0.01351.17×1.17\times / 0.00121.47×1.47\times / 0.0028
ρ=1.0\rho=1.00.97×0.97\times / 0.00461.60×1.60\times / 0.01351.17×1.17\times / 0.00121.47×1.47\times / 0.0028
grace G=4G=41.12×1.12\times / 0.00451.59×1.59\times / 0.01341.16×1.16\times / 0.00121.46×1.46\times / 0.0028
per-step reselection0.89×0.89\times / 0.00401.33×1.33\times / 0.01050.98×0.98\times / 0.00111.17×1.17\times / 0.0027

Table 10. One-factor sensitivity around the default on three further documents. Each cell gives per-request speedup over dense under native decoding, where every method, dense included, decodes its own 512-token trajectory, followed by KL to dense measured separately on dense’s denoising states for the same documents. The default ρ\rho is 0.5 on DreamReasoner and 1.0 on SDAR.

MethodDreamReasoner 16KDreamReasoner 32KSDAR 16KSDAR 32K
Dense46.9 / 92.743.8 / 93.86.2 / 51.018.8 / 68.8
RISA (default)46.9 / 92.740.6 / 92.76.2 / 51.012.5 / 65.6
MAGE 4K43.8 / 92.740.6 / 91.76.2 / 47.99.4 / 58.3
QUEST 4K46.9 / 94.834.4 / 84.46.2 / 42.76.2 / 56.2
LoSA 1K31.2 / 80.231.2 / 81.218.8 / 64.634.4 / 78.1
RISA KV reads (%)53.829.753.129.1
RISA HOT cap active (% of blocks)0100100100

Table 11. Multi-needle retrieval in long prompts. Three needles with two distractors in 16 documents, two layouts (n=32n=32 per cell). Each cell gives in-order exact match (all three needles answered in order) / per-needle recall (share of the three needles retrieved), in %. The final rows give RISA’s realized KV reads relative to dense and the share of its sparse blocks whose reused plan hits the HOT cap, averaged over requests. MAGE and QUEST use 4,096 tokens and LoSA 1,024.

MethodHotpotQALong-context MATH-500MATH-500GSM8K
DreamReasoner-8B
RISA (default)0.00870.00900.00310.0013
RISA, no recovery (q=0q = 0)0.01410.04790.02320.0013
MAGE 8K0.00550.00540.00030.0001
QUEST 8K0.00710.01150.00050.0001
LoSA 1K0.21010.16870.04560.0074
SDAR-8B-Chat
RISA (default)0.00120.00340.00030.0000
RISA, no recovery (q=0q = 0)0.00120.02690.00030.0000
MAGE 8K0.00100.00190.00020.0000
QUEST 8K0.00490.00500.00030.0000
LoSA 1K0.03960.18640.03460.0000

Table 12. Fidelity on benchmark prompts: KL to dense along dense’s own generation. In long-context MATH-500, a MATH-500 problem follows a long document, so generated history ages into COLD before the answer is complete.

Attention Profiling Protocol

Method16K32K64K128K
Dense3.2015.4389.82918.763
QUEST5.3555.2116.1237.772
MAGE2.7852.9545.28010.159
RISA2.1352.3482.7993.394

Table 13. Absolute efficiency results on DreamReasoner-8B. Attention latency in milliseconds per denoising step for the original configuration (QUEST and MAGE at 1,024 tokens), from one profiled source per context with block work amortized over 15.6 steps; 64K and 128K lie beyond the configured 32K window and measure execution only. Bold marks the fastest sparse method.

Table 13 records the absolute attention measurements underlying panels (a) and (c) of Figure 6. Attention includes selection, indices, merge, cache operations, and finalization attention. Block work uses the common 15.6-step horizon on DreamReasoner and 10.3-step horizon on SDAR.

Measurement scope. Kernels are assigned to their serving call sites. Captured kernels inherit same-name stage shares from an eager run of the same requests, while fused work is counted once. For the attention scaling in Figure 6, kernel profiles cover blocks 10–15 of a 512-token decode. For Figure 7, we fit kernel time as a function of denoising-step count across all output blocks after the first and amortize the fitted per-block cost over the model’s mean dense steps. Index totals the positive excess over dense in matrix multiplications, elementwise operations, and other kernels. Merge and Cache record the excess epilogue and memory operation time, respectively. The stacks sum kernel durations, including overlapping kernels such as asynchronous selection.

Memory footprint. Exact recovery does not imply KV compression. Peak allocation rises from dense’s 27.62 to 29.38 GiB at 16K and from 39.27 to 44.20 GiB at 64K. Packed copies, metadata, and scratch trade additional storage for fewer repeated reads, consistent with RISA’s focus on read efficiency rather than cache compression.

Block Execution Algorithm

The pseudocode below makes the ordering explicit: the seed forward observes the outgoing grace-window block before admission, the resulting seed queries build one historical read plan if denoising continues, and finalization remains a full-prefix forward pass. ADMIT applies the mass score in Equation (2) and the allocator in Section B.

Algorithm 1: RISA: denoising one block.

Input: Masked block xx; KV cache MM; statistics AA; frozen model FF

Output: Committed block xx; updated MM, AA

(M,a)←StageRecent⁡(M,G)(M, a) \leftarrow\operatorname{StageRecent}(M, G)

(z,Q,o)←SeedForward⁡(F,x,M,q)(z, Q, o) \leftarrow\operatorname{SeedForward}(F, x, M, q)

A←Update⁡(A,o)A \leftarrow\operatorname{Update}(A, o)

if a≠∅a \ne\varnothing then (M,A)←Admit⁡(M,a,A)(M, A) \leftarrow\operatorname{Admit}(M, a, A)

x←Commit⁡(x,z,0)x \leftarrow\operatorname{Commit}(x, z, 0); t←1t \leftarrow1

if Continue⁡(x,t)\operatorname{Continue}(x, t) then

foreach layer ℓ\ell and KV head hh do

(H,C,R)←Pages⁡(M,ℓ,h)(H, C, R) \leftarrow\operatorname{Pages}(M, \ell, h)

Hp←First4⁡(H)∪Last8⁡(H)H_p \leftarrow\operatorname{First4}(H) \cup\operatorname{Last8}(H); E←H∖HpE \leftarrow H \setminus H_p

u←Score⁡(Qℓ,h,E∪C)u \leftarrow\operatorname{Score}(Q_{\ell,h}, E \cup C); k←Budget⁡(∣E∣)k \leftarrow\operatorname{Budget}(\lvert E\rvert)

Sℓ,h←R∪rows⁡(Hp)∪Top⁡k(u,E)∪Top⁡q(u,C)S_{\ell,h} \leftarrow R \cup\operatorname{rows}(H_p) \cup\operatorname{Top}_k(u, E) \cup\operatorname{Top}_q(u, C)

while Continue⁡(x,t)\operatorname{Continue}(x, t) do

(z,o)←SparseForward⁡(F,x,M,S)(z, o) \leftarrow\operatorname{SparseForward}(F, x, M, S)

A←Update⁡(A,o)A \leftarrow\operatorname{Update}(A, o)

x←Commit⁡(x,z,t)x \leftarrow\operatorname{Commit}(x, z, t); t←t+1t \leftarrow t + 1

M←FinalizeKV⁡(F,x,M)M \leftarrow\operatorname{FinalizeKV}(F, x, M)

return (x,M,A)(x, M, A)

Additional Evaluation Details

Benchmark protocols. DreamReasoner-8B is evaluated on GSM8K, MATH-500, AIME25, LiveCodeBench v6, and HotpotQA; SDAR-8B-Chat is evaluated on GSM8K, MATH-500, and HotpotQA. All methods use identical prompts, tokenization, decoding settings, and generation limits within each model and benchmark. GSM8K and MATH use mathematical-answer scoring [5, 8]; LiveCodeBench [10] uses execution pass @1; HotpotQA uses LongBench token F1 [3]. For LiveCodeBench, the final fenced Python program must pass the benchmark tests under Python 3.12.13; missing code scores zero. AIME25 uses temperature 0.6 with a 32,768 total-position limit.

Configuration. The same frozen configuration is used across the reported tasks for each model. DreamReasoner uses G=8G = 8, Kmax⁡=512K_{\max} = 512, q=16q = 16, and ρ=0.5\rho= 0.5; SDAR uses the same history-management settings with ρ=0.75\rho= 0.75. Both use the learned admission correction. QUEST and MAGE use their stated 1,024-token budgets under the common carrier and decoder used throughout the main evaluation.

Motivation cohort. Motivation diagnostics use a fixed request cohort and sample every fourth eligible block at five layers and all KV heads. Recovery error is per-query relative Euclidean error against full-domain attention, with a 10−610^{-6} denominator floor. At quotas 16/32/64, additional HOT candidates are insufficient in 2.1/48.2/84.9% of layer/head states; these mismatches are retained rather than treated as successful matches in the reported summaries.

References

  1. [1]Marianne Arriola, Aaron Gokaslan, Justin Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sahoo, and Volodymyr Kuleshov. Block diffusion: Interpolating between autoregressive and diffusion language models. In International Conference on Learning Representations, volume 2025, pp. 50726–50753, 2025.
  2. [2]Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in neural information processing systems, 34:17981–17993, 2021.arxiv.org/abs/2107.03006
  3. [3]Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, et al. Longbench: A bilingual, multitask benchmark for long context understanding. In Proceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: Long papers), pp. 3119–3137, 2024.arxiv.org/abs/2308.14508
  4. [4]Shuang Cheng, Yihan Bian, Dawei Liu, Linfeng Zhang, Qian Yao, Zhongbo Tian, Wenhai Wang, Qipeng Guo, Kai Chen, Biqing Qi, and Bowen Zhou. Sdar: A synergistic diffusion-autoregression paradigm for scalable sequence generation, 2025. URL https://arxiv.org/abs/2510.06303.
  5. [5]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021. URL https://arxiv.org/abs/2110.14168.
  6. [6]Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. In International Conference on Learning Representations, volume 2024, pp. 35549–35562, 2024.arxiv.org/abs/2307.08691
  7. [7]Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in neural information processing systems, 35:16344–16359, 2022.arxiv.org/abs/2205.14135
  8. [8]Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset, 2021. URL https://arxiv.org/abs/2103.03874.
  9. [10]Naman Jain, Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. In Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (eds.), International Conference on Learning Representations, volume 2025, pp. 58791–58831, 2025. URL https://proceedings.iclr.cc/paper_files/paper/2025/file/94074dd5a072d28ff75a76dabed43767-Paper-Conference.pdf.arxiv.org/abs/2403.07974
  10. [11]Omin Kwon, Yeonjae Kim, Doyeon Kim, Minseo Kim, Yeonhong Park, and Jae W Lee. Mage: All-[mask] block already knows where to look in block diffusion llm. arXiv preprint arXiv:2602.14209, 2026.arxiv.org/abs/2602.14209
  11. [12]Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. Snapkv: Llm knows what you are looking for before generation. Advances in Neural Information Processing Systems, 37:22947–22970, 2024.arxiv.org/abs/2404.14469
  12. [13]Zichang Liu, Aditya Desai, Fangshuo Liao, Weitao Wang, Victor Xie, Zhaozhuo Xu, Anastasios Kyrillidis, and Anshumali Shrivastava. Scissorhands: Exploiting the persistence of importance hypothesis for llm kv cache compression at test time. Advances in Neural Information Processing Systems, 36:52342–52364, 2023.arxiv.org/abs/2305.17118
  13. [14]Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution. arXiv preprint arXiv:2310.16834, 2023.arxiv.org/abs/2310.16834
  14. [15]Xinyin Ma, Runpeng Yu, Gongfan Fang, and Xinchao Wang. dkv-cache: The cache for diffusion language models. Advances in Neural Information Processing Systems, 38:149009–149033, 2026.arxiv.org/abs/2505.15781
  15. [16]Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models. Advances in Neural Information Processing Systems, 38:50608–50646, 2026.arxiv.org/abs/2502.09992
  16. [17]Luka Ribar, Ivan Chelombiev, Luke Hudlass-Galley, Charlie Blake, Carlo Luschi, and Douglas Orr. Sparq attention: Bandwidth-efficient llm inference, 2024. URL https://arxiv.org/abs/2312.04985.
  17. [18]Subham S Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language models. Advances in Neural Information Processing Systems, 37:130136–130184, 2024.arxiv.org/abs/2406.07524
  18. [19]Yuerong Song, Xiaoran Liu, Ruixiao Li, Zhigen Liu, Zengfeng Huang, Qipeng Guo, Ziwei He, and Xipeng Qiu. Sparse-dllm: Accelerating diffusion llms with dynamic cache eviction, 2025. URL https://arxiv.org/abs/2508.02558.
  19. [20]Jiaming Tang, Yilong Zhao, Kan Zhu, Guangxuan Xiao, Baris Kasikci, and Song Han. Quest: Query-aware sparsity for efficient long-context llm inference. arXiv preprint arXiv:2406.10774, 2024.arxiv.org/abs/2406.10774
  20. [21]Zeqing Wang, Gongfan Fang, Xinyin Ma, Xingyi Yang, and Xinchao Wang. Sparsed: Sparse attention for diffusion language models. In International Conference on Learning Representations, volume 2026, pp. 99320–99337, 2026.arxiv.org/abs/2509.24014
  21. [22]Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding. In International Conference on Learning Representations, volume 2026, pp. 57027–57051, 2026a.arxiv.org/abs/2606.19257
  22. [23]Zirui Wu, Lin Zheng, Jiacheng Ye, Shanshan Gong, Xueliang Zhao, Yansong Feng, Wei Bi, and Lingpeng Kong. Dreamreasoner-8b: Block-size curriculum learning for diffusion reasoning models. arXiv preprint arXiv:2606.19257, 2026b.
  23. [24]Haocheng Xi, Harman Singh, Yuezhou Hu, Coleman Hooper, Rishabh Tiwari, Aditya Tomar, Minjae Lee, Wonjun Kang, Michael Mahoney, Chenfeng Xu, et al. Losa: Locality aware sparse attention for block-wise diffusion language models. arXiv preprint arXiv:2604.12056, 2026.
  24. [25]Chaoyin Xiao, Pengle Zhang, Xu Han, Guangxuan Xiao, Yankai Lin, Zhengyan Zhang, Zhiyuan Liu, and Maosong Sun. Inflmm: Training-free long-context extrapolation for llms with an efficient kv cache and parallel decoding. Advances in neural information processing systems, 37:119638–119661, 2024a.arxiv.org/abs/2309.17453
  25. [26]Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. In International Conference on Learning Representations, volume 2024, pp. 21875–21895, 2024b.arxiv.org/abs/2410.10819

Paper details

vlnc:W_bh65aqvn
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 3

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Generative AI tools

View statement
Looking for related papers…
Contents