betaFeedback

WorksComputer Science

DecoupAttn: Decoupled Sparse Attention for Training-Free Acceleration of Video Diffusion

Abstract

Sparse attention accelerates few-step video diffusion transformers by computing a few key blocks exactly and compensating for the rest. However, existing sparse attention still faces key challenges, including identification overhead from finding critical blocks at every denoising step, suboptimal budget allocation that sets sparsity budgets uniformly or by local metrics, and cross-step error accumulation as reused masks or clusters go stale. Observing that the errors left on skipped blocks cancel each other once every key is compensated, we propose DecoupAttn, a new paradigm of decoupled sparse attention: semantic clusters compensate every key, and exact attention corrects their error on a few blocks fixed once offline by position. To support this paradigm, we introduce Distortion Budget, a rate-distortion objective for budget allocation that places exact computation across timesteps, layers, and heads where it removes the most distortion that compensation leaves in the final latent. Built on this budget, we further propose Reassign Cache, in which each step reassigns the drifting keys to cached centroids and refreshes compensation while the budgeted blocks stay fixed. Moreover, DecoupAttn is training-free: a few calibration clips compile one plan offline, and inference runs standard block-sparse kernels with one fused compensation kernel. Our code will be released after submission.

Introduction

In recent years, video diffusion transformers have been applied to a wide range of generation tasks, from text-to-video synthesis [34] to controlled generation [24] and neural rendering [15], gradually moving from many-step samplers toward few-step distilled generators [6, 44]. Sparse attention makes these generators fast without retraining: since attention grows quadratically with video length, it computes a few key blocks exactly and compensates for the rest, keeping near-dense fidelity (Figure 1). However, existing methods tie compensation to block selection: the groups that pick the exact blocks also summarize the skipped keys, so compensation is costly at every step of a few-step sampler and impossible once the selection is fixed offline.

Figure 1. DecoupAttn keeps dense-attention quality at a lower latency. Frames from dense attention and DecoupAttn on Cosmos-Predict2.5 (4 steps) and HunyuanVideo (50 steps), with latency, speedup on all ten models (filled: these two), and PSNR and VBench against speedup for DecoupAttn and the seven baselines of Table 1, averaged over the ten models.

Sparse attention for video generation has followed three main lines of work, as shown in Figure 2. First, online routing (dynamic sparse attention) identifies critical blocks at every step from the current queries and keys, by scoring pooled blocks [39, 45] or clustering tokens by semantics [33, 41], treating sparsity as a property of each input. Second, offline calibration (static sparse attention) fixes sparse patterns before deployment, measured on a few prompts [43] or derived from how attention decays with distance [13], treating sparsity as a property of the model. Third, compensation approximates the skipped blocks instead of dropping them, from cluster centroids [53], block-wise Taylor expansions [12], or fitted residuals [32], recov ering much of the attention that selection leaves out.

Figure 2. Three lines of sparse attention and DecoupAttn. (a) Online routing re-picks exact blocks each step and drops the rest, (b) offline calibration fixes them but drops the rest, and (c) compensation reuses them and reroutes every step. (d) DecoupAttn fixes them offline and compensates every key by semantic clusters. Right: schematic error and time against DecoupAttn (dashed), excess in red.

However, these methods still face three challenges. (i) Identification overhead. Online routing and the compensation built on its groups identify critical blocks at every denoising step [11, 12, 17, 32, 33, 37, 39, 40, 41, 45, 48, 49, 53], an overhead that can offset the gains of sparsity in few-step samplers, while static patterns fixed in advance [13, 43, 46] discard the remaining blocks and incur information loss. (ii) Suboptimal budget allocation. Existing methods set sparsity budgets across heads, layers, and timesteps uniformly or by local metrics such as retained attention mass [20, 41, 43, 48, 53], or by the effect of dropping blocks on the output [38, 51, 52]. Retained attention mass, however, does not determine the output error [32], which depends on the residual left after compensation and on how unevenly later layers and steps amplify it, so exact computation is spent where it barely changes the final video. (iii) Cross-step error accumulation. To avoid redundant recomputation, recent methods reuse masks, clusters, or earlier centroids across steps, or predict masks ahead [2, 10, 16, 18, 20, 29, 40, 41], but attention patterns evolve across denoising steps, so the reused groups go stale and their errors accumulate, while refreshing them repeats the identification.

Our key observation is that once every key is compensated, the residuals of the skipped blocks point in many directions and cancel each other like dithered quantization errors [27], so blocks fixed once offline match blocks reselected at every step. Based on this observation, we propose DecoupAttn, a new paradigm of decoupled sparse attention: semantic clusters compensate every key, and exact attention corrects their residual on a few blocks fixed once offline by position (Figure 2d). Selecting blocks costs nothing at inference, and the residual of every fixed block can be priced offline. Few-step samplers amplify this residual unevenly, so we introduce Distortion Budget, a rate–distortion objective for budget allocation that prices every timestep, layer, and head by the distortion the residual leaves in the final latent. Built on this budget, we further propose Reassign Cache, which limits the growth of these residuals by reassigning the drifting keys to cached centroids at every step, while the blocks and the budget stay fixed. DecoupAttn thus moves block identification and budget allocation into one plan compiled offline with no training, and inference only refreshes the clusters, running standard block-sparse kernels and one fused compensation kernel.

We evaluate DecoupAttn on ten video generation, controlled generation, and neural rendering models, where it outperforms online-routing, offline-calibrated, and compensation-based methods on nine of them and delivers a substantial speedup over dense attention.

Related Work

Summarizing skipped attention. Transformers summarize attention by clusters, near- and far-field splits, or sparse-plus-low-rank estimators [1, 14, 21, 25, 35, 50], and video sparse attention compensates the skipped keys by cluster or block means [17, 53], Taylor terms [12, 48], or fitted or cached residuals [2, 32, 40]. Masks frozen early or predicted ahead [16, 18] drop the keys outside them, and routers that reuse centroids [10, 20, 41] still let the clusters pick the blocks. DecoupAttn summarizes the skipped keys by semantic clusters that never pick blocks.

Allocating computation by sensitivity. Coders and quantizers allocate bits by rate–distortion cost or layer sensitivity [3, 30, 31], and sparse attention sets density by retained attention mass, profiling, sampler curvature, or block error [20, 41, 43, 48, 53], or by each unit’s effect on the output [38, 51, 52]. Distortion Budget instead prices the residual left after compensation, not the error of dropping a block, by its effect on the final latent.

Preliminaries

Definition 1: Block-sparse attention call. A self-attention call at denoising step tt, layer ll, and head hh maps the queries qiq_i, keys kjk_j, and values vjv_j of its LL tokens to outputs yiy_i, with the tokens partitioned by position into query tiles gg and key blocks bb, the units of block-sparse kernels [42]:

yi=NiZi=∑bNib∑bZib,Nib=∑j∈besijvj,Zib=∑j∈besij(1)y_i = \frac{N_i}{Z_i} = \frac{\sum_b N_{ib}}{\sum_b Z_{ib}}, \qquad N_{ib} = \sum_{j \in b} e^{s_{ij}}v_j, \qquad Z_{ib} = \sum_{j \in b} e^{s_{ij}} \tag*{(1)}

where sij=qiTkj/ds_{ij}=q_i^\mathsf{T}k_j/\sqrt{d} and dd is the head dimension. A sparse call computes NibN_{ib} and ZibZ_{ib} exactly on a block set SgS_g and summarizes the rest. Existing compensation writes N^i=∑b∈SgNib+∑b∉SgN~ib\hat{N}_i=\sum_{b\in S_g}N_{ib}+\sum_{b\notin S_g}\tilde{N}_{ib} with N~ib\tilde{N}_{ib} from the statistics of block bb itself, and every change of SgS_g rebuilds the compensation terms [12, 53].

Definition 2: Sparse attention configuration. A sampler runs TT denoising steps. Each head of each call is a unit u=(t,l,h)u=(t,l,h) of the unit set UU, and the heads of a call (t,l)(t,l) share one compensation estimator. A configuration, or plan, π=({ou}u∈U,{ηtl})\pi=(\{o_u\}_{u\in U},\{\eta_{tl}\}) assigns every unit an option and every call an estimator, which determines the final latent and the density:

π⇒x^0(π),X(π)=∑uwuXu(ou)∑uwu,(2)\pi\Rightarrow\hat{x}_0(\pi), \qquad X(\pi)=\frac{\sum_u w_uX_u(o_u)}{\sum_u w_u}, \tag*{(2)}

where ou∈Oo_u\in\mathcal{O} is the unit’s option (Eq. (8)), ηtl∈{FULL,CENTROID}\eta_{tl}\in\{\mathrm{FULL},\mathrm{CENTROID}\} the call’s estimator (Eq. (4)), XuX_u the density of an option, the fraction of blocks it executes with every block padded to the kernel block size and the DENSE option counted as one, and wuw_u the number of sampler steps the unit represents. The first step and the first layer always stay dense and lie outside UU, and the estimator adds a fixed overhead to XX (Appendix § 8.9).

Problem statement. Given a pretrained video diffusion transformer with frozen weights and a few calibration clips, we seek a plan that is fixed before deployment, that keeps the density within a target ρ\rho, and that keeps the final latent closest to the dense one:

π⋆=arg⁡min⁡π Ex∥x^0(π)−x0∥22s.t.X(π)≤ρ,(3)\pi^\star=\underset{\pi}{\arg\min}\ \mathbb{E}_x\lVert\hat{x}_0(\pi)-x_0\rVert_2^2 \quad\mathrm{s.t.}\quad X(\pi)\leq\rho, \tag*{(3)}

where xx is the input (prompt, conditions, and seed) and x0x_0 the dense final latent for the same input.

Method

As Figure 3 shows, DecoupAttn centers on one quantity, the residual that compensation leaves on each key block: the decoupled estimator makes this residual the only error that reaches the output and keeps it small (§ 4.1), Distortion Budget spends exact computation where the residual most distorts the final latent (§ 4.2), and Reassign Cache limits its growth as keys drift (§ 4.3).

Figure 3. How DecoupAttn computes one attention call (Cosmos-Predict2.5). (a) Reassign Cache clusters the keys once, caches the centroids, and at every later step moves each key to its nearest cached centroid and refreshes the cluster statistics, (b) Distortion Budget prices the options of every unit offline and keeps the cheapest, which fixes the exact blocks SgS_g, and (c) exact attention on SgS_g and centroid compensation of every other key meet only through the counts ngcn_{gc} and share one softmax denominator (pink: approximation, blue: exact).

Decoupled Sparse Attention

To keep compensation when the selection is fixed offline, decoupled sparse attention follows the difference estimators of survey sampling [26]: a cheap algorithm predicts the contribution of every key, and exact attention on a few blocks replaces the prediction there (Figure 3c).

Centroid compensation with two corrections. At the first sparse step, the keys of every head are clustered by k-means into clusters cc, and every later call summarizes each cluster by its centroid with two second-order corrections computed from the cluster’s current members:

ωic=ϕic(1+γic),aic=ωicvˉc+ϕicHqid,(4)\omega_{ic}=\phi_{ic}(1+\gamma_{ic}), \qquad a_{ic}=\omega_{ic}\bar{v}_c+\phi_{ic}\frac{Hq_i}{\sqrt{d}}, \tag*{(4)}

where ϕic=eqiTkˉc/d\phi_{ic}=e^{q_i^\mathsf{T}\bar{k}_c/\sqrt{d}} is the centroid weight, and kˉc\bar{k}_c, vˉc\bar{v}_c, and σc2\sigma_c^2 are the cluster’s mean key, mean value, and per-dimension key variance. The variance correction γic=12d∑pqip2σcp2\gamma_{ic}=\frac{1}{2d}\sum_p q_{ip}^2\sigma_{cp}^2 is the diagonal part of the second-order term of eqiTk/de^{q_i^\mathsf{T}k/\sqrt{d}} around kˉc\bar{k}_c and restores this part of the mass that a centroid misses on a spread-out cluster (Lemmas A.3 and A.4). The within-cluster cross-covariance H=1L(∑jvjkjT−∑c∣c∣vˉckˉcT)H=\frac{1}{L}\left(\sum_jv_jk_j^\mathsf{T}-\sum_c|c|\bar{v}_c\bar{k}_c^\mathsf{T}\right), pooled over the clusters of a head, gives their shared first-order value correction (Lemma A.5). The CENTROID estimator drops both corrections.

Decoupled estimator. Each query tile gg computes its exact blocks SgS_g in full and covers every other key with its cluster centroid, and both parts share one softmax denominator:

y^i=N^iZ^i=∑b∈SgNib+∑cngc aic∑b∈SgZib+∑cngc ωic,(5)\hat{y}_i=\frac{\hat{N}_i}{\hat{Z}_i}=\frac{\sum_{b\in S_g}N_{ib}+\sum_c n_{gc}\,a_{ic}}{\sum_{b\in S_g}Z_{ib}+\sum_c n_{gc}\,\omega_{ic}}, \tag*{(5)}

References

  1. [1]Beidi Chen, Tri Dao, Eric Winsor, Zhao Song, Atri Rudra, and Christopher Ré. Scatterbrain: Unifying sparse and low-rank attention approximation. In NeurIPS, 2021. 2, 4, 5arxiv.org/abs/2110.15343
  2. [2]Jiarui Chen, Zeqiang Lai, Jiangshan Wang, Ziheng Ouyang, Ye Huang, Xiangyu Yue, Cewu Lu, and Chunchao Guo. MC-Sparse: Deconstructing and closing the dense-sparse attention gap in diffusion transformers. arXiv preprint arXiv:2610.06801, 2026. 2arxiv.org/abs/2610.06801
  3. [3]Zhen Dong, Zhewei Yao, Amir Gholami, Michael W. Mahoney, and Kurt Keutzer. HAWQ: Hessian Aware quantization of neural networks with mixed-precision. In ICCV, 2019. 2, 5arxiv.org/abs/1905.03696
  4. [4]Hugh Everett III. Generalized Lagrange multiplier method for solving problems of optimum allocation of resources. Operations Research, 11(3):399–417, 1963. 5, 7DOI
  5. [5]Kai He, Ruofan Liang, Jacob Munkberg, Jon Hasselgren, Nandita Vijaykumar, Alexander Keller, Sanja Fidler, Igor Gilitschenski, Zan Gojcic, and Zian Wang. UniRelight: Learning joint decomposition and synthesis for video relighting. In NeurIPS, 2025. 6arxiv.org/abs/2506.15673
  6. [6]Xun Huang, Zhenqiu Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion. In NeurIPS, pages 167283–167308, 2025. 1arxiv.org/abs/2506.08009
  7. [7]Ziqi Huang, Yinah He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Comprehensive benchmark suite for video generative models. In CVPR, 2024. 7, 22arxiv.org/abs/2311.17982
  8. [8]Zheng-Hui Huang, Zhixiang Wang, Jiaming Tan, Ruihan Yu, Yidan Zhang, Bo Zheng, Yu-Lun Liu, Yung-Yu Chuang, and Kaipeng Zhang. Generative world renderer. arXiv preprint arXiv:2604.02329, 2026. 6arxiv.org/abs/2604.02329
  9. [9]Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. HunyuanVideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024. 5arxiv.org/abs/2412.03603
  10. [10]Dongyeun Lee, Amir Zandieh, Vahab Mirrokni, Junmo Kim, and Insu Han. HyperVAttention: Efficient sparse attention with spatio-temporal clustering for video diffusion. arXiv preprint arXiv:2607.03012, 2026. 2arxiv.org/abs/2607.03012
  11. [11]Haopeng Li, Yitong Li, Junsong Chen, Tian Ye, Haozhe Liu, Jincheng Yu, Duomin Wang, Ruihua Zhang, Zeke Xie, Enze Xie, and Song Han. Sol-Attn: Accelerating video generation inference via on-the-fly attention sparsification. arXiv preprint arXiv:2607.24027, 2026. 2, 6arxiv.org/abs/2607.24027
  12. [12]Haopeng Li, Shitong Shao, Wenliang Zhong, Zikai Zhou, Lichen Bai, Hui Xiong, and Zeke Xie. PISA: Piecewise sparse attention is wiser for efficient diffusion transformers. In NeurIPS, 2026. arXiv:2602.01077. 1, 2, 3, 6arxiv.org/abs/2602.01077
  13. [13]Xinyang Li, Muyang Li, Tianle Cai, Haocheng Xi, Shuo Yang, Yujun Lin, Liyun Zhang, Songlin Yang, Jinbo Hu, Kelly Peng, Maneesh Agrawala, Ion Stoica, Kurt Keutzer, and Song Han. Radial attention: O(n log n) sparse attention with energy decay for long video generation. In NeurIPS, 2025. 1, 2
  14. [14]Yu Li, Binxu Li, and Tian Lan. MomentKV: Closing the directional gap in KV cache eviction for long-context inference. arXiv preprint arXiv:2606.01563, 2026. 2, 4arxiv.org/abs/2606.01563
  15. [15]Ruofan Liang, Zan Gojcic, Huan Ling, Jacob Munkberg, Jon Hasselgren, Chih-Hao Lin, Jun Gao, Alexander Keller, Nandita Vijaykumar, Sanja Fidler, and Zian Wang. Diffusion renderer: Neural inverse and forward rendering with video diffusion models. In CVPR, pages 26069–26080, 2025. 1, 6arxiv.org/abs/2501.18590
  16. [16]Enhuai Liu, Yunke Wang, Yutong Wang, Changming Sun, and Chang Xu. LoSA: Near-lossless sparse attention for training-free video diffusion acceleration. arXiv preprint arXiv:2608.12032, 2026. 2arxiv.org/abs/2608.12032
  17. [17]Xuewen Liu, Zikai Li, Jing Zhang, Mengjuan Chen, and Qingyi Gu. Rectified SpaAttn: Revisiting attention sparsity for efficient video generation. arXiv preprint arXiv:2511.19835, 2025. 2arxiv.org/abs/2511.19835
  18. [18]Yuxi Liu, Yipeng Hu, Zekun Zhang, Kunze Jiang, and Kun Yuan. Mixture of distributions matters: Dynamic sparse attention for efficient video diffusion transformers. arXiv preprint arXiv:2601.11641, 2026. 2arxiv.org/abs/2601.11641
  19. [19]Stuart P. Lloyd. Least squares quantization in PCM. IEEE Transactions on Information Theory, 28(2):129–137, 1982. 5, 8DOI
  20. [20]Jiayi Luo, Jiayu Chen, Jiankun Wang, Cong Wang, Hanxin Zhu, Qingyun Sun, Chen Gao, Zhibo Chen, and Jianxin Li. Attention sparsity is input-stable: Training-free sparse attention for video generation via offline sparsity profiling and online QK co-clustering. In ICML, pages 82177–82195, 2026. 2, 6arxiv.org/abs/2603.18636
  21. [21]Tan M. Nguyen, Vai Suliafu, Stanley J. Osher, Long Chen, and Bao Wang. FMMformer: Efficient and flexible transformer via composed near-field and far-field attention. In NeurIPS, 2021. 2
  22. [22]NVIDIA. Cosmos 3: Omnimodal world models for physical AI. arXiv preprint arXiv:2606.02800, 2026. 5
  23. [23]NVIDIA, Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, et al. Cosmos world foundation model platform for physical AI. arXiv preprint arXiv:2501.03575, 2025. 5
  24. [24]NVIDIA, Arslan Ali, Junjie Bai, Maciej Bala, Yogesh Balaji, Aaron Blakeman, Tiffany Cai, Jiaxin Cao, Tianshi Cao, et al. World simulation with video foundation models for physical AI. arXiv preprint arXiv:2511.00062, 2025. 1, 5, 6
  25. [25]Aurko Roy, Mohammad Saffar, Ashish Vaswani, and David Grangier. Efficient content-based sparse attention with routing transformers. Transactions of the Association for Computational Linguistics, 9:53–68, 2021. 2
  26. [26]Carl-Erik Särndal, Bengt Swensson, and Jan Wretman. Model Assisted Survey Sampling. Springer-Verlag, New York, 1992. 3, 5
  27. [27]Leonard Schuchman. Dither signals and their effect on quantization noise. IEEE Transactions on Communication Technology, 12(4):162–165, 1964. 2, 4, 5DOI
  28. [28]Jay Shah, Ganesh S. Kandbid, Ying Zhang, Vijay Thakkar, Pradeep Ramani, and Tri Dao. FlashAttention-3: Fast and accurate attention with asynchronous and low-precision. In NeurIPS, 2024. 7arxiv.org/abs/2407.08608
  29. [29]Dor Shmilovich, Tony Wu, Aviad Dahan, and Yuval Domb. LiteAttention: A temporal sparse attention for diffusion transformers. arXiv preprint arXiv:2511.11062, 2025. 2arxiv.org/abs/2511.11062
  30. [30]Yair Shoham and Allen Gersho. Efficient bit allocation for an arbitrary set of quantizers. IEEE Transactions on Acoustics, Speech, and Signal Processing, 36(9):1445–1453, 1988. 2, 4DOI
  31. [31]Gary J. Sullivan and Thomas Wiegand. Rate-distortion optimization for video compression. IEEE Signal Processing Magazine, 15(6):74–90, 1998. 2, 4DOI
  32. [32]Pardis Taghavi, Reza Langari, and Gaurav Pandey. Partition the support, reconstruct the residual: Training-free sparse attention for video generation and world models. arXiv preprint arXiv:2608.18484, 2026. 1, 2, 6arxiv.org/abs/2608.18484
  33. [33]Haoyue Tan, Shengnan Wang, Yulin Qiao, Juncheng Zhang, Youhui Bai, Ping Gong, Zewen Jin, and Cheng Li. Ada-Cluster: Adaptive query-key clustering for sparse attention in video generation. In CVPR, 2026. 1, 2DOI
  34. [34]Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025. 1, 5arxiv.org/abs/2503.20314
  35. [35]Apoorv Vyas, Angelos Katharopoulos, and Francois Fleuret. Fast transformers with clustered attention. In NeurIPS, 2020. 2
  36. [36]Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004. 7pubmed.ncbi.nlm.nih.gov/15376593
  37. [37]Haocheng Xi, Shuo Yang, Yilong Zhao, Chenfeng Xu, Muyang Li, Xiuyu Li, Yujun Lin, Han Cai, Jintao Zhang, Dacheng Li, Jianfei Chen, Ion Stoica, Kurt Keutzer, and Song Han. Sparse VideoGen: Accelerating video diffusion transformers with spatial-temporal sparsity. In ICML, 2025. 2arxiv.org/abs/2502.01776
  38. [38]Tianze Xia, Xiaoling Zhou, Ruya Lan, Yiqiao Liao, Chengru Song, and Shiming Xiang. Sparsity should be AL-LOTted by output: Training-free allocation of block-sparse attention for video diffusion transformers. arXiv preprint arXiv:2610.29441, 2026. 2, 3arxiv.org/abs/2610.29441
  39. [39]Ruyi Xu, Guangxuan Xiao, Haofeng Huang, Junxian Guo, and Song Han. XAttention: Block sparse attention with antidiagonal scoring. In ICML, 2025. 1, 2arxiv.org/abs/2503.16428
  40. [40]Tiange Xu, Kai Liu, Linghe Kong, and Yulun Zhang. RECAP: Reciprocal sparse attention and hierarchical caching for efficient video diffusion. arXiv preprint arXiv:2610.26244, 2026. 2arxiv.org/abs/2610.26244
  41. [41]Shuo Yang, Haocheng Xi, Yilong Zhao, Muyang Li, Jintao Zhang, Han Cai, Yujun Lin, Xiuyu Li, Chenfeng Xu, Kelly Peng, Jianfei Chen, Song Han, Kurt Keutzer, and Ion Stoica. Sparse VideoGen2: Accelerate video generation with sparse attention via semantic-aware permutation. In NeurIPS, 2025. 1, 2, 6
  42. [42]Zihao Ye, Legun Chen, Ruihang Lai, Wuwei Lin, Yieng Zhang, Stephanie Wang, Tianqi Chen, Baris Kasikci, Vinod Grover, Arvind Krishnamurthy, and Luis Ceze. FlashInfer: Efficient and customizable attention engine for LLM inference serving. In MLSys, 2025. 3, 7arxiv.org/abs/2501.01005
  43. [43]Shai Yehezkel, Shahar Yadin, Noam Elata, Yaron Ostrovsky-Berman, and Bahjat Kawar. Accelerating text-to-video generation with calibrated sparse attention. arXiv preprint arXiv:2603.05503, 2026. 1, 2, 6, 13, 14arxiv.org/abs/2603.05503
  44. [44]Tianwei Yin, Qiang Zhang, Richard Zhang, William T. Freeman, Frédo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast autoregressive video diffusion models. In CVPR, pages 22963–22974, 2025. 1arxiv.org/abs/2412.07772
  45. [45]Jintao Zhang, Chendong Xiang, Haofeng Huang, Jia Wei, Haocheng Xi, Jun Zhu, and Jianfei Chen. SpargeAttention: Accurate and training-free sparse attention accelerating any model inference. In ICML, 2025. 1, 2arxiv.org/abs/2502.18137
  46. [46]Peiyuan Zhang, Yongqi Chen, Runlong Su, Hangliang Ding, Ion Stoica, Zhengchong Liu, and Hao Zhang. Fast video generation with sliding tile attention. In ICML, 2025. 2arxiv.org/abs/2502.04507
  47. [47]Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018. 7arxiv.org/abs/1801.03924
  48. [48]Wentai Zhang, Ronghui Xi, Shiyao Peng, Jiayu Huang, Haoran Luo, Zichen Tang, and Haihong E. Ride the wave: Precision-allocated sparse attention for smooth video generation. arXiv preprint arXiv:2604.12119, 2026. 2, 14, 16
  49. [49]Qi Zhao, Qirui Li, Hanlin Tang, Yiduo Li, Zhen Guo, Cuifeng Shen, Chao Xu, Zhaosheng Chi, Xiaojin Lu, Kan Liu, Tao Lan, Lin Qu, and Xi Li. SCOPE: Subspace clustering with online per-head top-k estimation for sparse video attention. arXiv preprint arXiv:2608.12780, 2026. 2arxiv.org/abs/2608.12780
  50. [50]Lin Zheng, Jianbo Yuan, Chong Wang, and Lingpeng Kong. Efficient attention via control variates. In ICLR, 2023. 2, 4, 5arxiv.org/abs/2302.04542
  51. [51]Xuzhe Zheng, Yuexiao Ma, Jing Xu, Xiawu Zheng, Rongrong Ji, and Fei Chao. HEART: Exploiting head heterogeneity in sparse attention for video diffusion. arXiv preprint arXiv:2605.14513, 2026. 2, 3arxiv.org/abs/2605.14513
  52. [52]Ruiliang Zhou, Xuecheng Wu, Kang He, Guangyun Han, Bin Liu, Qinqin Chen, Wende Xu, Qingjie Zhao, and Chengru Song. ScalingAttention: Discovering intrinsic sparse attention topology for video diffusion transformers. arXiv preprint arXiv:2606.23019, 2026. 2, 3arxiv.org/abs/2606.23019
  53. [53]Xuanyi Zhou, Qiyuang Mang, Shuo Yang, Haocheng Xi, Jintao Zhang, Huan zhi Mao, Joseph E. Gonzalez, Kurt Keutzer, Ion Stoica, and Alvin Cheung. SVG-EAR: Parameter-free linear compensation for sparse video generation via error-aware routing. In ECCV, pages 167–185, 2026. 1, 2, 3, 6arxiv.org/abs/2603.08982

Paper details

vlnc:W_vy6xscda
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 3

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Claude

View statement
Looking for related papers…
Contents