WorksComputer Science
ReelCache: An Efficient KV Cache Compression and Management System for Multi-Round Video Generation
Abstract
Autoregressive video diffusion models produce large KV caches during multi-round video generation. Keeping all these KV chunks would consume substantial memory, while rebuilding discarded caches would significantly delay video chunk generation. This paper presents ReelCache, a KV cache compression system for efficient multi-round, long video generation. ReelCache combines an intra-chunk compressor with an inter-chunk cache manager. The compressor applies low-rank decomposition across channels and video coding to the resulting coefficients, predicting the decomposition rank per transformer layer online to meet retained-energy targets. The cache manager shares basis directions across consecutive chunks and organizes chunk states in a tree, allowing generation branches to share historical state while supporting restoration for resume, rewind, and fork. Experiments across five video generation models show that ReelCache reduces KV storage by 14.46-21.13x relative to baselines and accelerates next-chunk generation by 1.25-3.36x, with less than 1% drop in six representative VBench metrics. GitHub: https://github.com/Tianyu9748/ReelCache
Introduction
Video generation tools such as Google Veo (Google DeepMind, [11]), ByteDance Seedance (Team Seedance et al., 2026), and Adobe Firefly (Adobe Inc., [1]) empower users to create video clips from text descriptions or reference images (Team Wan et al., [35]; Brooks et al., [2]). These tools make it possible to produce professional visual content without extensive animation experience, significantly lowering the entry barrier to video creation. When creating a longer video, a user may start with a short clip and extend it through successive rounds of generation. Each continuation needs to build on the preceding content, preserving the appearance of recurring characters and objects and the continuity of the scene.
Autoregressive video diffusion models support this continuity by generating video chunk by chunk, with each new chunk conditioned on preceding content (Yin et al., [42]; Huang et al., [13]; Yang et al., 2025; Sand.ai et al., [33]). To avoid repeatedly processing preceding chunks, these systems cache their attention keys and values (KV). When generating a new chunk, the model attends directly to the cached KV rather than recomputing these representations from the historical video latents. However, retaining these per-layer KV representations consumes substantial GPU memory. For example, in our benchmark with LongLive-1.0-1.3B at 1080p resolution, the KV cache for five seconds of video occupies 45 GB memory (§ 2.2). Keeping multiple
such histories would quickly exceed GPU memory capacity.
Multi-round video creation makes this problem more demanding (§ 2.1). When generating a longer video, users typically review a few seconds or tens of seconds of generated video before deciding how to proceed. For example, they may adjust the prompt for subsequent chunks, return to an earlier point to replace an unsatisfactory segment, or explore alternative continuations. These operations can revisit generation points whose KV is no longer in the active cache. The system should therefore either retain extended KV for later reuse which is memory intensive, or reconstruct it when needed which comes with significant computing delay.
In this paper, we present ReelCache, a KV-cache compression and management system for multi-round, long video generation. ReelCache compresses each chunk’s KV as it is generated, reducing the storage needed to retain long generation histories. When a user edits an earlier chunk, it retrieves and restores the compressed KV of the required preceding chunks, avoiding costly recomputation from historical video latents. It uses CPU/NVENC for video encoding and dedicated GPU video decoders for coefficient decoding, limiting interference with ongoing video generations.
ReelCache’s design is guided by three key findings from our characterization of video KV caches: complementary channel and spatiotemporal redundancy, rank requirements that vary across prompts, layers, and models, and shared low-rank structure across consecutive chunks (§ 3).
Our first finding is that video KV contains complementary redundancy across channels and spatial and temporal positions (§ 3.1). Prior work has used low-rank decomposition and video codecs for LLM KV compression [34, 4] (Xu et al. 2025; Mi et al. 2026). Our characterization of video KV reveals how these techniques can work well together: low-rank decomposition reduces channel redundancy, while the resulting coefficient channels retain spatial and temporal correlations that a video codec can further exploit. We also find that, at the same rank, compressing keys before RoPE better preserves subsequent video generation quality than compressing them after RoPE. These findings guide us to design an intra-chunk compressor that factorizes pre-RoPE keys and values, then encodes their coefficient channels with a video codec (§ 4.1). The combination reduces storage beyond either method alone.
Nevertheless, our measurements also show that keys and values differ in their sensitivity to compression, and that a fixed retained-energy target requires substantially different ranks across layers, prompts, and models. A uniform rank (for low-rank decomposition-based compression) therefore either wastes storage on some KV matrices or discards too much information from others (§ 3.2). Computing a full singular spectrum to select each rank, however, would add substantial computing overhead. In ReelCache we propose a small-rank one-time energy measurement coupled with a learned rank-energy relationship to predict the required rank, then verify the result against the energy target (§ 4.2). This makes per-matrix rank selection possible with significantly lower computation overhead.
Our second finding is that although neighboring video chunks’ raw KV tensors differ substantially, their channel subspaces often remain similar. This creates an opportunity to share a common basis among consecutive KV caches while storing separate coefficients for each chunk (§ 3.3). To exploit this opportunity, ReelCache checks whether the previous chunk’s basis captures enough energy in the new chunk’s KV matrix. If so, the new chunk stores only its own coefficients and references the shared basis. Otherwise, ReelCache adds directions to capture the residual, or fits a new basis if doing so requires less storage. To support multi-round video editing, we further propose an inter-chunk manager that organizes chunks into a chunk state tree, where branches represent alternative continuations and share their common history. Each node stores the chunk’s encoded coefficients and references to its shared basis. Coefficients are encoded separately for each chunk, allowing it to be restored using its referenced basis without decoding earlier chunks’ coefficients (§ 4.3).
We implement ReelCache in 21K LoC and integrate it into SGLang-Diffusion [45]. The head-to-head comparison against two baselines across five video generation models shows that ReelCache reduces KV storage by up to and accelerates next chunk generation by up to , without noticeable video quality degradation.
This paper makes the following contributions:
We characterize the compression opportunities in video KV caches across channels, spatial and temporal positions, and consecutive chunks through benchmark studies.
Guided by the findings from these benchmarks, we design ReelCache, an efficient KV-cache compression system that exploits both inner- and inter-chunk KV redundancy.
We integrate ReelCache into SGLang-Diffusion and demonstrate it consistently outperforms baselines.
Related Works
KV-cache compression for LLMs. KV cache takes significant memory on long-context LLM inference. Prior works can be divided into the following categories: token eviction/discarding keeps only a budget of “important” tokens and drops the rest [43, 38, 3] (Li et al. 2024b); quantization lowers bits per element [12, 17] (Li et al. 2024b); low-rank/structural methods project tensors, which lie in a low-rank subspace, onto a small basis [4, 34]. Close to ReelCache, LLM.265 (Xu et al., 2025) shows H.264/H.265 codec can be leveraged to quantize LLM KV cache to bits, thereby reducing storage overhead. KVCodec (Mi et al., 2026) further adds a codec-friendly layout and a GPU-native NVENC/NVDEC pipeline for lossless remote KV reuse and migration.
ReelCache differs from these works in two aspects. First, these works optimize LLM decode bandwidth per step subject to accuracy, whereas ReelCache optimizes persistent bytes per branchable snapshot subject to reproducibility. Second, the codec line runs on the raw tensor and never removes the cross-channel redundancy that, for video KV that ReelCache targets at, is the larger one (§ 2).
Inter-step activation caching for faster diffusion inference. Prior works [26, 44] exploit similarity between intermediate activations across denoising steps to avoid repeating selected computations during generation. ReelCache addresses a different but complementary source of overhead: recovering the historical KV required to generate the next chunk. Thus, inter-step activation caching reduces computation within the current denoising process, whereas ReelCache reduces the cost of making historical KV available. Detailed comparison is provided in Appendix § 9. We also review efficient and parallel video generation systems in Appendix § 10.
Background and Motivation
In this section, we first characterize video generation (§ 2.1) and then explain how retaining KV caches for the multi-turn video generation leads to prohibitive memory footprint (§ 2.2). The illustration of video denoising and KV conditioning in autoregressive video diffusion is listed in Appendix § 11.
Characterizing Long Video Generation
While retaining only the latest or active KV window, as LongLive Yang et al., [40], supports efficient forward video generation, it is insufficient for long video generation tasks. In these tasks, video creation often involves multiple rounds of generation and revision. For example, after generating a one-minute video of a person entering a cafe, ordering a drink, and sitting by the window, a user may decide that the person should instead walk toward the table immediately after entering, around the 5th second. The video generation system should then return to that early point, preserve the first 5s footage, and generate a new continuation. At this point, however, the active KV window would contain only the context near the end of the original one-minute video, while the KVs needed to continue from the 5th second may have already been evicted long ago. As a result, without a saved copy of the required KV state, the server has to reconstruct it before generating the revised sequence, imposing prolonged waiting times on users.
Figure 4 quantifies this overhead on an RTX PRO 6000 GPU. We observed that the overhead of rebuilding historical KV grows sharply with video history length across all models. Compared with reusing resident KV (on GPU), rebuilding the cache from latents raises total next-chunk latency to – the baseline for a 5s history. For a 60s history, this latency gap increases to –, with slowdowns of , , , and for LongLive 1.0 Yang et al., [40], LongLive 2.0 Chen et al., [7], RAVEN (Lu et al., 2026), and CausVid Yin et al., [42], respectively.

Figure 4. Next-chunk latency with history KV cache resident and recomputed from latents across models and history durations on an RTX PRO 6000. Text encoding and VAE decoding are excluded.
The Excessive Cost of Retaining Historical KV
A straightforward way to avoid repeated reconstruction and delay is to retain all previously generated KV chunks. However, preserving this KV history incurs a substantial memory cost. Video represents each frame as a spatial grid of tokens (e.g., pixels), and each chunk spans several such frames, producing a large number of tokens even for a short sequence. Each token further requires keys and values to be stored across transformer layers. Retaining all KV therefore makes storage a major obstacle to historical state reuse.
Figure 1 shows the estimated FP16 historical KV cache for four models, assuming 24 FPS video. Each second of generated video adds approximately 1.95 GB of KV for LongLive 2.0 and 1.73 GB for CausVid. For LingBot (a 13.6B model at 720p), this number grows to 17.0 GB per second. Consequently, even if all 96 GB of an RTX PRO 6000 were dedicated to historical KV, it would hold less than one minute of history for the smaller models and only 5.6s for LingBot. Note that these are optimistic capacity bounds. In practice, model weights and inference activations would compete with KV cache for the same GPU memory.

Figure 1. Size of the KV and the longest history that fits in one RTX PRO 6000 with 96 GB memory or in 1 TB of host memory.
How about offloading to the host memory? Historical KV can instead be offloaded to host memory, as in LLM serving systems [18, 23]. However, its large footprint remains a significant concern. For instance, as shown in Figure 1, a dedicated 1 TB host-memory budget holds approximately 8.6 minutes of LongLive 2.0 history or only 59 seconds of LingBot history, before accounting for other uses of host memory.
Moreover, due to the limited PCIe bandwidth, migrating KV cache from the host memory or hard disk to GPU still comes with significant delay. For example, for CausVid with 30s of history, the required KV occupies 51.8 GB. Resident KV on the GPU yields the next chunk generation in 3.64s. In contrast, migrating KV from host memory or SSD can take 2.46s and 11.23s, respectively.
Basic Idea and Design Rationale
In this paper, we revisit a basic storage strategy for multi-turn video generation: compress historical KV when retaining it for future reuse, and decompress the required state on demand. This reduces both the storage footprint of historical KV cache and the amount of data transferred between main memory and GPU when restoring an earlier generation point. However, realizing this basic idea requires answering three core questions:
How to exploit the redundancy within each chunk’s KV?
How much to compress each KV matrix while preserving generation quality?
Whether consecutive chunks share structure that can further reduce storage and compression overhead?
We investigate these questions through the measurements below, which establish the rationale for our design.
Measurement setup. We characterize both the structure of historical KV and the effect of its compression on subsequent generation. Because reconstructed KV conditions future chunks, we assess compression fidelity using the Structural Similarity Index Measure (SSIM) of the decoded continuation relative to an uncompressed-cache reference generated with the same prompt and random seed. We use VBench (Huang et al., [14]) to assess generated-video quality. Unless otherwise noted, experiments use CausVid (Yin et al., [42]) at a fixed resolution of pixels.
Complementary Redundancy Within a Chunk
Observation One: Video KV cache exhibits complementary channel and spatiotemporal redundancy. The measurement motivates us to combine low-rank factorization with video codec encoding, using the pre-RoPE representation for the Key.
• Across channels, we found that the KV matrices exhibit a concentrated singular spectrum, indicating that many channel features can be represented by a smaller set of shared directions. Figure 2 shows a concrete example: despite its 1536-dimensional channel representation, 96 singular directions (6.25%) capture 98% of the pre-RoPE Key’s energy.
• Within each channel, KV matrices also exhibit extensive spatial and temporal correlations, as shown in Figure 5. It is understandable that nearby regions within each frame often depict similar content, while consecutive frames share much of the same scene, with changes concentrated around moving objects. These spatial-temporal redundancies are reflected in the corresponding KV features, which create redundancy across spatial positions and neighboring frames.

Figure 2. Video KV is low-rank across the channel dimension: a small basis captures almost all of the matrix’s energy.
Figure 5. Per-layer spatial and temporal correlation (mean, max, and min) within each pre-RoPE Key and Value channel of CausVid.
These two degrees of redundancy motivate us a two-stage compression pipeline: a) apply SVD to reduce redundancy across channels, representing the KV tensor with fewer coefficient channels and a compact basis, and b) use a video codec to compress spatial and temporal redundancy within these coefficient channels. Figure 3 illustrates the potential of combining these mechanisms. For the pre-RoPE Key, HEVC (a.k.a., H.265, a representative video codec) alone achieves 7.82× compression with a continuation SSIM of 0.986, while rank-256 SVD achieves 4.52× with SSIM 0.991. Applying HEVC to the SVD coefficients increases compression to 14.12×, with SSIM slightly drops to 0.981.

Figure 3. Compression ratio and SSIM of pre-RoPE Key compression (CausVid generates one chunk with SVD rank at 256).
While the initial results are promising, we also found that the representation being compressed also matters. RoPE applies position-dependent rotations to the Key, changing its structure across token positions. Table 1 compares low-rank compression before and after RoPE at matched ranks and compression ratios, leaving the Value uncompressed. The pre-RoPE representation consistently yields higher continuation SSIM across the tested models, with larger differences at lower ranks. For example, at rank 64, CausVid achieves SSIM 0.970 with pre-RoPE Key compression versus 0.918 with post-RoPE compression. These results favor compressing the Key before RoPE and restoring its positional encoding after reconstruction.
| Model | Rank 64 pre-RoPE | Rank 64 post-RoPE | Rank 256 pre-RoPE | Rank 256 post-RoPE |
| LongLive 2.0 | 0.823 | 0.769 | 0.924 | 0.885 |
| RAVEN | 0.989 | 0.966 | 0.998 | 0.996 |
| CausVid | 0.970 | 0.918 | 0.995 | 0.990 |
Table 1. Next-chunk SSIM with history Key compressed by SVD before or after RoPE (one history chunk, Value uncompressed).
Heterogeneous Rank Requirements
Observation Two: The SVD rank needed to meet a retained-energy target varies across layers, prompts, and models. Efficient compression thus requires per-matrix rank selection with low selection overhead.
When applying SVD, we have to determine how many singular directions (i.e., rank) to retain. A smaller rank compresses more, but also loses more information that may affect subsequent chunk frame generation. Conversely, a larger rank preserves more information but increases storage. We first examine retained spectral energy as a proxy for compression fidelity, and then characterize the rank needed to meet a given energy target.
As shown in Figure 6, we found that the SSIM metric generally improves as more energy is retained, but the Key and Value exhibit different sensitivities. As the figure shows, on CausVid, retaining 90%, 95%, and 98% of the Value energy (with the Key uncompressed) yields an SSIM of 0.94, 0.98, and 0.99, respectively. In contrast, retaining the same fractions of the Key energy (with the Value uncompressed) yields an SSIM of only 0.79, 0.89, and 0.95, respectively. Thus, equal retained-energy fractions do not imply equal effects on the generated video.

Figure 6. SSIM and VBench standard deviation relative to the uncompressed cache, for CausVid and LongLive 2.0 across retained Key and Value energy. The red box marks 98% Key / 95% Value.
Suggested by the above benchmark result, we retain 98% of the Key energy and 95% of the Value energy to preserve generated-video quality. Under joint Key and Value compression, these targets keep VBench scores within 0.2 points of the uncompressed reference on both CausVid and LongLive 2.0, indicating minimal quality degradation on the evaluated benchmark. We therefore use these targets in the following characterization to examine compression opportunities while maintaining near-reference video quality.
A fixed energy target requires different ranks. We then run experiments to explore the rank required to meet the retained-energy target in each layer of CausVid across the VBench dataset. Figure 7 shows substantial variation across layers and prompts. For CausVid, the mean required rank ranges from 1 to 153 for Key and from 16 to 269 for Value across different layers. For LongLive 2.0, this mean rank ranges from 1 to 538 for Key and from 39 to 613 for Value.

Figure 7. Smallest rank that retains 98% of Key energy and 95% of Value energy for each layer of CausVid and LongLive 2.0. The dashed line is the uniform rank that meets the target everywhere.
We also found the required rank differs across models as well. On average, CausVid requires ranks of 83 for Key and 162 for Value out of 1536 channels. LongLive 2.0, on the other hand, requires ranks of 269 and 412 out of 3072 channels. To meet the targets for every KV, a uniform rank would need to increase to 209 and 333 on CausVid, and 661 and 743 on LongLive 2.0. Such a uniform rank setup would allocate unnecessary singular directions to matrices that admit substantially lower-rank representations, wasting precious storage. Compared with giving each matrix its own rank, it stores 1.21 more SVD factors on CausVid and 1.06 more on LongLive 2.0. This variation indicates that we should select the rank separately for each KV matrix in order to maximize the compression ratio.
Shared Subspaces Across Consecutive Chunks
Observation Three. Adjacent chunks frequently share substantial channel structure, creating an opportunity to reuse bases.
Compressing each chunk independently would create a separate, heavy channel basis for every KV matrix. In practice, however, neighboring chunks often depict a continuing scene, suggesting that their KV representations may share structure across chunks. We investigate whether this shared structure persists in the channel subspaces, even when the raw KV tensors change.
Figure 8 shows the results1. We observed that the raw KV tensors can vary substantially across consecutive chunks. For example, consecutive LongLive 2.0 chunks have mean cosine similarities of only 0.76 for the Key and 0.41 for the Value. Despite these changes, their channel subspaces remain highly similar. At rank , adjacent chunks generated under the same prompt have a mean channel-subspace overlap of approximately 0.89 in CausVid and 0.86 in LongLive 2.0 for both Key and Value, with 90% of consecutive pairs exceeding 0.80. These results indicate substantial shared channel structure across neighboring chunks despite changes in their token-level representations.

Figure 8. Subspace overlap and raw KV overlap of adjacent chunks under the same prompt for Key and Value at SVD rank 256.
This shared structure creates an opportunity to reuse a common basis across chunks while storing separate coefficients to capture each chunk’s distinct content. However, harvesting it encounters two challenges. First, computing subspace overlap requires fitting the basis for the new chunk, which conflicts with the goal of avoiding repeated basis computation. Second, even a high overlap does not ensure that the reused basis retains enough energy from the new KV matrix. For example, two adjacent-chunk Key comparisons in the same CausVid rollout yield high overlaps of 0.951 and 0.930. Yet, under the same storage budget as a freshly fitted basis, the preceding chunk’s basis retains 98.2% of the new matrix’s energy in one case but only 96.6% in the other, meeting the 98% target for only the first.
System Design
We present ReelCache, a historical KV cache compression framework for multi-round, long video generation. It compresses the KV states of completed chunks for future reuse and restores the required historical context when a user resumes, rewinds, or branches a generation session. Figure 9 illustrates the compression and restoration workflows.
Figure 9. Overview of ReelCache.
Intra-Chunk KV Compression
ReelCache compresses each chunk’s KV through two stages: low-rank factorization across channels, followed by video coding over spatial and temporal positions. Figure 9 (part 1) illustrates the whole compression process and its reconstruction, overlapping with ongoing video generation.
Step One: low-rank factorization. For each transformer layer, ReelCache captures the chunk’s Key before RoPE and its Value, and compresses them separately. We represent each tensor as a matrix , where is the number of tokens across frames, each with an token grid. is the channel dimension. ReelCache approximates as:
where contains orthonormal channel directions and contains the corresponding coefficients for each token. The rank determines the storage-approximation tradeoff: the coefficient matrix and the basis matrix together contain values, compared with in the original matrix. For a new factorization, ReelCache fits a basis that approximates the leading right singular subspace of , selecting its rank described in § 4.2. The same representation also supports bases reused across chunks, as explained in § 4.3.
Step Two: video coding of coefficients . Recall that in Eq. (1), each column of contains one coefficient per token. ReelCache reshapes this column into an field using the tokens’ original spatial and temporal positions. It then quantizes the coefficient fields into codec-compatible samples and compresses them with HEVC. The compressed
representation consists of the encoded coefficients, the channel basis, and the metadata needed to reverse the layout and transformations. An explanation of why SVD first is in §A.
This factorization-first-then-video-coding order allows the two stages to exploit complementary redundancy: factorization reduces the number of fields to encode from original channels to coefficient channels, while video coding then exploits spatial and temporal correlations within these fields to further reduce storage. Thus, the codec operates on the numerical coefficients produced by factorization, while the basis is retained separately for reconstruction.
Running and restoration. In ReelCache, the basis fitting and coefficient projection run on the GPU, while encoding runs on NVENC if available (runs on a CPU worker pool otherwise). Note that either encoding method won’t compete with ongoing video denoising for GPU’s computing resources. During restoration, NVDEC first decodes the coefficients, which are dequantized and rearranged into a reconstructed coefficient matrix . ReelCache then computes , reapplies RoPE to the reconstructed Key at its original token positions, and writes Key and Value into the GPU’s memory. To overlap decoding with generation, ReelCache prepares the next layer’s coefficients while the current layer computes, as shown in Figure 9 (part 3).
Online Rank Selection
ReelCache estimates the rank for each layer’s KV matrix individually in three steps. It first computes a low-rank SVD of the targeting matrix and then uses the captured energy to predict the rank needed to meet the retained-energy target. After that, ReelCache verifies the energy captured with the predicted rank and updates the predictor. This is motivated by the observation in Figure 10: within each layer, prompts can differ substantially in the energy captured at small ranks (the blue shaded region), but their residual energy decays at similar relative rates in the spectral tails (the orange shaded region). Accordingly, the low-rank decomposition provides us a good starting point (step one), from which the learned decay function predicts the rank required to reach the target (step two), and verifies the energy captured (step three).

Figure 10. Rank-energy curve for various prompts of CausVid’s layer 15. The energy variance decreases as the rank increases. Each prompt’s energy curve can be found in Appendix § 13.
To better understand this idea, consider the highlighted curve in Figure 10. A rank-32 SVD captures 90.42% of ’s energy. Using this measurement and the layer’s learned decay function, ReelCache predicts that a rank of 145 will retain 98% of the energy, without computing the full singular spectrum of . We elaborate this algorithm below.
• Step One: measure the captured energy. For a KV matrix , ReelCache constructs a basis with a low-rank of , and then measures the captured-energy fraction as follows:
Given the basis, this measurement costs (), significantly lower than the full matrix decomposition of with overhead. If (the targeting energy ratio), then the basis already meets the target energy. Otherwise, ReelCache uses the measured residual energy, , to predict the required rank as follows.
• Step Two: predict the required rank. We approximate the residual-energy decay with a power law function and estimate the required rank using the following equation2:
where describes the decay pattern for layer and energy target . The ratio here measures how much the residual energy must decrease to meet the target. We learn and for each layer and KV type online to account for their decay patterns and approximation error in the measured basis. Both two coefficients are updated online through least square optimization, as explained below.
• Step Three: energy verification and coefficient update. If captured energy after step 2 does not meet the target energy, ReelCache adds an extra bucket of directions (32) and repeats the check until meeting the target. As shown in Figure 11,

Figure 11. Extra rank adjustments required after rank prediction.
more than 90% of Keys and 83% of Values of CausVid don’t need extra rank adjustment, while one extra adjustment can cover more than 99.8% of Keys and Values. The maximum rank adjustment is 2.
We then use these data to update the predictor online. Specifically, we use the unrounded exact rank value, , as the label, so bucket rounding doesn’t inflate the learned rank. For fits requiring more than the anchor rank , let and . Each layer maintains its online regression state, regularization initialized with identity matrix and coefficients initialized with . After verification, we update coefficients using least square optimization as follows:
This update incorporates the verified fit into subsequent predictions, with state retained across prompts.
Inter-Chunk KV Cache Management
In ReelCache, we store shared basis once and keep separate coefficients for each chunk. For each layer and KV type, a new chunk reuses its precedent chunk’s basis if it meets the retained-energy target (e.g., 98%). Otherwise, it adds directions to capture the missing energy, or fits a new basis when that requires less storage. We propose chunk state tree to record these basis references alongside the generation history, which can easily support resume, rewind, and forks.
Chunk state tree. As shown in Figure 12(a), each node represents a chunk and records its HEVC-encoded coefficients, references to the basis directions it uses, and reconstruction metadata. Each edge connects a parent chunk to its child, so a root-to-node path, such as , represents a consecutive video generation history.
Figure 12. Chunk state tree in ReelCache. (a) Three operations for basis sharing in consecutive chunks: reuse, augment, and refit. (b) Session pointer and user operations: resume, rewind, and fork.
For each new chunk, ReelCache first attempts to reuse its parent’s basis. If this basis captures sufficient energy to meet the target threshold, which is , the child stores its own HEVC-encoded coefficients while referencing the existing basis. For example, chunks and store separate coefficients and but share the stored basis .
If the inherited basis falls short of the target, ReelCache incrementally extends it with additional basis directions , adding directions at a time (e.g., 32). Once the extended basis meets the target, the child stores the additional directions and coefficients for the extended basis, as illustrated by chunk . If the required total rank exceeds the rank used in the most recent fit , ReelCache instead refits a new basis with the methods in § 4.1 and § 4.2. For example, the edited chunk differs substantially from its ancestor , triggering a refit that produces a new basis .
The session maintains a pointer to its current chunk node, as shown in Figure 12(b). The multi-round video generation then becomes tree operations: resume continues from this node; rewind selects an ancestor; and fork starts a new branch whose first chunk is a child of any existing node and leaves the parent’s branch untouched. Existing nodes remain unchanged, allowing branches to share their common history without copying it. The detailed algorithm for determining if a node can share the basis of its parent node is in §H.
Evaluation
We implement ReelCache in approximately 21K LoC and integrate it into SGLang-Diffusion ([45]) with 2K LoC. ReelCache adopts LRU to manage cache.
Experimental Setup
Testbed. We conduct experiments on a server with an AMD EPYC 9555 64-core processor, eight RTX PRO 6000 GPUs with 96GB GDDR7 each, and 512 GB host memory.
Models. We evaluate five autoregressive DiT models with three model sizes. CausVid ([42]), RAVEN ((Lu et al., 2026)), and LongLive 1.0 ([40]) have 1.3B parameters. LongLive 2.0 ([7]) has 5B parameters, and LingBot ((Robbyant Team et al., 2026)) has 13.6B parameters. We preserve each model’s native generation granularity, attention visibility, positional encoding, and conditioning semantics.
Datasets. Since no public multi-round video generation dataset is available for evaluation, we build the Synthetic500 dataset by extending MemFlow’s interactive multi-prompt benchmark ((Ji et al., 2025)) to 500 narratives with 10,000 main-path and 5,000 alternative prompts. An additional 1,328 prompts support branches visited during evaluation. Appendix § 16 details the corpus.
Workloads. We generate requests and set up different workloads for evaluation. We characterize workloads by two parameters: request rate: the aggregate number of submitted prompts per second; and rewind frequency: the probability of selecting a non-latest checkpoint. Rewind generates a new branch to a random historical checkpoint. Other requests extend the latest checkpoint. We fix rewind frequency to 0.2 in § 5.2. In §J, we perform the sensitivity study over rewind frequencies and measure ReelCache’s speedup.
Metrics. For storage, we report the compression ratio: the size of the uncompressed BF16 KV divided by the bytes ReelCache stores for it. For video quality, we measure VBench scores([14]), a gold-standard evaluation metric suite that quantifies video generation quality, and SSIM to measure frame-wise quality.
Baselines. We compare ReelCache with two baselines: Recompute and Offload. Recompute keeps no history KV outside the active GPU cache. When a session resumes or rewinds, it rebuilds the historical KV by replaying the forward passes for video latents. Offload keeps the uncompressed KV of generations in GPU memory and host memory, evicting the least recently used chunks when storage is full. When a request needs evicted KV, it recomputes the missed KV from video latents.
End-to-End Performance Comparison
Compression Ratio. Figure 13(a) reports the mean KV compression ratio and standard deviation for all five models. ReelCache achieves mean compression ratios ranging from for LongLive 2.0 to for CausVid. For RAVEN, LongLive 1.0, and LingBot, the corresponding ratios are , , and , respectively. The standard deviations are small relative to the means, indicating limited variation across the evaluated samples within each model. These reductions allow ReelCache to retain substantially more historical KV under a fixed storage budget than the uncompressed Offload baseline, potentially reducing evictions and subsequent recomputation under memory pressure.
Figure 13. Mean and standard deviation of compression ratio and SSIM for models under ReelCache. Higher is better.
Video Quality. We next examine whether this KV compression comes at the cost of video quality. Figure 14 compares generated videos against matched references produced without KV compression using six representative VBench dimensions ([14]).
Figure 14. Six-dimensional VBench scores between reference videos and videos generated by ReelCache.
Across all five models, ReelCache’s scores closely track the uncompressed references on every dimension. For example, motion smoothness changes by at most 0.1 points, while subject and background consistency remain within 0.6 points of the references. Aesthetic quality, imaging quality, and dynamic degree also remain close, with the largest reduction across all six dimensions being 0.97 points on the 0–100 scale. These results show that ReelCache substantially reduces KV storage while largely preserving video quality. We show a few frame comparisons in Appendix § 18.
We also assess frame-wise fidelity using SSIM against the corresponding reference frames. Figure 13(b) reports the mean SSIM and standard deviation for each model. All five models achieve mean SSIM values of at least 0.915, ranging from 0.915 for LingBot to 0.986 for LongLive 1.0. These results indicate that the generated videos retain high structural similarity to their uncompressed references.
Next-Chunk Latency. We compare the P50 and P90 next-chunk latencies under four workload configurations combining short/long histories (20/40 chunks) with low/high request loads. As shown in Figure 15, across all 20 model-workload combinations, ReelCache achieves mean P50 speedups of over Offload and over Recompute. The corresponding P90 speedups are and .

Figure 15. P50 and P90 next-chunk latency of ReelCache compared to baselines across models and workloads.
Specifically, when the request load is low, ReelCache remains beneficial, providing mean P50 speedups of over both baselines. Its average tail-latency advantage is more pronounced under long histories and high request loads: the mean P90 speedup over Offload increases from in the short-history, low-load configuration to , while the corresponding speedup over Recompute increases from to .
This benefit is because within a fixed memory budget, ReelCache can retain more compressed KVs than Offload, reducing capacity-induced misses and their reconstruction costs. Offload transfers retained raw KV back to the GPU and rebuilds required KV from latents when it has been evicted. Recompute instead retains latents rather than a historical KV archive and reconstructs the required context through model forwards. ReelCache restores compressed checkpoints on demand, avoiding the expensive recomputation costs and reducing waiting times for queued requests.
System Efficiency
We then evaluate system efficiency by measuring the time ReelCache spends on compressing and decompressing KV. We also evaluate how much compression and decompression operations slow down next-chunk generation, respectively.
Compression. Figure 16 lists the time breakdown of the chunk compression time and the comparison with the chunk generation time. Compressing a chunk takes 1.31–4.72s,
Figure 16. Compression time breakdown in ReelCache and model generation time.
– the time to generate the KV. SVD (rank selection, fitting, projection, and verification) and HEVC encoding are the two most expensive operations, together accounting for 88% of the compression time on average. Note that ReelCache keeps this work off the critical path: it schedules SVD on a dedicated CUDA stream, and a chunk’s SVD is deferred while a video chunk is generating and runs between chunks, unless uncompressed chunks will be offloaded to host memory by the cache policy. HEVC encoding runs on the CPU or NVENC engine, so it proceeds during generation without occupying the streaming multiprocessors. Interference therefore arises only when a request arrives while an SVD is still running, and the request then shares the GPU with the remainder of that SVD. As listed in Table 2, compression slows an ongoing video generation job running on the same GPU by at most 0.8%.
| Model | Attention | Compress | Decompress |
| CausVid | Full | 0.2% | 3.2% |
| RAVEN | Full | 0.8% | 4.2% |
| LingBot | Full | 0.1% | 3.7% |
| LongLive 1.0 | Sliding window | <0.1% | <0.1% |
| LongLive 2.0 | Sliding window | <0.1% | 0.6% |
Table 2. Slowdown of next-chunk generation caused by ReelCache’s compression and decompression.
Decompression. ReelCache overlaps decompression with video generation layer by layer: NVDEC decodes the coefficients of the next layer while the current layer computes, and reconstructing each K or V matrix requires only one matrix multiplication of its coefficients and basis. Generation stalls only when the first forward pass reaches a layer whose KV is not yet restored. Because full-attention models restore every history chunk, decompression slows video generation by 3.2% for CausVid, 4.2% for RAVEN, and 3.7% for LingBot. Sliding-window attention-based LongLive 1.0 and LongLive 2.0 restore only the limited chunks in their attention window, regardless of history length. Hence decompression slows their generation by merely <0.1% and 0.6%, respectively. These delays are far less than the latency ReelCache saves over KV recomputation (§ 5.2).
Ablation Study
We next quantify the contribution of each design part toward compression ratio. Stage-1: HEVC compression that encodes KV with HEVC. Stage-2: SVD compression that decomposes KV with SVD. Stage-3: SVD+HEVC compression pipeline that decomposes KV with SVD first, then encodes SVD coefficients with HEVC. Stage-4: Adaptive rank that dynamically adjusts SVD rank across layers during compression. We designate Stage-5 as the standard ReelCache configuration, representing the fully implemented system on our testbed. For the fixed-rank SVD configurations in Stages 2 and 3, we use and , where denotes the channel dimension.
Figure 17 breaks down the compression ratio of each stage. Stage-1 (HEVC only) and Stage-2 (SVD only) compress the least, by an average of 2.71× and 3.30× across models. Each applies a single transform/encoding and therefore removes only one kind of redundancy (intra-channel for

Figure 17. Breakdown of average compression ratio across incremental stages, where Stage-5 is ReelCache.
HEVC, inter-channel for SVD), leaving the other on the table. Combining the two in Stage-3 nearly doubles the ratio to 7.24× (2.19× over Stage-2). Stage-3, however, applies one fixed rank to every prompt and layer, so many layers keep far more rank than they need to reach the energy target (§4.2), which caps the achievable compression. Enabling the adaptive rank for layers in Stage-4 removes this excess rank and raises the mean ratio by 1.87×, from 7.24× to 13.56×. Finally, Stage-5, the default ReelCache configuration, shares bases across consecutive chunks. This basis sharing further reduces storage by 22.9%, raising the mean compression ratio to 17.59×.
We further examine ReelCache’s performance to workload sensitivity and put the results in Appendix § 17.
Conclusion
We have presented the design, implementation, and evaluation of ReelCache, an efficient KV cache compression and management system for multi-round video generation. Within each chunk, it leverages SVD and video codec to remove both inter- and intra-channel redundancy. Besides, it employs an online adaptive rank determination to reduce storage of bases and coefficients without degrading video quality. Across chunks, ReelCache maintains a chunk state tree and dynamically shares bases across consecutive chunks for less SVD basis storage. It also enables basis augmentation and basis refit when necessary to maintain the video quality. ReelCache reduces KV storage by 14.46×-21.13× with less than 1% drop in VBench metrics.
AI Usage Disclosure
We used generative AI tools in the following parts of this work.
Code generation. We used Fabel 5.1 as a coding assistant to implement parts of ReelCache. All AI-assisted code was reviewed, tested, and validated by the authors.
Data generation. We used GPT-6 Astra to author 400 of the 500 narratives in the Synthetic-500 evaluation corpus (Appendix § 16) that extend each script in MemFlow (Ji et al., 2025) to 20 consecutive prompts. We also leverage GPT-6 Astra to write 5,000 alternative prompts and 1,328 additional prompts to support user rewind and fork operations in multi-round video generation workloads. These prompts serve only as workload inputs.
Writing and references. We used GPT-6 Astra to rephrase and polish the text for clarity and readability.
Generative AI was not used for research ideation, the design of ReelCache, experiment design, data analysis, or generating any of the results, claims, or findings reported in this paper. We have reviewed all AI-assisted work and take full responsibility for the content of this submission.
Why SVD First Then Video Codec
Leveraging both SVD and a video codec (e.g., HEVC) offers a high compression ratio, as shown in Figure 3. ReelCache employs a SVD-first-then-HEVC compression pipeline. Reversing the order, HEVC followed by SVD, changes the object being approximated: SVD operates on an entropy-coded byte stream rather than the KV tensor. Its numerical approximation does not enforce the syntax required by the HEVC decoder. All video chunk streams reconstructed after SVD failed decoding in our offline experiments.
Autoregressive Diffusion Transformers
In ReelCache, we target chunk-autoregressive video diffusion transformers (AR-DiTs) that expose reusable, layer-wise self-attention keys and values for completed video context [42] (Yang et al. 2025). These models generate video sequentially, with each new chunk attending to the cached KV of selected preceding chunks. This causal historical KV therefore conditions subsequent chunks without repeatedly processing the preceding video, subject to the model’s attention window. ReelCache compresses and retains this historical state for efficient restoration when generation resumes or branches from an earlier point.
The historical video KV targeted here is distinct from text-conditioning KV used in cross-attention, where visual queries attend to keys and values derived from text representations (Chen et al., 2024a;b; Saharia et al., 2022; Rombach et al., 2022), as depicted in Figure 18(a). Visual tokens that are jointly updated during bidirectional denoising or in-place editing generally produce step- or edit-dependent hidden states, whose KV cannot be assumed to remain valid as an unchanged historical prefix. Hence, these scenarios are out of ReelCache’s scope.
Figure 18. Illustration for (a) KV reuse in cross-attention, (b) intermediate activation caching during denoising, and (c) ReelCache’s historical KV cache in autoregressive video generation.
Our objective also differs from intermediate-activation caching [21, 26, 44, 36] (Ma et al. 2024). These methods exploit similarity between denoising steps to reuse attention outputs, block features, or other intermediate results, reducing computations within the denoising process, as shown in Figure 18(b). In contrast, ReelCache preserves the KV of completed video chunks for subsequent chunk generation and interactive continuation. It accelerates recovery of the required historical context, rather than skipping the denoising computations of the new chunk, as illustrated in Figure 18(c).
Efficient and Parallel Video Inference
Two orthogonal lines make video-DiT inference practical, and ReelCache coexists with both. Step-cache works [10, 21] (Li et al. 2024a; Ma et al. 2024) exploit the temporal redundancy of the denoising loop and skip recomputation for nearly identical activations. Parallel-serving systems, on the other hand, distribute the model or tokens with sequence parallelism (SP), because a video DiT’s 3D activations and KV cache might exceed one GPU’s capacity. For instance, Ulysses [15] shards the token sequence where each GPU owns whole attention heads (head/channel split); RingAttention [22] keeps a contiguous token shard and rotates keys/values around a ring (token split). USP [8] composes both. Inference engines [9, 41, 45] integrate SP with other parallelism (e.g., pipeline), and recent work (Wu et al., 2025) proposes latent parallelism for video diffusion to cut the heavy cross-GPU activation traffic. SP fixes the cache layout shared by head/token that any KV compressor must accept before compression runs.
Autoregressive Diffusion Video Generation Primer
Modern autoregressive video diffusion models, such as CausVid [42] and LongLive [40, 7], generate a video one chunk at a time, conditioning each new chunk on previously generated content. As illustrated in Figure 19 (top), generation proceeds at two timescales: autoregressive generation across chunks and iterative denoising within each chunk. For each chunk, a diffusion transformer (DiT) starts from noisy latent tokens and progressively refines them through multiple denoising steps, conditioned on a text prompt and the preceding video context. A variational autoencoder (VAE) then decodes the resulting clean latents into RGB frames.
Figure 19. Autoregressive video generation workflow. Middle: the model generates a video chunk by chunk, each chunk conditioned on the preceding ones. Top: within chunk , the DiT iteratively denoises noisy latents into clean latents over steps, and the VAE decodes into RGB frames. Bottom: the KV of the most recent chunks is kept in a rolling cache window (three chunks shown). Chunk attends to the cached KV. Once it is generated, its KV is appended, and the oldest is evicted.
To maintain temporal consistency, the DiT denoises each new chunk while attending to previously generated content. Models such as LongLive [40] cache the layer-wise keys and values (KV) of preceding chunks within a rolling window, allowing subsequent chunks to reuse these representations without recomputing them, thereby reducing a significant amount of video generation latency. As illustrated in Figure 19 (bottom), this cache gets updated with generation, enabling efficient continuation while evicting older entries outside the retained context.
Channel-Subspace Similarity Across Chunks
To measure channel-subspace similarity across chunks, we use normalized subspace overlap:
where and are row-orthonormal rank- channel bases for corresponding KV matrices in chunks and , respectively. This score is the mean squared cosine of the principal angles between the two subspaces and is invariant to the choice of orthonormal basis within each subspace.
Full Energy Curve Plot
Figure 20 lists energy curves of prompts for CausVid’s layers 15 and 25 (pre-RoPE Key and Value). Within each layer, prompts can differ substantially in the energy captured at small ranks, but their residual energy decays at similar relative rates in the spectral tails.

Figure 20. Prompts’ energy curves of CausVid’s layers 15 and 25.
Derivation of the Predictor
We approximate the residual-energy decay with a power law function:
where denotes the energy fraction captured by the optimal rank- approximation, describes the decay pattern for layer and energy target . Substituting in above equation with measured by Eq. (2) and further solving for the rank at which the captured energy reaches gives:
Taking the logarithm on both sides, we have a rank estimator:
The ratio here measures how much the residual energy must decrease to meet the target. We learn and separately for each layer and KV type to account for their decay patterns and approximation error in the measured basis. In the online estimation, both coefficients and are updated through least-squares optimization.
Inter-Chunk Basis Reuse
Basis reuse and augmentation. For each layer and KV type, a new chunk first tests the basis inherited from its parent. Rather than comparing two independently fitted
Algorithm 1 Commit matrix of a new chunk, given the branch’s active basis .
// project
// certify
if then
keep the shortest prefix of reaching // share
else
// new directions
if storing costs more than a fresh fit then
// refit
end if
// augment
end if
// encode with HEVC and store the new basis
bases, ReelCache directly measures how much of the new matrix the inherited basis captures as follows:
If , the chunk reuses the shortest prefix of that meets the target and stores its own encoded coefficients, without adding basis directions. Otherwise, the residual contains the information missing from the inherited basis. To reach , the additional directions must capture at least energy, or a fraction of the residual energy. We therefore fit the residual with this target:
The new directions are orthogonalized against and appended to it. The chunk stores coefficients for the extended basis, but only the newly added basis directions. The resulting representation is checked against the same energy target as an independently fitted one. Algorithm 1 describes this whole process. On an RTX PRO 6000 GPU, the projection-energy test takes only 0.11 ms.
When to refit. Repeated augmentation can increase the number of coefficients needed by subsequent chunks. ReelCache therefore fits a fresh basis when its estimated storage cost is lower than that of augmentation, accounting for both coefficients and newly stored basis directions. Specifically, it estimates the fresh rank from the layer’s most recent fit, avoiding a separate decomposition just to make this decision. Large changes in content can favor refitting by requiring many additional directions, without a separate scene-change detector. A refit replaces the active basis for subsequent chunks; earlier nodes retain their original basis references. Likewise, a fork inherits the basis at its branch point and can extend or replace it independently, without modifying the other branch.
Random access. Although chunks share basis directions, in ReelCache each chunk’s coefficients are encoded as a separate codec stream. This allows a chunk to be restored from its coefficients and referenced basis without decoding preceding chunks’ coefficient streams, supporting random access for rewind and fork.
Corpus Construction
We construct our content corpus by extending MemFlow interactive multi-prompt video generation benchmark (Ji et al., 2025), which contains 100 narrative scripts with six successive prompts each. We extend each script to 20 consecutive prompts, and author 400 additional 20-prompt narratives with GPT-6 Astra. For every narrative, we also write two alternative continuations after each of five prefixes that end at the 4th, 8th, 12th, 16th, and 20th prompt. The resulting
Synthetic-500 corpus contains 500 narratives, 10,000 main-path prompts, and 5,000 alternative prompts. Since users can rewind to arbitrary earlier chunks and continue sessions beyond 20 prompts, we author 1,328 additional prompts for branches they visit. For each narrative, we author prompt-alternative branches after each prefix ending at prompt 4, 8, 12, 16, or 20. Each branch forms a consecutive alternative continuation. For LingBot (Robbyant Team et al., 2026), it requires interactive control instead of new prompts. We use the randomly generated camera/action sequences as the narratives. The request arrival pattern follows a Poisson distribution.
Workload Sensitivity
We vary the rewind frequency from 0% to 50% in 10-percentage-point increments, where controls the fraction of requests that resume from a historical chunk rather than extend the latest chunk. Figure 21 reports P50 and P90 next-chunk latency speedups over Offload and Recompute for all five models under short-history (20 chunks) and long-history (40 chunks) configurations. Within each configuration, session count, request load, and memory budget remain fixed.
Figure 21. ReelCache’s speedup (P50 and P90) over baselines under various rewind frequency .
ReelCache outperforms both baselines throughout the sweep. Its relative advantage generally decreases as rewinds become more frequent. Frequent rewinds can limit forward-history growth, reducing the baselines’ historical KV reconstruction costs and narrowing the latency gap. For CausVid under the long-history configuration, P50 speedups over both baselines decrease from at to at , while P90 speedups remain above at . Across all models and both configurations, P50 speedups at 50% rewind frequency remain –. The degree of sensitivity varies by model. For the sliding-window attention-based LongLive 1.0, the short-history curves remain relatively flat between 10% and 50%. LingBot’s gains stay around – across the sweep. LingBot’s expensive denoising step in its next-chunk latency limits the improvement space for faster historical KV recovery.
Overall, ReelCache benefits both ordinary per-session continuation () and interactive rewind workloads ().
Video Frame Comparison
Figure 22 presents sampled frames from reference videos generated without KV compression and matched videos generated with ReelCache. For CausVid, RAVEN, LongLive 1.0, and LongLive 2.0, we use the same four-prompt narrative: P1: A mint clay cat sets a brown teapot on a low table inside a tiny tea house with round windows. P2: The mint clay cat lifts the teapot lid, releasing a soft steam curl above three empty cups on the table. P3: The clay cat catches steam beneath a clear lid, where tiny green
Figure 22. Comparison between reference videos without KV compression and matched videos generated with ReelCache.
shapes gather over the first empty cup. P4: The mint clay cat taps the clear lid with one paw, and the tiny green shapes scatter in a swirl before drifting back together beneath it. For LingBot, we use a fixed scene prompt followed by a camera-control action sequence. The scene prompt is: A man with a shaved head and navy apron opens a shallow wooden shadow box on a studio table beside clean shells and sea glass. Rather than changing the text prompt between rounds, subsequent actions control the camera movement.
Across the displayed examples, the sampled frames generated with ReelCache closely resemble their uncompressed references, including those from LingBot’s camera-controlled sequence. These qualitative comparisons complement the quantitative quality evaluation and illustrate that ReelCache can preserve visual fidelity while compressing historical KV state.
References
- [1]Adobe Inc. Adobe firefly. Generative AI model. 2025. URL https://firefly.adobe.com/. Accessed: 2026-10-25.
- [2]Brooks, T., Peebles, B., Holmes, C., DePue, W., Guo, Y., Jing, L., Schnurr, D., Taylor, J., Luhman, T., Luhman, E., Ng, C., Wang, R., and Ramesh, A. Video generation models as world simulators. OpenAI technical report, 2024.
- [3]Cai, Z., Zhang, Y., Gao, B., Liu, Y., Li, Y., Liu, T., Lu, K., Xiong, W., Dong, Y., Hu, J., and Xiao, W. Pyramidx: Dynamic kv cache compression based on pyramidal information funneling. arXiv preprint arXiv:2406.02069, 2024.arxiv.org/abs/2406.02069
- [4]Chang, C.-C., Lin, W.-C., Lin, C.-Y., Chen, C.-Y., Hu, Y.-F., Wang, P.-S., Huang, N.-C., Ceze, L., Abdelfattah, M. S., and Wu, K.-C. Palu: KV-cache compression with low-rank projection. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=LWMS4pk2vK.arxiv.org/abs/2407.21118
- [7]Chen, Y., Wang, L., Huang, W., Yang, S., Zhang, B., Xiao, Y., Chu, R., Mao, W., Hu, Q., Liu, S., Zhao, Y., Mao, H., Chen, Y.-C., Xie, E., Qi, X., and Han, S. Longlive-2.0: An nfv4p parallel infrastructure for long video generation. arXiv preprint arXiv:2605.18739, 2026.arxiv.org/abs/2605.18739
- [8]Fang, J. and Zhao, S. Usp: A unified sequence parallelism approach for long context generative ai. arXiv preprint arXiv:2405.07719, 2024.
- [9]Fang, J., Pan, J., Sun, X., Li, A., and Wang, J. xdit: an inference engine for diffusion transformers (dits) with massive parallelism. arXiv preprint arXiv:2411.01738, 2024.
- [10]Fang, J., Pan, J., Li, A., Sun, X., and Jiannan, W. Pipefusion: Patch-level pipeline parallelism for diffusion transformers inference. Advances in Neural Information Processing Systems, 38:98434–98455, 2026.arxiv.org/abs/2405.14430
- [11]Google DeepMind. Veo: A text-to-video generation system. Technical report, Google DeepMind, 2025. URL https://storage.googleapis.com/deepmind-media/veo/Veo-3-Tech-Report.pdf.
- [12]Hooper, C., Kim, S., Mohammadzadeh, H., Mahoney, M. W., Shao, Y. S., Keutzer, K., and Gholami, A. Kvquant: Towards 10 million context length llm inference with kv cache quantization. Advances in Neural Information Processing Systems, 37:1270–1303, 2024.
- [13]Huang, X., Li, Z., He, G., Zhou, M., and Schechtman, E. Self forcing: Bridging the train-test gap in autoregressive video diffusion. Advances in Neural Information Processing Systems, 38:167283–167308, 2026.
- [14]Huang, Z., He, Y., Yu, J., Zhang, F., Si, C., Jiang, Y., Zhang, Y., Wu, T., Jin, Q., Chanpaisit, N., Wang, Y., Chen, X., Wang, L., Lin, D., Qiao, Y., and Liu, Z. Vbench: Comprehensive benchmark suite for video generative models. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 21807–21818. IEEE, 2024.
- [15]Jacobs, S. A., Tanaka, M., Zhang, C., Zhang, M., Song, S. L., Rajbhandari, S., and He, Y. Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models. arXiv preprint arXiv:2309.14509, 2023.
- [17]Kang, H., Zhang, Q., Kundu, S., Jeong, G., Liu, Z., Krishna, T., and Zhao, T. Gear: An efficient kv cache compression recipe for near-lossless generative inference of llm. arXiv preprint arXiv:2403.05527, 2024.
- [18]Lee, W., Lee, J., Seo, J., and Sim, J. InfiniGen: efficient generative inference of large language models with dynamic KV cache management. In Proceedings of the 18th USENIX Conference on Operating Systems Design and Implementation, OSDI’24, USA, 2024. USENIX Association. ISBN 978-1-939133-40-3.
- [21]Liu, F., Zhang, S., Wang, X., Wei, Y., Qiu, H., Zhao, Y., Zhang, Y., Ye, Q., and Wan, F. Timestep embedding tells: It’s time to cache for video diffusion model. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7353–7363. IEEE, 2025a.arxiv.org/abs/2411.19108
- [22]Liu, H., Zaharia, M., and Abbeel, P. Ringattention with blockwise transformers for near-infinite context. In International Conference on Learning Representations, volume 2024, pp. 3992–4008, 2024a.arxiv.org/abs/2310.01889
- [23]Liu, Y., Cheng, Y., Yao, J., An, Y., Chen, X., Feng, S., Huang, Y., Shen, S., Zhang, R., Du, K., and Jiang, J. Lmcache: An efficient kv cache layer for enterprise-scale llm inference. arXiv preprint arXiv:2510.09665, 2025b.arxiv.org/abs/2510.09665
- [26]Lyu, Z., Si, C., Song, J., Yang, Z., Qiao, Y., Liu, Z., and Wong, K.-Y. K. Fastercache: Training-free video diffusion model acceleration with high quality. In International Conference on Learning Representations, volume 2025, pp. 33132–33156, 2025.
- [33]Sand.ai, Teng, H., Jia, H., Sun, L., Li, L., Li, M., Tang, M., Han, S., Zhang, T., Zhang, W. Q., Luo, W., Wang, X., Sun, Y., Cao, Y., Huang, Y., Lin, Y., Fang, Y., Tao, Z., Zhang, Z., Wang, Z., Liu, Z., Shi, D., Su, G., Sun, H., Pan, H., Wang, J., Sheng, J., Cui, M., Hu, M., Yan, M., Yin, S., Zhang, S., Liu, T., Yin, X., Yang, X., Song, X., Hu, X., Zhang, Y., and Li, Y. Magi-1: Autoregressive video generation at scale. arXiv preprint arXiv:2505.13211, 2025.
- [34]Sun, H., Chang, L.-W., Bao, W., Zheng, S., Zheng, N., Liu, X., Dong, H., Chi, Y., and Chen, B. Shadowkv: Kv cache in shadows for high-throughput long-context llm inference. arXiv preprint arXiv:2410.21465, 2024. F., Yi, B., Yin, X., You, Y., Yuan, L., Zeng, W., Zeng, X., Zeng, Y., Zhai, S., Zhai, Z., Zhang, B., Zhang, C., Zhang, H., Zhang, J., Zhang, M., Zhang, P., Zhang, S., Zhang, X., Zhang, X., Zhang, X., Zhang, X., Zhang, Y., Zhang, Z., Zhao, H., Zhao, H., Zhao, L., Zhao, Y., Zheng, G., Zheng, J., Zheng, X., Zheng, Z., Zhu, K., and Zuo, F. Seedance 2.0: Advancing video generation for world complexity. arXiv preprint arXiv:2604.14148, 2026.
- [35]Team Wan, Wang, A., Ai, B., Wen, B., Mao, C., Xie, C.-W., Chen, D., Yu, F., Zhao, H., Yang, J., Zeng, J., Wang, J., Zhang, J., Zhou, J., Wang, J., Chen, J., Zhu, K., Zhao, K., Yan, K., Huang, L., Feng, M., Zhang, N., Li, P., Wu, P., Chu, R., Feng, R., Zhang, S., Sun, S., Fang, T., Wang, T., Gui, T., Weng, T., Shen, T., Lin, W., Wang, W., Wang, W., Zhou, W., Wang, W., Shen, W., Yu, W., Shi, X., Huang, X., Xu, X., Kou, Y., Lv, Y., Li, Y., Liu, Y., Wang, Y., Zhang, Y., Huang, Y., Li, Y., Wu, Y., Liu, Y., Pan, Y., Zheng, Y., Hong, Y., Shi, Y., Feng, Y., Jiang, Z., Han, Z., Wu, Z.-F., and Liu, Z. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025.arxiv.org/abs/2503.20314
- [36]Wimbauer, F., Wu, B., Schoenfeld, E., Dai, X., Hou, J., He, Z., Sanakoyeu, A., Zhang, P., Tsai, S., Kohler, J., Rupprecht, C., Cremers, D., Vajda, P., and Wang, J. Cache me if you can: Accelerating diffusion models through block caching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6211–6220, June 2024.arxiv.org/abs/2312.03209
- [38]Xiao, G., Tian, Y., Chen, B., Han, S., and Lewis, M. Efficient streaming language models with attention sinks. In International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2309.17453
- [40]Yang, S., Huang, W., Chu, R., Xiao, Y., Zhao, Y., Wang, X., Li, M.-Xie, E., Chen, Y., Lu, Y., Han, S., and Chen, Y. Longlive: Real-time interactive long video generation. arXiv preprint arXiv:2509.22622, 2025.arxiv.org/abs/2509.22622
- [41]Yin, P., Zhu, J., Gao, H., Zheng, C., Huang, Y., Zhou, T., Yang, R., Liu, W., Chen, W., Guo, C., Deng, D., Mo, Z., Wang, C., Cheng, J., Wang, R., and Liu, H. vllm-omni: Fully disaggregated serving for any-to-any multimodal models. arXiv preprint arXiv:2602.02204, 2026.arxiv.org/abs/2602.02204
- [42]Yin, T., Zhang, Q., Zhang, R., Freeman, W. T., Durand, F., Shechtman, E., and Huang, X. From slow bidirectional to fast autoregressive video diffusion models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 22963–22974. IEEE, 2025.arxiv.org/abs/2412.07772
- [43]Zhang, Z., Sheng, Y., Zhou, T., Chen, T., Zheng, L., Cai, R., Song, Z., Tian, Y., Ré, C., Barrett, C., Wang, Z., and Chen, B. H2O: Heavy-hitter oracle for efficient generative inference of large language models. Advances in neural information processing systems, 36:34661–34710, 2023.arxiv.org/abs/2306.14048
- [44]Zhao, X., Jin, X., Wang, K., and You, Y. Real-time video generation with pyramid attention broadcast. In International Conference on Learning Representations, volume 2025, pp. 3296–3319, 2025.arxiv.org/abs/2408.12588
- [45]Zheng, L., Yin, L., Xie, Z., Sun, C., Huang, J., Yu, C. H., Cao, S., Kozyrakis, C., Stoica, I., Gonzalez, J. E., Barrett, C., and Sheng, Y. Slang: Efficient execution of structured language model programs. Advances in neural information processing systems, 37:625557–62583, 2024.arxiv.org/abs/2312.07104









