WorksComputer Science
TileCache: Serving Multi-Round Diffusion Image Editing with Compressed Activation Caches
Abstract
Multi-round diffusion image editing can reduce computation by reusing intermediate activations, but retaining these activations across editing rounds introduces substantial memory and bandwidth overheads. Compression can reduce these overheads, but efficient cache construction is challenging: compressing only after editing requires buffering over 100 GB of activations per request. Moreover, the compressed cache must support selective reuse, as future masked edits require only a subset of the cached activations. It also exposes a scheduling challenge, as cached edits are faster than initial generations, causing naive policies to severely delay either initial generations or editing requests. In this paper, we aim to address both challenges through compression-informed serving. We develop TileCache, a serving system for multi-round diffusion editing. It compresses reusable activations and incorporates a scheduler that balances both edits and initial generation requests. Compared with the state-of-the-art image editing baseline, TileCache sustains over 2x the request arrival rate at 90% SLO attainment. It completes edits 2.4-4.5x and initial generations 1.3-2.2x faster than the baseline, while preserving image quality.
Introduction
Diffusion models have become an important tool for interactive image creation ([1, 30]), allowing users to generate images and refine them through successive edits ([5] (Qwen Team 2025)). In each round, users provide an instruction and a mask specifying the region to modify. The diffusion model then edits only the masked region according to the instruction while preserving consistency between the unmasked regions and the input image. As interactive workflows become increasingly prevalent, efficient edit serving systems are essential for delivering responsive services despite the substantial computational overhead of diffusion models.
To efficiently serve image editing tasks, prior studies have proposed leveraging mask-aware activation reuse to reduce computation ([13] (Li et al. 2022; Yu et al. 2024)). This technique recomputes activations only for the masked region while reusing cached activations for the unmasked region, thereby avoiding repeated processing of unchanged image regions across multiple editing turns. Despite the computation efficiency, activation reuse in turn requires substantial storage to maintain intermediate activations across editing rounds. For instance, caching activations for FLUX.2 base at resolution over 50 denoising steps requires 107 GB per session (§ 2.1). Maintaining these caches limits the sessions that fit in memory and introduces additional data transfer when users return for subsequent edits.
Cache compression makes activation reuse more efficient by reducing its memory footprint and data transfer ([27 b]), which enables the serving system to retain reusable states across more sessions while reducing data transfer overhead, translating computational savings into end-to-end latency reductions. However, these optimizations create substantial latency disparities between masked edits with reusable caches and requests without cache reuse. Such heterogeneity can bias scheduling decisions toward faster requests, potentially starving requests with higher computational demands. Therefore, reducing cache size alone does not necessarily improve serving quality. Efficiently serving multi-round editing requires not only compressing cached activations but also carefully scheduling requests to balance waiting time and execution interference.
However, achieving this goal poses challenges in both cache compression and request scheduling. Prior studies have shown that diffusion model activations exhibit similarity across both blocks and denoising steps ([16, 24]), suggesting opportunities for multi-dimensional compression. However, naively analyzing similarity after all activations are computed incurs prohibitive memory overhead, as retaining these activations throughout a single editing request can require over 100 GB of storage. Furthermore, since subsequent requests reuse cached activations only in unmasked regions, the compressed representation must support composable reuse, as the regions modified by future edits are unknown in advance. Therefore, the first challenge is how to design an efficient and composable cache compression mechanism to manage the historical cache. For the scheduler, considering only data transfer and computation latency is insufficient, as the substantial memory footprint of cached activations can lead to out-of-memory errors. Moreover, interference among concurrent requests further complicates scheduling decisions. Designing an effective request placement policy that jointly accounts for latency, memory constraints, and inter-request interference remains challenging.
This work introduces TileCache,1 a serving system for multi-round diffusion image editing that combines cache compression with compression-informed scheduling. To address the first challenge, TileCache develops a compression method that exploits redundancy along two dimensions: denoising steps and transformer blocks. Across steps, TileCache maintains a running sum and a few projections for each block, updating them as new activations are generated. This allows raw activations to be discarded immediately after each step, avoiding the need to retain all intermediate activations. Across blocks, TileCache compresses each block as a residual relative to the reconstructed previous block, exploiting the similarity between adjacent blocks. The key insight is that sums and projections are linear: the residual’s sums and projections can be obtained by subtraction, without storing the residual activations. After generation, TileCache processes the blocks sequentially and compresses their residuals using these sums and projections, without accessing the raw activations (§ 3.2).
To address the second challenge, TileCache develops a compression-informed scheduler that accounts for cache memory requirements, execution times, and interference between concurrent requests, all of which depend on the compressed representation. The scheduler first identifies GPUs with sufficient memory to accommodate an incoming request’s model activations and cache buffers. For each feasible placement, it estimates the completion time of the incoming request and the additional delay imposed on requests already running on the GPU. Based on these estimates, TileCache determines whether the request should wait or be colocated. Before colocating a new request on a busy GPU, the scheduler checks whether each running request can finish within a given multiple of its single-request latency. The balance between initial generations and edits changes with the request mix and GPU load. Every few minutes, the scheduler compares the slowdown of the two classes and adjusts the latency budget and delay penalty to
give higher priority to the class that is falling behind (§ 3.3).
We evaluate TileCache on three diffusion models: FLUX.2 base, distilled FLUX.2, and Qwen-Image-Edit. On a H100 platform and 70% edit-request ratio based on prior work ([13] (Li et al. 2025)), TileCache sustains over the arrival rate of the baseline at 90% attainment and, completes edits 2.4–4.5 and initial generations 1.3–2.2 faster. Against the baseline equipped with the same compressed caches, it sustains a 14% higher arrival rate on H100, and at that load it slows edits and initial generations alike, improving fairness by 32% (§ 4.2–§ 5).
We summarize the contributions of this work as follows:
We develop a streaming two-dimensional compression method that builds compressed activation caches without storing all intermediate activations. The resulting caches support selective reconstruction and foreground updates during multi-round editing (§ 3.2).
We design a compression-informed scheduler that considers cache memory requirements, execution times, and interference when placing and colocating requests. It balances initial generations and edits by adjusting its delay budget and penalties based on each request class’s observed latency slowdown (§ 3.3).
We implement these techniques in TileCache and evaluate its serving performance, compression quality, and individual components across different workloads and GPU platforms. We also validate the simulator against real GPU executions (§ 4).
Background and Motivation
Activation Reuse and Cache Overhead
Multi-round image editing allows users to refine an image through editing requests. A typical session starts with an initial image generation, followed by edits that modify selected regions of the image. Each edit specifies an instruction and a mask indicating which parts of the image should change.
Existing work has shown that image editing can be accelerated by reusing intermediate activations from previous rounds ([13] (Li et al. 2022; Yu et al. 2024)). Instead of recomputing the entire image, mask-aware reuse recomputes tokens in the masked area while retrieving cached activations for tokens in the unmasked area. A cache stores residual-stream outputs from transformer blocks across denoising steps, with separate outputs for each guidance branch when applicable.
Cache footprint and transfer costs. Activation reuse introduces substantial storage and transfer overhead. At resolution, an uncompressed cache occupies 107.4 GB for FLUX.2 ([5]) and 87.3 GB for Qwen-Image-Edit-2511 (Qwen Team, [32]). A 400 GB host cache can therefore hold only three to four sessions, even before accounting for other memory usage.
Figure 1 shows both the benefits of activation reuse and the overhead of cache transfers. Our measurements of FLUX.2 show that reusing activations from host memory reduces edit latency from 83.9 s to 29.1 s on an RTX 6000 Ada GPU and from 21.5 s to 10.5 s on an H100. Retrieving the cache from SSD increases the latency to 35.6 s and 17.8 s, including 6.5 s and 7.3 s of exposed transfer time, respectively. The faster H100 hides less of the transfer, making this overhead larger relative to the edit itself. These costs motivate cache compression. Compression, however, only lowers the cost of each edit in isolation. The observed latency also depends on when and where the edit runs, which we examine next.

Figure 1. Activation reuse reduces edit latency, but cache transfer introduces overhead. Latency comparison for FLUX.2 at with 5.9% masked tokens.
Scheduling Trade-offs
Compression also changes where an edit can run. An edit can share a GPU with a running request only if its cache fits in the remaining GPU memory: an uncompressed FLUX.2 cache (107 GB) cannot reside on an 80 GB H100, whereas a compressed package (4.3 GB) can. Compressed caches thus allow edits to run without waiting for an idle GPU, raising the question of when an edit should run and whether it should share a GPU.
This question is difficult because the two request classes use GPU resources differently. Initial generations execute the full diffusion process and produce reusable activations, while subsequent edits perform less computation but require access to cached data. An edit waiting behind an initial generation may spend more time in the queue than it needs for execution. Giving edits higher priority can reduce their waiting time, but may substantially delay initial generations.
Figure 2 illustrates this trade-off under increasing request rates. We report latency as slowdown (Harchol-Balter, [9]), a request’s response latency divided by its single-request latency. We compare first-come-first-served (FCFS) scheduling with colocation against shortest-job-first (SJF) that tends to prioritize shorter editing jobs, both using the same compressed caches. At 0.22 requests/s, edit P95 response latency under FCFS reaches 11.7 times its single-request latency. SJF reduces this ratio to 5.9, but increases initial-generation P95 latency from 5.6 to 7.0 times its single-request latency.

Figure 2. Slowdowns due to FCFS and SJF scheduling.
Sharing a GPU does not remove this trade-off. FCFS already allows colocation, but edit latency remains high. On H100 GPUs, a batched step with two colocated requests takes 93–97% of their combined single-request step latencies, leaving little room for throughput gains. Colocating an edit can reduce its waiting time, but the additional GPU work delays requests already running. The scheduler must
therefore weigh an edit’s waiting time against the delay it would impose, and both depend on the compressed cache: its size determines whether it can stay on the GPU and how much data must be transferred before the edit can start.
Design Challenges
The above observations highlight two challenges in serving multi-round image editing workloads: compressing large activation caches without sacrificing efficient reuse, and scheduling requests whose execution and data-access costs depend on the compressed cache.
Challenge 1: Streaming compression with selective access. Compression must reduce cache storage and transfer costs without sacrificing image quality or introducing significant memory and reconstruction overhead. Cached activations are redundant across denoising steps, which produce similar activations for the same token ([23] (Ma et al. 2024)), and across adjacent transformer blocks, which share a residual stream ([22]). Retaining all raw activations during compression incurs substantial storage overhead, making it necessary to exploit both forms of redundancy in a streaming manner.
Since diffusion models generate activations sequentially, compressing earlier activations limits their availability for subsequent steps, requiring efficient algorithms to preserve the compression quality. And for transformer blocks, predicting each block’s activations from the reconstructed previous block ensures consistent decoding, but stepwise compression requires the complete denoising trajectory, so the previous block’s reconstructed activations are not available when the current block’s activations are generated. Combining the two without retaining raw activations is the core difficulty.
Compression must also support the access pattern of multi-round editing. Reconstructing the full cache across all blocks and denoising steps at once would consume substantial GPU memory. Therefore, the compressed cache must support on-demand reconstruction of activations in the unmasked area and updates to activations in the masked area for subsequent editing rounds.
Challenge 2: Scheduling generations and edits. As shown in Figure 2, FCFS and SJF each lead to high latency for one request class. This trade-off appears even at a moderate edit ratio. When edits account for 50% of requests, prioritizing them can substantially delay initial generations, while FCFS leaves edits waiting behind long-running requests. Thus, the scheduler must adjust its priorities based on the observed latency of each class rather than using a fixed policy.
The scheduler must also decide when to schedule an edit and whether to colocate it with a running request. These decisions depend on the edit’s expected waiting time, the remaining work of running requests, and the additional delay caused by colocation. GPU memory further constrains placement decisions, as the model, running requests, and incoming edit must share the available memory, and both an edit’s memory need and the delay it causes depend on the size and location of its compressed cache. The compressed cache determines an edit’s memory requirements, cache loading time, and per-step reconstruction work, which affect placement feasibility and colocation interference.
Cache transfers also affect scheduling. After an initial generation, the newly built cache must be transferred out of GPU memory before that memory can be released. An edit must load cached activations and write back regenerated activations in the masked area for subsequent rounds. These transfers share bandwidth and can delay cache availability or GPU memory release. An edit may therefore be unable to start even when GPU compute is available.
TileCache Design
Overview
Figure 3 shows TileCache’s architecture and cache lifecycle across an image-editing session. The system consists of a scheduler, GPU workers, and a compressor. Each request includes a session ID and a prompt; edit requests also include a mask specifying the region to modify. The scheduler uses the compressed cache’s memory requirements and access costs to guide request placement, with class-level latency feedback helping balance latency between initial generations and edits. The cache lifecycle has three stages:
Figure 3. Overview of TileCache in multi-round editing.
(1) During initial generation, a worker generates the full image. The compressor accumulates compact summaries of each block’s activations on the GPU as denoising proceeds, without retaining raw activations from all steps (§ 3.2).
(2) Once generation completes, the compressor uses these summaries to build a compressed cache package on the GPU. The package is transferred to pinned host memory for reuse and written to SSD for longer-term storage, releasing the GPU memory occupied by the cache. (§ 3.2)
(3) For a subsequent edit, TileCache retrieves the cache and schedules the request (§ 3.3). The worker reuses cached activations for the unmasked region, recomputes masked activations, and updates the cache for future edits.
Two-Dimensional Activation Compression
An editing cache contains activations collected across multiple denoising steps and transformer blocks. TileCache exploits redundancy along both dimensions through step-
Algorithm 1 Compression-Informed Scheduling
Scope: Incoming edit $r$; all GPUs are busy.
State: Delay budget $\tau_{\mathrm{eff}}$ and class weight $w$.
1: if the latency-feedback window has completed then
2: Compute class P95 slowdowns from requests completed in the window
3: if at least one class has sufficient samples then
4: Set $\tau_{\mathrm{eff}}$ to the larger of $\tau$ and the highest available P95 slowdown
5: if both classes have sufficient samples then
6: Adjust $w$ using the bounded ratio of initial-generation to edit P95 slowdowns
7: WAIT.impact $\leftarrow$ single-request latency + estimated waiting time
8: options $\leftarrow \{\mathrm{WAIT}\}$
9: for each busy GPU $g$ do
10: cache $\leftarrow \mathrm{FITCACHE}(r,g)$ // GPU first; host streaming
11: if cache is infeasible then continue
12: for each mode in $\{\mathrm{shared},\mathrm{pausing}\}$ do
13: option $\leftarrow (g,\mathrm{cache},\mathrm{mode})$
14: Estimate running-request latencies for option from profiles
15: if any running request’s estimated slowdown exceeds $\tau_{\mathrm{eff}}$ ({xref}`Eq. 3`) then
16: continue
17: option.impact $\leftarrow \mathrm{LATENCYIMPACT}(r,\mathrm{option},w)$ ({xref}`Eq. 2`)
18: Add option to options
19: return the lowest-impact option in options
wise and blockwise compression. It accumulates sums and stepwise projections during generation, then uses them to compress residuals between adjacent transformer blocks without accessing the raw cache (Figure 3). We organize cached activations into spatial rows, each containing all channels of one horizontal row of image tokens across the cached denoising steps. Each row serves as the unit of compression and reconstruction, allowing partial edits without decompressing the entire cache.
Online accumulation and blockwise compression. To capture redundancy across denoising steps without retaining the full activation sequence, TileCache accumulates a running sum and stepwise projections as activations arrive. The running sum captures the mean activation across denoising steps, while the projections capture variations around that mean. TileCache uses a fixed DCT dictionary (Ahmed et al., [4]) that provides predefined patterns across denoising steps, allowing these projections to be updated at each step without accessing earlier activations. For each transformer block , we denote the accumulated sum and DCT projections as and , respectively. The accumulated sums and projections provide the inputs for subsequent blockwise residual compression and basis fitting without retaining the raw activation sequence. This limits GPU memory use during initial generations.
Adjacent transformer blocks often produce similar activations, making their differences easier to compress than the activations themselves. TileCache therefore compresses the first transformer block directly and represents each subsequent block as the difference between its activations and the reconstructed activations of the preceding transformer block. Let and denote the original and reconstructed activation sequences of transformer block , respectively, and let
Because sums and projections are linear, TileCache computes the residual’s sums and projections by subtracting those of the reconstructed previous block from those accumulated for the current block, without storing raw residuals:
where and denote the sum and DCT projections across denoising steps, respectively. After compressing a residual, TileCache reconstructs the current block one step at a time, updates the sums and projections needed for the next block, and discards each reconstructed step immediately afterward.
After generation, TileCache uses the accumulated stepwise projections to adapt the fixed DCT dictionary into a data-dependent stepwise basis for quantization. For the first block, it computes the mean and eigendecomposes the Gram matrix of the stepwise projections to obtain a rotation. Subsequent blocks apply the same procedure to their blockwise residuals. The compressor then assigns bit widths to the mean and coefficients under a fixed budget per image row using a greedy Gaussian distortion model (Huang & Schulthess, [12]). The distortion model weighs mean error by the number of cached denoising steps and each coefficient’s error by 1. Some coefficient slots may receive zero bits. For each mean or coefficient, TileCache uses a shared bit width across channels but computes quantization offsets and step sizes separately for each channel. The quantized values are then bit-packed into the cache package. Package size affects the memory and transfer costs of subsequent edits.
Reconstruction and partial edits. During an edit, the decompressor reconstructs the rows containing required unmasked tokens block by block, by adding the decoded residual to the preceding block’s reconstructed activations. The worker recomputes tokens in the masked area and updates their cached activations for subsequent editing rounds, while keeping the remaining activations compressed. Each row carries a version tag identifying its basis and quantization parameters. This way, rows compressed in different rounds can coexist, avoiding the overhead of fully reconstructing and recompressing the cache after each edit. Updating only modified rows also reduces write-back traffic.
Compression-Informed Scheduling
Placement by latency impact. Waiting and interference are two main sources of additional latency that can cause SLO misses. Colocating an edit can reduce its waiting time, but may delay requests already running on the GPU. Conversely, avoiding colocation can leave edits waiting long enough to miss their latency targets. Therefore, TileCache estimates the latency impact of each placement, defined as the incoming request’s response time plus the additional delay it imposes on running requests, and selects the placement with the lowest impact.
When a GPU is idle, an incoming request runs on it immediately. Otherwise, TileCache decides whether the request should wait for a GPU or, for an edit, be colocated on a busy GPU. To support colocation, TileCache adopts FlashPS’s step-level continuous batching [13], using one of two execution modes. In shared execution, colocated requests run together in batched steps. In pausing, initial generations are temporarily suspended while the incoming edit executes, and other edits can continue running. The scheduler evaluates both modes for each feasible colocation. Algorithm 1 summarizes the scheduling process for an incoming edit when all GPUs are busy.
The compressed cache’s memory requirements, staging time, and per-step execution costs determine which colocations are feasible and their latency and interference costs (§ 3.2). Each placement must first fit in GPU memory alongside the model weights and the running requests. Initial generations require model activations and online accumulation buffers, which replace the activations from all denoising steps. Edits additionally require compressed caches, reconstruction buffers, and accumulation buffers for updated activations. TileCache keeps compressed caches on the GPU when memory permits and streams them from host memory otherwise. Their smaller footprint creates more opportunities for GPU-side colocation, and the package size and location set the transfer and reconstruction time of each edit, as estimated below.
For each feasible colocation, TileCache estimates the incoming edit’s latency from cache staging (), execution (), and cache updates (), using profiled execution and transfer costs. Cache staging prepares the compressed package for execution, with its cost depending on the cache size, location, and exposed transfer overhead. Execution includes reconstruction in the unmasked area and recomputation in the masked area, while row-level updates limit recompression and write-back to modified rows. TileCache also estimates the total additional delays to running edits () and initial generations (). Under shared execution, TileCache estimates the additional delay to each running request from the incoming edit’s profiled per-step execution cost and the number of denoising steps over which the two requests overlap. Under pausing, an initial generation incurs the full pause duration, while delays to concurrently running edits are estimated using the same overlap-based model. A running initial generation must also complete GPU-side compression before releasing its buffers. Delaying it can therefore postpone GPU availability for queued initial generations. TileCache increases the weight of such delays based on the number of queued initial generations , normalized by the total number of GPUs . An adaptive weight further adjusts this weight based on the observed latency imbalance between the two request classes, as described below. The resulting latency impact of colocating edit on GPU is
Waiting has latency impact equal to the request’s single-request latency plus the estimated time until a GPU becomes available. The scheduler compares this impact with those of colocation and selects the lowest-impact action satisfying the delay constraints described below. By accounting for both waiting and interference, TileCache reduces unnecessary waiting without ignoring the delays imposed on running requests.
Latency-feedback balancing. Latency impact aggregates the delays of all affected requests, so selecting the lowest-impact placement may still cause excessive delays to individual requests or repeatedly impose more delay on one request class. TileCache therefore incorporates class-level latency feedback to adjust the relative weight of interference.
To prevent a low-impact placement from excessively delaying individual requests, TileCache applies a delay budget before allowing colocation. Since initial generations and edits have different single-request latencies, the same additional delay can result in different relative slowdowns. The budget therefore limits each running request’s estimated slowdown, consistent with the SLO normalization (§ 4.1). The scheduler estimates each request’s completion time using its remaining steps and profiled execution costs under the selected execution mode. A base threshold () controls how much slowdown is allowed. Under high load, a fixed delay budget can reject colocation even when running requests have already accumulated substantial delays. This can leave more incoming edits waiting until they miss their SLOs. Adapting the budget allows the scheduler to consider these placements while still accounting for their interference delays. The scheduler periodically computes the P95 slowdown for each request class from requests completed in the preceding window, requiring at least two samples for a class to be included. TileCache takes the largest available P95 slowdown as to adjust the effective delay budget.
The delay budget restricts colocation but does not determine how the scheduler balances the two request classes among feasible placements. TileCache therefore adjusts the weight in Eq. (2) using a bounded ratio of the P95 slowdowns for initial generations () and edits (). Larger slowdowns for initial generations increase the weight of delaying them, whereas larger slowdowns for edits reduce this weight and encourage colocation. This feedback adjusts how colocation interference is distributed between the two classes based on their observed tail latencies. The weight is updated only when both classes have sufficient samples.
For each candidate colocation on GPU , the scheduler checks the running requests () using each request’s arrival time (), estimated completion time (), and single-request latency (). The effective delay budget () is the larger of and . Colocation is allowed only if
If a window completes fewer than two requests in both classes, the scheduler keeps the effective budget from the previous window.
Impact-based placement can leave initial generations waiting when edits repeatedly occupy newly available GPUs. To prevent excessive waiting, initial generations waiting longer than their single-request latency receive priority on available GPUs, with the longest-waiting request scheduled first. During this time, pending edits can only join busy GPUs.
Evaluation
Setup and Methodology
Platforms and implementation. We implement TileCache’s compression and decompression using PyTorch (Paszke et al. [31]) and Triton (Tillet et al. [39]). Its serving runtime consists of a coordinator, one worker per GPU, a shared pinned-memory pool, and direct SSD I/O. We deploy TileCache on 4× H100 SXM GPUs (80 GB each) and 4× RTX 6000 Ada GPUs (48 GB each). For serving evaluation, we use a discrete-event simulator that runs the runtime’s scheduling code unmodified, calibrated with GPU execution and transfer measurements from each platform. We validate the simulator against GPU execution at 0.13–0.28 requests/s on H100 and 0.04–0.07 requests/s on RTX 6000 Ada. Simulated class-mean latency differs by at most 1.7%, P95 slowdown by at most 0.05, and 95% of scheduling decisions match those of the GPU runtime. The default storage configuration has 400 GB of host cache and 4 TB of SSD, maintaining a 10:1 SSD-to-host-cache capacity ratio throughout the cache-size sensitivity experiments.
Models and workloads. Unless noted otherwise, serving experiments use FLUX.2 base (Black Forest Labs [5]) at 1024 × 1024 resolution and 50 denoising steps. We also evaluate distilled FLUX.2 (4 steps) and Qwen-Image-Edit (Qwen Team [32]) (50 steps). Sessions follow Midjourney iterative-prompt threads (Don-Yehiya et al. [6]), with Poisson session arrivals (Jiang et al. [13, 14]), and masked areas sampled from the FlashPS editing trace (Jiang et al. [13]). The default workload contains 70% edits, following a diffusion serving trace (Li et al. (2025)) and FlashPS (Jiang et al. [13]). Each run includes a 24-hour warm-up followed by 48 hours of measurement.
Baselines. FlashPS (Jiang et al. [13]) uses lossless cache reuse and its original scheduler; caches are preloaded, and SSD writes are excluded. FlashPS+Compression uses the same compressed caches as TileCache but retains the FlashPS scheduler, isolating the benefit of compression-informed scheduling.
Serving metrics and load. SLO attainment is the fraction of requests completing within 3× their class’s single-request latency (§ 3.3) (Li et al. (2023); ); Zhong et al. [49]). Following DistServe (Zhong et al. [49]), goodput is the highest arrival rate that achieves at least 90% SLO attainment. We measure fairness between initial generations and edits as (Tavakkol et al. [38]), where is class ’s mean latency normalized by its single-request latency; indicates equal slowdowns. We sweep arrival rates through and slightly beyond TileCache’s goodput. We set the base delay budget to and both the latency-feedback interval and window size to 300 s (§ 3.3). The sensitivity of this delay budget is evaluated in Appendix § 8.§ 1.
Image quality. We evaluate editing quality on MagicBrush (Zhang et al. [48]), PIE-Bench (Ju et al. (2024)), and five-round ImgEdit sessions (Ye et al. (2025)) at 512 × 512 or 496 × 496 resolution. We use CLIP similarity (Radford et al. [33]) to measure how well an edited image matches its text prompt. Peak signal-to-noise ratio (PSNR) measures pixel-level similarity to a reference image, while structural similarity (SSIM) (Wang et al. [42]) captures similarity in image structure. Fréchet Inception Distance (FID) (Heusel et al. [10]) compares the feature distributions of generated and reference images. Higher CLIP, PSNR, and SSIM scores and lower FID indicate better quality. For PSNR, SSIM, and FID, we use the unmodified Diffusers pipeline (von Platen et al. [41]), which recomputes all image tokens without activation reuse. We also compare PSNR and SSIM against FlashPS’s lossless reuse (Jiang et al. [13]) to identify compression error.
End-to-End Serving Performance
With lossless reuse, FlashPS never reaches the 90% SLO attainment target, as its large uncompressed caches limit the number of sessions that fit in the 400 GB host cache (Figure 4). Its attainment is already 82% at 0.125 requests/s, less than half of TileCache’s goodput. Across the loads it sustains, its edits take 2.4–4.5× and its initial generations 1.3–2.2× longer than under TileCache. Giving FlashPS the same compressed caches as TileCache removes this limit. TileCache then achieves a goodput of 0.26 requests/s, 14% higher than FlashPS+Compression. Since both use the same compressed caches, the improvement comes from compression-informed scheduling.

Figure 4. Serving performance on 4× H100 with 70% edits.
At 0.25 requests/s, TileCache improves SLO attainment over FlashPS+Compression from 80.4% to 94.7% and also the fairness. It reduces edit slowdown by 38% on average, with initial-generation slowdown nearly unchanged under FlashPS+Compression. This improvement comes from comparing the estimated latency costs of colocation and waiting while limiting interference with ongoing requests (§ 3.3). TileCache maintains through 0.25 requests/s, but fairness declines to 0.68 at 0.275 requests/s, beyond its goodput, as more initial generations wait for idle GPUs.
Compression Quality
(1) Editing quality. Despite achieving 23.1–26.4× compression for the 50-step models and 4.6× for distilled FLUX.2, TileCache preserves editing quality close to FlashPS’s lossless reuse (Table 1). Across the evaluated datasets, including five-round editing sessions, CLIP scores differ by at most 0.1, while direct output comparisons yield 34.0–46.4 dB PSNR and 0.973–0.996 SSIM.
| Model | Dataset | Ratio | CLIP | Deviation from the Diffusers edit (no reuse) | Error added by compression | ||||||||
| Diffusers | FlashPS | TileCache | PSNR FlashPS | PSNR TileCache | SSIM FlashPS | SSIM TileCache | FID FlashPS | FID TileCache | TileCache vs. FlashPS edit PSNR | TileCache vs. FlashPS edit SSIM | |||
| FLUX.2 distilled | MagicBrush | 4.6× | 26.1 | 26.2 | 26.2 | 27.1 | 26.5 | 0.937 | 0.937 | 31.5 | 31.5 | 43.4 | 0.994 |
| PIE-Bench | 4.6× | 26.2 | 26.0 | 26.0 | 21.6 | 21.6 | 0.868 | 0.868 | 60.6 | 60.8 | 38.2 | 0.989 | |
| ImgEdit (5 rounds) | 4.6× | 27.6 | 27.8 | 27.8 | 23.9 | 23.9 | 0.911 | 0.911 | 21.4 | 21.5 | 42.2 | 0.993 | |
| FLUX.2 base | MagicBrush | 25.1× | 26.0 | 26.1 | 26.1 | 33.4 | 33.1 | 0.974 | 0.974 | 9.8 | 9.6 | 46.4 | 0.996 |
| PIE-Bench | 23.1× | 25.3 | 25.2 | 25.2 | 29.9 | 29.8 | 0.962 | 0.962 | 20.1 | 20.4 | 43.5 | 0.994 | |
| ImgEdit (5 rounds) | 24.6× | 27.7 | 27.7 | 27.7 | 28.7 | 28.6 | 0.951 | 0.951 | 16.2 | 16.2 | 45.3 | 0.996 | |
| Qwen-Image-Edit | MagicBrush | 26.4× | 26.2 | 26.4 | 26.5 | 28.3 | 28.3 | 0.946 | 0.947 | 19.4 | 19.1 | 36.7 | 0.980 |
| PIE-Bench | 23.1× | 25.9 | 25.7 | 25.7 | 33.4 | 33.4 | 0.948 | 0.947 | 26.6 | 26.0 | 40.6 | 0.981 | |
| ImgEdit (5 rounds) | 25.7× | 27.8 | 27.9 | 27.9 | 24.1 | 24.1 | 0.903 | 0.903 | 25.6 | 26.0 | 34.0 | 0.973 |
Table 1. End-to-end edit quality of TileCache and FlashPS
(2) Comparison with KVTC. At comparable compression ratios, TileCache improves masked-area PSNR over KVTC (Staniszewski & Łancucki, [36]) by 4.3–13.4 dB on 50-step FLUX.2 base and Qwen-Image-Edit (Figure 5a–Figure 5b). On 4-step distilled FLUX.2, TileCache achieves 2.4 dB higher PSNR at 4.5× compression (Figure 5c).

Figure 5. Masked-area PSNR on MagicBrush.
(3) Multiple editing rounds. The quality advantage persists across repeated edits (Figure 6). Over 200 five-round ImgEdit sessions, masked-area PSNR relative to lossless reuse decreases from 39.5 to 36.0 dB for FLUX.2 base and from 35.5 to 32.8 dB for distilled FLUX.2. TileCache still outperforms KVTC in every round, retaining its quality advantage across successive edits.

Figure 6. Masked-area PSNR across 200 ImgEdit sessions.
Ablation Study
(1) Compression components. Compared with stepwise compression using fixed 8-bit quantization, greedy bit allocation improves masked-area PSNR by 6.3 dB on FLUX.2 base and 4.6 dB on Qwen-Image-Edit, while blockwise residuals further improve 1.2 and 1.3 dB, respectively (Figure 7a). For distilled FLUX.2, fixed 8-bit quantization cannot achieve the target 4.5× compression ratio. With only 4 denoising steps, stepwise redundancy is limited, making blockwise residuals particularly valuable. The resulting PSNR is 2.8 dB higher at this ratio and exceeds KVTC by 2.4 dB. Against KVTC, TileCache achieves its largest gain at 4.5×, as shown in Figure 5c. At 2.5×, both compressors exceed 38 dB, leaving little room for improvement; at 7×, the advantage narrows to 0.4 dB under a tight bit budget. Appendix A.2 reports 6- and 8-step distilled versions.

Figure 7. Ablation study on TileCache components.
(2) Scheduler mechanisms. Impact-based placement is critical to SLO attainment (Figure 7b). Without it, edits join busy GPUs whenever memory permits, increasing interference and reducing attainment from 94.8% to 82.6% at 0.25 requests/s. Goodput also drops by 11%. Although fairness increases from 0.90 to 0.94, both classes experience higher slowdowns, showing that fairness alone does not capture serving performance.
Other mechanisms primarily improve fairness. Removing the delay budget, latency feedback, or initial-generation priority reduces fairness from 0.90 to 0.79, 0.82, and 0.72, respectively, with negligible impact on SLO attainment. Appendix A.1 further studies the delay budget.
Scheduler Comparison and Sensitivity Analysis
(1) Scheduling policies. To isolate scheduling effects, we compare TileCache with FIFO, shortest-job-first (SJF), deficit round robin (DRR) ([34]), and earliest-deadline-first (EDF), all using the same compressed caches (Figure 8). Each baseline dispatches the head of its queue to an idle GPU, or colocates it on a busy GPU when memory permits; DRR always colocates, while FIFO, SJF, and EDF do so only when it beats waiting for a free GPU. EDF sets each request’s deadline to its arrival time plus three times its single-request latency (i.e., the SLO). DRR assigns edits twice the GPU-time weight of initial generations, reflecting their 70% request share and greater sensitivity to waiting. Since edits have roughly half the single-request latency, the same waiting time contributes approximately twice as much to their slowdown. DRR falls short of TileCache’s goodput by 11%, and SJF and EDF by 23%. At 0.25 requests/s, TileCache achieves fairness of 0.91, compared with 0.74 for DRR, 0.59 for SJF, and 0.58 for EDF. FIFO achieves 20% lower goodput and a fairness of 0.54. All four baselines delay edits more than initial generations, since colocated edits run at the slower batched step rate. At 0.275 requests/s, TileCache still meets SLO for 85% of requests, compared with 60–69% for the baselines.

Figure 8. Scheduling comparison on 4× H100 with 70% edits.
(2) GPU platform. TileCache retains its advantage on different GPU platforms (Figure 9). Goodput improves over FlashPS+Compression by 12% on RTX 6000 Ada and 14% on H100. The smaller gain on Ada is consistent with its 48 GB GPU memory, which leaves fewer opportunities for edits to join busy GPUs. At FlashPS+Compression’s goodput on each platform, TileCache increases fairness from 0.77 to 0.99 on Ada and from 0.74 to 0.98 on H100.

Figure 9. GPU platform sensitivity with 70% edits.
(3) Model. The gains extend across models with different denoising lengths (Figure 10). TileCache improves goodput over FlashPS+Compression by 14% on FLUX.2 base, 22% on 4-step distilled FLUX.2, and 20% on Qwen-Image-Edit. The gain on Qwen-Image-Edit is due to SSD reads on FlashPS+Compression’s critical path following host-cache misses. TileCache also improves fairness by 19–21% on Qwen-Image-Edit at 0.16–0.18 requests/s and by 14% on distilled FLUX.2 at 4.6 requests/s.

Figure 10. Model sensitivity on H100 across diffusion models.
(4) Edit fraction. The scheduling benefit persists across mixtures with 20–90% edits (Figure 11a). TileCache delivers 8% higher goodput than FlashPS+Compression at 20% edits and 12–14% higher goodput at 50–90% edits, where more incoming edits can join busy GPUs. At the baseline’s goodput for each mixture, TileCache maintains fairness of 0.94–0.99, compared with 0.68–0.76 for FlashPS+Compression.

Figure 11. Edit-fraction and cache-size sensitivity on 4 × H100.
(5) Cache capacity. TileCache remains effective even with a small host cache (Figure 11b). As host cache capacity decreases 16× from 800 to 50 GB, TileCache’s cache hit rate falls from 96% to 32%, yet its goodput drops by only 4%. Across this range, TileCache maintains higher goodput and fairness than FlashPS+Compression. Appendix § 8.§ 3 breaks down how edits are served at each cache size.
Related Work
KV caching and compression. LLM serving systems compress KV state with techniques such as cross-layer prediction, transform coding, and low-bit quantization ([40, 35, 36, 17, 47]), and cache it across requests in host-memory, disk, or GPU/CPU tiers with loads overlapped against compute or streamed in compressed form ([7, 26] (Yu et al. 2025)). These methods compress completed KV states for reuse by later requests. TileCache instead accumulates compact statistics as DiT block outputs are generated, exploiting redundancy across denoising steps and blocks without retaining the activations from all denoising steps. It also uses the compressed cache’s memory and transfer costs to decide whether an edit should join a GPU running an initial generation to achieve better SLO attainment.
Diffusion reuse. Step-level caching reuses or forecasts features across denoising steps within one generation ([23 c] (Ma et al. 2024)), and approximate caching resumes generation from noisy latents cached from similar prompts ([2, 37]); neither holds the per-step block outputs that masked editing of a specific image requires. SIGE (Li et al., 2022) and FISEdit (Yu et al., 2024) cache the original image’s per-step activations and recompute only edited regions with sparse kernels. FlashPS (Jiang et al., [13]) brings this into a serving system, caching large, uncompressed block outputs and batching edits at step granularity, but it treats both the cache’s size and its creation as outside the serving problem. TileCache tackles both problems by compressing cached activations to keep the working set in memory and scheduling the initial generations that create caches with the edits that consume them.
Scheduling for model serving. Model serving systems schedule LLM requests at iteration granularity over paged KV memory ([18] (Yu et al. 2022)) and exploit predictable execution costs and per-request latency state to meet SLOs ([8, 11]); diffusion serving systems route queries through model cascades or vary per-request parallelism across denoising steps to meet deadlines ([3] (Lu et al. 2026)). None of these systems address scheduling for a cache lifecycle which involves the initial generation and subsequent edits that reuse the cache. TileCache uses the resulting memory and latency costs to compare waiting with colocation. It allows an edit to join a busy GPU only when memory permits and the estimated slowdown of running requests remains within the delay budget.
Conclusions
This paper presents TileCache, a serving system for multi-round diffusion editing. It compresses reusable activations and incorporates a scheduler that balances both edits and initial generation requests. Streaming two-dimensional compression avoids retaining raw activations, and compression-informed colocation bounds the delay an edit imposes on running requests. We evaluate TileCache with three diffusion models and demonstrate over 2× the arrival rate of FlashPS at 90% SLO attainment and 14–22% more than FlashPS equipped with the same compressed caches, with edits completing 2.4–4.5× and initial generations 1.3–2.2× faster, without noticeable quality difference.
Additional Results
Delay-Budget Sensitivity
The delay budget improves fairness with little impact on goodput (Table A1). Removing the delay budget allows edits to colocate more freely, but increases goodput by only 2%. At 0.25 requests/s, fairness falls from 0.91 at to 0.86 at , 0.81 at , and 0.80 without the budget, as looser budgets allow edits to join busy GPUs and delay initial generations. At 0.275 requests/s, beyond goodput, removing the budget reduces fairness from 0.68 to 0.56. Since achieves nearly the same fairness as , we use , the largest tested budget that maintains at 0.25 requests/s.
| Delay budget | Goodput (%) | 0.25 requests/s SLO attainment (%) | 0.25 requests/s Fairness | 0.275 requests/s SLO attainment (%) | 0.275 requests/s Fairness |
| 1.5 | -0.1 | 94.6 | 0.91 | 84.4 | 0.69 |
| 2 (default) | 0 | 94.7 | 0.91 | 84.6 | 0.68 |
| 3 | +0.4 | 95.3 | 0.86 | 84.8 | 0.68 |
| 5 | +0.2 | 94.8 | 0.81 | 85.0 | 0.62 |
| (none) | +1.9 | 95.1 | 0.80 | 87.4 | 0.56 |
Table A1. Delay-budget sensitivity on H100.
Distilled FLUX.2 with More Denoising Steps
TileCache’s advantage over KVTC grows with the number of denoising steps at higher compression ratios (Figure A1). At 4.5× compression, it improves masked-area PSNR by 2.4, 3.8, and 4.0 dB at 4, 6, and 8 steps, respectively. Blockwise residuals further improve PSNR by 2.5–3.0 dB over stepwise-only compression. At 2.5×, both methods achieve 38–41 dB, with differences below 0.6 dB. At 7×, TileCache’s advantage grows from 0.4 dB at 4 steps to 2.0 and 3.2 dB at six and eight steps, as longer denoising trajectories provide more opportunities to exploit redundancy across steps.

Figure A1. Masked-area PSNR on MagicBrush for distilled FLUX.2 at 4, 6, and 8 denoising steps.
Host-Cache Hit Rates
A smaller host cache shifts more edits from host-memory hits to SSD fetches (Table A2). As host-cache capacity shrinks from 800 GB to 50 GB, with SSD capacity set to ten times host capacity, the host-cache hit rate falls from 96% to 32%, the share of edits requiring SSD fetches rises from 3% to 62%, and the share whose caches are no longer available on SSD increases from 1.1% to 5.6%.
| Host cache (GB) | 50 | 100 | 200 | 400 | 800 |
| Hit rate (%) | 32 | 58 | 82 | 93 | 96 |
Table A2. Host-cache hit rate versus host-cache size at 0.25 requests/s on H100 with 70% edits.
References
- [1]Adobe. Photoshop Generative Fill, 2025. URL https://www.adobe.com/products/photoshop/generative-fill.html.
- [2]Agarwal, S., Mitra, S., Chakraborty, S., Karanam, S., Mukherjee, K., and Saini, S. K. Approximate caching for efficiently serving text-to-image diffusion models. In Proceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI), pp. 1173–1189, 2024.
- [3]Ahmad, S., Yang, Q., Wang, H., Sitaraman, R. K., and Guan, H. DiffServe: Efficiently serving text-to-image diffusion models with query-aware model scaling. In Proceedings of Machine Learning and Systems (MLSys), volume 7, 2025.arxiv.org/abs/2411.15381
- [4]Ahmed, N., Natarajan, T., and Rao, K. R. Discrete cosine transform. IEEE Transactions on Computers, C-23(1): 90–93, 1974. doi: 10.1109/T-C.1974.223784.DOI
- [5]Black Forest Labs. FLUX.2 [klein] 9B Base, January 2026. URL https://huggingface.co/black-forest-labs/FLUX.2-klein-base-9B.
- [6]Don-Yehiya, S., Choshen, L., and Abend, O. Human learning by model feedback: The dynamics of iterative prompting with Midjourney. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4146–4161, 2023. doi: 10.18653/v1/2023.emnlp-main.253.
- [7]Gao, B., He, Z., Sharma, P., Kang, Q., Jeyvdić, D., Deng, J., Yang, X., Yu, Z., and Zuo, P. Cost-efficient large language model serving for multi-turn conversations with CachedAttention. In Proceedings of the USENIX Annual Technical Conference (ATC), pp. 111–126, 2024.arxiv.org/abs/2403.19708
- [8]Gujarati, A., Karimi, R., Alzayat, S., Hao, W., Kaufmann, A., Vigfusson, Y., and Mace, J. Serving DNNs like clockwork: Performance predictability from the bottom up. In Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI), pp. 443–462, 2020.arxiv.org/abs/2006.02464
- [9]Harchol-Balter, M. Performance Modeling and Design of Computer Systems: Queueing Theory in Action. Cambridge University Press, 2013. ISBN 978-1-107-02750-3.
- [10]Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Advances in Neural Information Processing Systems (NeurIPS), volume 30, pp. 6626–6637, 2017.
- [11]Hong, K., Li, X., Chen, L., Mao, Q., Dai, G., Ning, X., Yan, S., Liang, Y., and Wang, Y. SLOA: Optimizing SLO attainment for large language model serving with state-aware scheduling. In Proceedings of Machine Learning and Systems (MLSys), volume 7, 2025.
- [12]Huang, J. J. Y. and Schultheiss, P. M. Block quantization of correlated Gaussian random variables. IEEE Transactions on Communications Systems, CS-11(3):289–296, 1963. doi: 10.1109/TCOM.1963.1088759.
- [13]Jiang, X., Li, S., Yang, L., Feng, T., Di, Z., Lu, W., Zhu, G., Lin, X., Liu, K., Yu, Y., Lan, T., Yang, G., Qu, L., Zhang, L., and Wang, W. FlashPS: Efficient generative image editing with mask-aware caching and scheduling. In Proceedings of the 21st European Conference on Computer Systems (EuroSys), pp. 2109–2125, 2026. doi: 10.1145/3767295.3769379.DOI
- [14]Jiang, Y., Fu, F., Yao, X., Wang, T., Cui, B., Klimovic, A., and Yoneki, E. ThunderServe: High-performance and cost-efficient LLM serving in cloud environments. In Proceedings of Machine Learning and Systems (MLSys), volume 7, 2025.arxiv.org/abs/2502.09334
- [16]Kahatapitiya, K., Liu, H., He, S., Liu, D., Jia, M., Zhang, C., Ryoo, M. S., and Xie, T. Adaptive caching for faster video generation with diffusion transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 15240–15252, 2025. doi: 10.1109/ICCV51701.2025.01414.DOI
- [17]Kai, J., Wang, Y., Zeng, B., Bai, H., Jiang, B., He, Z., and Lin, Z. FreqKV: Key-value compression in frequency domain for context window extension. In International Conference on Learning Representations (ICLR), 2026.arxiv.org/abs/2505.00570
- [18]Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., and Stoica, I. Efficient memory management for large language model serving with PagedAttention. In Proceedings of the 29th ACM Symposium on Operating Systems Principles (SOSP), pp. 611–626, 2023. doi: 10.1145/3600006.3613165.DOI
- [22]Liu, A., Liu, J., Pan, Z., He, Y., Haffari, G., and Zhuang, B. MiniCache: KV cache compression in depth dimension for large language models. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, pp. 139997–140031, 2024a. doi: 10.52202/079017-4443.DOI
- [23]Liu, F., Zhang, S., Wang, X., Wei, Y., Qiu, H., Zhao, Y., Zhang, Y., Ye, Q., and Wan, F. Timestep embedding tells: It’s time to cache for video diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7353–7363, 2025a. doi: 10.1109/CVPR52734.2025.00689.
- [24]Liu, J., Wang, X., Lin, Y., Wang, Z., Wang, P., Cai, P., Zhou, Q., Yan, Z., Yan, Z., Shi, Z., Zou, C., Ma, Y., and Zhang, L. A survey on cache methods in diffusion models: Toward efficient multi-modal generation. arXiv preprint arXiv:2510.19755, 2025b.arxiv.org/abs/2510.19755
- [26]Liu, Y., Li, H., Cheng, Y., Ray, S., Huang, Y., Zhang, Q., Du, K., Yao, J., Lu, S., Ananthanarayanan, G., Maire, M., Hoffmann, H., Holtzman, A., and Jiang, J. CacheGen: KV cache compression and streaming for fast large language model serving. In Proceedings of the ACM SIGCOMM 2024 Conference, pp. 38–56, 2024b. doi: 10.1145/3651890.3672274.
- [27]Liu, Z., Yuan, J., Jin, H., Zhong, S., Xu, Z., Braverman, V., Chen, B., and Hu, X. KIVI: A tuning-free asymmetric 2bit quantization for KV cache. In Proceedings of the International Conference on Machine Learning (ICML), pp. 32332–32344, 2024c.DOI
- [30]Midjourney. Editor – Midjourney, 2025. URL https://docs.midjourney.com/hc/en-us/articles/32764383466893-Editor.
- [31]Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köp f, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems (NeurIPS), volume 32, pp. 8026–8037, 2019.arxiv.org/abs/1912.01703
- [32]Qwen Team. Qwen-Image-Edit-2511, December 2025. URL https://huggingface.co/Qwen/Qwen-Image-Edit-2511.
- [33]Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning (ICML), volume 139 of PMLR, pp. 8748–8763, 2021.arxiv.org/abs/2103.00020
- [34]Shreedhar, M. and Varghese, G. Efficient fair queuing using deficit round-robin. IEEE/ACM Transactions on Networking, 4(3):375–385, 1996. doi: 10.1109/90.502236.DOI
- [35]Shutova, A., Malinovskii, V., Egiazarian, V., Kuznedelev, D., Mazur, D., Surkov, N., Ermakov, I., and Alistarh, D. Cache me if you must: Adaptive key-value quantization for large language models. In Proceedings of the 42nd International Conference on Machine Learning (ICML), volume 267 of Proceedings of Machine Learning Research, pp. 55451–55473, 2025.arxiv.org/abs/2501.19392
- [36]Staniszewski, K. and Łańcucki, A. KV cache transform coding for compact storage in LLM inference. In International Conference on Learning Representations (ICLR), 2026.arxiv.org/abs/2511.01815
- [37]Sun, D., Tian, H., Lu, T., and Liu, S. FlexCache: Flexible approximate cache system for video diffusion. arXiv preprint arXiv:2501.04012, 2025.arxiv.org/abs/2501.04012
- [38]Tavakkol, A., Sadrosadati, M., Ghose, S., Kim, J. S., Luo, Y., Wang, Y., Ghiasi, N. M., Orosa, L., Gómez-Luna, J., and Mutlu, O. FLIN: Enabling fairness and enhancing performance in modern NVMe solid state drives. In Proceedings of the 45th Annual International Symposium on Computer Architecture (ISCA), pp. 397–410, 2018. doi: 10.1109/ISCA.2018.00041.DOI
- [39]Tillet, P., Kung, H. T., and Cox, D. Triton: An intermediate language and compiler for tiled neural network computations. In Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages (MAPL), pp. 10–19, 2019. doi: 10.1145/3335508.3329973.DOI
- [40]Tomar, A., Hooper, C., Lee, M., Xi, H., Tiwari, R., Kang, W., Manolache, L., Mahoney, M. W., Keutzer, K., and Gholami, A. XQuant: Breaking the memory wall for LLM inference with KV cache rematerialization. arXiv preprint arXiv:2508.10395, 2025.arxiv.org/abs/2508.10395
- [41]von Platen, P., Patil, S., Lozhkov, A., Cuenca, P., Lambert, N., Rasul, K., Davaadorj, M., Nair, D., Paul, S., Berman, W., Xu, Y., Liu, S., and Wolf, T. Diffusers: State-of-the-art diffusion models, 2022. URL https://github.com/huggingface/diffusers.
- [42]Wang, Z., Bovik, A. C., Sheikh, H. R., and Simoncelli, E. P. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004. doi: 10.1109/TIP.2003.819861.DOI
- [47]Zandieh, A., Daliri, M., Hadian, M., and Mirrokni, V. TurboQuant: Online vector quantization with near-optimal distortion rate. In International Conference on Learning Representations (ICLR), 2026.arxiv.org/abs/2504.19874
- [48]Zhang, K., Mo, L., Chen, W., Sun, H., and Su, Y. MagicBrush: A manually annotated dataset for instruction-guided image editing. In Advances in Neural Information Processing Systems (NeurIPS), 2023.arxiv.org/abs/2306.10012
- [49]Zhong, Y., Liu, S., Chen, J., Hu, J., Zhu, Y., Liu, X., Jin, X., and Zhang, H. DistServe: Disaggregating prefill and decoding for goodput-optimized large language model serving. In Proceedings of the 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI), pp. 193–210, 2024.arxiv.org/abs/2401.09670
