WorksComputer Science
FADE: Fractional Anomalous Dynamics Extrapolation for Training-Free Diffusion Transformers Acceleration
Abstract
Diffusion Transformers (DiTs) have established themselves as the preeminent paradigm for high-fidelity generative modeling across various modalities, yet their practical deployment is hindered by the severe computational overhead of iterative sampling. While recent feature-caching methods mitigate this latency through feature reuse or forecasting based on integer-order operators, they remain limited in adapting to the non-stationary and highly curved dynamics of DiT latent trajectories. In this work, we reveal that DiT feature evolution exhibits anomalous diffusion characteristics and a certain degree of spatiotemporal heterogeneity, rendering static, integer-order forecasting operators inadequate and prone to error accumulation. To address this, we propose Fractional Anomalous Dynamics Extrapolation (FADE), a unified framework for training-free diffusion transformers acceleration. FADE models DiT feature evolution via anomalous diffusion and employs a Mittag–Leffler fractional operator to better adapt to high-curvature and non-linear latent trajectories of DiTs. Leveraging the empirical observation that these complex dynamics maintain cross-sample stability, we further construct a Look-Up Table with multiple dimensions through offline profiling. This enables negligible LUT-retrieval overhead, spatiotemporally adaptive parameter retrieval during online inference. Extensive evaluations across diverse architectures and modalities demonstrate that FADE achieves state-of-the-art acceleration, preserving exceptional generation fidelity under aggressive skip intervals with negligible latency overhead. Our project page is available at https://youngkinlon.github.io/fade-project-page/.
Introduction
Diffusion Models Ho et al. [ [10] have become a dominant paradigm for generative modeling across images Chen et al. [ [4], Cai et al. [ [3], Labs [ [17], videos Kong et al. [ [15], Wan et al. [ [41], and audio Kong et al. [ [16], Liu et al. [ [21], owing to their strong high-fidelity synthesis capability. Recent advances in Diffusion Transformers (DiTs) Peebles and Xie [ [33] have further improved this capability, but also introduce substantial computational overhead. Their iterative sampling process requires sequential forward passes through large-scale models, resulting in high latency that hinders practical deployment and motivates training-free acceleration without modifying model weights.
To alleviate this bottleneck, feature caching has emerged as a training-free acceleration strategy. Traditional caching methods Ma et al. [ [30], Selvaraju et al. [ [37] reuse features from previous timesteps by exploiting temporal consistency in intermediate representations. However, this cache-and-reuse paradigm degrades generation quality under large skip intervals, as feature similarity diminishes over longer temporal distances Selvaraju et al. [ [37], Ma et al. [ [30]. Methods such as TaylorSeer Liu et al. [ [22] and HiCache Feng et al. [ [7] formulate feature forecasting based on integer-order forecasting operators. Yet such formulations implicitly assume locally smooth dynamics with a fixed coefficient law, providing only limited approximations to DiT latent trajectories with complex geometry. As a result, they struggle to capture the intrinsic curvature, stage-dependent variations, and abrupt trajectory changes observed across denoising stages.
This limitation reflects a deeper geometric mismatch between existing forecasting schemes and DiT latent dynamics. Prior studies have shown that flat Euclidean assumptions are insufficient to capture the topology of deep generative latent spaces Kalatzis et al. [ [14], while recent analyses further suggest the existence of fractal phase-transition structures in diffusion latents Lobashev et al. [ [24]. Since anomalous diffusion commonly arises on fractal structures Oliveira et al. [ [32], this connection motivates us to investigate whether DiT feature evolution exhibits anomalous diffusion behavior. In anomalous diffusion studies, the power-law scaling exponent of the mean squared displacement (MSD) serves as a standard empirical diagnostic for deviations from Brownian diffusion Metzler and Klafter [ [31]. Using this diagnostic, we observe that the normalized MSD of DiT features deviates from Brownian linear scaling, as shown in Figure 1(a), thereby supporting a fractional perspective for modeling such dynamics. To further characterize the structure of feature evolution, we examine its variation across denoising stages, network layers, and input samples from an anomalous-diffusion perspective. As shown in Figure 1(b), feature dynamics vary substantially across layers and timesteps, while exhibiting relatively smaller variation across input samples. This observation suggests structured stage- and layer-dependent dynamics with a degree of cross-sample consistency, providing empirical support for calibrating forecasting parameters offline by denoising stage and network location and reusing them across samples.

Figure 1. Empirical evidence of anomalous and structured dynamics in DiT. (a) The normalized mean squared displacement across layers follows an anomalous diffusion scaling law. (b) Cross-sample trajectory consistency across different layers and denoising steps.
Building on these observations, our key insight is to treat DiT feature evolution as an anomalous diffusion process over complex latent geometry, while exploiting its empirical cross-sample regularity for offline calibration. This motivates a forecasting framework that combines a fractional dynamical model with offline-calibrated stage-aware adaptation. Accordingly, we propose FADE (Fractional Anomalous Dynamics Extrapolation), a training-free framework that models DiT feature evolution via anomalous diffusion for diffusion transformers acceleration. A Mittag–Leffler fractional operator is introduced to generalize the classical integer-order shift operator, enabling flexible prediction of non-Brownian feature evolution. To operationalize this adaptive formulation, FADE profiles the optimal fractional parameters offline and stores them in a multi-dimensional LUT, which allows stage-aware feature forecasting through retrieval during inference. This design improves extrapolation robustness under large skip intervals while preserving generation quality with negligible overhead.
In summary, the principal contributions of our work are as follows:
Fractional Anomalous Dynamics Extrapolation (FADE): We propose a training-free forecasting framework that models DiT feature evolution as an anomalous diffusion process. By generalizing the classical exponential shift operator with a Mittag–Leffler fractional operator, FADE better adapts to the complex, high-curvature dynamics of latent trajectories.
Zero-Overhead Adaptive Forecasting: To translate our fractional framework into practical acceleration, we construct a Look-Up Table indexed by multiple dimensions through offline profiling. This design enables spatiotemporally adaptive parameter retrieval in constant time, ensuring robust feature forecasting without introducing any additional inference latency.
Empirical Characterization of DiT Feature Dynamics: We reveal that DiT feature evolution exhibits anomalous diffusion behavior and structured spatiotemporal heterogeneity across denoising stages and network locations. Meanwhile, these dynamics remain relatively stable across input samples, providing empirical support for offline calibration.
State-of-the-Art Acceleration: Extensive evaluations across diverse architectures and modalities, including DiT-XL/2, FLUX.1, and HunyuanVideo, demonstrate that FADE consistently outperforms existing baselines, maintaining exceptional generation fidelity and negligible latency overhead even under long skip intervals.
Related Work
Computation Reduction Strategies. Previous efforts to accelerate diffusion models focus on either reducing the number of sampling steps or compressing the denoising network. Step-reduction methods accelerate the generative process through efficient ODE solvers like DDIM Song et al. [ [39] and DPM-Solver Lu et al. [ [26], Zhao et al. [2023], Lu et al. [ [27], knowledge distillation Salimans and Ho [ [36], Luo et al. [ [28], or few-step paradigms such as Rectified Flow Liu et al. [ [23] and Consistency Models Song et al. [ [40]. However, these approaches often degrade generation quality or require expensive retraining Ma et al. [ [29]. Alternatively, network compression techniques such as pruning Fang et al. [ [6], Yuan et al. [2024], quantization Li et al. [ [19], Shang et al. [ [38], Yang et al. [ [44], and token merging Zhang et al. [2024b], Bolya and Hoffman [ [2] reduce the FLOPs of each denoising step. However, their efficiency gains often come at the cost of reduced model expressivity, which may compromise generation fidelity under aggressive compression. In contrast, feature caching reduces redundant intermediate feature computation to accelerate inference.
Feature Caching. As a prominent training-free alternative, feature caching skips redundant intermediate computations. Existing methods generally fall into two paradigms: cache-then-reuse and cache-then-forecast. Reuse-based approaches reuse cached representations, including direct reuse methods such as DeepCache Ma et al. [ [30], PAB Zhao et al. [2024], and FORA Selvaraju et al. [ [37], token-level methods such as ToCa Zou et al. [2025], TokenCache Lou et al. [ [25], and DaTo Zhang et al. [2024a], and error-aware or adaptive caching variants such as TeaCache Liu et al. [ [20], AdaCache Kahatapitiya et al. [ [13], and ERTACache Peng et al. [ [34]. MEDA Zhou et al. [2026] uses multiobjective evolutionary search to optimize feature-reuse schedules, balancing computational cost and generation quality. However, larger skip intervals inevitably enlarge the temporal representation mismatch, degrading generation quality. However, larger skip intervals inevitably enlarge the temporal representation mismatch, degrading generation quality. Forecasting-based approaches, pioneered by TaylorSeer Liu et al. [ [22], actively predict feature evolution. Recent advancements seek to refine this paradigm: HiCache Feng et al. [ [7] improves the standard Taylor expansion by introducing scaled Hermite coefficients, while FOCA Zheng et al. [2026] drives the forecasting process by utilizing both the features themselves and their first-order finite-difference derivatives. BRACE Yang et al. [ [43] further explores barycentric rational forecasting with adapted Chebyshev weights, directly aggregating cached historical features to accommodate local trajectory irregularities. Despite these algorithmic refinements, these methods treat the non-equilibrium denoising trajectory as a globally uniform dynamical system, limiting their capacity to characterize the anomalous diffusion properties and complex temporal phase transitions of latent features. This ultimately results in trajectory deviation under large skip intervals.
Method
Preliminaries
Diffusion models generate data by iteratively reversing a predefined forward noising process via a neural network that estimates the denoising direction at each timestep. Starting from Gaussian noise, the latent variable is progressively refined over timesteps toward a clean sample:
Since diffusion sampling requires sequential model evaluations, inference is costly, especially for large-scale DiTs Peebles and Xie [ [33]. To reduce full forward passes at selected timesteps, Predictive feature caching reduces this cost by forecasting intermediate features at layer . With a recursive discrete difference operator , the -th order Taylor predictor over distance is:
Based on operator calculus, the above polynomial extrapolation over a prediction distance can be interpreted as applying the exponential shift operator to the current feature , where and denotes the temporal derivative operator:
From this perspective, conventional integer-order extrapolation methods typically rely on finite-order local approximations of the shift operator, which may struggle to capture the complex and non-stationary dynamics of DiT’s latent trajectories.
Overall Framework of FADE
Motivated by the anomalous diffusion of DiT latent feature evolution, we propose Fractional Anomalous Dynamics Extrapolation (FADE). As shown in Figure 2, FADE consists of an offline FADE Profiling Framework and an online FADE Inference Architecture. In the profiling stage, calibration samples are used for Optimal Order Search with Directional Alignment, where the optimal fractional order is obtained by minimizing the directional loss between the predicted feature and the ground-truth feature . The calibrated orders are then stored in a static look-up table (LUT), where indexes the network location(e.g., layer, module, stream, etc). In the inference stage, denoising follows a cache interval and alternates between Full Compute Steps and Prediction Steps: Full Compute Steps perform forward passes and cache the finite-difference set , while Prediction Steps retrieve from the LUT and apply the Fractional Operator to generate the predicted feature . The geometric view in Figure 2 further illustrates that FADE better adapts to the complex latent manifold of DiTs.
Figure 2. Overall framework of FADE. FADE consists of an offline profiling stage and an online inference stage. The offline stage calibrates and stores the optimal fractional orders in a static LUT, while the online stage alternates between full-compute and prediction steps using LUT-retrieved orders for feature forecasting. The upper-right panel illustrates the geometric contrast between the FADE trajectory and the integer-order trajectory in the latent space.
Modeling DiT Feature Evolution via Anomalous Diffusion
Building upon the observed latent dynamics and prior studies on DiT latent geometry, we model DiT feature evolution from an anomalous-diffusion perspective. Anomalous transport on fractal structures provides a geometric motivation for this view Oliveira et al. [ [32], while our observed deviations from Brownian MSD scaling provide empirical support. To formulate a forecasting rule for such dynamics, we draw on classical fractional dynamics, where Mittag–Leffler functions commonly arise in propagation and relaxation solutions Metzler and Klafter [ [31]. Since the Mittag–Leffler function reduces to the exponential function at unit order, we adopt it as a fractional operator prior to generalize the classical exponential shift operator. This provides a flexible coefficient law for controlling how higher-order local variations contribute to feature prediction:
Here, denotes the Gamma function used to parameterize the fractional coefficients. Following this fractional coefficient law, we construct a finite-order feature predictor as
Under this formulation, the fractional order controls the coefficient law used for feature extrapolation. Since DiT denoising is highly non-stationary, feature dynamics vary across timesteps and latent states, often shifting between sub- and super-diffusive-like behaviors. This makes a fixed integer-order rule insufficient for modeling local evolution. FADE therefore assigns state-specific fractional orders to adapt the extrapolation law to heterogeneous anomalous dynamics. To obtain these orders without runtime search, we introduce a Stage-Aware Calibration Strategy.
Stage-Aware Calibration Strategy
As established in our empirical analysis, existing static caching frameworks overlook a key structural property of Diffusion Transformers: feature dynamics vary substantially across latent states and timesteps, yet exhibit a certain degree of cross-sample regularity. Leveraging this cross-sample consistency, we propose a Stage-Aware Calibration Strategy that offline pre-computes the optimal fractional order for each state and timestep , thereby avoiding online search overhead with a short one-time calibration stage and negligible LUT storage.
Directional Monitoring Objective. Calibration is formulated as a local trajectory-matching problem. We use cosine distance to emphasize directional agreement between predicted and reference features. A purely directional objective is therefore adopted, and is defined as the cosine distance between the predicted feature and the ground-truth feature :
Optimal Order Search and Global Refinement. Using a small calibration dataset , the optimal fractional order is obtained by minimizing the expected directional error over a discrete search grid , which spans both sub- and super-diffusion-like regimes:
The offline-calibrated fractional orders may still exhibit a global scale bias in practical inference. To account for this potential bias, we introduce a global refinement factor and use , providing a lightweight global adjustment to the calibrated orders. The refined orders are then stored in the LUT described in § 3.2 for inference-time retrieval. Detailed calibration settings are provided in Appendix § 10.
Error Analysis from a Fractional Perspective
We provide a local and conditional error analysis under a fractional model of local latent evolution. Assume that the local ground-truth evolution admits the following representation:
where denotes the intrinsic propagation scale along the local feature trajectory, which reflects the underlying distance traveled by features on the latent manifold, while denotes the prediction distance used in practice. This distinction captures a potential propagation-scale mismatch. The profiled order is used as an estimate of the intrinsic order , and FADE applies the globally refined order at inference time. Substituting Eq. (5) into Eq. (8), we obtain
The second term combines the intrinsic–extrinsic scale mismatch and the coefficient mismatch. For Taylor extrapolation in Eq. (2), the corresponding mismatch is
Since Taylor extrapolation corresponds to the case of the fractional coefficient family, it can be viewed as a special case of FADE. In contrast, FADE calibrates a stage-adaptive order to accommodate the heterogeneous dynamics across DiT timesteps and network states, and applies a refined order at inference time. The global factor can be viewed as a lightweight correction that helps accommodate residual propagation-scale mismatch. Together, these designs provide a more flexible local approximation of the intrinsic dynamics, consistent with the improved robustness observed under large prediction distances.
Experiments
Experimental Setup
FADE is evaluated across three representative generative tasks: (i) class-conditional image generation on ImageNet-1K Deng et al. [ [5] using DiT-XL/2 Peebles and Xie [ [33]; (ii) text-to-image generation on DrawBench Saharia et al. [ [35] using the rectified flow-based FLUX.1-dev Labs et al. [ [18], Labs et al. [ [17], Liu et al. [ [23]; and (iii) text-to-video generation using HunyuanVideo Kong et al. [ [15] on the HunyuanLarge architecture. For DiT-XL/2, we generate images at resolution using the DDIM sampler with 50 denoising steps. For FLUX.1-dev, we use the Flow Match Euler scheduler with 50 sampling steps at resolution. HunyuanVideo follows its official inference pipeline with 50 flow-matching sampling steps and default video generation settings.
We compare FADE with representative training-free caching methods, including FORA Selvaraju et al. [ [37], TaylorSeer Liu et al. [ [22], and HiCache Feng et al. [ [7]. Within each comparison, caching methods use the same pretrained checkpoint, sampler, number of sampling steps, resolution, guidance settings, and evaluation inputs. Qualitative comparisons additionally use matched random seeds and acceleration schedules to facilitate direct visual assessment.
Quantitative performance is assessed using FID-50K Heusel et al. [ [9], sFID, and Inception Score for DiT-XL/2. For FLUX.1-dev, we report ImageReward Xu et al. [ [42], CLIP Score Hessel et al. [ [8], and CycleReward Bahng et al. [ [1], together with PSNR, SSIM Horé and Ziou [ [11], and LPIPS Zhang et al. [2018] to assess consistency with the full-compute baseline. HunyuanVideo is evaluated using the 16-dimensional VBench suite Huang et al. [ [12]. Computational efficiency is assessed using FLOPs and measured generation latency. FLOPs reduction and wall-clock speedup are computed separately as the ratios of baseline FLOPs and latency to those of the accelerated method, respectively. Offline calibration cost is reported separately from inference latency.
Empirical Validation of Fractional Dynamics
To validate the fractional prior underlying FADE, we profile the optimal fractional orders across representative architectures and modalities. If DiT feature evolution were well described by conventional integer-order dynamics, the optimal states would concentrate around , which is not observed in practice. On DiT-XL/2, Figure 3 shows that the global distribution of is clearly shifted away from , indicating that standard integer-order extrapolation captures only a limited portion of the effective generative states. The spatiotemporal heatmap further reveals a U-shaped temporal pattern: the middle denoising phase is more sub-diffusion-like, whereas the early and late phases are more super-diffusion-like in a phenomenological sense.

Figure 3. Empirical validation of fractional dynamics on DiT-XL/2. The optimal fractional orders deviate markedly from the static integer-order assumption and exhibit a pronounced non-stationary temporal pattern across denoising timesteps.
This heterogeneity also extends across architectures. FLUX.1 shows an inverted temporal trend and clear DoubleStream–SingleStream decoupling, while HunyuanVideo exhibits a broader super-diffusion-like tendency. Together, these results suggest that optimal feature dynamics are non-integer, architecture-dependent, and stage-dependent, motivating the stage-aware calibration design of FADE. More details are provided in Appendix D. Meanwhile, the cross-sample variance of is consistently small on DiT-XL/2 and remains within an acceptable range on FLUX.1 and HunyuanVideo.
Class-Conditional Image Generation
As demonstrated in Table 1, FADE consistently dominates both generation quality and efficiency metrics across all acceleration ratios on DiT-XL/2. When operating at a caching interval of , FADE achieves an outstanding FID of 2.41 and an sFID of 4.79. This not only crushes the 15-step DDIM (FID 5.09) with fewer FLOPs, but also outperforms TaylorSeer’s Liu et al. [ [22] FID of 2.59 and HiCache’s Feng et al. [ [7] 2.51 under strictly identical computational costs. This superiority becomes even more pronounced under aggressive acceleration regimes. When scaling to , FADE maintains a highly competitive FID of 2.87, completely mitigating the severe structural collapse observed in FORA Selvaraju et al. [ [37] and standard Taylor caching Liu et al. [ [22], which suffer from severely degraded FIDs of 9.22 and 3.17 respectively.
| Method | Lat. (s) | FLOPs (T) | Spd. | FID | sFID | IS |
| DDIM-50 steps Song et al. [ [39] | 1.18 | 23.74 | 1.00 | 2.29 | 4.29 | 237.31 |
| DDIM-25 steps Song et al. [ [39] | 0.62 | 11.87 | 2.00 | 2.91 | 4.57 | 232.22 |
| DDIM-15 steps Song et al. [ [39] | 0.42 | 7.12 | 3.33 | 5.09 | 6.11 | 205.54 |
| FORA() Selvaraju et al. [ [37] | 0.45 | 6.66 | 3.56 | 4.73 | 8.45 | 214.20 |
| Taylor (, ) Liu et al. [ [22] | 0.61 | 6.66 | 3.56 | 2.59 | 5.18 | 232.90 |
| HiCache (, ) Feng et al. [ [7] | 0.57 | 6.66 | 3.56 | 2.51 | 5.20 | 232.72 |
| FADE(, ) | 0.55 | 6.66 | 3.56 | 2.41 | 4.79 | 236.76 |
| DDIM-12 steps Song et al. [ [39] | 0.33 | 5.70 | 4.17 | 7.94 | 8.07 | 183.88 |
| FORA() Selvaraju et al. [ [37] | 0.37 | 5.24 | 4.53 | 5.81 | 9.61 | 199.96 |
| Taylor (, ) Liu et al. [ [22] | 0.55 | 5.24 | 4.53 | 2.73 | 5.34 | 228.75 |
| HiCache (, ) Feng et al. [ [7] | 0.50 | 5.24 | 4.53 | 2.67 | 5.48 | 229.44 |
| FADE(, ) | 0.49 | 5.24 | 4.53 | 2.55 | 5.15 | 230.89 |
| DDIM-10 steps Song et al. [ [39] | 0.30 | 4.75 | 5.00 | 12.17 | 11.24 | 159.20 |
| FORA() Selvaraju et al. [ [37] | 0.34 | 4.76 | 4.98 | 9.22 | 14.86 | 169.02 |
| Taylor (, ) Liu et al. [ [22] | 0.50 | 4.76 | 4.98 | 3.17 | 6.36 | 222.00 |
| HiCache (, ) Feng et al. [ [7] | 0.46 | 4.76 | 4.98 | 3.09 | 6.30 | 220.55 |
| FADE(, ) | 0.45 | 4.76 | 4.98 | 2.87 | 5.73 | 225.78 |
Table 1. Quantitative comparison of class-conditional image generation with DiT-XL/2.
Text-to-Image Generation
Quantitative Analysis Table 2 reports the quantitative results on FLUX. FADE achieves a strong quality–efficiency trade-off across acceleration settings, consistently outperforming existing baselines on semantic alignment and structural fidelity at standard configurations (). Under a more aggressive skip interval, FADE still maintains competitive generation quality with negligible latency overhead, demonstrating the robustness of fractional forecasting.
| Method | Latency(s) | FLOPs(T) | Speed | Img. Reward | CLIP Score | Cycle Reward | PSNR | SSIM | LPIPS |
| FLUX.1 [dev] - 50 steps | 17.18 | 3719.50 | 1.00 | 0.9898 | 27.4611 | 0.8800 | 1.0000 | 0.0000 | |
| FLUX.1 [dev] - 25 steps | 8.80 | 1859.75 | 2.00 | 0.9375 | 27.3388 | 0.8466 | 19.2599 | 0.7749 | 0.3583 |
| FLUX.1 [dev] - 20 steps | 7.12 | 1487.80 | 2.62 | 0.9387 | 27.2505 | 0.8466 | 18.1371 | 0.7484 | 0.4006 |
| FORA () Selvaraju et al. [ [37] | 5.01 | 893.54 | 4.16 | 0.8253 | 27.2377 | 0.8353 | 15.9899 | 0.6677 | 0.5206 |
| TaylorSeer (, ) Liu et al. [ [22] | 5.14 | 893.54 | 4.16 | 0.9919 | 27.5143 | 0.8540 | 18.4497 | 0.7424 | 0.4146 |
| HiCache (, ) Feng et al. [ [7] | 5.12 | 893.54 | 4.16 | 0.9350 | 27.4034 | 0.8346 | 19.2595 | 0.7503 | 0.3911 |
| FADE(, ) | 5.04 | 893.54 | 4.16 | 1.0071 | 27.5724 | 0.8600 | 19.6683 | 0.7593 | 0.3815 |
| FORA () Selvaraju et al. [ [37] | 4.40 | 744.81 | 4.99 | 0.7836 | 27.1541 | 0.8087 | 15.5290 | 0.6663 | 0.5253 |
| TaylorSeer (, ) Liu et al. [ [22] | 4.57 | 744.81 | 4.99 | 0.9850 | 27.5199 | 0.8480 | 17.5396 | 0.7078 | 0.4657 |
| HiCache (, ) Feng et al. [ [7] | 4.55 | 744.81 | 4.99 | 0.9427 | 27.6898 | 0.8539 | 18.5159 | 0.7155 | 0.4503 |
| HiCache (, ) Feng et al. [ [7] | 5.03 | 744.81 | 4.99 | 0.9892 | 27.6586 | 0.8428 | 18.5044 | 0.7268 | 0.4316 |
| FADE(, ) | 4.43 | 744.81 | 4.99 | 1.0019 | 27.7060 | 0.8706 | 18.7114 | 0.7320 | 0.4207 |
| FORA () Selvaraju et al. [ [37] | 4.10 | 670.44 | 5.55 | 0.7289 | 27.0135 | 0.7962 | 15.5290 | 0.6548 | 0.5437 |
| TaylorSeer (, ) Liu et al. [ [22] | 4.31 | 670.44 | 5.55 | 0.9420 | 27.4039 | 0.7970 | 16.8563 | 0.6844 | 0.5062 |
| HiCache (, ) Feng et al. [ [7] | 4.20 | 670.44 | 5.55 | 0.9191 | 27.6949 | 0.8494 | 18.2778 | 0.7032 | 0.4708 |
| HiCache (, ) Feng et al. [ [7] | 4.72 | 670.44 | 5.55 | 0.9881 | 27.5651 | 0.8384 | 17.9911 | 0.7098 | 0.4592 |
| FADE (, ) | 4.12 | 670.44 | 5.55 | 0.9861 | 27.8089 | 0.8621 | 18.1925 | 0.7137 | 0.4491 |
| FADE (, ) | 4.25 | 670.44 | 5.55 | 0.9914 | 27.7372 | 0.8592 | 17.7636 | 0.7106 | 0.4561 |
Table 2. Quantitative comparison of text-to-image generation on FLUX.
Qualitative comparison. As illustrated in Figure 4, FADE maintains close visual consistency with the original FLUX.1 baseline in text rendering, texture, and scene composition. It preserves the “Text to Image” storefront and the “Diffusion” sign, whereas FORA Selvaraju et al. [ [37] alters the sign geometry and produces unclear lettering. For the brick cube, FADE retains the reddish material and mortar details, while TaylorSeer Liu et al. [ [22] produces a pale stone-like appearance, FORA generates smoother surfaces, and HiCache Feng et al. [ [7] changes the brick arrangement. FADE also preserves the cat press-conference composition and animal yearbook scene, while TaylorSeer introduces sunglasses and more human-like bodies in the latter. In the magnifying-glass example, FADE retains the train imagery within the lens, whereas FORA replaces it with text-like patterns.
Figure 4. Visualization examples for different acceleration methods on Flux.
Text-to-Video Generation
Quantitative Analysis. Table 3 reports the quantitative results on HunyuanVideo. Under the same FLOPs budget and acceleration ratio, FADE achieves competitive latency and the best VBench score, indicating that fractional forecasting remains effective for text-to-video generation.
| Method | Lat. (s) | FLOPs (T) | Spd. | VBench (%) |
| Original (50 steps) | 190.2 | 29773 | 1.00× | 80.68 |
| DDIM (22% steps) Song et al. [ [39] | 42.7 | 6550 | 4.55× | 78.73 |
| FORA () Selvaraju et al. [ [37] | 37.1 | 5359 | 5.56× | 78.86 |
| TaylorSeer (, ) Liu et al. [ [22] | 37.8 | 5359 | 5.56× | 79.87 |
| HiCache (, ) Feng et al. [ [7] | 37.8 | 5359 | 5.56× | 79.93 |
| FADE (, ) | 37.9 | 5359 | 5.56× | 80.12 |
Table 3. Quantitative comparison of text-to-video generation on HunyuanVideo.
Qualitative Comparison. As shown in Figure 5, FADE preserves clearer text and structure in the static case, correctly rendering the word “STOP”, while better maintaining motion semantics and scene context in dynamic prompts. In particular, FADE better captures the intended temporal behavior and preserves prompt-specific motion cues, rather than merely maintaining frame-level appearance. In contrast, the baselines either miss the intended intense shaking effect in the astronaut case or introduce incorrect motion directions and off-road backgrounds in the motorcycle case.
Figure 5. Qualitative comparison on text-to-video generation.
Ablation Study
On DiT-XL/2, we compare several strategies for determining the fractional order, including fixed global tuning, polynomial fitting over denoising progress, and the proposed stage-aware LUT retrieval. We also compare MSE-based and directional calibration objectives. As shown in Table 4, the full FADE design achieves the best performance, suggesting that the optimal fractional dynamics cannot be well captured by either a static setting or a simple smooth fit, and that directional calibration provides a more effective signal for stable offline calibration.
| Method | Speed↑ | FID↓ | sFID↓ |
| DDIM-50 | 1.00× | 2.29 | 4.29 |
| DDIM-10 | 5.00× | 12.17 | 11.24 |
| HiCache | 4.98× | 3.09 | 6.30 |
| FADE () | 4.98× | 3.17 | 6.36 |
| FADE (fixed ) | 4.98× | 3.06 | 6.05 |
| FADE (poly-fit ) | 4.98× | 3.00 | 6.19 |
| FADE (LUT + MSE) | 4.98× | 2.97 | 5.91 |
| FADE (LUT + Dir.) | 4.98× | 2.87 | 5.73 |
Table 4. Ablation on DiT-XL/2.
On FLUX.1, we study two practical aspects of offline calibration: the calibration set size and the global refinement factor . Table 5 shows that FADE remains effective with a small calibration set, saturating around . A moderate refinement factor gives the best overall performance, with achieving the best ImageReward and the best CLIPScore. This indicates that is a useful lightweight correction, though its optimal value depends on the prediction interval and sampling setting. Additional ablations are provided in Appendix § 13.
| Method | Speed↑ | IR↑ | CLIP↑ |
| FADE () | 4.99× | 0.9963 | 27.6781 |
| FADE () | 4.99× | 1.0015 | 27.7454 |
| FADE () | 4.99× | 1.0019 | 27.7060 |
| FADE () | 4.99× | 1.0008 | 27.7256 |
| FADE () | 4.16× | 0.9974 | 27.4618 |
| FADE () | 4.16× | 0.9994 | 27.6198 |
| FADE () | 4.16× | 1.0071 | 27.5724 |
| FADE () | 4.16× | 0.9858 | 27.5050 |
Table 5. Ablation on FLUX.1.
We further report the practical cost of offline calibration and LUT retrieval. Using 10 calibration samples, end-to-end offline calibration takes approximately 50 s, 550 s, and 2,000 s for DiT-XL/2, FLUX.1, and HunyuanVideo, respectively, on their respective evaluation hardware. The LUT has a small storage footprint, occupying only 44.8 kB for DiT-XL/2. The accumulated LUT-retrieval time per generation is 0.14 ms, 0.57 ms, and 0.61 ms, respectively.
Conclusion and Future Work
This work revisits training-free DiT acceleration from the perspective of fractional dynamics. Motivated by the limitations of cache-based methods under complex stage-dependent latent dynamics, we view DiT feature evolution through the lens of anomalous diffusion and formulate FADE using a Mittag–Leffler-based fractional operator. With an offline-profiled LUT, FADE enables adaptive forecasting at inference time with retrieval and no extra runtime overhead. Experiments across diverse architectures and modalities show that FADE achieves a stronger quality–efficiency trade-off than existing baselines while remaining robust under large prediction distances.
FADE relies on manually designed calibration, and its Mittag–Leffler operator is chosen primarily for simplicity. Future work will explore alternative fractional operators and order parameterizations to improve forecasting accuracy, stability, and the quality–efficiency trade-off across denoising stages, network locations, and skip intervals.
Acknowledgments and Disclosure of Funding
This work was supported by National Natural Science Foundation of China under Grant No.62376172.
References
Technical appendices
Detailed Derivation of the FADE Predictive Formulation
In this section, we provide the rigorous step-by-step mathematical derivation detailing the transition from the classical integer-order extrapolation to our generalized Fractional Anomalous Dynamics Extrapolation (FADE) formulation.
From Polynomial Extrapolation to the Exponential Shift Operator
Based on operator calculus, the polynomial extrapolation in Eq. (2) can be interpreted as applying an exponential shift operator to the current feature representation. Let denote the feature at timestep , and let denote the prediction distance from to . We define the temporal derivative operator as
For a sufficiently smooth continuous trajectory, the Taylor expansion of around can be written as
Equivalently, by collecting the derivative operator terms, we obtain
Since the infinite series corresponds to the Maclaurin expansion of the exponential function, this transformation can be compactly expressed as
By defining
the above relation becomes
where the approximation emphasizes the practical setting in which the continuous operator is implemented through finite-order truncation and discrete feature differences.
In practical predictive caching, the infinite-dimensional operator is truncated to the -th order:
By approximating the continuous derivatives with discrete feature differences, i.e.,
we recover the polynomial forecasting form:
Therefore, the commonly used -th order polynomial extrapolation can be viewed as a truncated and discretized approximation of the exponential shift operator .
Detailed Derivation of the FADE Predictive Formulation
In this section, we provide a step-by-step derivation of the FADE predictive formulation, showing how the classical integer-order extrapolation can be generalized through a fractional Mittag–Leffler operator.
Operator Equivalence. We first revisit the continuous integer-order baseline. Let denote the latent feature at timestep , and let denote the prediction distance from to . For a sufficiently smooth feature trajectory, the Taylor expansion of around is given by
By defining the temporal derivative operator as
the above expansion can be written in operator form as
Equivalently, by defining
the integer-order extrapolation can be interpreted as applying the exponential shift operator:
where the approximation emphasizes the practical setting in which the continuous operator is implemented through finite-order truncation and discrete feature differences.
Continuous Fractional Generalization. To model the anomalous and stage-dependent evolution of DiT features, FADE replaces the classical exponential operator with a Mittag–Leffler (ML) operator. The ML function is defined as
where controls the fractional propagation order. Substituting the exponential shift variable into the ML operator gives
By isolating the zero-order term and using , we obtain
Truncation and Numerical Discretization. For tractable inference, the infinite fractional series is truncated to a finite order . Meanwhile, since diffusion sampling proceeds over discrete timesteps, the continuous derivative is approximated by the -th order discrete feature difference, denoted by . The backward difference operator is recursively defined as
Substituting the continuous derivatives with discrete feature differences yields the FADE predictive formulation:
Here, denotes the effective prediction distance, instantiated in practice as the skip interval between the latest computed step and the target prediction step.
Degeneration to the Integer-Order Baseline. The classical integer-order extrapolation is recovered as a special case when . Using the Gamma function identity
Eq. (30) becomes
Therefore, the conventional integer-order polynomial forecasting rule can be viewed as the instance of the more general FADE formulation, while provides a flexible coefficient law for modeling anomalous, stage-dependent feature dynamics.
Implementation Details of Stage-Aware Calibration
Model-Specific Calibration and Extreme Data Efficiency
A significant advantage of FADE is its extreme data efficiency during the offline calibration phase. We construct the calibration dataset using only 10 randomly sampled conditions paired with standard Gaussian noise initializations. To ensure the calibration perfectly aligns with the native inference pipelines of different generation paradigms, the conditioning inputs are specifically configured as follows:
Class-Conditional Models (e.g., DiT-XL/2): is constructed by randomly sampling 10 class labels from the ImageNet training label space, coupled with random initial noise seeds. The calibration seeds are kept disjoint from those used for final evaluation.
Text-to-Image Models (e.g., FLUX.1): The calibration conditions are formed by randomly selecting 10 diverse text prompts from the MS-COCO training split, paired with random initial seeds. These calibration prompts and seeds are not reused in the final DrawBench evaluation.
Text-to-Video Models (e.g., HunyuanVideo): For video generation, is constructed using 10 randomly sampled prompts from T2V-CompBench, together with random initial noise seeds. These calibration prompts and seeds are kept disjoint from those used in the final VBench evaluation.
Despite the limited sample size () and the inherent stochasticity of diverse semantic conditions, the derived Look-Up Table (LUT) for captures a stable profile of latent dynamics that generalizes effectively to unseen prompts during online inference.
Stage-Aware Calibration Procedure
Calibration follows the online compute–predict schedule using a closed-loop procedure, as summarized in Algorithm 1. We approximate the calibration objective in Eq. (7) through sample-wise directional grid searches followed by aggregation. At each prediction location , candidate orders in are evaluated using Eqs. (5) and (6). The prediction associated with the minimum directional loss is injected into the residual stream:
where is the current hidden residual state and is the corresponding model gate. Subsequent calibration therefore accounts for recursively propagated prediction errors.
For each timestep–location pair , the selected orders are collected across calibration samples and aggregated to obtain . The refined order is then stored in the LUT. The feature cache contains the zeroth-order feature and its finite differences, where denotes the latest full-compute timestep.
Dissecting Trajectory Dynamics: Implicit Regularization vs. Dynamic Rectification
To rigorously validate the efficacy of the optimal fractional orders found during the aforementioned offline calibration, we shift our focus to the long-horizon error propagation during online inference. By injecting the extrapolated features back into the recursive forward pass, we compare our FADE framework against the Ground-Truth full computation (GT-FI) and Standard Taylor extrapolation (ST-FI).
The Illusion of Implicit Regularization in Taylor Expansion
A critical observation in our closed-loop experiments is the highly non-linear nature of the diffusion trajectory. Due to the inherent architectural complexity of Diffusion Transformers (e.g., the global receptive field of self-attention mechanisms), the true latent manifold (GT-FI) exhibits dense, high-frequency structural transitions and abrupt numerical spikes across layers.
However, when employing the standard Taylor expansion (ST-FI) for caching, we observe a phenomenon we term Implicit Regularization. The strict integer-order assumption () acts as a rigid mathematical constraint, aggressively smoothing out these necessary trajectory spikes. It forces the feature evolution into a sub-optimal, overly-regularized path. While this may locally suppress extreme outliers, it fundamentally destroys the high-frequency semantics required for complex generation, inevitably leading to catastrophic directional divergence in later stages.
FADE: Dual-Stage Rectification under Recursive Error
In contrast, FADE avoids this rigid regularization by treating the fractional order as a dynamic, stage-aware degree of freedom. By explicitly simulating the recursive error accumulation during the offline calibration, FADE introduces a Dual-Stage Rectification mechanism to faithfully reconstruct the GT trajectory:
Directional Rectification via Fractional Order: Instead of suppressing the structural spikes, the optimized Mittag–Leffler parameters () flexibly adapt to the local curvature
Algorithm 1 Stage-Aware Closed-Loop Offline Calibration for FADE
Input: calibration dataset , candidate grid , compute-step schedule including warm-up, maximum order , refinement factor
Output: refined fractional-order table
Initialize an empty collection for each target pair
for each calibration sample do
Initialize the diffusion latent and conditioning from
for each denoising timestep do
if then
Run the standard full forward pass to obtain model output
Update feature histories and caches for all target locations
else
Compute the prediction distance from to
for each target location in forward execution order do
Advance normal forward computations to location
Obtain the reference feature by full computation
Evaluate all using cache , distance , and Eq. (5)
Select minimizing Eq. (6) and retain its prediction
Append to
end for
Complete the remaining forward computations to obtain model output
end if
Advance the diffusion latent using and the native sampler
end for
end for
for each pair with do
end for
return
of the manifold. By strictly monitoring the cosine similarity (Algorithm 1), FADE actively corrects the directional drift that plagues standard caching methods.
Intrinsic–Extrinsic Mismatch Correction: The stage-aware LUT provides state-specific fractional orders, but practical inference still uses a proxy distance instead of the intrinsic propagation scale. To account for this residual propagation mismatch, FADE applies a global refinement factor to obtain , while preserving the state-specific structure encoded in the LUT.
Through this decoupled rectification strategy, FADE is able to faithfully trace the abrupt fluctuations of the ground-truth manifold, completely bypassing the implicit regularization trap of Euclidean-based forecasting methods.
Empirical Validation of High Sample Consistency on DiT
To rigorously justify the use of a minuscule calibration set, we conducted an extensive empirical study focusing on the consistency of the optimal fractional order specifically within the DiT-XL/2 architecture. We computed the optimal across 10 independent random noise initializations on ImageNet-512.
As summarized in Table 6, the optimal fractional order at the stable denoising phase () exhibits a clear layer-dependent pattern. While the early and middle layers naturally display moderate variance to adapt to diverse initial noise inputs, the deep layers (Layers 19–27) maintain strong cross-sample consistency. Most importantly, empirical generation results confirm that the moderate standard deviation observed in the shallow stages has negligible impact on the final perceptual quality. The stable deeper layers effectively absorb these initial fluctuations, ensuring that a single offline calibration remains highly robust across any unseen prompts.
| Network Phase | Layer Indices | Attention Module (Attn) | |||
| Early Layers | Layers 0–9 | Mean | 0.64 | 0.144 | 0.128 |
| Middle Layers | Layers 10–18 | Mean | 0.96 | 0.129 | 0.119 |
| Deep Layers | Layers 19–27 | Mean | 0.46 | 0.128 | 0.109 |
| Global Summary | All Layers | Mean | 0.68 | 0.120 | 0.123 |
Table 6. Cross-sample consistency of the optimal fractional order in DiT attention layers. We highlight the statistics at the most stable denoising phase (Timestep ). The consistent behavior in deep layers and its robustness to early-stage variance formally validate our offline calibration strategy.
Error Propagation and Stability Analysis
Scope of the analysis. We provide a local and conditional error analysis for the regime where the local latent evolution is better described by an intrinsic fractional propagation law. The goal is not to prove a universal advantage of FADE over all possible predictors, but to explain why a state-adaptive fractional coefficient law can reduce the mismatch between the intrinsic feature trajectory and its practical approximation based on an extrinsic prediction scale.
Notation. Let denote the denoising timestep and denote the latent state index. We write
and let denote the -th order backward finite difference computed from cached historical features at state . For compactness, define the fractional coefficient
Here, denotes the extrinsic prediction scale used in practical inference, while denotes the intrinsic propagation scale along the local feature trajectory. The difference between these scales gives rise to a potential intrinsic–extrinsic propagation mismatch.
Assumption 1: intrinsic fractional local evolution. For each timestep–state pair , assume that there exists a local intrinsic fractional order such that the ground-truth local evolution admits the following truncated intrinsic expansion:
where the truncation remainder satisfies
This assumption is local and conditional: it only states that the nearby feature evolution can be approximated by a truncated intrinsic ML-type expansion.
FADE predictor. FADE uses the extrinsic prediction scale and the globally refined fractional order
where is obtained from offline stage-aware calibration. The corresponding predictor is
Theorem 1: residual-aware FADE error bound. Under Assumption 1, the prediction error of FADE satisfies
Proof. Subtracting (38) from (35) gives
Taking norms and applying the triangle inequality yields
Using (36) and the definition of gives (39). □
Decomposing the mismatch term. The second term in (39) contains both intrinsic–extrinsic scale mismatch and coefficient mismatch. Indeed, for each order , we have
This shows explicitly that FADE can reduce the coefficient mismatch by calibrating and refining it with , while the remaining scale discrepancy arises from the difference between the extrinsic prediction scale and the intrinsic propagation scale .
Smoothness of the coefficient law. Assume that the relevant fractional orders lie in a compact interval
Since is smooth on , there exists a finite constant
such that
Therefore, (42) implies
This bound separates the error into an intrinsic–extrinsic scale mismatch and an order-induced coefficient mismatch.
Taylor predictor. The vanilla Taylor predictor corresponds to the special case , since
Its prediction form is
Following the same derivation as above, we obtain
Conditional advantage over Taylor. The truncation term in (39) and (47) is shared. Therefore, FADE has a tighter derived upper bound than Taylor whenever
This condition states that the refined fractional coefficient law must approximate the intrinsic local evolution more closely than the rigid Taylor coefficient law, accounting for the difference between intrinsic and extrinsic propagation scales. It is a local and conditional statement, rather than a universal guarantee over arbitrary latent processes.
Role of the global refinement factor . The offline-calibrated fractional orders may still exhibit a global scale bias in practical inference. To account for this potential bias, we introduce a global refinement factor and use
This provides a lightweight global adjustment to the calibrated orders while preserving the state-specific structure captured by the offline LUT.
Summary of the local error analysis. Under the intrinsic fractional local evolution assumption, the prediction error is bounded by a truncation term and a mismatch term involving propagation scales and fractional coefficients. Taylor extrapolation corresponds to the case of the fractional coefficient family. FADE calibrates stage-adaptive orders and applies the refined orders during inference. When its mismatch term is smaller than that of Taylor, FADE has a tighter derived upper bound. These designs provide a more flexible local approximation of the intrinsic dynamics, consistent with the improved robustness observed under large prediction distances.
Comprehensive Analysis of Latent Trajectory Dynamics
Empirical Analysis of Fractional Dynamics on DiT-XL/2
We analyze the optimal fractional orders across the 28 layers of DiT-XL/2 during a standard 50-step generation process. Statistics are averaged over 50 randomly sampled generation trajectories. Figure 6 shows the global distribution and spatiotemporal variation of the calibrated orders.

Figure 6. (a) Global Density Distribution of (b) Spatiotemporal Heatmap of . Spatiotemporal visualization of the optimal fractional orders () on DiT-XL/2. (a) Global KDE distribution of . (b) Variation across denoising timesteps and transformer layers, showing a broadly shared U-shaped temporal pattern.
Global Distribution: Fractional Regimes Dominate. As shown in Figure 6(a), orders below 0.9 occur more frequently than those above 1.1, while the near-integer regime () accounts for approximately 13% of the analyzed states. Offline calibration therefore frequently selects orders that differ from the unit order used in standard Taylor extrapolation.
Temporal Heterogeneity: A U-Shaped Phase Pattern. Figure 6(b) reveals a U-shaped temporal pattern: lower orders are concentrated in the middle phase, while higher orders appear more frequently in the early and late phases. Spatial Consistency across Network Depth. At a given timestep, the calibrated orders remain relatively aligned across network layers. Together with the temporal variation, this motivates stage-aware selection of the forecasting order.
Empirical Analysis of Fractional Dynamics on FLUX.1
We extend the analysis to FLUX.1, which uses rectified flow matching and contains distinct DoubleStream and SingleStream blocks. For this analysis, we trace over 50 generation trajectories using text prompts randomly sampled from DrawBench. Figure 7 shows the calibrated orders across both block types.

Figure 7. (a) DoubleStream Blocks (b) SingleStream Blocks. Spatiotemporal decoupling of latent dynamics across FLUX.1 architectures. Calibrated orders in (a) DoubleStream and (b) SingleStream blocks. SingleStream blocks exhibit a greater proportion of higher-order states. Across both streams, higher orders are concentrated in the middle stages of .
Architectural Decoupling: A Shift toward Higher-Order Fractional Regimes. In the DoubleStream blocks (Figure 7(a)), the higher-order region () accounts for 56.8% of the states, while orders below 0.9 account for 32.4%. In the SingleStream blocks (Figure 7(b)), the higher-order proportion increases to 87.0%, while the near-integer regime () accounts for only approximately 0.5%.
Temporal Heterogeneity: An Inverted Phase Pattern. Along the timeline , FLUX exhibits a temporal pattern opposite to the U-shaped trend observed in DiT. Relatively lower orders appear in the early and late phases, while higher orders are concentrated in the middle phase, as shown in Figure 7.
These observations motivate adapting the forecasting order across both architectural blocks and temporal stages.
Empirical Analysis of Fractional Dynamics on HunyuanVideo
We extend the analysis to HunyuanVideo by tracing the optimal fractional orders across its transformer blocks using diverse video text prompts. Figure 8 presents the distributions across its DoubleStream and SingleStream blocks.

Figure 8. Spatiotemporal decoupling of latent dynamics across HunyuanVideo architectures. Both (a) DoubleStream and (b) SingleStream blocks show a broad shift toward higher-order regimes (), while near-integer states () are comparatively uncommon.
A Broad Shift toward Super-Diffusion-Like Regimes. Figure 8 shows a broad shift toward higher-order regimes () across both block types. This trend resembles the behavior observed in the SingleStream blocks of FLUX.1 (Section D.2).
Limitations of a Uniform Integer-Order Assumption. The near-integer regime () accounts for only a small fraction of the calibrated states, while orders below 0.9 are also less prominent than in the image DiT analyzed above. These distributions show that calibration frequently selects orders beyond the fixed baseline.
Together with the results on DiT and FLUX, these observations motivate adapting the forecasting order across denoising stages and model architectures.
Additional Ablation Study
Ablation on Fractional-Order Scheduling
We further study different fractional-order scheduling strategies on FLUX.1 under the setting. This ablation is designed to examine whether a manually specified fixed order or a simple time-dependent fitted schedule can reliably replace stage-aware calibration.
As shown in Table 7, different fractional-order schedules favor different evaluation metrics. A fixed order of achieves the best ImageReward, while the dynamic fitted schedule obtains the highest CLIP score. This suggests that time-dependent order variation can better align with text-image semantic consistency, but does not uniformly dominate fixed-order schedules across all metrics. These results indicate that fractional-order scheduling is sensitive to both the metric and denoising regime, further motivating stage-aware calibration instead of relying on a single manually specified order.
| Method | Speed | IR | CLIP |
| FADE (fixed ) | 4.99 | 0.9883 | 27.5061 |
| FADE (fixed ) | 4.99 | 1.0016 | 27.6207 |
| FADE (fixed ) | 4.99 | 0.9978 | 27.6239 |
| FADE (fixed ) | 4.99 | 0.9936 | 27.6617 |
| FADE (dynamic fit ) | 4.99 | 0.9948 | 27.7077 |
Table 7. Additional ablation on fractional-order scheduling strategies for FLUX.1.
We also evaluate simple fractional-order scheduling strategies on DiT-XL/2. This study compares the original integer-order Taylor baseline with a manually tuned fixed fractional order and a fitted time-dependent schedule.
As shown in Table 8, both the fixed fractional order and the fitted time-dependent schedule improve over the integer-order Taylor baseline in terms of FID. However, the fitted schedule does not consistently outperform the manually tuned fixed order, suggesting that simple parametric schedules can be sensitive to the chosen functional form. Together with the FLUX.1 results, this supports the use of stage-aware calibration, which avoids imposing a single fixed or manually fitted schedule.
| Method | FID | sFID |
| DDIM-50 | 2.29 | 4.29 |
| HiCache | 2.51 | 5.20 |
| Taylor | 2.590 | 5.18 |
| FADE (fixed ) | 2.49 | 4.81 |
| FADE (Bernoulli-fit ) | 2.51 | 5.14 |
Table 8. Additional ablation on fractional-order scheduling strategies for DiT-XL/2. “–” indicates that the metric was not recorded.
Detailed Experimental Protocols
As mentioned in the main text, experiments are conducted on three representative diffusion transformer models across different generative tasks: FLUX.1-dev Labs [ [17] for text-to-image generation, HunyuanVideo Kong et al. [ [15] for text-to-video generation, and DiT-XL/2 Peebles and Xie [ [33] for class-conditional image generation. This section provides detailed experimental configurations, including hardware infrastructure, sampling protocols, FADE-specific calibration settings, and baseline hyperparameters.
Hardware and Software Infrastructure
Hardware platforms are selected according to the memory and computation requirements of each model. Specifically, DiT-XL/2 is evaluated on 32GB NVIDIA vGPUs, FLUX.1-dev on NVIDIA H800 GPUs, and HunyuanVideo on NVIDIA A800 GPUs. All models follow their official inference pipelines and default precision settings. Classifier-free guidance (CFG) scales, schedulers, input resolutions, and prompt settings are kept consistent with the standard configurations of the corresponding models.
Model Configurations and Sampling Parameters
Across all evaluated models, official pretrained checkpoints and inference pipelines are strictly followed. Unless otherwise specified, all methods are evaluated under the same number of inference steps, sampling schedules, random seeds, and acceleration ratios for fair comparison.
DiT-XL/2 (ImageNet-1K): The pretrained DiT-XL/2 model is used for class-conditional image generation at a spatial resolution of 256 256. The inference process is driven by the DDIM sampler with a total of 50 denoising steps.
FLUX.1-dev (DrawBench): The rectified-flow-based FLUX.1-dev model is evaluated for high-fidelity text-to-image generation. Sampling is performed using the default Flow Match Euler scheduler at a resolution of 1024 1024, with 50 inference steps.
HunyuanVideo (VBench): Text-to-video generation experiments are conducted using the official HunyuanVideo inference pipeline. The default flow-matching sampler is adopted
with 50 denoising steps, and all video generation settings follow the official configuration unless otherwise stated.
FADE and Baseline Configurations
A detailed configuration scheme for the proposed FADE method and the reproduced baselines is provided below.
FADE (Ours): FADE is implemented as a training-free feature forecasting framework. For each model, we first run an offline calibration stage to profile the optimal fractional order for different denoising stages and network locations. The calibration objective is defined as a directional trajectory matching loss based on cosine distance between the FADE-predicted feature and the corresponding full-computation feature. This choice avoids over-penalizing magnitude variations caused by normalization layers and focuses on the angular consistency of latent feature trajectories.
The calibrated fractional orders are stored in a static lookup table (LUT), which is indexed by model-specific structural states. For DiT-XL/2, the LUT is indexed by timestep, layer, and module. For FLUX.1-dev, the LUT additionally considers the stream and block structure of the model. For HunyuanVideo, we adopt a coarser LUT indexing strategy due to the substantially higher computational cost and inherent spatiotemporal redundancy of video generation. The LUT is therefore constructed based on timestep and coarse video-transformer block indices. During online inference, FADE directly retrieves the corresponding from the LUT and performs fractional-order feature extrapolation without any additional optimization or adaptive search.
Baselines: We compare FADE with representative training-free diffusion acceleration methods, including Selvaraju et al. [ [37], Feng et al. [ [7], and Liu et al. [ [22]. All reproduced baselines strictly follow the implementation protocols and default hyperparameter settings reported in their original papers and open-source repositories. For a fair comparison, all baselines are evaluated using the same base model, sampler, number of inference steps, resolution, CFG scale, prompt set, and acceleration schedule as FADE.
Offline calibration protocol. For each calibration trajectory, a full-computation reference pass collects the oracle features at prediction locations. Given the cached feature basis and prediction distance , each candidate order is evaluated by
where is the number of retained basis terms and the zeroth-order coefficient is 1. The candidate minimizing the directional error is selected:
The selected predictions are propagated using the closed-loop procedure in Algorithm 1.
For DiT-XL/2, indexes the layer and module, and we use with all available cached orders. For FLUX.1-dev, additionally specifies the stream and block, and we use with a two-term cached basis for numerical stability. HunyuanVideo follows the FLUX.1-dev protocol with a coarser search and LUT indexed by timestep and video-transformer block. The calibrated orders are stored in a static LUT for inference.
Online inference settings. For each model and cache interval , we tune offline and compute the refined orders:
The refined orders are stored in the LUT and retrieved directly during inference. We keep fixed across all evaluation prompts. The settings are summarized in Table 9.
| Model | Cache interval | Refinement factor |
| DiT-XL/2 | 4 | 0.6 |
| DiT-XL/2 | 5 | 0.6 |
| DiT-XL/2 | 6 | 0.7 |
| FLUX.1-dev | 5 | 1.3 |
| FLUX.1-dev | 6 | 1.0 |
| FLUX.1-dev | 7 | 1.1 |
| HunyuanVideo | default | 1.0 |
Table 9. Online inference settings of FADE. The refinement factor is fixed after offline calibration and introduces no online search overhead.
References
- [1]Hyojin Bahng, Caroline Chan, Fredo Durand, and Phillip Isola. Cycle consistency as reward: Learning image-text alignment without human preferences. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 22934–22946, October 2025.arxiv.org/abs/2506.02095
- [2]Daniel Bolya and Judy Hoffman. Token merging for fast stable diffusion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4599–4603, 2023.arxiv.org/abs/2303.17604
- [3]Qi Cai, Jingwen Chen, Yang Chen, Yehao Li, Fuchen Long, Yingwei Pan, Zhaofan Qiu, Yiheng Zhang, Fengbin Gao, Peihan Xu, Yimeng Wang, Kai Yu, Wenxuan Chen, Ziwei Feng, Zijian Gong, Jianzhuang Pan, Yi Peng, Rui Tian, Siyu Wang, Bo Zhao, Ting Yao, and Tao Mei. Hidream-ii: A high-end efficient image generative foundation model with sparse diffusion transformer, 2025. URL https://arxiv.org/abs/2505.22705.
- [4]Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-α: Fast training of diffusion transformer for photorealistic text-to-image synthesis, 2023. URL https://arxiv.org/abs/2310.00426.
- [5]Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009. doi: 10.1109/CVPR.2009.5206848.DOI
- [6]Gongfan Fang, Xinyin Ma, and Xinchao Wang. Structural pruning for diffusion models, 2023. URL https://arxiv.org/abs/2305.10924.
- [7]Liang Feng, Shikang Zheng, Jiacheng Liu, Yuqi Lin, Qinming Zhou, Peiliang Cai, Xinyu Wang, Junjie Chen, Chang Zhou, Yue Ma, and Linfeng Zhang. Hicache: A plug-in scaled-hermite upgrade for taylor-style cache-then-forecast diffusion acceleration. In International Conference on Learning Representations (ICLR), 2026.arxiv.org/abs/2508.16984
- [8]Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning, 2022. URL https://arxiv.org/abs/2104.08718.
- [9]Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium, 2018. URL https://arxiv.org/abs/1706.08500.
- [10]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 6840–6851. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/file/4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf.arxiv.org/abs/2006.11239
- [11]Alain Horé and Djemel Ziou. Image quality metrics: Psnr vs. ssim. In 2010 20th International Conference on Pattern Recognition, pages 2366–2369, 2010. doi: 10.1109/ICPR.2010.579.DOI
- [12]Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21807–21818, 2024.arxiv.org/abs/2311.17982
- [13]Kumara Kahatapitiya, Haozhe Liu, Sen He, Ding Liu, Menglin Jia, Chenyang Zhang, Michael S Ryoo, and Tian Xie. Adaptive caching for faster video generation with diffusion transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15240–15252, 2025.arxiv.org/abs/2411.02397
- [14]Dimitris Kalatzis, David Eklund, Georgios Arvanitidis, and Søren Hauberg. Variational autoencoders with riemannian brownian motion priors. arXiv preprint arXiv:2002.05227, 2020.arxiv.org/abs/2002.05227
- [15]Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024.arxiv.org/abs/2412.03603
- [16]Zhifeng Kong, Wei Ping, Jiaj i Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis, 2021. URL https://arxiv.org/abs/2009.09761.
- [17]Black Forest Labs. Flux. https://github.com/black-forest-labs/flux, 2024.
- [18]Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, Sumith Kulal, Kyle Lacey, Yam Levi, Cheng Li, Dominik Lorenz, Jonas Müller, Dustin Podell, Robin Rombach, Harry Saini, Axel Sauer, and Luke Smith. Flux.1 Kontext: Flow matching for in-context image generation and editing in latent space, 2025. URL https://arxiv.org/abs/2506.15742.
- [19]Xiuyu Li, Yijiang Liu, Long Lian, Huanrui Yang, Zhen Dong, Daniel Kang, Shanghang Zhang, and Kurt Keutzer. Q-diffusion: Quantizing diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17535–17545, 2023.arxiv.org/abs/2302.04304
- [20]Feng Liu, Shiwei Zhang, Xiaofeng Wang, Yujie Wei, Haonan Qiu, Yuzhong Zhao, Yingya Zhang, Qixiang Ye, and Fang Wan. Timestep embedding tells: It’s time to cache for video diffusion model. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 7353–7363, 2025a.
- [21]Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley. AudioIdm: Text-to-audio generation with latent diffusion models. arXiv preprint arXiv:2301.12503, 2023.
- [22]Jiacheng Liu, Chang Zou, Yuanhuiyi Lyu, Junjie Chen, and Linfeng Zhang. From reusing to forecasting: Accelerating diffusion models with taylors eers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15853–15863, 2025b.
- [23]Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow, 2022. URL https://arxiv.org/abs/2209.03003.
- [24]Alexander Lobashev, Dmitry Guskov, Maria Larchenko, and Mikhail Tamm. Hessian geometry of latent space in generative models. arXiv preprint arXiv:2506.10632, 2025.arxiv.org/abs/2506.10632
- [25]Jinming Lou, Wenyang Luo, Yufan Liu, Bing Li, Xinmiao Ding, Weiming Hu, Yuming Li, and Chengguang Ma. Token caching for diffusion transformer acceleration. arXiv preprint arXiv:2409.18523, 2024.arxiv.org/abs/2409.18523
- [26]Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in neural information processing systems, 35:5775–5787, 2022.
- [27]Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. Machine Intelligence Research, 22(4):730–751, June 2025. ISSN 2731-5398. doi: 10.1007/s11633-025-1562-4. URL http://dx.doi.org/10.1007/s11633-025-1562-4.
- [28]Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference, 2023. URL https://arxiv.org/abs/2310.04378.
- [29]Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden, and Saining Xie. SiT: Exploring flow and diffusion-based generative models with scalable interpolant transformers. In European Conference on Computer Vision, pages 23–40. Springer, 2024a.
- [30]Xinyin Ma, Gongfan Fang, and Xinchao Wang. Deepcache: Accelerating diffusion models for free. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15762–15772, 2024b.arxiv.org/abs/2312.00858
- [31]Ralf Metzler and Joseph Klafter. The random walk’s guide to anomalous diffusion: a fractional dynamics approach. Physics Reports, 339(1):1–77, 2000. ISSN 0370-1573. doi: https://doi.org/10.1016/S0370-1573(00)00070-3. URL https://www.sciencedirect.com/science/article/pii/S0370157300000703.
- [32]Fernando A Oliveira, Rogelma MS Ferreira, Luciano C Lapas, and Mendeli H Vainstein. Anomalous diffusion: A basic mechanism for the evolution of inhomogeneous systems. Frontiers in Physics, 7:18, 2019.DOI
- [33]William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023.arxiv.org/abs/2212.09748
- [34]Xurui Peng, Chenqian Yan, Hong Liu, Rui Ma, Fangmin Chen, Xing Wang, Zhihua Wu, Songwei Liu, and Mingbao Lin. Ertacache: Error rectification and timesteps adjustment for efficient diffusion. arXiv preprint arXiv:2508.21091, 2025.arxiv.org/abs/2508.21091
- [35]Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding, 2022. URL https://arxiv.org/abs/2205.11487.
- [36]Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models, 2022. URL https://arxiv.org/abs/2202.00512.
- [37]Pratheba Selvaraju, Tianyu Ding, Tianyi Chen, Ilya Zharkov, and Luming Liang. Fora: Fast-forward caching in diffusion transformer acceleration, 2024. URL https://arxiv.org/abs/2407.01425.
- [38]Yuzhang Shang, Zhihang Yuan, Bin Xie, Bingzhe Wu, and Yan Yan. Post-training quantization on diffusion models. In CVPR, 2023.arxiv.org/abs/2211.15736
- [39]Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models, 2022. URL https://arxiv.org/abs/2010.02502.
- [40]Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models, 2023. URL https://arxiv.org/abs/2303.01469.
- [41]Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Zhang, Siyang Sun, Tao Fang, Tianxing Wang, Tianyi Gui, Tingyu Weng, Tong Shen, Wei Lin, Wei Wang, Wei Wang, Wenmeng Zhou, Wente Wang, Wenting Shen, Wenyuan Yu, Xianzhong Shi, Xiaoming Huang, Xin Xu, Yan Kou, Yangyu Lv, Yifei Li, Yijing Liu, Yiming Wang, Yingya Zhang, Yitong Huang, Yong Li, You Wu, Yu Liu, Yulin Pan, Yun Zheng, Yuntao Hong, Yupeng Shi, Yutong Feng, Zeyinzi Jiang, Ze Han, Zhi-Fan Wu, and Ziyu Liu. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025.arxiv.org/abs/2503.20314
- [42]Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imageward: learning and evaluating human preferences for text-to-image generation. In Proceedings of the 37th International Conference on Neural Information Processing Systems, pages 15903–15935, 2023.arxiv.org/abs/2304.05977
- [43]Jinlong Yang, Jinke Wu, Lizilin, and Yao Zhou. Brace: Taming sharp irregularities via barycentric rational forecasting for fast diffusion transformers inference, 2026. URL https://arxiv.org/abs/2608.07572.
- [44]Lianwei Yang, Haokun Lin, Tianchen Zhao, Yichen Wu, Hongyu Zhu, Ruiqi Xie, Zhenan Sun, Yu Wang, and Qingyi Gu. Lrq-dit: Log-rotation post-training quantization of diffusion transformers for image and video generation. arXiv preprint arXiv:2508.03485, 2025.arxiv.org/abs/2508.03485


