betaFeedback

WorksComputer Science

FADE: Fractional Anomalous Dynamics Extrapolation for Training-Free Diffusion Transformers Acceleration

Abstract

Diffusion Transformers (DiTs) have established themselves as the preeminent paradigm for high-fidelity generative modeling across various modalities, yet their practical deployment is hindered by the severe computational overhead of iterative sampling. While recent feature-caching methods mitigate this latency through feature reuse or forecasting based on integer-order operators, they remain limited in adapting to the non-stationary and highly curved dynamics of DiT latent trajectories. In this work, we reveal that DiT feature evolution exhibits anomalous diffusion characteristics and a certain degree of spatiotemporal heterogeneity, rendering static, integer-order forecasting operators inadequate and prone to error accumulation. To address this, we propose Fractional Anomalous Dynamics Extrapolation (FADE), a unified framework for training-free diffusion transformers acceleration. FADE models DiT feature evolution via anomalous diffusion and employs a Mittag–Leffler fractional operator to better adapt to high-curvature and non-linear latent trajectories of DiTs. Leveraging the empirical observation that these complex dynamics maintain cross-sample stability, we further construct a Look-Up Table with multiple dimensions through offline profiling. This enables negligible LUT-retrieval overhead, spatiotemporally adaptive parameter retrieval during online inference. Extensive evaluations across diverse architectures and modalities demonstrate that FADE achieves state-of-the-art acceleration, preserving exceptional generation fidelity under aggressive skip intervals with negligible latency overhead. Our project page is available at https://youngkinlon.github.io/fade-project-page/.

Introduction

Diffusion Models Ho et al. [ [10] have become a dominant paradigm for generative modeling across images Chen et al. [ [4], Cai et al. [ [3], Labs [ [17], videos Kong et al. [ [15], Wan et al. [ [41], and audio Kong et al. [ [16], Liu et al. [ [21], owing to their strong high-fidelity synthesis capability. Recent advances in Diffusion Transformers (DiTs) Peebles and Xie [ [33] have further improved this capability, but also introduce substantial computational overhead. Their iterative sampling process requires sequential forward passes through large-scale models, resulting in high latency that hinders practical deployment and motivates training-free acceleration without modifying model weights.

To alleviate this bottleneck, feature caching has emerged as a training-free acceleration strategy. Traditional caching methods Ma et al. [ [30], Selvaraju et al. [ [37] reuse features from previous timesteps by exploiting temporal consistency in intermediate representations. However, this cache-and-reuse paradigm degrades generation quality under large skip intervals, as feature similarity diminishes over longer temporal distances Selvaraju et al. [ [37], Ma et al. [ [30]. Methods such as TaylorSeer Liu et al. [ [22] and HiCache Feng et al. [ [7] formulate feature forecasting based on integer-order forecasting operators. Yet such formulations implicitly assume locally smooth dynamics with a fixed coefficient law, providing only limited approximations to DiT latent trajectories with complex geometry. As a result, they struggle to capture the intrinsic curvature, stage-dependent variations, and abrupt trajectory changes observed across denoising stages.

This limitation reflects a deeper geometric mismatch between existing forecasting schemes and DiT latent dynamics. Prior studies have shown that flat Euclidean assumptions are insufficient to capture the topology of deep generative latent spaces Kalatzis et al. [ [14], while recent analyses further suggest the existence of fractal phase-transition structures in diffusion latents Lobashev et al. [ [24]. Since anomalous diffusion commonly arises on fractal structures Oliveira et al. [ [32], this connection motivates us to investigate whether DiT feature evolution exhibits anomalous diffusion behavior. In anomalous diffusion studies, the power-law scaling exponent of the mean squared displacement (MSD) serves as a standard empirical diagnostic for deviations from Brownian diffusion Metzler and Klafter [ [31]. Using this diagnostic, we observe that the normalized MSD of DiT features deviates from Brownian linear scaling, as shown in Figure 1(a), thereby supporting a fractional perspective for modeling such dynamics. To further characterize the structure of feature evolution, we examine its variation across denoising stages, network layers, and input samples from an anomalous-diffusion perspective. As shown in Figure 1(b), feature dynamics vary substantially across layers and timesteps, while exhibiting relatively smaller variation across input samples. This observation suggests structured stage- and layer-dependent dynamics with a degree of cross-sample consistency, providing empirical support for calibrating forecasting parameters offline by denoising stage and network location and reusing them across samples.

Empirical evidence of anomalous and structured dynamics in DiT

Figure 1. Empirical evidence of anomalous and structured dynamics in DiT. (a) The normalized mean squared displacement across layers follows an anomalous diffusion scaling law. (b) Cross-sample trajectory consistency across different layers and denoising steps.

Building on these observations, our key insight is to treat DiT feature evolution as an anomalous diffusion process over complex latent geometry, while exploiting its empirical cross-sample regularity for offline calibration. This motivates a forecasting framework that combines a fractional dynamical model with offline-calibrated stage-aware adaptation. Accordingly, we propose FADE (Fractional Anomalous Dynamics Extrapolation), a training-free framework that models DiT feature evolution via anomalous diffusion for diffusion transformers acceleration. A Mittag–Leffler fractional operator is introduced to generalize the classical integer-order shift operator, enabling flexible prediction of non-Brownian feature evolution. To operationalize this adaptive formulation, FADE profiles the optimal fractional parameters offline and stores them in a multi-dimensional LUT, which allows stage-aware feature forecasting through O(1)\mathcal{O}(1) retrieval during inference. This design improves extrapolation robustness under large skip intervals while preserving generation quality with negligible overhead.

In summary, the principal contributions of our work are as follows:

  • Fractional Anomalous Dynamics Extrapolation (FADE): We propose a training-free forecasting framework that models DiT feature evolution as an anomalous diffusion process. By generalizing the classical exponential shift operator with a Mittag–Leffler fractional operator, FADE better adapts to the complex, high-curvature dynamics of latent trajectories.

  • Zero-Overhead Adaptive Forecasting: To translate our fractional framework into practical acceleration, we construct a Look-Up Table indexed by multiple dimensions through offline profiling. This design enables spatiotemporally adaptive parameter retrieval in constant time, ensuring robust feature forecasting without introducing any additional inference latency.

  • Empirical Characterization of DiT Feature Dynamics: We reveal that DiT feature evolution exhibits anomalous diffusion behavior and structured spatiotemporal heterogeneity across denoising stages and network locations. Meanwhile, these dynamics remain relatively stable across input samples, providing empirical support for offline calibration.

  • State-of-the-Art Acceleration: Extensive evaluations across diverse architectures and modalities, including DiT-XL/2, FLUX.1, and HunyuanVideo, demonstrate that FADE consistently outperforms existing baselines, maintaining exceptional generation fidelity and negligible latency overhead even under long skip intervals.

Related Work

Computation Reduction Strategies. Previous efforts to accelerate diffusion models focus on either reducing the number of sampling steps or compressing the denoising network. Step-reduction methods accelerate the generative process through efficient ODE solvers like DDIM Song et al. [ [39] and DPM-Solver Lu et al. [ [26], Zhao et al. [2023], Lu et al. [ [27], knowledge distillation Salimans and Ho [ [36], Luo et al. [ [28], or few-step paradigms such as Rectified Flow Liu et al. [ [23] and Consistency Models Song et al. [ [40]. However, these approaches often degrade generation quality or require expensive retraining Ma et al. [ [29]. Alternatively, network compression techniques such as pruning Fang et al. [ [6], Yuan et al. [2024], quantization Li et al. [ [19], Shang et al. [ [38], Yang et al. [ [44], and token merging Zhang et al. [2024b], Bolya and Hoffman [ [2] reduce the FLOPs of each denoising step. However, their efficiency gains often come at the cost of reduced model expressivity, which may compromise generation fidelity under aggressive compression. In contrast, feature caching reduces redundant intermediate feature computation to accelerate inference.

Feature Caching. As a prominent training-free alternative, feature caching skips redundant intermediate computations. Existing methods generally fall into two paradigms: cache-then-reuse and cache-then-forecast. Reuse-based approaches reuse cached representations, including direct reuse methods such as DeepCache Ma et al. [ [30], PAB Zhao et al. [2024], and FORA Selvaraju et al. [ [37], token-level methods such as ToCa Zou et al. [2025], TokenCache Lou et al. [ [25], and DaTo Zhang et al. [2024a], and error-aware or adaptive caching variants such as TeaCache Liu et al. [ [20], AdaCache Kahatapitiya et al. [ [13], and ERTACache Peng et al. [ [34]. MEDA Zhou et al. [2026] uses multiobjective evolutionary search to optimize feature-reuse schedules, balancing computational cost and generation quality. However, larger skip intervals inevitably enlarge the temporal representation mismatch, degrading generation quality. However, larger skip intervals inevitably enlarge the temporal representation mismatch, degrading generation quality. Forecasting-based approaches, pioneered by TaylorSeer Liu et al. [ [22], actively predict feature evolution. Recent advancements seek to refine this paradigm: HiCache Feng et al. [ [7] improves the standard Taylor expansion by introducing scaled Hermite coefficients, while FOCA Zheng et al. [2026] drives the forecasting process by utilizing both the features themselves and their first-order finite-difference derivatives. BRACE Yang et al. [ [43] further explores barycentric rational forecasting with adapted Chebyshev weights, directly aggregating cached historical features to accommodate local trajectory irregularities. Despite these algorithmic refinements, these methods treat the non-equilibrium denoising trajectory as a globally uniform dynamical system, limiting their capacity to characterize the anomalous diffusion properties and complex temporal phase transitions of latent features. This ultimately results in trajectory deviation under large skip intervals.

Method

Preliminaries

Diffusion models generate data by iteratively reversing a predefined forward noising process via a neural network ϵθ(xt,t)\epsilon_\theta(x_t,t) that estimates the denoising direction at each timestep. Starting from Gaussian noise, the latent variable xtx_t is progressively refined over TT timesteps toward a clean sample:

xt−1=1αt(xt−1−αt1−αˉtϵθ(xt,t))+σtϵ.(1)\mathbf{x}_{t-1}=\frac{1}{\sqrt{\alpha_t}}\left(\mathbf{x}_t-\frac{1-\alpha_t}{\sqrt{1-\bar{\alpha}_t}}\epsilon_\theta(\mathbf{x}_t,t)\right)+\sigma_t\epsilon. \tag*{(1)}

Since diffusion sampling requires TT sequential model evaluations, inference is costly, especially for large-scale DiTs Peebles and Xie [ [33]. To reduce full forward passes at selected timesteps, Predictive feature caching reduces this cost by forecasting intermediate features Fl\mathbf{F}^{l} at layer ll. With a recursive discrete difference operator Δ\Delta, the mm-th order Taylor predictor over distance δ\delta is:

Fpred(xt−δl)=F(xtl)+∑i=1mΔiF(xtl)i!(−δ)i.(2)\mathbf{F}_{\mathrm{pred}}(\mathbf{x}^{l}_{t-\delta})=\mathbf{F}(\mathbf{x}^{l}_{t})+\sum_{i=1}^{m}\frac{\Delta^{i}\mathbf{F}(\mathbf{x}^{l}_{t})}{i!}(-\delta)^{i}. \tag*{(2)}

Based on operator calculus, the above polynomial extrapolation over a prediction distance δ\delta can be interpreted as applying the exponential shift operator eze^{z} to the current feature Ft\mathbf{F}_t, where z=−δ∇tz=-\delta\nabla_t and ∇t=∂/∂t\nabla_t=\partial/\partial t denotes the temporal derivative operator:

Ft−δ≈ezFt.(3)\mathbf{F}_{t-\delta}\approx e^{z}\mathbf{F}_t. \tag*{(3)}

From this perspective, conventional integer-order extrapolation methods typically rely on finite-order local approximations of the shift operator, which may struggle to capture the complex and non-stationary dynamics of DiT’s latent trajectories.

Overall Framework of FADE

Motivated by the anomalous diffusion of DiT latent feature evolution, we propose Fractional Anomalous Dynamics Extrapolation (FADE). As shown in Figure 2, FADE consists of an offline FADE Profiling Framework and an online FADE Inference Architecture. In the profiling stage, calibration samples are used for Optimal Order Search with Directional Alignment, where the optimal fractional order γt,s∗\gamma^{*}_{t,s} is obtained by minimizing the directional loss Ldir\mathcal{L}_{\mathrm{dir}} between the predicted feature FFADE\mathbf{F}_{\mathrm{FADE}} and the ground-truth feature FGT\mathbf{F}_{\mathrm{GT}}. The calibrated orders are then stored in a static look-up table (LUT), where ss indexes the network location(e.g., layer, module, stream, etc). In the inference stage, denoising follows a cache interval NN and alternates between Full Compute Steps and Prediction Steps: Full Compute Steps perform forward passes and cache the finite-difference set {ΔiFt}i=1m\{\Delta^{i}\mathbf{F}_t\}_{i=1}^{m}, while Prediction Steps retrieve γ^t,s\hat{\gamma}_{t,s} from the LUT and apply the Fractional Operator Eγ(ξ)E_{\gamma}(\xi) to generate the predicted feature Fpred\mathbf{F}_{\mathrm{pred}}. The geometric view in Figure 2 further illustrates that FADE better adapts to the complex latent manifold of DiTs.

Figure 2. Overall framework of FADE. FADE consists of an offline profiling stage and an online inference stage. The offline stage calibrates and stores the optimal fractional orders in a static LUT, while the online stage alternates between full-compute and prediction steps using LUT-retrieved orders for feature forecasting. The upper-right panel illustrates the geometric contrast between the FADE trajectory and the integer-order trajectory in the latent space.

Modeling DiT Feature Evolution via Anomalous Diffusion

Building upon the observed latent dynamics and prior studies on DiT latent geometry, we model DiT feature evolution from an anomalous-diffusion perspective. Anomalous transport on fractal structures provides a geometric motivation for this view Oliveira et al. [ [32], while our observed deviations from Brownian MSD scaling provide empirical support. To formulate a forecasting rule for such dynamics, we draw on classical fractional dynamics, where Mittag–Leffler functions commonly arise in propagation and relaxation solutions Metzler and Klafter [ [31]. Since the Mittag–Leffler function reduces to the exponential function at unit order, we adopt it as a fractional operator prior to generalize the classical exponential shift operator. This provides a flexible coefficient law for controlling how higher-order local variations contribute to feature prediction:

Eγ(ξ)=∑k=0∞ξkΓ(γk+1).(4)E_{\gamma}(\xi)=\sum_{k=0}^{\infty}\frac{\xi^{k}}{\Gamma(\gamma k+1)}. \tag*{(4)}

Here, Γ(⋅)\Gamma(\cdot) denotes the Gamma function used to parameterize the fractional coefficients. Following this fractional coefficient law, we construct a finite-order feature predictor as

Fpred(xt−δl)=F(xtl)+∑i=1mΔiF(xtl)Γ(γi+1)(−δ)i.(5)\mathbf{F}_{\mathrm{pred}}(x_{t-\delta}^{l}) = \mathbf{F}(x_{t}^{l}) + \sum_{i=1}^{m} \frac{\Delta^{i}\mathbf{F}(x_{t}^{l})}{\Gamma(\gamma i + 1)}(-\delta)^{i}. \tag*{(5)}

Under this formulation, the fractional order γ\gamma controls the coefficient law used for feature extrapolation. Since DiT denoising is highly non-stationary, feature dynamics vary across timesteps and latent states, often shifting between sub- and super-diffusive-like behaviors. This makes a fixed integer-order rule insufficient for modeling local evolution. FADE therefore assigns state-specific fractional orders to adapt the extrapolation law to heterogeneous anomalous dynamics. To obtain these orders without runtime search, we introduce a Stage-Aware Calibration Strategy.

Stage-Aware Calibration Strategy

As established in our empirical analysis, existing static caching frameworks overlook a key structural property of Diffusion Transformers: feature dynamics vary substantially across latent states and timesteps, yet exhibit a certain degree of cross-sample regularity. Leveraging this cross-sample consistency, we propose a Stage-Aware Calibration Strategy that offline pre-computes the optimal fractional order γt,s∗\gamma_{t,s}^{*} for each state ss and timestep tt, thereby avoiding online search overhead with a short one-time calibration stage and negligible LUT storage.

Directional Monitoring Objective. Calibration is formulated as a local trajectory-matching problem. We use cosine distance to emphasize directional agreement between predicted and reference features. A purely directional objective is therefore adopted, and Ldir\mathcal{L}_{\mathrm{dir}} is defined as the cosine distance between the predicted feature FFADE(γ)\mathbf{F}_{\mathrm{FADE}}(\gamma) and the ground-truth feature FGT\mathbf{F}_{\mathrm{GT}}:

Ldir(γ)=1−FFADE(γ)⋅FGT∥FFADE(γ)∥2∥FGT∥2.(6)\mathcal{L}_{\mathrm{dir}}(\gamma) = 1 - \frac{\mathbf{F}_{\mathrm{FADE}}(\gamma)\cdot\mathbf{F}_{\mathrm{GT}}}{\lVert\mathbf{F}_{\mathrm{FADE}}(\gamma)\rVert_{2}\lVert\mathbf{F}_{\mathrm{GT}}\rVert_{2}}. \tag*{(6)}

Optimal Order Search and Global Refinement. Using a small calibration dataset Dcalib\mathcal{D}_{\mathrm{calib}}, the optimal fractional order is obtained by minimizing the expected directional error over a discrete search grid G\mathcal{G}, which spans both sub- and super-diffusion-like regimes:

γt,s∗=arg⁡min⁡γ∈G Ex∼Dcalib[Ldir(FFADE(γ);x,t,s)].(7)\gamma_{t,s}^{*} = \underset{\gamma\in\mathcal{G}}{\arg\min}\ \mathbb{E}_{\mathbf{x}\sim\mathcal{D}_{\mathrm{calib}}}\left[\mathcal{L}_{\mathrm{dir}}\left(\mathbf{F}_{\mathrm{FADE}}(\gamma); \mathbf{x}, t, s\right)\right]. \tag*{(7)}

The offline-calibrated fractional orders γt,s∗\gamma_{t,s}^{*} may still exhibit a global scale bias in practical inference. To account for this potential bias, we introduce a global refinement factor λ\lambda and use γ^t,s=λ⋅γt,s∗\widehat{\gamma}_{t,s} = \lambda\cdot\gamma_{t,s}^{*}, providing a lightweight global adjustment to the calibrated orders. The refined orders γ^t,s\widehat{\gamma}_{t,s} are then stored in the LUT described in § 3.2 for inference-time retrieval. Detailed calibration settings are provided in Appendix § 10.

Error Analysis from a Fractional Perspective

We provide a local and conditional error analysis under a fractional model of local latent evolution. Assume that the local ground-truth evolution admits the following representation:

Ft−δ,sGT=Ft,s+∑i=1mΔiFt,sΓ(γt,s∗i+1)(−δM,t,s)i+Rm+1(t,s),∥Rm+1(t,s)∥≤Ct,sδM,t,sm+1,(8)\mathbf{F}^{\mathrm{GT}}_{t-\delta,s} = \mathbf{F}_{t,s} + \sum_{i=1}^{m} \frac{\Delta^{i}\mathbf{F}_{t,s}}{\Gamma(\gamma_{t,s}^{*}i+1)}(-\delta_{M,t,s})^{i} + \mathbf{R}^{(t,s)}_{m+1}, \qquad\left\lVert\mathbf{R}^{(t,s)}_{m+1}\right\rVert\le C_{t,s}\delta_{M,t,s}^{m+1}, \tag*{(8)}

where δM,t,s\delta_{M,t,s} denotes the intrinsic propagation scale along the local feature trajectory, which reflects the underlying distance traveled by features on the latent manifold, while δ\delta denotes the prediction distance used in practice. This distinction captures a potential propagation-scale mismatch. The profiled order γt,s∗\gamma_{t,s}^{*} is used as an estimate of the intrinsic order γt,s\gamma_{t,s}, and FADE applies the globally refined order γ^t,s=λγt,s∗\widehat{\gamma}_{t,s} = \lambda\gamma_{t,s}^{*} at inference time. Substituting Eq. (5) into Eq. (8), we obtain

∥Ft−δ,sGT−F^t−δ,sFADE∥≤Ct,sδM,t,sm+1+∑i=1m∥ΔiFt,s∥∣(−δM,t,s)iΓ(γt,s∗i+1)−(−δ)iΓ(λγt,s∗i+1)∣.(9)\left\lVert\mathbf{F}^{\mathrm{GT}}_{t-\delta,s} - \widehat{\mathbf{F}}^{\mathrm{FADE}}_{t-\delta,s}\right\rVert\le C_{t,s}\delta_{M,t,s}^{m+1} + \sum_{i=1}^{m}\left\lVert\Delta^{i}\mathbf{F}_{t,s}\right\rVert\left\lvert\frac{(-\delta_{M,t,s})^{i}}{\Gamma(\gamma_{t,s}^{*}i+1)}-\frac{(-\delta)^{i}}{\Gamma(\lambda\gamma_{t,s}^{*}i+1)}\right\rvert. \tag*{(9)}

The second term combines the intrinsic–extrinsic scale mismatch and the coefficient mismatch. For Taylor extrapolation in Eq. (2), the corresponding mismatch is

∑i=1m∥ΔiFt,s∥∣(−δM,t,s)iΓ(γt,s∗i+1)−(−δ)ii!∣.(10)\sum_{i=1}^{m}\left\lVert\Delta^{i}\mathbf{F}_{t,s}\right\rVert\left\lvert\frac{(-\delta_{M,t,s})^{i}}{\Gamma(\gamma_{t,s}^{*}i+1)}-\frac{(-\delta)^{i}}{i!}\right\rvert. \tag*{(10)}

Since Taylor extrapolation corresponds to the γ=1\gamma= 1 case of the fractional coefficient family, it can be viewed as a special case of FADE. In contrast, FADE calibrates a stage-adaptive order γt,s∗\gamma_{t,s}^{*} to accommodate the heterogeneous dynamics across DiT timesteps and network states, and applies a refined order γ^t,s=λγt,s∗\widehat{\gamma}_{t,s} = \lambda\gamma_{t,s}^{*} at inference time. The global factor λ\lambda can be viewed as a lightweight correction that helps accommodate residual propagation-scale mismatch. Together, these designs provide a more flexible local approximation of the intrinsic dynamics, consistent with the improved robustness observed under large prediction distances.

Experiments

Experimental Setup

FADE is evaluated across three representative generative tasks: (i) class-conditional image generation on ImageNet-1K Deng et al. [ [5] using DiT-XL/2 Peebles and Xie [ [33]; (ii) text-to-image generation on DrawBench Saharia et al. [ [35] using the rectified flow-based FLUX.1-dev Labs et al. [ [18], Labs et al. [ [17], Liu et al. [ [23]; and (iii) text-to-video generation using HunyuanVideo Kong et al. [ [15] on the HunyuanLarge architecture. For DiT-XL/2, we generate images at 256×256256 \times256 resolution using the DDIM sampler with 50 denoising steps. For FLUX.1-dev, we use the Flow Match Euler scheduler with 50 sampling steps at 1024×10241024 \times1024 resolution. HunyuanVideo follows its official inference pipeline with 50 flow-matching sampling steps and default video generation settings.

We compare FADE with representative training-free caching methods, including FORA Selvaraju et al. [ [37], TaylorSeer Liu et al. [ [22], and HiCache Feng et al. [ [7]. Within each comparison, caching methods use the same pretrained checkpoint, sampler, number of sampling steps, resolution, guidance settings, and evaluation inputs. Qualitative comparisons additionally use matched random seeds and acceleration schedules to facilitate direct visual assessment.

Quantitative performance is assessed using FID-50K Heusel et al. [ [9], sFID, and Inception Score for DiT-XL/2. For FLUX.1-dev, we report ImageReward Xu et al. [ [42], CLIP Score Hessel et al. [ [8], and CycleReward Bahng et al. [ [1], together with PSNR, SSIM Horé and Ziou [ [11], and LPIPS Zhang et al. [2018] to assess consistency with the full-compute baseline. HunyuanVideo is evaluated using the 16-dimensional VBench suite Huang et al. [ [12]. Computational efficiency is assessed using FLOPs and measured generation latency. FLOPs reduction and wall-clock speedup are computed separately as the ratios of baseline FLOPs and latency to those of the accelerated method, respectively. Offline calibration cost is reported separately from inference latency.

Empirical Validation of Fractional Dynamics

To validate the fractional prior underlying FADE, we profile the optimal fractional orders γ∗\gamma^* across representative architectures and modalities. If DiT feature evolution were well described by conventional integer-order dynamics, the optimal states would concentrate around γ=1\gamma= 1, which is not observed in practice. On DiT-XL/2, Figure 3 shows that the global distribution of γ∗\gamma^* is clearly shifted away from γ=1\gamma= 1, indicating that standard integer-order extrapolation captures only a limited portion of the effective generative states. The spatiotemporal heatmap further reveals a U-shaped temporal pattern: the middle denoising phase is more sub-diffusion-like, whereas the early and late phases are more super-diffusion-like in a phenomenological sense.

Empirical validation of fractional dynamics on DiT-XL/2, showing the global distribution and spatiotemporal heatmap of optimal fractional order $\gamma^*$

Figure 3. Empirical validation of fractional dynamics on DiT-XL/2. The optimal fractional orders deviate markedly from the static integer-order assumption and exhibit a pronounced non-stationary temporal pattern across denoising timesteps.

This heterogeneity also extends across architectures. FLUX.1 shows an inverted temporal trend and clear DoubleStream–SingleStream decoupling, while HunyuanVideo exhibits a broader super-diffusion-like tendency. Together, these results suggest that optimal feature dynamics are non-integer, architecture-dependent, and stage-dependent, motivating the stage-aware calibration design of FADE. More details are provided in Appendix D. Meanwhile, the cross-sample variance of γ∗\gamma^* is consistently small on DiT-XL/2 and remains within an acceptable range on FLUX.1 and HunyuanVideo.

Class-Conditional Image Generation

As demonstrated in Table 1, FADE consistently dominates both generation quality and efficiency metrics across all acceleration ratios on DiT-XL/2. When operating at a caching interval of N=4N = 4, FADE achieves an outstanding FID of 2.41 and an sFID of 4.79. This not only crushes the 15-step DDIM (FID 5.09) with fewer FLOPs, but also outperforms TaylorSeer’s Liu et al. [ [22] FID of 2.59 and HiCache’s Feng et al. [ [7] 2.51 under strictly identical computational costs. This superiority becomes even more pronounced under aggressive acceleration regimes. When scaling to N=6N = 6, FADE maintains a highly competitive FID of 2.87, completely mitigating the severe structural collapse observed in FORA Selvaraju et al. [ [37] and standard Taylor caching Liu et al. [ [22], which suffer from severely degraded FIDs of 9.22 and 3.17 respectively.

MethodLat. (s)↓\downarrowFLOPs (T)↓\downarrowSpd.↑\uparrowFID↓\downarrowsFID↓\downarrowIS↑\uparrow
DDIM-50 steps Song et al. [ [39]1.1823.741.00×\times2.294.29237.31
DDIM-25 steps Song et al. [ [39]0.6211.872.00×\times2.914.57232.22
DDIM-15 steps Song et al. [ [39]0.427.123.33×\times5.096.11205.54
FORA(N=4N=4) Selvaraju et al. [ [37]0.456.663.56×\times4.738.45214.20
Taylor (N=4N=4, m=4m=4) Liu et al. [ [22]0.616.663.56×\times2.595.18232.90
HiCache (N=4N=4, m=4m=4) Feng et al. [ [7]0.576.663.56×\times2.515.20232.72
FADE(N=4N=4, m=4m=4)0.556.663.56×\times2.414.79236.76
DDIM-12 steps Song et al. [ [39]0.335.704.17×\times7.948.07183.88
FORA(N=5N=5) Selvaraju et al. [ [37]0.375.244.53×\times5.819.61199.96
Taylor (N=5N=5, m=4m=4) Liu et al. [ [22]0.555.244.53×\times2.735.34228.75
HiCache (N=5N=5, m=4m=4) Feng et al. [ [7]0.505.244.53×\times2.675.48229.44
FADE(N=5N=5, m=4m=4)0.495.244.53×\times2.555.15230.89
DDIM-10 steps Song et al. [ [39]0.304.755.00×\times12.1711.24159.20
FORA(N=6N=6) Selvaraju et al. [ [37]0.344.764.98×\times9.2214.86169.02
Taylor (N=6N=6, m=4m=4) Liu et al. [ [22]0.504.764.98×\times3.176.36222.00
HiCache (N=6N=6, m=4m=4) Feng et al. [ [7]0.464.764.98×\times3.096.30220.55
FADE(N=6N=6, m=4m=4)0.454.764.98×\times2.875.73225.78

Table 1. Quantitative comparison of class-conditional image generation with DiT-XL/2.

Text-to-Image Generation

Quantitative Analysis Table 2 reports the quantitative results on FLUX. FADE achieves a strong quality–efficiency trade-off across acceleration settings, consistently outperforming existing baselines on semantic alignment and structural fidelity at standard configurations (N∈{5,6}\mathcal{N} \in\{5,6\}). Under a more aggressive skip interval, FADE still maintains competitive generation quality with negligible latency overhead, demonstrating the robustness of fractional forecasting.

MethodLatency(s)↓\downarrowFLOPs(T)↓\downarrowSpeed↑\uparrowImg. Reward↑\uparrowCLIP Score↑\uparrowCycle Reward↑\uparrowPSNR↑\uparrowSSIM↑\uparrowLPIPS↓\downarrow
FLUX.1 [dev] - 50 steps†^\dagger17.183719.501.00×\times0.989827.46110.8800∞\infty1.00000.0000
FLUX.1 [dev] - 25 steps8.801859.752.00×\times0.937527.33880.846619.25990.77490.3583
FLUX.1 [dev] - 20 steps7.121487.802.62×\times0.938727.25050.846618.13710.74840.4006
FORA (N=5N=5) Selvaraju et al. [ [37]5.01893.544.16×\times0.825327.23770.835315.98990.66770.5206
TaylorSeer (N=5N=5, m=1m=1) Liu et al. [ [22]5.14893.544.16×\times0.991927.51430.854018.44970.74240.4146
HiCache (N=5N=5, m=1m=1) Feng et al. [ [7]5.12893.544.16×\times0.935027.40340.834619.25950.75030.3911
FADE‡^\ddagger(N=5N=5, m=1m=1)5.04893.544.16×\times1.007127.57240.860019.66830.75930.3815
FORA (N=6N=6) Selvaraju et al. [ [37]4.40744.814.99×\times0.783627.15410.808715.52900.66630.5253
TaylorSeer (N=6N=6, m=1m=1) Liu et al. [ [22]4.57744.814.99×\times0.985027.51990.848017.53960.70780.4657
HiCache (N=6N=6, m=1m=1) Feng et al. [ [7]4.55744.814.99×\times0.942727.68980.853918.51590.71550.4503
HiCache (N=6N=6, m=2m=2) Feng et al. [ [7]5.03744.814.99×\times0.989227.65860.842818.50440.72680.4316
FADE‡^\ddagger(N=6N=6, m=1m=1)4.43744.814.99×\times1.001927.70600.870618.71140.73200.4207
FORA (N=7N=7) Selvaraju et al. [ [37]4.10670.445.55×\times0.728927.01350.796215.52900.65480.5437
TaylorSeer (N=7N=7, m=1m=1) Liu et al. [ [22]4.31670.445.55×\times0.942027.40390.797016.85630.68440.5062
HiCache (N=7N=7, m=1m=1) Feng et al. [ [7]4.20670.445.55×\times0.919127.69490.849418.27780.70320.4708
HiCache (N=7N=7, m=2m=2) Feng et al. [ [7]4.72670.445.55×\times0.988127.56510.838417.99110.70980.4592
FADE‡^\ddagger (N=7N=7, m=1m=1)4.12670.445.55×\times0.986127.80890.862118.19250.71370.4491
FADE‡^\ddagger (N=7N=7, m=2m=2)4.25670.445.55×\times0.991427.73720.859217.76360.71060.4561

Table 2. Quantitative comparison of text-to-image generation on FLUX.

Qualitative comparison. As illustrated in Figure 4, FADE maintains close visual consistency with the original FLUX.1 baseline in text rendering, texture, and scene composition. It preserves the “Text to Image” storefront and the “Diffusion” sign, whereas FORA Selvaraju et al. [ [37] alters the sign geometry and produces unclear lettering. For the brick cube, FADE retains the reddish material and mortar details, while TaylorSeer Liu et al. [ [22] produces a pale stone-like appearance, FORA generates smoother surfaces, and HiCache Feng et al. [ [7] changes the brick arrangement. FADE also preserves the cat press-conference composition and animal yearbook scene, while TaylorSeer introduces sunglasses and more human-like bodies in the latter. In the magnifying-glass example, FADE retains the train imagery within the lens, whereas FORA replaces it with text-like patterns.

Figure 4. Visualization examples for different acceleration methods on Flux.

Text-to-Video Generation

Quantitative Analysis. Table 3 reports the quantitative results on HunyuanVideo. Under the same FLOPs budget and acceleration ratio, FADE achieves competitive latency and the best VBench score, indicating that fractional forecasting remains effective for text-to-video generation.

MethodLat. (s)↓\downarrowFLOPs (T)↓\downarrowSpd.↑\uparrowVBench (%)↑\uparrow
Original (50 steps)190.2297731.00×80.68
DDIM (22% steps) Song et al. [ [39]42.765504.55×78.73
FORA (N=6N=6) Selvaraju et al. [ [37]37.153595.56×78.86
TaylorSeer (N=6N=6, m=1m=1) Liu et al. [ [22]37.853595.56×79.87
HiCache (N=6N=6, m=1m=1) Feng et al. [ [7]37.853595.56×79.93
FADE (N=6N=6, m=1m=1)37.953595.56×80.12

Table 3. Quantitative comparison of text-to-video generation on HunyuanVideo.

Qualitative Comparison. As shown in Figure 5, FADE preserves clearer text and structure in the static case, correctly rendering the word “STOP”, while better maintaining motion semantics and scene context in dynamic prompts. In particular, FADE better captures the intended temporal behavior and preserves prompt-specific motion cues, rather than merely maintaining frame-level appearance. In contrast, the baselines either miss the intended intense shaking effect in the astronaut case or introduce incorrect motion directions and off-road backgrounds in the motorcycle case.

Figure 5. Qualitative comparison on text-to-video generation.

Ablation Study

On DiT-XL/2, we compare several strategies for determining the fractional order, including fixed global tuning, polynomial fitting over denoising progress, and the proposed stage-aware LUT retrieval. We also compare MSE-based and directional calibration objectives. As shown in Table 4, the full FADE design achieves the best performance, suggesting that the optimal fractional dynamics cannot be well captured by either a static setting or a simple smooth fit, and that directional calibration provides a more effective signal for stable offline calibration.

MethodSpeed↑FID↓sFID↓
DDIM-501.00×2.294.29
DDIM-105.00×12.1711.24
HiCache4.98×3.096.30
FADE (γ=1\gamma= 1)4.98×3.176.36
FADE (fixed γ=0.85\gamma= 0.85)4.98×3.066.05
FADE (poly-fit γ(t)\gamma(t))4.98×3.006.19
FADE (LUT + MSE)4.98×2.975.91
FADE (LUT + Dir.)4.98×2.875.73

Table 4. Ablation on DiT-XL/2.

On FLUX.1, we study two practical aspects of offline calibration: the calibration set size and the global refinement factor λ\lambda. Table 5 shows that FADE remains effective with a small calibration set, saturating around ∣Dcalib∣=10|\mathcal{D}_{\mathrm{calib}}| = 10. A moderate refinement factor gives the best overall performance, with λ=1.3\lambda= 1.3 achieving the best ImageReward and λ=1.2\lambda= 1.2 the best CLIPScore. This indicates that λ\lambda is a useful lightweight correction, though its optimal value depends on the prediction interval and sampling setting. Additional ablations are provided in Appendix § 13.

MethodSpeed↑IR↑CLIP↑
FADE (∣Dcalib∣=1|\mathcal{D}_{\mathrm{calib}}| = 1)4.99×0.996327.6781
FADE (∣Dcalib∣=5|\mathcal{D}_{\mathrm{calib}}| = 5)4.99×1.001527.7454
FADE (∣Dcalib∣=10|\mathcal{D}_{\mathrm{calib}}| = 10)4.99×1.001927.7060
FADE (∣Dcalib∣=20|\mathcal{D}_{\mathrm{calib}}| = 20)4.99×1.000827.7256
FADE (λ=1.0\lambda= 1.0)4.16×0.997427.4618
FADE (λ=1.2\lambda= 1.2)4.16×0.999427.6198
FADE (λ=1.3\lambda= 1.3)4.16×1.007127.5724
FADE (λ=1.4\lambda= 1.4)4.16×0.985827.5050

Table 5. Ablation on FLUX.1.

We further report the practical cost of offline calibration and LUT retrieval. Using 10 calibration samples, end-to-end offline calibration takes approximately 50 s, 550 s, and 2,000 s for DiT-XL/2, FLUX.1, and HunyuanVideo, respectively, on their respective evaluation hardware. The LUT has a small storage footprint, occupying only 44.8 kB for DiT-XL/2. The accumulated LUT-retrieval time per generation is 0.14 ms, 0.57 ms, and 0.61 ms, respectively.

Conclusion and Future Work

This work revisits training-free DiT acceleration from the perspective of fractional dynamics. Motivated by the limitations of cache-based methods under complex stage-dependent latent dynamics, we view DiT feature evolution through the lens of anomalous diffusion and formulate FADE using a Mittag–Leffler-based fractional operator. With an offline-profiled LUT, FADE enables adaptive forecasting at inference time with O(1)O(1) retrieval and no extra runtime overhead. Experiments across diverse architectures and modalities show that FADE achieves a stronger quality–efficiency trade-off than existing baselines while remaining robust under large prediction distances.

FADE relies on manually designed calibration, and its Mittag–Leffler operator is chosen primarily for simplicity. Future work will explore alternative fractional operators and order parameterizations to improve forecasting accuracy, stability, and the quality–efficiency trade-off across denoising stages, network locations, and skip intervals.

Acknowledgments and Disclosure of Funding

This work was supported by National Natural Science Foundation of China under Grant No.62376172.

References

Technical appendices

Detailed Derivation of the FADE Predictive Formulation

In this section, we provide the rigorous step-by-step mathematical derivation detailing the transition from the classical integer-order extrapolation to our generalized Fractional Anomalous Dynamics Extrapolation (FADE) formulation.

From Polynomial Extrapolation to the Exponential Shift Operator

Based on operator calculus, the polynomial extrapolation in Eq. (2) can be interpreted as applying an exponential shift operator to the current feature representation. Let Ft=F(xtl)F_t = F(x_t^l) denote the feature at timestep tt, and let δ\delta denote the prediction distance from tt to t−δt-\delta. We define the temporal derivative operator as

∇t=∂∂t.(11)\nabla_t = \frac{\partial}{\partial t}. \tag*{(11)}

For a sufficiently smooth continuous trajectory, the Taylor expansion of Ft−δF_{t-\delta} around tt can be written as

Ft−δ=Ft+∑i=1∞∇tiFti!(−δ)i.(12)F_{t-\delta} = F_t + \sum_{i=1}^{\infty} \frac{\nabla_t^i F_t}{i!}(-\delta)^i. \tag*{(12)}

Equivalently, by collecting the derivative operator terms, we obtain

Ft−δ=(∑i=0∞(−δ∇t)ii!)Ft.(13)F_{t-\delta} = \left(\sum_{i=0}^{\infty} \frac{(-\delta\nabla_t)^i}{i!}\right) F_t. \tag*{(13)}

Since the infinite series corresponds to the Maclaurin expansion of the exponential function, this transformation can be compactly expressed as

Ft−δ=e−δ∇tFt.(14)\mathbf{F}_{t-\delta}=e^{-\delta\nabla_t}\mathbf{F}_t. \tag*{(14)}

By defining

z=−δ∇t,(15)z=-\delta\nabla_t, \tag*{(15)}

the above relation becomes

Ft−δ≈ezFt,(16)\mathbf{F}_{t-\delta}\approx e^z\mathbf{F}_t, \tag*{(16)}

where the approximation emphasizes the practical setting in which the continuous operator is implemented through finite-order truncation and discrete feature differences.

In practical predictive caching, the infinite-dimensional operator is truncated to the mm-th order:

e−δ∇tFt≈(∑i=0m(−δ∇t)ii!)Ft=Ft+∑i=1m∇tiFti!(−δ)i.(17)e^{-\delta\nabla_t}\mathbf{F}_t\approx\left(\sum_{i=0}^{m}\frac{(-\delta\nabla_t)^i}{i!}\right)\mathbf{F}_t=\mathbf{F}_t+\sum_{i=1}^{m}\frac{\nabla_t^i\mathbf{F}_t}{i!}(-\delta)^i. \tag*{(17)}

By approximating the continuous derivatives with discrete feature differences, i.e.,

ΔiF(xtl)≈∇tiFt,(18)\Delta^i\mathbf{F}(x_t^l)\approx\nabla_t^i\mathbf{F}_t, \tag*{(18)}

we recover the polynomial forecasting form:

Fpred(xt−δl)=F(xtl)+∑i=1mΔiF(xtl)i!(−δ)i.(19)\mathbf{F}_{\mathrm{pred}}(x_{t-\delta}^l)=\mathbf{F}(x_t^l)+\sum_{i=1}^{m}\frac{\Delta^i\mathbf{F}(x_t^l)}{i!}(-\delta)^i. \tag*{(19)}

Therefore, the commonly used mm-th order polynomial extrapolation can be viewed as a truncated and discretized approximation of the exponential shift operator e−δ∇te^{-\delta\nabla_t}.

Detailed Derivation of the FADE Predictive Formulation

In this section, we provide a step-by-step derivation of the FADE predictive formulation, showing how the classical integer-order extrapolation can be generalized through a fractional Mittag–Leffler operator.

Operator Equivalence. We first revisit the continuous integer-order baseline. Let Ft=F(xtl)\mathbf{F}_t=\mathbf{F}(x_t^l) denote the latent feature at timestep tt, and let δ\delta denote the prediction distance from tt to t−δt-\delta. For a sufficiently smooth feature trajectory, the Taylor expansion of Ft−δ\mathbf{F}_{t-\delta} around tt is given by

Ft−δ=∑i=0∞(−δ)ii!∂iFt∂ti.(20)\mathbf{F}_{t-\delta}=\sum_{i=0}^{\infty}\frac{(-\delta)^i}{i!}\frac{\partial^i\mathbf{F}_t}{\partial t^i}. \tag*{(20)}

By defining the temporal derivative operator as

∇t=∂∂t,(21)\nabla_t=\frac{\partial}{\partial t}, \tag*{(21)}

the above expansion can be written in operator form as

Ft−δ=(∑i=0∞(−δ∇t)ii!)Ft=e−δ∇tFt.(22)\mathbf{F}_{t-\delta}=\left(\sum_{i=0}^{\infty}\frac{(-\delta\nabla_t)^i}{i!}\right)\mathbf{F}_t=e^{-\delta\nabla_t}\mathbf{F}_t. \tag*{(22)}

Equivalently, by defining

z=−δ∇t,(23)z=-\delta\nabla_t, \tag*{(23)}

the integer-order extrapolation can be interpreted as applying the exponential shift operator:

Ft−δ≈ezFt,(24)\mathbf{F}_{t-\delta}\approx e^z\mathbf{F}_t, \tag*{(24)}

where the approximation emphasizes the practical setting in which the continuous operator is implemented through finite-order truncation and discrete feature differences.

Continuous Fractional Generalization. To model the anomalous and stage-dependent evolution of DiT features, FADE replaces the classical exponential operator with a Mittag–Leffler (ML) operator. The ML function is defined as

Eγ(z)=∑i=0∞ziΓ(γi+1),(25)E_{\gamma}(z)=\sum_{i=0}^{\infty}\frac{z^{i}}{\Gamma(\gamma i+1)}, \tag*{(25)}

where γ\gamma controls the fractional propagation order. Substituting the exponential shift variable z=−δ∇tz=-\delta\nabla_t into the ML operator gives

Eγ(−δ∇t)Ft=∑i=0∞(−δ∇t)iΓ(γi+1)Ft.(26)E_{\gamma}(-\delta\nabla_t)\mathbf{F}_t=\sum_{i=0}^{\infty}\frac{(-\delta\nabla_t)^i}{\Gamma(\gamma i+1)}\mathbf{F}_t. \tag*{(26)}

By isolating the zero-order term and using Γ(1)=1\Gamma(1)=1, we obtain

Eγ(−δ∇t)Ft=Ft+∑i=1∞(−δ)iΓ(γi+1)∇tiFt.(27)E_{\gamma}(-\delta\nabla_t)\mathbf{F}_t=\mathbf{F}_t+\sum_{i=1}^{\infty}\frac{(-\delta)^i}{\Gamma(\gamma i+1)}\nabla_t^i\mathbf{F}_t. \tag*{(27)}

Truncation and Numerical Discretization. For tractable inference, the infinite fractional series is truncated to a finite order mm. Meanwhile, since diffusion sampling proceeds over discrete timesteps, the continuous derivative ∇tiFt\nabla_t^i\mathbf{F}_t is approximated by the ii-th order discrete feature difference, denoted by ΔiFt\Delta^i\mathbf{F}_t. The backward difference operator is recursively defined as

Δ1Ft=Ft−Ft−1,(28)\Delta^1\mathbf{F}_t=\mathbf{F}_t-\mathbf{F}_{t-1}, \tag*{(28)}
ΔiFt=Δi−1Ft−Δi−1Ft−1,i≥2.(29)\Delta^i\mathbf{F}_t=\Delta^{i-1}\mathbf{F}_t-\Delta^{i-1}\mathbf{F}_{t-1},\quad i\ge2. \tag*{(29)}

Substituting the continuous derivatives with discrete feature differences yields the FADE predictive formulation:

Fpred(xt−δl)=F(xtl)+∑i=1mΔiF(xtl)Γ(γi+1)(−δ)i.(30)\mathbf{F}_{\mathrm{pred}}(x_{t-\delta}^{l})=\mathbf{F}(x_t^{l})+\sum_{i=1}^{m}\frac{\Delta^i\mathbf{F}(x_t^{l})}{\Gamma(\gamma i+1)}(-\delta)^i. \tag*{(30)}

Here, δ\delta denotes the effective prediction distance, instantiated in practice as the skip interval between the latest computed step and the target prediction step.

Degeneration to the Integer-Order Baseline. The classical integer-order extrapolation is recovered as a special case when γ=1\gamma=1. Using the Gamma function identity

Γ(i+1)=i!,(31)\Gamma(i+1)=i!, \tag*{(31)}

Eq. (30) becomes

Fpred(xt−δl)∣γ=1=F(xtl)+∑i=1mΔiF(xtl)i!(−δ)i.(32)\left.\mathbf{F}_{\mathrm{pred}}(x_{t-\delta}^{l})\right|_{\gamma=1}=\mathbf{F}(x_t^{l})+\sum_{i=1}^{m}\frac{\Delta^i\mathbf{F}(x_t^{l})}{i!}(-\delta)^i. \tag*{(32)}

Therefore, the conventional integer-order polynomial forecasting rule can be viewed as the γ=1\gamma=1 instance of the more general FADE formulation, while γ≠1\gamma\ne1 provides a flexible coefficient law for modeling anomalous, stage-dependent feature dynamics.

Implementation Details of Stage-Aware Calibration

Model-Specific Calibration and Extreme Data Efficiency

A significant advantage of FADE is its extreme data efficiency during the offline calibration phase. We construct the calibration dataset Dcalib\mathcal{D}_{\mathrm{calib}} using only 10 randomly sampled conditions paired with standard Gaussian noise initializations. To ensure the calibration perfectly aligns with the native inference pipelines of different generation paradigms, the conditioning inputs are specifically configured as follows:

  • Class-Conditional Models (e.g., DiT-XL/2): Dcalib\mathcal{D}_{\mathrm{calib}} is constructed by randomly sampling 10 class labels from the ImageNet training label space, coupled with random initial noise seeds. The calibration seeds are kept disjoint from those used for final evaluation.

  • Text-to-Image Models (e.g., FLUX.1): The calibration conditions are formed by randomly selecting 10 diverse text prompts from the MS-COCO training split, paired with random initial seeds. These calibration prompts and seeds are not reused in the final DrawBench evaluation.

  • Text-to-Video Models (e.g., HunyuanVideo): For video generation, Dcalib\mathcal{D}_{\mathrm{calib}} is constructed using 10 randomly sampled prompts from T2V-CompBench, together with random initial noise seeds. These calibration prompts and seeds are kept disjoint from those used in the final VBench evaluation.

Despite the limited sample size (N=10N = 10) and the inherent stochasticity of diverse semantic conditions, the derived Look-Up Table (LUT) for γ^t,s\hat{\gamma}_{t,s} captures a stable profile of latent dynamics that generalizes effectively to unseen prompts during online inference.

Stage-Aware Calibration Procedure

Calibration follows the online compute–predict schedule using a closed-loop procedure, as summarized in Algorithm 1. We approximate the calibration objective in Eq. (7) through sample-wise directional grid searches followed by aggregation. At each prediction location (t,s)(t, s), candidate orders in G\mathcal{G} are evaluated using Eqs. (5) and (6). The prediction associated with the minimum directional loss is injected into the residual stream:

Ht,s←Ht,s+gate⁡t,s⊙F^t,s∗,(33)\mathbf{H}_{t,s} \leftarrow\mathbf{H}_{t,s} + \operatorname{gate}_{t,s} \odot\hat{\mathbf{F}}^{*}_{t,s}, \tag*{(33)}

where Ht,s\mathbf{H}_{t,s} is the current hidden residual state and gate⁡t,s\operatorname{gate}_{t,s} is the corresponding model gate. Subsequent calibration therefore accounts for recursively propagated prediction errors.

For each timestep–location pair (t,s)(t, s), the selected orders are collected across calibration samples and aggregated to obtain γt,s∗\gamma^{*}_{t,s}. The refined order γ^t,s=λγt,s∗\hat{\gamma}_{t,s} = \lambda\gamma^{*}_{t,s} is then stored in the LUT. The feature cache contains the zeroth-order feature Δ0Ftc,s=Ftc,s\Delta^{0}\mathbf{F}_{t_c,s} = \mathbf{F}_{t_c,s} and its finite differences, where tct_c denotes the latest full-compute timestep.

Dissecting Trajectory Dynamics: Implicit Regularization vs. Dynamic Rectification

To rigorously validate the efficacy of the optimal fractional orders found during the aforementioned offline calibration, we shift our focus to the long-horizon error propagation during online inference. By injecting the extrapolated features back into the recursive forward pass, we compare our FADE framework against the Ground-Truth full computation (GT-FI) and Standard Taylor extrapolation (ST-FI).

The Illusion of Implicit Regularization in Taylor Expansion

A critical observation in our closed-loop experiments is the highly non-linear nature of the diffusion trajectory. Due to the inherent architectural complexity of Diffusion Transformers (e.g., the global receptive field of self-attention mechanisms), the true latent manifold (GT-FI) exhibits dense, high-frequency structural transitions and abrupt numerical spikes across layers.

However, when employing the standard Taylor expansion (ST-FI) for caching, we observe a phenomenon we term Implicit Regularization. The strict integer-order assumption (γ=1.0\gamma= 1.0) acts as a rigid mathematical constraint, aggressively smoothing out these necessary trajectory spikes. It forces the feature evolution into a sub-optimal, overly-regularized path. While this may locally suppress extreme outliers, it fundamentally destroys the high-frequency semantics required for complex generation, inevitably leading to catastrophic directional divergence in later stages.

FADE: Dual-Stage Rectification under Recursive Error

In contrast, FADE avoids this rigid regularization by treating the fractional order as a dynamic, stage-aware degree of freedom. By explicitly simulating the recursive error accumulation during the offline calibration, FADE introduces a Dual-Stage Rectification mechanism to faithfully reconstruct the GT trajectory:

  • Directional Rectification via Fractional Order: Instead of suppressing the structural spikes, the optimized Mittag–Leffler parameters (γ∗\gamma^{*}) flexibly adapt to the local curvature

Algorithm 1 Stage-Aware Closed-Loop Offline Calibration for FADE

  1. Input: calibration dataset Dcalib\mathcal{D}_{\mathrm{calib}}, candidate grid G\mathcal{G}, compute-step schedule C\mathcal{C} including warm-up, maximum order mm, refinement factor λ\lambda

  2. Output: refined fractional-order table LUT[t,s]\mathrm{LUT}[t,s]

  3. Initialize an empty collection At,s\mathcal{A}_{t,s} for each target pair (t,s)(t,s)

  4. for each calibration sample q∈Dcalib\mathbf{q} \in\mathcal{D}_{\mathrm{calib}} do

  5. Initialize the diffusion latent and conditioning from q\mathbf{q}

  6. for each denoising timestep tt do

  7. if t∈Ct \in\mathcal{C} then

  8. Run the standard full forward pass to obtain model output yt\mathbf{y}_t

  9. Update feature histories and caches Hs←{ΔiFt,s}i=0m\mathcal{H}_s \leftarrow\{\Delta^i\mathbf{F}_{t,s}\}_{i=0}^{m} for all target locations ss

  10. tc←tt_c \leftarrow t

  11. else

  12. Compute the prediction distance δ\delta from tct_c to tt

  13. for each target location ss in forward execution order do

  14. Advance normal forward computations to location ss

  15. Obtain the reference feature FGT(t,s)\mathbf{F}_{\mathrm{GT}}(t,s) by full computation

  16. Evaluate all γ∈G\gamma\in\mathcal{G} using cache Hs\mathcal{H}_s, distance δ\delta, and Eq. (5)

  17. Select γt,s∗(q)\gamma^*_{t,s}(\mathbf{q}) minimizing Eq. (6) and retain its prediction F^t,s∗\hat{\mathbf{F}}^*_{t,s}

  18. Append γt,s∗(q)\gamma^*_{t,s}(\mathbf{q}) to At,s\mathcal{A}_{t,s}

  19. Ht,s←Ht,s+gatet,s⊙F^t,s∗\mathbf{H}_{t,s} \leftarrow\mathbf{H}_{t,s} + \mathrm{gate}_{t,s} \odot\hat{\mathbf{F}}^*_{t,s}

  20. end for

  21. Complete the remaining forward computations to obtain model output yt\mathbf{y}_t

  22. end if

  23. Advance the diffusion latent using yt\mathbf{y}_t and the native sampler

  24. end for

  25. end for

  26. for each pair (t,s)(t,s) with At,s≠∅\mathcal{A}_{t,s} \ne\varnothing do

  27. γt,s∗←Aggregate⁡(At,s)\gamma^*_{t,s} \leftarrow\operatorname{Aggregate}(\mathcal{A}_{t,s})

  28. LUT[t,s]←λγt,s∗\mathrm{LUT}[t,s] \leftarrow\lambda\gamma^*_{t,s}

  29. end for

  30. return LUT\mathrm{LUT}

of the manifold. By strictly monitoring the cosine similarity (Algorithm 1), FADE actively corrects the directional drift that plagues standard caching methods.

  • Intrinsic–Extrinsic Mismatch Correction: The stage-aware LUT provides state-specific fractional orders, but practical inference still uses a proxy distance instead of the intrinsic propagation scale. To account for this residual propagation mismatch, FADE applies a global refinement factor λ\lambda to obtain γ^t,s=λγt,s∗\hat{\gamma}_{t,s} = \lambda\gamma^*_{t,s}, while preserving the state-specific structure encoded in the LUT.

Through this decoupled rectification strategy, FADE is able to faithfully trace the abrupt fluctuations of the ground-truth manifold, completely bypassing the implicit regularization trap of Euclidean-based forecasting methods.

Empirical Validation of High Sample Consistency on DiT

To rigorously justify the use of a minuscule calibration set, we conducted an extensive empirical study focusing on the consistency of the optimal fractional order specifically within the DiT-XL/2 architecture. We computed the optimal γt,s∗\gamma^*_{t,s} across 10 independent random noise initializations on ImageNet-512.

As summarized in Table 6, the optimal fractional order at the stable denoising phase (t=40t = 40) exhibits a clear layer-dependent pattern. While the early and middle layers naturally display moderate variance to adapt to diverse initial noise inputs, the deep layers (Layers 19–27) maintain strong cross-sample consistency. Most importantly, empirical generation results confirm that the moderate standard deviation observed in the shallow stages has negligible impact on the final perceptual quality. The stable deeper layers effectively absorb these initial fluctuations, ensuring that a single offline calibration remains highly robust across any unseen prompts.

Network PhaseLayer IndicesAttention Module (Attn)
Early LayersLayers 0–9Mean γ∗\gamma^*0.640.1440.128
Middle LayersLayers 10–18Mean γ∗\gamma^*0.960.1290.119
Deep LayersLayers 19–27Mean γ∗\gamma^*0.460.1280.109
Global SummaryAll LayersMean γ∗\gamma^*0.680.1200.123

Table 6. Cross-sample consistency of the optimal fractional order γ∗\gamma^* in DiT attention layers. We highlight the statistics at the most stable denoising phase (Timestep t=40t = 40). The consistent behavior in deep layers and its robustness to early-stage variance formally validate our offline calibration strategy.

Error Propagation and Stability Analysis

Scope of the analysis. We provide a local and conditional error analysis for the regime where the local latent evolution is better described by an intrinsic fractional propagation law. The goal is not to prove a universal advantage of FADE over all possible predictors, but to explain why a state-adaptive fractional coefficient law can reduce the mismatch between the intrinsic feature trajectory and its practical approximation based on an extrinsic prediction scale.

Notation. Let tt denote the denoising timestep and ss denote the latent state index. We write

Ft,s:=F(xt;s),\mathbf{F}_{t,s} := \mathbf{F}(\mathbf{x}_t; s),

and let ΔiFt,s\Delta^i\mathbf{F}_{t,s} denote the ii-th order backward finite difference computed from cached historical features at state ss. For compactness, define the fractional coefficient

bi(γ):=1Γ(γi+1).(34)b_i(\gamma) := \frac{1}{\Gamma(\gamma i + 1)}. \tag*{(34)}

Here, δ\delta denotes the extrinsic prediction scale used in practical inference, while δM,t,s\delta_{M,t,s} denotes the intrinsic propagation scale along the local feature trajectory. The difference between these scales gives rise to a potential intrinsic–extrinsic propagation mismatch.

Assumption 1: intrinsic fractional local evolution. For each timestep–state pair (t,s)(t,s), assume that there exists a local intrinsic fractional order γt,s†>0\gamma^\dagger_{t,s} > 0 such that the ground-truth local evolution admits the following truncated intrinsic expansion:

Ft−δ,sGT=Ft,s+∑i=1mbi(γt,s†)ΔiFt,s(−δM,t,s)i+Rm+1(t,s),(35)\mathbf{F}^{\mathrm{GT}}_{t-\delta,s} = \mathbf{F}_{t,s} + \sum_{i=1}^{m} b_i(\gamma^\dagger_{t,s})\Delta^i\mathbf{F}_{t,s}(-\delta_{M,t,s})^i + \mathbf{R}^{(t,s)}_{m+1}, \tag*{(35)}

where the truncation remainder satisfies

∥Rm+1(t,s)∥≤Ct,sδM,t,sm+1.(36)\left\|\mathbf{R}^{(t,s)}_{m+1}\right\| \le C_{t,s}\delta_{M,t,s}^{m+1}. \tag*{(36)}

This assumption is local and conditional: it only states that the nearby feature evolution can be approximated by a truncated intrinsic ML-type expansion.

FADE predictor. FADE uses the extrinsic prediction scale δ\delta and the globally refined fractional order

γ^t,s=λγt,s∗,(37)\hat{\gamma}_{t,s} = \lambda\gamma^*_{t,s}, \tag*{(37)}

where γt,s∗\gamma^*_{t,s} is obtained from offline stage-aware calibration. The corresponding predictor is

F^t−δ,sFADE=Ft,s+∑i=1mbi(λγt,s∗)ΔiFt,s(−δ)i.(38)\widehat{\mathbf{F}}^{\mathrm{FADE}}_{t-\delta,s} = \mathbf{F}_{t,s} + \sum_{i=1}^{m} b_i(\lambda\gamma^*_{t,s})\Delta^i\mathbf{F}_{t,s}(-\delta)^i. \tag*{(38)}

Theorem 1: residual-aware FADE error bound. Under Assumption 1, the prediction error of FADE satisfies

∥Ft−δ,sGT−F^t−δ,sFADE∥≤CtδM,t,sm+1+∑i=1m∥ΔiFt,s∥∣(−δM,t,s)iΓ(γt,s†i+1)−(−δ)iΓ(λγt,s⋆i+1)∣.(39)\left\|\mathbf{F}^{\mathrm{GT}}_{t-\delta,s}-\widehat{\mathbf{F}}^{\mathrm{FADE}}_{t-\delta,s}\right\| \le C_t \delta_{\mathcal{M},t,s}^{m+1} +\sum_{i=1}^{m}\left\|\Delta^i\mathbf{F}_{t,s}\right\|\left|\frac{(-\delta_{\mathcal{M},t,s})^i}{\Gamma(\gamma^\dagger_{t,s}i+1)}-\frac{(-\delta)^i}{\Gamma(\lambda\gamma^\star_{t,s}i+1)}\right|. \tag*{(39)}

Proof. Subtracting (38) from (35) gives

Ft−δ,sGT−F^t−δ,sFADE=Rm+1(t,s)+∑i=1m[bi(γt,s†)(−δM,t,s)i−bi(λγt,s⋆)(−δ)i]ΔiFt,s.(40)\mathbf{F}^{\mathrm{GT}}_{t-\delta,s}-\widehat{\mathbf{F}}^{\mathrm{FADE}}_{t-\delta,s}=\mathbf{R}^{(t,s)}_{m+1} +\sum_{i=1}^{m}\left[b_i(\gamma^\dagger_{t,s})(-\delta_{\mathcal{M},t,s})^i-b_i(\lambda\gamma^\star_{t,s})(-\delta)^i\right]\Delta^i\mathbf{F}_{t,s}. \tag*{(40)}

Taking norms and applying the triangle inequality yields

∥Ft−δ,sGT−F^t−δ,sFADE∥≤∥Rm+1(t,s)∥+∑i=1m∥ΔiFt,s∥∣bi(γt,s†)(−δM,t,s)i−bi(λγt,s⋆)(−δ)i∣.(41)\left\|\mathbf{F}^{\mathrm{GT}}_{t-\delta,s}-\widehat{\mathbf{F}}^{\mathrm{FADE}}_{t-\delta,s}\right\| \le\left\|\mathbf{R}^{(t,s)}_{m+1}\right\| +\sum_{i=1}^{m}\left\|\Delta^i\mathbf{F}_{t,s}\right\|\left|b_i(\gamma^\dagger_{t,s})(-\delta_{\mathcal{M},t,s})^i-b_i(\lambda\gamma^\star_{t,s})(-\delta)^i\right|. \tag*{(41)}

Using (36) and the definition of bi(⋅)b_i(\cdot) gives (39). □

Decomposing the mismatch term. The second term in (39) contains both intrinsic–extrinsic scale mismatch and coefficient mismatch. Indeed, for each order ii, we have

∣bi(γt,s†)(−δM,t,s)i−bi(λγt,s⋆)(−δ)i∣≤∣bi(γt,s†)∣∣δM,t,si−δi∣⏟intrinsic–extrinsic scale mismatch+δi∣bi(γt,s†)−bi(λγt,s⋆)∣⏟fractional coefficient mismatch.(42)\left|b_i(\gamma^\dagger_{t,s})(-\delta_{\mathcal{M},t,s})^i-b_i(\lambda\gamma^\star_{t,s})(-\delta)^i\right| \le\underbrace{\left|b_i(\gamma^\dagger_{t,s})\right|\left|\delta_{\mathcal{M},t,s}^i-\delta^i\right|}_{\text{intrinsic–extrinsic scale mismatch}} +\underbrace{\delta^i\left|b_i(\gamma^\dagger_{t,s})-b_i(\lambda\gamma^\star_{t,s})\right|}_{\text{fractional coefficient mismatch}}. \tag*{(42)}

This shows explicitly that FADE can reduce the coefficient mismatch by calibrating γt,s⋆\gamma^\star_{t,s} and refining it with λ\lambda, while the remaining scale discrepancy arises from the difference between the extrinsic prediction scale δ\delta and the intrinsic propagation scale δM,t,s\delta_{\mathcal{M},t,s}.

Smoothness of the coefficient law. Assume that the relevant fractional orders lie in a compact interval

I=[γmin⁡,γmax⁡]⊂(0,∞).I=[\gamma_{\min},\gamma_{\max}]\subset(0,\infty).

Since bi(γ)=1/Γ(γi+1)b_i(\gamma)=1/\Gamma(\gamma i+1) is smooth on II, there exists a finite constant

Li:=sup⁡γ∈I∣ddγbi(γ)∣<∞,(43)L_i:=\sup_{\gamma\in I}\left|\frac{d}{d\gamma}b_i(\gamma)\right|<\infty, \tag*{(43)}

such that

∣bi(γ1)−bi(γ2)∣≤Li∣γ1−γ2∣,∀γ1,γ2∈I.(44)\left|b_i(\gamma_1)-b_i(\gamma_2)\right|\le L_i|\gamma_1-\gamma_2|,\qquad\forall\gamma_1,\gamma_2\in I. \tag*{(44)}

Therefore, (42) implies

∣bi(γt,s†)(−δM,t,s)i−bi(λγt,s⋆)(−δ)i∣≤∣bi(γt,s†)∣∣δM,t,si−δi∣+Li∣γt,s†−λγt,s⋆∣δi.(45)\left|b_i(\gamma^\dagger_{t,s})(-\delta_{\mathcal{M},t,s})^i-b_i(\lambda\gamma^\star_{t,s})(-\delta)^i\right| \le\left|b_i(\gamma^\dagger_{t,s})\right|\left|\delta_{\mathcal{M},t,s}^i-\delta^i\right|+L_i\left|\gamma^\dagger_{t,s}-\lambda\gamma^\star_{t,s}\right|\delta^i. \tag*{(45)}

This bound separates the error into an intrinsic–extrinsic scale mismatch and an order-induced coefficient mismatch.

Taylor predictor. The vanilla Taylor predictor corresponds to the special case γ=1\gamma= 1, since

bi(1)=1Γ(i+1)=1i!.b_i(1) = \frac{1}{\Gamma(i + 1)} = \frac{1}{i!}.

Its prediction form is

F^t−δ,sTaylor=Ft,s+∑i=1m1i!ΔiFt,s(−δ)i.(46)\widehat{\mathbf{F}}^{\mathrm{Taylor}}_{t-\delta,s} = \mathbf{F}_{t,s} + \sum_{i=1}^{m} \frac{1}{i!} \Delta^{i}\mathbf{F}_{t,s}(-\delta)^{i}. \tag*{(46)}

Following the same derivation as above, we obtain

∥Ft−δ,sGT−F^t−δ,sTaylor∥≤Ct,sδM,t,sm+1+∑i=1m∥ΔiFt,s∥∣(−δM,t,s)iΓ(γt,s†i+1)−(−δ)ii!∣.(47)\left\lVert\mathbf{F}^{\mathrm{GT}}_{t-\delta,s} - \widehat{\mathbf{F}}^{\mathrm{Taylor}}_{t-\delta,s} \right\rVert\le C_{t,s}\delta_{M,t,s}^{m+1} + \sum_{i=1}^{m} \left\lVert\Delta^{i}\mathbf{F}_{t,s} \right\rVert\left| \frac{(-\delta_{M,t,s})^{i}}{\Gamma(\gamma^{\dagger}_{t,s}i + 1)} - \frac{(-\delta)^{i}}{i!} \right|. \tag*{(47)}

Conditional advantage over Taylor. The truncation term in (39) and (47) is shared. Therefore, FADE has a tighter derived upper bound than Taylor whenever

∑i=1m∥ΔiFt,s∥∣(−δM,t,s)iΓ(γt,s†i+1)−(−δ)iΓ(λγt,s⋆i+1)∣<∑i=1m∥ΔiFt,s∥∣(−δM,t,s)iΓ(γt,s†i+1)−(−δ)ii!∣.(48)\sum_{i=1}^{m} \left\lVert\Delta^{i}\mathbf{F}_{t,s} \right\rVert\left| \frac{(-\delta_{M,t,s})^{i}}{\Gamma(\gamma^{\dagger}_{t,s}i + 1)} - \frac{(-\delta)^{i}}{\Gamma(\lambda\gamma^{\star}_{t,s}i + 1)} \right| < \sum_{i=1}^{m} \left\lVert\Delta^{i}\mathbf{F}_{t,s} \right\rVert\left| \frac{(-\delta_{M,t,s})^{i}}{\Gamma(\gamma^{\dagger}_{t,s}i + 1)} - \frac{(-\delta)^{i}}{i!} \right|. \tag*{(48)}

This condition states that the refined fractional coefficient law must approximate the intrinsic local evolution more closely than the rigid Taylor coefficient law, accounting for the difference between intrinsic and extrinsic propagation scales. It is a local and conditional statement, rather than a universal guarantee over arbitrary latent processes.

Role of the global refinement factor λ\lambda. The offline-calibrated fractional orders γt,s⋆\gamma^{\star}_{t,s} may still exhibit a global scale bias in practical inference. To account for this potential bias, we introduce a global refinement factor λ\lambda and use

γ^t,s=λγt,s⋆.\widehat{\gamma}_{t,s} = \lambda\gamma^{\star}_{t,s}.

This provides a lightweight global adjustment to the calibrated orders while preserving the state-specific structure captured by the offline LUT.

Summary of the local error analysis. Under the intrinsic fractional local evolution assumption, the prediction error is bounded by a truncation term and a mismatch term involving propagation scales and fractional coefficients. Taylor extrapolation corresponds to the γ=1\gamma= 1 case of the fractional coefficient family. FADE calibrates stage-adaptive orders γt,s⋆\gamma^{\star}_{t,s} and applies the refined orders γ^t,s=λγt,s⋆\widehat{\gamma}_{t,s} = \lambda\gamma^{\star}_{t,s} during inference. When its mismatch term is smaller than that of Taylor, FADE has a tighter derived upper bound. These designs provide a more flexible local approximation of the intrinsic dynamics, consistent with the improved robustness observed under large prediction distances.

Comprehensive Analysis of Latent Trajectory Dynamics

Empirical Analysis of Fractional Dynamics on DiT-XL/2

We analyze the optimal fractional orders γ⋆\gamma^{\star} across the 28 layers of DiT-XL/2 during a standard 50-step generation process. Statistics are averaged over 50 randomly sampled generation trajectories. Figure 6 shows the global distribution and spatiotemporal variation of the calibrated orders.

Global density distribution and spatiotemporal heatmap of optimal fractional orders

Figure 6. (a) Global Density Distribution of γ∗\gamma^{*} (b) Spatiotemporal Heatmap of γ∗\gamma^{*}. Spatiotemporal visualization of the optimal fractional orders (γ∗\gamma^{*}) on DiT-XL/2. (a) Global KDE distribution of γ∗\gamma^{*}. (b) Variation across denoising timesteps and transformer layers, showing a broadly shared U-shaped temporal pattern.

Global Distribution: Fractional Regimes Dominate. As shown in Figure 6(a), orders below 0.9 occur more frequently than those above 1.1, while the near-integer regime (γ≈1\gamma\approx1) accounts for approximately 13% of the analyzed states. Offline calibration therefore frequently selects orders that differ from the unit order used in standard Taylor extrapolation.

Temporal Heterogeneity: A U-Shaped Phase Pattern. Figure 6(b) reveals a U-shaped temporal pattern: lower orders are concentrated in the middle phase, while higher orders appear more frequently in the early and late phases. Spatial Consistency across Network Depth. At a given timestep, the calibrated orders remain relatively aligned across network layers. Together with the temporal variation, this motivates stage-aware selection of the forecasting order.

Empirical Analysis of Fractional Dynamics on FLUX.1

We extend the analysis to FLUX.1, which uses rectified flow matching and contains distinct DoubleStream and SingleStream blocks. For this analysis, we trace γ∗\gamma^{*} over 50 generation trajectories using text prompts randomly sampled from DrawBench. Figure 7 shows the calibrated orders across both block types.

Spatiotemporal dynamics of latent trajectory curvature in DoubleStream and SingleStream blocks

Figure 7. (a) DoubleStream Blocks (b) SingleStream Blocks. Spatiotemporal decoupling of latent dynamics across FLUX.1 architectures. Calibrated orders in (a) DoubleStream and (b) SingleStream blocks. SingleStream blocks exhibit a greater proportion of higher-order states. Across both streams, higher orders are concentrated in the middle stages of T0→T1T_{0} \to T_{1}.

Architectural Decoupling: A Shift toward Higher-Order Fractional Regimes. In the DoubleStream blocks (Figure 7(a)), the higher-order region (γ>1.1\gamma> 1.1) accounts for 56.8% of the states, while orders below 0.9 account for 32.4%. In the SingleStream blocks (Figure 7(b)), the higher-order proportion increases to 87.0%, while the near-integer regime (γ≈1\gamma\approx1) accounts for only approximately 0.5%.

Temporal Heterogeneity: An Inverted Phase Pattern. Along the timeline T0→T1T_{0} \to T_{1}, FLUX exhibits a temporal pattern opposite to the U-shaped trend observed in DiT. Relatively lower orders appear in the early and late phases, while higher orders are concentrated in the middle phase, as shown in Figure 7.

These observations motivate adapting the forecasting order across both architectural blocks and temporal stages.

Empirical Analysis of Fractional Dynamics on HunyuanVideo

We extend the analysis to HunyuanVideo by tracing the optimal fractional orders γ∗\gamma^* across its transformer blocks using diverse video text prompts. Figure 8 presents the distributions across its DoubleStream and SingleStream blocks.

Spatiotemporal Dynamics of Latent Trajectory Curvature for DoubleStream and SingleStream Blocks

Figure 8. Spatiotemporal decoupling of latent dynamics across HunyuanVideo architectures. Both (a) DoubleStream and (b) SingleStream blocks show a broad shift toward higher-order regimes (γ>1.1\gamma> 1.1), while near-integer states (γ≈1\gamma\approx1) are comparatively uncommon.

A Broad Shift toward Super-Diffusion-Like Regimes. Figure 8 shows a broad shift toward higher-order regimes (γ>1.1\gamma> 1.1) across both block types. This trend resembles the behavior observed in the SingleStream blocks of FLUX.1 (Section D.2).

Limitations of a Uniform Integer-Order Assumption. The near-integer regime (γ≈1\gamma\approx1) accounts for only a small fraction of the calibrated states, while orders below 0.9 are also less prominent than in the image DiT analyzed above. These distributions show that calibration frequently selects orders beyond the fixed γ=1\gamma= 1 baseline.

Together with the results on DiT and FLUX, these observations motivate adapting the forecasting order across denoising stages and model architectures.

Additional Ablation Study

Ablation on Fractional-Order Scheduling

We further study different fractional-order scheduling strategies on FLUX.1 under the N=6N = 6 setting. This ablation is designed to examine whether a manually specified fixed order or a simple time-dependent fitted schedule can reliably replace stage-aware calibration.

As shown in Table 7, different fractional-order schedules favor different evaluation metrics. A fixed order of γ=1.6\gamma= 1.6 achieves the best ImageReward, while the dynamic fitted schedule obtains the highest CLIP score. This suggests that time-dependent order variation can better align with text-image semantic consistency, but does not uniformly dominate fixed-order schedules across all metrics. These results indicate that fractional-order scheduling is sensitive to both the metric and denoising regime, further motivating stage-aware calibration instead of relying on a single manually specified order.

MethodSpeed↑\uparrowIR↑\uparrowCLIP↑\uparrow
FADE (fixed γ=1.2\gamma= 1.2)4.99×\times0.988327.5061
FADE (fixed γ=1.6\gamma= 1.6)4.99×\times1.001627.6207
FADE (fixed γ=1.8\gamma= 1.8)4.99×\times0.997827.6239
FADE (fixed γ=2.0\gamma= 2.0)4.99×\times0.993627.6617
FADE (dynamic fit γ(t)\gamma(t))4.99×\times0.994827.7077

Table 7. Additional ablation on fractional-order scheduling strategies for FLUX.1.

We also evaluate simple fractional-order scheduling strategies on DiT-XL/2. This study compares the original integer-order Taylor baseline with a manually tuned fixed fractional order and a fitted time-dependent schedule.

As shown in Table 8, both the fixed fractional order and the fitted time-dependent schedule improve over the integer-order Taylor baseline in terms of FID. However, the fitted schedule does not consistently outperform the manually tuned fixed order, suggesting that simple parametric schedules can be sensitive to the chosen functional form. Together with the FLUX.1 results, this supports the use of stage-aware calibration, which avoids imposing a single fixed or manually fitted γ\gamma schedule.

MethodFID↓\downarrowsFID↓\downarrow
DDIM-502.294.29
HiCache2.515.20
Taylor2.5905.18
FADE (fixed γ=0.85\gamma= 0.85)2.494.81
FADE (Bernoulli-fit γ(t)\gamma(t))2.515.14

Table 8. Additional ablation on fractional-order scheduling strategies for DiT-XL/2. “–” indicates that the metric was not recorded.

Detailed Experimental Protocols

As mentioned in the main text, experiments are conducted on three representative diffusion transformer models across different generative tasks: FLUX.1-dev Labs [ [17] for text-to-image generation, HunyuanVideo Kong et al. [ [15] for text-to-video generation, and DiT-XL/2 Peebles and Xie [ [33] for class-conditional image generation. This section provides detailed experimental configurations, including hardware infrastructure, sampling protocols, FADE-specific calibration settings, and baseline hyperparameters.

Hardware and Software Infrastructure

Hardware platforms are selected according to the memory and computation requirements of each model. Specifically, DiT-XL/2 is evaluated on 32GB NVIDIA vGPUs, FLUX.1-dev on NVIDIA H800 GPUs, and HunyuanVideo on NVIDIA A800 GPUs. All models follow their official inference pipelines and default precision settings. Classifier-free guidance (CFG) scales, schedulers, input resolutions, and prompt settings are kept consistent with the standard configurations of the corresponding models.

Model Configurations and Sampling Parameters

Across all evaluated models, official pretrained checkpoints and inference pipelines are strictly followed. Unless otherwise specified, all methods are evaluated under the same number of inference steps, sampling schedules, random seeds, and acceleration ratios for fair comparison.

  • DiT-XL/2 (ImageNet-1K): The pretrained DiT-XL/2 model is used for class-conditional image generation at a spatial resolution of 256 ×\times 256. The inference process is driven by the DDIM sampler with a total of 50 denoising steps.

  • FLUX.1-dev (DrawBench): The rectified-flow-based FLUX.1-dev model is evaluated for high-fidelity text-to-image generation. Sampling is performed using the default Flow Match Euler scheduler at a resolution of 1024 ×\times 1024, with 50 inference steps.

  • HunyuanVideo (VBench): Text-to-video generation experiments are conducted using the official HunyuanVideo inference pipeline. The default flow-matching sampler is adopted

with 50 denoising steps, and all video generation settings follow the official configuration unless otherwise stated.

FADE and Baseline Configurations

A detailed configuration scheme for the proposed FADE method and the reproduced baselines is provided below.

  • FADE (Ours): FADE is implemented as a training-free feature forecasting framework. For each model, we first run an offline calibration stage to profile the optimal fractional order γ∗\gamma^* for different denoising stages and network locations. The calibration objective is defined as a directional trajectory matching loss based on cosine distance between the FADE-predicted feature and the corresponding full-computation feature. This choice avoids over-penalizing magnitude variations caused by normalization layers and focuses on the angular consistency of latent feature trajectories.

    The calibrated fractional orders are stored in a static lookup table (LUT), which is indexed by model-specific structural states. For DiT-XL/2, the LUT is indexed by timestep, layer, and module. For FLUX.1-dev, the LUT additionally considers the stream and block structure of the model. For HunyuanVideo, we adopt a coarser LUT indexing strategy due to the substantially higher computational cost and inherent spatiotemporal redundancy of video generation. The LUT is therefore constructed based on timestep and coarse video-transformer block indices. During online inference, FADE directly retrieves the corresponding γ\gamma from the LUT and performs fractional-order feature extrapolation without any additional optimization or adaptive search.

  • Baselines: We compare FADE with representative training-free diffusion acceleration methods, including Selvaraju et al. [ [37], Feng et al. [ [7], and Liu et al. [ [22]. All reproduced baselines strictly follow the implementation protocols and default hyperparameter settings reported in their original papers and open-source repositories. For a fair comparison, all baselines are evaluated using the same base model, sampler, number of inference steps, resolution, CFG scale, prompt set, and acceleration schedule as FADE.

Offline calibration protocol. For each calibration trajectory, a full-computation reference pass collects the oracle features Ft,sGT\mathbf{F}^{\mathrm{GT}}_{t,s} at prediction locations. Given the cached feature basis {Dt,sk}k=0K−1\{\mathbf{D}^{k}_{t,s}\}_{k=0}^{K-1} and prediction distance xx, each candidate order γ\gamma is evaluated by

F^t,s(γ)=∑k=0K−11Γ(γk+1)Dt,skxk,(49)\widehat{\mathbf{F}}_{t,s}(\gamma)=\sum_{k=0}^{K-1}\frac{1}{\Gamma(\gamma k+1)}\mathbf{D}^{k}_{t,s}x^k, \tag*{(49)}

where KK is the number of retained basis terms and the zeroth-order coefficient is 1. The candidate minimizing the directional error is selected:

γt,s∗=arg⁡min⁡γ∈G[1−cos⁡(F^t,s(γ),Ft,sGT)].(50)\gamma^*_{t,s}=\underset{\gamma\in\mathcal{G}}{\arg\min}\left[1-\cos\left(\widehat{\mathbf{F}}_{t,s}(\gamma),\mathbf{F}^{\mathrm{GT}}_{t,s}\right)\right]. \tag*{(50)}

The selected predictions are propagated using the closed-loop procedure in Algorithm 1.

For DiT-XL/2, ss indexes the layer and module, and we use G={0.1,0.2,…,3.0}\mathcal{G}=\{0.1,0.2,\ldots,3.0\} with all available cached orders. For FLUX.1-dev, ss additionally specifies the stream and block, and we use G={0.1,0.2,…,5.0}\mathcal{G}=\{0.1,0.2,\ldots,5.0\} with a two-term cached basis for numerical stability. HunyuanVideo follows the FLUX.1-dev protocol with a coarser search and LUT indexed by timestep and video-transformer block. The calibrated orders are stored in a static LUT for inference.

Online inference settings. For each model and cache interval NN, we tune λ\lambda offline and compute the refined orders:

γ^t,s=λγt,s∗.(51)\widehat{\gamma}_{t,s}=\lambda\gamma^*_{t,s}. \tag*{(51)}

The refined orders are stored in the LUT and retrieved directly during inference. We keep λ\lambda fixed across all evaluation prompts. The settings are summarized in Table 9.

ModelCache interval NNRefinement factor λ\lambda
DiT-XL/240.6
DiT-XL/250.6
DiT-XL/260.7
FLUX.1-dev51.3
FLUX.1-dev61.0
FLUX.1-dev71.1
HunyuanVideodefault1.0

Table 9. Online inference settings of FADE. The refinement factor λ\lambda is fixed after offline calibration and introduces no online search overhead.

References

  1. [1]Hyojin Bahng, Caroline Chan, Fredo Durand, and Phillip Isola. Cycle consistency as reward: Learning image-text alignment without human preferences. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 22934–22946, October 2025.arxiv.org/abs/2506.02095
  2. [2]Daniel Bolya and Judy Hoffman. Token merging for fast stable diffusion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4599–4603, 2023.arxiv.org/abs/2303.17604
  3. [3]Qi Cai, Jingwen Chen, Yang Chen, Yehao Li, Fuchen Long, Yingwei Pan, Zhaofan Qiu, Yiheng Zhang, Fengbin Gao, Peihan Xu, Yimeng Wang, Kai Yu, Wenxuan Chen, Ziwei Feng, Zijian Gong, Jianzhuang Pan, Yi Peng, Rui Tian, Siyu Wang, Bo Zhao, Ting Yao, and Tao Mei. Hidream-ii: A high-end efficient image generative foundation model with sparse diffusion transformer, 2025. URL https://arxiv.org/abs/2505.22705.
  4. [4]Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-α: Fast training of diffusion transformer for photorealistic text-to-image synthesis, 2023. URL https://arxiv.org/abs/2310.00426.
  5. [5]Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009. doi: 10.1109/CVPR.2009.5206848.DOI
  6. [6]Gongfan Fang, Xinyin Ma, and Xinchao Wang. Structural pruning for diffusion models, 2023. URL https://arxiv.org/abs/2305.10924.
  7. [7]Liang Feng, Shikang Zheng, Jiacheng Liu, Yuqi Lin, Qinming Zhou, Peiliang Cai, Xinyu Wang, Junjie Chen, Chang Zhou, Yue Ma, and Linfeng Zhang. Hicache: A plug-in scaled-hermite upgrade for taylor-style cache-then-forecast diffusion acceleration. In International Conference on Learning Representations (ICLR), 2026.arxiv.org/abs/2508.16984
  8. [8]Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning, 2022. URL https://arxiv.org/abs/2104.08718.
  9. [9]Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium, 2018. URL https://arxiv.org/abs/1706.08500.
  10. [10]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 6840–6851. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/file/4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf.arxiv.org/abs/2006.11239
  11. [11]Alain Horé and Djemel Ziou. Image quality metrics: Psnr vs. ssim. In 2010 20th International Conference on Pattern Recognition, pages 2366–2369, 2010. doi: 10.1109/ICPR.2010.579.DOI
  12. [12]Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21807–21818, 2024.arxiv.org/abs/2311.17982
  13. [13]Kumara Kahatapitiya, Haozhe Liu, Sen He, Ding Liu, Menglin Jia, Chenyang Zhang, Michael S Ryoo, and Tian Xie. Adaptive caching for faster video generation with diffusion transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15240–15252, 2025.arxiv.org/abs/2411.02397
  14. [14]Dimitris Kalatzis, David Eklund, Georgios Arvanitidis, and Søren Hauberg. Variational autoencoders with riemannian brownian motion priors. arXiv preprint arXiv:2002.05227, 2020.arxiv.org/abs/2002.05227
  15. [15]Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024.arxiv.org/abs/2412.03603
  16. [16]Zhifeng Kong, Wei Ping, Jiaj i Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis, 2021. URL https://arxiv.org/abs/2009.09761.
  17. [17]Black Forest Labs. Flux. https://github.com/black-forest-labs/flux, 2024.
  18. [18]Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, Sumith Kulal, Kyle Lacey, Yam Levi, Cheng Li, Dominik Lorenz, Jonas Müller, Dustin Podell, Robin Rombach, Harry Saini, Axel Sauer, and Luke Smith. Flux.1 Kontext: Flow matching for in-context image generation and editing in latent space, 2025. URL https://arxiv.org/abs/2506.15742.
  19. [19]Xiuyu Li, Yijiang Liu, Long Lian, Huanrui Yang, Zhen Dong, Daniel Kang, Shanghang Zhang, and Kurt Keutzer. Q-diffusion: Quantizing diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17535–17545, 2023.arxiv.org/abs/2302.04304
  20. [20]Feng Liu, Shiwei Zhang, Xiaofeng Wang, Yujie Wei, Haonan Qiu, Yuzhong Zhao, Yingya Zhang, Qixiang Ye, and Fang Wan. Timestep embedding tells: It’s time to cache for video diffusion model. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 7353–7363, 2025a.
  21. [21]Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley. AudioIdm: Text-to-audio generation with latent diffusion models. arXiv preprint arXiv:2301.12503, 2023.
  22. [22]Jiacheng Liu, Chang Zou, Yuanhuiyi Lyu, Junjie Chen, and Linfeng Zhang. From reusing to forecasting: Accelerating diffusion models with taylors eers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15853–15863, 2025b.
  23. [23]Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow, 2022. URL https://arxiv.org/abs/2209.03003.
  24. [24]Alexander Lobashev, Dmitry Guskov, Maria Larchenko, and Mikhail Tamm. Hessian geometry of latent space in generative models. arXiv preprint arXiv:2506.10632, 2025.arxiv.org/abs/2506.10632
  25. [25]Jinming Lou, Wenyang Luo, Yufan Liu, Bing Li, Xinmiao Ding, Weiming Hu, Yuming Li, and Chengguang Ma. Token caching for diffusion transformer acceleration. arXiv preprint arXiv:2409.18523, 2024.arxiv.org/abs/2409.18523
  26. [26]Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in neural information processing systems, 35:5775–5787, 2022.
  27. [27]Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. Machine Intelligence Research, 22(4):730–751, June 2025. ISSN 2731-5398. doi: 10.1007/s11633-025-1562-4. URL http://dx.doi.org/10.1007/s11633-025-1562-4.
  28. [28]Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference, 2023. URL https://arxiv.org/abs/2310.04378.
  29. [29]Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden, and Saining Xie. SiT: Exploring flow and diffusion-based generative models with scalable interpolant transformers. In European Conference on Computer Vision, pages 23–40. Springer, 2024a.
  30. [30]Xinyin Ma, Gongfan Fang, and Xinchao Wang. Deepcache: Accelerating diffusion models for free. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15762–15772, 2024b.arxiv.org/abs/2312.00858
  31. [31]Ralf Metzler and Joseph Klafter. The random walk’s guide to anomalous diffusion: a fractional dynamics approach. Physics Reports, 339(1):1–77, 2000. ISSN 0370-1573. doi: https://doi.org/10.1016/S0370-1573(00)00070-3. URL https://www.sciencedirect.com/science/article/pii/S0370157300000703.
  32. [32]Fernando A Oliveira, Rogelma MS Ferreira, Luciano C Lapas, and Mendeli H Vainstein. Anomalous diffusion: A basic mechanism for the evolution of inhomogeneous systems. Frontiers in Physics, 7:18, 2019.DOI
  33. [33]William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023.arxiv.org/abs/2212.09748
  34. [34]Xurui Peng, Chenqian Yan, Hong Liu, Rui Ma, Fangmin Chen, Xing Wang, Zhihua Wu, Songwei Liu, and Mingbao Lin. Ertacache: Error rectification and timesteps adjustment for efficient diffusion. arXiv preprint arXiv:2508.21091, 2025.arxiv.org/abs/2508.21091
  35. [35]Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding, 2022. URL https://arxiv.org/abs/2205.11487.
  36. [36]Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models, 2022. URL https://arxiv.org/abs/2202.00512.
  37. [37]Pratheba Selvaraju, Tianyu Ding, Tianyi Chen, Ilya Zharkov, and Luming Liang. Fora: Fast-forward caching in diffusion transformer acceleration, 2024. URL https://arxiv.org/abs/2407.01425.
  38. [38]Yuzhang Shang, Zhihang Yuan, Bin Xie, Bingzhe Wu, and Yan Yan. Post-training quantization on diffusion models. In CVPR, 2023.arxiv.org/abs/2211.15736
  39. [39]Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models, 2022. URL https://arxiv.org/abs/2010.02502.
  40. [40]Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models, 2023. URL https://arxiv.org/abs/2303.01469.
  41. [41]Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Zhang, Siyang Sun, Tao Fang, Tianxing Wang, Tianyi Gui, Tingyu Weng, Tong Shen, Wei Lin, Wei Wang, Wei Wang, Wenmeng Zhou, Wente Wang, Wenting Shen, Wenyuan Yu, Xianzhong Shi, Xiaoming Huang, Xin Xu, Yan Kou, Yangyu Lv, Yifei Li, Yijing Liu, Yiming Wang, Yingya Zhang, Yitong Huang, Yong Li, You Wu, Yu Liu, Yulin Pan, Yun Zheng, Yuntao Hong, Yupeng Shi, Yutong Feng, Zeyinzi Jiang, Ze Han, Zhi-Fan Wu, and Ziyu Liu. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025.arxiv.org/abs/2503.20314
  42. [42]Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imageward: learning and evaluating human preferences for text-to-image generation. In Proceedings of the 37th International Conference on Neural Information Processing Systems, pages 15903–15935, 2023.arxiv.org/abs/2304.05977
  43. [43]Jinlong Yang, Jinke Wu, Lizilin, and Yao Zhou. Brace: Taming sharp irregularities via barycentric rational forecasting for fast diffusion transformers inference, 2026. URL https://arxiv.org/abs/2608.07572.
  44. [44]Lianwei Yang, Haokun Lin, Tianchen Zhao, Yichen Wu, Hongyu Zhu, Ruiqi Xie, Zhenan Sun, Yu Wang, and Qingyi Gu. Lrq-dit: Log-rotation post-training quantization of diffusion transformers for image and video generation. arXiv preprint arXiv:2508.03485, 2025.arxiv.org/abs/2508.03485

Paper details

vlnc:W_edxmzjkn
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents