betaFeedback

WorksComputer Science

Finite-Step Geometry of Marginal-Preserving Velocity Fields in Generative Flows

Abstract

Generative flows can admit different velocity fields that induce the same continuous marginal evolution but behave differently after numerical discretization. We study this finite-step non-equivalence while keeping the marginal path and numerical method fixed. Projected modified equations characterize the component of local integration error that changes the marginal law and give sufficient conditions for higher marginal convergence order. For affine fixed-shape distributions, we characterize exact finite-step reachability and derive spectral obstructions and sharp nearest-reachable Wasserstein errors. For diffusion probability-flow ODEs, we construct skew-score perturbations that preserve the exact marginal path under exact scores and quantify their leakage under score error. Controlled Gaussian and mixture experiments recover the predicted order, reachability, and obstruction phenomena.

Why the Velocity Field Still Matters

A generative flow turns a simple random input into a sample by following a velocity field. Diffusion probability-flow ODEs and flow-matching models implement this idea with a neural network [35, 3] (Song et al. 2021):

X˙t=vt(Xt),X0∼ρ0.\dot{X}_t = v_t(X_t), \qquad X_0 \sim\rho_0.

Here ρt\rho_t denotes the distribution of particles at time tt. Sampling requires a numerical approximation to this motion, and each velocity evaluation usually costs one network call. With only a few calls, the numerical distribution can differ substantially from the intended one.

Can we improve that approximation by changing the velocity without changing the intended continuous distributions? A distribution records probability mass, not particle identities. Different motions can therefore produce the same continuous distributions but different numerical outputs.

A simple example: expansion with or without rotation. Imagine a two-dimensional Gaussian cloud whose radius grows exponentially. Let β>0\beta> 0 set its expansion rate, and let JJ rotate a vector by a quarter turn. The fields

ρt=N(0,e2βtI2),(1)\rho_t = \mathcal{N}\left(0,e^{2\beta t}I_2\right), \tag*{(1)}
wω(x)=(βI+ωJ)x,J=(0−110)(2)w_\omega(x) = (\beta I+\omega J)x,\qquad J=\begin{pmatrix}0 & -1 \\ 1 & 0\end{pmatrix} \tag*{(2)}

all produce this Gaussian path: the exact motion expands by eβte^{\beta t} and rotates by ωt\omega t, leaving the circular distributions unchanged.

Now take one Euler step of length hh. Without rotation, its radial multiplier is 1+βh1+\beta h, smaller than the exact multiplier eβhe^{\beta h}. With rotation, the covariance multiplier becomes

(1+βh)2+ω2h2.(3)(1+\beta h)^2+\omega^2h^2. \tag*{(3)}

Choosing

ωh2=e2βh−(1+βh)2h2(4)\omega_h^2=\frac{e^{2\beta h}-(1+\beta h)^2}{h^2} \tag*{(4)}

therefore makes the numerical distribution exact. This choice is real for β,h>0\beta,h>0. On a uniform grid it matches every grid marginal, although numerical and exact particle endpoints still differ.

Euler follows a straight segment rather than the curved motion. The sideways displacement from rotation lengthens that segment and supplies the missing expansion (Fig. 1). The straightest representative, which also has the least kinetic energy in this family, is therefore not the best representative for this numerical method.

What we hold fixed. The primary comparison keeps the continuous distribution path, solver, timesteps and network-call budget fixed. Other approaches change the learned transport, the update formula or the schedule ([36, 27, 37] (Sabour et al. 2024; Shaul et al. 2024)). Selecting the correction has an offline cost beyond sampling calls; Section G.2 and Table 8 specify what is recorded and what remains unmeasured. Tasks that require a particular initial-to-final pairing, such as a prescribed inversion map, impose an additional constraint; improving a sampled distribution alone does not solve them.

ComponentRecorded scope and limitation
Coarse samplingActual network calls are reported with each solver comparison. Worker generation time excludes fitting, feature extraction, I/O and scheduling.
Offline selectionCalibration, PCA, fitting, dense references and six validation candidates are additional costs. No controlled end-to-end deployment or break-even measurement is available.
Diagnostic timingElapsed local-shard hours include feature calls/I/O; calibration-cloud wall seconds and sampling worker-hours are distinct and cannot be added.

Table 8. Cost accounting and its limits. Detailed recorded timings are in the code supplement; heterogeneous execution timings do not establish latency gains or amortization.

Illustration of straight, rotational, and angular-to-radial representatives

Figure 1. Rotation can repair an under-expanding Euler step. In this exact two-dimensional Gaussian example (βh=0.7\beta h = 0.7), the radial step falls short (left). Adding rotation preserves the continuous distribution and gives Euler the correct endpoint radius (middle), while individual particle endpoints remain different. The line–circle intersection on the right gives the required rotation. Circles are Gaussian density contours, not a bounded support. These are analytic illustrations, not benchmark measurements.

Contributions. We answer three questions about a fixed distribution path.

  1. Marginal error and order. We project modified-equation errors onto the component that changes the law. Cancelling successive components gives sufficient conditions for higher marginal convergence order, including quantitative tolerances for approximate cancellation.

  1. Exactness and unavoidable error. For affine fixed-shape laws, we characterize finite-step reachability and its spectral obstructions. Rank, operator-norm and energy budgets give sharp nearest-reachable Wasserstein errors; the selected representative depends on the solver and grid.

  1. Constructions and empirical mechanisms. Skew-score perturbations preserve exact diffusion marginals, with a separate leakage bound for score error. Controlled and pretrained-model tests examine solver transfer, native window fitting, local response and integration against each field’s own dense flow. An exact Gaussian calibration analysis separates minibatch and penalty effects from these measured neural outcomes.

Notation and detailed attribution appear in Table 2 and Section H.

SymbolsMeaning
ρt\rho_t, ρ~t\tilde{\rho}_t, ρtB\rho^{\mathrm{B}}_t, ρtG\rho^{\mathrm{G}}_tPrescribed density, comparison-flow density, and actual learned Base/Gauge densities.
v,u,w=v+uv,u,w = v+uBase velocity, admissible correction, and selected representative.
Πρ\Pi_\rho, Ψhw\Psi^w_h, Rt,h\mathcal{R}_{t,h}, Mt,h\mathcal{M}_{t,h}Weighted gradient projection, numerical step map, response law, and reachable set.
mtm_t, Σt\Sigma_t, Σ⋆\Sigma_\starCenter (mean when integrable), scatter (covariance when normalized), and whitened target scatter.
ZZ, PPCAP_{\mathrm{PCA}}Spherical/Gaussian random input and fixed calibration PCA basis.
BB, B^=L−1BL\widehat{B}=L^{-1}BL, SS, KK, JJAffine generator, whitened generator, symmetric strain, skew control, and orthogonal complex structure.
HΣH_\Sigma, GΣG_\Sigma, ToptT^{\mathrm{opt}}Whitened covariance tangent/curvature and positive optimal-transport map.
AσA_\sigma, MskM_{\mathrm{sk}}, LresσL^\sigma_\mathrm{res}, KtR​​FK^{\mathrm{R​​F}}_tDimensionless EDM skew operator, score mobility, residual factor Aσ/σA_\sigma/\sigma, and independent-path RF skew operator.
UfitU_{\mathrm{fit}}, VfitV_{\mathrm{fit}}, bσb_\sigmaLatent fit factors and their noise-dependent profile coefficients.
CcalC_{\mathrm{cal}}, BpmethB_{p_{\mathrm{meth}}}Finite empirical calibration objective and population projected modified-error objective.
pmethp_{\mathrm{meth}}, plinp_{\mathrm{lin}}, qq, ℓmod\ell_{\mathrm{mod}}, EjE_j, QhQ_hClassical method order, scalar linear order, first active marginal index, expansion cutoff, modified coefficients and remainder.
rr, ϱ\varrho, υ\upsilon, ℓrad\ell_{\mathrm{rad}}, ℓjet\ell_{\mathrm{jet}}Rank-pair budget, radial coordinate, source quantile, radial power and jet degree.
R(z)R(z), NN, hh, σ\sigma, κEDM\kappa_{\mathrm{EDM}}Stability function, step count, signed step, noise level and EDM grid exponent (7).
s=∇log⁡ρs=\nabla\log\rho, ς\varsigma, ff, ggScore, isotropic scale, and SDE drift/diffusion coefficient.
DθD_\theta, DdiffD_{\mathrm{diff}}, Ξ\XiDenoiser, diffusion covariance, and antisymmetric current potential.
VρV_\rho, EuE_u, CpostC_{\mathrm{post}}, χm\chi_mState variance, gauge energy, posterior velocity covariance and mixture responsibilities.
DjD_j, IFI_F, GGramG_{\mathrm{Gram}}Linearized projected residual operator, dictionary right inverse and gradient Gram matrix.
ξmix\xi_{\mathrm{mix}}, ζexp\zeta_{\mathrm{exp}}Mixing and expansion coordinates in Figure 2; neither is the mean.

Table 2. Recurring notation. Definitions and assumptions remain in the indicated text; the table is a reading aid.

What It Means to Keep the Distribution Fixed

The Gaussian rotation illustrates a general constraint. The continuity equation describes how a velocity redistributes probability mass:

∂tρt+div⁡(ρtvt)=0.(5)\partial_t \rho_t + \operatorname{div}(\rho_t v_t) = 0. \tag*{(5)}

Adding utu_t leaves this equation unchanged exactly when

div⁡(ρtut)=0.(6)\operatorname{div}(\rho_t u_t) = 0. \tag*{(6)}

The quantity ρtut\rho_t u_t is a probability current; zero divergence means that this additional motion creates no net local gain or loss of mass. We call it a marginal gauge. Under the regularity and transport-uniqueness assumptions stated in Section A, v+uv+u then produces the same continuous distribution as vv. The constraint must hold throughout the interval, not just at its first node. A correction need not move along density contours to satisfy it.

The numerical response is simply the output distribution. Start a step with particles distributed as ρt\rho_t, apply the chosen numerical map Ψt,hv+u\Psi_{t,h}^{v+u}, and inspect their new distribution:

Rt,h(u)=(Ψt,hv+u)#ρt.(7)\mathcal{R}_{t,h}(u) = \left(\Psi_{t,h}^{v+u}\right)_{\#}\rho_t. \tag*{(7)}

The symbol #ρt\#\rho_t means the distribution obtained by applying the map to a draw from ρt\rho_t. Exact integration always returns the prescribed ρt+h\rho_{t+h} for an admissible correction. One numerical step need not. A low-rank restriction or an amplitude cap limits which responses are available.

We use this single output map in three ways. At small hh, we ask what its leading distributional error is. At a fixed hh, we ask whether any allowed correction reaches ρt+h\rho_{t+h} exactly. If it cannot, we minimize the distance to that target. We use W2W_2, the Wasserstein distance that measures the least root-mean-square displacement needed to transport one distribution to another. The formal reachable-set and minimization definitions are in Section A.1.

Why local success is not enough. Later steps can amplify local error. Under Lipschitz stability, the final transport error is bounded by local errors weighted by their subsequent amplification (Section A.2). This motivates separate local, own-field integration and image-quality checks.

Which Integration Errors Change the Law?

For general smooth fields, we test whether the omitted motion changes probability mass.

Euler misses acceleration, but not all acceleration matters. For w=v+uw = v + u, the exact trajectory begins as x+hw+h2aw/2+O(h3)x + hw + h^2a_w/2 + O(h^3), where

aw=∂tw+(∇w)w(8)a_w = \partial_t w + (\nabla w)w \tag*{(8)}

is the acceleration along the trajectory. Euler keeps only x+hwx + hw. Nevertheless, its leading distributional error depends only on the weighted divergence of the omitted acceleration:

(Ψh,wE)#ρt−ρt+h=h22div⁡(ρtaw)+O(h3).(9)(\Psi_{h,w}^{\mathrm{E}})\#\rho_t - \rho_{t+h} = \frac{h^2}{2}\operatorname{div}(\rho_t a_w) + O(h^3). \tag*{(9)}

This expansion is weak: it compares expectations of smooth test functions. If div⁡(ρtaw)=0\operatorname{div}(\rho_t a_w) = 0, the leading particle error merely rearranges mass and the leading density error disappears.

We can separate any field into a component that changes the distribution and an orthogonal component that only rearranges it. Write Πρ\Pi_\rho for the projection onto the former, the closure of gradient fields in L2(ρ)L^2(\rho). For Euler the relevant component is

Πρtaw.(10)\Pi_{\rho_t}a_w. \tag*{(10)}

It can vanish even when the full acceleration is nonzero. The weighted projection also supplies the local Wasserstein error coefficient [7, 5]; its construction and norm identity appear in Section A.

The same idea applies beyond Euler. A modified equation represents a numerical method as the exact flow of a nearby velocity, expanded in powers of hh [19]. Its successive coefficients Epmeth,Epmeth+1,…E_{p_{\mathrm{meth}}}, E_{p_{\mathrm{meth}}+1}, \ldots describe errors introduced by a method of classical order pmethp_{\mathrm{meth}}. We test each coefficient with the same mass-conservation condition as in Eq. (6).

References

  1. [1]Assyr Abdulle, Gilles Vilmart, and Konstantinos C. Zyg alakis. High order numerical approximation of the invariant measure of ergodic SDEs. SIAM Journal on Numerical Analysis, 52(4):1600–1622, 2014. doi: 10.1137/130935616.DOI
  2. [2]J. F. Adams. Vector fields on spheres. Annals of Mathematics, 75(3):603–632, 1962. doi: 10.2307/1970213.DOI
  3. [3]Michael S. Albergo, Nicolas M. Boffi, and Eric Vanden-Eijnden. Stochastic interpolants: A unifying framework for flows and diffusion. Journal of Machine Learning Research, 26(209):1–80, 2025. URL https://jmlr.org/papers/v26/23-1605.html.
  4. [4]Ethem Alpaydin and Cenk Kaynak. Optical recognition of handwritten digits. UCI Machine Learning Repository, 1998.
  5. [5]Luigi Ambrosio, Nicola Gigli, and Giuseppe Savaré. Gradient Flows in Metric Spaces and in the Space of Probability Measures. Birkhäuser, 2 edition, 2008. doi: 10.1007/978-3-7643-8722-8.
  6. [6]Vansh Bansal, Saptarshi Roy, Alessandro Rinaldo, and Purnamrita Sarkar. On the convergence and straightness of rectified flow. In Proceedings of the 29th International Conference on Artificial Intelligence and Statistics, volume 300 of Proceedings of Machine Learning Research, pages 4636–4644, 2026.
  7. [7]Jean-David Benamou and Yann Brenier. A computational fluid mechanics solution to the Monge–Kantorovich mass transfer problem. Numerische Mathematik, 84(3):375–393, 2000. doi: 10.1007/s002110050002.DOI
  8. [8]Rajendra Bhatia, Tanvi Jain, and Yongdo Lim. On the Bures–Wasserstein distance between positive definite matrices. Expositiones Mathematicae, 37(2):165–191, 2019. doi: 10.1016/j.exmath.2018.01.002.DOI
  9. [9]Mikolaj Bińkowski, Danica J. Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD GANs. In International Conference on Learning Representations, 2018.arxiv.org/abs/1801.01401
  10. [11]Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-Woo Ha. StarGAN v2: Diverse image synthesis for multiple domains. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8188–8197, 2020.arxiv.org/abs/1912.01865
  11. [12]Min Jin Chong and David Forsyth. Effectively unbiased FID and inception score and where to find them. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6070–6079, 2020. URL https://opencaccess.thecvf.com/content_CVPR_2020/html/Chong_Effectively_Unbiased_FID_and_Inception_Score_and_Where_to_Find_CVPR_2020_paper.html.arxiv.org/abs/1911.07023
  12. [13]Arsène Claustre, Hugo Negrel, Claire Boyer, Kimia Nadjahi, and Eric Vanden-Eijnden. Gaussian flow-matching schedules: Implications for sampling and training. arXiv preprint arXiv:2609.25839, 2026.arxiv.org/abs/2609.25839
  13. [14]Arnaud Debussche and Erwan Faou. Weak backward error analysis for SDEs. SIAM Journal on Numerical Analysis, 50(3):1735–1752, 2012. doi: 10.1137/110831544.DOI
  14. [15]A. B. Duncan, G. A. Pavliotis, and K. C. Zygalakis. Nonreversible langevin samplers: Splitting schemes, analysis and implementation. arXiv preprint arXiv:1701.04247, 2017. URL https://arxiv.org/abs/1701.04247.
  15. [16]Yansong Gao, Zhihong Pan, Xin Zhou, Le Kang, and Pratik Chaudhari. Fast diffusion probabilistic model sampling through the lens of backward error analysis. arXiv preprint arXiv:2304.11446, 2023.arxiv.org/abs/2304.11446
  16. [17]Matthias Gelbrich. On a formula for the L² Wasserstein metric between measures on Euclidean and Hilbert spaces. Mathematische Nachrichten, 147(1):185–203, 1990. doi: 10.1002/mana.19901470121.DOI
  17. [18]Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(25):723–773, 2012. URL https://jmlr.org/papers/v13/gretton12a.html.DOI
  18. [19]Ernst Hairer, Christian Lubich, and Gerhard Wanner. Geometric Numerical Integration: Structure-Preserving Algorithms for Ordinary Differential Equations. Springer, 2 edition, 2006. doi: 10.1007/3-540-30668-8.
  19. [20]Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Advances in Neural Information Processing Systems, volume 30, 2017.
  20. [21]Nicholas J. Higham. Functions of Matrices: Theory and Computation. Society for Industrial and Applied Mathematics, 2008. doi: 10.1137/1.9780898717778.DOI
  21. [22]Christian Horvat and Jean-Pascal Pfister. On gauge freedom, conservativity and intrinsic dimensionality estimation in diffusion models. In International Conference on Learning Representations, 2024.arxiv.org/abs/2402.03845
  22. [23]Yilie Huang, Wenpin Tang, and Xun Yu Zhou. Adaptive reparameterized time for score-based diffusion sampling. arXiv preprint arXiv:2607.02137v3, 2026.arxiv.org/abs/2607.02137
  23. [24]Samuel Hurault, Thomas Moreau, and Gabriel Peyré. Geometry-aware discretization error of diffusion models. arXiv preprint arXiv:2605.08392, 2026.arxiv.org/abs/2605.08392
  24. [25]Arieh Iserles, G. R. W. Quispel, and P. S. P. Tse. B-series methods cannot be volume-preserving. BIT Numerical Mathematics, 47(2):351–378, 2007. doi: 10.1007/s10543-006-0114-8.DOI
  25. [26]Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4401–4410, 2019.DOI
  26. [27]Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. In Advances in Neural Information Processing Systems, volume 35, pages 25655–26577, 2022.arxiv.org/abs/2206.00364
  27. [28]Naïl B. Khelifa, Richard E. Turner, and Ramji Venkataramanan. Diffusion models observe only gradients: A geometric perspective on score matching errors. arXiv preprint arXiv:2606.06179v2, 2026.arxiv.org/abs/2606.06179
  28. [29]Valentin Khrulkov, Gleb Ryz h akov, Andrei Chertkov, and Ivan Oseledets. Understanding DDPM latent codes through optimal transport. In International Conference on Learning Representations, 2023.arxiv.org/abs/2202.07477
  29. [30]Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009. URL https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf.
  30. [31]Adrien Laurent and Gilles Vilmart. Order conditions for sampling the invariant measure of ergodic stochastic differential equations on manifolds. Foundations of Computational Mathematics, 22:649–695, 2022. doi: 10.1007/s10208-021-09495-y.DOI
  31. [32]Hugo Lavenant and Filippo Santambrogio. The flow map of the fokker–planck equation does not provide optimal transport. Applied Mathematics Letters, 133:108225, 2022. doi: 10.1016/j.aml.2022.108225.DOI
  32. [33]Tony Lelièvre, Francis Nier, and Grigorios A. Pavliotis. Optimal non-reversible linear drift for the convergence to equilibrium of a diffusion. Journal of Statistical Physics, 152(2):237–274, 2013. doi: 10.1007/s10955-013-0769-x.DOI
  33. [34]Qinchan Li, Pedro Cisneros-Velarde, Keru Fu, Samuel Antunes Miranda, Sharan Vaswani, and Hao Zhang. CAT-Flow: Curvature-adaptive steps for flow matching. arXiv preprint arXiv:2609.01746, 2026.arxiv.org/abs/2609.01746
  34. [35]Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. In International Conference on Learning Representations, 2023.arxiv.org/abs/2210.02747
  35. [36]Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In International Conference on Learning Representations, 2023.arxiv.org/abs/2209.03003
  36. [37]Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM-Solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps. In Advances in Neural Information Processing Systems, volume 35, pages 5775–5787, 2022.arxiv.org/abs/2206.00927
  37. [38]Hengyuan Ma, Li Zhang, Xiatian Zhu, and Jianfeng Feng. Accelerating score-based generative models with preconditioned diffusion sampling. In Computer Vision – ECCV 2022, volume 13683 of Lecture Notes in Computer Science, pages 1–16. Springer, 2022. doi: 10.1007/978-3-031-20050-2_1. URL https://www.ecva.net/papers/eccv_2022/papers_ECCV/papers/136830001.pdf.DOI
  38. [39]Hengyuan Ma, Xiatian Zhu, Jianfeng Feng, and Li Zhang. Preconditioned score-based generative models. International Journal of Computer Vision, 133(7):4837–4863, 2025. doi: 10.1007/s11263-025-02410-w.DOI
  39. [41]Muhammad Ferjad Naeem, Seong Joon Oh, Youngjung Uh, Yunjey Choi, and Jaegun Yoo. Reliable fidelity and diversity metrics for generative models. In Proceedings of the 37th International Conference on Machine Learning, volume 119, pages 7176–7185. PMLR, 2020. URL https://proceedings.mlr.press/v119/naeem20a.html.arxiv.org/abs/2002.09797
  40. [42]Ewsey Obzherin and Valentin Khrulkov. Density-level backward error for low-NFE generative flows, 2026. URL https://arxiv.org/abs/2610.29475. arXiv:2610.29475.

Paper details

vlnc:W_79vfa3jp
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Generative AI tools

View statement
Looking for related papers…
Contents