betaFeedback

WorksMathematics

The Limits of Closed-Form Couplings for Few-Step Generation

Abstract

Few-step generators via flow models require complicated technical solutions such as a pretrained flow model or a consistency loss. While rectified flows have gained popularity for their success in few-step sampling, they pay a cost of needing a pre-trained model and sampling a large number of couplings per dataset. Yet for a finite dataset and a Gaussian probability path, the probability-flow vector field of the empirical distribution is available in closed form, so one could use this field as the teacher and skip pretraining altogether. We demonstrate that, surprisingly, this approach is impossible for problems of practical relevance. In particular, the analytic solution of the diffusion process in high dimensions becomes almost a linear transformation. The integration returns the noise vector drawn at initialisation, plus a drift that is empirically a faint copy of the data point, almost independent of the noise. The coupling therefore differs from an independent pairing only through this drift. As a result, training a model on these couplings is almost equivalent to training the model on the independent draws of the noise and the data, which does not provide a useful teacher. We highlight that this is the direct consequence of the high-dimensionality of the data, and that for low enough dimensions or large enough datasets, analytic couplings become more efficient teachers. We quantify this effect by the fraction of endpoint energy carried by the drift, which scales as log⁡Nd\frac{\log N}{d} for a dataset of size NN in a dd-dimensional space. Finally, we provide an empirical study confirming all the theoretical results presented in the paper.

Introduction

Diffusion and flow-based generative models learn a time-dependent vector field that transports samples from a simple base distribution to the data distribution by integrating an ODE [1, 2, 3, 4]. Flow Matching (FM) trains a parametric field by regressing to the conditional velocity of a chosen probability path, enabling simulation-free training. Sampling, however, still requires integrating the ODE over many steps, and reducing this cost remains an active area of research.

One line of work learns a direct map between different points along the generative trajectory, enabling large jumps in time with a single model evaluation, either by distilling a pretrained model or by training from scratch [5, 6, 7, 8]. While distillation requires a pretrained teacher, direct training from scratch can be optimisation-sensitive and therefore requires substantial stabilisation and tuning. An alternative approach keeps the FM objective but changes the coupling between noise and data on which it trains. The regression target depends on how the endpoints are paired, and the choice of couplings can yield straighter trajectories that can be integrated in fewer steps. Several approaches construct such couplings using optimal transport [9, 10, 11]. These couplings require no pretrained model, but the improvement they yield is limited. The most prominent approach, rectified flows [12], instead obtains couplings from a pretrained teacher. In this case, each noise sample is integrated through the teacher’s probability-flow ODE to generate a datapoint, and a student is trained with FM on the resulting pairs. Recent improvements have shown that a single rectification step can be enough for state-of-the-art few-step generation [13, 14]. However, this procedure remains expensive, as it requires pretraining the teacher.

This raises a natural question: if rectification benefits from “good” couplings, could we avoid pretrained couplings entirely by using closed-form couplings? Concretely, given a Gaussian probability path and a dataset defining the target marginal, one can derive an “empirical” optimal regression target (vector field) for FM/diffusion models [15, 16, 17]. Park et al. [2026] successfully exploit this idea for discrete flow models using closed-form inversion, suggesting that a similar shortcut might be possible for rectification in the continuous case.

There is, however, a fundamental ambiguity in turning this vector field into couplings. The empirical probability-flow ODE maps entire regions of prior space to the same data point, so it determines which region corresponds to a data point, but not a unique endpoint within that region (see Figure 4). The question is therefore whether a useful pairing can be selected directly from this closed-form flow, without relying on a pretrained teacher. We show that the natural closed-form construction produces couplings that perform close to random pairings and therefore fail to train a good student. Surprisingly, the pretrained and closed-form constructions largely associate the same data points with the same regions of prior space, but select very different endpoints within those regions. We explain this failure theoretically, identify the two quantities that control it, the dataset size NN and the dimension dd, and show empirically that by moving the whole construction into lower-dimensional spaces, the closed-form couplings become better teachers.

Figure 4. Many-to-one structure of the closed-form probability-flow ODE. Left: An empirical two-dimensional dataset p^0\hat{p}_0, with each colour denoting one data point. Right: The corresponding partition of the Gaussian prior space p1p_1 induced by the closed-form PFPF-ODEODE. Each coloured region contains all prior points that are mapped back to the same data point when the flow is integrated in reverse. The matching colours indicate the associated data point and region. Stars mark the endpoints obtained by forward integration from the conditional mean, xϵ=αϵx0x_\epsilon= \alpha_\epsilon x_0 (λ=0\lambda= 0). Thus, a single data point corresponds to an entire region of prior space rather than a unique endpoint.

Our contributions are as follows:

  • Closed-form couplings are poor teachers (§ 4). Students trained on closed-form probability-flow couplings perform far worse than students trained on pretrained couplings, and only marginally better than students trained on random pairings. We show that this happens, even though the pretrained couplings themselves correspond to one of the pairings admitted by the closed-form field, with more than 99.9% matching accuracy.

  • Closed-form couplings are dominated by a linear noise map (§ 5). We prove an endpoint decomposition for the closed-form flow (Proposition 5.1). This shows that the terminal coupling point x1x_1 decomposes into the injected initialisation noise and a trajectory-dependent drift. We further derive an exact cutoff decomposition that bounds this drift in terms of the trajectory’s tracking error (Proposition 5.2). Empirically, the drift is small and largely independent of the injected noise, so each datapoint is paired essentially with the fresh Gaussian sample used to initialise its integration, plus a small datapoint-dependent correction, which explains the poor performance of closed-form couplings.

  • Closed-form couplings require exponentially more data in higher dimensions (§ 5.2 and § 6). Combining our drift bound with the collapse time of empirical diffusion dynamics predicts that the informative share of the coupling scales as log⁡Nd\frac{\log N}{d}. Therefore, maintaining a fixed drift share as the dimension grows requires the dataset size to scale exponentially with dd. We test the complementary prediction by progressively reducing the image resolution. The drift share increases as 1d\frac{1}{d} and student performance improves, although the gap to pretrained couplings remains substantial even at the lowest resolutions we study.

Related Work

Couplings and rectified flows. The choice of couplings used for training can significantly affect the straightness of Flow Matching trajectories. In minibatch optimal-transport methods, non-independent couplings are constructed to obtain straighter flows and therefore faster sampling [9, 10], with recent improvements by developing scalable semi-discrete variants [11]. Instead of relying on semi-discrete couplings, rectified flows use couplings obtained from a pre-trained teacher instead [12]. This idea has enabled high-quality few-step generation [19, 13, 20, 14]. Based on these advancements, recent theoretical studies investigate when rectification produces straight or optimal couplings [21, 22]. Berthelot et al. [2026] instead distil couplings from a pretrained normalising flow, whose endpoints are likewise obtained from noised images but, unlike in our closed-form construction, remain a learned function of the data. Since all these works require a pretrained teacher, we ask whether it can be removed altogether by obtaining endpoint pairs from the closed-form vector field.

Closed-form empirical flows. For a finite empirical distribution and Gaussian probability path, the optimal diffusion score or Flow Matching velocity can be evaluated analytically. Prior work has used these closed-form dynamics to study memorisation and generalisation [17, 15, 16, 24], including the temporal phases of the dynamics and the transition from averaging over many data points to the collapse on individual training examples [25, 26]. Recent work further characterises the geometry produced by the exact semi-discrete Flow Matching dynamics [27]. In contrast, we ask whether the endpoint coupling obtained from the empirical probability flow can itself serve as a teacher for few-step generation. We show that its datapoint-dependent information is confined to a small drift term and characterise when this signal becomes large enough to improve over random pairings.

Background

Let x0∼p0x_0 \sim p_0 denote the data distribution on Rd\mathbb{R}^d. We consider the Gaussian conditional path

pt(xt∣x0)=N(xt∣αtx0,σt2I)t∈(0,1],(1)p_t(x_t \mid x_0) = \mathcal{N}(x_t \mid\alpha_t x_0, \sigma_t^2 \mathbf{I}) \qquad t \in(0,1], \tag*{(1)}

for differentiable schedules αt,σt∈R∗\alpha_t,\sigma_t \in\mathbb{R}^* which induce marginals pt(x)=∫pt(x∣x0)p0(x0)dx0p_t(x) = \int p_t(x \mid x_0)p_0(x_0)\mathrm{d}x_0. A velocity field transporting these marginals can be obtained from a conditional velocity field that, together with pt(xt∣x0)p_t(x_t \mid x_0), satisfies the continuity equation. For (1), the conditional velocity field is

vt(xt∣x0)=dαtdtx0+dlog⁡σtdt(xt−αtx0).(2)v_t(x_t \mid x_0) = \frac{\mathrm{d}\alpha_t}{\mathrm{d}t}x_0 + \frac{\mathrm{d}\log\sigma_t}{\mathrm{d}t}(x_t-\alpha_t x_0). \tag*{(2)}

The marginal probability-flow vector field is given by the posterior average vt(x)=∫vt(x∣x0)pt(x0∣x)dx0v_t(x) = \int v_t(x \mid x_0)p_t(x_0 \mid x)\mathrm{d}x_0. For an empirical distribution p^0(x)=1N∑i=1Nδ(x−x0i)\hat{p}_0(x) = \frac{1}{N}\sum_{i=1}^{N}\delta(x-x_0^i), this integral reduces to the finite sum

vtPF(x)=∑i=1Nwti(x)(dαtdtx0i+dlog⁡σtdt(x−αtx0i)),wti(x)=N(x∣αtx0i,σt2I)∑j=1NN(x∣αtx0j,σt2I).(3)v_t^{PF}(x) = \sum_{i=1}^{N}w_t^i(x)\left(\frac{\mathrm{d}\alpha_t}{\mathrm{d}t}x_0^i + \frac{\mathrm{d}\log\sigma_t}{\mathrm{d}t}(x-\alpha_t x_0^i)\right), \qquad w_t^i(x) = \frac{\mathcal{N}(x \mid\alpha_t x_0^i,\sigma_t^2\mathbf{I})}{\sum_{j=1}^{N}\mathcal{N}(x \mid\alpha_t x_0^j,\sigma_t^2\mathbf{I})}. \tag*{(3)}

Since vtPFv_t^{PF} is induced by the empirical data distribution, its reverse dynamics are strongly contracting as t→0t \to0. In particular, starting from a point in the Gaussian prior at t=1t=1 and integrating the exact field backwards to t=0t=0 sends it to one of the data points in the support of p^0\hat{p}_0. Hence, whole regions of prior space can contract onto the same training example (Figure 4).

In practice, Flow Matching approximates vtPFv_t^{PF} with a parametric model vFM(x,t)v^{FM}(x,t) by least-squares regression to a Monte Carlo draw of the conditional velocity. Specifically, sample z∼N(0,I)z \sim\mathcal{N}(0,\mathbf{I}), set xt=αtx0+σtzx_t = \alpha_t x_0 + \sigma_t z, and use the target dxtdt=dαtdtx0+dσtdtz\frac{\mathrm{d}x_t}{\mathrm{d}t} = \frac{\mathrm{d}\alpha_t}{\mathrm{d}t}x_0 + \frac{\mathrm{d}\sigma_t}{\mathrm{d}t}z. Theoretically, a model with sufficient capacity would eventually converge to Equation (3) [28, 29].

Closed-form PF-ODE couplings are poor teachers

A natural way to avoid pretraining a Flow Matching teacher is to construct couplings directly from the closed-form probability-flow field introduced in (3). Surprisingly, this construction fails in practice. As shown in Figure 1, students trained on these couplings perform much closer to students trained on random noise-data pairings than to those trained on pretrained-model couplings.

Figure 1. data x0x_0; initial noise zz; αϵx0+σϵz\alpha_\epsilon x_0 + \sigma_\epsilon z; initialise trajectory; Forward Integrate ODE; Pre-trained vectorfield; Closed-form vectorfield; x1FMx_1^{\mathrm{FM}}; x1PFx_1^{\mathrm{PF}}; Informative coupling; strong few-step generation; Weak coupling; poor few-step generation. Why? The closed-form endpoint is mostly the initial noise. x1PF=z+Δ(xϵ)x_1^{\mathrm{PF}} = z + \Delta(x_\epsilon). The initial noise survives, scaled to full magnitude. The signal survives but scaled to low energy.

Figure 1 (PDF p. 4). FID against sampling steps for students trained on πPF\pi^{PF}, πFM\pi^{FM} and πRandom\pi^{Random}. For a fair comparison, each model is trained exactly on the NN forward couplings permitted by the datasets. This limits the FID value, but enables us to provide a clean comparison between couplings. Although the gap narrows as the number of integration steps increases, it is substantial in the few-step regime.

To construct a coupling for each datapoint x0ix_0^i, we sample from the conditional path in Equation (1) at a small time ϵ>0\epsilon> 0, avoiding the singularity at t=0t = 0:

xϵ=αϵx0+σϵz,z∼N(0,I).(4)x_\epsilon= \alpha_\epsilon x_0 + \sigma_\epsilon z,\qquad z \sim\mathcal{N}(0,I). \tag*{(4)}

Starting from xϵix_\epsilon^i, we integrate either vPFv^{PF} or vFMv^{FM} forward from t=ϵt = \epsilon to t=1t = 1, obtaining the endpoint x1ix_1^i. We use ϵ=10−3\epsilon= 10^{-3} unless stated otherwise. This gives π(⋅)={(x0i,x1i)}i=1N\pi^{(\cdot)} = \{(x_0^i, x_1^i)\}_{i=1}^N, with marginals π0(⋅)={x0i}i=1N\pi_0^{(\cdot)} = \{x_0^i\}_{i=1}^N and π1(⋅)={x1i}i=1N\pi_1^{(\cdot)} = \{x_1^i\}_{i=1}^N. Here, πPF\pi^{PF} denotes couplings generated with the closed-form field and πFM\pi^{FM} those generated with the pretrained Flow Matching model. Random couplings πRandom\pi^{Random} are obtained by independently pairing datapoints with Gaussian noise samples. We use exactly NN forward couplings in all cases, such that the data marginal π0(⋅)\pi_0^{(\cdot)} is identical across constructions, and differences can be attributed to the induced pairing. Further details are given in Appendix § 13.

Figure 1 compares rectified-flow students trained with the Flow Matching objective on the three coupling sets πPF\pi^{PF}, πFM\pi^{FM}, and πRandom\pi^{Random}. Within each dataset, all students are trained on the same number of couplings, with the same architecture and optimisation settings. The only difference is that, on CIFAR-10 and CelebA, students on πFM\pi^{FM} are trained for fewer steps, since their FID stops improving earlier (Appendix § 13). The difference is most pronounced at a single sampling step, where students trained on πPF\pi^{PF} have roughly 4–5× higher FID than those trained on πFM\pi^{FM} across datasets. This gap is particularly striking because vPFv^{PF} is the exact probability-flow field of the empirical distribution, whereas vFMv^{FM} is only learned as an approximation to it. We therefore ask whether the pretrained endpoints nevertheless fall within the same regions the closed-form PFPF-ODEODE assigns to that specific datapoint.

The large performance gap could suggest that the pretrained model learns a fundamentally different correspondence between data and prior space. To test this, we take the endpoints produced by vFMv^{FM} and integrate them backwards with the closed-form field vPFv^{PF}. On all three datasets the trajectory returns to the data point from which the pretrained coupling originated in more than 99.9% of the cases (Appendix § 10). Thus, although vFMv^{FM} and vPFv^{PF} define different forward maps, the pretrained endpoints almost always lie within the closed-form PF-ODE region associated with the same datapoint. Combined with the observation above, this suggests that the pretrained model selects a much narrower subset within the broader regions admitted by the closed-form flow. While vPFv^{PF} is highly sensitive to the initial perturbation, vFMv^{FM} maps different perturbations of the same datapoint to much more similar endpoints (Table 1).

datasetcouplingnoise sharecos⁡(x1,x1′)\cos(x_1, x_1')NN match
CIFAR-10teacher6.1%0.94100%
CIFAR-10closed-form99.6%0.0040.05%
CelebAteacher8.2%0.92100%
CelebAclosed-form99.9%0.0010.05%
ImageNetteacher4.3%0.96100%
ImageNetclosed-form99.8%0.0020.03%

Table 1. Teacher couplings select a narrow set of endpoints within each region. For the first 2000 training images on each data point, the endpoints from three independent noise draws (λ=1,ϵ=10−3)(\lambda= 1, \epsilon= 10^{-3}) are integrated with 250 Euler steps using vFMv^{FM} and vPFv^{PF}. Then, we measure the noise share, i.e. fraction of the endpoint variance explained by the injected noise, the cosine similarity between two endpoints of the same data point and fraction of endpoints whose nearest neighbour among the endpoints of another noise draw belongs to the same data point (chance level 0.05%).

The results from this section isolate the source of the failure. Although the closed-form and pretrained models induce different forward maps, their endpoints are largely associated with the same datapoint regions in prior space. The difference seems to lie in how they select endpoints within these regions. The closed-form construction is highly sensitive to the initial perturbation and appears to produce a particularly uninformative pairing. In the next section, we characterise this selection explicitly and show that the endpoint is dominated by the noise used to initialise the integration.

An exact endpoint decomposition: the flow returns the initialisation noise

Following the empirical results, we now show that the endpoint of the closed-form flow admits a simple decomposition. The integration returns the injected noise z\mathbf{z} (Equation (4)) plus a non-linear data point dependent drift Δ(xϵ)\Delta(\mathbf{x}_{\epsilon}), which captures how the trajectory deviates from x0\mathbf{x}_0 over time. This drift term is empirically small, carrying less than 0.6% of the endpoint’s squared norm in all datasets we investigate, and preserves the structure of the originating data point. Thus, the closed-form coupling is close to an independent noise-data pairing, with the drift Δ(xϵ)\Delta(\mathbf{x}_{\epsilon}) providing a small datapoint-dependent correction.

We now derive this decomposition exactly. First, we split the closed-form field into a part that is linear in x\mathbf{x} and a nonlinear remainder. Using ∑iwti(x)=1\sum_i w_t^i(\mathbf{x}) = 1, Equation (3) can be rewritten as

vtPF(x)=dlog⁡σtdtx+γtmt(x),γt:=dαtdt−αtdlog⁡σtdt,mt(x):=∑i=1Nwti(x)x0i.(5)\mathbf{v}^{PF}_t(\mathbf{x}) = \frac{\mathrm{d}\log\sigma_t}{\mathrm{d}t}\mathbf{x} + \gamma_t m_t(\mathbf{x}), \qquad\gamma_t := \frac{\mathrm{d}\alpha_t}{\mathrm{d}t} - \alpha_t\frac{\mathrm{d}\log\sigma_t}{\mathrm{d}t}, \qquad m_t(\mathbf{x}) := \sum_{i=1}^{N} w_t^i(\mathbf{x})\mathbf{x}_0^i . \tag*{(5)}

where mt(x)m_t(\mathbf{x}) is the posterior mean of the data given xt=x\mathbf{x}_t = \mathbf{x}. The first term is linear in x\mathbf{x} and depends on the data only through the schedule. All nonlinearity of the field is carried by the posterior mean. The linear part can be integrated exactly, which is what produces the following identity.

Proposition 5.1 (informal). Let (xt)t∈[ϵ,1](\mathbf{x}_t)_{t\in[\epsilon,1]} solve dxtdt=vtPF(xt)\frac{\mathrm{d}\mathbf{x}_t}{\mathrm{d}t} = \mathbf{v}^{PF}_t(\mathbf{x}_t) with initial condition xϵ=αϵx0+λσϵz\mathbf{x}_{\epsilon} = \alpha_{\epsilon}\mathbf{x}_0 + \lambda\sigma_{\epsilon}\mathbf{z}, λ∈[0,1]\lambda\in[0,1]. Then, with γt\gamma_t and mtm_t as in Equation (5),

x1=λz+Δ(xϵ),Δ(xϵ):=∫ϵ1γtσt(mt(xt)−x0) dt.(6)\mathbf{x}_1 = \lambda\mathbf{z} + \Delta(\mathbf{x}_{\epsilon}), \qquad\Delta(\mathbf{x}_{\epsilon}) := \int_{\epsilon}^{1} \frac{\gamma_t}{\sigma_t}\left(m_t(\mathbf{x}_t)-\mathbf{x}_0\right)\,\mathrm{d}t . \tag*{(6)}

The formal proposition and its proof are given in Appendix § 11. The injected noise therefore survives exactly as an additive term at the endpoint. Although it enters the initial condition only through the small perturbation λσϵz\lambda\sigma_{\epsilon}\mathbf{z}, its endpoint contribution is λz\lambda\mathbf{z}, corresponding to an amplification by 1σϵ\frac{1}{\sigma_{\epsilon}}. For our choice σt=t\sigma_t=t and ϵ=10−3\epsilon=10^{-3}, this factor is 10310^3, explaining the strong sensitivity to even small amounts of initialisation noise. This exact recovery of the injected noise leaves the drift as the only component which also depends on the data point. We therefore turn to Δ(xϵ)\Delta(\mathbf{x}_{\epsilon}) and ask what information it carries. Visually, it resembles a faint copy of the corresponding data point (Figure 2, with further examples in Appendix § 9), suggesting that it is largely determined by x0\mathbf{x}_0 rather than by the particular noise realisation. We first isolate this component by setting λ=0\lambda=0 and removing the noise from the integration, then test its dependence on the injected noise, and finally measure its magnitude relative to the full endpoint.

Figure 2. Endpoint decomposition of an ImageNet-64 integration (λ=1\lambda= 1, ϵ=10−3\epsilon= 10^{-3}, closed-form velocity against all 1.28M training images). The endpoint x1x_1 looks like pure Gaussian noise. Subtracting the injected noise λz\lambda z reveals the drift Δ(xϵ)\Delta(x_{\epsilon}), a faint copy of the datapoint that keeps most of its structure. The drift is well aligned with the data, with cos⁡(Δ(xϵ),x0)=0.88\cos(\Delta(x_{\epsilon}), x_0) = 0.88 for this image (0.86 on average). The drift of the noise-free PFPF-ODE (λ=0\lambda= 0) integration, Δ(αϵx0)\Delta(\alpha_{\epsilon}x_0), is almost identical to it, with cos⁡(Δ(xϵ),Δ(αϵx0))=0.96\cos(\Delta(x_{\epsilon}), \Delta(\alpha_{\epsilon}x_0)) = 0.96 (0.95 on average). The injected noise therefore barely changes the information the drift carries about x0x_0. The datapoint x0x_0 on the right is shown for reference. All panels use the same colour scale.

The case λ=0\lambda=0. Setting λ=0\lambda=0 removes the injected noise and gives x1=Δ(αϵx0)\mathbf{x}_1=\Delta(\alpha_{\epsilon}\mathbf{x}_0), the drift evaluated along the mean-initialised trajectory. This provides a direct way to visualise the data point dependent part of the endpoint. As shown in Figure 2 and Figure 6 and Table 2(b), starting from xϵ=αϵx0\mathbf{x}_{\epsilon}=\alpha_{\epsilon}\mathbf{x}_0 produces structure-preserving endpoints rather than Gaussian noise. For datasets with reflection symmetries this is exact, as the noise-free trajectory never leaves the line through its data point, so that Δ(αϵx0)\Delta(\alpha_{\epsilon}\mathbf{x}_0) is a rescaled copy of x0\mathbf{x}_0 (Proposition A.2). Therefore, while setting λ=0\lambda=0 removes the injected noise that appears to make the coupling difficult to learn from, the resulting endpoints no longer have the required Gaussian marginal and cannot be used as training couplings.

Figure 6. Noisy versus noise-free initialisation on ImageNet-64 (ϵ=10−3\epsilon= 10^{-3}). The two initial conditions differ only by σϵz\sigma_\epsilon\mathbf{z} and are visually indistinguishable. Their endpoints, however, differ completely. Starting from the noisy initialisation (a) produces endpoints that resemble Gaussian noise, whereas starting from the conditional mean (b) produces faint, low-contrast copies of the data points, from which each pair can be identified by eye.

DatasetE∥Δ(xϵ)∥2E∥x1∥2\frac{\mathbb{E}\lVert\Delta(x_\epsilon)\rVert^2}{\mathbb{E}\lVert x_1\rVert^2}E∥Δ(xϵ)−Δ(xϵ′)∥2E∥Δ(xϵ)∥2\frac{\mathbb{E}\lVert\Delta(x_\epsilon)-\Delta(x'_\epsilon)\rVert^2}{\mathbb{E}\lVert\Delta(x_\epsilon)\rVert^2}E∥Δ(xϵ)−Δ(αϵx0)∥2E∥Δ(xϵ)∥2\frac{\mathbb{E}\lVert\Delta(x_\epsilon)-\Delta(\alpha_\epsilon x_0)\rVert^2}{\mathbb{E}\lVert\Delta(x_\epsilon)\rVert^2}ggremainder
CIFAR-100.52%25.5%15.5%1.1913.3%
CelebA0.14%27.3%16.8%1.2114.2%
ImageNet0.18%22.0%14.0%1.1812.0%
Datasetcos⁡(Δ(xϵ),Δ(xϵ′))\cos(\Delta(x_\epsilon),\Delta(x'_\epsilon))cos⁡(Δ(xϵ),Δ(αϵx0))\cos(\Delta(x_\epsilon),\Delta(\alpha_\epsilon x_0))cos⁡(Δ(xϵ),x0)\cos(\Delta(x_\epsilon),x_0)cos⁡(Δ(αϵx0),x0)\cos(\Delta(\alpha_\epsilon x_0),x_0)
CIFAR-100.8990.9450.8670.932
CelebA0.8880.9390.7460.796
ImageNet0.9110.9520.8640.923

Table 2. Empirical verification of Equation (7). Every data point is integrated twice with independent noises zz, z′∼N(0,I)z' \sim\mathcal{N}(0,I) (λ=1\lambda=1) and once from αϵx0\alpha_\epsilon x_0 (PF-ODE (λ=0\lambda=0)), for 25000 data points per dataset, with the full training set as the empirical distribution and ϵ=10−3\epsilon=10^{-3}. (a) Share of the endpoint energy carried by the drift (first column), energy of the differences between the drifts Δ(xϵ)=x1−z\Delta(x_\epsilon)=x_1-z, Δ(xϵ′)=x1′−z′\Delta(x'_\epsilon)=x'_1-z' and Δ(αϵx0)\Delta(\alpha_\epsilon x_0) relative to the drift energy E∥Δ(xϵ)∥2\mathbb{E}\lVert\Delta(x_\epsilon)\rVert^2 (second and third columns), and a least-squares fit Δ(xϵ)≈gΔ(αϵx0)\Delta(x_\epsilon)\approx g\Delta(\alpha_\epsilon x_0) with its gain gg and the energy share of the remainder (last two columns). (b) Cosine similarities, averaged over data points.

Empirically, the drift depends only weakly on the noise. We test this by integrating the flow twice from every data point, with two independent noise samples z,z′∼N(0,I)\mathbf{z},\mathbf{z}'\sim\mathcal{N}(\mathbf{0},\mathbf{I}), and subtracting the injected noise from each endpoint. This gives the two drifts Δ(xϵ)=x1−z\Delta(\mathbf{x}_{\epsilon})=\mathbf{x}_1-\mathbf{z} and Δ(xϵ′)=x1′−z′\Delta(\mathbf{x}'_{\epsilon})=\mathbf{x}'_1-\mathbf{z}'. A third, noise-free integration from xϵ=αϵx0\mathbf{x}_{\epsilon}=\alpha_{\epsilon}\mathbf{x}_0 gives the PFPF-ODEODE (λ=0\lambda=0) endpoint Δ(αϵx0)\Delta(\alpha_{\epsilon}\mathbf{x}_0). If the drift depended strongly on the noise, the two drifts would differ, and neither would resemble Δ(αϵx0)\Delta(\alpha_{\epsilon}\mathbf{x}_0). However, the drift is closely aligned with the PFPF-ODEODE (λ=0\lambda=0) endpoint, cos⁡(Δ(xϵ),Δ(αϵx0))=0.94\cos(\Delta(\mathbf{x}_{\epsilon}),\Delta(\alpha_{\epsilon}\mathbf{x}_0))=0.94–0.950.95, and is well described by an amplified copy of it. We found that Δ(xϵ)≈gΔ(αϵx0)\Delta(\mathbf{x}_{\epsilon})\approx g\Delta(\alpha_{\epsilon}\mathbf{x}_0) with g≈1.2g\approx1.2 on all datasets, which leaves only 12–14% of the drift energy unexplained. The drifts of the two noises agree almost as well, cos⁡(Δ(xϵ),Δ(xϵ′))=0.89\cos(\Delta(\mathbf{x}_{\epsilon}),\Delta(\mathbf{x}'_{\epsilon}))=0.89–0.910.91. They share the component gΔ(αϵx0)g\Delta(\alpha_{\epsilon}\mathbf{x}_0) and differ only by a small noise-dependent remainder (Table 2; see Appendix § 12.§ 1 for a detailed discussion). Empirically, we therefore have the approximation

Δ(xϵ)≈gΔ(αϵx0)⟺x1(λ,z)≈gΔ(αϵx0)+λz.(7)\Delta(\mathbf{x}_{\epsilon}) \approx g\Delta(\alpha_{\epsilon}\mathbf{x}_0) \qquad\Longleftrightarrow\qquad\mathbf{x}_1(\lambda,\mathbf{z}) \approx g\Delta(\alpha_{\epsilon}\mathbf{x}_0) + \lambda\mathbf{z}. \tag*{(7)}

The flow therefore returns the injected noise unchanged, on top of a drift that is almost the same for every noise (Figure 2).

Empirically, the drift is small. Relative to the Gaussian endpoint, for which E∥x1∥2=d\mathbb{E}\lVert x_1\rVert^2 = d, it accounts for only 0.52%, 0.14%, and 0.18% of the squared norm on CIFAR-10, CelebA and ImageNet-64, respectively (Table 2)(a). Interestingly, the absolute drift norm remains similar across datasets despite their different dimensionalities (Appendix § 12.§ 1).

Together with Equation (7), this gives a concrete picture of the closed-form coupling. Each data point is paired with the Gaussian sample drawn to initialise its own integration, plus a weak copy of itself, leaving only a weak datapoint-dependent signal in x1x_1. This explains why training on πPF\pi^{PF} performs closer to training on πRandom\pi^{Random} than on πFM\pi^{FM}.

A general bound on the drift

Having established the endpoint decomposition and the empirical properties of the drift, we now ask why Δ(xϵ)\Delta(x_{\epsilon}) remains small. We derive a general bound showing that the drift is determined by how long the PFPF-ODEODE trajectory remains close to the conditional path of its datapoint. If it remains close to this path until the signal-to-noise ratio (SNR) is already low, the drift must remain small. Empirically, this is precisely the regime we observe across all datasets, in line with the collapse behaviour identified in previous work [30, 25, 26, 24].

Proposition 5.2 (informal). Let (xt)t∈[ϵ,1](x_t)_{t\in[\epsilon,1]} solve dxtdt=vtPF(xt)\frac{\mathrm{d}x_t}{\mathrm{d}t} = v_t^{PF}(x_t) with initial condition xϵ=αϵx0+σϵzx_{\epsilon} = \alpha_{\epsilon}x_0 + \sigma_{\epsilon}z and rt:=1σt(xt−(αtx0+σtz))r_t := \frac{1}{\sigma_t}\left(x_t - (\alpha_t x_0 + \sigma_t z)\right) defines the normalised deviation between the conditional and unconditional trajectories. Then, for every τ∈(ϵ,1)\tau\in(\epsilon,1), the endpoint drift from Equation (6) satisfies

∥Δ(xϵ)∥L2≤∥rτ∥L2+2ατστV,(8)\lVert\Delta(x_{\epsilon})\rVert_{L^2} \le\lVert r_{\tau}\rVert_{L^2} + 2\frac{\alpha_{\tau}}{\sigma_{\tau}}\sqrt{V}, \tag*{(8)}

with VV being the total variance of the empirical data distribution and ∥⋅∥L2\lVert\cdot\rVert_{L^2} the root-mean-square norm over x0∼p^0x_0 \sim\hat{p}_0 and z∼N(0,I)z \sim\mathcal{N}(0,I).

The full proposition statement including proof can be found in Appendix § 11. The bound separates the drift into a tracking-error term and an SNR-dependent term. In Figure 8 we evaluate these for all datasets on different values of τ\tau (see Appendix § 12.§ 2). We observe that the bound is dominated by the term 2ατστV2\frac{\alpha_{\tau}}{\sigma_{\tau}}\sqrt{V} for most of the trajectory, while the normalised tracking error ∥rτ∥\lVert r_{\tau}\rVert stays extremely small. This observation is consistent with previous work, which identified a regime in which the dynamics are dominated by, or even collapse onto, an individual training sample [25, 26]. As τ\tau increases, the trajectory starts to deviate from its conditional path while the schedule ratio ατστ→0\frac{\alpha_{\tau}}{\sigma_{\tau}} \to0, causing the bound to become increasingly tight.

Figure 8. Evaluation of the drift bound. We evaluate both terms of Proposition 5.2 on all three datasets for different values of the cutoff τ\tau, using the entire training set. The bound becomes increasingly tight as τ\tau increases, showing that Proposition 5.2 provides a practical bound on the drift of the PFPF-ODEODE trajectory. Furthermore, the tracking error ∥rτ∥L2\lVert r_{\tau}\rVert_{L^2} remains negligible for small τ\tau, supporting the assumption in Equation (12).

Together, the bound and our empirical results show that the drift is governed primarily by the SNR at which the trajectory stops tracking its originating data point, scaled by the overall spread of the data. The tracking error itself remains small until this transition. Understanding the drift magnitude therefore reduces to understanding when the trajectory loses track of its originating data point. In the next section, we show how this transition depends on the dataset size and dimension.

Scaling the drift

The drift is the only component of the endpoint that depends on the data point, so its significance is set by its size relative to the Gaussian noise. As the drift becomes larger relative to the Gaussian component, the datapoint-dependent signal becomes more prominent in the coupling and may therefore be easier for a student to exploit. We now ask how this ratio scales with NN and dd.

Proposition 5.2 reduces the size of the drift to the signal-to-noise ratio αtσt\frac{\alpha_t}{\sigma_t} at which the trajectory stops to track its generating datapoint. Let τ\tau denote the timestep of this transition. As long as the trajectory remains close to its conditional path, xt≈αtx0i+σtz\mathbf{x}_t \approx\alpha_t\mathbf{x}_0^i+\sigma_t\mathbf{z}, the relative posterior weight of a competing datapoint x0j\mathbf{x}_0^j satisfies

log⁡wj(xt)wi(xt)≈αtσtzT(x0j−x0i)⏟random advantage−αt22σt2∥x0j−x0i∥2⏟distance penalty.(9)\log\frac{w^j(\mathbf{x}_t)}{w^i(\mathbf{x}_t)} \approx \underbrace{\frac{\alpha_t}{\sigma_t}\mathbf{z}^{\mathsf{T}}(\mathbf{x}_0^j-\mathbf{x}_0^i)}_{\text{random advantage}} - \underbrace{\frac{\alpha_t^2}{2\sigma_t^2}\lVert\mathbf{x}_0^j-\mathbf{x}_0^i\rVert^2}_{\text{distance penalty}}. \tag*{(9)}

In high dimensions, if pairwise distances concentrate, a typical squared distance satisfies ∥x0j−x0i∥2=Θ(V)\lVert\mathbf{x}_0^j-\mathbf{x}_0^i\rVert^2=\Theta(V). Hence, conditional on the data points,

zT(x0j−x0i)∼N(0,∥x0j−x0i∥2)≈N(0,V),(10)\mathbf{z}^{\mathsf{T}}(\mathbf{x}_0^j-\mathbf{x}_0^i)\sim\mathcal{N}\left(0,\lVert\mathbf{x}_0^j-\mathbf{x}_0^i\rVert^2\right)\approx\mathcal{N}(0,V), \tag*{(10)}

so a typical fluctuation has scale V\sqrt{V}. Across NN competing data points, the largest such fluctuation is of order Vlog⁡N\sqrt{V\log N}. The largest random advantage therefore has scale αtσtVlog⁡N\frac{\alpha_t}{\sigma_t}\sqrt{V\log N}, whereas the distance penalty has scale αt2σt2V\frac{\alpha_t^2}{\sigma_t^2}V. The posterior stops concentrating on the generating data point when these become close, i.e.

ατ2στ2V=Θ(log⁡N).(11)\frac{\alpha_\tau^2}{\sigma_\tau^2}V=\Theta(\log N). \tag*{(11)}

(11) predicts the SNR at which the posterior loses track of the generating datapoint. We test this prediction in Figure 9 (left) by varying the data dimension, which in turn changes the total variance VV, and measuring the corresponding collapse time. Across CIFAR-10, CelebA and ImageNet-64, the predicted transition closely tracks the measured one. This is the same high-dimensional collapse transition studied by Biroli et al. [ [25]. Their analysis identifies log⁡Nd\frac{\log N}{d} as the parameter controlling the time when the trajectory collapses to a data point and hence the start of memorisation in the reverse generative dynamics. Related analyses of the closed-form vector field observed the same temporal transition from averaging over multiple training points to dynamics dominated by a single data point [15, 26, 24].

Figure 9. Empirical validation of the scaling argument. Left: The posterior collapse time τ\tau as the data dimension is varied by downsampling. Dashed lines show the prediction (ατστ)2V=Clog⁡N\left(\frac{\alpha_{\tau}}{\sigma_{\tau}}\right)^2 V = C\log N from Equation (11), with a single constant CC fitted per dataset, which closely tracks the measured transition across CIFAR-10, CelebA and ImageNet-64. Right: Drift magnitude as a function of the dataset size NN. Solid lines show fits of the form E∥Δ(xε)∥=alog⁡N+b\mathbb{E}\lVert\Delta(x_{\varepsilon})\rVert= a\sqrt{\log N}+b, confirming the log⁡N\sqrt{\log N} scaling predicted by Equation (12). Dashed lines extrapolate the fits beyond the full training set, and error bars are 95% confidence intervals.

Choosing τ\tau at the beginning of this transition, the normalized tracking error ∥rτ∥L2\lVert r_\tau\rVert_{L^2} is empirically small (Figure 8) compared with the SNR term ατστV\frac{\alpha_\tau}{\sigma_\tau}\sqrt{V}. Hence, by Proposition 5.2 and Equation (11) we have

∥Δ(xϵ)∥L2≲ατστV=O(log⁡N).(12)\lVert\Delta(\mathbf{x}_\epsilon)\rVert_{L^2}\lesssim\frac{\alpha_\tau}{\sigma_\tau}\sqrt{V}=O\left(\sqrt{\log N}\right). \tag*{(12)}

This characterisation implies that the drift grows only as log⁡N\sqrt{\log N}. Figure 9 (right) tests this prediction directly by varying the size of the empirical distribution (details in Appendix D.3). We found that across all three datasets, the measured drift closely follows E∥Δ(xϵ)∥≈1.3log⁡N\mathbb{E}\lVert\Delta(\mathbf{x}_\epsilon)\rVert\approx1.3\sqrt{\log N} up to a constant offset.

Finally, the exact probability-flow ODE preserves the prescribed marginals. Since α1=0\alpha_1=0 and σ1=1\sigma_1=1, its endpoint satisfies x1∼N(0,Id)\mathbf{x}_1\sim\mathcal{N}(0,\mathbf{I}_d), and therefore E∥x1∥2=d\mathbb{E}\lVert\mathbf{x}_1\rVert^2=d. The fraction of endpoint energy carried by the drift is consequently

E∥Δ(xϵ)∥2E∥x1∥2=O(log⁡Nd).(13)\frac{\mathbb{E}\lVert\Delta(\mathbf{x}_\epsilon)\rVert^2}{\mathbb{E}\lVert\mathbf{x}_1\rVert^2} =O\left(\frac{\log N}{d}\right). \tag*{(13)}

(13) identifies the lever we could use to increase the informative fraction, as it scales linear in 1d\frac{1}{d} but only logarithmic in NN. This motivates our next section, in which we study closed-form couplings on the same datasets for different resolutions, hence controlling dd.

Reducing the dimension improves closed-form couplings

(13) predicts that the informative share of the closed-form coupling grows linearly in 1d\frac{1}{d} but only logarithmically in NN. We test this by downsampling the images and repeating the coupling construction of Section 4, leaving all other parts of the setup unchanged. Concretely, we vary d=3r2d=3r^2 through the resolution rr of the same images, using r∈{16,32,64}r\in\{16,32,64\} for CelebA and ImageNet and r∈{16,24,32}r\in\{16,24,32\} for CIFAR-10. Since the dataset size NN is fixed across resolutions, it does not confound the comparison. Our goal is not to reach the best possible FID, but to track how closed-form couplings perform relative to random and pretrained couplings as dd decreases. Full experimental details are given in Appendix § 13 and a detailed discussion about the connection between the drift share and learnability can be found in Appendix § 15.

Lower dimension makes the coupling more informative. We first test the prediction of Equation (13) that the drift share grows linearly in 1d\frac{1}{d}. As shown in Figure 3 (left), the drift norm barely changes across resolutions, so its share of the endpoint energy grows almost exactly as 1d\frac{1}{d}, from ∼0.15%\sim0.15\% at d=12288d=12288 to ∼2\sim2–2.5%2.5\% at d=768d=768. The same trend appears in the coupling statistics. The cosine similarity between each data point and its endpoint rises almost fourfold from 64×6464\times64 to 16×1616\times16 on CelebA and ImageNet, and doubles from 32×3232\times32 to 16×1616\times16 on CIFAR-10 (Table 5). This matches the endpoint decomposition. Since zz is independent of x0x_0, we have cos⁡(x0,x1)≈∥Δ(xϵ)∥dcos⁡(Δ(xϵ),x0)\cos(x_0,x_1)\approx\frac{\lVert\Delta(x_\epsilon)\rVert}{\sqrt{d}}\cos(\Delta(x_\epsilon),x_0), which scales as 1/d1/\sqrt{d} at fixed drift norm. Meanwhile, the alignment of the drift with its data point, cos⁡(Δ(xϵ),x0)\cos(\Delta(x_\epsilon),x_0), stays nearly constant for each dataset. The drift therefore carries similar information at every resolution, but this information becomes easier to extract as the noise shrinks relative to it. A linear readout from x1x_1 recovers 39% of the variance of held-out ImageNet images at 64×6464\times64 and 56% at 16×1616\times16, with the same trend on CelebA and CIFAR-10 (Table 6). Teacher couplings, in contrast, are strongly aligned and linearly decodable at every resolution, and even at 16×1616\times16 the closed-form couplings remain below them (Table 5 and Table 6).

Figure 3. Closed-form couplings against the dimension. Left: Share of the endpoint’s squared norm carried by the drift, E∥Δ(xϵ)∥2/E∥x1∥2\mathbb{E}\lVert\Delta(x_\epsilon)\rVert^2/\mathbb{E}\lVert x_1\rVert^2 with Δ(xϵ)=x1−z\Delta(x_\epsilon)=x_1-z. The dashed lines show ∥Δ(xϵ)∥2/d\lVert\Delta(x_\epsilon)\rVert^2/d with the drift norm held fixed at its measured value. Each dataset follows this scaling closely. Right: Gap score GG (Equation (14)) of the closed-form students at 1 and 2 Euler steps. G=0G=0 means the closed-form student is no better than the random one, and G=1G=1 means it matches the student trained on teacher couplings. πPF\pi^{\mathrm{PF}} improves as the dimension shrinks on all datasets, but does not reach πFM\pi^{\mathrm{FM}}. Note that GG is not comparable across datasets. On ImageNet, the random student cannot fit its 1.28M random training pairs, which inflates GG (Appendix § 16).

datasetpxddπRandom\pi^{Random}πPF\pi^{PF}πFM\pi^{FM}cos⁡(Δ(xϵ),x0)\cos(\Delta(x_{\epsilon}), x_0)
CIFAR-10167680.0000.1200.3350.852
CIFAR-102417280.0000.0820.2840.861
CIFAR-103230720.0000.0620.2300.867
CelebA167680.0000.1030.2420.703
CelebA3230720.0000.0540.1710.730
CelebA64122880.0000.0280.1080.746
ImageNet167680.0000.1350.3600.826
ImageNet3230720.0000.0700.2710.850
ImageNet64122880.0000.0360.2040.864

Table 5. Alignment between data points and their coupled endpoints. Cosine similarity cos⁡(x0,x1)\cos(x_0, x_1) for the three couplings, averaged over data points. The last column is the alignment of the closed-form drift with its data point, cos⁡(Δ(xϵ),x0)\cos(\Delta(x_{\epsilon}), x_0).

datasetpxddπRandom\pi^{Random} trainπRandom\pi^{Random} testπPF\pi^{PF} trainπPF\pi^{PF} testπFM\pi^{FM} trainπFM\pi^{FM} test
CIFAR-10167680.010.000.540.520.860.85
CIFAR-102417280.03-0.010.520.480.840.82
CIFAR-103230720.05-0.010.520.450.810.78
CelebA167680.000.000.600.600.840.84
CelebA3230720.02-0.010.550.530.790.78
CelebA64122880.09-0.030.560.450.800.74
ImageNet167680.000.000.560.560.850.85
ImageNet3230720.020.000.510.480.820.81
ImageNet64122880.09-0.020.520.390.810.75

Table 6. Linear read-out of the data point from the coupled endpoint. Held-out and training R2R^2 of a ridge regression x0−xˉ≈W⊤x1x_0-\bar{x}\approx W^\top x_1, i.e. the fraction of the data variance a linear map recovers from the endpoint x1x_1 (0: predicts the mean image, 1: exact recovery). Fitted on the first 40000 (CIFAR-10) or 100000 (CelebA, ImageNet) pairs, tested on the next 10000.

More informative couplings produce better students. We next test whether the more informative coupling translates into better students. Since FID values are not directly comparable across resolutions, we measure the gap of the closed-form student relative to the random and FM students at the same resolution,

G=FIDrnd−FIDPFFIDrnd−FIDFM,(14)G=\frac{\mathrm{FID}_{\mathrm{rnd}}-\mathrm{FID}_{\mathrm{PF}}}{\mathrm{FID}_{\mathrm{rnd}}-\mathrm{FID}_{\mathrm{FM}}}, \tag*{(14)}

so that G=0G=0 corresponds to the random student and G=1G=1 to the student trained on teacher couplings. As shown in Figure 3 (right), GG increases consistently as dd decreases (FIDs are in Table 4). At a single step, it more than doubles on CIFAR-10 and rises by roughly 60% on CelebA and 20% on ImageNet. Since GG depends on the random baseline, only its trend within a dataset is meaningful, not its absolute level (Appendix § 16). Two aspects of our setup could confound the results. Downsampling changes the image content along with the dimension, and with a fixed network, the number of parameters per dimension grows as dd decreases. However, neither accounts for the trend. When we compress CIFAR-10 with a learned autoencoder instead of downsampling it, GG again grows as dd decreases, from 0.14 at d=1024d = 1024 to 0.49 at d=256d = 256 (Appendix § 17.§ 1). Likewise, when we scale the network width to keep the number of parameters per dimension fixed, GG still doubles from 32 to 16 px (Appendix § 17.§ 2). A lower FID could, in principle, also come from students that merely memorise their training pairs. To test whether they actually use the drift, we construct coupling endpoints from held-out images. On these, πPF\pi^{PF} students reconstruct 33%–44% of the image variance on CIFAR-10, 47%–61% on CelebA and 40%–51% on ImageNet, again improving as dd decreases (Table 7). The students therefore exploit the information in the drift rather than only their training pairs. Nor do they reproduce training images from fresh noise. Even at 16 px, where the CIFAR-10 network has more parameters than the training set has pixel values, neither the students nor their teachers copy training images (Appendix § 17.§ 3). However, the gap to the teacher couplings does not close. πFM\pi^{FM} students reconstruct more than 92% of the held-out variance at every resolution, and their FID approaches that of the teacher at low resolution (Table 3 and Table 4). Teacher endpoints thus retain most of the information about their data point in a form that is easy to extract. In short, lowering the dimension makes closed-form couplings better teachers, but not competitive with pretrained ones.

datasetpxdd64 steps128 steps
CIFAR-10167682.62.1
CIFAR-102417283.43.1
CIFAR-103230723.63.1
CelebA167681.51.2
CelebA3230722.62.7
CelebA64122884.43.4
ImageNet167681.81.3
ImageNet3230723.83.2
ImageNet64122888.17.2

Table 3. FID of the flow-matching teachers used to generate πFM\pi^{FM}. FID of the best checkpoint at 64 and 128 Euler steps.

πRandom\pi^{Random}πRandom\pi^{Random}πPF\pi^{PF}πPF\pi^{PF}πFM\pi^{FM}πFM\pi^{FM}
datasetpx1 step2 steps1 step2 steps1 step2 steps
CIFAR-101661.0±1.961.0 \pm1.938.1±0.738.1 \pm0.731.3±1.231.3 \pm1.220.2±1.120.2 \pm1.14.6±0.04.6 \pm0.03.9±0.03.9 \pm0.0
CIFAR-102472.7±0.972.7 \pm0.949.1±0.649.1 \pm0.651.5±1.551.5 \pm1.536.5±1.136.5 \pm1.19.3±0.29.3 \pm0.28.3±0.38.3 \pm0.3
CIFAR-103282.9±0.782.9 \pm0.760.7±6.160.7 \pm6.168.0±2.368.0 \pm2.346.5±2.546.5 \pm2.515.2±0.515.2 \pm0.512.4±0.312.4 \pm0.3
CelebA1638.3±1.038.3 \pm1.029.2±0.829.2 \pm0.821.3±0.121.3 \pm0.114.7±0.114.7 \pm0.12.0±0.12.0 \pm0.12.0±0.02.0 \pm0.0
CelebA3243.1±2.143.1 \pm2.129.6±1.129.6 \pm1.129.1±1.829.1 \pm1.820.2±0.720.2 \pm0.74.8±0.04.8 \pm0.06.4±0.26.4 \pm0.2
CelebA6470.4±0.470.4 \pm0.450.7±0.450.7 \pm0.453.1±0.253.1 \pm0.238.4±0.038.4 \pm0.012.3±0.612.3 \pm0.68.3±0.38.3 \pm0.3
ImageNet16430.1158.097.544.22.31.6
ImageNet32417.8184.1145.159.09.16.3
ImageNet64415.5175.6164.681.229.722.3

Table 4. FID of the students presented in Figure 3. One- and two-step FID of the students trained on πRandom\pi^{Random}, πPF\pi^{PF} and πFM\pi^{FM}, from which the gap score GG (Equation (14)) is computed. For CIFAR-10 and CelebA we report mean ±\pm standard deviation over three seeds.

datasetpXp_XddπRandom\pi^{Random} trainπRandom\pi^{Random} testπPF\pi^{PF} trainπPF\pi^{PF} testπFM\pi^{FM} trainπFM\pi^{FM} test
CIFAR-10167680.96−0.54-0.540.970.441.000.98
CIFAR-102417280.95−0.51-0.510.960.401.000.96
CIFAR-103230720.92−0.59-0.590.920.330.990.92
CelebA167680.95−0.52-0.520.970.611.000.99
CelebA3230720.89−0.50-0.500.920.521.000.98
CelebA64122880.85−0.47-0.470.920.470.990.98
ImageNet167680.62−0.35-0.350.820.511.000.99
ImageNet3230720.17−0.06-0.060.780.430.990.98
ImageNet64122880.02−0.01-0.010.650.400.990.97

Table 7. Reconstruction of held-out images. R2=1−E∥x^0−x0∥2/VR^2 = 1 - \mathbb{E}\lVert\hat{x}_0 - x_0\rVert^2/V of the one-step student output x^0\hat{x}_0 against the paired image, on the student’s own training couplings (train) and on couplings built the same way for held-out images (test). VV is the variance of the training images, so R2=0R^2 = 0 is the accuracy of predicting the mean image. Test couplings of πPF\pi^{PF} integrate the closed-form field of the training images together with the held-out images.

Conclusion

We asked whether the couplings used for rectified flows, usually obtained from a pretrained teacher, can instead be constructed directly from the closed-form probability-flow field of the empirical data distribution. In our experiments, they cannot: students trained on these closed-form pairs perform much closer to those trained on random pairings than to those trained on teacher-generated pairs. The problem is not that the two constructions associate noise with different data points. They largely agree on which regions of noise space correspond to which data points, but choose very different endpoints within those regions. Our analytical endpoint decomposition explains why: the endpoint produced by the closed-form flow is dominated by the Gaussian noise used to initialise the trajectory, with only a small data-dependent drift. The relative contribution of this drift scales as log⁡Nd\frac{\log N}{d}, so the coupling becomes less informative as the dimension grows unless the dataset size increases exponentially. We confirm empirically that closed-form couplings improve in lower-dimensional spaces. Even there, however, they remain substantially worse teachers than pretrained couplings.

This suggests that the key advantage of a learned teacher is not the coarse assignment of noise-space regions to data points, but how it selects an endpoint within each region. Understanding what determines that selection, and whether the same structure can be recovered without training a teacher, is the main open question left by our results.

Acknowledgments and Disclosure of Funding

Part of this work was supported by the Helmholtz International Lab Causal Cell Dynamics (InterLabs-0029) through grant funding from the Initiative and Networking Fund of the Hermann von Helmholtz-Association Deutscher Forschungszentren e.V. The authors gratefully acknowledge the Gauss Centre for Supercomputing e.V. (www.gauss-centre.eu) for funding this project by providing computing time through the John von Neumann Institute for Computing (NIC) on the GCS Supercomputer JUPITER — JUWELS [31] at Jülich Supercomputing Centre (JSC). Furthermore, the authors acknowledge the computational resources provided by the National High Performance Computing Centre (www.nhr.kit.edu). TH acknowledges support from G-Research. This research was enabled in part by compute resources provided by Mila (mila.quebec) and the Digital Research Alliance of Canada (alliancecan.ca). KN acknowledges support from IVADO and the Canada First Research Excellence Fund.

References

Building intuition for the closed-form PF-ODE

In this section we build intuition for the closed-form field vPFv^{\mathrm{PF}} and the couplings it induces. We start with a two-dimensional toy dataset, where the partition of prior space can be drawn directly and we can visualise the entire distribution. We then show that, for datasets with reflection symmetries, both this partition and the noise-free endpoints can be characterised exactly. Finally, we show that the same picture carries over to ImageNet-64.

Many-to-one contraction. Figure 4 colours every point of the prior by the data point it converges to when vPFv^{\mathrm{PF}} is integrated backwards from t=1t = 1 to t=0t = 0. Because the empirical dynamics contract as t→0t \to0, each data point attracts not a single point but an entire region of prior space, and together these regions tile the plane. The regions differ considerably in shape and size. Datapoints at the two ends of the arc own large regions that extend far into the tails, while those near its centre are confined to thin bands through the origin. Their probability mass under the prior, however, is identical. The exact flow transports N(0,I)\mathcal{N}(0, I) onto the uniform empirical distribution, so every region carries mass 1N\frac{1}{N}.

The partition also shows why the closed-form coupling is ambiguous. Every point in the region of x0ix_0^i is a valid endpoint for x0ix_0^i, and which one we obtain is decided entirely by the initial condition. Starting from xϵ=αϵx0+σϵzx_\epsilon= \alpha_\epsilon x_0 + \sigma_\epsilon z, the endpoint can land anywhere in the region, depending on zz. Removing the noise, i.e. starting from xϵ=αϵx0x_\epsilon= \alpha_\epsilon x_0 (λ=0\lambda= 0), instead yields a single deterministic endpoint per data point. These endpoints trace out a contracted version of the data arc and preserve the order of the data points along it, but they cluster around the origin and no longer follow the Gaussian prior. For symmetric datasets, both the partition and this contraction can be characterised exactly, as we show next.

Symmetric datasets. For a general dataset, the regions in Figure 4 have no simple description. They depend on the global arrangement of the data. If the dataset has reflection symmetries, however, we can characterise them exactly. Both results below rely on the same observation (Lemma C.1). If an orthogonal map QQ maps the dataset onto itself, i.e. the dataset is symmetric under QQ, then vtPFv_t^{\mathrm{PF}} commutes with QQ, vtPF(Qx)=QvtPF(x)v_t^{\mathrm{PF}}(Qx) = Qv_t^{\mathrm{PF}}(x). Consequently, the set of points that QQ leaves unchanged is invariant under the flow: a trajectory that starts there stays there. We apply this to two families of reflections. If the dataset is invariant under reflections across the bisector hyperplanes between neighbouring data points, these hyperplanes become barriers that no trajectory can cross, so each Voronoi cell of the data is mapped to its own data point (Proposition A.1). If it is invariant under reflections across the lines through individual data points, the noise-free PFPF-ODEODE (λ=0\lambda= 0) trajectory of each data point stays on its own line, so its endpoint is a rescaled copy of that data point (Proposition A.2). Formal statements and proofs are given in Appendix C.1.

The first family describes the partition of prior space. If all data points have the same norm, every bisector hyperplane passes through the origin, and the Voronoi cells of the data are cones.

Proposition A.1 (Invariant Voronoi cells, informal). Let all data points have the same norm, and let ViV_i denote the Voronoi cell of x0ix_0^i. Assume that the dataset is symmetric under the reflection across the bisector hyperplane of any two neighbouring data points, i.e. data points whose Voronoi cells share a facet. Then vtPFv_t^{PF} is tangent to every such bisector, and a trajectory of the PFPF-ODEODE that lies in ViV_i at one time lies in ViV_i at all times.

Because the cells are cones, ViV_i is also the Voronoi cell of the conditional mean αtx0i\alpha_t x_0^i at every time tt, so the partition does not move along the path. Since the backward flow ends at a data point and x0ix_0^i is the only data point in ViV_i, the region of prior space that the closed-form flow assigns to x0ix_0^i is exactly its Voronoi cone. In the symmetric case, the closed-form semi-discrete coupling is therefore simply a nearest-neighbour assignment of prior samples to data points (Figure 5). In terms of Proposition 5.1, a noisy trajectory that starts in ViV_i must also end in ViV_i, so the drift Δ(xϵ)\Delta(x_\epsilon) is exactly the correction that moves the injected noise zz into the cell of its data point. Where in the cone the endpoint lands is still decided by zz, so the ambiguity of the coupling persists even in this simple geometry.

Figure 5. Prior-space partition of a symmetric dataset. Left: Eight equally spaced data points on a circle, which satisfy the assumptions of Propositions A.1 and A.2. Right: The partition of the prior induced by the closed-form PFPF-ODEODE, computed as in Figure 4. The regions coincide with the Voronoi cones of the data (dashed lines), as predicted by Proposition A.1. The noise-free endpoints (λ=0\lambda= 0, stars) lie on the rays through their data points (dotted lines), i.e. they are rescaled copies of the data points, as predicted by Proposition A.2.

The second family describes the noise-free endpoints, marked by stars in Figure 4 and Figure 5.

Proposition A.2 (Invariant lines, informal). Assume that the dataset is symmetric under the reflection across the line through a data point x0ix_0^i. Then the noise-free trajectory starting from xϵ=αϵx0ix_\epsilon= \alpha_\epsilon x_0^i (λ=0\lambda= 0) stays on this line, and its endpoint is a rescaled copy of the data point, Δ(αϵx0i)=cx0i\Delta(\alpha_\epsilon x_0^i) = c x_0^i for some c∈Rc \in\mathbb{R}.

In two dimensions, this is the ordinary reflection across the line through x0ix_0^i. In higher dimensions, the required symmetry is more complex and much harder to satisfy. Both assumptions hold, for instance, for NN equally spaced points on a circle and for the points {±ek}k=1d\{\pm e_k\}_{k=1}^d in Rd\mathbb{R}^d. For such datasets, the closed-form flow maps the noise-free initial condition of every data point along its own ray, so the PFPF-ODEODE (λ=0\lambda= 0) endpoints preserve the configuration of the data up to a rescaling of each point (Figure 5). Natural images are not symmetric in this sense, and we cannot expect the statement to hold exactly. However, Figure 6 and Figure 7 below show that it holds approximately. The PFPF-ODEODE (λ=0\lambda= 0) endpoints are faint copies of their data points, with an average cosine similarity cos⁡(Δ(αϵx0),x0)\cos(\Delta(\alpha_\epsilon x_0), x_0) between 0.80 and 0.93 across datasets (Table 2).

Visualising the endpoint decomposition on ImageNet-64

Figure 7. Visualising the endpoint decomposition on ImageNet-64 (λ=1\lambda= 1, ϵ=10−3\epsilon= 10^{-3}). The endpoints x1\mathbf{x}_1 resemble pure Gaussian noise, yet subtracting the injected noise λz\lambda\mathbf{z} reveals the drift Δ(xϵ)\Delta(\mathbf{x}_\epsilon), a faint, structure-preserving copy of each data point that closely matches the PFPF-ODEODE (λ=0\lambda= 0) endpoint Δ(αϵx0)\Delta(\alpha_\epsilon\mathbf{x}_0) of a noise-free integration. The two drifts differ mainly in magnitude, with Δ(xϵ)≈1.2 Δ(αϵx0)\Delta(\mathbf{x}_\epsilon) \approx1.2\,\Delta(\alpha_\epsilon\mathbf{x}_0), while their directions agree closely, with an average cosine similarity of 0.95 (Table 2).

Noisy versus noise-free initialisation. Figure 6 shows that the two initialisations behave analogously on ImageNet-64. We integrate vPFv^{PF} forward either from the noisy initial condition xϵ=αϵx0+σϵz\mathbf{x}_\epsilon= \alpha_\epsilon\mathbf{x}_0 + \sigma_\epsilon\mathbf{z} used to construct πPF\pi^{PF}, or from the noise-free initial condition xϵ=αϵx0\mathbf{x}_\epsilon= \alpha_\epsilon\mathbf{x}_0 used by PFPF-ODEODE (λ=0\lambda= 0). Note that the latter is the mean of the conditional path in Equation (1) at time ϵ\epsilon. Although the two initial conditions are visually indistinguishable, their endpoints differ enormously. The noisy initialisation produces endpoints that look like pure Gaussian noise, with no visible trace of the image. The noise-free initialisation produces a scaled down version of the image that is easily matched to its data point. This is the high-dimensional counterpart of Figure 4. The injected noise decides where within its region the endpoint lands and, in doing so, hides the data point, while the noise-free endpoint retains the structure of the data point but is far too small in norm to be a sample from the prior.

Recovering the drift. Finally, Figure 7 connects the two initialisations through the endpoint decomposition of Proposition 5.1. Since x1=λz+Δ(xϵ)\mathbf{x}_1 = \lambda\mathbf{z} + \Delta(\mathbf{x}_\epsilon) holds exactly, subtracting the injected noise from the Gaussian-looking endpoints removes precisely the component that hides the image. What remains is the drift Δ(xϵ)\Delta(\mathbf{x}_\epsilon), again a faint copy of the data point, and nearly indistinguishable from the PFPF-ODEODE (λ=0\lambda= 0) endpoint Δ(αϵx0)\Delta(\alpha_\epsilon\mathbf{x}_0). The information about x0\mathbf{x}_0 is therefore not lost during the noisy integration. It is merely buried under the injected noise, which carries more than 99% of the endpoint energy. Moreover, the drift looks almost the same whether or not noise was injected, which is the approximation stated in Equation (7). This is a high-dimensional effect. In two dimensions, the drift must carry every endpoint back into the region of its data point (Figure 4, and exactly into its Voronoi cone for symmetric data, Proposition A.1) and therefore depends more strongly on the noise, and even across our image resolutions, cos⁡(Δ(xϵ),x0)\cos(\Delta(\mathbf{x}_\epsilon), \mathbf{x}_0) drops slightly as the dimension decreases (Table 5). We quantify these observations in Appendix D.1.

Comparing teacher and closed-form couplings

In this section, we provide the details for the two observations from § 4 that compare teacher and closed-form couplings. First, the teacher couplings are consistent with the closed-form field, since teacher endpoints lie in the closed-form region of their own data point. Second, within this region, the teacher selects a much narrower set of endpoints than the closed-form flow.

We take 10000 random teacher couplings (x0i,x1i)(x_0^i, x_1^i) per dataset and integrate each endpoint x1ix_1^i backwards through the closed-form field of the full training set, from t=1t = 1 to t=10−3t = 10^{-3}. A coupling counts as a match if the nearest training image of the result is x0ix_0^i, which by chance happens with probability 1N\frac{1}{N}. The matching accuracy is 99.95% on CIFAR-10, 99.94% on CelebA and 100% on ImageNet. Almost every teacher endpoint therefore lies in the region that the closed-form flow assigns to its own data point.

To compare how the two couplings select endpoints within these regions, we integrate the first 2000 training images of each dataset from three independent noise draws through each teacher, using its best checkpoint and 250 Euler steps, and compare them with closed-form couplings of the same images (Table 1). The two couplings behave in opposite ways. For the teacher, the injected noise explains only 4–8% of the endpoint variance. Two endpoints of the same data point are nearly identical, with a cosine similarity of 0.92–0.96, and the nearest neighbour of every teacher endpoint among the endpoints of another noise draw belongs to the same data point. For the closed-form flow, the injected noise explains more than 99% of the endpoint variance. Two endpoints of the same data point are almost orthogonal, like two independent Gaussian samples, and the nearest neighbour identifies the data point only at chance level.

Proofs

In this section, we provide the proofs of our propositions.

Proposition C.1 (Endpoint decomposition). Let (xt)t∈[ϵ,1](x_t)_{t\in[\epsilon,1]} solve dxtdt=vtPF(xt)\frac{\mathrm{d}x_t}{\mathrm{d}t}=v_t^{PF}(x_t) with initial condition xϵ=αϵx0+λϵzx_\epsilon=\alpha_\epsilon x_0+\lambda_\epsilon z, for arbitrary x0,z∈Rdx_0,z\in\mathbb{R}^d, λ∈R\lambda\in\mathbb{R} and ϵ∈(0,1)\epsilon\in(0,1). Then, with γt\gamma_t and mtm_t as in Equation (5),

x1=α1x0+λσ1z+σ1Δ(xϵ),Δ(xϵ):=∫ϵ1γtσt(mt(xt)−x0) dt.(15)x_1=\alpha_1x_0+\lambda\sigma_1z+\sigma_1\Delta(x_\epsilon), \qquad\Delta(x_\epsilon):=\int_\epsilon^1\frac{\gamma_t}{\sigma_t}\left(m_t(x_t)-x_0\right)\,\mathrm{d}t. \tag*{(15)}

For the schedules of this paper (αt=1−t, σt=t)(\alpha_t=1-t,\ \sigma_t=t) this reads x1=λz+Δ(xϵ)x_1=\lambda z+\Delta(x_\epsilon) with Δ(xϵ)=−∫ϵ1t−2(mt(xt)−x0) dt\Delta(x_\epsilon)=-\int_\epsilon^1t^{-2}\left(m_t(x_t)-x_0\right)\,\mathrm{d}t. In particular: if the posterior remains concentrated on the generating datapoint, i.e. mt(xt)=x0m_t(x_t)=x_0 for all t∈[ϵ,1]t\in[\epsilon,1] (e.g. for N=1N=1), then Δ(xϵ)=0\Delta(x_\epsilon)=0 and the flow returns the injected noise exactly, x1=λzx_1=\lambda z.

Proof. Since σt>0\sigma_t>0 on [ϵ,1][\epsilon,1] and the weights wtiw_t^i are smooth there, vtPFv_t^{PF} is smooth on [ϵ,1]×Rd[\epsilon,1]\times\mathbb{R}^d and the trajectory (xt)t∈[ϵ,1](x_t)_{t\in[\epsilon,1]} is well defined. Recall the split of the field into its linear and nonlinear parts, Equation (5),

vtPF(x)=dlog⁡σtdtx+γtmt(x),γt=dαtdt−αtdlog⁡σtdt.(16)v_t^{PF}(x)=\frac{\mathrm{d}\log\sigma_t}{\mathrm{d}t}x+\gamma_t m_t(x), \qquad\gamma_t=\frac{\mathrm{d}\alpha_t}{\mathrm{d}t}-\alpha_t\frac{\mathrm{d}\log\sigma_t}{\mathrm{d}t}. \tag*{(16)}

Define the conditional trajectory yt:=αtx0+λσtzy_t:=\alpha_tx_0+\lambda\sigma_tz. Direct substitution gives

dytdt=dαtdtx0+λdσtdtz=dlog⁡σtdtyt+γtx0,(17)\frac{\mathrm{d}y_t}{\mathrm{d}t}=\frac{\mathrm{d}\alpha_t}{\mathrm{d}t}x_0+\lambda\frac{\mathrm{d}\sigma_t}{\mathrm{d}t}z=\frac{\mathrm{d}\log\sigma_t}{\mathrm{d}t}y_t+\gamma_tx_0, \tag*{(17)}

i.e. yty_t solves the same ODE as xtx_t with mt(⋅)m_t(\cdot) replaced by the constant x0x_0. We will derive an ODE for the deviation of the true trajectory from this reference and integrate it, this will give us the endpoint decomposition. Let et:=xt−yte_t := x_t-y_t denote the deviation. Subtracting the two equations,

detdt=dlog⁡σtdtet+γt(mt(xt)−x0),eϵ=xϵ−yϵ=0.(18)\frac{d e_t}{d t}=\frac{d\log\sigma_t}{d t}e_t+\gamma_t\left(m_t(x_t)-x_0\right),\qquad e_\epsilon=x_\epsilon-y_\epsilon=0. \tag*{(18)}

Using the integrating factor method (with the factor exp⁡(−∫tdlog⁡σsds ds)=1σt\exp\left(-\int^t \frac{d\log\sigma_s}{d s}\,d s\right)=\frac{1}{\sigma_t}), we get

ddt(etσt)=γtσt(mt(xt)−x0).(19)\frac{d}{d t}\left(\frac{e_t}{\sigma_t}\right)=\frac{\gamma_t}{\sigma_t}\left(m_t(x_t)-x_0\right). \tag*{(19)}

Integrating from ϵ\epsilon to 11 with eϵ=0e_\epsilon=0 yields e1=σ1∫ϵ1γtσt(mt(xt)−x0)dt=σ1Δ(xϵ)e_1=\sigma_1\int_\epsilon^1\frac{\gamma_t}{\sigma_t}\left(m_t(x_t)-x_0\right)d t=\sigma_1\Delta(x_\epsilon). Hence x1=y1+e1=α1x0+λσ1z+σ1Δ(xϵ)x_1=y_1+e_1=\alpha_1x_0+\lambda\sigma_1z+\sigma_1\Delta(x_\epsilon).

If mt(xt)=x0m_t(x_t)=x_0 for all t∈[ϵ,1]t\in[\epsilon,1], the integrand vanishes, so Δ(xϵ)=0\Delta(x_\epsilon)=0 and x1=λzx_1=\lambda z. For N=1N=1 we have mt(x)=x0m_t(x)=x_0 identically, so this case applies with no further assumption.

Proposition C.2 (Exact cutoff decomposition). Let x0∼p^0x_0\sim\hat{p}_0 and z∼N(0,I)z\sim\mathcal{N}(0,I) independently, and let (xt)t∈[ϵ,1](x_t)_{t\in[\epsilon,1]} solve the empirical probability-flow ODE from xϵ=αϵx0+σϵzx_\epsilon=\alpha_\epsilon x_0+\sigma_\epsilon z. Assume that αt\alpha_t and σt\sigma_t are continuously differentiable, σt>0\sigma_t>0 on [ϵ,1][\epsilon,1], α1=0\alpha_1=0, σ1=1\sigma_1=1, and that ut:=αtσt≥0u_t:=\frac{\alpha_t}{\sigma_t}\geq0 is nonincreasing, and set ωt:=−dutdt≥0\omega_t:=-\frac{d u_t}{d t}\geq0. Let xˉ:=1N∑j=1Nx0j\bar{x}:=\frac{1}{N}\sum_{j=1}^N x_0^j and V:=1N∑j=1N∥x0j−xˉ∥2V:=\frac{1}{N}\sum_{j=1}^N\lVert x_0^j-\bar{x}\rVert^2 be the mean and total variance of the dataset, and write ∥Y∥L2:=(Ex0,z[∥Y∥2])1/2\lVert Y\rVert_{L^2}:=\left(\mathbb{E}_{x_0,z}[\lVert Y\rVert^2]\right)^{1/2} for a random vector Y=Y(x0,z)Y=Y(x_0,z). For a cutoff τ∈(ϵ,1)\tau\in(\epsilon,1), let rτ:=1στ(xτ−(ατx0+στz))r_\tau:=\frac{1}{\sigma_\tau}\left(x_\tau-(\alpha_\tau x_0+\sigma_\tau z)\right) denote the normalised deviation of the trajectory from its conditional path at time τ\tau. Then, for every τ∈(ϵ,1)\tau\in(\epsilon,1), the endpoint drift from Equation (15) satisfies the pathwise identity

Δ(xϵ)=uτ(x0−xˉ)+rτ−∫τ1ωt(mt(xt)−xˉ)dt,(20)\Delta(x_\epsilon)=u_\tau(x_0-\bar{x})+r_\tau-\int_\tau^1\omega_t\left(m_t(x_t)-\bar{x}\right)d t, \tag*{(20)}

and consequently

∥Δ(xϵ)∥L2≤∥rτ∥L2+2uτV.(21)\lVert\Delta(x_\epsilon)\rVert_{L^2}\leq\lVert r_\tau\rVert_{L^2}+2u_\tau\sqrt{V}. \tag*{(21)}

Proof. Let yt=αtx0+σtzy_t=\alpha_t x_0+\sigma_t z and et:=xt−yte_t:=x_t-y_t be the reference path and the deviation from the proof of Proposition C.1 with λ=1\lambda=1. Using

γtσt=ddt(αtσt)=−ωt,(22)\frac{\gamma_t}{\sigma_t}=\frac{d}{d t}\left(\frac{\alpha_t}{\sigma_t}\right)=-\omega_t, \tag*{(22)}

we can rewrite the ODE for the relative deviation, Equation (19), as

ddt(etσt)=−ωt(mt(xt)−x0).(23)\frac{d}{d t}\left(\frac{e_t}{\sigma_t}\right)=-\omega_t\left(m_t(x_t)-x_0\right). \tag*{(23)}

At t=1t=1, the reference path equals y1=zy_1=z, because α1=0\alpha_1=0 and σ1=1\sigma_1=1. Hence e1σ1=x1−z=Δ(xϵ)\frac{e_1}{\sigma_1}=x_1-z=\Delta(x_\epsilon). At t=τt=\tau, we have eτστ=rτ\frac{e_\tau}{\sigma_\tau}=r_\tau by definition of rτr_\tau. Integrating the previous identity from τ\tau to 11 therefore gives

Δ(xϵ)−rτ=e1σ1−eτστ=−∫τ1ωt(mt(xt)−x0)dt.(24)\Delta(x_\epsilon)-r_\tau=\frac{e_1}{\sigma_1}-\frac{e_\tau}{\sigma_\tau}=-\int_\tau^1\omega_t\left(m_t(x_t)-x_0\right)d t. \tag*{(24)}

To isolate the dependence on x0x_0, we write mt(xt)−x0=(mt(xt)−xˉ)−(x0−xˉ)m_t(x_t)-x_0=(m_t(x_t)-\bar{x})-(x_0-\bar{x}) inside the integral. The second term does not depend on tt, and the multiplier ωt\omega_t integrates to

∫τ1ωt dt=−∫τ1dutdt dt=uτ−u1=uτ,(25)\int_\tau^1\omega_t\,d t=-\int_\tau^1\frac{d u_t}{d t}\,d t=u_\tau-u_1=u_\tau, \tag*{(25)}

since u1=α1σ1=0u_1=\frac{\alpha_1}{\sigma_1}=0. Solving for Δ(xϵ)\Delta(x_\epsilon) yields the exact decomposition

Δ(xϵ)=uτ(x0−xˉ)+rτ−∫τ1ωt(mt(xt)−xˉ)dt.(26)\Delta(x_\epsilon)=u_\tau(x_0-\bar{x})+r_\tau-\int_\tau^1\omega_t\left(m_t(x_t)-\bar{x}\right)d t. \tag*{(26)}

It remains to bound the three terms in L2L^2. Since xϵ=αϵx0+σϵzx_\epsilon=\alpha_\epsilon x_0+\sigma_\epsilon z, the initial state has distribution pϵp_\epsilon. The probability-flow ODE therefore has marginal distribution ptp_t at every time, so xt∼ptx_t\sim p_t. Moreover, mtm_t is the posterior mean under the Gaussian path. A posterior mean cannot have more variance than the variable it estimates, and the law of total variance gives

EX∼pt[∥mt(X)−xˉ∥2]≤V.(27)\mathbb{E}_{X\sim p_t}\left[\lVert m_t(X)-\bar{x}\rVert^2\right]\le V. \tag*{(27)}

Applying the L2L^2 triangle inequality, also to the time integral,

∥Δ(xϵ)∥L2≤uτ∥x0−xˉ∥L2+∥rτ∥L2+∫τ1ωt(EX∼pt[∥mt(X)−xˉ∥2])12 dt≤uτV+∥rτ∥L2+V∫τ1ωt dt=∥rτ∥L2+2uτV.(28)\begin{aligned} \lVert\Delta(x_\epsilon)\rVert_{L^2}\le u_\tau\lVert x_0-\bar{x}\rVert_{L^2}+\lVert r_\tau\rVert_{L^2}+\int_\tau^1\omega_t\left(\mathbb{E}_{X\sim p_t}\left[\lVert m_t(X)-\bar{x}\rVert^2\right]\right)^{\frac{1}{2}}\,dt \\ &\le u_\tau\sqrt{V}+\lVert r_\tau\rVert_{L^2}+\sqrt{V}\int_\tau^1\omega_t\,dt \\ &=\lVert r_\tau\rVert_{L^2}+2u_\tau\sqrt{V}. \tag*{(28)} \end{aligned}

This proves the claim.

Symmetry results

Both propositions from Appendix § 9 follow from one lemma, which we prove first before stating the propositions formally. If an orthogonal map permutes the data points, the closed-form field commutes with it, and the set of points it leaves fixed is invariant under the flow. Throughout, we assume that αt\alpha_t and σt\sigma_t are continuously differentiable.

Lemma C.1 (Equivariance). Let Q∈Rd×dQ\in\mathbb{R}^{d\times d} be orthogonal and assume that QQ permutes the dataset, i.e. there is a permutation π\pi of {1,…,N}\{1,\ldots,N\} with Qx0k=x0π(k)Qx_0^k=x_0^{\pi(k)} for all kk. Let FQ:={x:Qx=x}F_Q:=\{x:Qx=x\}. Then

(i) vtPF(Qx)=QvtPF(x)v_t^{PF}(Qx)=Qv_t^{PF}(x) for all x∈Rdx\in\mathbb{R}^d and t∈(0,1]t\in(0,1];

(ii) vtPF(x)∈FQv_t^{PF}(x)\in F_Q for all x∈FQx\in F_Q and t∈(0,1]t\in(0,1];

(iii) if (xt)t∈[ϵ,1](x_t)_{t\in[\epsilon,1]} solves dxtdt=vtPF(xt)\frac{dx_t}{dt}=v_t^{PF}(x_t) and xs∈FQx_s\in F_Q for some s∈[ϵ,1]s\in[\epsilon,1], then xt∈FQx_t\in F_Q for all t∈[ϵ,1]t\in[\epsilon,1].

Proof. (i) Since QQ is orthogonal, ∥Qx−αtx0π(k)∥=∥Qx−αtQx0k∥=∥x−αtx0k∥\lVert Qx-\alpha_tx_0^{\pi(k)}\rVert=\lVert Qx-\alpha_tQx_0^k\rVert=\lVert x-\alpha_tx_0^k\rVert, hence N(Qx∣αtx0π(k),σt2I)=N(x∣αtx0k,σt2I)\mathcal{N}(Qx\mid\alpha_tx_0^{\pi(k)},\sigma_t^2I)=\mathcal{N}(x\mid\alpha_tx_0^k,\sigma_t^2I) for every kk. Because π\pi is a bijection, the normalising sums in Equation (3) agree as well, so wtπ(k)(Qx)=wtk(x)w_t^{\pi(k)}(Qx)=w_t^k(x). For the posterior mean of Equation (5), this gives

mt(Qx)=∑k=1Nwtπ(k)(Qx)x0π(k)=∑k=1Nwtk(x)Qx0k=Qmt(x),(29)m_t(Qx)=\sum_{k=1}^{N}w_t^{\pi(k)}(Qx)x_0^{\pi(k)}=\sum_{k=1}^{N}w_t^k(x)Qx_0^k=Qm_t(x), \tag*{(29)}

and therefore vtPF(Qx)=dlog⁡σtdtQx+γtQmt(x)=QvtPF(x)v_t^{PF}(Qx)=\frac{d\log\sigma_t}{dt}Qx+\gamma_t Qm_t(x)=Qv_t^{PF}(x).

(ii) For x∈FQx\in F_Q, (i) gives QvtPF(x)=vtPF(Qx)=vtPF(x)Qv_t^{PF}(x)=v_t^{PF}(Qx)=v_t^{PF}(x).

(iii) Since vtPFv_t^{PF} is continuous in tt and Lipschitz in xx on [ϵ,1][\epsilon,1], solutions of the PFPF-ODEODE exist and are unique. Now let xs∈FQx_s\in F_Q and consider the reflected trajectory yt:=Qxty_t:=Qx_t. By (i),

dytdt=QvtPF(xt)=vtPF(Qxt)=vtPF(yt),ys=Qxs=xs.(30)\frac{dy_t}{dt}=Qv_t^{PF}(x_t)=v_t^{PF}(Qx_t)=v_t^{PF}(y_t),\qquad y_s=Qx_s=x_s. \tag*{(30)}

Thus, yty_t is itself a solution of the PFPF-ODEODE, and at time ss it passes through the same point as xtx_t, since xs∈FQx_s\in F_Q. Two solutions that agree at one time agree at all times, so yt=xty_t=x_t. Hence Qxt=xtQx_t=x_t, i.e. xt∈FQx_t\in F_Q, for all t∈[ϵ,1]t\in[\epsilon,1].

Proposition C.3 (Invariant Voronoi cells, formal version of Proposition A.1). Let the data points {x0i}i=1N⊂Rd\{x_0^i\}_{i=1}^{N}\subset\mathbb{R}^d be distinct with ∥x0i∥=∥x0j∥\lVert x_0^i\rVert=\lVert x_0^j\rVert for all i,ji,j, and let Vi:={x:∥x−x0i∥≤∥x−x0j∥ ∀j}V_i:=\{x:\lVert x-x_0^i\rVert\le\lVert x-x_0^j\rVert\ \forall j\} be the Voronoi cell of x0ix_0^i. Call jj a neighbour of ii if ViV_i and VjV_j share a facet, i.e. Vi∩VjV_i\cap V_j has dimension d−1d-1. For such a pair, let nij:=x0j−x0in_{ij}:=x_0^j-x_0^i, let Hij:={x:⟨nij,x⟩=0}H_{ij}:=\{x:\langle n_{ij},x\rangle=0\} be the bisector hyperplane, and let

Qij:=I−2nijnijT∥nij∥2(31)Q_{ij}:=I-2\frac{n_{ij}n_{ij}^{\mathsf{T}}}{\lVert n_{ij}\rVert^2} \tag*{(31)}

be the reflection across it. Assume that, for every pair of neighbours (i,j)(i,j), QijQ_{ij} permutes the dataset, i.e. {Qijx0k}k=1N={x0k}k=1N\{Q_{ij}x_0^k\}_{k=1}^{N}=\{x_0^k\}_{k=1}^{N}. Then, for every pair of neighbours, every t∈(0,1]t\in(0,1] and every x∈Hijx\in H_{ij},

⟨nij,vtPF(x)⟩=0,(32)\langle n_{ij},v_t^{PF}(x)\rangle=0, \tag*{(32)}

i.e. vtPFv_t^{PF} is tangent to HijH_{ij}. Consequently, every Voronoi cell is invariant under the flow. If (xt)t∈[ϵ,1](x_t)_{t\in[\epsilon,1]} solves dxtdt=vtPF(xt)\frac{dx_t}{dt}=v_t^{PF}(x_t) and xs∈Vix_s\in V_i for some s∈[ϵ,1]s\in[\epsilon,1], then xt∈Vix_t\in V_i for all t∈[ϵ,1]t\in[\epsilon,1].

Proof. Geometry of the cells. For any i≠ji \ne j,

∥x−x0i∥2−∥x−x0j∥2=2⟨nij,x⟩+∥x0i∥2−∥x0j∥2=2⟨nij,x⟩,(33)\lVert x-x_0^i\rVert^2-\lVert x-x_0^j\rVert^2=2\langle n_{ij},x\rangle+\lVert x_0^i\rVert^2-\lVert x_0^j\rVert^2=2\langle n_{ij},x\rangle, \tag*{(33)}

using the equal norms. Hence HijH_{ij} is the bisector of x0ix_0^i and x0jx_0^j, and Vi={x:⟨nij,x⟩≤0 ∀j≠i}V_i=\{x:\langle n_{ij},x\rangle\le0\ \forall j\ne i\} is a full-dimensional convex polyhedral cone. Like every such polyhedron, it is already described by the half-spaces of its facets, Vi={x:⟨nij,x⟩≤0 for all neighbours j of i}V_i=\{x:\langle n_{ij},x\rangle\le0\text{ for all neighbours }j\text{ of }i\}.

Tangency. QijQ_{ij} is orthogonal, and its fixed-point set is HijH_{ij}. Since QijQ_{ij} permutes the dataset, Lemma C.1(ii) gives vtPF(x)∈Hijv_t^{\mathrm{PF}}(x)\in H_{ij}, i.e. ⟨nij,vtPF(x)⟩=0\langle n_{ij},v_t^{\mathrm{PF}}(x)\rangle=0, for all x∈Hijx\in H_{ij}. Moreover, by Lemma C.1(iii), a trajectory that lies in HijH_{ij} at one time lies in HijH_{ij} at all times.

Invariance of the cells. Let xs∈Vix_s\in V_i, fix a neighbour jj of ii, and consider g(t):=⟨nij,xt⟩g(t):=\langle n_{ij},x_t\rangle, which is continuous with g(s)≤0g(s)\le0. If gg vanishes at some time, the trajectory lies in HijH_{ij} at that time, and therefore at all times, so g≡0g\equiv0. Otherwise, gg never vanishes and by continuity keeps the sign of g(s)<0g(s)<0. In both cases, g(t)≤0g(t)\le0 for all t∈[ϵ,1]t\in[\epsilon,1]. Since this holds for every neighbour jj, we have xt∈Vix_t\in V_i for all t∈[ϵ,1]t\in[\epsilon,1].

Proposition C.4 (Invariant lines, formal version of Proposition A.2). Fix a data point x0i≠0x_0^i\ne0 and let

Qi:=2x0i(x0i)⊤∥x0i∥2−I(34)Q_i:=2\frac{x_0^i(x_0^i)^\top}{\lVert x_0^i\rVert^2}-I \tag*{(34)}

be the reflection across the line span⁡(x0i)\operatorname{span}(x_0^i). Assume that QiQ_i permutes the dataset, i.e. {Qix0k}k=1N={x0k}k=1N\{Q_i x_0^k\}_{k=1}^N=\{x_0^k\}_{k=1}^N. Then vtPF(x)∈span⁡(x0i)v_t^{\mathrm{PF}}(x)\in\operatorname{span}(x_0^i) for every x∈span⁡(x0i)x\in\operatorname{span}(x_0^i) and t∈(0,1]t\in(0,1]. Consequently, the trajectory with noise-free initial condition xϵ=αϵx0ix_\epsilon=\alpha_\epsilon x_0^i (λ=0\lambda=0) satisfies xt=ctx0ix_t=c_t x_0^i for all t∈[ϵ,1]t\in[\epsilon,1] with scalars ctc_t. In particular, the drift of the noise-free integration is an exact rescaled copy of the data point,

Δ(αϵx0i)=c1x0i.(35)\Delta(\alpha_\epsilon x_0^i)=c_1x_0^i. \tag*{(35)}

If, in addition, all data points have the same norm and the dataset has mean zero, then ct>0c_t>0 for all t∈[ϵ,1]t\in[\epsilon,1].

Proof. QiQ_i is orthogonal, and Qix=xQ_i x=x holds if and only if xx equals its projection onto span⁡(x0i)\operatorname{span}(x_0^i), so its fixed-point set is span⁡(x0i)\operatorname{span}(x_0^i). Since QiQ_i permutes the dataset, Lemma C.1(ii) gives vtPF(x)∈span⁡(x0i)v_t^{\mathrm{PF}}(x)\in\operatorname{span}(x_0^i) for all x∈span⁡(x0i)x\in\operatorname{span}(x_0^i). Since xϵ=αϵx0i∈span⁡(x0i)x_\epsilon=\alpha_\epsilon x_0^i\in\operatorname{span}(x_0^i), Lemma C.1(iii) gives xt∈span⁡(x0i)x_t\in\operatorname{span}(x_0^i), i.e. xt=ctx0ix_t=c_t x_0^i, for all t∈[ϵ,1]t\in[\epsilon,1]. Finally, Proposition C.1 with λ=0\lambda=0, α1=0\alpha_1=0 and σ1=1\sigma_1=1 gives x1=Δ(αϵx0i)x_1=\Delta(\alpha_\epsilon x_0^i), hence Δ(αϵx0i)=c1x0i\Delta(\alpha_\epsilon x_0^i)=c_1x_0^i.

For the sign, note that with equal norms all weights at the origin are equal, w0k(0)=1/Nw_0^k(0)=1/N, so mt(0)m_t(0) is the data mean, which is 00. Hence vtPF(0)=0v_t^{\mathrm{PF}}(0)=0, and xt≡0x_t\equiv0 solves the PF-ODE. By uniqueness, no other trajectory reaches 00, so ctc_t cannot change sign, and cϵ=αϵ>0c_\epsilon=\alpha_\epsilon>0 gives ct>0c_t>0 for all t∈[ϵ,1]t\in[\epsilon,1].

Empirical investigation of the drift

This section contains the empirical measurements to support § 5. Appendix D.1 quantifies the two empirical properties of the drift on which the endpoint decomposition builds. First, the drift is small, and second, it depends only weakly on the injected noise. Appendix D.2 evaluates both terms of the bound in Proposition 5.2 along the trajectory, and Appendix D.3 tests the two predictions of the scaling argument in § 5.2. Unless stated otherwise, the empirical distribution is the full training set of each dataset, and we use ϵ=10−3\epsilon=10^{-3}.

The properties of the drift

Table 2 is based on three integrations per data point. Two start from independent noisy initial conditions (λ=1\lambda=1) and give the drifts Δ(xϵ)=x1−z\Delta(x_\epsilon)=x_1-z and Δ(xϵ′)=x1′−z′\Delta(x'_\epsilon)=x'_1-z'. The third starts from the noise-free initial condition αϵx0\alpha_\epsilon x_0 and gives the PFPF-ODEODE (λ=0\lambda=0) endpoint Δ(αϵx0)\Delta(\alpha_\epsilon x_0).

The drift is small. On all three datasets, the drift carries well below 1% of the endpoint energy (Table 2(a). In absolute terms, its norm is nearly the same across datasets, between 4.0 and 4.7 (Figure 8), although the dimension of CelebA and ImageNet-64 is four times that of CIFAR-10. The smaller share on these two datasets therefore reflects the larger dimension rather than a smaller drift. This is the 1d\frac{1}{d} dependence that § 5.2 predicts and § 6 exploits.

The drift depends only weakly on the noise. The least-squares fit in Table 2(a) shows that, on every dataset, the drift is an amplified copy of the noise-free drift, with gain g≈1.2g\approx1.2, and the remainder carries only 12–14% of the drift energy. The cosine similarities in Table 2(b) show that this remainder is the part that changes with the noise. Each noisy drift is closely aligned with the noise-free drift, with cos⁡(Δ(xϵ),Δ(αϵx0))=0.94\cos(\Delta(x_\epsilon),\Delta(\alpha_\epsilon x_0))=0.94–0.950.95. If every noisy drift consists of the noise-free drift plus an independent noise-specific part, Δ(xϵ)\Delta(x_\epsilon) and Δ(xϵ′)\Delta(x'_\epsilon) would share only the noise-free drift, and their cosine similarity should be approximately the square of this value, 0.88–0.91. The measured cos⁡(Δ(xϵ),Δ(xϵ′))\cos(\Delta(x_\epsilon),\Delta(x'_\epsilon)) of 0.89–0.91 matches this closely. Therefore, Δ(αϵx0)\Delta(\alpha_\epsilon x_0) captures what the noisy drifts have in common, and only the small remainder varies with the noise. This supports the approximation in Equation (7), up to the gain gg.

The drift carries the data point. Finally, both drifts are well aligned with the data point itself (Table 2(b)). The noise-free drift is consistently the better aligned of the two, as expected if the noise-specific remainder carries little information about x0x_0. The alignment is lowest on CelebA, but even there the drift retains much of the structure of its data point. Figure 2 and Figure 7 show what these numbers look like on ImageNet-64.

The bound on the drift investigated

Figure 8 evaluates the two terms of Proposition 5.2 as a function of the cutoff τ\tau, together with the measured drift.

The two terms trade off along the trajectory. For small τ\tau, the trajectory tracks the conditional path of its data point almost perfectly. The tracking error ∥rτ∥L2\lVert r_\tau\rVert_{L^2} stays between 10−310^{-3} and 10−110^{-1}, and the bound is dominated by the SNR term 2ατστV2\frac{\alpha_\tau}{\sigma_\tau}\sqrt{V}, which is still large because the SNR is high. Only late in the trajectory, from τ≈0.55\tau\approx0.55 on CIFAR-10, τ≈0.8\tau\approx0.8 on CelebA and τ≈0.75\tau\approx0.75 on ImageNet-64, does the tracking error rise as the trajectory leaves the conditional path. By then, the SNR term has decayed, and the bound approaches the measured drift. At τ=1\tau=1 the bound is exact, since α1=0\alpha_1=0 and r1=x1−z=Δ(xϵ)r_1=x_1-z=\Delta(x_\epsilon).

This supports the step from Proposition 5.2 to Equation (12) in § 5.2. Before the trajectory stops tracking its data point, the tracking error is negligible, and the drift is controlled by the SNR term alone. At such τ\tau, the bound exceeds the measured drift by roughly an order of magnitude, so it captures how the drift scales rather than its exact value.

Scaling of the drift contribution

Figure 9 tests the two predictions of § 5.2, specifically the collapse criterion in Equation (11) and the resulting log⁡N\sqrt{\log N} growth of the drift in Equation (12).

Collapse time. To measure the collapse time, we follow the conditional path xt=αtx0i+σtzx_t = \alpha_t x_0^i + \sigma_t z for 10000 data points and evaluate the posterior weights wtj(xt)w_t^j(x_t) of Equation (3) against the full training set. The collapse time τ\tau is the first tt at which the mean maximum weight max⁡jwtj(xt)\max_j w_t^j(x_t) falls below 12\frac{1}{2}. For the prediction, we solve Equation (11) for τ\tau, using the measured data variance VV at each resolution and fitting a single constant CC per dataset. For our linear schedule, this gives τ=(1+Clog⁡N/V)−1\tau= \left(1+\sqrt{C\log N/V}\right)^{-1}. Lowering the dimension reduces VV and with it the distance penalty that keeps the posterior on its data point, so the posterior collapses earlier. The collapse time drops from τ≈0.9\tau\approx0.9 at d=12288d = 12288 to τ≈0.65\tau\approx0.65–0.70.7 at d=768d = 768, and the prediction follows this trend on every dataset with CC between 2.95 and 3.62 (Figure 9, left).

Drift magnitude. To vary NN, we restrict the empirical distribution to random subsets of the training set, from N=102N = 10^2 up to the full dataset. Over four orders of magnitude in NN, the mean drift norm grows only from about 2.5 to 4.6 and is well described by alog⁡N+ba\sqrt{\log N}+b with a≈1.3a \approx1.3 on all three datasets (Figure 9, right). The three curves nearly coincide, even though the datasets differ in content and CIFAR-10 has a quarter of the dimension of the other two. This is consistent with Equation (12), in which the drift depends on NN but not on dd, and with the nearly constant drift norm across resolutions in Figure 3. Extrapolating the fits shows how little more data would help. Even at N=108N = 10^{8}, the drift norm would reach only about 5.3, which at d=12288d = 12288 is still only about 0.2% of the endpoint energy.

Experimental Details

This section describes our experimental setup. We first describe the datasets and how we obtain the different resolutions, then the flow-matching teachers, the construction of the three coupling sets, and finally the training and evaluation of the rectified-flow students.

Datasets and resolutions

We use the training sets of CIFAR-10 [32], CelebA [33] (the standard training partition of the aligned-and-cropped release) and ImageNet-1k [34], all unconditional, without augmentation, with pixel values scaled to [−1,1][-1,1]. The dimension d=3r2d = 3r^{2} is varied by changing the resolution rr. All resolutions of a dataset are generated from the same source images, so neighbouring resolutions differ in nothing but resolution.

Downsampling. For CelebA, we take the aligned 178 × 218 images, crop the central 140 × 140 pixels and resize them with bilinear interpolation to 16, 32 and 64 px. ImageNet images are centre-cropped to a square and box-downsampled to 64 px, following Chrabaszcz et al. [ [35]. The 32 and 16 px versions are 2 × 2 and 4 × 4 box averages of these 64 px images. CIFAR-10 is only available at 32 px. We obtain the 16 px version as a 2 × 2 box average, and the 24 px version, a non-integer factor of 4/3, with PIL’s area filter, which generalises the box average to non-integer factors. Across all resolutions, the per-pixel standard deviation stays between 0.48 and 0.55, so we use the same preconditioning constant σdata=0.5\sigma_{\mathrm{data}} = 0.5 throughout.

Flow-matching teachers

All teachers are trained with the flow-matching objective on the linear path αt=1−t\alpha_t = 1 - t, σt=t\sigma_t = t, with the EDM preconditioning of Karras et al. [ [36] around the network (σdata=0.5\sigma_{\mathrm{data}} = 0.5), the EDM loss weight, and timesteps tt drawn from a logit-normal distribution in log-SNR (mean 2.5, standard deviation 2.5). Optimisation uses Adam, an exponential moving average of the weights with rate 0.9999, and a linear learning-rate warm-up over the first 20000 steps. Model selection uses the checkpoint with the lowest FID-50k at 32 Euler steps. Teacher quality (FID-50k, best checkpoint) is reported in Table 3 at 64 and 128 Euler steps. To have a fair comparison between the teacher model and the PF-ODE, we train the model without data augmentation.

CIFAR-10 and CelebA. The network is the DDPM++ UNet of Song et al. [ [3] following the exact configuration used in Karras et al. [ [36] for CIFAR-10. At 16, 24 and 32 px, this network is used unchanged (only the spatial size of its feature maps changes). However, at 64 px for CelebA we use EDM’s FFHQ-64 variant with channel multipliers [1,2,2,2][1, 2, 2, 2] and attention at 16 × 16. Teachers train for 300000 steps at global batch size 512 with learning rate 10−310^{-3} (2×10−42 \times10^{-4} with gradient clipping at norm 1 for the 64 px teacher, following EDM’s 64-px setting).

ImageNet. The network is the ADM UNet of Dhariwal and Nichol [ [37] following Karras et al. [ [36] ImageNet-64 configuration. The identical network is used at 16, 32 and 64 px, so the three ImageNet rungs share their capacity exactly. Only the attention resolutions scale with the image. Teachers train with learning rate 10−410^{-4} at global batch size 1024 in bf16 mixed precision for 400000 steps (600000 at 64 px).

Couplings

Every coupling set pairs each of the NN training images x0ix_0^i with one noise endpoint x1ix_1^i and the students are then trained on these fixed pairs only.

Random couplings. To construct πRandom\pi^{Random} we draw x1i∼N(0,I)x_1^i \sim\mathcal{N}(0, I) and couple them independently with x0ix_0^i.

Closed-form couplings. For πPF\pi^{PF} we start from xϵi∼N(αϵx0i,σϵ2I)x_\epsilon^i \sim\mathcal{N}(\alpha_\epsilon x_0^i,\sigma_\epsilon^2 I) with ϵ=10−3\epsilon= 10^{-3} and λ=1\lambda= 1 and integrate the closed-form probability-flow field of the empirical distribution of all NN training images from t=ϵt = \epsilon to t=1t = 1 with an adaptive Dormand–Prince solver (relative tolerance 10−810^{-8}, absolute tolerance 10−1010^{-10}). The field in Equation (3) is evaluated exactly against all NN training images, without the nearest-neighbour approximations used in Scarvelis et al. [ [17], Bertrand et al. [ [15]. The injected noise zz is then used to measure the drift Δ=x1−z\Delta= x_1 - z. Each field evaluation touches the whole training set, which makes it memory intensive. Therefore, for CelebA and ImageNet the NN trajectories are integrated in chunks against the full set of NN guiding points held on the GPU.

Teacher couplings. To obtain πFM\pi^{FM} we integrate the same initial states x0ix_0^i forward through the flow-matching teacher’s EMA weights with Euler steps on a uniform time grid to t=1t = 1. For each dataset we use 128 Euler steps from ϵ=10−3\epsilon= 10^{-3} and the teacher’s best checkpoint.

Rectified-flow students

For every coupling set one rectified-flow student is trained on the fixed pairs (x0i,x1i)(x_0^i, x_1^i) with the flow-matching loss along the straight interpolant. The timesteps tt are drawn from the tilted-uniform distribution of Lee et al. [ [13] with tilt a=4a = 4 (antithetic, symmetric). We use Adam with 5000 warm-up steps and an EMA of rate 0.9999. The student has the same architecture as the teacher of its resolution. On CIFAR-10 and CelebA the students use learning rate 2×10−42 \times10^{-4}, global batch 512 and the EDM loss weight. The students on random and closed-form couplings train for 200000 steps and students on teacher couplings for 100000 steps, after which their FID no longer improves. On ImageNet all students use learning rate 10−410^{-4}, global batch 512, no loss weighting, bf16 mixed precision and 200000 steps. Students never see augmented data, since the coupling fixes the pairing.

Repeats. To estimate the robustness of our findings, we repeat experiments on CIFAR-10 and CelebA for three seeds. Each repeat k∈{1,2,3}k \in\{1, 2, 3\} draws fresh random couplings and fresh injected noise for the closed-form couplings with seed kk and trains its students with seed 41+k41 + k. The teacher couplings are the same file in all three repeats, so their repeats differ only in the student’s initialisation and data order.

Evaluation. All FIDs are estimated using 50000 samples against the Inception-v3 pool-3 statistics of the full training set at the same resolution. Students are sampled with the Euler method on a uniform time grid with k∈{1,2,4,8,16,32}k \in\{1, 2, 4, 8, 16, 32\} steps from t=1t = 1 to t=10−3t = 10^{-3}, using the EMA weights, and every saved checkpoint (every 25000 steps) is evaluated separately at each kk. Teachers are sampled the same way with 64 and 128 Euler steps.

The gap score GG in Equation (14) compares the three students of a setting at equal kk. G=0G = 0 means the closed-form student is no better than the random one, and G=1G = 1 that it matches the student trained on the teacher’s couplings. Since the three students of a setting share the architecture, optimisation settings and data, GG is comparable across the rungs of one dataset, however across datasets it is not, because the score depends heavily on the random baseline. For further discussion see Appendix H.

Additional results on reducing the dimension

In this section we provide the results from Section 6 in more detail. We first confirm that the teachers used to generate πFM\pi^{FM} are strong, and then report the FIDs of the three students, from which the gap score is computed. We then show that lowering the dimension makes closed-form couplings more informative, both in how closely each endpoint aligns with its data point and in how much of the data point can be recovered from it, whereas teacher couplings are highly informative at every resolution. Finally, we show that students trained on closed-form couplings use this information beyond their training pairs.

The teachers are strong. All teachers reach low FIDs (Table 3), between 1.2 and 3.4 at 128 Euler steps in every setting except ImageNet-64, the hardest one, where they reach 7.2. These numbers are competitive given that we have not used data augmentation or class-conditional generation.

FIDs behind the gap score. Table 4 shows the one- and two-step FIDs from which the gap score in Figure 3 is computed. The ordering is the same in every resolution. In each setting, the πFM\pi^{FM} student is best, the πRandom\pi^{Random} student worst, and the πPF\pi^{PF} student in between. Absolute FIDs, however, are not comparable across resolutions, which is why Section 6 compares the students through GG. At the lowest resolution, the πFM\pi^{FM} students come close to their teachers, with one-step FIDs of 2.0–4.6 against 2.1–2.1 for the teachers at 128 steps. On ImageNet, the random students reach FIDs above 400 at one step. They cannot fit their 1.28M random training pairs and simply predict the dataset mean, which inflates GG on this (Appendix H).

Lower dimension makes closed-form couplings more informative. Table 5 measures how closely each endpoint aligns with its data point. For random couplings, the cosine similarity is zero, as expected for independent high-dimensional vectors. For closed-form couplings, it is small but grows as the dimension decreases, almost fourfold from 64 to 16 px on CelebA and ImageNet and twofold from 32 to 16 px on CIFAR-10, in line with the 1d\frac{1}{\sqrt{d}} scaling derived in § 6. The last column shows why. The alignment of the drift with its data point stays roughly constant across resolutions, so the drift carries similar information at every resolution, and only its weight relative to the injected noise changes. Teacher couplings, in contrast, are strongly aligned at every resolution, roughly four to six times more than closed-form couplings at the native resolution. Even at 16 px, closed-form couplings remain less aligned than teacher couplings at the highest resolution, except on CelebA, where the two are about equal.

The information becomes easier to extract. Table 6 asks how much of the data point a linear map can recover from the endpoint. Random endpoints carry no information, and the held-out R2R^2 is zero. For closed-form couplings, the held-out R2R^2 rises as the dimension decreases, from 0.45 to 0.52 on CIFAR-10, from 0.45 to 0.60 on CelebA and from 0.39 to 0.56 on ImageNet. Since the drift keeps its alignment with the data point, this gain comes from the noise masking less of it. Teacher couplings reach 0.74–0.85 at every resolution, above the best closed-form value in any setting. At the highest resolutions, the training R2R^2 noticeably exceeds the held-out one, e.g. 0.52 against 0.39 for closed-form couplings on ImageNet-64. The ridge regression starts to overfit, as the non-zero training R2R^2 of random couplings in the same setting shows, so the held-out values are the relevant ones.

Students use the drift beyond their training pairs. Finally, Table 7 tests whether the students exploit this information, or merely memorise their training pairs. We build couplings for held-out images in the same way and compare the one-step output of each student with the paired image. Students trained on closed-form couplings reconstruct 33–61% of the held-out variance, and again more at lower dimensions. This is similar to what the linear read-out recovers from the same kind of endpoints, whereas students trained on teacher couplings reconstruct at least 92% at every resolution.

Random students behave differently. On CIFAR-10 and CelebA, they reproduce their training pairs with R2≥0.85R^2 \ge0.85, but reach a negative R2R^2 on held-out images, because they output plausible images that are unrelated to the paired data point. On ImageNet, they fit their training pairs only partially at 16 px and hardly at all at 64 px, which we discuss in Appendix § 16.

Drift energy and learnability

The endpoint decomposition gives us a simple picture. The closed-form flow returns the injected noise plus a drift that, in our experiments, is small, resembles its originating image, and depends only weakly on the injected noise. This small drift can contain much information about the image, but it might be difficult to recover this structure from the endpoint, where the drift is mixed with noise. In the following, we examine how the drift share in Equation (13) relates to the accessibility of this structure, and clarify its usefulness and limitations as a proxy for coupling learnability. As a simple measure of learnability, we use the fraction of image variance recovered by the best linear predictor from the endpoint. This does not characterise what every nonlinear model can learn, but on Table 6 and Table 7 we saw that our empirical linear readouts follow the reconstruction values achieved by the closed-form students.

Let uk\mathbf{u}_k be the principal directions of the data, with variances λk\lambda_k, total variance V=∑kλk>0V=\sum_k \lambda_k>0, and variance shares μk=λk/V\mu_k=\lambda_k/V. Write sk=uk⊤(x0−xˉ)s_k=\mathbf{u}_k^\top(x_0-\bar{x}) for the kk-th data component, and ek=uk⊤x1e_k=\mathbf{u}_k^\top x_1 and δk=uk⊤Δ(xϵ)\delta_k=\mathbf{u}_k^\top\Delta(x_{\epsilon}) for the endpoint and drift components in the same basis. Since x1∼N(0,I)x_1\sim\mathcal{N}(0,I), the endpoint components are uncorrelated and have unit variance. For each λk>0\lambda_k > 0, let

rk=∑j=1dCov⁡(sk,ej)2λk=∑j=1dCov⁡(sk,δj)2λk∈[0,1](36)r_k = \sum_{j=1}^{d} \frac{\operatorname{Cov}(s_k,e_j)^2}{\lambda_k} = \sum_{j=1}^{d} \frac{\operatorname{Cov}(s_k,\delta_j)^2}{\lambda_k} \in[0,1] \tag*{(36)}

be the fraction of its variance recovered by the best linear predictor from the full endpoint x1x_1. The second equality holds because the injected noise zz is independent of x0x_0, so only the drift contributes to the covariance. Thus, rk=1r_k = 1 means that the component can be recovered perfectly, while rk=0r_k = 0 means that a linear predictor does no better than predicting its mean.

The optimal linear reconstruction recovers

Rlin2=∑kμkrk=∥Cov⁡(x0,Δ(xϵ))∥F2V(37)R_{\mathrm{lin}}^2 = \sum_k \mu_k r_k = \frac{\left\lVert\operatorname{Cov}(x_0,\Delta(x_\epsilon)) \right\rVert_F^2}{V} \tag*{(37)}

of the total data variance, where the second equality follows from the invariance of the Frobenius norm under the orthogonal change of basis. Throughout, sums over data components include only λk>0\lambda_k > 0; components with zero variance are constant, are already reconstructed by the mean, and do not contribute to the covariance.

Since the injected noise is independent of the data, all linear dependence between the endpoint and the image comes from the drift. This gives the following connection.

Proposition G.1 (Drift energy and linear recovery). Let x1=z+Δ(xϵ)x_1 = z + \Delta(x_\epsilon) be the endpoint of the closed-form flow from Proposition C.1. Let Δ^\widehat{\Delta} be the best linear predictor of Δ(xϵ)\Delta(x_\epsilon) from x0−xˉx_0 - \bar{x}, and define

η:=E∥Δ^∥2E∥Δ(xϵ)∥2∈[0,1](38)\eta:= \frac{\mathbb{E}\lVert\widehat{\Delta}\rVert^2}{\mathbb{E}\lVert\Delta(x_\epsilon)\rVert^2} \in[0,1] \tag*{(38)}

as the fraction of drift energy linearly explained by the data, with η=0\eta= 0 if the drift energy is zero. Then

∑krk=ηE∥Δ(xϵ)∥2≤E∥Δ(xϵ)∥2.(39)\sum_k r_k = \eta\mathbb{E}\lVert\Delta(x_\epsilon)\rVert^2 \le\mathbb{E}\lVert\Delta(x_\epsilon)\rVert^2. \tag*{(39)}

Proof. Let s~k=skλk\tilde{s}_k = \frac{s_k}{\sqrt{\lambda_k}}. These variables have zero mean, unit variance, and are mutually uncorrelated. The probability-flow ODE preserves the marginals, so x1∼N(0,I)x_1 \sim\mathcal{N}(0,I). Since Cov⁡(x1)=I\operatorname{Cov}(x_1) = I, the best linear predictor of s~k\tilde{s}_k from x1x_1 explains the variance

rk=∥Cov⁡(x1,s~k)∥2=∥Cov⁡(Δ(xϵ),s~k)∥2,(40)r_k = \left\lVert\operatorname{Cov}(x_1,\tilde{s}_k) \right\rVert^2 = \left\lVert\operatorname{Cov}(\Delta(x_\epsilon),\tilde{s}_k) \right\rVert^2, \tag*{(40)}

where the second equality follows from the independence of zz and x0x_0.

Both x1x_1 and zz have zero mean, so the drift does as well. Its best linear predictor from the centred data is

Δ^=∑kCov⁡(Δ(xϵ),s~k)s~k.(41)\widehat{\Delta} = \sum_k \operatorname{Cov}(\Delta(x_\epsilon),\tilde{s}_k)\tilde{s}_k. \tag*{(41)}

By orthonormality,

E∥Δ^∥2=∑k∥Cov⁡(Δ(xϵ),s~k)∥2=∑krk.(42)\mathbb{E}\lVert\widehat{\Delta}\rVert^2 = \sum_k \lVert\operatorname{Cov}(\Delta(x_\epsilon),\widetilde{s}_k)\rVert^2 = \sum_k r_k. \tag*{(42)}

The identity follows from the definition of η\eta, and the inequality follows because linear prediction cannot explain more than the total drift energy.

The proposition therefore connects linear recovery to two properties of the drift. First, its energy and second, how much of it is linearly related to the data. These two determine how well the data components can be recovered collectively. However, reconstruction quality also depends on which components are recovered. Recovering components that contain much of the image variance contributes more than recovering those that contain little. Based on this, we now examine when the drift-energy share is a useful proxy for learnability and when it can be unreliable.

Which components are recovered matters. The connection to the drift share is particularly simple when the data variance is spread equally across all dd directions. In this case, each component contributes the same amount to reconstruction, so μk=1/d\mu_k = 1/d. Using Proposition G.1 and E∥x1∥2=d\mathbb{E}\lVert x_1\rVert^2 = d, we obtain

Rlin2=1d∑krk=ηE∥Δ(xϵ)∥2E∥x1∥2.(43)R^2_{\mathrm{lin}} = \frac{1}{d}\sum_k r_k = \eta\frac{\mathbb{E}\lVert\Delta(x_\epsilon)\rVert^2}{\mathbb{E}\lVert x_1\rVert^2}. \tag*{(43)}

Hence, for isotropic data, the fraction of image variance recovered by a linear predictor is exactly the drift-energy share multiplied by η\eta, the fraction of drift energy linearly explained by the data.

For images, however, some components contain much more variance than others. Recovering these components improves reconstruction more than recovering components with little variance. The readout therefore depends not only on the drift energy, but also on which components remain readable through the noise. This is captured by the weights μk\mu_k in Equation (37). Directions with no data variance demonstrate this issue particularly clearly. They contribute neither to the covariance nor to VV in Equation (37), so they leave Rlin2R^2_{\mathrm{lin}} unchanged, but they still count towards the ambient dimension dd and therefore lower the drift-energy share without changing learnability. The weights μk\mu_k account for this distinction, whereas normalisation by dd alone does not. Therefore, for general data, the drift-energy share alone does not determine reconstruction quality. The same sum ∑krk\sum_k r_k can give different values of Rlin2R^2_{\mathrm{lin}}, depending on whether the readable components contain much or little of the image variance. We therefore examine which components remain readable in our experiments.

What this means for closed-form couplings. Under the scaling approximation in Equation (12), Proposition G.1 gives ∑krk=O(log⁡N)\sum_k r_k = O(\log N). In our experiments, E∥Δ(xϵ)∥2≈16\mathbb{E}\lVert\Delta(x_\epsilon)\rVert^2 \approx16–2222, which constrains linear recovery across components, but does not rule out substantial variation in the fraction of the readable components contain much of the image variance.

We examine which components remain readable using the directional correlation ρk2\rho_k^2 in Table 8, which gives a lower bound on rkr_k.

datasetpxddπPF\pi^{PF} directionsπPF\pi^{PF} varianceπFM\pi^{FM} directionsπFM\pi^{FM} variance
CIFAR-1016768558%23898%
CIFAR-10241728348%17494%
CIFAR-10323072347%15485%
CelebA16768670%25497%
CelebA323072664%42589%
CelebA6412288661%526284%
ImageNet16768661%24897%
ImageNet323072656%17390%
ImageNet6412288653%14885%

Table 8. Strong directional readability is concentrated in a few leading directions for closed-form couplings. We report the number of principal directions with ρk2>12\rho_k^2 > \frac{1}{2} and the share of data variance they hold, for closed-form and teacher endpoints. This threshold selects directions in which a linear predictor using the corresponding endpoint coordinate recovers more than half of the component’s variance.

ρk2=Cov⁡(sk,ϵk)2λk≤rk.(44)\rho_k^2 = \frac{\operatorname{Cov}(s_k,\epsilon_k)^2}{\lambda_k} \le r_k. \tag*{(44)}

For closed-form endpoints, only 3–6 leading directions have ρk2>1/2\rho_k^2 > 1/2, and these hold 47–70% of the data variance. Thus, strong directional readout is concentrated in a few leading components. The full linear predictor may also extract information from other endpoint coordinates, which is measured separately in Table 6.

These observations support the initial signal-to-noise picture: the drift retains image structure, but some components are much easier to extract from the noisy endpoint than others. At lower resolution, the strongly readable directions hold a larger share of image variance, while the drift energy barely changes. This is consistent with the improved linear readout and student reconstruction in Table 6 and Table 7, and explains why their improvement need not follow the 1d\frac{1}{d} growth of the drift share.

Fitting the training pairs and the random baseline

On ImageNet, the gap score GG is considerably higher than on CIFAR-10 and CelebA (Figure 3), although the closed-form couplings are no more informative there (Table 5 and Table 6). In this section, we show that the higher score is caused by the random baseline, which cannot fit its 1.28M training pairs.

Every student is trained on a fixed set of NN pairs, and a student that learns its coupling well will also fit these pairs, i.e. map each training endpoint x1ix_1^i close to its paired image x0ix_0^i. Fitting the training pairs is therefore not a sign of overfitting. For the teacher coupling, which is a smooth deterministic map from noise to data, fitting the pairs and generalising coincide, as the πFM\pi^{FM} students reconstruct 92%–99% of the variance of held-out images from unseen endpoints (Table 7). The situation is different for the random coupling. Since x1x_1 is independent of x0x_0, the optimal velocity at t=1t = 1 is E[x1−x0∣x1]=x1−xˉ\mathbb{E}[x_1 - x_0 \mid x_1] = x_1 - \bar{x}, so the optimal one-step student outputs the mean image xˉ\bar{x}. A random student can only improve on the mean image by fitting its finite set of pairs, and its quality therefore depends on whether NN pairs can be fitted within the training budget.

Probing pair fitting. We feed each student its own training endpoints and compare the one-step output with the paired image, which gives the training R2R^2 in Table 7. On CIFAR-10 and CelebA, both the random and the closed-form students fit their pairs, with R2R^2 between 0.85 and 0.97. On ImageNet, with 80 epochs over 1.28M pairs, the random student fits its pairs only partially at 16 px (R2=0.62R^2 = 0.62) and hardly at all at 32 and 64 px (0.17 and 0.02), and it has no information in its coupling to fall back on. At 64 px it is effectively a mean predictor, with an FID of 415, close to the FID of the mean image (446). The closed-form student does not fit its pairs perfectly either, but it still recovers 65–82% of the paired image’s variance and 40–51% on held-out images, consistent with it decoding the drift.

Control with 50k pairs. To isolate the effect of NN, we train the ImageNet-16 students on the first 50k pairs of each coupling, with the same network, batch size and number of steps (Table 9). The random student improves eightfold at one step and reaches the level of the CIFAR-10 random student, and GG drops to the CIFAR-10 level. The πFM\pi^{FM} student becomes worse. Its coupling already contains the full noise-to-data map, so the student’s quality is set by how densely the pairs sample this map, and 50k pairs sample it 25 times more sparsely than 1.28M.

datasetpairsπRandom\pi^{Random} 1 stepπRandom\pi^{Random} 2 stepsπPF\pi^{PF} 1 stepπPF\pi^{PF} 2 stepsπFM\pi^{FM} 1 stepπFM\pi^{FM} 2 stepsGG 1 stepGG 2 steps
ImageNet-161.28M430.1158.097.544.22.31.60.780.73
ImageNet-1650k53.828.235.325.112.111.60.440.18

Table 9. Controlling for the number of pairs. ImageNet-16 students trained on all pairs (80 epochs) and on the first 50k pairs (2048 epochs), with the same network, batch size and number of steps (best checkpoint).

In conclusion, the high GG on ImageNet reflects a random baseline that cannot fit its pairs, not closed-form couplings that are better than on the other datasets. Since the random students are similarly poor at every ImageNet resolution, with one-step FIDs between 416 and 430 (Table 4), the increase of GG as dd decreases remains meaningful within ImageNet.

Control experiments

The experiment in § 6 reduces the dimension by downsampling and keeps the network fixed across resolutions. This leaves possible confounders: downsampling changes the image content along with the dimension, a fixed network has more capacity per data dimension at low resolution, and at low resolution the teachers might memorise their training images. We address each of them in turn. Unless stated otherwise, all experiments follow the setup of Appendix E.

Reducing the dimension with a learned autoencoder

Downsampling removes fine image detail, which could change how hard the data is to learn independently of dd. To test if our results depend on the dimension reduction technique, we therefore reduce the dimension in a different way. We use a learned autoencoder that preserves the image content with high fidelity, and repeats the full construction of § 6 in its latent space.

Setup. On CIFAR-10, we train a KL-regularised autoencoder following Rombach et al. [ [38] with an L1, a perceptual (VGG16) and an adversarial (PatchGAN) loss, and a KL weight of 10−510^{-5}. The encoder downsamples the images by a factor of 4 to 8×88 \times8 latents with 4, 8 or 16 channels, i.e. d∈{256,512,1024}d \in\{256, 512, 1024\}. The reconstruction FID of the training set is at most 0.95 (Table 10), far below all student FIDs, so the autoencoder does not limit the comparison. In each latent space we train a flow-matching teacher, which reaches an FID of 7.4, 6.1 and 6.3 for d=256d = 256, 512 and 1024. The three coupling sets are constructed exactly as in pixel space. All FIDs are computed on decoded samples against the training images.

ddlatentrFIDπRandom\pi^{Random} 1 stepπRandom\pi^{Random} 2 stepsπPF\pi^{PF} 1 stepπPF\pi^{PF} 2 stepsπFM\pi^{FM} 1 stepπFM\pi^{FM} 2 stepsGG 1 stepGG 2 steps
2564×8×84 \times8 \times80.9569.341.743.832.417.717.70.490.39
5128×8×88 \times8 \times80.5571.140.349.434.920.820.90.430.28
102416×8×816 \times8 \times80.3771.256.064.848.224.123.10.140.24

Table 10. Dimension reduction with a learned autoencoder (CIFAR-10). One- and two-step FID of the students trained on πRandom\pi^{Random}, πPF\pi^{PF} and πFM\pi^{FM} in the latent space of the autoencoder (mean over three seeds), and the resulting gap score GG (Equation (14)). rFID is the FID of the autoencoder’s reconstructions of the training set.

Results. Table 10 shows the same trend as in pixel space. At one step, GG grows from 0.14 at d=1024d = 1024 to 0.49 at d=256d = 256, and at two steps from 0.24 to 0.39. Since the autoencoder alters the data in a very different way than downsampling, the improvement of the closed-form couplings at lower dimension does not depend on how the dimension is reduced. The absolute values of GG are not comparable to those in pixel space, since the latent distribution differs from the image distribution. As discussed in Appendix § 15, readability depends on how the data variance is distributed across directions and not on dd alone.

Equal capacity per dimension

In § 6, the students on CIFAR-10 use the same network at every resolution, so the number of parameters per data dimension grows from 18.4k at 32 px to 73.4k at 16 px. To remove this confounder, we reduce the base width of the network from 128 to 96 channels at 24 px and to 64 channels at 16 px, which gives 18.4k parameters per data dimension at every resolution, and retrain all three students on the same couplings. Table 11 shows that the smaller networks leave the students on teacher couplings essentially unaffected, but make the random and closed-form students worse in absolute FID. The largest effect is on the random student at 16 px, which worsens on fitting its training pairs (Appendix § 16). Nevertheless, the gap score changes by at most 0.04 (0.01 at two steps) and still doubles from 32 to 16 px. Lowering the dimension therefore improves the closed-form students also at equal capacity per dimension.

pxddwidthparametersper dimensionπRandom\pi^{Random}πPF\pi^{PF}πFM\pi^{FM}GG
32307212856.4M18.4k85.169.214.70.23
24172812856.4M32.6k74.350.89.50.36
2417289631.7M18.4k84.858.59.10.35
1676812856.4M73.4k62.933.04.70.51
167686414.1M18.4k93.952.14.70.47

Table 11. Students at equal capacity per dimension (CIFAR-10). One-step FID of the students trained on πRandom\pi^{Random}, πPF\pi^{PF} and πFM\pi^{FM} with the base width of the network either fixed (128) or scaled with dd, and the resulting gap score GG (Equation (14)).

Memorisation

At 16 px, the CIFAR-10 network has more parameters (56.4M) than the training set has pixel values (38.4M). A teacher that memorises its training images would, like the closed-form flow, send each noise to a training image. Its couplings would then trivially agree with the closed-form regions (Appendix § 10), and the comparison between teacher and closed-form couplings would lose its meaning. We test for this in two ways.

Teacher samples are novel. We generate 5000 samples from fresh noise with the 16 px teachers of CIFAR-10 and CelebA and compare each with its nearest training images. A copy of a training image would be much closer to it than to any other training image, so the ratio between the distances to the nearest and the second-nearest training image would be close to 0. Instead, the median ratio is 0.98 on CIFAR-10 and 0.97 on CelebA, and the samples are about as far from the training set as held-out test images. The students do not copy training images either, with median ratios of 0.97–0.98 on CIFAR-10 at 16 px.

Teacher couplings are insensitive to the injected noise. Since the couplings are constructed in the direction from data to noise, we also test the teacher’s map in this direction. A memorising teacher would have to spread the small perturbations around each training image over its entire region of prior space and would therefore be as sensitive to the injected noise as the closed-form flow. Repeating the noise-sensitivity analysis of Table 1 at every resolution shows the opposite (Table 12). At most 8.2% of the teacher’s endpoint variance is due to the injected noise, compared with at least 97.7% for the closed-form flow.

datasetpxddπFM\pi^{FM}πPF\pi^{PF}
CIFAR-10167682.1%98.2%
CIFAR-102417282.7%99.2%
CIFAR-103230726.1%99.6%
CelebA167681.2%98.2%
CelebA3230723.8%99.6%
CelebA64122888.2%99.9%
ImageNet167682.8%97.7%
ImageNet3230723.8%99.4%
ImageNet64122884.3%99.8%

Table 12. Noise share across resolutions. Fraction of the endpoint variance explained by the injected noise for teacher and closed-form couplings, measured as in Table 1 on the first 2000 training images with three independent noise draws (λ=1\lambda= 1, ϵ=10−3\epsilon= 10^{-3}).

Hence, the teachers do not memorise their training images. In addition, the region agreement of Appendix § 10 is measured at the original resolutions, where the networks have far fewer parameters than the training set has pixel values (0.37 per pixel value on CIFAR-10). The agreement therefore reflects the consistency of the teacher with the closed-form partition and not memorisation.

References

  1. [1]Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, 2015.
  2. [2]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.arxiv.org/abs/2006.11239
  3. [3]Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
  4. [4]Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. In International Conference on Learning Representations, 2023.arxiv.org/abs/2210.02747
  5. [5]Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Representations, 2022.arxiv.org/abs/2202.00512
  6. [6]Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In International Conference on Machine Learning, 2023.arxiv.org/abs/2303.01469
  7. [7]Michael Albergo, Nicholas M Boffi, and Eric Vanden-Eijnden. Stochastic interpolants: A unifying framework for flows and diffusions. Journal of Machine Learning Research, 26(209):1–80, 2025.
  8. [8]Nicholas Matthew Boffi, Michael Samuel Albergo, and Eric Vanden-Eijnden. Flow map matching with stochastic interpolants: A mathematical framework for consistency models. Transactions on Machine Learning Research, 2025. ISSN 2835-8856.arxiv.org/abs/2406.07507
  9. [9]Aram-Alexandre Pooladian, Heli Ben-Hamu, Carles Domingo-Enrich, Brandon Amos, Yaron Lipman, and Ricky T Chen. Multisample flow matching: Straightening flows with minibatch couplings. International Conference on Machine Learning, 2023.arxiv.org/abs/2304.14772
  10. [10]Alexander Tong, Kilian Fatrás, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector-Brooks, Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based generative models with minibatch optimal transport. Transactions on Machine Learning Research, pages 1–34, 2024.arxiv.org/abs/2302.00482
  11. [11]Alireza Mousavi-Hosseini, Stephen Zhang, Michal Klein, et al. Flow matching with semidiscrete couplings. In International Conference on Learning Representations, 2026.
  12. [12]Xingchao Liu, Chengyue Gong, et al. Flow straight and fast: Learning to generate and transfer data with rectified flow. In International Conference on Learning Representations, 2023.
  13. [13]Sangyun Lee, Zinan Lin, and Giulia Fanti. Improving the training of rectified flows. Advances in neural information processing systems, 37:63082–63109, 2024.
  14. [14]Beomsu Kim, Yu-Guan Hsieh, Michal Klein, Jong Chul Ye, Bahjat Kawar, James Thornton, et al. Simple reflow: Improved techniques for fast flow models. In International Conference on Learning Representations, 2025.
  15. [15]Quentin Bertrand, Anne Gagneux, Mathurin Massias, and Rémi Emonet. On the closed-form of flow matching: Generalization does not arise from target stochasticity. arXiv preprint arXiv:2506.03719, 2025.
  16. [16]Weiguo Gao and Ming Li. How do flow matching models memorize and generalize in sample data subspaces. npj Artificial Intelligence, 2026.
  17. [17]Christopher Scarvelis, Haitz Sáez de Ocáriz Borde, and Justin Solomon. Closed-form diffusion models. Transactions on Machine Learning Research, 2025.
  18. [19]Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, and qiang liu. Instaflow: One step is enough for high-quality diffusion-based text-to-image generation. In International Conference on Learning Representations, 2024.
  19. [20]Hanshu Yan, Xingchao Liu, Jiachun Pan, Jun Hao Liew, Qiang Liu, and Jiashi Feng. Perflow: Piecewise rectified flow as universal plug-and-play accelerator. Advances in Neural Information Processing Systems, 37, 2024.
  20. [21]Vansh Bansal, Saptarshi Roy, Alessandro Rinaldo, and Purnamrita Sarkar. On the convergence and straightness of rectified flow. In International Conference on Artificial Intelligence and Statistics, 2026.
  21. [22]Antonin Chambolle and Johannes Hertrich. Limit points of reflow with minibatch optimal transport. arXiv preprint arXiv:2608.07042, 2026.
  22. [24]Haoming Liu, Jinnuo Liu, Yanhao Li, Liuyang Bai, Yunkai Ji, Yuanhe Guo, Shenji Wan, and Hongyi Wen. From navigation to refinement: Revealing the two-stage nature of flow-based diffusion models through oracle velocity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2649–2658, 2026.arxiv.org/abs/2512.02826
  23. [25]Giulio Biroli, Tony Bonnaire, Valentin De Bortoli, and Marc Mézard. Dynamical regimes of diffusion models. Nature Communications, 15(1):9957, 2024.DOI
  24. [26]Anne Gagneux, Ségolène Martin, Rémi Gribonval, and Mathurin Massias. The generation phases of flow matching: A denoising perspective. arXiv preprint arXiv:2510.24830, 2025.
  25. [27]Emile Pierret, Johannes Hertrich, Samuel Hurault, and Julie Delon. Tessellations of semi-discrete flow matching. arXiv preprint arXiv:2605.07513, 2026.arxiv.org/abs/2605.07513
  26. [28]Mingyang Yi, Jiacheng Sun, and Zhenguo Li. On the generalization of diffusion model. arXiv preprint arXiv:2305.14712, 2023.arxiv.org/abs/2305.14712
  27. [29]Mason Kamb and Surya Ganguli. An analytic theory of creativity in convolutional diffusion models. In International Conference on Machine Learning, 2025.arxiv.org/abs/2412.20292
  28. [30]Beatrice Achilli, Enrico Ventura, Gianluigi Silvestri, Bao Pham, Gabriel Raya, Dmitry Krotov, Carlo Lucibello, and Luca Ambrogioni. Losing dimensions: Geometric memorization in generative diffusion. arXiv preprint arXiv:2410.08727, 2024.arxiv.org/abs/2410.08727
  29. [31]Jülich Supercomputing Centre. JUWELS Cluster and Booster: Exascale Pathfinder with Modular Supercomputing Architecture at Jülich Supercomputing Centre. Journal of large-scale research facilities, 7(A138), 2021. doi: 10.17815/jlsrf-7-183.
  30. [32]Alex Krizhevsky et al. Learning multiple layers of features from tiny images. 2009.
  31. [33]Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2015.arxiv.org/abs/1411.7766
  32. [34]Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3), 2015.
  33. [35]Patryk Chrabaszcz, Ilya Loshchilov, and Frank Hutter. A downsampled variant of imagenet as an alternative to the cifar datasets. arXiv preprint arXiv:1707.08819, 2017.arxiv.org/abs/1707.08819
  34. [36]Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35:26565–26577, 2022.arxiv.org/abs/2206.00364
  35. [37]Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794, 2021.
  36. [38]Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.arxiv.org/abs/2112.10752

Paper details

vlnc:W_wj9bkh9w
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents