betaFeedback

WorksComputer Science

Physics-Informed Distillation of Diffusion Models for PDE-Constrained Generation

Abstract

Modeling physical systems in a generative manner offers several advantages, including the ability to handle partial observations, generate diverse solutions, and address both forward and inverse problems. Recently, diffusion models have gained increasing attention in the modeling of physical systems, particularly those governed by partial differential equations (PDEs). However, diffusion models only access noisy data xt\boldsymbol{x}_t at intermediate steps, making it infeasible to directly enforce constraints on the clean sample x0\boldsymbol{x}_0 at each noisy level. As a workaround, constraints are typically applied to the expectation of clean samples E[x0∣xt]\mathbb{E}[\boldsymbol{x}_0|\boldsymbol{x}_t], which is estimated using the learned score network. However, imposing PDE constraints on the expectation does not strictly represent the one on the true clean data, known as Jensen's Gap. This gap creates a trade-off: enforcing PDE constraints may come at the cost of reduced accuracy in generative modeling. To address this, we propose a simple yet effective post-hoc distillation approach, where PDE constraints are not injected directly into the diffusion process, but instead enforced during a post-hoc distillation stage. We term our method as Physics-Informed Distillation of Diffusion Models (PIDDM). This distillation not only facilitates single-step generation with improved PDE satisfaction, but also support both forward and inverse problem solving and reconstruction from randomly partial observation. Extensive experiments across various PDE benchmarks demonstrate that PIDDM significantly improves PDE satisfaction over several recent and competitive baselines, such as PIDM, DiffusionPDE, and ECI-sampling, with less computation overhead. Our approach can shed light on more efficient and effective strategies for incorporating physical constraints into diffusion models.

Introduction

Solving partial differential equations (PDEs) underpins innumerable applications in physics, biology, and engineering, spanning fluid flow Davidson [ [11], heat transfer Incropera et al. [ [25], elasticity Timoshenko and Goodier [ [63], electromagnetism Jackson [ [26], and chemical diffusion Crank [ [10]. Classical discretization schemes such as finite-difference Smith [ [56] and finite-element methods LeVeque [ [33] provide reliable solutions, but their computational cost grows sharply with mesh resolution, dimensionality, and parameter sweeps, limiting their practicality for large-scale or real-time simulations Hughes [ [24]. This bottleneck has fueled a surge of learning-based solvers that approximate or accelerate PDE solutions, from early physics-informed neural networks (PINNs) Raissi et al. [ [51] to modern operator-learning frameworks such as DeepONet Lu et al. [2019], Fourier Neural Operators Li et al. [ [35] and Physics-Informed Neural Operator Li et al. [ [36], offering faster inference, uncertainty quantification, and seamless integration into inverse or data-driven tasks.

Among these learning-based solvers, diffusion models Ho et al. [ [21], Song et al. [ [59] provide a promising framework for generative modeling of physical systems. For PDEs, a diffusion model can learn the joint distribution of solution and coefficient fields, x0=(u,a)x_0 = (u, a), from data, where

aa denotes input parameters that satisfy the boundary operator B\mathcal{B} (for example, material properties or initial conditions) and uu is the corresponding solution that satisfies the PDE operator F\mathcal{F}. After training, the model can sample (u,a)(u, a) from this learned distribution, enabling forward simulation (sample uu given aa), inverse recovery (sample aa given uu), and conditional reconstruction (complete missing components of uu or aa) within a single framework, which prior non-diffusion approaches Li et al. [ [35], Lu et al. [2019], Raissi et al. [ [51] do not provide. However, while diffusion models perform well under soft, high-level constraints Rombach et al. [ [52], Esser et al. [ [15], Ho and Salimans [ [20], Chung et al. [ [9], Ben-Hamu et al. [ [4], PDE applications often require strict, low-level constraints dictated by F\mathcal{F} and B\mathcal{B}.

Enforcing such PDE constraints within diffusion models is nontrivial. A core difficulty is that, at an individual noise level tt, diffusion models operate on noisy variables xtx_t rather than the clean physical field x0x_0, where constraints such as F[x0]=0\mathcal{F}[x_0] = 0 are defined. To address this, one option is to reconstruct x0x_0 by running the full deterministic sampling trajectory, but this is computationally expensive since it requires many forward passes, and enforcing constraints through backpropagation often causes gradient issues Bastek et al. [ [3]. A more common alternative is to approximate x0x_0 with the posterior mean E[x0∣xt]\mathbb{E}[x_0|x_t], which can be efficiently computed via Tweedie’s formula Bastek et al. [ [3], Huang et al. [ [22], Cheng et al. [ [8], Jacobsen et al. [ [27], Kawar et al. [ [29], Wang et al. [ [67], Zhu et al. [ [75], Zhang et al. [ [70], Song et al. [2023a], Tang et al. [ [61], Huang et al. [ [23] (see the right panel of Figure 1(a)). However, this introduces a mismatch: enforcing constraints on the posterior mean, F[E[x0∣xt]]\mathcal{F}[\mathbb{E}[x_0|x_t]], is not equivalent to enforcing the expected constraint, E[F[x0∣xt]]\mathbb{E}[\mathcal{F}[x_0|x_t]], due to Jensen’s inequality. This issue, known as the Jensen’s Gap in inverse-problem diffusion guidance Chung et al. [ [9] and later studied in PDE-constrained diffusion models Bastek et al. [ [3], Huang et al. [ [22], can lead to degraded physical fidelity.

Illustration of physics-constrained diffusion generation and our proposed framework

Figure 1. Illustration of physics-constrained diffusion generation and our proposed framework. (a) Existing methods Huang et al. [ [22], Cheng et al. [ [8], Bastek et al. [ [3], Jacobsen et al. [ [27] impose PDE losses or guidance on the posterior mean E[x0∣xt]\mathbb{E}[x_0|x_t] in diffusion training and sampling, introducing the Jensen’s Gap. (b) We train and sample a diffusion model using vanilla methods to generate a paired noise-data dataset for distillation. (c) Our proposed framework distills the teacher diffusion model and directly enforces physical constraints on the final generated sample x0x_0, avoiding posterior-mean surrogate constraints in the student objective.

Contributions. We propose an effective framework that enforces PDE constraints in diffusion models via post-hoc distillation, enabling reliable and efficient generation under physical laws. As shown in Figure 1(c), our method sidesteps the limitations of existing constraint-guided diffusion-based approaches by decoupling physics enforcement from the diffusion trajectory. Our main contributions are:

  • Empirical confirmation of Jensen’s Gap: We provide an empirical demonstration and quantitative analysis of the Jensen’s Gap in PDE-constrained diffusion, a discrepancy that arises when PDE constraints are imposed on the posterior mean E[x0∣xt]\mathbb{E}[x_0|x_t], rather than the final clean sample x0x_0.

  • Final-sample physics enforcement: PIDDM changes where and when physics is enforced: instead of guiding iterative denoising through posterior-mean surrogates, it imposes PDE supervision on the final output of a post-hoc distilled student. This removes the posterior-mean versus final-sample mismatch from the student objective, while our distributional fidelity claims remain empirical.

  • Versatile and efficient inference: The distilled student model preserves the full generative capabilities of the teacher, supporting physical simulation, reconstruction, and unified forward and inverse PDE solving within a single model, while enabling one-step generation for fast inference. Experiments across diverse PDEs show that PIDDM surpasses posterior-mean-based methods Huang et al. [2024a], Cheng et al. [2025], Bastek et al. [2025], Jacobsen et al. [2024] in both generation quality and constraint satisfaction.

Problem Setup: Jensen's Gap in Diffusion Models with PDE Constraints

In scientific machine learning, there exist many hard and low-level constraints that are mathematically strict and non-negotiable LeVeque and LeVeque [1992], Hansen et al. [2023], Mouli et al. [2024], Saad et al. [2022]. In this section, we discuss how existing works impose these constraints in diffusion-generated data, and the Jensen’s Gap introduced by posterior-mean guidance in inverse problems Chung et al. [2023] and inherited by PDE-constrained diffusion methods Bastek et al. [2025], Huang et al. [2024a].

Preliminaries on Physical Constraints

Physical constraints are typically expressed as a partial differential equation (PDE) F\mathcal{F} defined over a solution domain Ω⊂Rd\Omega\subset\mathbb{R}^{d}, together with a boundary condition operator B\mathcal{B} defined on the coefficient domain Ω′\Omega':

F[u(ξ)]=0 for ξ∈Ω,B[a(ξ′)]=0 for ξ′∈Ω′.(1)\mathcal{F}[\mathbf{u}(\xi)] = 0 \text{ for } \xi\in\Omega,\qquad\mathcal{B}[\mathbf{a}(\xi')] = 0 \text{ for } \xi' \in\Omega'. \tag*{(1)}

In practice, the domains Ω\Omega and Ω′\Omega' are discretized into a uniform grid, typically of size H×WH \times W, and the fields u\mathbf{u} and a\mathbf{a} are evaluated at those grid points to produce the observed data x0=(u,a)\mathbf{x}_0 = (\mathbf{u}, \mathbf{a}), where diffusion models are trained to learn the joint distribution p(x0)=p((u,a))p(\mathbf{x}_0) = p((\mathbf{u}, \mathbf{a})). While PINNs Raissi et al. [2019] model the mapping ξ,ξ′↦(u(ξ),a(ξ′))\xi,\xi' \mapsto(\mathbf{u}(\xi),\mathbf{a}(\xi')) with differentiable neural networks to enable automatic differentiation Paszke et al. [2019], Abadi et al. [2016], grid-based approaches commonly approximate the differential operators in F\mathcal{F} via finite difference methods Smith [1985], LeVeque [2007]. To quantify the extent to which a generated sample x0\mathbf{x}_0 violates the physical constraints, the physics residual error is often defined by:

R(x0)=R((u,a)):=[F[u],B[a]]T(2)\mathcal{R}(\mathbf{x}_0) = \mathcal{R}((\mathbf{u},\mathbf{a})) := \left[\mathcal{F}[\mathbf{u}], \mathcal{B}[\mathbf{a}]\right]^{\mathsf{T}} \tag*{(2)}

Here, R(x)\mathcal{R}(\mathbf{x}) measures the discrepancy between the sample x\mathbf{x} and the expected PDE F\mathcal{F} and boundary conditions B\mathcal{B}. The physics residual loss is often defined by the squared norm of this physics residual error, i.e., ∥R(x)∥2\|\mathcal{R}(\mathbf{x})\|^2.

Imposing PDE Constraints in Diffusion Models

The physical constraints R\mathcal{R} are often defined on the clean field x0x_0, while during training or sampling of the diffusion model, the model only observes the noisy state xtx_t. Therefore, direct optimization or controlled generation based on the physical residual loss R(x0)\mathcal{R}(x_0) is generally impractical. A practical workaround is to evaluate the constraint on an estimate of x0x_0 from xtx_t, and a common choice is to use the estimated posterior mean: E[x0∣xt]\mathbb{E}[x_0 \mid x_t] based on the score network in a diffusion model [3, 22]. As a simplified example, consider the forward process defined as xt=x0+σtϵx_t = x_0 + \sigma_t\epsilon, where σt\sigma_t denotes the noise level at time tt and ϵ∼N(0,I)\epsilon\sim\mathcal{N}(0, I) is standard Gaussian noise. Then, the posterior mean can be efficiently estimated via Tweedie’s formula:

x^θ(xt,t):=xt+σt2sθ(xt,t)≈xt+σt2∇log⁡p(xt)≈E[x0∣xt],\hat{x}_{\theta}(x_t,t) := x_t + \sigma_t^2s_{\theta}(x_t,t) \approx x_t + \sigma_t^2\nabla\log p(x_t) \approx\mathbb{E}[x_0 \mid x_t],

where sθs_{\theta} is a learned score function approximating the gradient of the log-density (see Appendix § 9.§ 2 for the derivation for the general diffusion process). Leveraging this approximation, several existing works incorporate PDE constraints by evaluating the PDE residual operator R\mathcal{R} on x^θ(xt,t)\hat{x}_{\theta}(x_t,t). For instance, PIDM Bastek et al. [ [3] integrates PDE constraints into a diffusion model at training time by augmenting the standard diffusion objective with an additional PDE residual loss R(x^θ(xt,t))\mathcal{R}(\hat{x}_{\theta}(x_t,t)). Similarly, at inference time, DiffusionPDE Huang et al. [ [22] and CoCoGen Jacobsen et al. [ [27] employ diffusion posterior sampling (DPS) Chung et al. [ [9], guiding each intermediate sample xtx_t using the gradient ∇xR(x^θ(xt,t))\nabla_x\mathcal{R}(\hat{x}_{\theta}(x_t,t)). On the other hand, ECI-sampling Cheng et al. [ [8] directly applies hard constraints to the posterior mean at each DDIM step using a correction operator (more detailed discussion on their implementations can be found in Appendix § 11.§ 4). Beyond PDE applications, constrained diffusion models for image inverse problems also rely on posterior-mean approximations for true posterior estimation. Representative examples include DDRM Kawar et al. [ [29], DDNM Wang et al. [ [67], LGD Song et al. [ [58], DPG Tang et al. [ [61], SCG Huang et al. [ [23], DCDPM Dong et al. [ [12], mid-point guidance Moufad et al. [ [47], DiffPIR Zhu et al. [ [75], and DAPS Zhang et al. [ [70] (see Appendix § 8.§ 2 for details). While these pioneering methods have been demonstrated to be effective in enforcing PDE constraints within diffusion models, they still suffer a theoretical inconsistency: PDE constraints are enforced on the posterior mean approximation E[x0∣xt]\mathbb{E}[x_0|x_t], which is not equivalent to the constraints on the true generated data x0x_0 due to Jensen’s inequality:

R(E[x0∣xt])≠E[R(x0)∣xt].(3)\mathcal{R}(\mathbb{E}[x_0|x_t]) \ne\mathbb{E}[\mathcal{R}(x_0)|x_t]. \tag*{(3)}

This discrepancy is commonly referred to as the Jensen’s Gap Chung et al. [ [9], Bastek et al. [ [3], Huang et al. [ [22], Gao et al. [ [16]. To mitigate this issue, PIDM and DiffusionPDE heuristically down-weight PDE constraints at early denoising steps (large tt) in training and sampling, respectively, where the gap is pronounced, and emphasize them near t→0t \to0, where the posterior mean approximation improves. ECI-sampling introduces stochastic resampling steps Wang et al. [ [65] to project intermediate samples back toward their correct distribution. Many other methods for image inverse problems also provide partial improvements to reduce approximation error Song et al. [2023a], Tang et al. [ [61], Huang et al. [ [23], Dong et al. [ [12], Moufad et al. [ [47] (Appendix § 8.§ 2). These methods address aspects of the approximation error, but still rely on posterior-mean surrogates during the diffusion trajectory.

Demonstration of the Jensen’s Gap

To better illustrate this posterior-mean mismatch and its negative effect in PDE-constrained generation, we conduct experiments on two synthetic datasets: a Mixture-of-Gaussians (MoG) dataset and a Stokes Problem dataset.

Sampling-time Jensen’s Gap. We demonstrate the sampling-time Jensen’s Gap using the Mixture-of-Gaussians (MoG) dataset, where the score function is analytically tractable, allowing us to isolate the effect of the diffusion process without interference from training error. The MoG is constructed in 2D: the first dimension follows a bimodal Gaussian distribution, while the second dimension encodes a discrete latent variable that serves as a hard constraint. Concretely, the joint distribution is defined as a mixture of two Gaussians, each supported on a distinct horizontal line:

p(x0)=0.5⋅N(x1;−1,σ2)⋅δ(x2+1)+0.5⋅N(x1;+1,σ2)⋅δ(x2−1),p(x_0)=0.5\cdot\mathcal{N}(x_1;-1,\sigma^2)\cdot\delta(x_2+1)+ 0.5\cdot\mathcal{N}(x_1;+1,\sigma^2)\cdot\delta(x_2-1),

Algorithm 1 PIDDM Training: Physics-Informed Distillation.

Require: Teacher Model vθ(x,t)v_\theta(x,t), Student Model dθ′d_{\theta'}, Batch Size BB, Terminal Time T=1T=1, Steps NsN_s, Step Size dt=1/Ns\mathrm{d}t=1/N_s, Physics Residual Error R\mathcal{R}, Loss Weight λtrain\lambda_{\mathrm{train}}, Learning Rate ηtrain\eta_{\mathrm{train}}

  1. repeat

  2. Sample ϵ1:B∼i.i.d.N(0,I)\epsilon_{1:B} \mathrel{\overset{\mathrm{i.i.d.}}{\sim}} \mathcal{N}(0,\mathbf{I}); xT←ϵ1:Bx_T \leftarrow\epsilon_{1:B}

  3. for t=T−dt,…,0t=T-\mathrm{d}t,\ldots,0 do

  4. xt←xt+dt−vθ(xt+dt,t+dt)dtx_t \leftarrow x_{t+\mathrm{d}t}-v_\theta(x_{t+\mathrm{d}t},t+\mathrm{d}t)\mathrm{d}t ▷ Sampling Phase

  5. end for

  6. xpred←dθ′(ϵ1:B)x_{\mathrm{pred}} \leftarrow d_{\theta'}(\epsilon_{1:B})

  7. L←1B∥xpred−x0∥2+λtrain∥R(xpred)∥2\mathcal{L} \leftarrow\frac{1}{B}\lVert x_{\mathrm{pred}}-x_0\rVert^2+\lambda_{\mathrm{train}}\lVert\mathcal{R}(x_{\mathrm{pred}})\rVert^2 ▷ Distillation Phase

  8. θ′←θ′−ηtrain∇θ′L\theta' \leftarrow\theta'-\eta_{\mathrm{train}}\nabla_{\theta'}\mathcal{L}

  9. until Converged

where δ(⋅)\delta(\cdot) denotes the Dirac delta function and σ=0.2\sigma=0.2. To examine the impact of Jensen’s Gap during sampling, we compare Diffusion Posterior Sampling (DPS) Chung et al. [ [9] which uses a latent code to guide the generation, with the ground-truth conditional ODE trajectory derived analytically. We evaluate three representative diffusion processes: Variance-Preserving (VP) Ho et al. [ [21], Sub-VP Song et al. [ [59], and Linear Liu et al. [ [38], and compare their velocity fields during inference to characterize Jensen’s Gap. To quantify amplitude errors, we compute the mean absolute error (MAE) and angular error between the DPS-predicted velocity field vDPS(x,t)v_{\mathrm{DPS}}(x,t) and the ground-truth velocity vGT(x,t)v_{\mathrm{GT}}(x,t): We observe that both DPS errors are significantly elevated at intermediate timesteps and only diminish as t→0t\to0, as shown in Figure 2a. Although DPS achieves accurate sampling in the unconstrained dimension (Figure 2b), it fails to respect the hard constraint in the constrained dimension (Figure 2c). This experiment illustrates the mismatch induced by posterior-mean guidance; it does not claim that single posterior samples along the trajectory resolve the gap.

Empirical illustration of the Jensen’s Gap in physics-constrained diffusion models

Figure 2. Empirical illustration of the Jensen’s Gap in physics-constrained diffusion models. (a) Absolute velocity error and angular discrepancy (1−cos⁡(θ))(1-\cos(\theta)) between Diffusion Posterior Sampling (DPS) and the ground-truth conditional ODE velocity on the MoG dataset. (b) and (c) Histograms comparing the first (unconstrained) and second (hard-constrained) dimensions of DPS-sampled MoG data against the ground-truth MoG. (d) Training-time manifestation: diffusion loss comparison on a Stokes problem dataset.

Training-time Jensen’s Gap. We examine the Jensen’s Gap during training using the synthetic Stokes dataset with target distribution pStokesp_{\mathrm{Stokes}}. The diffusion model vθv_\theta adopts a Fourier Neural Operator (FNO) Li et al. [ [35] architecture and follows a standard linear noise schedule Lipman et al. [ [37], Liu et al. [ [39]. Dataset and training details are given in Appendices C and D. We take PIDM Bastek et al. [ [3] as a representative method, which augments the diffusion loss with a PDE residual term R(x^θ(xt,t))\mathcal{R}(\hat{x}_\theta(x_t,t)), and compare its performance with standard diffusion training. To assess generative quality, we monitor the diffusion loss, which theoretically corresponds to the evidence lower bound (ELBO) Ho et al. [ [21], Kingma et al. [ [32]. The comparison results are shown in Figure 2d, revealing a significant increase in diffusion loss when the PDE residual loss is incorporated. This suggests that the PDE residual loss does not improve modeling of the PDE-consistent data distribution. This observation also corroborates findings from PIDM Bastek et al. [ [3], which identified that residual supervision on the posterior mean can create “a conflicting objective between the data and residual loss”, where the data loss represents the original diffusion training objective. These results provide further evidence of the posterior-mean mismatch in training, as enforcing constraints on E[x0∣xt]\mathbb{E}[x_0|x_t] may interfere with maximizing the likelihood of the true data distribution.

Method: Physics-Informed Distillation of Diffusion Models

In Section 2, we have demonstrated the posterior-mean mismatch that appears when incorporating physical constraints into diffusion training and sampling, as observed in prior works. To address this issue, we propose a distillation-based framework that sidesteps this mismatch in the student objective. Specifically, instead of enforcing constraints on the posterior mean during the diffusion process, we apply physical constraints directly to the final generated samples in a post-hoc distillation stage.

Diffusion Training

To decouple physical constraint enforcement from the diffusion process itself, we first conduct standard diffusion model training using its original denoising objective, without adding any constraint-based loss. To obtain smoother sampling trajectories that benefit later noise-data distillation Liu et al. [ [39], we adopt a linear diffusion process and apply the ν\nu-prediction parameterization Liu et al. [ [39], Lipman et al. [ [37], Liu et al. [ [40], Cheng et al. [ [8], Esser et al. [ [15], which is Algorithm 2 PIDDM Inference: Physics Data Simulation

  1. Input Student Model dθ′d_{\theta'}, Physics Residual Error R\mathcal{R}, Refinement Step Number NfN_f, Refinement Step Size ηref\eta_{\mathrm{ref}}, Latent Noise ϵ∼N(0,I)\epsilon\sim\mathcal{N}(0,I).

  2. for i=1,…,Nfi = 1,\ldots,N_f do

  3. ϵ←ϵ−ηref∇ϵ∥R(dθ′(ϵ))∥2\epsilon\leftarrow\epsilon- \eta_{\mathrm{ref}}\nabla_{\epsilon}\lVert\mathcal{R}(d_{\theta'}(\epsilon))\rVert^2

  4. end for

  5. Output dθ′(ϵ)d_{\theta'}(\epsilon)

Algorithm 3 PIDDM Inference for Forward/Inverse/Reconstruction

  1. Input Student Model dθ′d_{\theta'}, Physics Residual Error R\mathcal{R}, Optimization Iterations NoN_o, Step Size ηinfer\eta_{\mathrm{infer}}, Observation x′x', Observation Mask MM, Loss Weight λinfer\lambda_{\mathrm{infer}}, Latent Noise ϵ∼N(0,I)\epsilon\sim\mathcal{N}(0,I).

  2. for i=1,…,Noi = 1,\ldots,N_o do

  3. xmix←x′⊙M+dθ′(ϵ)⊙(1−M)x_{\mathrm{mix}} \leftarrow x' \odot M + d_{\theta'}(\epsilon) \odot(1-M)

  4. ϵ←ϵ−ηinfer∇ϵ[∥(dθ′(ϵ)−x′)⊙M∥2+λinfer∥R(xmix)∥2]\epsilon\leftarrow\epsilon- \eta_{\mathrm{infer}}\nabla_{\epsilon}\left[\lVert(d_{\theta'}(\epsilon)-x')\odot M\rVert^2+\lambda_{\mathrm{infer}}\lVert\mathcal{R}(x_{\mathrm{mix}})\rVert^2\right]

  5. end for

  6. x←x′⊙M+dθ′(ϵ)⊙(1−M)x \leftarrow x' \odot M + d_{\theta'}(\epsilon) \odot(1-M)

  7. Output xx

commonly referred to as a flow model. Specifically, the training objective is

L(θ)=Et∼U(0,1), x0∼pϵ∼N(0,I)[∥vθ(xt,t)−(ϵ−x0)∥2]\mathcal{L}(\theta)=\mathbb{E}_{t\sim U(0,1),\,x_0\sim p \atop\epsilon\sim\mathcal{N}(0,I)}\left[\left\lVert v_\theta(x_t,t)-(\epsilon-x_0)\right\rVert^2\right]

where xt=(1−t)x0+tϵx_t=(1-t)x_0+t\epsilon, p(x0)p(x_0) is the distribution of joint data containing both solution and coefficient fields x=(u,a)x=(u,a), ϵ\epsilon is sampled from a standard Gaussian distribution, and vθv_\theta is the neural network used as the diffusion model. This formulation allows the model to learn to reverse the diffusion process without entangling it with physical supervision, thereby preserving generative fidelity.

Imposing PDE Constraints in Distillation

After training the teacher diffusion model using the standard denoising objective, we proceed to the distillation stage, where we transfer its knowledge to a student model designed for efficient one-step generation. Crucially, this post-hoc distillation stage is where we impose PDE constraints, moving physics enforcement away from posterior-mean surrogates used during diffusion training or sampling. This distillation process is guided by two complementary objectives: (1) learning to map a noise sample to the final generated output predicted by the teacher model, and (2) enforcing physical consistency on this output via PDE residual minimization. Concretely, we begin by sampling a noise input ϵ∼N(0,I)\epsilon\sim\mathcal{N}(0,I) and generating a target sample x0x_0 using the pre-trained teacher model via deterministic integration of the reverse-time ODE:

xt−dt=xt−vθ(xt,t) dt(4)x_{t-\mathrm{d}t}=x_t-v_\theta(x_t,t)\,\mathrm{d}t \tag*{(4)}

which proceeds from t=1t=1 to t=0t=0 using a fixed step size dt\mathrm{d}t. This yields a paired noise-data dataset D={ϵ,x0}\mathcal{D}=\{\epsilon,x_0\} for distillation, as shown in Figure 1(b). A student model dθ′(ϵ)d_{\theta'}(\epsilon) is then trained to predict x0x_0 in one step, as shown in Figure 1(c). Meanwhile, to enforce physical consistency, we evaluate the physics residual error on the output x=dθ′(ϵ)x=d_{\theta'}(\epsilon), i.e., ∥R(x)∥2\lVert\mathcal{R}(x)\rVert^2. The overall training objective is:

Ltotal=Lsample+λtrainLPDE=E(ϵ,x0)∼D[∥dθ′(ϵ)−x0∥2+λtrain∥R(x)∥2](5)\begin{aligned} \mathcal{L}_{\mathrm{total}}=\mathcal{L}_{\mathrm{sample}}+\lambda_{\mathrm{train}}\mathcal{L}_{\mathrm{PDE}} \\ &=\mathbb{E}_{(\epsilon,x_0)\sim\mathcal{D}}\left[\lVert d_{\theta'}(\epsilon)-x_0\rVert^2+\lambda_{\mathrm{train}}\lVert\mathcal{R}(x)\rVert^2\right] \tag*{(5)} \end{aligned}

where λtrain\lambda_{\mathrm{train}} balances generative fidelity and physical constraint satisfaction. Unlike prior work Bastek et al. [ [3], Huang et al. [ [22], the residual is evaluated on final student outputs rather than posterior-mean surrogates E[x0∣xt]\mathbb{E}[x_0|x_t], which makes the physical objective directly aligned with sample-level constraint satisfaction (see Table 4). Training is repeated until convergence (Algorithm 1). This noise-to-sample distillation is often difficult to learn due to the high curvature of sampling trajectories, yielding noise–data pairs that are far apart in Euclidean space Liu et al. [ [40]. To reduce curvature and improve learnability, we adopt linear-flow distillation Lipman et al. [ [37], Liu et al. [ [39], and we further evaluate Distribution Matching Distillation (DMD) Yin et al. [ [69], Rectified Flow, and Consistency Model Song et al. [ [60] to strengthen coupling and distribution alignment. These choices produce consistent gains (see Table 4).

TaskMetricPIDDM+RF-1+RF-2+DMD+CM−HCVPsub-VP
ForwardMSE (×10−1\times10^{-1})0.3160.2780.1270.2550.2830.7050.3980.372
PDE Error (×10−4\times10^{-4})0.1450.1290.0980.1340.0830.3540.1540.157
InverseMSE (×10−1\times10^{-1})0.2360.1950.1360.1880.1820.5030.2840.271
PDE Error (×10−4\times10^{-4})0.1150.1260.0790.1210.1090.3210.1430.139
ReconstructCoef MSE (×10−1\times10^{-1})0.1280.1070.0910.0950.0850.2940.1330.138
ReconstructSol MSE (×10−1\times10^{-1})0.1020.0840.0630.0730.0720.2390.1270.119
ReconstructPDE Error (×10−4\times10^{-4})0.1430.1180.0850.1040.0830.4640.1590.158

Table 4. Evaluation on various downstream tasks on Darcy datasets under different PIDDM settings. PIDDM denotes the raw method, +RF-1 and +RF-2 denote one and two rounds of reflowing Liu et al. [ [40], +DMD denotes distribution matching distillation Yin et al. [ [69], and +CM denotes the consistency model Song et al. [ [60]. −HC refers to the ablation without the hard-constraint replacement in Eq. (6) during PIDDM inference. VP and sub-VP refer to ablations on VP and sub-VP diffusion processes. PDE error is the MSE of the evaluated physics residual error. The best results are in bold.

To clarify the optimization mechanism, Appendix § 9.§ 3 analyzes a low-rank Gaussian target, where the low-rank direction mimics a hard equality constraint in PDE problems. The main intuition is that diffusion training and posterior-mean guidance operate in score space along a noisy trajectory. Near the clean endpoint, the score component associated with the constrained direction becomes increasingly sharp, so small approximation errors in this component can persist as visible constraint violations in generated samples. PIDDM instead moves the physics penalty to data space after the endpoint sample is formed. The student therefore does not need to resolve the sharp score field at every noise level; it is trained to suppress the violating component of its final output directly. This explains why final-sample distillation can reduce sample-level PDE residuals, while our distributional fidelity claims remain empirical rather than an end-to-end theoretical guarantee.

Downstream Tasks

Our method naturally supports one-step generation of physically constrained data, jointly producing both coefficient and solution fields. Beyond this intrinsic functionality, it also retains the flexibility of the teacher diffusion model, enabling various downstream tasks such as forward and inverse problem solving, and reconstruction from partial observations. Compared to the teacher model, our method achieves these capabilities with improved computational efficiency and stronger physical alignment.

Generative Modeling. We aim to sample physically consistent pairs x0=(w,a)x_0 = (w, a) from a learned distribution that satisfies the governing PDE system. The student model supports this via efficient one-step generation: given ϵ∼N(0,I)\epsilon\sim\mathcal{N}(0, I), it outputs x0=dθ′(ϵ)x_0 = d_{\theta'}(\epsilon), approximating a valid solution–coefficient pair. We further provide an optional refinement stage based on constraint-driven optimization (Algorithm 2), which reduces the physics residual by updating ϵ\epsilon with gradient descent. This design is inspired by noise prompting methods Ben-Hamu et al. [ [4], Guo et al. [ [17] that optimize the final sample with respect to the initial noise. However, in contrast to those prior works, which backpropagate through an entire sampling trajectory and incur high cost and unstable gradients, our refinement operates in a one-step setting. While optional, it offers additional control that is useful in scientific applications requiring strict physical consistency LeVeque and Leveque [ [34], Hansen et al. [ [19], Mouli et al. [ [48], Saad et al. [ [53].

Forward/Inverse Problem and Reconstruction. PIDDM handles all downstream problems as conditional generation over the joint field x=(u,a)x = (u, a). Forward inference draws uu from known aa; inverse inference recovers aa from observed uu; reconstruction fills in missing entries of (u,a)(u, a) given a partial observation x′x'. We solve this via optimization-based inference on the latent variable ε\varepsilon, using the same student model dθ′d_{\theta'} as in generation, as described in Algorithm 3. Let x=dθ′(ε)x = d_{\theta'}(\varepsilon) denote the generated sample, and let MM be a binary observation mask indicating the known entries in x′x' with respect to xx. To ensure hard consistency with observed values (e.g., boundary conditions B\mathcal{B}), we define a mixed sample by injecting observed entries into the generated output, following ECI-sampling Cheng et al. [ [8], and then update ε\varepsilon using a combined objective:

Ltotal=∥(x−x′)⊙M∥2+λ∥R(xmix)∥2,\mathcal{L}_{\mathrm{total}} = \lVert(x - x') \odot M \rVert^2 + \lambda\lVert\mathcal{R}(x_{\mathrm{mix}}) \rVert^2,
xmix=x′⊙M+x⊙(1−M).(6)x_{\mathrm{mix}} = x' \odot M + x \odot(1 - M). \tag*{(6)}

Interestingly, we also find that applying this masking not only enhances hard constraints on B\mathcal{B}, but also improves satisfaction of F\mathcal{F}, as demonstrated in our ablation study in Table 4. Classical inverse solvers Li et al. [ [35, 36], Lu et al. [2019], Raissi et al. [ [51] learn a deterministic map u↦au \mapsto a and therefore require full observations of aa to evaluate F[u,a]=0\mathcal{F}[u, a] = 0, a condition rarely met in practice. DiffusionPDE Huang et al. [ [22] relaxes this by sampling missing variables, but enforces physics on the posterior mean, i.e. F[E[x0∣xt]]\mathcal{F}[\mathbb{E}[x_0|x_t]], and thus suffers from the Jensen’s Gap. Our method avoids this inconsistency by imposing constraints directly on the final sample F[x0]\mathcal{F}[x_0], yielding more reliable and physically consistent inverse solutions.

Experiments

Experiment Setup. We consider three widely used PDE benchmarks in the main text: Darcy flow, Poisson equation, and Burgers’ equation. These datasets are readily accessible from FNO Li et al. [ [35] and DiffusionPDE Huang et al. [ [22]. We also provide results on other benchmarks in Appendix § 12. We consider ECI Cheng et al. [ [8], DiffusionPDE Huang et al. [ [22], D-Flow Cheng et al. [ [8], Ben-Hamu et al. [ [4], PIDM Bastek et al. [ [3], and vanilla teacher diffusion models as baseline methods, with detailed implementations in Appendix § 11.§ 4. We follow ECI-sampling Cheng et al. [ [8] to use FNO as both the teacher diffusion model and the student distillation model. We provide the full specification of our experiment setup in Appendix § 11.

To quantitatively evaluate generative performance, we report MMSE, SMSE, FPD, and PDE error following prior work Cheng et al. [ [8], Kerrigan et al. [ [30], Bastek et al. [ [3], Jacobsen et al. [ [27]: MMSE measures the mean squared error of the sample mean; SMSE evaluates the error of the sample standard deviation; FPD evaluates the Frechet distance between hidden representations extracted by the pre-trained PDE foundation model; PDE error quantifies the violation of physical constraints using the physics residual error ∣R(x)∣2\lvert\mathcal{R}(x) \rvert^2. The number of function evaluations (NFE) reflects computational cost during inference. For downstream tasks, we further report MSE on solution fields, coefficient fields, or both, depending on the problem setting, reflecting the accuracy of PDE solving.

Empirical Evaluations

PIDDM samples the joint field (u,a)(u, a), enabling forward (u∣a)(u|a), inverse (a∣u)(a|u), and reconstruction (partial u,au, a) tasks (§ 3.3). DiffusionPDE Huang et al. [ [22] reports only reconstruction MSE, while ECI-sampling Cheng et al. [ [8] and PIDM Bastek et al. [ [3] cover at most one task, limited to either unconditional generation or forward solving. For a fair comparison, we evaluate all methods on all three tasks, providing a unified view of generative quality and physical fidelity.

Generative Tasks. We first evaluate the generative performance of our method across three representative PDE systems: Darcy, Poisson, and Burgers’ equation. As shown in Table 1, our one-step model (PIDDM-1) achieves competitive MMSE and SMSE scores while requiring extremely low computational cost (1 NFE). Notably, PIDDM-1 already surpasses prior methods that incorporate physical constraints during training or sampling, such as PIDM, DiffusionPDE, and ECI-sampling, which impose constraints on posterior-mean surrogates and only exhibit marginal improvements over vanilla diffusion baselines. Our optional refinement stage (PIDDM-ref) further reduces both statistical errors and physical PDE residuals, outperforming all baselines. Meanwhile, ECI, which only enforces hard constraints on boundary conditions, achieves moderate improvements but remains less effective on field-level physical consistency. Although D-Flow enforces physical constraints through final-sample trajectory optimization, it requires thousands of NFEs and often suffers from gradient instability.

DatasetMetricPIDDM-1PIDDM-refECIDiffusionPDED-FlowPIDMTeacher
DarcyMMSE (×10−2\times10^{-2})0.1120.0370.1530.4190.1290.5150.108
DarcySMSE (×10−2\times10^{-2})0.0820.0020.1030.1630.0850.3680.069
DarcyPDE Error (×10−4\times10^{-4})0.2260.1481.5821.0710.5321.2361.585
DarcyFPD0.7540.3850.9211.4370.9951.9830.782
DarcyNFE (×103\times10^{3})0.0010.0800.5000.1005.0000.1000.100
PoissonMMSE (×10−2\times10^{-2})0.1620.1130.1830.8610.1720.9480.150
PoissonSMSE (×10−2\times10^{-2})0.3260.2740.2910.4830.4750.7010.353
PoissonPDE Error (×10−9\times10^{-9})0.0730.0502.4201.2700.8311.5932.443
PoissonFPD1.2810.6591.5321.8351.6772.3581.342
PoissonNFE (×103\times10^{3})0.0010.0800.5000.1005.0000.1000.100
BurgersMMSE (×10−2\times10^{-2})0.1520.0120.2940.0640.3050.9480.264
BurgersSMSE (×10−2\times10^{-2})0.1330.1010.1050.1030.2070.7010.114
BurgersPDE Error (×10−3\times10^{-3})0.4660.1741.5721.0320.7301.5931.334
BurgersFPD0.1290.0540.3871.1330.6951.4370.118
BurgersNFE (×103\times10^{3})0.0010.0800.5000.1005.0000.1000.100

Table 1. Generative metrics on various PDE problems. PDE error is the MSE of the evaluated physics residual error. The best results are in bold and the second-best results are underlined.

Compute-Matched Comparison. Table 3 further compares inference cost and quality on Darcy generation under the same hardware. PIDDM-1 uses one forward evaluation and no backward pass, giving millisecond-level generation while improving PDE loss and distributional metrics over posterior-mean-guided baselines. With the same 80-step online budget as DiffusionPDE, PIDDM-ref uses final-sample refinement to obtain lower PDE loss, FPD, MMSE, and SMSE. D-Flow optimizes the final sample directly but requires 2000 forward and backward passes, making it substantially slower despite weaker statistical metrics. Here, NFE denotes the number of forward function evaluations, and BWD denotes the number of inference-time backward passes.

MethodTimeNFEBWDPDEFPDMMSESMSE
PIDDM-10.004100.2260.7540.1120.082
PIDDM-ref1.17080800.1480.3850.0370.002
DiffPDE1.25080801.1291.4520.4310.173
PIDM1.28130001.2742.0180.5220.374
D-Flow19.302200020000.8851.2520.2540.209
ECI1.29430001.1060.8370.1380.112

Table 3. Compute-matched Darcy generation. Best or tied-best entries are in bold.

Forward/Inverse Solving and Reconstruction. We further demonstrate the versatility of our method in forward and inverse problem solving on the Darcy dataset. Since the original PIDM Bastek et al. [ [3] implementation addresses only unconditional generation, we pair it with Diffusion Posterior Sampling (DPS) Chung et al. [ [9] to extend it to downstream tasks (forward, inverse, and reconstruction). Following the test protocol of D-Flow, we apply inference-time optimization over the initial noise to match given observations while satisfying physical laws. As shown in Table 2, our method (PIDDM) achieves the best results across all metrics, including MSE and PDE error, while being significantly more efficient than D-Flow, which requires 5000 function evaluations. Compared to ECI and DiffusionPDE, our method yields lower residuals and better predictive accuracy, reflecting its superior handling of physical and observational constraints jointly.

TaskMetricPIDDMECIDiffusionPDED-FlowPIDM
ForwardMSE0.3160.7760.6910.5390.380
PDE Error0.1451.5731.5760.5841.248
NFE0.0800.5000.1005.0000.100
InverseMSE0.2360.5450.4560.4280.468
PDE Error0.1261.5051.4020.4381.113
NFE0.0800.5000.1005.0000.100
ReconstructCoef MSE0.1280.3950.2400.1580.179
Sol MSE0.1020.2190.1430.1250.147
PDE Error0.1431.2051.2390.6051.240
NFE0.0800.5000.1005.0000.100

Table 2. Evaluation on various downstream tasks on Darcy datasets. PDE error is the MSE of the evaluated physics residual error. The units of MSE, PDE error, and NFE are ×10−1\times10^{-1}, ×10−4\times10^{-4}, and ×103\times10^{3}, respectively. The best results are in bold.

Time is wall-clock seconds per sample; PDE, MMSE, and SMSE are scaled as in Table 1.

Ablation Studies

To better understand the effect of key design choices in PIDDM, we perform ablations on five factors: teacher sampling steps NsN_s; distillation weight λtrain\lambda_{\mathrm{train}}; inference weight λinfer\lambda_{\mathrm{infer}}; diffusion schedule (VP, sub-VP, linear); and advanced distillation variants (Rectified Flow, DMD, Consistency Model). Figure 3 presents three key ablation studies on the Darcy dataset. Panel (a) shows that increasing the teacher model’s sampling steps NsN_s consistently improves both generative quality and physical alignment, as reflected by lower MMSE, SMSE, and PDE residuals of the distilled student, highlighting the importance of high-fidelity supervision. Panels (b) and (c) examine the impact of the PDE loss weight during distillation and inference, respectively. We observe that across a wide range of weights, all metrics achieve strong performance. In particular, compared with baselines in Table 2, the PDE error is reduced by an order of magnitude. This improvement is consistent with the design goal of enforcing PDE constraints on final generated samples rather than posterior-mean surrogates.

Ablation studies on the effect of several factors on the performance of PIDDM on the Darcy dataset

Figure 3. Ablation studies on the effect of several factors on the performance of PIDDM on the Darcy dataset. Panels (a), (b), and (c) show the effects of NsN_s, λtrain\lambda_{\mathrm{train}}, and λinfer\lambda_{\mathrm{infer}}, respectively.

We also explore whether more sophisticated distillation strategies can improve the quality of the student model. As shown in Table 4, advanced techniques such as Rectified Flow (RF-1, RF-2), Distribution Matching Distillation (DMD), and Consistency Model yield better MMSE and SMSE than our raw method, while maintaining competitive PDE residuals. This indicates that tighter coupling between noise and data trajectories during distillation facilitates noise-data learning. In addition, we analyze the effect of imposing hard constraints during downstream inference. Following the strategy inspired by ECI-sampling, we directly replace the masked entries in the generated sample with observed values before computing the PDE residual. This ensures that the known information is preserved when evaluating physical consistency. As shown by the −HC variant in Table 4, removing this hard-constraint replacement significantly degrades the PDE residual errors across all tasks. We also validate our design using other linear diffusion processes in Table 4 (i.e., VP and sub-VP).

Appendix G provides additional diagnostics for the main design choices. Table 6 separates one-step distillation from final-sample PDE supervision, showing that distillation mainly gives speed while final-sample supervision drives physical consistency; Table 13 confirms the same pattern across RF, DMD, and consistency-model backbones. Table 7 further shows that PIDDM preserves sample diversity better than vanilla distillation. We also test robustness to teacher quality and architecture in Tables 8 and 9, and report dynamic weighting and latent-optimization stability in Tables 10–12.

Conclusion and Limitation

We propose PIDDM, a lightweight post-hoc distillation framework for physics-constrained diffusion models. Unlike methods that impose PDE constraints on the posterior mean, leading to a Jensen’s Gap and a trade-off between quality and constraint satisfaction, PIDDM enforces constraints directly on the final generated sample. This design improves sample-level physical consistency while empirically preserving distributional fidelity. We provide empirical illustrations of the Jensen’s Gap in diffusion training and sampling. Experiments show PIDDM improves physical and distributional fidelity across forward, inverse, and partial reconstruction tasks, with robustness to hyperparameter choice. The distilled student model also enables efficient one-step physics simulation for repeated-query and latency-sensitive scientific workflows such as uncertainty quantification, online data assimilation, virtual sensing, and design optimization.

Limitations. PIDDM requires a well-trained teacher model and a reliable PDE residual operator, which can be challenging to obtain. The one-step student’s performance may degrade if the teacher is poorly calibrated or lacks diversity. Although performance is robust across a wide range of PDE loss weights, some hyperparameter tuning is still needed to obtain optimal results. Addressing these limitations remains important future work.

Acknowledgements

We sincerely thank the reviewers and the area chair for their careful reading, constructive feedback, and thoughtful suggestions, which helped us improve the clarity and presentation of this work. D. Zou and Y. Zhang are supported in part by NSFC 62306252, Hong Kong ECS award 27309624, Guangdong NSF 2024A1515012444, and the central fund from HKU. P. Wang is supported in part by the University of Macau SRG2025-00043-FST and UMDF-TISF-I/2026/013/FST, and in part by the Macau Science and Technology Development Fund (FDCT) 0091/2025/ITP2.

Impact Statement

This work makes physics-constrained diffusion models practical by enforcing PDE constraints post hoc, sidestepping the posterior-mean mismatch in the student objective and enabling fast single-step generation. By improving physical fidelity and reducing compute and tuning burden, PIDDM can accelerate simulation, inverse design, and data completion workflows in engineering and scientific domains.

Related Work

Diffusion Models

Diffusion models Song et al. [ [59], Ho et al. [ [21], Karras et al. [ [28] learn a score function, ∇log⁡p(xt)\nabla\log p(x_t), to reverse a predefined diffusion process, typically of the form xt=x0+σtεx_t = x_0 + \sigma_t\varepsilon. A key characteristic of diffusion models is that sampling requires iteratively reversing this process over a sequence of timesteps. This iterative nature presents a challenge for controlled generation: to guide the sampling trajectory effectively, we often need to first estimate the current denoised target x0x_0 in order to determine the correct guidance direction. In other words, to decide how to get there, we must first understand where we are. However, obtaining this information through full iterative sampling is computationally expensive and often impractical in an optimization regime.

A practical workaround is to leverage an implicit one-step data estimate provided by diffusion models via Tweedie’s formula Efron [ [14], which requires only a single network forward pass:

x^0≈E[x0∣xt]=xt+σt2∇log⁡p(xt),\hat{x}_0 \approx\mathbb{E}[x_0 \mid x_t] = x_t + \sigma_t^2 \nabla\log p(x_t),

where x^0\hat{x}_0 denotes a one-step denoised estimate of the clean sample from xtx_t. This approximation improves as t→0t \to0. Although this posterior mean E[x0∣xt]\mathbb{E}[x_0 \mid x_t] is not theoretically equivalent to the final sample obtained after full denoising, in practice, this estimate serves as a useful proxy for the underlying data and enables approximate guidance for controlled generation, without the need to complete the entire sampling trajectory.

Constrained Generation for PDE Systems

Diffusion models have demonstrated strong potential for physical-constraint applications due to their generative nature. This generative capability naturally supports the basic task of simulating physical data and also extends to downstream applications such as reconstruction from partial observations and solving both forward and inverse problems. However, many scientific tasks require strict adherence to physical laws, often expressed as PDE constraints on the data. These constraints, applied at the sample level xx, are not easily enforced within diffusion models, which are trained to model the data distribution p(x)p(x). To address this, prior works have proposed three main strategies for incorporating physical constraints into diffusion models.

Training-time Loss Injection. PG-Diffusion Shu et al. [ [55] employs Classifier-Free Guidance (CFG), where a conditional diffusion model is trained using the PDE residual error as a conditioning input. However, CFG is well known to suffer from theoretical inconsistencies—specifically, the interpolated conditional score function does not match the true conditional score—which limits its suitability for enforcing precise physical constraints. To avoid this issue, PIDM Bastek et al. [ [3] introduces a loss term based on the residual evaluated at the posterior mean, E[x0∣xt]\mathbb{E}[x_0 \mid x_t]. While this approach avoids the theoretical pitfalls of CFG, the constraint is still not imposed on the actual sample x0x_0, inheriting the Jensen-gap issue originally discussed in diffusion posterior sampling for inverse problems Chung et al. [ [9].

Sampling-time Guidance. Diffusion Posterior Sampling (DPS), used in DiffusionPDE Huang et al. [ [22] and CoCoGen Jacobsen et al. [ [27], applies guidance during each sampling step by using the gradient of the PDE residual evaluated on the posterior mean E[x0∣xt]\mathbb{E}[x_0 \mid x_t]. Therefore, they inherit the Jensen’s Gap issue, as the guidance operates on an estimate of the final sample rather than the sample itself. Moreover, DPS assumes that the residual error follows a Gaussian distribution—a condition that may not hold in real-world PDE systems. Meanwhile, to support hard constraints, ECI-sampling Cheng et al. [ [8] directly modifies the posterior mean E[x0∣xt]\mathbb{E}[x_0 \mid x_t] using known boundary conditions.

Noise Prompting. Another stream of research—often called noise prompting or golden-noise optimization—directly tunes the initial noise so that the resulting sample satisfies a target constraint Ben-Hamu et al. [ [4], Guo et al. [ [17], Zhou et al. [ [74], Wang et al. [ [66], Mao et al. [ [45], Chen et al. [ [7]. In the physics domain, this idea is used to minimize the true PDE residual R(x)R(x) evaluated on the final sample, rather than the surrogate residual R(E[x0∣xt])R(\mathbb{E}[x_0 \mid x_t]). Because the constraint is imposed on the actual output, noise prompting avoids posterior-mean surrogate constraints and therefore serves as a strong baseline in ECI-sampling Cheng et al. [ [8] and PIDM Bastek et al. [ [3]. The main drawback is efficiency: optimizing the noise requires backpropagating through the entire sampling trajectory, which is computationally expensive and prone to gradient instability.

Recently, diffusion-based techniques for solving image inverse problems have demonstrated competitive performance Zheng et al. [ [72]. However, a common limitation is that they rely on the posterior mean, i.e., E[x0∣xt]\mathbb{E}[x_0 \mid x_t], as a surrogate for the true posterior. For example, DDRM Kawar et al. [ [29] and DDNM Wang et al. [ [67] exploit singular value decomposition (SVD) and pseudo-inverse operations to fill in the missing components of E[x0∣xt]\mathbb{E}[x_0 \mid x_t] during sampling, which is conceptually similar to ECI-sampling. Likewise, methods such as DiffPIR Zhu et al. [ [75] and DAPS Zhang et al. [ [70] optimize or run Langevin MCMC updates on the posterior mean in order to enforce observation consistency. Another line of work approximates the likelihood p(y∣xt)p(y \mid x_t). For instance, DPS Chung et al. [ [9] treats p(x0∣xt)p(x_0 \mid x_t) as a point mass centered at E[x0∣xt]\mathbb{E}[x_0 \mid x_t], while LGD Song et al. [ [58] and DPG Tang et al. [ [61] use a Gaussian distribution with mean E[x0∣xt]\mathbb{E}[x_0 \mid x_t] for approximation. To reduce this approximation error, some methods trade off computational cost: Monte Carlo–based approaches Dou and Song [ [13], Cardoso et al. [ [6], Huang et al. [ [23] and variational inference–based approaches Wu et al. [ [68] avoid the direct mean approximation, but they either require simulating a large number of samples or solving intermediate optimization problems during sampling, both of which are computationally expensive. Finally, there are methods that explicitly aim to reduce the Jensen’s Gap by modifying the sampling dynamics. Examples include mid-point schemes Moufad et al. [ [47] and user-defined intermediate potentials Dong et al. [ [12]. While these approaches can shorten the gap, they still rely on E[x0∣xt]\mathbb{E}[x_0 \mid x_t] for posterior estimation, and moreover, they often incur high computational cost due to additional variational inference or Langevin MCMC steps.

Distillation of Diffusion Models

Sampling in diffusion models involves integrating through a reverse diffusion process, which is computationally expensive. Even with the aid of high-order ODE solvers Maoutsa et al. [ [46], Song et al. [ [57], Lu et al. [ [41], Zhou et al. [ [73], parallel sampling Shih et al. [ [54], and better training schedules Karras et al. [ [28], Nichol and Dhariwal [ [49], Liu et al. [ [38], the process remains iterative and typically requires hundreds of network forward passes. To alleviate this inefficiency, distillation-based methods have been developed to enable one-step generation by leveraging the deterministic nature of samplers (e.g., DDIM), where the noise–data pairs become fixed. The most basic formulation, Knowledge Distillation Luhman and Luhman [ [44], trains a student model to replicate the teacher’s deterministic noise-to-data mapping. However, subsequent studies have shown that directly learning this raw mapping is challenging for neural networks, as the high curvature of sampling trajectories often yields noise–data pairs that are distant in Euclidean space, making the regression task ill-conditioned and hard to generalize.

To address this, recent research has proposed three complementary strategies. (1) Noise–data coupling refinement: Rectified Flow Liu et al. [ [39] distills the sampling process into a structure approximating optimal transport, where the learned mapping corresponds to minimal-cost trajectories between noise and data. InstaFlow Liu et al. [ [40] further demonstrates that such near-optimal-transport couplings significantly ease the learning process for student models. (2) Distribution-level distillation: Rather than matching individual noise–data pairs, DMD Yin et al. [ [69] trains the student via score-matching losses that align the overall data distribution, thereby bypassing the need to regress complex mappings directly. (3) Trajectory distillation: Instead of only supervising on initial (xT)(x_T) and final (x0)(x_0) states, this approach provides supervision at intermediate states xtx_t along the ODE trajectory Berthelot et al. [ [5], Zheng et al. [ [71], Song et al. [ [60], He et al. [2024]. This decomposition allows the student model to learn the generative process in a piecewise manner, which improves stability and sample fidelity. We note that among existing approaches, Physics-Informed Distillation (PID) Tee et al. [ [62] has a name similar to our method’s but differs fundamentally in both objective and methodology. Specifically, PID distills ODE trajectories from teacher models using a PINN-like strategy, whereas our method distills diffusion models for PDE-constrained generation by applying physical supervision directly to the final samples.

Mixture-of-Gaussians (MoG) Dataset

To study the sampling-time behavior of constrained diffusion models, we design a synthetic 2D Mixture-of-Gaussians (MoG) dataset with analytical score functions. Each sample x=(x1,x2)∈R2x=(x_1,x_2)\in\mathbb{R}^2 consists of a data dimension x1x_1 and a fixed latent code x2x_2 that serves as a hard constraint.

Specifically, we define a mixture model where x1x_1 is sampled from a Gaussian mixture conditioned on the latent code z∈{−1,+1}z\in\{-1,+1\}, and x2x_2 is deterministically set to zz. The full distribution is:

x2=z∈{−1,+1},x1∼N(μz,σ2),(7)x_2=z\in\{-1,+1\},\qquad x_1\sim\mathcal{N}(\mu_z,\sigma^2), \tag*{(7)}

with μ−1=−1\mu_{-1}=-1, μ+1=+1\mu_{+1}=+1, and fixed standard deviation σ=0.2\sigma=0.2. The full 2D data point is thus given by:

x=[x1x2],with x1∼N(μx2,σ2),x2∈{−1,+1}.(8)x=\begin{bmatrix}x_1\\x_2\end{bmatrix},\qquad\text{with }x_1\sim\mathcal{N}(\mu_{x_2},\sigma^2),\quad x_2\in\{-1,+1\}. \tag*{(8)}

The resulting joint density p(x)p(x) is a mixture of two Gaussians supported on parallel horizontal lines:

p(x)=12N(x1;−1,σ2)⋅δ(x2+1)+12N(x1;+1,σ2)⋅δ(x2−1),(9)p(x)=\frac{1}{2}\mathcal{N}(x_1;-1,\sigma^2)\cdot\delta(x_2+1)+\frac{1}{2}\mathcal{N}(x_1;+1,\sigma^2)\cdot\delta(x_2-1), \tag*{(9)}

where δ(⋅)\delta(\cdot) denotes the Dirac delta function. In our experiment comparing DPS in Sec. 2.3, we tune the weight of DPS guidance to be 0.035, as it gives stable performance.

Derivation of Score Function of the MoG Dataset

MoG distributions admit analytical diffusion objectives. Specifically, consider a MoG with the form:

x0∼1K∑k=1KN(μk,σk2⋅I).x_0\sim\frac{1}{K}\sum_{k=1}^{K}\mathcal{N}(\mu_k,\sigma_k^2\cdot I).

where KK is the number of Gaussian components, μk\mu_k and σk2\sigma_k^2 are the means and variances of the Gaussian components, respectively. Suppose the solution of the diffusion process follows:

xt=αtx0+σt⋅ξwhere ξ∼N(0,I).x_t=\alpha_t x_0+\sigma_t\cdot\xi\qquad\text{where }\xi\sim\mathcal{N}(0,I).

Since x0x_0 and ξ\xi are both sampled from Gaussian distributions, their linear combination xtx_t also forms a Gaussian distribution, i.e.,

xt∼1K∑k=1KN(αtμk,(σk2αt2+σt2)⋅I).x_t\sim\frac{1}{K}\sum_{k=1}^{K}\mathcal{N}(\alpha_t\mu_k,(\sigma_k^2\alpha_t^2+\sigma_t^2)\cdot I).

Let pi(xt)=N(xt;αtμi,(σi2αt2+σt2)I)p_i(x_t)=\mathcal{N}(x_t;\alpha_t\mu_i,(\sigma_i^2\alpha_t^2+\sigma_t^2)I). Then,

∇pt(xt)=1K∑i=1K∇xtpi(xt)=1K∑i=1Kpi(xt)⋅−(xt−αtμi)σi2αt2+σt2.\begin{aligned} \nabla p_t(x_t)=\frac{1}{K}\sum_{i=1}^{K}\nabla_{x_t}p_i(x_t) \\ &=\frac{1}{K}\sum_{i=1}^{K}p_i(x_t)\cdot\frac{-(x_t-\alpha_t\mu_i)}{\sigma_i^2\alpha_t^2+\sigma_t^2}. \end{aligned}

We can also calculate the score of xtx_t, i.e.,

∇log⁡pt(xt)=∇pt(xt)pt(xt)=∑i=1Kpi(xt)⋅(−(xt−αtμi)σi2αt2+σt2)∑i=1Kpi(xt).\nabla\log p_t(x_t)=\frac{\nabla p_t(x_t)}{p_t(x_t)} =\frac{\sum_{i=1}^{K}p_i(x_t)\cdot\left(\frac{-(x_t-\alpha_t\mu_i)}{\sigma_i^2\alpha_t^2+\sigma_t^2}\right)} {\sum_{i=1}^{K}p_i(x_t)}.

Deviation of Velocity Field of Reverse ODE and DPS

Diffusion models define a forward diffusion process to perturb the data distribution pdatap_{\mathrm{data}} to a Gaussian distribution. Formally, the diffusion process is an Itô SDE dxt=f(xt)+g(t)dw\mathrm{d}x_t=f(x_t)+g(t)\mathrm{d}w, where dw\mathrm{d}w is the Brownian motion and tt flows forward from 00 to TT. The solution of this diffusion process gives a transition distribution pt(xt∣x0)=N(xt∣αtx0,σt2I)p_t(x_t \mid x_0) = \mathcal{N}(x_t \mid\alpha_t x_0, \sigma_t^2 I), where αt=exp⁡(∫0tf(s) ds)\alpha_t = \exp\left(\int_0^t f(s)\,ds\right) and σt2=1−exp⁡(−∫0tg(s)2 ds)\sigma_t^2 = 1 - \exp\left(-\int_0^t g(s)^2\,ds\right). For the linear interpolation used in our experiments, αt=1−t\alpha_t = 1 - t and σt=t\sigma_t = t. To sample from the diffusion model, a typical approach is to apply a reverse-time SDE that reverses the diffusion process Anderson [1982]:

dxt=(f(xt)−g(t)2∇xtlog⁡pt(xt))dt+g(t)dw‾,d x_t = \left(f(x_t) - g(t)^2 \nabla_{x_t} \log p_t(x_t)\right)dt + g(t)d\overline{w},

where dw‾d\overline{w} is the Brownian motion and tt flows backward from TT to 0. For all reverse-time SDEs, there exist corresponding deterministic processes that share the same density evolution, i.e., {pt(xt)}t=0T\{p_t(x_t)\}_{t=0}^{T} Song et al. [2020]. Specifically, this deterministic process follows an ODE:

dxt=(f(xt)−12g(t)2∇xtlog⁡pt(xt))dt,d x_t = \left(f(x_t) - \frac{1}{2}g(t)^2 \nabla_{x_t} \log p_t(x_t)\right)dt,

where tt flows backward from TT to 0. The deterministic process defines a velocity field,

vGT(x,t)=[f(xt)−12g(t)2∇xtlog⁡pt(xt)].v_{\mathrm{GT}}(x,t) = \left[f(x_t) - \frac{1}{2}g(t)^2\nabla_{x_t}\log p_t(x_t)\right].

Here, we also define the velocity field by v(xt,t)=f(xt)−12g(t)2∇xtlog⁡pt(xt)v(x_t,t) = f(x_t) - \frac{1}{2}g(t)^2\nabla_{x_t}\log p_t(x_t).

The posterior mean can be estimated from the score by:

E[x0∣xt]=xt+σt2∇log⁡pt(xt)αt.\mathbb{E}[x_0 \mid x_t] = \frac{x_t + \sigma_t^2 \nabla\log p_t(x_t)}{\alpha_t}.

The posterior mean can also be estimated from the velocity field by:

E[x0∣xt]=σ˙txt−σtv(xt,t)αtσ˙t−σtα˙t.\mathbb{E}[x_0 \mid x_t] = \frac{\dot{\sigma}_t x_t - \sigma_t v(x_t,t)}{\alpha_t\dot{\sigma}_t - \sigma_t\dot{\alpha}_t}.

Theoretical Explanation of the Effectiveness of Distillation

In this subsection, we provide a theoretical explanation for why the post-hoc distillation stage improves sampling efficiency compared to both standard diffusion models and models with low-rank guidance. Here, we consider a target distribution x0∼N(0,Λ)x_0 \sim\mathcal{N}(0,\Lambda), where Λ=diag⁡(λ1,…,λd−1,0)\Lambda= \operatorname{diag}(\lambda_1,\ldots,\lambda_{d-1},0) is diagonal without loss of generality and 1=λ1≥⋯≥λd−1>01 = \lambda_1 \ge\cdots\ge\lambda_{d-1} > 0. The zero eigenvalue λd=0\lambda_d = 0 is introduced to simulate the hard equality constraint on the data, and the corresponding eigendirection ede_d is referred to as the constraint direction.

We consider the variance-preserving SDE with the following forward process:

dxt=−12βxtdt+βdwt,0≤t≤T,d x_t = -\frac{1}{2}\beta x_t dt + \sqrt{\beta}d w_t,\qquad0 \le t \le T,

where β>0\beta> 0 is a noise schedule and wtw_t is a standard Wiener process Song et al. [2020]. This implies

xt=αtx0+σtϵ,where αt=exp⁡(−12∫0tβdy), σt=1−αt2, and ϵ∼N(0,I).x_t = \alpha_t x_0 + \sigma_t \epsilon,\qquad\text{where }\alpha_t = \exp\left(-\frac{1}{2}\int_0^t \beta d y\right),\ \sigma_t = \sqrt{1-\alpha_t^2},\ \text{and }\epsilon\sim\mathcal{N}(0,I).

One can easily verify that the score function is

∇log⁡pt(x)=−(αt2Λ+σt2I)−1x.\nabla\log p_t(x) = -\left(\alpha_t^2\Lambda+ \sigma_t^2 I\right)^{-1}x.

For ease of exposition, let θ∗:=−diag⁡(αt2Λ+σt2I)−1\theta^* := -\operatorname{diag}\left(\alpha_t^2\Lambda+ \sigma_t^2 I\right)^{-1} denote the diagonal entries and ∇log⁡pt(x)=diag⁡(θ∗)x\nabla\log p_t(x) = \operatorname{diag}(\theta^*)x. In particular, along the constrained direction ede_d, we have θd∗(t)=−σt−2\theta_d^*(t) = -\sigma_t^{-2}, whose magnitude diverges as σt2→0\sigma_t^2 \to0.

Score network. To learn the score function at each time tt, we parameterize the score network as sθ(x,t)=diag⁡(θ)xs_\theta(x,t) = \operatorname{diag}(\theta)x and consider the denoising score matching loss:

Ex0,ϵ∼N(0,I)[∥sθ(xt,t)−∇log⁡pt(xt)∥2].\mathbb{E}_{x_0,\epsilon\sim\mathcal{N}(0,I)}\left[\left\|s_\theta(x_t,t)-\nabla\log p_t(x_t)\right\|^2\right].

Using the above loss, we learn the score function s^θ\hat{s}_{\theta} from training samples {x(i)}i=1M∼N(0,Λ)\{x^{(i)}\}_{i=1}^{M} \sim\mathcal{N}(0,\Lambda) by solving the following problem:

Lt(θ):=1M∑i=1MEϵ∼N(0,I)[∥sθ(αtx0(i)+σtϵ,t)−∇log⁡p(xt∣x0(i))∥2]=1M∑i=1MEϵ∼N(0,I)[∥diag⁡(θ)(αtx0(i)+σtϵ)+1σtϵ∥2]=1M∑i=1MEϵ∼N(0,I)[∥αtdiag⁡(θ)x0(i)+(σtdiag⁡(θ)+1σtI)ϵ∥2]=αt2M∑i=1M∥diag⁡(θ)x0(i)∥2+∥σtdiag⁡(θ)+1σtI∥F2.(10)\begin{aligned} \mathcal{L}_{t}(\theta) &:= \frac{1}{M}\sum_{i=1}^{M}\mathbb{E}_{\epsilon\sim\mathcal{N}(0,I)}\left[\left\|s_{\theta}(\alpha_{t}x_{0}^{(i)}+\sigma_{t}\epsilon,t)-\nabla\log p(x_{t}\mid x_{0}^{(i)})\right\|^{2}\right]\\ &= \frac{1}{M}\sum_{i=1}^{M}\mathbb{E}_{\epsilon\sim\mathcal{N}(0,I)}\left[\left\|\operatorname{diag}(\theta)(\alpha_{t}x_{0}^{(i)}+\sigma_{t}\epsilon)+\frac{1}{\sigma_{t}}\epsilon\right\|^{2}\right]\\ &= \frac{1}{M}\sum_{i=1}^{M}\mathbb{E}_{\epsilon\sim\mathcal{N}(0,I)}\left[\left\|\alpha_{t}\operatorname{diag}(\theta)x_{0}^{(i)}+\left(\sigma_{t}\operatorname{diag}(\theta)+\frac{1}{\sigma_{t}}I\right)\epsilon\right\|^{2}\right]\\ &= \frac{\alpha_{t}^{2}}{M}\sum_{i=1}^{M}\left\|\operatorname{diag}(\theta)x_{0}^{(i)}\right\|^{2}+\left\|\sigma_{t}\operatorname{diag}(\theta)+\frac{1}{\sigma_{t}}I\right\|_{F}^{2}. \tag*{(10)} \end{aligned}

Moreover, the effective score function of diffusion models with guidance Guo et al. [ [18] is

sθg=sθ(xt,t)−γ⟨xt−σtsθ(xt,t)αt,ed⟩ed,s_{\theta}^{g}=s_{\theta}(x_{t},t)-\gamma\left\langle\frac{x_{t}-\sigma_{t}s_{\theta}(x_{t},t)}{\alpha_{t}},e_{d}\right\rangle e_{d},

where the guidance is designed by enforcing the predicted clean data x^0∣t=xt−σtsθ(xt,t)αt\hat{x}_{0\mid t}=\frac{x_{t}-\sigma_{t}s_{\theta}(x_{t},t)}{\alpha_{t}} to have a small constraint-satisfaction loss (x^0∣t⊤ed)2(\hat{x}_{0\mid t}^{\top}e_{d})^{2}, and γ\gamma is the guidance strength. Based on sθ(x,t)=diag⁡(θ)xs_{\theta}(x,t)=\operatorname{diag}(\theta)x, one can express the guided score as sθg=diag⁡(θg)xs_{\theta}^{g}=\operatorname{diag}(\theta^{g})x.

Then, let x^0(ϵ)\hat{x}_{0}(\epsilon) be the data generated from the input random variable ϵ∼N(0,I)\epsilon\sim\mathcal{N}(0,I) through the ODE solver using the trained score function sθs_{\theta}. The student network vw(ϵ)=diag⁡(w)ϵv_{w}(\epsilon)=\operatorname{diag}(w)\epsilon is trained according to (5), i.e.,

Ldistill(w)=Eϵ[∥vw(ϵ)−x^0(ϵ)∥22]+λ(vw(ϵ)⊤ed)2.\mathcal{L}_{\mathrm{distill}}(w)=\mathbb{E}_{\epsilon}\left[\left\|v_{w}(\epsilon)-\hat{x}_{0}(\epsilon)\right\|_{2}^{2}\right]+\lambda(v_{w}(\epsilon)^{\top}e_{d})^{2}.

Constrained direction may be imperfectly learned via vanilla diffusion models and diffusion models with guidance. Based on the above setup, we can analyze the learning ability of these methods in terms of the hard constraint on the final coordinate. In particular, we will show that even in the simple linearized setting, learning the constrained coordinate θd∗(t)\theta_{d}^{\ast}(t) will be slow at small noise levels, i.e., t≪1t\ll1.

Theorem B.1. Suppose that the number of training samples MM is sufficiently large and we apply gradient descent to solve the empirical loss in Problem (10), starting from θ0=1\theta^{0}=1 and using step size η∼Θ(1)\eta\sim\Theta(1) for ℓ\ell iterations. Then, with high probability, for the constrained coordinate dd, the iterates satisfy

θdℓ−θd∗(t)=(1−2ησt2)ℓ(θd0−θd∗(t)),ℓ=0,1,2,…,\theta_{d}^{\ell}-\theta_{d}^{\ast}(t)=\left(1-2\eta\sigma_{t}^{2}\right)^{\ell}\left(\theta_{d}^{0}-\theta_{d}^{\ast}(t)\right),\qquad\ell=0,1,2,\ldots,

and consequently, for any regime with ℓσt2≪1\ell\sigma_{t}^{2}\ll1,

∣θdℓ−θd∗(t)∣∣θd∗(t)∣=1−O(ℓσt2),\frac{\left|\theta_{d}^{\ell}-\theta_{d}^{\ast}(t)\right|}{\left|\theta_{d}^{\ast}(t)\right|}=1-O(\ell\sigma_{t}^{2}),

i.e., the learned score coefficient in the constrained direction remains a constant-factor away from the optimum unless ℓ=Ω(σt−2)\ell=\Omega(\sigma_{t}^{-2}). For diffusion models with guidance, we have

θdℓ,g=(1+γσtαt)θdℓ−γαt.\theta_{d}^{\ell,g}=\left(1+\frac{\gamma\sigma_{t}}{\alpha_{t}}\right)\theta_{d}^{\ell}-\frac{\gamma}{\alpha_{t}}.

Proof. At time tt, the considered denoising score matching loss (10) over the training samples {x(i)}i=1M∼N(0,Σ)\{x^{(i)}\}_{i=1}^{M}\sim\mathcal{N}(0,\Sigma) is as follows:

Lt(θ)=αt2M∑i=1M∥diag⁡(θ)x(i)∥2+∥σtdiag⁡(θ)+1σtI∥F2.(11)\mathcal{L}_{t}(\theta)=\frac{\alpha_{t}^{2}}{M}\sum_{i=1}^{M}\left\|\operatorname{diag}(\theta)x^{(i)}\right\|^{2}+\left\|\sigma_{t}\operatorname{diag}(\theta)+\frac{1}{\sigma_{t}}I\right\|_{F}^{2}. \tag*{(11)}

For ease of exposition, let

λ^j:=1M∑i=1M(xj(i))2.(12)\hat{\lambda}_j := \frac{1}{M}\sum_{i=1}^{M}\left(x_j^{(i)}\right)^2. \tag*{(12)}

Note that since Σ=diag⁡(λ1,…,λd−1,0)\Sigma=\operatorname{diag}(\lambda_1,\ldots,\lambda_{d-1},0) with 1=λ1≥⋯≥λd−1>01=\lambda_1\ge\cdots\ge\lambda_{d-1}>0, it holds with high probability that λ^j=(1−o(1))λj\hat{\lambda}_j=(1-o(1))\lambda_j for all j=1,…,d−1j=1,\ldots,d-1 and λ^d=0\hat{\lambda}_d=0 almost surely. Then, we have

Lt(θ)=∑j=1d(αt2λ^jθj2+(σtθj+1σt)2).\mathcal{L}_t(\theta)=\sum_{j=1}^{d}\left(\alpha_t^2\hat{\lambda}_j\theta_j^2+\left(\sigma_t\theta_j+\frac{1}{\sigma_t}\right)^2\right).

With initialization θ0=1d\theta^0=1_d, the gradient descent update is

θℓ+1=θℓ−η∇Lt(θℓ),ℓ=0,1,2,….\theta^{\ell+1}=\theta^\ell-\eta\nabla\mathcal{L}_t(\theta^\ell),\qquad\ell=0,1,2,\ldots.

This is equivalent to

θjℓ+1=θjℓ−2η((αt2λ^j+σt2)θjℓ+1)=(1−2η(αt2λ^j+σt2))θjℓ−2η,∀j=1,…,d.\begin{aligned} \theta_j^{\ell+1}=\theta_j^\ell-2\eta\left(\left(\alpha_t^2\hat{\lambda}_j+\sigma_t^2\right)\theta_j^\ell+1\right) \\ &=\left(1-2\eta\left(\alpha_t^2\hat{\lambda}_j+\sigma_t^2\right)\right)\theta_j^\ell-2\eta,\quad\forall j=1,\ldots,d. \end{aligned}

Obviously, the optimal solution is

θj∗=−1αt2λ^j+σt2,∀j=1,…,d.\theta_j^*=-\frac{1}{\alpha_t^2\hat{\lambda}_j+\sigma_t^2},\quad\forall j=1,\ldots,d.

Then, we have

θjℓ+1−θj∗=(1−2η(αt2λ^j+σt2))(θjℓ−θj∗).\theta_j^{\ell+1}-\theta_j^*=\left(1-2\eta\left(\alpha_t^2\hat{\lambda}_j+\sigma_t^2\right)\right)\left(\theta_j^\ell-\theta_j^*\right).

To guarantee monotonically decreasing for j=1j=1, we have

η<12(αt2λ^1+σt2)≈12,\eta<\frac{1}{2\left(\alpha_t^2\hat{\lambda}_1+\sigma_t^2\right)}\approx\frac{1}{2},

where the last inequality follows from λ^1≈1\hat{\lambda}_1\approx1. Then, we set η=1/4\eta=1/4. Next, we have

θdℓ+1−θd∗=(1−2ησt2)(θdℓ−θd∗)=(1−σt22)(θdℓ−θd∗).\theta_d^{\ell+1}-\theta_d^*=\left(1-2\eta\sigma_t^2\right)\left(\theta_d^\ell-\theta_d^*\right)=\left(1-\frac{\sigma_t^2}{2}\right)\left(\theta_d^\ell-\theta_d^*\right).

This, together with θd0=1\theta_d^0=1, implies for all ℓ=1,2,…\ell=1,2,\ldots,

θdℓ−θd∗=(1−σt22)ℓ(1−θd∗).\theta_d^\ell-\theta_d^*=\left(1-\frac{\sigma_t^2}{2}\right)^\ell\left(1-\theta_d^*\right).

Then by the approximation (1−2ησt2)ℓ=1−O(ℓσt2)\left(1-2\eta\sigma_t^2\right)^\ell=1-O(\ell\sigma_t^2) when ℓσt2≪1\ell\sigma_t^2\ll1, we can obtain

∣θdℓ−θd∗(t)∣∣θd∗(t)∣=1−O(ℓσt2).\frac{\left|\theta_d^\ell-\theta_d^*(t)\right|}{\left|\theta_d^*(t)\right|}=1-O(\ell\sigma_t^2).

Thus, the learned score function for standard diffusion models, s^θ(x,t)=diag⁡(θℓ)x\hat{s}_\theta(x,t)=\operatorname{diag}(\theta^\ell)x, can remain inaccurate in the constrained coordinate.

Next, we consider the effective score function of diffusion models with guidance:

s^θg(x,t)=s^θ(x,t)−γ⟨x−σts^θ(x,t)αt,ed⟩ed=∑j=1d−1θjℓxjej+(θdℓ−γαt(1−σtθdℓ))xded.\begin{aligned} \hat{s}_\theta^{\mathrm{g}}(x,t)=\hat{s}_\theta(x,t)-\gamma\left\langle\frac{x-\sigma_t\hat{s}_\theta(x,t)}{\alpha_t},e_d\right\rangle e_d \\ &=\sum_{j=1}^{d-1}\theta_j^\ell x_j e_j+\left(\theta_d^\ell-\frac{\gamma}{\alpha_t}\left(1-\sigma_t\theta_d^\ell\right)\right)x_d e_d. \end{aligned}

Then, the last component is

θdℓ,g:=(1+γσtαt)θdℓ−γαt.\theta_d^{\ell,\mathrm{g}}:=\left(1+\frac{\gamma\sigma_t}{\alpha_t}\right)\theta_d^\ell-\frac{\gamma}{\alpha_t}.

This theorem shows that, for vanilla diffusion models, the learning error on the constrained coordinate remains large in gradient descent, leading to slow correction during sampling. Although guidance amplifies the update on the constrained dimension to reduce the sampling error caused by imperfect training of the score function, achieving θdg≈θd∗=−σt−2\theta_d^g \approx\theta_d^* = -\sigma_t^{-2} at small σt\sigma_t typically requires large γ\gamma, which is not reasonable in practice because it may introduce other instability issues. Moreover, this also requires very precise choices of γ\gamma for different inference times tt, which may require substantially more intensive hyperparameter tuning.

Post-hoc distillation Let x^0(ϵ)\hat{x}_0(\epsilon) denote the teacher sample obtained by running an ODE solver (probability flow ODE) starting from noise ϵ∼N(0,I)\epsilon\sim\mathcal{N}(0,I) using a trained score model (with or without guidance). We distill this teacher into a student mapping

vw(ϵ)=diag⁡(w)ϵ,v_w(\epsilon) = \operatorname{diag}(w)\epsilon,

trained with a squared loss plus a hard-constraint penalty on the final coordinate:

Ldistill(w)=Eϵ[∥vw(ϵ)−x^0(ϵ)∥22]+λEϵ[⟨vw(ϵ),ed⟩2],λ≥0.\mathcal{L}_{\mathrm{distill}}(w) = \mathbb{E}_{\epsilon}\left[\left\|v_w(\epsilon)-\hat{x}_0(\epsilon)\right\|_2^2\right] + \lambda\mathbb{E}_{\epsilon}\left[\langle v_w(\epsilon),e_d\rangle^2\right], \qquad\lambda\ge0.

Then, the following theorem characterizes the solution of the distilled student learner and proves the constraint error achieved by the post-hoc distillation.

Theorem B.2. Assume ϵ∼N(0,I)\epsilon\sim\mathcal{N}(0,I) and vw(ϵ)=diag⁡(w)ϵv_w(\epsilon)=\operatorname{diag}(w)\epsilon. Let x^0(ϵ)\hat{x}_0(\epsilon) be any teacher output with E∥x^0(ϵ)∥22<∞\mathbb{E}\|\hat{x}_0(\epsilon)\|_2^2 < \infty. Then Ldistill(w)\mathcal{L}_{\mathrm{distill}}(w) is strictly convex and its unique minimizer satisfies, for each coordinate jj,

wj∗=E[ϵjx^0,j(ϵ)]1+λ1{j=d}.w_j^* = \frac{\mathbb{E}[\epsilon_j\hat{x}_{0,j}(\epsilon)]}{1+\lambda\mathbf{1}\{j=d\}}.

In particular, the constrained coordinate is shrunk by a factor 1/(1+λ)1/(1+\lambda):

wd∗=11+λE[ϵdx^0,d(ϵ)],E[⟨vw∗(ϵ),ed⟩2]=(wd∗)2≤1(1+λ)2E[x^0,d(ϵ)2].w_d^* = \frac{1}{1+\lambda}\mathbb{E}[\epsilon_d\hat{x}_{0,d}(\epsilon)], \qquad\mathbb{E}[\langle v_{w^*}(\epsilon),e_d\rangle^2] = (w_d^*)^2 \le\frac{1}{(1+\lambda)^2}\mathbb{E}[\hat{x}_{0,d}(\epsilon)^2].

Proof. Expand the loss coordinate-wise. Since vw(ϵ)j=wjϵjv_w(\epsilon)_j=w_j\epsilon_j,

Ldistill(w)=∑j=1dE[(wjϵj−x^0,j)2]+λE[(wdϵd)2].\mathcal{L}_{\mathrm{distill}}(w)=\sum_{j=1}^{d}\mathbb{E}\left[(w_j\epsilon_j-\hat{x}_{0,j})^2\right]+\lambda\mathbb{E}\left[(w_d\epsilon_d)^2\right].

Using E[ϵj2]=1\mathbb{E}[\epsilon_j^2]=1, we get

E[(wjϵj−x^0,j)2]=wj2−2wjE[ϵjx^0,j]+E[x^0,j2].\mathbb{E}\left[(w_j\epsilon_j-\hat{x}_{0,j})^2\right]=w_j^2-2w_j\mathbb{E}[\epsilon_j\hat{x}_{0,j}]+\mathbb{E}[\hat{x}_{0,j}^2].

Thus, for j≠dj\ne d, the derivative is 2wj−2E[ϵjx^0,j]2w_j-2\mathbb{E}[\epsilon_j\hat{x}_{0,j}], giving wj∗=E[ϵjx^0,j]w_j^*=\mathbb{E}[\epsilon_j\hat{x}_{0,j}]. For j=dj=d, the coefficient of wd2w_d^2 becomes 1+λ1+\lambda, giving wd∗=E[ϵdx^0,d]/(1+λ)w_d^*=\mathbb{E}[\epsilon_d\hat{x}_{0,d}]/(1+\lambda). The bound on the constraint second moment follows by direct substitution. □\square

Remark. Based on the above theorem, we can see that even if the teacher x^0(ϵ)\hat{x}_0(\epsilon) has a nonzero constraint violation in the dd-th coordinate due to imperfect score learning and/or numerical solver error, the distilled student, through the proposed distillation loss function, can systematically reduce this violation through the penalty parameter λ\lambda. Moreover, the penalization is applied only along one constrained direction, so learning on the remaining directions is still maintained, and the overall generation quality need not be degraded.

Datasets

We consider the following widely used PDE benchmarks. Each dataset contains paired solution and coefficient fields defined on a 128×128128\times128 grid. These datasets are readily accessible from FNO Li et al. [ [35], DiffusionPDE Huang et al. [ [22], and ECI-Sampling Cheng et al. [ [8].

Darcy Flow

We adopt the Darcy Flow setup introduced in DiffusionPDE Huang et al. [ [22], with the dataset released by FNO Li et al. [ [35]. For completeness, we describe the generation process here. Specifically, we consider the steady-state Darcy flow equation on a 2D rectangular domain Ω⊂R2\Omega\subset\mathbb{R}^{2} with no-slip boundary conditions:

−∇⋅(a(c)∇u(c))=q(c),c∈Ω,u(c)=0,c∈∂Ω.-\nabla\cdot(a(c)\nabla u(c)) = q(c), \quad c \in\Omega,\quad u(c) = 0,\quad c \in\partial\Omega.

Here, a(c)a(c) is the spatially varying permeability field with binary values, and q(c)q(c) is set to 1 for constant forcing. The pair (u,a)(u,a) is jointly modeled by the diffusion model.

Inhomogeneous Helmholtz Equation and Poisson Equation

We adopt the setup introduced in DiffusionPDE Huang et al. [ [22], with the dataset released by FNO Li et al. [ [35]. For completeness, we describe the generation process here. As a special case of the inhomogeneous Helmholtz equation, the Poisson equation is obtained by setting k=0k = 0:

∇2u(c)=a(c),c∈Ω,u(c)=0,c∈∂Ω.\nabla^{2}u(c) = a(c),\quad c \in\Omega,\quad u(c) = 0,\quad c \in\partial\Omega.

Here, a(c)a(c) is a piecewise constant forcing function. The pair (u,a)(u,a) is jointly modeled by the diffusion model.

Burgers' Equation

We adopt the Burgers’ Equation setup introduced in DiffusionPDE Huang et al. [ [22], with the dataset released by FNO Li et al. [ [35]. For completeness, we describe the generation process here. We study the 1D viscous Burgers’ equation with periodic boundary conditions on a spatial domain Ω=(0,1)\Omega= (0,1) and temporal domain τ∈(0,T]\tau\in(0,T]:

∂τu(c,τ)+∂c(u2(c,τ)2)=ν∂cc2u(c,τ),u(c,0)=a(c),c∈Ω.\partial_{\tau}u(c,\tau) + \partial_{c}\left(\frac{u^{2}(c,\tau)}{2}\right) = \nu\partial^{2}_{cc}u(c,\tau),\quad u(c,0) = a(c),\quad c \in\Omega.

In our experiments, we set ν=0.01\nu= 0.01. Specifically, we use 128 temporal steps, where each trajectory has shape 128×128128 \times128. The pair (u,a)(u,a) is jointly modeled by the diffusion model.

Stokes Problem

We adopt the Stokes problem setup introduced in ECI-Sampling Cheng et al. [ [8] and use their released generation code. For completeness, we describe the generation process below.

The 1D Stokes problem is governed by the heat equation:

ut=νuxx,x∈[0,1], t∈[0,1],u_{t} = \nu u_{xx},\quad x \in[0,1],\ t \in[0,1],

with the following boundary and initial conditions:

u(x,0)=Ae−kxcos⁡(kx),x∈[0,1],u(0,t)=Acos⁡(ωt),t∈[0,1],u(x,0) = Ae^{-kx}\cos(kx),\quad x \in[0,1],\quad u(0,t) = A\cos(\omega t),\quad t \in[0,1],

where ν≥0\nu\ge0 is the viscosity, A>0A > 0 is the amplitude, ω\omega is the oscillation frequency, and k=ω2νk = \sqrt{\frac{\omega}{2\nu}} controls the spatial decay. The analytical solution is given by:

uexact(x,t)=Ae−kxcos⁡(kx−ωt).u_{\mathrm{exact}}(x,t) = Ae^{-kx}\cos(kx-\omega t).

In our experiments, we set A=2A = 2, k=5k = 5 and take a:=ω∼U[2,8]a := \omega\sim\mathcal{U}[2,8] as the coefficient field to jointly model with uu.

Heat Equation

We adopt the heat equation setup introduced in ECI-Sampling Cheng et al. [ [8] and use their released generation code. For completeness, we describe the generation process below.

The 1D heat (diffusion) equation with periodic boundary conditions is defined as:

ut=αuxx,x∈[0,2π], t∈[0,1],(13)u_{t} = \alpha u_{xx},\quad x \in[0,2\pi],\ t \in[0,1], \tag*{(13)}

with the initial and boundary conditions:

u(x,0)=sin⁡(x+φ),u(0,t)=u(2π,t).u(x,0)=\sin(x+\varphi), \quad u(0,t)=u(2\pi,t).

Here, α\alpha denotes the diffusion coefficient and φ\varphi controls the phase of the sinusoidal initial condition. The exact solution is:

uexact(x,t)=e−αtsin⁡(x+φ).u_{\mathrm{exact}}(x,t)=e^{-\alpha t}\sin(x+\varphi).

In our experiments, we set α=3\alpha=3 and take a:=φ∼U[0,π]a:=\varphi\sim\mathcal{U}[0,\pi] as the coefficient to jointly model with uu.

Navier–Stokes Equation

We adopt the 2D Navier–Stokes (NS) setup from ECI-Sampling Cheng et al. [ [8] and use their released generation code. The NS equation in vorticity form for an incompressible fluid with periodic boundary conditions is given as:

∂tw(x,t)+u(x,t)⋅∇w(x,t)=νΔw(x,t)+f(x),x∈[0,1]2, t∈[0,T],∇⋅u(x,t)=0,x∈[0,1]2, t∈[0,T],w(x,0)=w0(x),x∈[0,1]2.\begin{aligned} \partial_t w(x,t)+u(x,t)\cdot\nabla w(x,t)&=\nu\Delta w(x,t)+f(x), \quad x\in[0,1]^2,\ t\in[0,T],\\ \nabla\cdot u(x,t)&=0, \quad x\in[0,1]^2,\ t\in[0,T],\\ w(x,0)&=w_0(x), \quad x\in[0,1]^2. \end{aligned}

Here, uu denotes the velocity field and w=∇×uw=\nabla\times u is the vorticity. The initial vorticity w0w_0 is sampled from N(0,73/2(−Δ+49I)−5/2)\mathcal{N}(0,7^{3/2}(-\Delta+49I)^{-5/2}), and the forcing term is defined as f(x)=0.12sin⁡(2π(x1+x2)+φ)f(x)=0.1\sqrt{2}\sin(2\pi(x_1+x_2)+\varphi), where φ∼U[0,π/2]\varphi\sim\mathcal{U}[0,\pi/2]. We take a:=w0a:=w_0 as the coefficient to jointly model with uu.

Porous Medium Equation

We use the Porous Medium Equation (PME) setup provided by ECI-Sampling Cheng et al. [ [8], with zero initial and time-varying Dirichlet left boundary conditions:

ut=∇⋅(um∇u),x∈[0,1], t∈[0,1],u(x,0)=0,x∈[0,1],u(0,t)=(mt)1/m,t∈[0,1],u(1,t)=0,t∈[0,1].\begin{aligned} u_t&=\nabla\cdot(u^m\nabla u), \quad x\in[0,1],\ t\in[0,1],\\ u(x,0)&=0, \quad x\in[0,1],\\ u(0,t)&=(mt)^{1/m}, \quad t\in[0,1],\\ u(1,t)&=0, \quad t\in[0,1]. \end{aligned}

The exact solution is uexact(x,t)=(m⋅ReLU⁡(t−x))1/mu_{\mathrm{exact}}(x,t)=(m\cdot\operatorname{ReLU}(t-x))^{1/m}. The exponent mm is sampled from U[1,5]\mathcal{U}[1,5]. We take a:=ma:=m as the coefficient to jointly model with uu.

Stefan Problem

We also adopt the Stefan problem configuration from ECI-Sampling Cheng et al. [ [8], which is a nonlinear case of the Generalized Porous Medium Equation (GPME) with fixed Dirichlet boundary conditions:

ut=∇⋅(k(u)∇u),x∈[0,1], t∈[0,T],u(x,0)=0,x∈[0,1],u(0,t)=1,t∈[0,T],u(1,t)=0,t∈[0,T].\begin{aligned} u_t&=\nabla\cdot(k(u)\nabla u), \quad x\in[0,1],\ t\in[0,T],\\ u(x,0)&=0, \quad x\in[0,1],\\ u(0,t)&=1, \quad t\in[0,T],\\ u(1,t)&=0, \quad t\in[0,T]. \end{aligned}

where k(u)k(u) is a step function defined by a shock value u∗u^*:

k(u)={1,u≥u∗,0,u<u∗.k(u)= \begin{cases} 1, & u\ge u^*,\\ 0, & u<u^*. \end{cases}

The exact solution is:

uexact(x,t)=1[u≥u∗](1−(1−u∗)erf⁡(x/(2t))erf⁡(α)).u_{\mathrm{exact}}(x,t)=\mathbb{1}_{[u\ge u^*]}\left(1-(1-u^*)\frac{\operatorname{erf}(x/(2\sqrt{t}))}{\operatorname{erf}(\alpha)}\right).

where α\alpha satisfies the nonlinear equation (1−u∗)/π=u∗erf⁡(α)αexp⁡(α2)(1-u^*)/\sqrt{\pi}=u^*\operatorname{erf}(\alpha)\alpha\exp(\alpha^2). We follow ECI-Sampling to take a:=u∗∼U[0.55,0.7]a:=u^*\sim\mathcal{U}[0.55,0.7] as the coefficient to jointly model with uu.

Experimental Setup

This section provides details on the model architecture, training configurations for diffusion and distillation, evaluation protocols, and baseline methods.

Model Structure

We follow ECI-sampling Cheng et al. [ [8] and adopt the Fourier Neural Operator (FNO) Li et al. [ [35] as both the teacher diffusion model and the student distillation model. A sinusoidal positional encoding Vaswani et al. [ [64] is appended as an additional input dimension. Specifically, we use a four-layer FNO with a frequency cutoff of 32×3232 \times32, a time embedding dimension of 3232, a hidden channel width of 6464, and a projection dimension of 256256.

Diffusion and Distillation Training Setup

For diffusion training, we employ a standard linear noise schedule [39 a], Lipman et al. [ [37], Liu et al. [ [40] with a batch size of 128128 and a total of 10,00010{,}000 iterations. The model is optimized using Adam Kingma and Ba [ [31] with a learning rate of 3×10−23 \times10^{-2}.

During distillation, we use Euler’s method with 100100 uniformly spaced timesteps from t=1t = 1 to t=0t = 0 for sampling. Every 100100 epochs, we resample 10241024 new noise–data pairs for supervision. Distillation is trained for 20002000 epochs using Adam (learning rate 3×10−23 \times10^{-2}), with early stopping based on the squared norm of the observation loss, i.e., ∥dθ′(ε)−x∥2\lVert d_{\theta^\prime}(\varepsilon) - x \rVert^2.

The physics constraint weight λtrain\lambda_{\mathrm{train}} is set to 1010 for Darcy Flow, Burgers’ Equation, Stokes Problem, Heat Equation, Navier–Stokes, Porous Medium Equation, and Stefan Problem. For Helmholtz and Poisson equations, we increase λtrain\lambda_{\mathrm{train}} to 10610^6 due to the stiffness of these PDEs. All experiments are conducted on an NVIDIA RTX 4090 GPU.

Evaluation Setup

For physics-based data simulation, we evaluate models with and without physics refinement: the number of gradient-based refinement steps NN is set to 00 or 5050. The step size η\eta is aligned with the dataset-specific λtrain\lambda_{\mathrm{train}} used during distillation.

In forward and inverse problems, the observation mask MM defines the known entries. For forward problems, the mask has ones at boundary entries. For partial reconstruction, the mask is sampled randomly with 20%20\% of entries set to 11 (observed), and the rest to 00 (missing). All evaluations are conducted on an NVIDIA RTX 4090 GPU.

Baseline Methods

We describe the configurations of all baseline methods used for comparison. Where necessary, we adapt our diffusion training and sampling codebase to implement their respective constraint mechanisms.

ECI-sampling. We follow the approach of directly substituting hard constraints into the posterior mean E[x0∣xt]\mathbb{E}[x_0 \mid x_t] based on a predefined observation mask. Specifically, we project these constraints at each DDIM step Song et al. [ [57] using a correction operator C\mathcal{C}:

xt−dt=C(x^θ(xt,t))⋅(1−t+dt)+(xt−x^θ(xt,t))⋅(t−dt),(14)x_{t-\mathrm{d}t} = \mathcal{C}(\hat{x}_\theta(x_t,t)) \cdot(1-t+\mathrm{d}t) + (x_t-\hat{x}_\theta(x_t,t)) \cdot(t-\mathrm{d}t), \tag*{(14)}

where tt flows backward from 11 to 00, and x^θ\hat{x}_\theta denotes the posterior mean estimated using Tweedie’s formula.

DiffusionPDE. This method employs diffusion posterior sampling (DPS) Chung et al. [ [9], where each intermediate sample xtx_t is guided by the gradient of the PDE residual evaluated on the posterior mean:

xt−dt=xt+vθ(xt,t)⋅dt−ηt∇xt∥R(x^θ(xt,t))∥2,(15)x_{t-\mathrm{d}t} = x_t + v_\theta(x_t,t) \cdot\mathrm{d}t - \eta_t \nabla_{x_t} \lVert\mathcal{R}(\hat{x}_\theta(x_t,t)) \rVert^2, \tag*{(15)}

where vθ(xt,t)v_\theta(x_t,t) is the learned velocity field from the reverse-time ODE sampler, and ηt\eta_t is a hyperparameter. In our experiments, we set ηt\eta_t equal to λtrain\lambda_{\mathrm{train}} for each dataset.

PIDM. This method incorporates an additional residual loss into the diffusion training objective, evaluated on the posterior mean E[x0∣xt]\mathbb{E}[x_0 \mid x_t]. Specifically, PIDM Bastek et al. [ [3] augments the standard diffusion loss with a physics-based term:

LPIDM=Ldiffusion+λt∥R(x^θ(xt,t))∥2,(16)\mathcal{L}_{\mathrm{PIDM}} = \mathcal{L}_{\mathrm{diffusion}} + \lambda_t \left\lVert R\left(\hat{x}_{\theta}(x_t,t)\right)\right\rVert^2, \tag*{(16)}

where Ldiffusion\mathcal{L}_{\mathrm{diffusion}} is the original diffusion training loss, and λt\lambda_t is the residual loss weight. In our experiments, we set λt\lambda_t to λtrain\lambda_{\mathrm{train}} for each dataset as it gives stable performance.

D-Flow. For this standard method Ben-Hamu et al. [ [4], we build on the official implementation of ECI-sampling Cheng et al. [ [8] and introduce an additional PDE residual loss evaluated on the final sample. The weighting λtrain\lambda_{\mathrm{train}} is aligned with our setup across datasets. Specifically, the implementation follows the D-Flow setup in ECI-sampling Cheng et al. [ [8]: we discretize the sampling trajectory into 100 denoising steps and perform gradient-based optimization on the input noise over 50 iterations to minimize the physics residual loss. At each iteration, gradients are backpropagated through the entire 100-step trajectory, resulting in a total of 5,000 function evaluations (NFE) per sample. This leads to significantly higher computational cost compared to our one-step method.

Teacher. This baseline refers to sampling directly from the trained teacher diffusion model without incorporating any PDE-based constraint or guidance mechanism.

Generative Evaluations on More Datasets

In this section, we report results on more datasets and compare them with other baseline methods, as shown in Table 5. PIDDM consistently improves over the baselines, especially in physics residual error.

Table 5: Generative metrics on various constrained PDEs. PDE error is the MSE of the evaluated physics residual error. The best results are in bold.

DatasetMetricPIDDM-1PIDDM-refECIDiffusionPDED-FlowPIDMFM
HelmholtzMMSE (×10−1\times10^{-1})0.2650.1850.3180.3350.1400.3520.296
HelmholtzSMSE (×10−1\times10^{-1})0.1950.1690.2890.3010.1060.3250.210
HelmholtzPDE Error (×10−9\times10^{-9})0.0540.0342.1351.8120.6801.1422.104
HelmholtzNFE (×103\times10^{3})0.0010.1000.5000.1005.0000.1000.100
Stokes ProblemMMSE (×10−2\times10^{-2})0.2980.1820.3350.3420.3010.3610.310
Stokes ProblemSMSE (×10−2\times10^{-2})0.4250.3120.4550.4690.4410.4840.430
Stokes ProblemPDE Error (×10−3\times10^{-3})0.2410.1940.5850.4980.3180.4320.578
Stokes ProblemNFE (×103\times10^{3})0.0010.1000.5000.1005.0000.1000.100
Heat EquationMMSE (×10−3\times10^{-3})0.9010.8454.6204.6001.4524.5804.544
Heat EquationSMSE (×10−2\times10^{-2})0.8160.7901.6121.5980.8921.5871.565
Heat EquationPDE Error (×10−5\times10^{-5})3.2652.9104.1204.1003.6984.1504.354
Heat EquationNFE (×103\times10^{3})0.0010.1000.5000.1005.0000.1000.100
Navier–Stokes EquationMMSE (×10−4\times10^{-4})0.2850.2640.3020.2990.2880.3060.294
Navier–Stokes EquationSMSE (×10−4\times10^{-4})0.2180.2100.3230.3210.2250.3270.314
Navier–Stokes EquationPDE Error (×10−5\times10^{-5})3.1842.9456.9106.7403.2006.9507.222
Navier–Stokes EquationNFE (×103\times10^{3})0.0010.1000.5000.1005.0000.1000.100
Porous Medium EquationMMSE (×10−3\times10^{-3})4.5554.2107.7427.6985.2037.7627.863
Porous Medium EquationSMSE (×10−1\times10^{-1})2.1432.0512.5732.6022.3272.5892.639
Porous Medium EquationPDE Error (×10−5\times10^{-5})3.4123.1104.9824.9453.5484.9175.523
Porous Medium EquationNFE (×103\times10^{3})0.0010.1000.5000.1005.0000.1000.100
Stefan ProblemMMSE (×10−3\times10^{-3})0.2310.2200.2480.2490.2380.2520.245
Stefan ProblemSMSE (×10−3\times10^{-3})0.2780.2680.3150.3180.2890.3200.307
Stefan ProblemPDE Error (×10−2\times10^{-2})0.0810.0700.4100.3980.0950.4050.458
Stefan ProblemNFE (×103\times10^{3})0.0010.1000.5000.1005.0000.1000.100

Table 5. Generative metrics on various constrained PDEs. PDE error is the MSE of the evaluated physics residual error. The best results are in bold.

Qualitative Results on the Darcy Forward Problem

Figure 4 compares the predicted Darcy pressure fields and their corresponding data- and PDE-error maps for each baseline and for our PIDDM. DiffusionPDE and ECI reproduce the coarse flow pattern but exhibit large pointwise errors and pronounced residual bands. In contrast, PIDDM produces the visually sharpest solution and the lowest error intensities in both maps, confirming the quantitative gains reported in the main text.

Qualitative comparison of ECI-sampling, DiffusionPDE, D-Flow, and PIDDM solution, data error, and PDE error fields

Figure 4. Qualitative comparison on the Darcy forward problem. Each column shows (left) the predicted solution field, (middle) pointwise data error, and (right) PDE residual error. Our PIDDM (bottom row) delivers visibly lower data and PDE errors than other baselines while maintaining sharp solution details.

Additional Experiments

Additional Review-Driven Ablations

We include additional experiments requested during review to clarify component attribution, diversity preservation, teacher-quality sensitivity, architecture robustness, dynamic loss weighting, and latent-optimization stability. These experiments use the same evaluation protocol as the main text and are reported on Darcy unless otherwise specified. The compute-matched comparison is reported in Table 3.

Component attribution. Table 6 separates the effects of one-step distillation and final-sample PDE supervision. Distillation alone mainly provides the efficiency gain but does not improve PDE satisfaction, whereas final-sample PDE supervision is the key component for physical consistency. PIDDM combines both effects and achieves a better quality–compute trade-off.

MetricPIDDM-refDistillTeacherD-Flow
MMSE (×10−2\times10^{-2})0.0370.1210.1080.129
SMSE (×10−2\times10^{-2})0.0020.1020.0690.085
PDE error (×10−4\times10^{-4})0.1481.6421.5850.532
FPD0.3850.8250.7820.995
NFE8011005000

Table 6. Component attribution on Darcy generation. Distill denotes pure one-step distillation without final-sample PDE supervision.

Diversity preservation. Beyond MMSE, SMSE, and FPD, we report sliced Wasserstein distance (SWD) and average pairwise distance difference (APDD). For samples x(1),…,x(N)x^{(1)}, \ldots, x^{(N)}, define

APD⁡=2N(N−1)∑i<j∥xi−xj∥2,APDD⁡=∣APD⁡S−APD⁡T∣APD⁡T,\operatorname{APD} = \frac{2}{N(N-1)} \sum_{i<j} \lVert x_i - x_j \rVert_2,\qquad\operatorname{APDD} = \frac{\lvert\operatorname{APD}_{S} - \operatorname{APD}_{T} \rvert}{\operatorname{APD}_{T}},

where APD⁡S\operatorname{APD}_{S} and APD⁡T\operatorname{APD}_{T} are computed on student and teacher samples, respectively. Table 7 shows that PIDDM preserves diversity better than vanilla distillation, with PIDDM-ref performing best.

MetricPIDDM-1PIDDM-refDistillationTeacher
SWD0.0580.0320.1030.089
APDD0.0500.0320.0670.034

Table 7. Diversity preservation on Darcy generation. Lower SWD and APDD indicate closer distributional diversity to the reference samples.

Teacher quality and architecture robustness. Table 8 shows that student performance improves with teacher checkpoint quality early in training, but gains become marginal after 6k teacher iterations. Table 9 further verifies that PIDDM improves physical consistency over DiffusionPDE under U-Net and CNN auto-encoder backbones, showing that the final-sample supervision strategy is not tied to FNO.

Metric2k4k6k8k10k
MMSE (×10−2\times10^{-2})0.1740.1360.1210.1190.112
SMSE (×10−2\times10^{-2})0.1240.0980.0850.0850.082
PDE error (×10−4\times10^{-4})0.3360.2470.2310.2290.226

Table 8. Sensitivity to teacher checkpoint quality on Darcy generation.

MetricPIDDM(UNet)DiffusionPDE(UNet)PIDDM(AE)DiffusionPDE(AE)
MMSE (×10−2\times10^{-2})0.2430.7530.3100.975
SMSE (×10−2\times10^{-2})0.2110.4310.2570.692
PDE error (×10−4\times10^{-4})0.7431.5660.9021.848
FPD0.9481.6511.1821.906

Table 9. Architecture robustness on Darcy generation. PIDDM improves over posterior-mean guidance when using U-Net and CNN auto-encoder backbones.

Dynamic weighting and latent optimization stability. For stiff systems such as Helmholtz and Poisson, raw PDE residual magnitudes can differ by orders of magnitude across PDE families. We therefore evaluate a dynamic weighting variant that rescales the PDE term by a detached running residual magnitude to target a comparable scale of 10−410^{-4}. Table 10 shows comparable performance to manually tuned weights. Table 11 and Table 12 show that the downstream latent optimization converges quickly and has lower variance across random initializations than DiffusionPDE.

MetricHelmholtz(dynamic)Helmholtz(manual)Poisson(dynamic)Poisson(manual)
MMSE (×10−1\times10^{-1})0.2570.2650.0170.016
SMSE (×10−1\times10^{-1})0.1840.1950.0350.036
PDE error (×10−9\times10^{-9})0.0530.0540.0690.073

Table 10. Dynamic weighting for stiff PDEs. Dynamic weighting rescales the PDE term by a detached running residual magnitude, targeting a comparable residual scale of 10−410^{-4}.

Metric1020406080
PDE loss (×10−4\times10^{-4})0.7270.1730.1520.1490.145
Data MSE0.1190.0120.0030.0020.002

Table 11. Convergence of latent optimization on the Darcy forward task.

MethodForwardInverseReconstruction
PIDDM-ref8.579.498.14
DiffusionPDE46.3954.6142.72

Table 12. Error variance across random initializations. Values are MSE variance (×10−5\times10^{-5}).

Final-sample PDE supervision across distillation backbones. Table 13 compares standard distillation backbones with and without final-sample PDE supervision (PS). Removing PS only slightly changes MSE but increases PDE error by about one order of magnitude, indicating that the backbone mainly affects one-step distillation quality while final-sample PDE supervision drives the physics-consistency gain.

MetricRF-2+PSRF-2 w/o PSDMD+PSDMD w/o PSCM+PSCM w/o PSDiffusionPDEPIDM
MSE (×10−1\times10^{-1})0.1270.1430.2550.2610.2830.2910.6910.380
PDE error (×10−4\times10^{-4})0.0981.3040.1341.2480.0831.3731.5761.248

Table 13. Effect of final-sample PDE supervision (PS) across one-step distillation backbones on the Darcy forward task.

Correlated MoG Constraint Satisfaction

We investigate a controlled Mixture-of-Gaussians (MoG) setting to evaluate constraint satisfaction in generative models. The target distribution is a correlated, two-component Gaussian mixture:

pMoG(x)=12N(x;[−1,−1]T,Σ)+12N(x;[+1,+1]T,Σ),(17)p_{\mathrm{MoG}}(\mathbf{x}) = \frac{1}{2}\mathcal{N}\left(\mathbf{x};[-1,-1]^{\mathsf{T}},\Sigma\right) + \frac{1}{2}\mathcal{N}\left(\mathbf{x};[+1,+1]^{\mathsf{T}},\Sigma\right), \tag*{(17)}

where

Σ=σ2[1ρρ1],σ2=0.04,ρ=0.99999.\Sigma= \sigma^2 \begin{bmatrix} 1 & \rho\\ \rho& 1 \end{bmatrix}, \qquad\sigma^2 = 0.04,\quad\rho= 0.99999.

The high correlation ρ=0.99999\rho= 0.99999 ensures that the analytic score function ∇xlog⁡pMoG(x)\nabla_{\mathbf{x}}\log p_{\mathrm{MoG}}(\mathbf{x}) remains well-defined, despite the near-singular covariance. The physical constraint is defined as:

F(x)=∣x0−x1∣2=0.(18)F(\mathbf{x}) = \lvert x_0-x_1\rvert^2 = 0. \tag*{(18)}

Baselines. DPS and ECI both integrate the analytical score using 1000-step Euler discretization over (0,1)(0,1). DPS applies constraint guidance via gradient descent on F(x)F(\mathbf{x}) at each step, using a loss weight of 300. ECI enforces the constraint by directly projecting the posterior mean to satisfy F(x)=0F(\mathbf{x})=0.

PIDDM. A teacher diffusion model is constructed using a probability-flow ODE with 100-step Euler integration, leveraging the analytic score. It generates 50,000 training pairs (ε,x0)(\varepsilon,x_0) that are used to train a one-step student model, a ReLU-activated MLP with two hidden layers (100 neurons each) via the loss:

Ltrain=1N∑i=1N∣dθ(εi)−x0,i∣2+λtrainF(dθ(εi)),λtrain=1.(19)\mathcal{L}_{\mathrm{train}} = \frac{1}{N}\sum_{i=1}^{N}\left\lvert d_{\theta}(\varepsilon_i)-x_{0,i}\right\rvert^2+\lambda_{\mathrm{train}}F(d_{\theta}(\varepsilon_i)),\qquad\lambda_{\mathrm{train}}=1. \tag*{(19)}

Training uses Adam optimizer (lr = 10−310^{-3}, batch size = 2048). During inference, latent noise ε\varepsilon is optimized via 80 steps of LBFGS with strong-Wolfe line search, learning rate 3×10−33\times10^{-3}, and gradient tolerance 10−710^{-7}, with λinfer=1\lambda_{\mathrm{infer}}=1.

Results. Figure 5(a) shows that all methods recover the bimodal structure of x1x_1. However, as shown in Figure 5(b), DPS fails to fully satisfy the constraint, with x0−x1x_0-x_1 spread over ±10−2\pm10^{-2}, while ECI enforces it exactly but distorts the marginal distribution. In contrast, PIDDM maintains both constraint satisfaction (standard deviation ≈2×10−3\approx2\times10^{-3}) and distributional fidelity.

Comparison of marginal distribution and constraint deviation for correlated MoG

Figure 5. (a) Marginal distribution over x1x_1 (b) Constraint deviation x0−x1x_0-x_1. Constraint satisfaction on correlated MoG. Comparison of generated samples using DPS, ECI, and PIDDM. PIDDM closely matches the target distribution while satisfying constraints.

References

  1. [3]Jan-Hendrik Bastek, WaiChing Sun, and Dennis Kochmann. Physics-informed diffusion models. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=tpYeermigrp.arxiv.org/abs/2403.14404
  2. [4]Heli Ben-Hamu, Omri Puny, Itai Gat, Brian Karrer, Uriel Singer, and Yaron Lipman. D-flow: Differentiating through flows for controlled generation. arXiv preprint arXiv:2402.14017, 2024.
  3. [5]David Berthelot, Arnaud Aufef, Jierui Lin, Dian Ang Yap, Shuangfei Zhai, Siyuan Hu, Daniel Zheng, Walter Talbott, and Eric Gu. Tract: Denoising diffusion models with transitive closure time-distillation. arXiv preprint arXiv:2303.04248, 2023.arxiv.org/abs/2303.04248
  4. [6]Gabriel Cardoso, Yazid Janati El Idrissi, Sylvain Le Corff, and Eric Moulines. Monte carlo guided denoising diffusion models for bayesian linear inverse problems. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=nHESwXvxWk.arxiv.org/abs/2308.07983
  5. [7]Changgu Chen, Libing Yang, Xiaoyan Yang, Lianggangxu Chen, Gaoqi He, Changbo Wang, and Yang Li. Find: Fine-tuning initial noise distribution with policy optimization for diffusion models. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 6735–6744, 2024.arxiv.org/abs/2407.19453
  6. [8]Chaoran Cheng, Boran Han, Danielle C. Maddix, Abdul Fatir Ansari, Andrew Stuart, Michael W. Mahoney, and Bernie Wang. Gradient-free generation for hard-constrained systems. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=teE4pl9fTk.arxiv.org/abs/2412.01786
  7. [9]Hyungjin Chung, Jeongsol Kim, Michael Thompson McCann, Marc Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=0nD9zGAGT0k.arxiv.org/abs/2209.14687
  8. [10]John Crank. The Mathematics of Diffusion. Clarendon Press, Oxford, 2 edition, 1975. ISBN 9780198534112.
  9. [11]Peter A. Davidson. Turbulence: An Introduction for Scientists and Engineers. Oxford University Press, 2015.
  10. [12]Yue-Jiang Dong, Hubery Yin, Fangyikang Wang, Yizhe Zhao, Chao Zhang, Chen Li, and Song-Hai Zhang. DC-DPM: A divide-and-conquer approach for diffusion reverse process. ICLR 2025 Conference Withdrawn Submission, 2024. URL https://openreview.net/forum?id=VbAxCwV2e3.
  11. [13]Zehao Dou and Yang Song. Diffusion posterior sampling for linear inverse problem solving: A filtering perspective. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=tp1XNcHzs1.
  12. [14]Bradley Efron. Tweedie’s formula and selection bias. Journal of the American Statistical Association, 106(496):1602–1614, 2011.pubmed.ncbi.nlm.nih.gov/22505788
  13. [15]Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning, 2024.arxiv.org/abs/2403.03206
  14. [16]Xiang Gao, Meera Sitharam, and Adrian E Roitberg. Bounds on the jensen gap, and implications for mean-concentrated distributions. arXiv preprint arXiv:1712.05267, 2017.arxiv.org/abs/1712.05267
  15. [17]Xiefan Guo, Jinlin Liu, Miaomiao Cui, Jiankai Li, Hongyu Yang, and Di Huang. Initno: Boosting text-to-image diffusion models via initial noise optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9380–9389, 2024a.arxiv.org/abs/2404.04650
  16. [18]Yingqing Guo, Hui Yuan, Yukang Yang, Minshuo Chen, and Mengdi Wang. Gradient guidance for diffusion models: An optimization perspective. In Advances in Neural Information Processing Systems (NeurIPS) 37, pages 90736–90770, 2024b.arxiv.org/abs/2404.14743
  17. [19]Derek Hansen, Danielle C Maddix, Shima Alizadeh, Gaurav Gupta, and Michael W Mahoney. Learning physical models that can respect conservation laws. In International Conference on Machine Learning, pages 12469–12510. PMLR, 2023.DOI
  18. [20]Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance, 2022. URL https://arxiv.org/abs/2207.12598.
  19. [21]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.arxiv.org/abs/2006.11239
  20. [22]Jiahe Huang, Guandao Yang, Zichen Wang, and Jeong Joon Park. DiffusionPDE: Generative PDE-solving under partial observation. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024a. URL https://openreview.net/forum?id=zOI2SBjNOR.arxiv.org/abs/2406.17763
  21. [23]Yujia Huang, Adishree Ghatare, Yuanzhe Liu, Ziniu Hu, Qinsheng Zhang, Chandramouli S Sastry, Siddharth Gururani, Sageev Oore, and Yisong Yue. Symbolic music generation with non-differentiable rule guided diffusion, 2024b. URL https://arxiv.org/abs/2402.14285.
  22. [24]Thomas J. R. Hughes. The Finite Element Method: Linear Static and Dynamic Finite Element Analysis. Dover Publications, Mineola, NY, 2000. ISBN 9780486411811.
  23. [25]Frank P. Incropera, David P. DeWitt, Theodore L. Bergman, and Adrienne S. Lavine. Fundamentals of Heat and Mass Transfer. John Wiley & Sons, Hoboken, NJ, 7 edition, 2011. ISBN 9780470501979.
  24. [26]John David Jackson. Classical Electrodynamics. John Wiley & Sons, 3 edition, 1998.
  25. [27]Christian Jacobsen, Yilin Zhuang, and Karthik Duraisamy. Cocogen: Physically-consistent and conditioned score-based generative models for forward and inverse problems, 2024. URL https://arxiv.org/abs/2312.10527.
  26. [28]Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35:265655–26577, 2022.arxiv.org/abs/2206.00364
  27. [29]Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models, 2022. URL https://arxiv.org/abs/2201.11793.
  28. [30]Gavin Kerrigan, Giosue Migliorini, and Padhraic Smyth. Functional flow matching. arXiv preprint arXiv:2305.17209, 2023.arxiv.org/abs/2305.17209
  29. [31]Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.arxiv.org/abs/1412.6980
  30. [32]Diederik P Kingma, Max Welling, et al. Auto-encoding variational bayes, 2013.arxiv.org/abs/1312.6114
  31. [33]Randall J LeVeque. Finite difference methods for ordinary and partial differential equations: steady-state and time-dependent problems. SIAM, 2007.
  32. [34]Randall J LeVeque and Randall J Leveque. Numerical methods for conservation laws, volume 132. Springer, 1992.
  33. [35]Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895, 2020.arxiv.org/abs/2010.08895
  34. [36]Zongyi Li, Hongkai Zheng, Nikola Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Kamyar Azizzadenesheli, and Anima Anandkumar. Physics-informed neural operator for learning partial differential equations. ACM/JMS Journal of Data Science, 1(3):1–27, 2024.arxiv.org/abs/2111.03794
  35. [37]Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=PvqMRDCt9t.arxiv.org/abs/2210.02747
  36. [38]Xingchao Liu, Chengyue Gong, and qiang liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, 2023a. URL https://openreview.net/forum?id=XVjTT1nw5z.arxiv.org/abs/2209.03003
  37. [39]Xingchao Liu, Chengyue Gong, and qiang liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, 2023b. URL https://openreview.net/forum?id=XVjTT1nw5z.arxiv.org/abs/2209.03003
  38. [40]Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, et al. Instaflow: One step is enough for high-quality diffusion-based text-to-image generation. In The Twelfth International Conference on Learning Representations, 2023c.arxiv.org/abs/2309.06380
  39. [41]Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775–5787, 2022a.arxiv.org/abs/2206.00927
  40. [44]Eric Luhman and Troy Luhman. Knowledge distillation in iterative generative models for improved sampling speed. arXiv preprint arXiv:2101.02388, 2021.arxiv.org/abs/2101.02388
  41. [45]Jiafeng Mao, Xueting Wang, and Kiyoharu Aizawa. The lottery ticket hypothesis in denoising: Towards semantic-driven initialization. In European Conference on Computer Vision, pages 93–109. Springer, 2024.arxiv.org/abs/2312.08872
  42. [46]Dimitra Maoutsa, Sebastian Reich, and Manfred Opper. Interacting particle solutions of fokker–planck equations through gradient–log–density estimation. Entropy, 22(8):802, 2020.
  43. [47]Badr Moufad, Yazid Janati, Lisa Bedin, Alain Durmus, Randal Douc, Eric Moulines, and Jimmy Olsson. Variational diffusion posterior sampling with midpoint guidance, 2024. URL https://arxiv.org/abs/2410.09945.
  44. [48]S Chandra Mouli, Danielle C Maddix, Shima Alizadeh, Gaurav Gupta, Andrew Stuart, Michael W Mahoney, and Yuyang Wang. Using uncertainty quantification to characterize and improve out-of-domain learning for pdes. arXiv preprint arXiv:2403.10642, 2024.arxiv.org/abs/2403.10642
  45. [49]Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pages 8162–8171. PMLR, 2021.arxiv.org/abs/2102.09672
  46. [51]Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686–707, 2019.DOI
  47. [52]Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022.arxiv.org/abs/2112.10752
  48. [53]Nadim Saad, Gaurav Gupta, Shima Alizadeh, and Danielle C Maddix. Guiding continuous operator learning through physics-based boundary constraints. arXiv preprint arXiv:2212.07477, 2022.arxiv.org/abs/2212.07477
  49. [54]Andy Shih, Suneel Belkhale, Stefano Ermon, Dorsa Sadigh, and Nima Anari. Parallel sampling of diffusion models. Advances in Neural Information Processing Systems, 36, 2024.arxiv.org/abs/2305.16317
  50. [55]Dule Shu, Zijie Li, and Amir Barati Farimani. A physics-informed diffusion model for high-fidelity flow field reconstruction. Journal of Computational Physics, 478:111972, 2023.DOI
  51. [56]Gordon D Smith. Numerical solution of partial differential equations: finite difference methods. Oxford university press, 1985.
  52. [57]Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=St1giarCHLP.arxiv.org/abs/2010.02502
  53. [58]Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming-Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for plug-and-play controllable generation. In Andreas Krause, Emma Brunskill, Kyungyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 32483–32498. PMLR, 23–29 Jul 2023. URL https://proceedings.mlr.press/v202/song23k.html.
  54. [59]Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2020.
  55. [60]Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models, 2023b. URL https://arxiv.org/abs/2303.01469.
  56. [61]Haoyue Tang, Tian Xie, Aosong Feng, Hanyu Wang, Chenyang Zhang, and Yang Bai. Solving general noisy inverse problem via posterior sampling: A policy gradient viewpoint, 2024. URL https://arxiv.org/abs/2403.10585.
  57. [62]Joshua Tian Jin Tee, Kang Zhang, Hee Suk Yoon, Dhananjaya Nagaraja Gowda, Chanwoo Kim, and Chang D Yoo. Physics informed distillation for diffusion models. arXiv preprint arXiv:2411.08378, 2024.arxiv.org/abs/2411.08378
  58. [63]Stephen P. Timoshenko and James N. Goodier. Theory of Elasticity. McGraw-Hill, New York, 3 edition, 1970. ISBN 9780070647206.
  59. [64]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.arxiv.org/abs/1706.03762
  60. [65]Hao Wang, Weihua Chen, Chenming Li, Wenjian Huang, Jingkai Zhou, Fan Wang, and Jianguo Zhang. Noise re-sampling for high fidelity image generation. Submitted to ICLR 2025, 2025. URL https://openreview.net/forum?id=GD4Tlqvwrq.
  61. [66]Ruoyu Wang, Huayang Huang, Ye Zhu, Olga Russakovsky, and Yu Wu. The silent prompt: Initial noise as implicit guidance for goal-driven image generation. arXiv preprint arXiv:2412.05101, 2024.arxiv.org/abs/2412.05101
  62. [67]Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model, 2022. URL https://arxiv.org/abs/2212.00490.
  63. [68]Zihui Wu, Yu Sun, Yifan Chen, Bingliang Zhang, Yisong Yue, and Katherine L. Bouman. Principled probabilistic imaging using diffusion models as plug-and-play priors, 2024. URL https://arxiv.org/abs/2405.18782.
  64. [69]Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6613–6623, 2024.arxiv.org/abs/2311.18828
  65. [70]Bingliang Zhang, Wenda Chu, Julius Berner, Chenlin Meng, Anima Anandkumar, and Yang Song. Improving diffusion inverse problem solving with decoupled noise annealing. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page 20895–20905. IEEE, June 2025. doi: 10.1109/cvpr52734.2025.01946. URL http://dx.doi.org/10.1109/CVPR52734.2025.01946.
  66. [71]Hongkai Zheng, Weili Nie, Arash Vahdat, Kamyar Aziz A denesheli, and Anima Anandkumar. Fast sampling of diffusion models via operator learning. In International conference on machine learning, pages 42390–42402. PMLR, 2023.arxiv.org/abs/2211.13449
  67. [72]Hongkai Zheng, Wenda Chu, Bingliang Zhang, Zihui Wu, Austin Wang, Berthy T. Feng, Caifeng Zou, Yu Sun, Nikola Kovachki, Zachary E. Ross, Katherine L. Bouman, and Yisong Yue. Inversebench: Benchmarking plug-and-play diffusion priors for inverse problems in physical sciences, 2025. URL https://arxiv.org/abs/2503.11043.
  68. [73]Zhenyu Zhou, Defang Chen, Can Wang, and Chun Chen. Fast ode-based sampling for diffusion models in around 5 steps. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7777–7786, 2024a.arxiv.org/abs/2312.00094
  69. [74]Zikai Zhou, Shitong Shao, Lichen Bai, Zhiqiang Xu, Bo Han, and Zeke Xie. Golden noise for diffusion models: A learning framework. arXiv preprint arXiv:2411.09502, 2024b.arxiv.org/abs/2411.09502
  70. [75]Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, and Luc Van Gool. Denoising diffusion models for plug-and-play image restoration, 2023. URL https: //arxiv.org/abs/2305.08995.arxiv.org/abs/2305.08995

Paper details

vlnc:W_wdqcp7n8
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 8

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents