WorksComputer Science
Physics-Informed Distillation of Diffusion Models for PDE-Constrained Generation
Abstract
Modeling physical systems in a generative manner offers several advantages, including the ability to handle partial observations, generate diverse solutions, and address both forward and inverse problems. Recently, diffusion models have gained increasing attention in the modeling of physical systems, particularly those governed by partial differential equations (PDEs). However, diffusion models only access noisy data at intermediate steps, making it infeasible to directly enforce constraints on the clean sample at each noisy level. As a workaround, constraints are typically applied to the expectation of clean samples , which is estimated using the learned score network. However, imposing PDE constraints on the expectation does not strictly represent the one on the true clean data, known as Jensen's Gap. This gap creates a trade-off: enforcing PDE constraints may come at the cost of reduced accuracy in generative modeling. To address this, we propose a simple yet effective post-hoc distillation approach, where PDE constraints are not injected directly into the diffusion process, but instead enforced during a post-hoc distillation stage. We term our method as Physics-Informed Distillation of Diffusion Models (PIDDM). This distillation not only facilitates single-step generation with improved PDE satisfaction, but also support both forward and inverse problem solving and reconstruction from randomly partial observation. Extensive experiments across various PDE benchmarks demonstrate that PIDDM significantly improves PDE satisfaction over several recent and competitive baselines, such as PIDM, DiffusionPDE, and ECI-sampling, with less computation overhead. Our approach can shed light on more efficient and effective strategies for incorporating physical constraints into diffusion models.
Introduction
Solving partial differential equations (PDEs) underpins innumerable applications in physics, biology, and engineering, spanning fluid flow Davidson [ [11], heat transfer Incropera et al. [ [25], elasticity Timoshenko and Goodier [ [63], electromagnetism Jackson [ [26], and chemical diffusion Crank [ [10]. Classical discretization schemes such as finite-difference Smith [ [56] and finite-element methods LeVeque [ [33] provide reliable solutions, but their computational cost grows sharply with mesh resolution, dimensionality, and parameter sweeps, limiting their practicality for large-scale or real-time simulations Hughes [ [24]. This bottleneck has fueled a surge of learning-based solvers that approximate or accelerate PDE solutions, from early physics-informed neural networks (PINNs) Raissi et al. [ [51] to modern operator-learning frameworks such as DeepONet Lu et al. [2019], Fourier Neural Operators Li et al. [ [35] and Physics-Informed Neural Operator Li et al. [ [36], offering faster inference, uncertainty quantification, and seamless integration into inverse or data-driven tasks.
Among these learning-based solvers, diffusion models Ho et al. [ [21], Song et al. [ [59] provide a promising framework for generative modeling of physical systems. For PDEs, a diffusion model can learn the joint distribution of solution and coefficient fields, , from data, where
denotes input parameters that satisfy the boundary operator (for example, material properties or initial conditions) and is the corresponding solution that satisfies the PDE operator . After training, the model can sample from this learned distribution, enabling forward simulation (sample given ), inverse recovery (sample given ), and conditional reconstruction (complete missing components of or ) within a single framework, which prior non-diffusion approaches Li et al. [ [35], Lu et al. [2019], Raissi et al. [ [51] do not provide. However, while diffusion models perform well under soft, high-level constraints Rombach et al. [ [52], Esser et al. [ [15], Ho and Salimans [ [20], Chung et al. [ [9], Ben-Hamu et al. [ [4], PDE applications often require strict, low-level constraints dictated by and .
Enforcing such PDE constraints within diffusion models is nontrivial. A core difficulty is that, at an individual noise level , diffusion models operate on noisy variables rather than the clean physical field , where constraints such as are defined. To address this, one option is to reconstruct by running the full deterministic sampling trajectory, but this is computationally expensive since it requires many forward passes, and enforcing constraints through backpropagation often causes gradient issues Bastek et al. [ [3]. A more common alternative is to approximate with the posterior mean , which can be efficiently computed via Tweedie’s formula Bastek et al. [ [3], Huang et al. [ [22], Cheng et al. [ [8], Jacobsen et al. [ [27], Kawar et al. [ [29], Wang et al. [ [67], Zhu et al. [ [75], Zhang et al. [ [70], Song et al. [2023a], Tang et al. [ [61], Huang et al. [ [23] (see the right panel of Figure 1(a)). However, this introduces a mismatch: enforcing constraints on the posterior mean, , is not equivalent to enforcing the expected constraint, , due to Jensen’s inequality. This issue, known as the Jensen’s Gap in inverse-problem diffusion guidance Chung et al. [ [9] and later studied in PDE-constrained diffusion models Bastek et al. [ [3], Huang et al. [ [22], can lead to degraded physical fidelity.

Figure 1. Illustration of physics-constrained diffusion generation and our proposed framework. (a) Existing methods Huang et al. [ [22], Cheng et al. [ [8], Bastek et al. [ [3], Jacobsen et al. [ [27] impose PDE losses or guidance on the posterior mean in diffusion training and sampling, introducing the Jensen’s Gap. (b) We train and sample a diffusion model using vanilla methods to generate a paired noise-data dataset for distillation. (c) Our proposed framework distills the teacher diffusion model and directly enforces physical constraints on the final generated sample , avoiding posterior-mean surrogate constraints in the student objective.
Contributions. We propose an effective framework that enforces PDE constraints in diffusion models via post-hoc distillation, enabling reliable and efficient generation under physical laws. As shown in Figure 1(c), our method sidesteps the limitations of existing constraint-guided diffusion-based approaches by decoupling physics enforcement from the diffusion trajectory. Our main contributions are:
Empirical confirmation of Jensen’s Gap: We provide an empirical demonstration and quantitative analysis of the Jensen’s Gap in PDE-constrained diffusion, a discrepancy that arises when PDE constraints are imposed on the posterior mean , rather than the final clean sample .
Final-sample physics enforcement: PIDDM changes where and when physics is enforced: instead of guiding iterative denoising through posterior-mean surrogates, it imposes PDE supervision on the final output of a post-hoc distilled student. This removes the posterior-mean versus final-sample mismatch from the student objective, while our distributional fidelity claims remain empirical.
Versatile and efficient inference: The distilled student model preserves the full generative capabilities of the teacher, supporting physical simulation, reconstruction, and unified forward and inverse PDE solving within a single model, while enabling one-step generation for fast inference. Experiments across diverse PDEs show that PIDDM surpasses posterior-mean-based methods Huang et al. [2024a], Cheng et al. [2025], Bastek et al. [2025], Jacobsen et al. [2024] in both generation quality and constraint satisfaction.
Problem Setup: Jensen's Gap in Diffusion Models with PDE Constraints
In scientific machine learning, there exist many hard and low-level constraints that are mathematically strict and non-negotiable LeVeque and LeVeque [1992], Hansen et al. [2023], Mouli et al. [2024], Saad et al. [2022]. In this section, we discuss how existing works impose these constraints in diffusion-generated data, and the Jensen’s Gap introduced by posterior-mean guidance in inverse problems Chung et al. [2023] and inherited by PDE-constrained diffusion methods Bastek et al. [2025], Huang et al. [2024a].
Preliminaries on Physical Constraints
Physical constraints are typically expressed as a partial differential equation (PDE) defined over a solution domain , together with a boundary condition operator defined on the coefficient domain :
In practice, the domains and are discretized into a uniform grid, typically of size , and the fields and are evaluated at those grid points to produce the observed data , where diffusion models are trained to learn the joint distribution . While PINNs Raissi et al. [2019] model the mapping with differentiable neural networks to enable automatic differentiation Paszke et al. [2019], Abadi et al. [2016], grid-based approaches commonly approximate the differential operators in via finite difference methods Smith [1985], LeVeque [2007]. To quantify the extent to which a generated sample violates the physical constraints, the physics residual error is often defined by:
Here, measures the discrepancy between the sample and the expected PDE and boundary conditions . The physics residual loss is often defined by the squared norm of this physics residual error, i.e., .
Imposing PDE Constraints in Diffusion Models
The physical constraints are often defined on the clean field , while during training or sampling of the diffusion model, the model only observes the noisy state . Therefore, direct optimization or controlled generation based on the physical residual loss is generally impractical. A practical workaround is to evaluate the constraint on an estimate of from , and a common choice is to use the estimated posterior mean: based on the score network in a diffusion model [3, 22]. As a simplified example, consider the forward process defined as , where denotes the noise level at time and is standard Gaussian noise. Then, the posterior mean can be efficiently estimated via Tweedie’s formula:
where is a learned score function approximating the gradient of the log-density (see Appendix § 9.§ 2 for the derivation for the general diffusion process). Leveraging this approximation, several existing works incorporate PDE constraints by evaluating the PDE residual operator on . For instance, PIDM Bastek et al. [ [3] integrates PDE constraints into a diffusion model at training time by augmenting the standard diffusion objective with an additional PDE residual loss . Similarly, at inference time, DiffusionPDE Huang et al. [ [22] and CoCoGen Jacobsen et al. [ [27] employ diffusion posterior sampling (DPS) Chung et al. [ [9], guiding each intermediate sample using the gradient . On the other hand, ECI-sampling Cheng et al. [ [8] directly applies hard constraints to the posterior mean at each DDIM step using a correction operator (more detailed discussion on their implementations can be found in Appendix § 11.§ 4). Beyond PDE applications, constrained diffusion models for image inverse problems also rely on posterior-mean approximations for true posterior estimation. Representative examples include DDRM Kawar et al. [ [29], DDNM Wang et al. [ [67], LGD Song et al. [ [58], DPG Tang et al. [ [61], SCG Huang et al. [ [23], DCDPM Dong et al. [ [12], mid-point guidance Moufad et al. [ [47], DiffPIR Zhu et al. [ [75], and DAPS Zhang et al. [ [70] (see Appendix § 8.§ 2 for details). While these pioneering methods have been demonstrated to be effective in enforcing PDE constraints within diffusion models, they still suffer a theoretical inconsistency: PDE constraints are enforced on the posterior mean approximation , which is not equivalent to the constraints on the true generated data due to Jensen’s inequality:
This discrepancy is commonly referred to as the Jensen’s Gap Chung et al. [ [9], Bastek et al. [ [3], Huang et al. [ [22], Gao et al. [ [16]. To mitigate this issue, PIDM and DiffusionPDE heuristically down-weight PDE constraints at early denoising steps (large ) in training and sampling, respectively, where the gap is pronounced, and emphasize them near , where the posterior mean approximation improves. ECI-sampling introduces stochastic resampling steps Wang et al. [ [65] to project intermediate samples back toward their correct distribution. Many other methods for image inverse problems also provide partial improvements to reduce approximation error Song et al. [2023a], Tang et al. [ [61], Huang et al. [ [23], Dong et al. [ [12], Moufad et al. [ [47] (Appendix § 8.§ 2). These methods address aspects of the approximation error, but still rely on posterior-mean surrogates during the diffusion trajectory.
Demonstration of the Jensen’s Gap
To better illustrate this posterior-mean mismatch and its negative effect in PDE-constrained generation, we conduct experiments on two synthetic datasets: a Mixture-of-Gaussians (MoG) dataset and a Stokes Problem dataset.
Sampling-time Jensen’s Gap. We demonstrate the sampling-time Jensen’s Gap using the Mixture-of-Gaussians (MoG) dataset, where the score function is analytically tractable, allowing us to isolate the effect of the diffusion process without interference from training error. The MoG is constructed in 2D: the first dimension follows a bimodal Gaussian distribution, while the second dimension encodes a discrete latent variable that serves as a hard constraint. Concretely, the joint distribution is defined as a mixture of two Gaussians, each supported on a distinct horizontal line:
Algorithm 1 PIDDM Training: Physics-Informed Distillation.
Require: Teacher Model , Student Model , Batch Size , Terminal Time , Steps , Step Size , Physics Residual Error , Loss Weight , Learning Rate
repeat
Sample ;
for do
▷ Sampling Phase
end for
▷ Distillation Phase
until Converged
where denotes the Dirac delta function and . To examine the impact of Jensen’s Gap during sampling, we compare Diffusion Posterior Sampling (DPS) Chung et al. [ [9] which uses a latent code to guide the generation, with the ground-truth conditional ODE trajectory derived analytically. We evaluate three representative diffusion processes: Variance-Preserving (VP) Ho et al. [ [21], Sub-VP Song et al. [ [59], and Linear Liu et al. [ [38], and compare their velocity fields during inference to characterize Jensen’s Gap. To quantify amplitude errors, we compute the mean absolute error (MAE) and angular error between the DPS-predicted velocity field and the ground-truth velocity : We observe that both DPS errors are significantly elevated at intermediate timesteps and only diminish as , as shown in Figure 2a. Although DPS achieves accurate sampling in the unconstrained dimension (Figure 2b), it fails to respect the hard constraint in the constrained dimension (Figure 2c). This experiment illustrates the mismatch induced by posterior-mean guidance; it does not claim that single posterior samples along the trajectory resolve the gap.

Figure 2. Empirical illustration of the Jensen’s Gap in physics-constrained diffusion models. (a) Absolute velocity error and angular discrepancy between Diffusion Posterior Sampling (DPS) and the ground-truth conditional ODE velocity on the MoG dataset. (b) and (c) Histograms comparing the first (unconstrained) and second (hard-constrained) dimensions of DPS-sampled MoG data against the ground-truth MoG. (d) Training-time manifestation: diffusion loss comparison on a Stokes problem dataset.
Training-time Jensen’s Gap. We examine the Jensen’s Gap during training using the synthetic Stokes dataset with target distribution . The diffusion model adopts a Fourier Neural Operator (FNO) Li et al. [ [35] architecture and follows a standard linear noise schedule Lipman et al. [ [37], Liu et al. [ [39]. Dataset and training details are given in Appendices C and D. We take PIDM Bastek et al. [ [3] as a representative method, which augments the diffusion loss with a PDE residual term , and compare its performance with standard diffusion training. To assess generative quality, we monitor the diffusion loss, which theoretically corresponds to the evidence lower bound (ELBO) Ho et al. [ [21], Kingma et al. [ [32]. The comparison results are shown in Figure 2d, revealing a significant increase in diffusion loss when the PDE residual loss is incorporated. This suggests that the PDE residual loss does not improve modeling of the PDE-consistent data distribution. This observation also corroborates findings from PIDM Bastek et al. [ [3], which identified that residual supervision on the posterior mean can create “a conflicting objective between the data and residual loss”, where the data loss represents the original diffusion training objective. These results provide further evidence of the posterior-mean mismatch in training, as enforcing constraints on may interfere with maximizing the likelihood of the true data distribution.
Method: Physics-Informed Distillation of Diffusion Models
In Section 2, we have demonstrated the posterior-mean mismatch that appears when incorporating physical constraints into diffusion training and sampling, as observed in prior works. To address this issue, we propose a distillation-based framework that sidesteps this mismatch in the student objective. Specifically, instead of enforcing constraints on the posterior mean during the diffusion process, we apply physical constraints directly to the final generated samples in a post-hoc distillation stage.
Diffusion Training
To decouple physical constraint enforcement from the diffusion process itself, we first conduct standard diffusion model training using its original denoising objective, without adding any constraint-based loss. To obtain smoother sampling trajectories that benefit later noise-data distillation Liu et al. [ [39], we adopt a linear diffusion process and apply the -prediction parameterization Liu et al. [ [39], Lipman et al. [ [37], Liu et al. [ [40], Cheng et al. [ [8], Esser et al. [ [15], which is Algorithm 2 PIDDM Inference: Physics Data Simulation
Input Student Model , Physics Residual Error , Refinement Step Number , Refinement Step Size , Latent Noise .
for do
end for
Output
Algorithm 3 PIDDM Inference for Forward/Inverse/Reconstruction
Input Student Model , Physics Residual Error , Optimization Iterations , Step Size , Observation , Observation Mask , Loss Weight , Latent Noise .
for do
end for
Output
commonly referred to as a flow model. Specifically, the training objective is
where , is the distribution of joint data containing both solution and coefficient fields , is sampled from a standard Gaussian distribution, and is the neural network used as the diffusion model. This formulation allows the model to learn to reverse the diffusion process without entangling it with physical supervision, thereby preserving generative fidelity.
Imposing PDE Constraints in Distillation
After training the teacher diffusion model using the standard denoising objective, we proceed to the distillation stage, where we transfer its knowledge to a student model designed for efficient one-step generation. Crucially, this post-hoc distillation stage is where we impose PDE constraints, moving physics enforcement away from posterior-mean surrogates used during diffusion training or sampling. This distillation process is guided by two complementary objectives: (1) learning to map a noise sample to the final generated output predicted by the teacher model, and (2) enforcing physical consistency on this output via PDE residual minimization. Concretely, we begin by sampling a noise input and generating a target sample using the pre-trained teacher model via deterministic integration of the reverse-time ODE:
which proceeds from to using a fixed step size . This yields a paired noise-data dataset for distillation, as shown in Figure 1(b). A student model is then trained to predict in one step, as shown in Figure 1(c). Meanwhile, to enforce physical consistency, we evaluate the physics residual error on the output , i.e., . The overall training objective is:
where balances generative fidelity and physical constraint satisfaction. Unlike prior work Bastek et al. [ [3], Huang et al. [ [22], the residual is evaluated on final student outputs rather than posterior-mean surrogates , which makes the physical objective directly aligned with sample-level constraint satisfaction (see Table 4). Training is repeated until convergence (Algorithm 1). This noise-to-sample distillation is often difficult to learn due to the high curvature of sampling trajectories, yielding noise–data pairs that are far apart in Euclidean space Liu et al. [ [40]. To reduce curvature and improve learnability, we adopt linear-flow distillation Lipman et al. [ [37], Liu et al. [ [39], and we further evaluate Distribution Matching Distillation (DMD) Yin et al. [ [69], Rectified Flow, and Consistency Model Song et al. [ [60] to strengthen coupling and distribution alignment. These choices produce consistent gains (see Table 4).
| Task | Metric | PIDDM | +RF-1 | +RF-2 | +DMD | +CM | −HC | VP | sub-VP |
| Forward | MSE () | 0.316 | 0.278 | 0.127 | 0.255 | 0.283 | 0.705 | 0.398 | 0.372 |
| PDE Error () | 0.145 | 0.129 | 0.098 | 0.134 | 0.083 | 0.354 | 0.154 | 0.157 | |
| Inverse | MSE () | 0.236 | 0.195 | 0.136 | 0.188 | 0.182 | 0.503 | 0.284 | 0.271 |
| PDE Error () | 0.115 | 0.126 | 0.079 | 0.121 | 0.109 | 0.321 | 0.143 | 0.139 | |
| Reconstruct | Coef MSE () | 0.128 | 0.107 | 0.091 | 0.095 | 0.085 | 0.294 | 0.133 | 0.138 |
| Reconstruct | Sol MSE () | 0.102 | 0.084 | 0.063 | 0.073 | 0.072 | 0.239 | 0.127 | 0.119 |
| Reconstruct | PDE Error () | 0.143 | 0.118 | 0.085 | 0.104 | 0.083 | 0.464 | 0.159 | 0.158 |
Table 4. Evaluation on various downstream tasks on Darcy datasets under different PIDDM settings. PIDDM denotes the raw method, +RF-1 and +RF-2 denote one and two rounds of reflowing Liu et al. [ [40], +DMD denotes distribution matching distillation Yin et al. [ [69], and +CM denotes the consistency model Song et al. [ [60]. −HC refers to the ablation without the hard-constraint replacement in Eq. (6) during PIDDM inference. VP and sub-VP refer to ablations on VP and sub-VP diffusion processes. PDE error is the MSE of the evaluated physics residual error. The best results are in bold.
To clarify the optimization mechanism, Appendix § 9.§ 3 analyzes a low-rank Gaussian target, where the low-rank direction mimics a hard equality constraint in PDE problems. The main intuition is that diffusion training and posterior-mean guidance operate in score space along a noisy trajectory. Near the clean endpoint, the score component associated with the constrained direction becomes increasingly sharp, so small approximation errors in this component can persist as visible constraint violations in generated samples. PIDDM instead moves the physics penalty to data space after the endpoint sample is formed. The student therefore does not need to resolve the sharp score field at every noise level; it is trained to suppress the violating component of its final output directly. This explains why final-sample distillation can reduce sample-level PDE residuals, while our distributional fidelity claims remain empirical rather than an end-to-end theoretical guarantee.
Downstream Tasks
Our method naturally supports one-step generation of physically constrained data, jointly producing both coefficient and solution fields. Beyond this intrinsic functionality, it also retains the flexibility of the teacher diffusion model, enabling various downstream tasks such as forward and inverse problem solving, and reconstruction from partial observations. Compared to the teacher model, our method achieves these capabilities with improved computational efficiency and stronger physical alignment.
Generative Modeling. We aim to sample physically consistent pairs from a learned distribution that satisfies the governing PDE system. The student model supports this via efficient one-step generation: given , it outputs , approximating a valid solution–coefficient pair. We further provide an optional refinement stage based on constraint-driven optimization (Algorithm 2), which reduces the physics residual by updating with gradient descent. This design is inspired by noise prompting methods Ben-Hamu et al. [ [4], Guo et al. [ [17] that optimize the final sample with respect to the initial noise. However, in contrast to those prior works, which backpropagate through an entire sampling trajectory and incur high cost and unstable gradients, our refinement operates in a one-step setting. While optional, it offers additional control that is useful in scientific applications requiring strict physical consistency LeVeque and Leveque [ [34], Hansen et al. [ [19], Mouli et al. [ [48], Saad et al. [ [53].
Forward/Inverse Problem and Reconstruction. PIDDM handles all downstream problems as conditional generation over the joint field . Forward inference draws from known ; inverse inference recovers from observed ; reconstruction fills in missing entries of given a partial observation . We solve this via optimization-based inference on the latent variable , using the same student model as in generation, as described in Algorithm 3. Let denote the generated sample, and let be a binary observation mask indicating the known entries in with respect to . To ensure hard consistency with observed values (e.g., boundary conditions ), we define a mixed sample by injecting observed entries into the generated output, following ECI-sampling Cheng et al. [ [8], and then update using a combined objective:
Interestingly, we also find that applying this masking not only enhances hard constraints on , but also improves satisfaction of , as demonstrated in our ablation study in Table 4. Classical inverse solvers Li et al. [ [35, 36], Lu et al. [2019], Raissi et al. [ [51] learn a deterministic map and therefore require full observations of to evaluate , a condition rarely met in practice. DiffusionPDE Huang et al. [ [22] relaxes this by sampling missing variables, but enforces physics on the posterior mean, i.e. , and thus suffers from the Jensen’s Gap. Our method avoids this inconsistency by imposing constraints directly on the final sample , yielding more reliable and physically consistent inverse solutions.
Experiments
Experiment Setup. We consider three widely used PDE benchmarks in the main text: Darcy flow, Poisson equation, and Burgers’ equation. These datasets are readily accessible from FNO Li et al. [ [35] and DiffusionPDE Huang et al. [ [22]. We also provide results on other benchmarks in Appendix § 12. We consider ECI Cheng et al. [ [8], DiffusionPDE Huang et al. [ [22], D-Flow Cheng et al. [ [8], Ben-Hamu et al. [ [4], PIDM Bastek et al. [ [3], and vanilla teacher diffusion models as baseline methods, with detailed implementations in Appendix § 11.§ 4. We follow ECI-sampling Cheng et al. [ [8] to use FNO as both the teacher diffusion model and the student distillation model. We provide the full specification of our experiment setup in Appendix § 11.
To quantitatively evaluate generative performance, we report MMSE, SMSE, FPD, and PDE error following prior work Cheng et al. [ [8], Kerrigan et al. [ [30], Bastek et al. [ [3], Jacobsen et al. [ [27]: MMSE measures the mean squared error of the sample mean; SMSE evaluates the error of the sample standard deviation; FPD evaluates the Frechet distance between hidden representations extracted by the pre-trained PDE foundation model; PDE error quantifies the violation of physical constraints using the physics residual error . The number of function evaluations (NFE) reflects computational cost during inference. For downstream tasks, we further report MSE on solution fields, coefficient fields, or both, depending on the problem setting, reflecting the accuracy of PDE solving.
Empirical Evaluations
PIDDM samples the joint field , enabling forward , inverse , and reconstruction (partial ) tasks (§ 3.3). DiffusionPDE Huang et al. [ [22] reports only reconstruction MSE, while ECI-sampling Cheng et al. [ [8] and PIDM Bastek et al. [ [3] cover at most one task, limited to either unconditional generation or forward solving. For a fair comparison, we evaluate all methods on all three tasks, providing a unified view of generative quality and physical fidelity.
Generative Tasks. We first evaluate the generative performance of our method across three representative PDE systems: Darcy, Poisson, and Burgers’ equation. As shown in Table 1, our one-step model (PIDDM-1) achieves competitive MMSE and SMSE scores while requiring extremely low computational cost (1 NFE). Notably, PIDDM-1 already surpasses prior methods that incorporate physical constraints during training or sampling, such as PIDM, DiffusionPDE, and ECI-sampling, which impose constraints on posterior-mean surrogates and only exhibit marginal improvements over vanilla diffusion baselines. Our optional refinement stage (PIDDM-ref) further reduces both statistical errors and physical PDE residuals, outperforming all baselines. Meanwhile, ECI, which only enforces hard constraints on boundary conditions, achieves moderate improvements but remains less effective on field-level physical consistency. Although D-Flow enforces physical constraints through final-sample trajectory optimization, it requires thousands of NFEs and often suffers from gradient instability.
| Dataset | Metric | PIDDM-1 | PIDDM-ref | ECI | DiffusionPDE | D-Flow | PIDM | Teacher |
| Darcy | MMSE () | 0.112 | 0.037 | 0.153 | 0.419 | 0.129 | 0.515 | 0.108 |
| Darcy | SMSE () | 0.082 | 0.002 | 0.103 | 0.163 | 0.085 | 0.368 | 0.069 |
| Darcy | PDE Error () | 0.226 | 0.148 | 1.582 | 1.071 | 0.532 | 1.236 | 1.585 |
| Darcy | FPD | 0.754 | 0.385 | 0.921 | 1.437 | 0.995 | 1.983 | 0.782 |
| Darcy | NFE () | 0.001 | 0.080 | 0.500 | 0.100 | 5.000 | 0.100 | 0.100 |
| Poisson | MMSE () | 0.162 | 0.113 | 0.183 | 0.861 | 0.172 | 0.948 | 0.150 |
| Poisson | SMSE () | 0.326 | 0.274 | 0.291 | 0.483 | 0.475 | 0.701 | 0.353 |
| Poisson | PDE Error () | 0.073 | 0.050 | 2.420 | 1.270 | 0.831 | 1.593 | 2.443 |
| Poisson | FPD | 1.281 | 0.659 | 1.532 | 1.835 | 1.677 | 2.358 | 1.342 |
| Poisson | NFE () | 0.001 | 0.080 | 0.500 | 0.100 | 5.000 | 0.100 | 0.100 |
| Burgers | MMSE () | 0.152 | 0.012 | 0.294 | 0.064 | 0.305 | 0.948 | 0.264 |
| Burgers | SMSE () | 0.133 | 0.101 | 0.105 | 0.103 | 0.207 | 0.701 | 0.114 |
| Burgers | PDE Error () | 0.466 | 0.174 | 1.572 | 1.032 | 0.730 | 1.593 | 1.334 |
| Burgers | FPD | 0.129 | 0.054 | 0.387 | 1.133 | 0.695 | 1.437 | 0.118 |
| Burgers | NFE () | 0.001 | 0.080 | 0.500 | 0.100 | 5.000 | 0.100 | 0.100 |
Table 1. Generative metrics on various PDE problems. PDE error is the MSE of the evaluated physics residual error. The best results are in bold and the second-best results are underlined.
Compute-Matched Comparison. Table 3 further compares inference cost and quality on Darcy generation under the same hardware. PIDDM-1 uses one forward evaluation and no backward pass, giving millisecond-level generation while improving PDE loss and distributional metrics over posterior-mean-guided baselines. With the same 80-step online budget as DiffusionPDE, PIDDM-ref uses final-sample refinement to obtain lower PDE loss, FPD, MMSE, and SMSE. D-Flow optimizes the final sample directly but requires 2000 forward and backward passes, making it substantially slower despite weaker statistical metrics. Here, NFE denotes the number of forward function evaluations, and BWD denotes the number of inference-time backward passes.
| Method | Time | NFE | BWD | PDE | FPD | MMSE | SMSE |
| PIDDM-1 | 0.004 | 1 | 0 | 0.226 | 0.754 | 0.112 | 0.082 |
| PIDDM-ref | 1.170 | 80 | 80 | 0.148 | 0.385 | 0.037 | 0.002 |
| DiffPDE | 1.250 | 80 | 80 | 1.129 | 1.452 | 0.431 | 0.173 |
| PIDM | 1.281 | 300 | 0 | 1.274 | 2.018 | 0.522 | 0.374 |
| D-Flow | 19.302 | 2000 | 2000 | 0.885 | 1.252 | 0.254 | 0.209 |
| ECI | 1.294 | 300 | 0 | 1.106 | 0.837 | 0.138 | 0.112 |
Table 3. Compute-matched Darcy generation. Best or tied-best entries are in bold.
Forward/Inverse Solving and Reconstruction. We further demonstrate the versatility of our method in forward and inverse problem solving on the Darcy dataset. Since the original PIDM Bastek et al. [ [3] implementation addresses only unconditional generation, we pair it with Diffusion Posterior Sampling (DPS) Chung et al. [ [9] to extend it to downstream tasks (forward, inverse, and reconstruction). Following the test protocol of D-Flow, we apply inference-time optimization over the initial noise to match given observations while satisfying physical laws. As shown in Table 2, our method (PIDDM) achieves the best results across all metrics, including MSE and PDE error, while being significantly more efficient than D-Flow, which requires 5000 function evaluations. Compared to ECI and DiffusionPDE, our method yields lower residuals and better predictive accuracy, reflecting its superior handling of physical and observational constraints jointly.
| Task | Metric | PIDDM | ECI | DiffusionPDE | D-Flow | PIDM |
| Forward | MSE | 0.316 | 0.776 | 0.691 | 0.539 | 0.380 |
| PDE Error | 0.145 | 1.573 | 1.576 | 0.584 | 1.248 | |
| NFE | 0.080 | 0.500 | 0.100 | 5.000 | 0.100 | |
| Inverse | MSE | 0.236 | 0.545 | 0.456 | 0.428 | 0.468 |
| PDE Error | 0.126 | 1.505 | 1.402 | 0.438 | 1.113 | |
| NFE | 0.080 | 0.500 | 0.100 | 5.000 | 0.100 | |
| Reconstruct | Coef MSE | 0.128 | 0.395 | 0.240 | 0.158 | 0.179 |
| Sol MSE | 0.102 | 0.219 | 0.143 | 0.125 | 0.147 | |
| PDE Error | 0.143 | 1.205 | 1.239 | 0.605 | 1.240 | |
| NFE | 0.080 | 0.500 | 0.100 | 5.000 | 0.100 |
Table 2. Evaluation on various downstream tasks on Darcy datasets. PDE error is the MSE of the evaluated physics residual error. The units of MSE, PDE error, and NFE are , , and , respectively. The best results are in bold.
Time is wall-clock seconds per sample; PDE, MMSE, and SMSE are scaled as in Table 1.
Ablation Studies
To better understand the effect of key design choices in PIDDM, we perform ablations on five factors: teacher sampling steps ; distillation weight ; inference weight ; diffusion schedule (VP, sub-VP, linear); and advanced distillation variants (Rectified Flow, DMD, Consistency Model). Figure 3 presents three key ablation studies on the Darcy dataset. Panel (a) shows that increasing the teacher model’s sampling steps consistently improves both generative quality and physical alignment, as reflected by lower MMSE, SMSE, and PDE residuals of the distilled student, highlighting the importance of high-fidelity supervision. Panels (b) and (c) examine the impact of the PDE loss weight during distillation and inference, respectively. We observe that across a wide range of weights, all metrics achieve strong performance. In particular, compared with baselines in Table 2, the PDE error is reduced by an order of magnitude. This improvement is consistent with the design goal of enforcing PDE constraints on final generated samples rather than posterior-mean surrogates.

Figure 3. Ablation studies on the effect of several factors on the performance of PIDDM on the Darcy dataset. Panels (a), (b), and (c) show the effects of , , and , respectively.
We also explore whether more sophisticated distillation strategies can improve the quality of the student model. As shown in Table 4, advanced techniques such as Rectified Flow (RF-1, RF-2), Distribution Matching Distillation (DMD), and Consistency Model yield better MMSE and SMSE than our raw method, while maintaining competitive PDE residuals. This indicates that tighter coupling between noise and data trajectories during distillation facilitates noise-data learning. In addition, we analyze the effect of imposing hard constraints during downstream inference. Following the strategy inspired by ECI-sampling, we directly replace the masked entries in the generated sample with observed values before computing the PDE residual. This ensures that the known information is preserved when evaluating physical consistency. As shown by the −HC variant in Table 4, removing this hard-constraint replacement significantly degrades the PDE residual errors across all tasks. We also validate our design using other linear diffusion processes in Table 4 (i.e., VP and sub-VP).
Appendix G provides additional diagnostics for the main design choices. Table 6 separates one-step distillation from final-sample PDE supervision, showing that distillation mainly gives speed while final-sample supervision drives physical consistency; Table 13 confirms the same pattern across RF, DMD, and consistency-model backbones. Table 7 further shows that PIDDM preserves sample diversity better than vanilla distillation. We also test robustness to teacher quality and architecture in Tables 8 and 9, and report dynamic weighting and latent-optimization stability in Tables 10–12.
Conclusion and Limitation
We propose PIDDM, a lightweight post-hoc distillation framework for physics-constrained diffusion models. Unlike methods that impose PDE constraints on the posterior mean, leading to a Jensen’s Gap and a trade-off between quality and constraint satisfaction, PIDDM enforces constraints directly on the final generated sample. This design improves sample-level physical consistency while empirically preserving distributional fidelity. We provide empirical illustrations of the Jensen’s Gap in diffusion training and sampling. Experiments show PIDDM improves physical and distributional fidelity across forward, inverse, and partial reconstruction tasks, with robustness to hyperparameter choice. The distilled student model also enables efficient one-step physics simulation for repeated-query and latency-sensitive scientific workflows such as uncertainty quantification, online data assimilation, virtual sensing, and design optimization.
Limitations. PIDDM requires a well-trained teacher model and a reliable PDE residual operator, which can be challenging to obtain. The one-step student’s performance may degrade if the teacher is poorly calibrated or lacks diversity. Although performance is robust across a wide range of PDE loss weights, some hyperparameter tuning is still needed to obtain optimal results. Addressing these limitations remains important future work.
Acknowledgements
We sincerely thank the reviewers and the area chair for their careful reading, constructive feedback, and thoughtful suggestions, which helped us improve the clarity and presentation of this work. D. Zou and Y. Zhang are supported in part by NSFC 62306252, Hong Kong ECS award 27309624, Guangdong NSF 2024A1515012444, and the central fund from HKU. P. Wang is supported in part by the University of Macau SRG2025-00043-FST and UMDF-TISF-I/2026/013/FST, and in part by the Macau Science and Technology Development Fund (FDCT) 0091/2025/ITP2.
Impact Statement
This work makes physics-constrained diffusion models practical by enforcing PDE constraints post hoc, sidestepping the posterior-mean mismatch in the student objective and enabling fast single-step generation. By improving physical fidelity and reducing compute and tuning burden, PIDDM can accelerate simulation, inverse design, and data completion workflows in engineering and scientific domains.
Related Work
Diffusion Models
Diffusion models Song et al. [ [59], Ho et al. [ [21], Karras et al. [ [28] learn a score function, , to reverse a predefined diffusion process, typically of the form . A key characteristic of diffusion models is that sampling requires iteratively reversing this process over a sequence of timesteps. This iterative nature presents a challenge for controlled generation: to guide the sampling trajectory effectively, we often need to first estimate the current denoised target in order to determine the correct guidance direction. In other words, to decide how to get there, we must first understand where we are. However, obtaining this information through full iterative sampling is computationally expensive and often impractical in an optimization regime.
A practical workaround is to leverage an implicit one-step data estimate provided by diffusion models via Tweedie’s formula Efron [ [14], which requires only a single network forward pass:
where denotes a one-step denoised estimate of the clean sample from . This approximation improves as . Although this posterior mean is not theoretically equivalent to the final sample obtained after full denoising, in practice, this estimate serves as a useful proxy for the underlying data and enables approximate guidance for controlled generation, without the need to complete the entire sampling trajectory.
Constrained Generation for PDE Systems
Diffusion models have demonstrated strong potential for physical-constraint applications due to their generative nature. This generative capability naturally supports the basic task of simulating physical data and also extends to downstream applications such as reconstruction from partial observations and solving both forward and inverse problems. However, many scientific tasks require strict adherence to physical laws, often expressed as PDE constraints on the data. These constraints, applied at the sample level , are not easily enforced within diffusion models, which are trained to model the data distribution . To address this, prior works have proposed three main strategies for incorporating physical constraints into diffusion models.
Training-time Loss Injection. PG-Diffusion Shu et al. [ [55] employs Classifier-Free Guidance (CFG), where a conditional diffusion model is trained using the PDE residual error as a conditioning input. However, CFG is well known to suffer from theoretical inconsistencies—specifically, the interpolated conditional score function does not match the true conditional score—which limits its suitability for enforcing precise physical constraints. To avoid this issue, PIDM Bastek et al. [ [3] introduces a loss term based on the residual evaluated at the posterior mean, . While this approach avoids the theoretical pitfalls of CFG, the constraint is still not imposed on the actual sample , inheriting the Jensen-gap issue originally discussed in diffusion posterior sampling for inverse problems Chung et al. [ [9].
Sampling-time Guidance. Diffusion Posterior Sampling (DPS), used in DiffusionPDE Huang et al. [ [22] and CoCoGen Jacobsen et al. [ [27], applies guidance during each sampling step by using the gradient of the PDE residual evaluated on the posterior mean . Therefore, they inherit the Jensen’s Gap issue, as the guidance operates on an estimate of the final sample rather than the sample itself. Moreover, DPS assumes that the residual error follows a Gaussian distribution—a condition that may not hold in real-world PDE systems. Meanwhile, to support hard constraints, ECI-sampling Cheng et al. [ [8] directly modifies the posterior mean using known boundary conditions.
Noise Prompting. Another stream of research—often called noise prompting or golden-noise optimization—directly tunes the initial noise so that the resulting sample satisfies a target constraint Ben-Hamu et al. [ [4], Guo et al. [ [17], Zhou et al. [ [74], Wang et al. [ [66], Mao et al. [ [45], Chen et al. [ [7]. In the physics domain, this idea is used to minimize the true PDE residual evaluated on the final sample, rather than the surrogate residual . Because the constraint is imposed on the actual output, noise prompting avoids posterior-mean surrogate constraints and therefore serves as a strong baseline in ECI-sampling Cheng et al. [ [8] and PIDM Bastek et al. [ [3]. The main drawback is efficiency: optimizing the noise requires backpropagating through the entire sampling trajectory, which is computationally expensive and prone to gradient instability.
Recently, diffusion-based techniques for solving image inverse problems have demonstrated competitive performance Zheng et al. [ [72]. However, a common limitation is that they rely on the posterior mean, i.e., , as a surrogate for the true posterior. For example, DDRM Kawar et al. [ [29] and DDNM Wang et al. [ [67] exploit singular value decomposition (SVD) and pseudo-inverse operations to fill in the missing components of during sampling, which is conceptually similar to ECI-sampling. Likewise, methods such as DiffPIR Zhu et al. [ [75] and DAPS Zhang et al. [ [70] optimize or run Langevin MCMC updates on the posterior mean in order to enforce observation consistency. Another line of work approximates the likelihood . For instance, DPS Chung et al. [ [9] treats as a point mass centered at , while LGD Song et al. [ [58] and DPG Tang et al. [ [61] use a Gaussian distribution with mean for approximation. To reduce this approximation error, some methods trade off computational cost: Monte Carlo–based approaches Dou and Song [ [13], Cardoso et al. [ [6], Huang et al. [ [23] and variational inference–based approaches Wu et al. [ [68] avoid the direct mean approximation, but they either require simulating a large number of samples or solving intermediate optimization problems during sampling, both of which are computationally expensive. Finally, there are methods that explicitly aim to reduce the Jensen’s Gap by modifying the sampling dynamics. Examples include mid-point schemes Moufad et al. [ [47] and user-defined intermediate potentials Dong et al. [ [12]. While these approaches can shorten the gap, they still rely on for posterior estimation, and moreover, they often incur high computational cost due to additional variational inference or Langevin MCMC steps.
Distillation of Diffusion Models
Sampling in diffusion models involves integrating through a reverse diffusion process, which is computationally expensive. Even with the aid of high-order ODE solvers Maoutsa et al. [ [46], Song et al. [ [57], Lu et al. [ [41], Zhou et al. [ [73], parallel sampling Shih et al. [ [54], and better training schedules Karras et al. [ [28], Nichol and Dhariwal [ [49], Liu et al. [ [38], the process remains iterative and typically requires hundreds of network forward passes. To alleviate this inefficiency, distillation-based methods have been developed to enable one-step generation by leveraging the deterministic nature of samplers (e.g., DDIM), where the noise–data pairs become fixed. The most basic formulation, Knowledge Distillation Luhman and Luhman [ [44], trains a student model to replicate the teacher’s deterministic noise-to-data mapping. However, subsequent studies have shown that directly learning this raw mapping is challenging for neural networks, as the high curvature of sampling trajectories often yields noise–data pairs that are distant in Euclidean space, making the regression task ill-conditioned and hard to generalize.
To address this, recent research has proposed three complementary strategies. (1) Noise–data coupling refinement: Rectified Flow Liu et al. [ [39] distills the sampling process into a structure approximating optimal transport, where the learned mapping corresponds to minimal-cost trajectories between noise and data. InstaFlow Liu et al. [ [40] further demonstrates that such near-optimal-transport couplings significantly ease the learning process for student models. (2) Distribution-level distillation: Rather than matching individual noise–data pairs, DMD Yin et al. [ [69] trains the student via score-matching losses that align the overall data distribution, thereby bypassing the need to regress complex mappings directly. (3) Trajectory distillation: Instead of only supervising on initial and final states, this approach provides supervision at intermediate states along the ODE trajectory Berthelot et al. [ [5], Zheng et al. [ [71], Song et al. [ [60], He et al. [2024]. This decomposition allows the student model to learn the generative process in a piecewise manner, which improves stability and sample fidelity. We note that among existing approaches, Physics-Informed Distillation (PID) Tee et al. [ [62] has a name similar to our method’s but differs fundamentally in both objective and methodology. Specifically, PID distills ODE trajectories from teacher models using a PINN-like strategy, whereas our method distills diffusion models for PDE-constrained generation by applying physical supervision directly to the final samples.
Mixture-of-Gaussians (MoG) Dataset
To study the sampling-time behavior of constrained diffusion models, we design a synthetic 2D Mixture-of-Gaussians (MoG) dataset with analytical score functions. Each sample consists of a data dimension and a fixed latent code that serves as a hard constraint.
Specifically, we define a mixture model where is sampled from a Gaussian mixture conditioned on the latent code , and is deterministically set to . The full distribution is:
with , , and fixed standard deviation . The full 2D data point is thus given by:
The resulting joint density is a mixture of two Gaussians supported on parallel horizontal lines:
where denotes the Dirac delta function. In our experiment comparing DPS in Sec. 2.3, we tune the weight of DPS guidance to be 0.035, as it gives stable performance.
Derivation of Score Function of the MoG Dataset
MoG distributions admit analytical diffusion objectives. Specifically, consider a MoG with the form:
where is the number of Gaussian components, and are the means and variances of the Gaussian components, respectively. Suppose the solution of the diffusion process follows:
Since and are both sampled from Gaussian distributions, their linear combination also forms a Gaussian distribution, i.e.,
Let . Then,
We can also calculate the score of , i.e.,
Deviation of Velocity Field of Reverse ODE and DPS
Diffusion models define a forward diffusion process to perturb the data distribution to a Gaussian distribution. Formally, the diffusion process is an Itô SDE , where is the Brownian motion and flows forward from to . The solution of this diffusion process gives a transition distribution , where and . For the linear interpolation used in our experiments, and . To sample from the diffusion model, a typical approach is to apply a reverse-time SDE that reverses the diffusion process Anderson [1982]:
where is the Brownian motion and flows backward from to 0. For all reverse-time SDEs, there exist corresponding deterministic processes that share the same density evolution, i.e., Song et al. [2020]. Specifically, this deterministic process follows an ODE:
where flows backward from to 0. The deterministic process defines a velocity field,
Here, we also define the velocity field by .
The posterior mean can be estimated from the score by:
The posterior mean can also be estimated from the velocity field by:
Theoretical Explanation of the Effectiveness of Distillation
In this subsection, we provide a theoretical explanation for why the post-hoc distillation stage improves sampling efficiency compared to both standard diffusion models and models with low-rank guidance. Here, we consider a target distribution , where is diagonal without loss of generality and . The zero eigenvalue is introduced to simulate the hard equality constraint on the data, and the corresponding eigendirection is referred to as the constraint direction.
We consider the variance-preserving SDE with the following forward process:
where is a noise schedule and is a standard Wiener process Song et al. [2020]. This implies
One can easily verify that the score function is
For ease of exposition, let denote the diagonal entries and . In particular, along the constrained direction , we have , whose magnitude diverges as .
Score network. To learn the score function at each time , we parameterize the score network as and consider the denoising score matching loss:
Using the above loss, we learn the score function from training samples by solving the following problem:
Moreover, the effective score function of diffusion models with guidance Guo et al. [ [18] is
where the guidance is designed by enforcing the predicted clean data to have a small constraint-satisfaction loss , and is the guidance strength. Based on , one can express the guided score as .
Then, let be the data generated from the input random variable through the ODE solver using the trained score function . The student network is trained according to (5), i.e.,
Constrained direction may be imperfectly learned via vanilla diffusion models and diffusion models with guidance. Based on the above setup, we can analyze the learning ability of these methods in terms of the hard constraint on the final coordinate. In particular, we will show that even in the simple linearized setting, learning the constrained coordinate will be slow at small noise levels, i.e., .
Theorem B.1. Suppose that the number of training samples is sufficiently large and we apply gradient descent to solve the empirical loss in Problem (10), starting from and using step size for iterations. Then, with high probability, for the constrained coordinate , the iterates satisfy
and consequently, for any regime with ,
i.e., the learned score coefficient in the constrained direction remains a constant-factor away from the optimum unless . For diffusion models with guidance, we have
Proof. At time , the considered denoising score matching loss (10) over the training samples is as follows:
For ease of exposition, let
Note that since with , it holds with high probability that for all and almost surely. Then, we have
With initialization , the gradient descent update is
This is equivalent to
Obviously, the optimal solution is
Then, we have
To guarantee monotonically decreasing for , we have
where the last inequality follows from . Then, we set . Next, we have
This, together with , implies for all ,
Then by the approximation when , we can obtain
Thus, the learned score function for standard diffusion models, , can remain inaccurate in the constrained coordinate.
Next, we consider the effective score function of diffusion models with guidance:
Then, the last component is
This theorem shows that, for vanilla diffusion models, the learning error on the constrained coordinate remains large in gradient descent, leading to slow correction during sampling. Although guidance amplifies the update on the constrained dimension to reduce the sampling error caused by imperfect training of the score function, achieving at small typically requires large , which is not reasonable in practice because it may introduce other instability issues. Moreover, this also requires very precise choices of for different inference times , which may require substantially more intensive hyperparameter tuning.
Post-hoc distillation Let denote the teacher sample obtained by running an ODE solver (probability flow ODE) starting from noise using a trained score model (with or without guidance). We distill this teacher into a student mapping
trained with a squared loss plus a hard-constraint penalty on the final coordinate:
Then, the following theorem characterizes the solution of the distilled student learner and proves the constraint error achieved by the post-hoc distillation.
Theorem B.2. Assume and . Let be any teacher output with . Then is strictly convex and its unique minimizer satisfies, for each coordinate ,
In particular, the constrained coordinate is shrunk by a factor :
Proof. Expand the loss coordinate-wise. Since ,
Using , we get
Thus, for , the derivative is , giving . For , the coefficient of becomes , giving . The bound on the constraint second moment follows by direct substitution.
Remark. Based on the above theorem, we can see that even if the teacher has a nonzero constraint violation in the -th coordinate due to imperfect score learning and/or numerical solver error, the distilled student, through the proposed distillation loss function, can systematically reduce this violation through the penalty parameter . Moreover, the penalization is applied only along one constrained direction, so learning on the remaining directions is still maintained, and the overall generation quality need not be degraded.
Datasets
We consider the following widely used PDE benchmarks. Each dataset contains paired solution and coefficient fields defined on a grid. These datasets are readily accessible from FNO Li et al. [ [35], DiffusionPDE Huang et al. [ [22], and ECI-Sampling Cheng et al. [ [8].
Darcy Flow
We adopt the Darcy Flow setup introduced in DiffusionPDE Huang et al. [ [22], with the dataset released by FNO Li et al. [ [35]. For completeness, we describe the generation process here. Specifically, we consider the steady-state Darcy flow equation on a 2D rectangular domain with no-slip boundary conditions:
Here, is the spatially varying permeability field with binary values, and is set to 1 for constant forcing. The pair is jointly modeled by the diffusion model.
Inhomogeneous Helmholtz Equation and Poisson Equation
We adopt the setup introduced in DiffusionPDE Huang et al. [ [22], with the dataset released by FNO Li et al. [ [35]. For completeness, we describe the generation process here. As a special case of the inhomogeneous Helmholtz equation, the Poisson equation is obtained by setting :
Here, is a piecewise constant forcing function. The pair is jointly modeled by the diffusion model.
Burgers' Equation
We adopt the Burgers’ Equation setup introduced in DiffusionPDE Huang et al. [ [22], with the dataset released by FNO Li et al. [ [35]. For completeness, we describe the generation process here. We study the 1D viscous Burgers’ equation with periodic boundary conditions on a spatial domain and temporal domain :
In our experiments, we set . Specifically, we use 128 temporal steps, where each trajectory has shape . The pair is jointly modeled by the diffusion model.
Stokes Problem
We adopt the Stokes problem setup introduced in ECI-Sampling Cheng et al. [ [8] and use their released generation code. For completeness, we describe the generation process below.
The 1D Stokes problem is governed by the heat equation:
with the following boundary and initial conditions:
where is the viscosity, is the amplitude, is the oscillation frequency, and controls the spatial decay. The analytical solution is given by:
In our experiments, we set , and take as the coefficient field to jointly model with .
Heat Equation
We adopt the heat equation setup introduced in ECI-Sampling Cheng et al. [ [8] and use their released generation code. For completeness, we describe the generation process below.
The 1D heat (diffusion) equation with periodic boundary conditions is defined as:
with the initial and boundary conditions:
Here, denotes the diffusion coefficient and controls the phase of the sinusoidal initial condition. The exact solution is:
In our experiments, we set and take as the coefficient to jointly model with .
Navier–Stokes Equation
We adopt the 2D Navier–Stokes (NS) setup from ECI-Sampling Cheng et al. [ [8] and use their released generation code. The NS equation in vorticity form for an incompressible fluid with periodic boundary conditions is given as:
Here, denotes the velocity field and is the vorticity. The initial vorticity is sampled from , and the forcing term is defined as , where . We take as the coefficient to jointly model with .
Porous Medium Equation
We use the Porous Medium Equation (PME) setup provided by ECI-Sampling Cheng et al. [ [8], with zero initial and time-varying Dirichlet left boundary conditions:
The exact solution is . The exponent is sampled from . We take as the coefficient to jointly model with .
Stefan Problem
We also adopt the Stefan problem configuration from ECI-Sampling Cheng et al. [ [8], which is a nonlinear case of the Generalized Porous Medium Equation (GPME) with fixed Dirichlet boundary conditions:
where is a step function defined by a shock value :
The exact solution is:
where satisfies the nonlinear equation . We follow ECI-Sampling to take as the coefficient to jointly model with .
Experimental Setup
This section provides details on the model architecture, training configurations for diffusion and distillation, evaluation protocols, and baseline methods.
Model Structure
We follow ECI-sampling Cheng et al. [ [8] and adopt the Fourier Neural Operator (FNO) Li et al. [ [35] as both the teacher diffusion model and the student distillation model. A sinusoidal positional encoding Vaswani et al. [ [64] is appended as an additional input dimension. Specifically, we use a four-layer FNO with a frequency cutoff of , a time embedding dimension of , a hidden channel width of , and a projection dimension of .
Diffusion and Distillation Training Setup
For diffusion training, we employ a standard linear noise schedule [39 a], Lipman et al. [ [37], Liu et al. [ [40] with a batch size of and a total of iterations. The model is optimized using Adam Kingma and Ba [ [31] with a learning rate of .
During distillation, we use Euler’s method with uniformly spaced timesteps from to for sampling. Every epochs, we resample new noise–data pairs for supervision. Distillation is trained for epochs using Adam (learning rate ), with early stopping based on the squared norm of the observation loss, i.e., .
The physics constraint weight is set to for Darcy Flow, Burgers’ Equation, Stokes Problem, Heat Equation, Navier–Stokes, Porous Medium Equation, and Stefan Problem. For Helmholtz and Poisson equations, we increase to due to the stiffness of these PDEs. All experiments are conducted on an NVIDIA RTX 4090 GPU.
Evaluation Setup
For physics-based data simulation, we evaluate models with and without physics refinement: the number of gradient-based refinement steps is set to or . The step size is aligned with the dataset-specific used during distillation.
In forward and inverse problems, the observation mask defines the known entries. For forward problems, the mask has ones at boundary entries. For partial reconstruction, the mask is sampled randomly with of entries set to (observed), and the rest to (missing). All evaluations are conducted on an NVIDIA RTX 4090 GPU.
Baseline Methods
We describe the configurations of all baseline methods used for comparison. Where necessary, we adapt our diffusion training and sampling codebase to implement their respective constraint mechanisms.
ECI-sampling. We follow the approach of directly substituting hard constraints into the posterior mean based on a predefined observation mask. Specifically, we project these constraints at each DDIM step Song et al. [ [57] using a correction operator :
where flows backward from to , and denotes the posterior mean estimated using Tweedie’s formula.
DiffusionPDE. This method employs diffusion posterior sampling (DPS) Chung et al. [ [9], where each intermediate sample is guided by the gradient of the PDE residual evaluated on the posterior mean:
where is the learned velocity field from the reverse-time ODE sampler, and is a hyperparameter. In our experiments, we set equal to for each dataset.
PIDM. This method incorporates an additional residual loss into the diffusion training objective, evaluated on the posterior mean . Specifically, PIDM Bastek et al. [ [3] augments the standard diffusion loss with a physics-based term:
where is the original diffusion training loss, and is the residual loss weight. In our experiments, we set to for each dataset as it gives stable performance.
D-Flow. For this standard method Ben-Hamu et al. [ [4], we build on the official implementation of ECI-sampling Cheng et al. [ [8] and introduce an additional PDE residual loss evaluated on the final sample. The weighting is aligned with our setup across datasets. Specifically, the implementation follows the D-Flow setup in ECI-sampling Cheng et al. [ [8]: we discretize the sampling trajectory into 100 denoising steps and perform gradient-based optimization on the input noise over 50 iterations to minimize the physics residual loss. At each iteration, gradients are backpropagated through the entire 100-step trajectory, resulting in a total of 5,000 function evaluations (NFE) per sample. This leads to significantly higher computational cost compared to our one-step method.
Teacher. This baseline refers to sampling directly from the trained teacher diffusion model without incorporating any PDE-based constraint or guidance mechanism.
Generative Evaluations on More Datasets
In this section, we report results on more datasets and compare them with other baseline methods, as shown in Table 5. PIDDM consistently improves over the baselines, especially in physics residual error.
Table 5: Generative metrics on various constrained PDEs. PDE error is the MSE of the evaluated physics residual error. The best results are in bold.
| Dataset | Metric | PIDDM-1 | PIDDM-ref | ECI | DiffusionPDE | D-Flow | PIDM | FM |
| Helmholtz | MMSE () | 0.265 | 0.185 | 0.318 | 0.335 | 0.140 | 0.352 | 0.296 |
| Helmholtz | SMSE () | 0.195 | 0.169 | 0.289 | 0.301 | 0.106 | 0.325 | 0.210 |
| Helmholtz | PDE Error () | 0.054 | 0.034 | 2.135 | 1.812 | 0.680 | 1.142 | 2.104 |
| Helmholtz | NFE () | 0.001 | 0.100 | 0.500 | 0.100 | 5.000 | 0.100 | 0.100 |
| Stokes Problem | MMSE () | 0.298 | 0.182 | 0.335 | 0.342 | 0.301 | 0.361 | 0.310 |
| Stokes Problem | SMSE () | 0.425 | 0.312 | 0.455 | 0.469 | 0.441 | 0.484 | 0.430 |
| Stokes Problem | PDE Error () | 0.241 | 0.194 | 0.585 | 0.498 | 0.318 | 0.432 | 0.578 |
| Stokes Problem | NFE () | 0.001 | 0.100 | 0.500 | 0.100 | 5.000 | 0.100 | 0.100 |
| Heat Equation | MMSE () | 0.901 | 0.845 | 4.620 | 4.600 | 1.452 | 4.580 | 4.544 |
| Heat Equation | SMSE () | 0.816 | 0.790 | 1.612 | 1.598 | 0.892 | 1.587 | 1.565 |
| Heat Equation | PDE Error () | 3.265 | 2.910 | 4.120 | 4.100 | 3.698 | 4.150 | 4.354 |
| Heat Equation | NFE () | 0.001 | 0.100 | 0.500 | 0.100 | 5.000 | 0.100 | 0.100 |
| Navier–Stokes Equation | MMSE () | 0.285 | 0.264 | 0.302 | 0.299 | 0.288 | 0.306 | 0.294 |
| Navier–Stokes Equation | SMSE () | 0.218 | 0.210 | 0.323 | 0.321 | 0.225 | 0.327 | 0.314 |
| Navier–Stokes Equation | PDE Error () | 3.184 | 2.945 | 6.910 | 6.740 | 3.200 | 6.950 | 7.222 |
| Navier–Stokes Equation | NFE () | 0.001 | 0.100 | 0.500 | 0.100 | 5.000 | 0.100 | 0.100 |
| Porous Medium Equation | MMSE () | 4.555 | 4.210 | 7.742 | 7.698 | 5.203 | 7.762 | 7.863 |
| Porous Medium Equation | SMSE () | 2.143 | 2.051 | 2.573 | 2.602 | 2.327 | 2.589 | 2.639 |
| Porous Medium Equation | PDE Error () | 3.412 | 3.110 | 4.982 | 4.945 | 3.548 | 4.917 | 5.523 |
| Porous Medium Equation | NFE () | 0.001 | 0.100 | 0.500 | 0.100 | 5.000 | 0.100 | 0.100 |
| Stefan Problem | MMSE () | 0.231 | 0.220 | 0.248 | 0.249 | 0.238 | 0.252 | 0.245 |
| Stefan Problem | SMSE () | 0.278 | 0.268 | 0.315 | 0.318 | 0.289 | 0.320 | 0.307 |
| Stefan Problem | PDE Error () | 0.081 | 0.070 | 0.410 | 0.398 | 0.095 | 0.405 | 0.458 |
| Stefan Problem | NFE () | 0.001 | 0.100 | 0.500 | 0.100 | 5.000 | 0.100 | 0.100 |
Table 5. Generative metrics on various constrained PDEs. PDE error is the MSE of the evaluated physics residual error. The best results are in bold.
Qualitative Results on the Darcy Forward Problem
Figure 4 compares the predicted Darcy pressure fields and their corresponding data- and PDE-error maps for each baseline and for our PIDDM. DiffusionPDE and ECI reproduce the coarse flow pattern but exhibit large pointwise errors and pronounced residual bands. In contrast, PIDDM produces the visually sharpest solution and the lowest error intensities in both maps, confirming the quantitative gains reported in the main text.

Figure 4. Qualitative comparison on the Darcy forward problem. Each column shows (left) the predicted solution field, (middle) pointwise data error, and (right) PDE residual error. Our PIDDM (bottom row) delivers visibly lower data and PDE errors than other baselines while maintaining sharp solution details.
Additional Experiments
Additional Review-Driven Ablations
We include additional experiments requested during review to clarify component attribution, diversity preservation, teacher-quality sensitivity, architecture robustness, dynamic loss weighting, and latent-optimization stability. These experiments use the same evaluation protocol as the main text and are reported on Darcy unless otherwise specified. The compute-matched comparison is reported in Table 3.
Component attribution. Table 6 separates the effects of one-step distillation and final-sample PDE supervision. Distillation alone mainly provides the efficiency gain but does not improve PDE satisfaction, whereas final-sample PDE supervision is the key component for physical consistency. PIDDM combines both effects and achieves a better quality–compute trade-off.
| Metric | PIDDM-ref | Distill | Teacher | D-Flow |
| MMSE () | 0.037 | 0.121 | 0.108 | 0.129 |
| SMSE () | 0.002 | 0.102 | 0.069 | 0.085 |
| PDE error () | 0.148 | 1.642 | 1.585 | 0.532 |
| FPD | 0.385 | 0.825 | 0.782 | 0.995 |
| NFE | 80 | 1 | 100 | 5000 |
Table 6. Component attribution on Darcy generation. Distill denotes pure one-step distillation without final-sample PDE supervision.
Diversity preservation. Beyond MMSE, SMSE, and FPD, we report sliced Wasserstein distance (SWD) and average pairwise distance difference (APDD). For samples , define
where and are computed on student and teacher samples, respectively. Table 7 shows that PIDDM preserves diversity better than vanilla distillation, with PIDDM-ref performing best.
| Metric | PIDDM-1 | PIDDM-ref | Distillation | Teacher |
| SWD | 0.058 | 0.032 | 0.103 | 0.089 |
| APDD | 0.050 | 0.032 | 0.067 | 0.034 |
Table 7. Diversity preservation on Darcy generation. Lower SWD and APDD indicate closer distributional diversity to the reference samples.
Teacher quality and architecture robustness. Table 8 shows that student performance improves with teacher checkpoint quality early in training, but gains become marginal after 6k teacher iterations. Table 9 further verifies that PIDDM improves physical consistency over DiffusionPDE under U-Net and CNN auto-encoder backbones, showing that the final-sample supervision strategy is not tied to FNO.
| Metric | 2k | 4k | 6k | 8k | 10k |
| MMSE () | 0.174 | 0.136 | 0.121 | 0.119 | 0.112 |
| SMSE () | 0.124 | 0.098 | 0.085 | 0.085 | 0.082 |
| PDE error () | 0.336 | 0.247 | 0.231 | 0.229 | 0.226 |
Table 8. Sensitivity to teacher checkpoint quality on Darcy generation.
| Metric | PIDDM(UNet) | DiffusionPDE(UNet) | PIDDM(AE) | DiffusionPDE(AE) |
| MMSE () | 0.243 | 0.753 | 0.310 | 0.975 |
| SMSE () | 0.211 | 0.431 | 0.257 | 0.692 |
| PDE error () | 0.743 | 1.566 | 0.902 | 1.848 |
| FPD | 0.948 | 1.651 | 1.182 | 1.906 |
Table 9. Architecture robustness on Darcy generation. PIDDM improves over posterior-mean guidance when using U-Net and CNN auto-encoder backbones.
Dynamic weighting and latent optimization stability. For stiff systems such as Helmholtz and Poisson, raw PDE residual magnitudes can differ by orders of magnitude across PDE families. We therefore evaluate a dynamic weighting variant that rescales the PDE term by a detached running residual magnitude to target a comparable scale of . Table 10 shows comparable performance to manually tuned weights. Table 11 and Table 12 show that the downstream latent optimization converges quickly and has lower variance across random initializations than DiffusionPDE.
| Metric | Helmholtz(dynamic) | Helmholtz(manual) | Poisson(dynamic) | Poisson(manual) |
| MMSE () | 0.257 | 0.265 | 0.017 | 0.016 |
| SMSE () | 0.184 | 0.195 | 0.035 | 0.036 |
| PDE error () | 0.053 | 0.054 | 0.069 | 0.073 |
Table 10. Dynamic weighting for stiff PDEs. Dynamic weighting rescales the PDE term by a detached running residual magnitude, targeting a comparable residual scale of .
| Metric | 10 | 20 | 40 | 60 | 80 |
| PDE loss () | 0.727 | 0.173 | 0.152 | 0.149 | 0.145 |
| Data MSE | 0.119 | 0.012 | 0.003 | 0.002 | 0.002 |
Table 11. Convergence of latent optimization on the Darcy forward task.
| Method | Forward | Inverse | Reconstruction |
| PIDDM-ref | 8.57 | 9.49 | 8.14 |
| DiffusionPDE | 46.39 | 54.61 | 42.72 |
Table 12. Error variance across random initializations. Values are MSE variance ().
Final-sample PDE supervision across distillation backbones. Table 13 compares standard distillation backbones with and without final-sample PDE supervision (PS). Removing PS only slightly changes MSE but increases PDE error by about one order of magnitude, indicating that the backbone mainly affects one-step distillation quality while final-sample PDE supervision drives the physics-consistency gain.
| Metric | RF-2+PS | RF-2 w/o PS | DMD+PS | DMD w/o PS | CM+PS | CM w/o PS | DiffusionPDE | PIDM |
| MSE () | 0.127 | 0.143 | 0.255 | 0.261 | 0.283 | 0.291 | 0.691 | 0.380 |
| PDE error () | 0.098 | 1.304 | 0.134 | 1.248 | 0.083 | 1.373 | 1.576 | 1.248 |
Table 13. Effect of final-sample PDE supervision (PS) across one-step distillation backbones on the Darcy forward task.
Correlated MoG Constraint Satisfaction
We investigate a controlled Mixture-of-Gaussians (MoG) setting to evaluate constraint satisfaction in generative models. The target distribution is a correlated, two-component Gaussian mixture:
where
The high correlation ensures that the analytic score function remains well-defined, despite the near-singular covariance. The physical constraint is defined as:
Baselines. DPS and ECI both integrate the analytical score using 1000-step Euler discretization over . DPS applies constraint guidance via gradient descent on at each step, using a loss weight of 300. ECI enforces the constraint by directly projecting the posterior mean to satisfy .
PIDDM. A teacher diffusion model is constructed using a probability-flow ODE with 100-step Euler integration, leveraging the analytic score. It generates 50,000 training pairs that are used to train a one-step student model, a ReLU-activated MLP with two hidden layers (100 neurons each) via the loss:
Training uses Adam optimizer (lr = , batch size = 2048). During inference, latent noise is optimized via 80 steps of LBFGS with strong-Wolfe line search, learning rate , and gradient tolerance , with .
Results. Figure 5(a) shows that all methods recover the bimodal structure of . However, as shown in Figure 5(b), DPS fails to fully satisfy the constraint, with spread over , while ECI enforces it exactly but distorts the marginal distribution. In contrast, PIDDM maintains both constraint satisfaction (standard deviation ) and distributional fidelity.

Figure 5. (a) Marginal distribution over (b) Constraint deviation . Constraint satisfaction on correlated MoG. Comparison of generated samples using DPS, ECI, and PIDDM. PIDDM closely matches the target distribution while satisfying constraints.
References
- [3]Jan-Hendrik Bastek, WaiChing Sun, and Dennis Kochmann. Physics-informed diffusion models. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=tpYeermigrp.arxiv.org/abs/2403.14404
- [4]Heli Ben-Hamu, Omri Puny, Itai Gat, Brian Karrer, Uriel Singer, and Yaron Lipman. D-flow: Differentiating through flows for controlled generation. arXiv preprint arXiv:2402.14017, 2024.
- [5]David Berthelot, Arnaud Aufef, Jierui Lin, Dian Ang Yap, Shuangfei Zhai, Siyuan Hu, Daniel Zheng, Walter Talbott, and Eric Gu. Tract: Denoising diffusion models with transitive closure time-distillation. arXiv preprint arXiv:2303.04248, 2023.arxiv.org/abs/2303.04248
- [6]Gabriel Cardoso, Yazid Janati El Idrissi, Sylvain Le Corff, and Eric Moulines. Monte carlo guided denoising diffusion models for bayesian linear inverse problems. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=nHESwXvxWk.arxiv.org/abs/2308.07983
- [7]Changgu Chen, Libing Yang, Xiaoyan Yang, Lianggangxu Chen, Gaoqi He, Changbo Wang, and Yang Li. Find: Fine-tuning initial noise distribution with policy optimization for diffusion models. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 6735–6744, 2024.arxiv.org/abs/2407.19453
- [8]Chaoran Cheng, Boran Han, Danielle C. Maddix, Abdul Fatir Ansari, Andrew Stuart, Michael W. Mahoney, and Bernie Wang. Gradient-free generation for hard-constrained systems. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=teE4pl9fTk.arxiv.org/abs/2412.01786
- [9]Hyungjin Chung, Jeongsol Kim, Michael Thompson McCann, Marc Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=0nD9zGAGT0k.arxiv.org/abs/2209.14687
- [10]John Crank. The Mathematics of Diffusion. Clarendon Press, Oxford, 2 edition, 1975. ISBN 9780198534112.
- [11]Peter A. Davidson. Turbulence: An Introduction for Scientists and Engineers. Oxford University Press, 2015.
- [12]Yue-Jiang Dong, Hubery Yin, Fangyikang Wang, Yizhe Zhao, Chao Zhang, Chen Li, and Song-Hai Zhang. DC-DPM: A divide-and-conquer approach for diffusion reverse process. ICLR 2025 Conference Withdrawn Submission, 2024. URL https://openreview.net/forum?id=VbAxCwV2e3.
- [13]Zehao Dou and Yang Song. Diffusion posterior sampling for linear inverse problem solving: A filtering perspective. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=tp1XNcHzs1.
- [14]Bradley Efron. Tweedie’s formula and selection bias. Journal of the American Statistical Association, 106(496):1602–1614, 2011.pubmed.ncbi.nlm.nih.gov/22505788
- [15]Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning, 2024.arxiv.org/abs/2403.03206
- [16]Xiang Gao, Meera Sitharam, and Adrian E Roitberg. Bounds on the jensen gap, and implications for mean-concentrated distributions. arXiv preprint arXiv:1712.05267, 2017.arxiv.org/abs/1712.05267
- [17]Xiefan Guo, Jinlin Liu, Miaomiao Cui, Jiankai Li, Hongyu Yang, and Di Huang. Initno: Boosting text-to-image diffusion models via initial noise optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9380–9389, 2024a.arxiv.org/abs/2404.04650
- [18]Yingqing Guo, Hui Yuan, Yukang Yang, Minshuo Chen, and Mengdi Wang. Gradient guidance for diffusion models: An optimization perspective. In Advances in Neural Information Processing Systems (NeurIPS) 37, pages 90736–90770, 2024b.arxiv.org/abs/2404.14743
- [19]Derek Hansen, Danielle C Maddix, Shima Alizadeh, Gaurav Gupta, and Michael W Mahoney. Learning physical models that can respect conservation laws. In International Conference on Machine Learning, pages 12469–12510. PMLR, 2023.DOI
- [20]Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance, 2022. URL https://arxiv.org/abs/2207.12598.
- [21]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.arxiv.org/abs/2006.11239
- [22]Jiahe Huang, Guandao Yang, Zichen Wang, and Jeong Joon Park. DiffusionPDE: Generative PDE-solving under partial observation. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024a. URL https://openreview.net/forum?id=zOI2SBjNOR.arxiv.org/abs/2406.17763
- [23]Yujia Huang, Adishree Ghatare, Yuanzhe Liu, Ziniu Hu, Qinsheng Zhang, Chandramouli S Sastry, Siddharth Gururani, Sageev Oore, and Yisong Yue. Symbolic music generation with non-differentiable rule guided diffusion, 2024b. URL https://arxiv.org/abs/2402.14285.
- [24]Thomas J. R. Hughes. The Finite Element Method: Linear Static and Dynamic Finite Element Analysis. Dover Publications, Mineola, NY, 2000. ISBN 9780486411811.
- [25]Frank P. Incropera, David P. DeWitt, Theodore L. Bergman, and Adrienne S. Lavine. Fundamentals of Heat and Mass Transfer. John Wiley & Sons, Hoboken, NJ, 7 edition, 2011. ISBN 9780470501979.
- [26]John David Jackson. Classical Electrodynamics. John Wiley & Sons, 3 edition, 1998.
- [27]Christian Jacobsen, Yilin Zhuang, and Karthik Duraisamy. Cocogen: Physically-consistent and conditioned score-based generative models for forward and inverse problems, 2024. URL https://arxiv.org/abs/2312.10527.
- [28]Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35:265655–26577, 2022.arxiv.org/abs/2206.00364
- [29]Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models, 2022. URL https://arxiv.org/abs/2201.11793.
- [30]Gavin Kerrigan, Giosue Migliorini, and Padhraic Smyth. Functional flow matching. arXiv preprint arXiv:2305.17209, 2023.arxiv.org/abs/2305.17209
- [31]Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.arxiv.org/abs/1412.6980
- [32]Diederik P Kingma, Max Welling, et al. Auto-encoding variational bayes, 2013.arxiv.org/abs/1312.6114
- [33]Randall J LeVeque. Finite difference methods for ordinary and partial differential equations: steady-state and time-dependent problems. SIAM, 2007.
- [34]Randall J LeVeque and Randall J Leveque. Numerical methods for conservation laws, volume 132. Springer, 1992.
- [35]Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895, 2020.arxiv.org/abs/2010.08895
- [36]Zongyi Li, Hongkai Zheng, Nikola Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Kamyar Azizzadenesheli, and Anima Anandkumar. Physics-informed neural operator for learning partial differential equations. ACM/JMS Journal of Data Science, 1(3):1–27, 2024.arxiv.org/abs/2111.03794
- [37]Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=PvqMRDCt9t.arxiv.org/abs/2210.02747
- [38]Xingchao Liu, Chengyue Gong, and qiang liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, 2023a. URL https://openreview.net/forum?id=XVjTT1nw5z.arxiv.org/abs/2209.03003
- [39]Xingchao Liu, Chengyue Gong, and qiang liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, 2023b. URL https://openreview.net/forum?id=XVjTT1nw5z.arxiv.org/abs/2209.03003
- [40]Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, et al. Instaflow: One step is enough for high-quality diffusion-based text-to-image generation. In The Twelfth International Conference on Learning Representations, 2023c.arxiv.org/abs/2309.06380
- [41]Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775–5787, 2022a.arxiv.org/abs/2206.00927
- [44]Eric Luhman and Troy Luhman. Knowledge distillation in iterative generative models for improved sampling speed. arXiv preprint arXiv:2101.02388, 2021.arxiv.org/abs/2101.02388
- [45]Jiafeng Mao, Xueting Wang, and Kiyoharu Aizawa. The lottery ticket hypothesis in denoising: Towards semantic-driven initialization. In European Conference on Computer Vision, pages 93–109. Springer, 2024.arxiv.org/abs/2312.08872
- [46]Dimitra Maoutsa, Sebastian Reich, and Manfred Opper. Interacting particle solutions of fokker–planck equations through gradient–log–density estimation. Entropy, 22(8):802, 2020.
- [47]Badr Moufad, Yazid Janati, Lisa Bedin, Alain Durmus, Randal Douc, Eric Moulines, and Jimmy Olsson. Variational diffusion posterior sampling with midpoint guidance, 2024. URL https://arxiv.org/abs/2410.09945.
- [48]S Chandra Mouli, Danielle C Maddix, Shima Alizadeh, Gaurav Gupta, Andrew Stuart, Michael W Mahoney, and Yuyang Wang. Using uncertainty quantification to characterize and improve out-of-domain learning for pdes. arXiv preprint arXiv:2403.10642, 2024.arxiv.org/abs/2403.10642
- [49]Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pages 8162–8171. PMLR, 2021.arxiv.org/abs/2102.09672
- [51]Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686–707, 2019.DOI
- [52]Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022.arxiv.org/abs/2112.10752
- [53]Nadim Saad, Gaurav Gupta, Shima Alizadeh, and Danielle C Maddix. Guiding continuous operator learning through physics-based boundary constraints. arXiv preprint arXiv:2212.07477, 2022.arxiv.org/abs/2212.07477
- [54]Andy Shih, Suneel Belkhale, Stefano Ermon, Dorsa Sadigh, and Nima Anari. Parallel sampling of diffusion models. Advances in Neural Information Processing Systems, 36, 2024.arxiv.org/abs/2305.16317
- [55]Dule Shu, Zijie Li, and Amir Barati Farimani. A physics-informed diffusion model for high-fidelity flow field reconstruction. Journal of Computational Physics, 478:111972, 2023.DOI
- [56]Gordon D Smith. Numerical solution of partial differential equations: finite difference methods. Oxford university press, 1985.
- [57]Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=St1giarCHLP.arxiv.org/abs/2010.02502
- [58]Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming-Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for plug-and-play controllable generation. In Andreas Krause, Emma Brunskill, Kyungyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 32483–32498. PMLR, 23–29 Jul 2023. URL https://proceedings.mlr.press/v202/song23k.html.
- [59]Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2020.
- [60]Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models, 2023b. URL https://arxiv.org/abs/2303.01469.
- [61]Haoyue Tang, Tian Xie, Aosong Feng, Hanyu Wang, Chenyang Zhang, and Yang Bai. Solving general noisy inverse problem via posterior sampling: A policy gradient viewpoint, 2024. URL https://arxiv.org/abs/2403.10585.
- [62]Joshua Tian Jin Tee, Kang Zhang, Hee Suk Yoon, Dhananjaya Nagaraja Gowda, Chanwoo Kim, and Chang D Yoo. Physics informed distillation for diffusion models. arXiv preprint arXiv:2411.08378, 2024.arxiv.org/abs/2411.08378
- [63]Stephen P. Timoshenko and James N. Goodier. Theory of Elasticity. McGraw-Hill, New York, 3 edition, 1970. ISBN 9780070647206.
- [64]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.arxiv.org/abs/1706.03762
- [65]Hao Wang, Weihua Chen, Chenming Li, Wenjian Huang, Jingkai Zhou, Fan Wang, and Jianguo Zhang. Noise re-sampling for high fidelity image generation. Submitted to ICLR 2025, 2025. URL https://openreview.net/forum?id=GD4Tlqvwrq.
- [66]Ruoyu Wang, Huayang Huang, Ye Zhu, Olga Russakovsky, and Yu Wu. The silent prompt: Initial noise as implicit guidance for goal-driven image generation. arXiv preprint arXiv:2412.05101, 2024.arxiv.org/abs/2412.05101
- [67]Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model, 2022. URL https://arxiv.org/abs/2212.00490.
- [68]Zihui Wu, Yu Sun, Yifan Chen, Bingliang Zhang, Yisong Yue, and Katherine L. Bouman. Principled probabilistic imaging using diffusion models as plug-and-play priors, 2024. URL https://arxiv.org/abs/2405.18782.
- [69]Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6613–6623, 2024.arxiv.org/abs/2311.18828
- [70]Bingliang Zhang, Wenda Chu, Julius Berner, Chenlin Meng, Anima Anandkumar, and Yang Song. Improving diffusion inverse problem solving with decoupled noise annealing. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page 20895–20905. IEEE, June 2025. doi: 10.1109/cvpr52734.2025.01946. URL http://dx.doi.org/10.1109/CVPR52734.2025.01946.
- [71]Hongkai Zheng, Weili Nie, Arash Vahdat, Kamyar Aziz A denesheli, and Anima Anandkumar. Fast sampling of diffusion models via operator learning. In International conference on machine learning, pages 42390–42402. PMLR, 2023.arxiv.org/abs/2211.13449
- [72]Hongkai Zheng, Wenda Chu, Bingliang Zhang, Zihui Wu, Austin Wang, Berthy T. Feng, Caifeng Zou, Yu Sun, Nikola Kovachki, Zachary E. Ross, Katherine L. Bouman, and Yisong Yue. Inversebench: Benchmarking plug-and-play diffusion priors for inverse problems in physical sciences, 2025. URL https://arxiv.org/abs/2503.11043.
- [73]Zhenyu Zhou, Defang Chen, Can Wang, and Chun Chen. Fast ode-based sampling for diffusion models in around 5 steps. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7777–7786, 2024a.arxiv.org/abs/2312.00094
- [74]Zikai Zhou, Shitong Shao, Lichen Bai, Zhiqiang Xu, Bo Han, and Zeke Xie. Golden noise for diffusion models: A learning framework. arXiv preprint arXiv:2411.09502, 2024b.arxiv.org/abs/2411.09502
- [75]Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, and Luc Van Gool. Denoising diffusion models for plug-and-play image restoration, 2023. URL https: //arxiv.org/abs/2305.08995.arxiv.org/abs/2305.08995