WorksPhysics and Astronomy
Only Linear Constraints Survive Coarse-Graining: Evaluating Physics-Constrained Neural Operators on Stochastically Forced Turbulence
Abstract
A physics-informed loss adds a discretised PDE residual to the data term, but is only strictly valid if the equations being enforced are closed on the fields being fitted. This fails on coarse-grained, stochastically forced data: nonlinear terms in the PDE generally do not commute with the filter, so the coarse-grained fields do not satisfy the original equations, and driving their residual to zero encodes an implicit closure into the learned operator. Under normalised, translation-invariant filtering on a periodic domain, constant-coefficient linear constraints remain valid on the filtered grid. For the 2D incompressible Navier-Stokes equations, two such constraints are continuity and global momentum balance. We enforce these exactly by applying a closed-form projection on the output of a Fourier neural operator (FNO) at cost. On two-dimensional isotropic turbulence, truncated so that its forcing lies entirely beyond the cutoff wavenumber, the projection achieves a continuity error of against 0.51 for a plain FNO and 0.17 for a physics-informed FNO, and a global momentum balance error of against and . The projection only costs an extra 2% of training time, and takes the fraction of rollouts whose energy remains bounded at 448 steps from 0.18 to 0.87. On resolved Kolmogorov flow, test-time optimisation costs approximately 3,300 as much per trajectory as the projection, while leaving maximum divergence three orders of magnitude higher. Nonlinear constraints can also be enforced once their unclosed terms are modelled: closing the global energy balance with a constant subgrid flux, fitted from the coarse training data alone, removes the 20.9% energy deficit that enforcing the unclosed balance produces and keeps every free-running rollout bounded over 2,000 steps.
Introduction
Learning an evolution operator from finitely many samples is intrinsically underdetermined: many operators can interpolate the observed data while representing fundamentally different dynamics away from the training set. Known physical laws provide information that the observations alone cannot. They constrain the admissible states and trajectories of the system independently of the particular initial conditions, sampling interval, or time horizon represented in the data, and therefore provide a principled means of constraining the model. This matters not only for extrapolation, but also because predictions are commonly used by downstream models and analyses that implicitly assume that those physical constraints are satisfied. Neural operators can incorporate this information either softly, through penalties added to the training objective, or exactly, by restricting the model architecture or its outputs to the physically admissible set. Physics-informed losses provide one widely used example of the former approach: a discretised partial differential equation (PDE) residual is added to the data-fitting objective, as in physics-informed neural networks [PINNs; 27] and their operator-learning counterpart, the physics-informed neural operator [PINO; 19]. Such a residual, however, represents a valid physical constraint only when the governing equations are closed on the fields being fitted. When this condition is satisfied, the PDE supplies additional information without requiring labelled solutions and can often be evaluated inexpensively throughout the entire domain. When it is not, enforcing the residual does not make the model more physical: it imposes an equation that the predicted fields cannot, in general, satisfy. The data-fitting and physics objectives then become incompatible, and training can only select a compromise between them [29].
An example of where physics-informed training is not valid is on coarse-grained, stochastically forced data. This is because a coarse-grained flow field, for example a macroscopic observation of a multiscale physical system, is a filtered image of a finer resolution state. Nonlinear terms in the equations governing that state generally do not commute with the filter, so the coarse-grained fields do not satisfy the original equations. This is the classical closure problem of large-eddy simulation [LES, 26, 30]. Recent work on this problem in the machine learning literature either learns the unresolved term from direct numerical simulation [10], inserts a classical subgrid model into a physics-informed loss [36], or evaluates the residual on a fine grid [33]. These all either rely on data not available from the coarse field alone, or inherit the shortcomings of classical LES models. A PDE-residual loss evaluated solely on the coarse field makes the same approximation as LES, but left implicit. Attempting to enforce the original equations as constraints by driving their residual to zero encodes an unintended closure into the learned operator, selected by the optimiser rather than the modeller.
Linear constraints that commute with the filter survive coarse-graining exactly. On a periodic domain, a normalised, translation-invariant filter commutes with constant-coefficient differential operators and preserves domain integrals; linearity alone does not guarantee this for arbitrary operators or boundary conditions. The criterion is therefore commutation with the filter rather than linearity as such; linearity is what then lets a surviving constraint be enforced exactly by projection. In this paper we use the 2D incompressible Navier–Stokes equations as a motivating example, for which two linear constraints that remain valid are continuity and global momentum balance obtained by integrating the momentum equation over the domain. By contrast, the pointwise momentum equation and global energy balance are both nonlinear, therefore neither remains valid to enforce as a constraint after filtering without some form of learned/modelled closure. Here we do not attempt to close the unresolved terms in the pointwise momentum equation. Instead we show, on stochastically forced two-dimensional turbulence, which constraints remain valid after filtering and what happens when a neural operator is trained as if they all were. The motivating setting is neural operators that see only coarse-grained observations (for example, geophysical fields in weather/climate prediction); the experiments here are a controlled proxy for that case.
In addition to remaining valid after filtering, another advantage of linear constraints is that they can often be imposed exactly via the solution of a small, equality-constrained, least-squares problem [6, 23]. Furthermore, on a periodic grid that closed form solution is diagonal in Fourier space, so it only requires one forward and one inverse transform; no linear system needs to be solved. We therefore enforce continuity and global momentum balance cheaply and exactly by applying this projection to the output of an FNO at cost. The projection has no parameters, is differentiable, and composes with the model in end to end training. Unlike a residual penalty, it requires no coefficient to weight against the data term in the loss. In situations where it is valid to do so, we can also impose global energy balance, a single quadratic equation, using a Gauss–Newton projection applied after the linear step [32]. The update is a rank-one correction whose direction is divergence-free wherever the field is and has no domain mean, so continuity and the momentum balance remain enforced. On coarse data the energy balance is not closed, so imposing it exactly requires a model for the unresolved energy flux across the cutoff. We test the simplest such model: a constant, fitted from the coarse training data alone, which shifts the target of the constraint by one scalar and turns a mis-specified constraint into a valid one.
We consider two test cases to both demonstrate the effects of enforcing unclosed constraints, as well as the effectiveness of exactly enforcing closed ones. The first is the (resolved) Kolmogorov flow benchmark from Li et al. [19], where the equations are closed up to discretisation error. The second is two-dimensional isotropic turbulence [3], simulated using a pseudo-spectral solver at and truncated to by a sharp spectral cutoff placed below the forcing band, so that the forcing lies entirely beyond the cutoff wavenumber. On this unresolved flow the forcing is invisible to the learner as it is entirely subgrid, so the energy dissipated in the resolved band is supplied entirely by nonlinear transfer from unresolved scales. The map from one coarse snapshot to the next is therefore driven by scales that are not in the input and are not deterministic functions of it: many fine states are compatible with the same coarse field, and they evolve differently. A deterministic operator trained under squared error can still learn the conditional mean of this stochastic transition. The constraints that remain valid for that mean are again the linear ones, because the expectation is linear as well.
Contributions.
A filtering criterion that separates the constraints which remain valid after coarse-graining from those which do not: a residual loss on filtered data is an implicit closure model.
A spectral projection that enforces continuity and global momentum balance jointly on an FNO, in closed form, at cost, together with a Gauss–Newton step for the global energy balance that leaves those two constraints intact.
A comparison of that projection against the PINO residual, with and without test-time optimisation, on a resolved control and an unresolved case, to demonstrate the effects of mis-specified physical constraints.
A proof of concept that a nonlinear global constraint can be made valid on coarse data by closing it: a constant subgrid energy flux, one scalar fitted from the coarse training pairs alone, removes the energy bias that enforcing the unclosed balance produces.
Relation to prior work. Closed-form projection onto linear equality constraints as a non-trainable neural network output layer was introduced in KKT-hPINN [6] and HardNet [23], is treated in generality by Grontas et al. [9], and appears in iterative or implicit form in DC3 [7] and in differentiable-optimisation layers [25, 5]. A spectral divergence-free projection appended to an FNO and trained end to end on two-dimensional Navier–Stokes has been published in [8, 35, 16]; what we add to that layer is the global momentum balance constraint in the same step. Quadratic integral constraints have been projected in PINNs [1] and imposed on FNOs by a learnable correction [20]; the energy step here is parameter-free, applied after the spectral projection, and constructed so that it does not undo continuity or the momentum balance. Hard constraints by reparameterisation are an alternative route to the same exactness [28, 2, 24, 21], and projections onto global conservation laws are established for PINNs and operators alike [11, 1, 20, 32]. The closest quantitative comparison of hard vs. soft constraints is Baez et al. [1], who report three to four orders of magnitude lower conservation error than a soft penalty. Harder et al. [12] enforce a linear coarse-to-fine mass balance in climate downscaling: a related constraint that also survives coarse-graining, but not the dynamical constraints considered here. None of this work uses a filtering argument to decide which constraints to enforce. The comparisons below use one architecture, one optimiser and a common schedule across methods within each flow, based on the training and evaluation pipeline of Wang et al. [34], so that differences are not artefacts of weak or unmatched baselines [22].
Results
The filtering criterion, the projection, and the description of two flows are given in Appendix § 6–Appendix § 8. For each experiment the eight methods share one architecture, one optimiser and one schedule, with five seeds each (Section C). These are a plain FNO, an FNO with the projection applied (continuity, continuity with the momentum balance, and both with the energy balance), a PINO with the full continuity + momentum residual or continuity only; and each PINO method with additional test-time optimisation (TTO) during the rollout. § 2.5 adds one further model on E2: the energy-balance projection with a fitted closure, trained under the same protocol. For both experiments we compare accuracy, constraint satisfaction, stability over long rollout horizons and computational cost.
Accuracy
Figure 1 shows the E1 rollout errors; Table 1 gives their final values. On E1, the Kolmogorov flow where the momentum equation closes, projecting the two linear constraints gives a small and consistent gain at short lead times: lower one-step error than the plain FNO, better on five seeds of five, and lower at step 16. By the final step of the 64-step rollout the gain has gone: the projected model predictions are 6.9 16.0% worse than the FNO, although that difference is well inside the seed spread. Adding the residual loss on the same flow is a different story. PINO is 45.8% worse than the FNO at step 16 and 61.9 37.3% worse at step 64, and PINO (cont only) is similar at 58.6%. On a resolved flow, where adding the PDE residual would be expected to help, it does not (at least when only applied during the training stage). Meanwhile, the exact satisfaction of the constraints that the projection provides is, at worst, free in terms of accuracy. Further optimisation of the full residual at test time (PINO+TTO) results in 10.1 12.6% lower error relative to the plain FNO at step 1, a gain inside the seed spread and no better than the projection’s, and 7.6% worse at step 64. Optimising continuity alone gives a final relative error of 0.425, 26 worse than the FNO, for the reason given in Section E.4.
Figure 1. E1 (the resolved case): relative error, the step-64 energy spectrum, maximum divergence and domain-mean velocity. Curves and shading show the median and interquartile range over five seeds, each averaged over 200 test rollouts (20 for TTO).
Table 1: Relative error at the final step of the 64-step rollout against the ground truth, constraint satisfaction over the same rollout, and training and inference costs, for both flows. Entries are means over five seeds with the seed standard deviation. is taken at the worst step of the rollout and at the final step. Cost columns are relative to the FNO for the same flow. Daggers mark E2 step-64 errors that are lower because the model has lost energy, not because it tracks the flow (§ 2.1).
| Model | E1 (resolved) step 64 | E1 (resolved) | E1 (resolved) | E1 (resolved) train | E1 (resolved) infer. | E2 (unresolved) step 64 | E2 (unresolved) | E2 (unresolved) | E2 (unresolved) train | E2 (unresolved) infer. |
| FNO | ||||||||||
| FNO+proj (cont) | ||||||||||
| FNO+proj (+mom) | ||||||||||
| FNO+proj (+energy) | ||||||||||
| PINO (cont only) | ||||||||||
| PINO (cont only) + TTO | ||||||||||
| PINO | ||||||||||
| PINO + TTO |
Table 1.
Figure 2 and Table 1 give the corresponding E2 results. On E2, the stochastically forced turbulent flow where the momentum equation does not close, the accuracy comparison is unchanged at short lead times. At step 1, the relative errors of the FNO, the three projected models and both PINOs span 0.0569 to 0.0574, and compared seed by seed none differs from the FNO by more than 0.5%. By step 64 (25.6 time units) every method has a relative error above 1.0, worse than predicting zero everywhere. Step 64 is eight e-folding times of the coarse field (3.2 time units), so the flow has “forgotten” its initial condition. The remaining differences, −1.9% for the projection, −3.9% for PINO and −9.0% when the energy balance is also imposed, therefore measure remaining energy, not agreement with the ground truth: the lowest error amongst the PINOs (without TTO) and the projected FNOs is the energy balance projection, which removes the most energy (§ 2.5). Further optimisation of the full residual at test time (PINO+TTO) gives a one-step error of 0.093 against 0.057 for PINO, and 0.109 when only continuity is optimised; at step 64 those errors are 4.5% and 4.1% lower, again a difference in energy. Both TTO methods lose energy from the first steps: in the free-running rollouts of § 2.3, the continuity-only TTO trajectories fall below a tenth of their initial energy at a median step of 22, so these step-64 errors describe a drained field.
Figure 2. As in Figure 1 but for E2 (the unresolved case), averaged over each seed’s 42 test rollouts.
Constraint satisfaction
The bottom two panels of Figure 1 and Figure 2 show constraint satisfaction over the rollout, and , which are also listed in Table 1. On E2, over the 64-step rollout, the projected models maintain at against the ground truth’s , contrasted with 0.51 for the FNO, 0.17 for PINO and 0.30 for PINO (cont only). Adding the momentum balance maintains the domain-mean velocity at the final rollout step at against the ground truth’s , whereas the FNO only manages , PINO and PINO (cont only) . The projections therefore give six orders of magnitude better constraint satisfaction on continuity and seven on the momentum balance. On E1, the projection likewise reaches the precision of the stored data: is against the ground truth’s for the projection vs. 0.23, 0.22 and 0.096 for the FNO, PINO and PINO (cont only), and is against the ground truth’s for the projection vs. , and for the FNO, PINO and PINO (cont only). Further optimisation of the full residual at test time (PINO+TTO) reduces to on E1 and on E2, below PINO but still two and a half
to four orders of magnitude above the projection. Optimising continuity alone gives and on the same diagnostic, and increases the domain-mean velocity by on E1 and on E2 (see Section E.4).
Long-term stability
The second panel of Figure 1 and Figure 2 shows the kinetic energy spectrum at the final step of the 64-step rollout. On E1, the ground truth contains only of its total energy above radial wavenumber 16, and at step 64 of the rollout the FNO has in that band; the projected FNO models and both PINOs follow the same spectrum. Further optimisation of the full residual at test time (PINO+TTO) also follows that spectrum, whereas optimising continuity alone leaves only of energy above mode 16. On E2, the ground truth now contains of its total energy in that same band. E2 uses 32 modes per axis to extend the spectral convolution to smaller scales; this rectangular mode selection differs from the radial band used for the diagnostic. At step 64 of the rollout the FNO, the three projected models and both PINOs have to above mode 16, whereas PINO+TTO has only of its energy above mode 16, and optimising continuity alone has . The momentum equation does not close on how, therefore driving its residual to zero at test time removes energy from that wavenumber band rather than matching the truth.
To test long-term stability, free-running rollouts were conducted on both E1 and E2 out to 2,000 steps (i.e. beyond the horizon of the ground truth, which is available to 64 steps on E1 and 448 steps on E2). Figure 6 reports the fraction of trajectories that remain bounded: defined as their kinetic energy staying within a factor of ten of its initial value, above or below (Section C). On E1, every method is still at 1.00 (i.e. 0% of trajectories have left the band) at step 64. At step 448 the FNO and the three projected models remain at 1.00, PINO at 0.91 and PINO (cont only) at 0.96. At 2,000 steps the projected models remain at 0.99 to 1.00 and the FNO at 0.96, while PINO has fallen to 0.42 and PINO (cont only) to 0.64. Further optimisation of the full residual at test time (PINO+TTO) does not prevent that fall (0.52), and nor does optimising continuity alone (0.38). On this flow, minimising the PDE residual costs stability at long rollout horizons, whereas the projection does not. On E2, the bounded fraction is also 1.00 for every method at step 64 except the two TTO methods, which lose energy rather than gain it: PINO+TTO is at 0.99 and PINO (cont only)+TTO at 0.12. By step 448 it is 0.18 for the plain FNO, 0.71 with continuity projected, 0.87 with the momentum balance, 1.00 with the energy balance, 0.89 for PINO and 0.40 for PINO (cont only). PINO+TTO has fallen to 0.38 and PINO (cont only)+TTO remains at 0.12; neither method blows up, but both drain substantial amounts of energy from the flow and leave the band from below. At 2,000 steps the FNO, the continuity-only projection and both PINOs are at 0.00, the momentum-balance projection at (one seed out of five has 0.62 whereas the other four are none), the energy-balance projection at 0.00, PINO+TTO at 0.20 and PINO (cont only)+TTO at 0.12. Again, the last three leave the band from below rather than above: they lose energy rather than gain it, with the energy balance projection losing it most slowly (its trajectories are the last to fall below , from step 448 onwards). The closed-energy model of § 2.5 is also shown, and remains at 1.00 throughout.
Figure 6. Free-running stability over 2,000 steps, five seeds of 40 rollouts on E1 (20 for the TTO methods) and 42 on E2. Top: fraction that have stayed within [0.1, 10] times their initial energy. Bottom: median energy ratio within each seed. Curves and shading show the median and interquartile range across seeds; dotted lines mark the energy bounds.
Computational cost
The last two columns of Table 1 show training and inference costs for each method relative to the plain FNO. On E1, projecting the two linear constraints costs an additional 4% of training time, the residual loss an additional 10%, and the energy projection 49%. On E2 the same overheads are 2%, 5% and 23%. At inference the projection costs and the FNO forward pass on E1 for the continuity projection or continuity plus momentum balance, and if the energy balance is included as well. On E2 the same costs are , and . PINO inference is a negligible additional cost, whereas further optimisation of the full residual at test time (PINO+TTO) is the FNO on E1 and on E2; TTO of continuity alone is and .
A simple energy balance closure
The energy balance does not close on E2 because the resolved band receives energy via nonlinear transfer across the cutoff (Appendix § 8). The unclosed energy-balance projection ends the rollout with energy below the ground truth. The truth’s measured mean residual of per step gives an estimated 22.7% deficit over 64 steps relative to its initial energy of . Restoring that balance requires a closure energy flux, and we test the simplest one: a constant energy source , fitted as the mean unclosed residual of the coarse training pairs, so that no fine-grid data is needed (Appendix § 9). On E2 the fit gives per unit time, and a radial shell-flux measurement at agrees with it within 7%.
With in the projector and nothing else changed, the bias disappears: the energy error at step 64 goes from to , and over 2,000 free-running steps the closed model is the only one on E2 whose median energy remains near its initial value (0.71 at step 2,000), with all 210 trajectories inside their initial energy (Figure 6). Accuracy and satisfaction of the two linear constraints are unchanged. The closure does not correct how that energy is distributed over wavenumbers: the fraction above wavenumber 16 still grows from the truth’s 0.18 to 0.50 by step 2,000 (Figure 7 in Appendix § 9). A nonlinear global constraint can therefore be enforced on coarse data once its unclosed term is modelled, and a single scalar is enough to restore the energy budget. However, matching the distribution of energy across scales requires a closure with spectral structure.
Figure 7. E2 energy spectra at the final step of the 64-step rollout (left) and the 448-step windows with ground truth (right), for the three projected models and the model with the closed energy balance. Curves on the left show the median over five seeds, each averaged over its test trajectories; those on the right show the median over the thirty seed–window rollouts. The dotted reference is the ground-truth spectrum at step 64 on the left and its stationary mean over the 2,688 test snapshots on the right.
Discussion
The introduction framed these experiments as a controlled proxy for neural operators that see only coarse-grained observations, for example geophysical fields in weather and climate prediction. E1 is the case where a residual loss is valid: the Kolmogorov flow is closed up to discretisation error. E2 is the case where it is not, since the forcing lies entirely beyond the cutoff and the energy in the resolved band arrives by nonlinear transfer from scales the learner never sees. On the ground truth data the pointwise momentum residual is 0.094 on E1 and 0.648 on E2 (Table 2). The comparisons in § 2 follow that split, and they add two things the filtering argument alone does not provide: the form the implicit closure takes in the learned operator, and the behaviour of penalised/constrained models at horizons where the flow has decorrelated from its initial condition.
| E1 (resolved) | E2 (unresolved) | ||
| linear | continuity, max | ||
| momentum balance, | |||
| nonlinear | pointwise momentum residual | 0.094 | 0.648 |
| global energy balance | 0.256 | 0.935 |
Table 2. Ground-truth constraint diagnostics. The linear rows are absolute errors; the nonlinear rows are relative residuals, with normalisations defined below.
On E2, driving the unclosed residual towards zero does not recover the missing subgrid flux. It removes energy from the wavenumber band the residual cannot account for, and the unclosed energy balance projection does the same just more slowly. This is the optimiser-selected closure described in the introduction: the data term and the physics term cannot be satisfied together, so training selects a compromise, and the compromise is a lower-energy field. Optimising the residual harder does not change the outcome. Test-time optimisation reduces the residual by one to two orders of magnitude and never blows up, but only because it drains the field: at step 64 it holds 0.20% of its energy above wavenumber 16 against the ground truth’s 18.3%.
On E1 the equations close, so the residual is a valid constraint, yet enforcing it as a penalty still does not help (§ 2.1, § 2.3). A valid residual is still a penalty with a coefficient that must be balanced against the data term; it is not equivalent to restricting the model to the admissible set.
The constraint columns of Table 1 measure what motivated enforcing constraints in the first place: downstream models and analyses assume that the constraints are satisfied. On both flows the projection maintains continuity and the domain-mean velocity constraints at the precision of the stored data — several orders of magnitude below the FNO and both PINOs — for an additional 2–4% of training time and 4–7% of inference time. This is a guarantee that a penalty cannot provide: it holds at every training step and at every rollout step, not on average over a training distribution. A divergence-free field can be passed to a tracer solver or inverted for a streamfunction without spurious sources; an exactly conserved domain mean cannot accumulate a drift over a long integration.
The closure result of § 2.5 qualifies the filtering criterion: a nonlinear global constraint is not invalid in itself, it is invalid until its unclosed term is modelled. One fitted scalar keeps the energy bounded under the modelled balance over 2,000 steps. The fraction of energy above wavenumber 16 still grows from the ground truth’s 0.18 to 0.50, so the model drifts away from the attractor of the flow despite remaining within the energy band. Keeping a learned operator on the attractor is the problem the Markov neural operator addresses [18], where a one-step operator is trained with a Sobolev-norm loss, which weights prediction errors across scales, together with a dissipativity regularisation that trains the operator to dissipate energy on states far away from the attractor. Both parts compose directly with the projection layer used here: the projection would enforce the linear constraints and the modelled energy balance, while the Sobolev-norm term penalises errors in scale-by-scale transfer that a single scalar cannot represent. The spectral drift also has an architectural component. The FNO concentrates its error near the spectral cutoff, and an architecture with less spectral bias, such as the multiscale wavelet transformer of Wang et al. [34], is a candidate backbone for the same projection. We leave both combinations to future work.
Limitations. The evidence is two flows, both two-dimensional, both on a periodic domain, and one architecture. The filter is a sharp spectral cutoff, which is the cleanest case for the argument: a smooth filter leaves a subgrid forcing that is small rather than exactly zero, and these experiments do not locate the boundary of that regime. The stability criterion is a band on the kinetic energy, which detects a blow-up or a drain but not a rollout that stays bounded while ceasing to resemble the flow; the closed-energy model is exactly such a case, and only the spectral diagnostics expose it. The linear constraint right-hand sides here are zero, so the momentum balance constraint reduces to zeroing one Fourier mode. A non-zero, time-dependent target would exercise more of (8). The constant-flux closure assumes a statistically stationary flow: it cannot represent state-dependent variation in the subgrid flux, and its accuracy here rests on E2 being stationary.
Conclusion
Linear constraints that commute with the filter survive coarse-graining; nonlinear terms generally require closure. A neural operator trained on coarse-grained data can therefore enforce the linear constraints exactly. On both experiments, we apply a closed-form spectral projection that enforces continuity and the global momentum balance at the precision of the stored data for an additional 2–4% of training time. Residual penalties added to the loss, either at training time or at test time, provide no consistent accuracy benefit and remove energy from the flow. A nonlinear constraint is not invalid in itself, it is invalid until its unclosed term is modelled. A constant subgrid flux fitted from the coarse training data restores the global energy balance, allowing it to be enforced alongside the linear constraints. The closed model maintains bounded energy under the modelled balance while still drifting from the attractor, as the constraint does not preserve the distribution of energy across scales.
Acknowledgments and Disclosure of Funding
The authors wish to acknowledge Xuesong Wang for providing the initial training and evaluation pipeline for the FNO models used in this paper. We thank the anonymous reviewers and the area chair for their constructive comments.
The filtering criterion
For a state on , write the PDE problem governing the evolution of that state as
with being the evolution operator, a set of constraints holding at every interior point, a boundary operator, and functionals of the whole field. Now, coarse-grain with a filter of width ,
with . We use a fixed, translation-invariant filter on a periodic domain, so it commutes with differentiation and preserves domain integrals. Split the state into the resolved part and the residual part .
Linear constraints remain valid. For a linear constraint operator that commutes with (2), . This holds for constant-coefficient differential operators such as divergence; normalisation also preserves unweighted domain integrals. Linearity alone does not establish commutation: variable coefficients and boundary conditions require separate compatibility with the filter. Under these assumptions, the constraint remains exact regardless of , the shape of or how much energy the filter removes.
The evolution equation does not. Split into its linear and nonlinear parts. Under the assumptions above, the linear part commutes with the filter. The nonlinear part generally leaves behind , which depends on unresolved products and cannot in general be evaluated from the coarse field alone, so representing it requires a closure model. A sharp spectral cutoff also removes forcing supported entirely beyond it: the filtered equation then has no explicit forcing term, while the flow it describes is forcing-driven.
A residual loss on filtered data is a closure model. Write the filtered evolution equation as , where is the unclosed residual. A physics-informed loss penalises , and driving that penalty to zero asserts , i.e. it is a closure model that sets the subgrid term to zero. Explicit learned closures [10, 36, 33, 14, 31] are approximations too, but they can be inspected, tested and reported. This asymmetry motivates the paper: a commuting linear constraint is exact on filtered data at no modelling cost, whereas a nonlinear residual on filtered data is a modelling choice, regardless of whether it is made deliberately.
Example: the Navier–Stokes equations. Consider the incompressible Navier–Stokes equations on a periodic domain, with viscosity and a linear drag coefficient ,
Filtering the continuity equation returns , because divergence commutes with this filter. However, the filtered momentum equation does not close: filtering the convective term produces the subgrid stress , containing contributions from the unresolved field, and the filtered equation becomes
A residual loss written on the coarse fields is (4) with the right-hand side truncated to , so minimising it asserts .
Integrating (4) over the periodic domain removes every divergence, included, as per the divergence theorem, along with the pressure gradient and the viscous term. What is left is the global momentum balance,
every term of which is linear in . We enforce (5) and continuity as exact constraints via the projection described in Appendix § 7.
Conversely, contracting (4) with and integrating gives the global energy balance,
The convective and pressure terms vanish after integration by parts because . The last term is the only one that does not close,
which is the net rate at which unresolved scales do work on the resolved field. For a sharp radial cutoff it equals of (10). E2 uses a rectangular cutoff, so the radial shell flux is a diagnostic comparison rather than the same integral. On E2, omitting this positive energy supply from (6) forces the resolved energy to decay. We enforce this balance in two forms: unclosed, as a probe of what a miss-specified constraint does to a model, and closed by the constant-flux model of Appendix D described in Section 2.5.
Exact linear constraints by projection
Once the admissible set is linear, the projection onto it has a closed form. Discretise the linear constraints from Appendix A as , with the state on the grid and carrying one row per constraint, being the total degrees of freedom on the grid and the number of discrete linear equalities. We seek the smallest correction to an unconstrained prediction that satisfies them, measured in the norm of a symmetric positive-definite mass matrix holding the grid weights (i.e. the discrete inner product whose weights turn a sum over grid values into an integral over the domain). The Karush–Kuhn–Tucker conditions of subject to are linear. They are necessary and sufficient, because the problem is a strictly convex quadratic with affine constraints. The Lagrangian is . Stationarity gives ; substituting into and eliminating returns
The Schur complement is invertible exactly when has full row rank. For the continuity rows is zero. For the momentum rows it is (5) integrated across one step, so that is the domain-mean momentum the coarse field is required to have at the end of the step; on both flows used in this paper the forcing has no domain mean and the initial mean is zero, so those rows set the mean velocity to zero.
The map has no parameters and is differentiable, so trains end to end by ordinary backpropagation [6, 8], and appending it costs no expressivity (any admissible field is a fixed point of [23]). A single contains every row, so pointwise and global constraints are imposed jointly rather than in sequence, imposing them one after the other would in general undo the first with the second. There is no coefficient anywhere in (8), which is the practical difference from a penalty: the constraint is satisfied at every step of training, at every step of the rollout, and at every point of the parameter space, including the initialisation.
Periodic grids. In this case is a positive multiple of the identity and therefore cancels from the projection formula, and both families of rows are diagonal in the Fourier transform. A divergence row acts as and couples only the components at one wavevector, so (8) reduces mode by mode to the Leray projection commonly used in pseudo-spectral methods. A domain-mean row is and touches only the zero wavevector, which the Leray projection rows leave untouched. The two families therefore act on disjoint modes, the joint solve is one pass over the spectrum, and the whole projection costs a forward and an inverse transform, in the number of grid points, with no linear system to assemble. The projection layer and its spectral form appear in prior work: (8) is the hard-constraint output layer of Chen et al. [6] and Min and Azizian [23], treated in generality by Grontas et al. [9], and its Leray form appended to an FNO and trained end to end on two-dimensional Navier–Stokes appears in Duruisseaux et al. [8], Xu et al. [35], Li et al. [16]. What this paper adds is the zero-wavevector rows in the same .
The energy balance. The global energy balance is a single scalar equation, quadratic in , so it has no row in . We impose it after (8) on the discrete trapezoidal balance between consecutive steps,
with , and , by at most 20 minimum-norm steps in Fourier space, with residual tolerance . The step direction is , a per-mode multiple of the field plus the forcing, so it is divergence-free and mean-free; the -scaled field and the forcing are and vanishes at whenever the domain means do. Unclosed, the step is an instrument for measuring what enforcing a non-closing constraint does to a model.
Equation (8) is the finite-dimensional case of a constrained reproducing-kernel construction [13, 15]; non-periodic geometries and non-uniform meshes are future work.
Experiments
The filtering criterion of Appendix A is a statement about the data, so it can be checked before any model exists. We use it to construct two flows, one resolved and one unresolved, both observed on the same grid.
E1, the resolved control. This is the Kolmogorov-flow benchmark of Li et al. [19]: Navier–Stokes at on the torus , forced by , with 4,000 trajectories of 65 snapshots. It is published in vorticity form; we recover velocity and pressure by spectral inversion of the Poisson equations. The reconstructed velocity is divergence-free by construction, but the unconstrained model’s velocity predictions need not satisfy this or the global momentum balance. The snapshot spacing is . The data are used at their native resolution. The forcing, the energy-containing scales and the dissipation all sit inside the grid, and the equation closes up to discretisation error. This makes E1 the case where a residual loss is valid to enforce.
E2, the unresolved case. We apply the Dedalus pseudo-spectral solver [4] to stochastically forced two-dimensional turbulence [3] on the periodic domain . Five realisations are integrated to at with and linear drag , under an Ornstein–Uhlenbeck forcing confined to with and correlation time , at a solver step of . A spin-up phase is conducted up to ; data prior to this is discarded. Fields are written at every 0.08 time units by a sharp spectral cutoff applied at , placed below the forcing band. We retain every fifth snapshot, giving a model step of . On the coarse field the forcing RMS is of the velocity RMS. The coarse dynamics in the resolved band are therefore driven entirely by nonlinear transfer from scales below the grid. E2 is the case where a residual loss is not valid to enforce without an explicit closure.
The two flows therefore differ in how the resolved band is supplied with energy. Write for the kinetic energy in the shell of wavenumber magnitude , for the net nonlinear transfer out of that shell, and for the energy the forcing injects. The shell budget is
Summing over for a statistically stationary flow with no forcing inside the band gives , with the cumulative nonlinear transfer into the band and the dissipation inside it.
Figure 3 plots both quantities. On E2 they coincide throughout the resolved wavenumber band: measured on 1,100 stationary snapshots of one realisation, is 1.002 at and 1.003 at . The resolved range is a net receiver of energy from beyond the cutoff, and there is no resolved forcing to replace what it dissipates. On E1 they do not coincide. The forcing sits at , and the ratio is at and at .
Figure 3. The energy-budget diagnostic of (10). Top: cumulative nonlinear transfer into the band at wavenumber , , as well as cumulative dissipation, . Bottom: nonlinear transfer . Forcing is marked ( on E1; on E2) by the green shading.
Table 2 reports the residuals of each constraint on the ground truth. On both flows the two linear constraints hold to numerical round-off. On E1 the pointwise momentum equation very nearly holds as well. On E2, both this residual and the global energy-balance residual are after normalisation.
The pointwise momentum residual is normalised by the root-mean-square one-step velocity increment. The relative energy residual is the mean over pairs of
with , , and defined in (9).
Figure 4 and Figure 5 show the vorticity field on one test trajectory of each flow.
Figure 4. Vorticity at the final step of a 64-step rollout on E1.
Figure 5. Vorticity at step 64 of the standard rollout (top) and step 448 of the extended rollout (bottom) on E2.
Architecture and training. Every method on both flows uses the same Fourier neural operator architecture: four spectral layers of width 64, GELU activations, a 128-unit channel-mixing head, and increment prediction , with the projection applied to the predicted state rather than the increment . The input is seven channels (, , and four periodic grid features) and the output is three (, , ). The data loss is relative , i.e. over the flattened () state. E1 retains 16 modes per axis and E2 retains 32. The Adam optimiser is used at a base learning rate of , weight decay , gradient-norm clipping at 1.0 and batch size 100. E1 trains for 280 epochs, with the learning rate halved at epochs 112, 196 and 252; E2 trains for 100 epochs, with halvings at 40, 70 and 90. Each configuration uses seeds 1–5 for initialisation, minibatch shuffling and input noise. Training uses one-step pairs, without an additional initial-condition loss.
The data term has unit weight. For full PINO, continuity has weight 0.0767 and each momentum component has weight 0.0051 on both flows. These weights were selected by one-step validation error in one-seed sweeps with factor-of-three spacing, over 50 epochs on E1 and the full 100-epoch schedule on E2. Continuity-only PINO uses 0.12 on E1, selected for rollout stability, and 0.026 on E2, selected by one-step validation error. These weights were chosen under the initial 16-mode protocol and retained for the final experiments.
Protocol. Eight methods are compared on both flows, all sharing the one architecture, optimiser and schedule described above: a plain FNO [17]; three adding the projection of Appendix § 7 — continuity alone, continuity with the momentum balance, and both with the energy balance; a PINO with the continuity residual and both momentum components; a PINO with the continuity residual only; and each of the two PINO methods again with test-time optimisation (TTO), in which the trained weights are further optimised on each test trajectory against its own PDE residual with no data term (Appendix § 10). We name the methods FNO, FNO + proj (cont), FNO + proj (+ mom), FNO + proj (+ energy), PINO, PINO (cont only), PINO + TTO and PINO (cont only) + TTO.
Numerical summaries use seed means and sample standard deviations. Accuracy differences are paired by seed and expressed as percentages of the FNO’s mean error across seeds. Energy error is .
The standard evaluation rollout is 64 steps on both flows. E1’s released trajectories are 65 snapshots long, so this covers the entire available trajectory. E2’s 2,688 test pairs give 42 non-overlapping 64-step windows; a separate extended evaluation uses six 448-step windows. Free-running evaluation uses 40 initial conditions per seed on E1 (20 for TTO) and 42 on E2, each for 2,000 steps. Across five seeds, this gives 200 trajectories per method on E1 (100 for TTO) and 210 on E2. Figure 6 records those free-running trajectories. Every method on E2 stays below the bound for the first 128 free-running steps, so no standard 64-step evaluation is cut short by a blow-up.
Separate train, validation and test splits are employed. Model training uses the training split only. On E1, checkpoint selection is restricted to epochs at the full input-noise amplitude, which under the ramp below is the final epoch. On E2, validation is evaluated every epoch and the checkpoint with the lowest one-step data loss is selected. The test split is used to report final results and is not used for checkpoint selection. On E1 the split is 3,600/200/200 trajectories. On E2 it is by realisation: train uses three seeds, validation another, and the final seed is used for testing, so that a model is tested on a realisation it has never seen. The split seeds and data preparation are specified in Appendix § 11.
Rollout stabilisers. The two flows make use of two different methods for stabilising the autoregressive rollout. E1 trains with Gaussian input noise at of the state, held at zero for the first fifth of the schedule and then ramped linearly to full amplitude. E2 trains with an additional vorticity data term, , measured as a relative against the truth. This helps stability by penalising errors at higher wavenumbers more strongly, since taking the curl multiplies each mode by its wavenumber. Its weight is 1.74 for every E2 method, selected in a stabilisation screen and equal to three times the weight that matches the vorticity and data terms at initialisation.
Closing the energy balance with a constant flux
The closure
The global energy balance does not close on E2 because it omits the nonlinear energy flux across the cutoff, of (7). We approximate this flux by a constant, , giving the closed discrete balance
Here is the unclosed balance in (9). The constant shifts the target of the constraint without changing its gradient, so we can use the Gauss–Newton update of Appendix § 7. The update continues to preserve continuity and global momentum balance. The only difference from the unclosed projection is the scalar .
We fit from the coarse training data alone. Evaluated on a pair of ground-truth snapshots, the residual of the unclosed balance equals the time integral of over that step, up to the trapezoidal2 rule error. This follows from the exact filtered energy balance, (6), which includes the subgrid flux. We therefore estimate as the mean of over the 6,885 E2 training pairs. The fitted value is per unit time, or per step. The standard error of this per-step source is , computed as the residual standard deviation divided by , without adjustment for temporal correlation.
Two checks support this estimate. The radial shell flux at is per unit time, within 7% of the fitted value, despite the difference between the radial diagnostic and the rectangular data filter. On the held-out validation split, the mean closed residual is per step, or 0.2% of the ground truth’s mean absolute one-step energy change of . Applying the same fit to E1 gives a per-step source whose magnitude is 3.0% of its mean absolute one-step energy change, compared with 1,367% on E2. This is consistent with E1 being resolved, so we do not train a separate model with an energy closure on that flow.
Results
We add to the FNO + proj (+ energy) configuration and train it end to end with five seeds under the same protocol as the other methods (Section C). Its accuracy and satisfaction of the two linear constraints are comparable to those of the other projected models: relative error is 0.0571 at step 1, max and . The closure substantially reduces the energy deficit. At step 64, the closed model’s energy error relative to the ground truth is , compared with a deficit of for the unclosed projection. At step 448, the respective errors are and .
Over 2,000 free-running steps (210 trajectories), the closed model’s median energy is 0.92 of its initial value at step 448 and 0.71 at step 2,000. Every trajectory ends within [0.1, 10] times its initial energy, making it the only E2 model for which all trajectories satisfy this criterion. The unclosed projection’s median energy is of its initial value at step 2,000. For the momentum-balance projection, the median over the 210 trajectories exceeds the upper limit of the [0.1, 10] band at step 975; for the continuity-only projection, it exceeds that limit at step 584.
Distribution of energy across scales
Although the closure restores the global energy budget, the predicted distribution of energy across scales continues to depart from the ground truth (Figure 7). The fraction of energy above wavenumber 16 is 0.20 at step 64, 0.28 at step 448 and 0.50 at step 2,000, compared with the ground truth’s 0.18. By step 256, the predicted vorticity range is 3.8× that of the ground truth. The constraint fixes the total energy budget but does not constrain how energy is distributed over wavenumbers. Correcting the transfer between scales requires a closure with spectral structure, which is beyond the constant-flux approximation considered here.
Test-time optimisation
Test-time optimisation (TTO) further minimises the PDE residual on each test trajectory by adapting the trained operator’s parameters. We compare its constraint satisfaction and computational cost with those of the projection. Unlike a penalty applied only during training, TTO requires a separate optimisation for every trajectory at inference.
Protocol
Starting from the trained PINO checkpoint, we optimise all parameters of the operator separately for each test trajectory, using the PDE residual evaluated over the adaptation horizon. The residual terms retain the weights used to train the checkpoint: continuity at 0.0767 and each momentum component at 0.0051 for the full PINO on both flows, and continuity at 0.12 on E1 and 0.026 on E2 for the continuity-only methods. No data term is used, since adaptation does not have access to the ground-truth trajectory.
Optimiser Adam with a learning rate of , halved at iterations 100 and 160, gradient-norm clipping at 1.0, and a maximum of 201 iterations.
Stopping Optimisation stops after 30 iterations without improvement. The weights with the lowest loss are restored at the end.
Guard If a trajectory’s loss exceeds ten times its lowest value so far, optimisation stops and the weights with the lowest finite loss are restored.
Evaluation 20 test trajectories per seed on E1; 42 64-step windows and, separately, six 448-step windows per seed on E2. Both flows use five seeds.
Loss guard. For full-residual TTO on E1, 24 of the 100 trajectories triggered the guard at the second iteration: the first optimiser step increased the loss to more than ten times its initial value. In each case, the lowest loss was that of the original checkpoint, so its weights were restored. These 24 trajectories are therefore evaluated as unadapted PINO rollouts in the TTO results. Of the remaining 76 trajectories, 42 reached the iteration limit and 34 stopped early. The reported means consequently include both adapted and unadapted trajectories, which should be considered when interpreting the accuracy comparison below.
Computational cost. We report the wall-clock time required to produce one test trajectory on one GPU, including both optimisation and the final rollout. Timing follows the same convention as for the methods without adaptation. It excludes training the initial checkpoint, whose cost is reported in the PINO row of Table 1.
Results on E1
On E1, TTO requires 66 ± 18 seconds per trajectory, compared with 0.020 ± 0.001 for the projected model and 0.019 ± 0.002 for a plain FNO. This is a factor of 3,300 relative to the projection. At step 1, its relative error is 10.1 ± 12.6% lower than the FNO’s, with an improvement on four of five seeds (1.48 × 10^{-3} compared with 1.64 × 10^{-3}). The mean improvement is smaller than the seed standard deviation, and at step 64 the error is 7.6% higher than the FNO’s.
TTO reduces the constraint residuals but does not enforce them exactly. The maximum divergence is 2.4 × 10^{-2} and the continuity residual is 2.2 × 10^{-3}, approximately three orders of magnitude above the projection’s and . The projection achieves these values with an additional 4% of training time and a small inference overhead.
Results on E2
For the extended E2 evaluation, adaptation uses the first 64 steps of each window, and the adapted weights are then used for the full 448-step rollout. This limits the cost of differentiating through the rollout, which would otherwise span 448 steps for every test trajectory. On E1, the same adaptation horizon covers the full available trajectory. This extended evaluation comprises six windows per seed, five seeds and two methods, giving sixty trajectories. None triggered the loss guard. Unlike E1, where the initial checkpoint already has a low residual, the unadapted E2 models produce divergent rollouts over this horizon.
| over the six 448-step windows | PINO | PINO + TTO | PINO (cont) | PINO (cont) + TTO |
| relative , horizon mean | 280 | 1.12 | 338 | 1.06 |
| relative at step 64 | 1.060 | 1.029 | 1.089 | 1.049 |
| domain-mean velocity, final | 69 | 111 | ||
| continuity residual | 63 | 92 | ||
| seconds per trajectory | 0.61 | 120 | 0.57 | 79 |
Table 3.
Adaptation prevents blow-up over this horizon and reduces the constraint diagnostics by four to nine orders of magnitude. These errors remain above those of the projection, whose maximum divergence and final domain-mean velocity are and on the same six windows, at 0.70 seconds per trajectory. All reported accuracy and constraint diagnostics improve under adaptation in this comparison, but both unadapted PINO models have diverged by step 448. The improvements therefore reflect suppression of that divergence. Both TTO methods lose energy, taking most trajectories below the 0.1× bound (§ 2.3).
Adapting against continuity alone
On E1, adapting against continuity alone reduces the objective while degrading prediction accuracy. The objective uses the continuity residual with its training weight of 0.12 and zero weight on the momentum terms.
| E1, final step of the 64-step rollout | before adaptation | after | PINO + TTO |
| continuity residual (the objective) | |||
| domain-mean velocity | |||
| relative | 0.023 | 0.017 | |
| seconds per trajectory | — | 137 | 66 |
Table 4.
This loss of accuracy occurs on all five seeds and is accompanied by a large increase in the domain-mean velocity. A spatially uniform velocity field has zero divergence, so the mode lies in the kernel of the divergence operator. A continuity-only objective therefore does not penalise changes to the domain mean. Over the course of the rollout, the domain-mean velocity increases by two orders of magnitude. The momentum residual, which is absent from the objective, rises from 0.093 (compared with 0.094 for the ground truth) to 0.72. When the objective includes continuity and both momentum components, adaptation reduces the domain-mean velocity and the momentum residual falls from 0.092 to 0.064.
Continuity satisfies the filtering criterion of Appendix § 6, but enforcing a valid constraint alone does not prevent drift in quantities that it leaves unconstrained. Here, continuity does not determine the mode. The projection addresses this by enforcing continuity and global momentum balance jointly in (8): both sets of constraints are included in , and the momentum rows fix the domain mean left unconstrained by the divergence rows.
Reproducibility
Availability. The code that produced every result in this paper is released at https://github.com/m-groom/physics-constrained-neural-operators. The E2 dataset, along with its normalisation statistics, is available on Zenodo at https://doi.org/10.5281/zenodo.23113478; the solver configuration that generated it is at https://github.com/m-groom/NS2D. The E1 dataset is available from the PINO repository of Li et al. [19]. The E2 simulations use Dedalus [4], version 3.0.4, with the equations and parameters in Appendix § 8.
Data preparation. E1 uses NS_fft_Re500_T4000.npy. Before spectral inversion to velocity and pressure, we remove the mean vorticity and, as a numerical convention, the Nyquist row and column to avoid ambiguity in the spectral derivatives. The pressure has zero domain mean. A NumPy random permutation with seed 1234 assigns the first 200 trajectories to validation, the next 200 to testing and the remaining 3,600 to training.
For E2, simulation seeds 1234–1236 supply training data, 1237 validation and 1238 testing. After the spin-up cut and temporal stride in Appendix § 8, each realisation contains 2,688 consecutive pairs. From each training realisation, the final 384 pairs are reserved as a secondary temporal test set, separated from training by eight strided snapshots. This leaves 2,295 pairs per training realisation, or 6,885 in total; the reported E2 results use the separate test realisation. Normalisation statistics are computed from training inputs only: each channel is centred on its mean, the velocity components share a standard deviation, and pressure has its own. The projection and physical residuals are evaluated after returning the fields to physical units.
Software. Training and evaluation use Python 3.12, PyTorch 2.5.1 and CUDA 12.2, with single-precision model parameters and fields.
Hardware and cost. Each training or evaluation run uses one NVIDIA H100 GPU with 94 GB of device memory. An FNO training run takes approximately six hours on E1 (280 epochs at about 80 seconds per epoch) and ten minutes on E2 (100 epochs at about 6 seconds per epoch). Table 1 gives the cost ratios for the other methods, and Appendix § 10 gives the absolute TTO times. The five E2 simulations each used 32 CPU ranks for 9–10 hours and produced approximately 22 GB of output. Preparing the coarse dataset took 15 minutes on eight CPU cores with a peak host-memory use of 39 GB. These costs exclude preliminary experiments and hyperparameter screens.
Asset licences. Dedalus is distributed under the GNU General Public License version 3, and the PINO repository under the Apache License 2.0. The E1 data are obtained from the original authors’ download linked above; no separate dataset licence is specified there.
References
- [1]A. Baez, W. Zhang, Z. Ma, L. Nguyen, S. Das, and L. Daniel. Guaranteeing conservation of integrals with projection in physics-informed neural networks. arXiv preprint arXiv:2511.09048, 2025. URL https://arxiv.org/abs/2511.09048.
- [2]T. Beucler, M. Pritchard, S. Rasp, J. Ott, P. Baldi, and P. Gentine. Enforcing analytic constraints in neural networks emulating physical systems. Physical Review Letters, 126:098302, 2021. doi: 10.1103/PhysRevLett.126.098302. URL https://arxiv.org/abs/1909.00912.
- [3]G. Boffetta and R. E. Ecke. Two-dimensional turbulence. Annual Review of Fluid Mechanics, 44:427–451, 2012. doi: 10.1146/annurev-fluid-120710-101240.DOI
- [4]K. J. Burns, G. M. Vasil, J. S. Oishi, D. LeCоanet, and B. P. Brown. Dedalus: A flexible framework for numerical simulations with spectral methods. Physical Review Research, 2(2):023068, 2020. doi: 10.1103/PhysRevResearch.2.023068. URL https://arxiv.org/abs/1905.10388.
- [5]N. Chalapathi, Y. Du, and A. Krishnapriyan. Scaling physics-informed hard constraints with mixture-of-experts. In International Conference on Learning Representations (ICLR), 2024. URL https://arxiv.org/abs/2402.13412.
- [6]H. Chen, G. E. Constante-Flores, and C. Li. Physics-informed neural networks with hard linear equality constraints. Computers & Chemical Engineering, 189:108764, 2024. URL https://arxiv.org/abs/2402.07251. arXiv:2402.07251.
- [7]P. L. Donti, D. Rolnick, and J. Z. Kolter. DC3: A learning method for optimization with hard constraints. In International Conference on Learning Representations (ICLR), 2021. URL https://arxiv.org/abs/2104.12225.
- [8]V. Duruisseaux, M. Liu-Schiaffini, J. Berner, and A. Anandkumar. Towards enforcing hard physics constraints in operator learning frameworks. In ICML 2024 AI for Science Workshop, 2024. URL https://openreview.net/forum?id=Zvxm14Rd1F.
- [9]P. D. Grontas, A. Terpin, E. C. Balta, R. D’Andrea, and J. Lygeros. Ilnet: Optimizing hard-constrained neural networks with orthogonal projection layers. In International Conference on Learning Representations (ICLR), 2026. URL https://arxiv.org/abs/2508.10480. arXiv:2508.10480.
- [10]Y. Guan, A. Subel, A. Chattopadhyay, and P. Hassanzadeh. Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate LES. Physica D: Non-linear Phenomena, 443:133568, 2023. URL https://www.sciencedirect.com/science/article/abs/pii/S016727892200272X. arXiv:2201.07347.DOI
- [11]D. Hansen, D. C. Maddix, S. Alizadeh, G. Gupta, and M. W. Mahoney. Learning physical models that can respect conservation laws. In Proceedings of the 40th International Conference on Machine Learning (ICML), volume 202 of PMLR, pages 12469–12510, 2023. URL https://arxiv.org/abs/2302.11002.
- [12]P. Harder, A. Hernandez-Garcia, V. Ramesh, Q. Yang, P. Sattigeri, D. Szwarcman, C. Watson, and D. Rolnick. Hard-constrained deep learning for climate downscaling. Journal of Machine Learning Research, 24(365):1–40, 2023. URL https://jmlr.org/papers/v24/23-0158.html.arxiv.org/abs/2208.05424
- [13]C. Jidling, N. Wahlström, A. Wills, and T. B. Schön. Linearly constrained Gaussian processes. In Advances in Neural Information Processing Systems (NeurIPS), volume 30, 2017. URL https://arxiv.org/abs/1703.00787.
- [14]D. Kochkov, J. A. Smith, A. Alieva, Q. Wang, M. P. Brenner, and S. Hoyer. Machine learning–accelerated computational fluid dynamics. Proceedings of the National Academy of Sciences, 118(21):e2101784118, 2021. URL https://www.pnas.org/doi/10.1073/pnas.2101784118.DOI
- [15]M. Lange-Hegermann. Algorithmic linearly constrained Gaussian processes. In Advances in Neural Information Processing Systems (NeurIPS), volume 31, 2018. URL https://arxiv.org/abs/1801.09197.
- [16]X. Li, H. Zhang, R. Jiang, D. Chen, C. Lin, L. Han, Y. Qi, X. Guo, and Y. Cheng. Project and generate: Divergence-free neural operators for incompressible flows. arXiv preprint arXiv:2603.24500, 2026. URL https://arxiv.org/abs/2603.24500.
- [17]Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations (ICLR), 2021. URL https://arxiv.org/abs/2010.08895.
- [18]Z. Li, M. Liu-Schiaffini, N. Kovachki, B. Liu, K. Azizzadenesheli, K. Bhattacharya, A. Stuart, and A. Anandkumar. Learning dissipative dynamics in chaotic systems. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, 2022. URL https://arxiv.org/abs/2106.06898.
- [19]Z. Li, H. Zheng, N. Kovachki, D. Jin, H. Chen, B. Liu, K. Azizzadenesheli, and A. Anandkumar. Physics-informed neural operator for learning partial differential equations. ACM/IMS Journal of Data Science, 2024. doi: 10.1145/3648506. URL https://arxiv.org/abs/2111.03794. arXiv:2111.03794 (v1 2021, v4 2023).
- [20]C. Liu, Y. Li, Z. Deng, C. Budd, and C.-B. Schönlieb. Adaptive correction for ensuring conservation laws in neural operators. arXiv preprint arXiv:2505.24579, 2025. URL https://arxiv.org/abs/2505.24579. v1 titled “Conservation-preserved Fourier Neural Operator through Adaptive Correction”.
- [21]N. Liu, Y. Fan, X. Zeng, M. Klöwer, L. Zhang, and Y. Yu. Harnessing the power of neural operators with automatically encoded conservation laws. In Proceedings of the 41st International Conference on Machine Learning (ICML), volume 235 of PMLR, pages 30965–30997, 2024. URL https://arxiv.org/abs/2312.11176. arXiv:2312.11176.
- [22]N. McGreivy and A. Hakim. Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations. Nature Machine Intelligence, 6(10): 1256–1269, 2024. URL https://www.nature.com/articles/s42256-024-00897-5.DOI
- [23]Y. Min and N. Azizan. HardNet: Hard-constrained neural networks with universal approximation guarantees. arXiv preprint arXiv:2410.10807, 2024. URL https://arxiv.org/abs/2410.10807.
- [24]A. T. Mohan, N. Lubbers, M. Chertkov, and D. Livescu. Embedding hard physical constraints in neural network coarse-graining of three-dimensional turbulence. Physical Review Fluids, 8:014604, 2023. doi: 10.1103/PhysRevFluids.8.014604. URL https://arxiv.org/abs/2002.00021. arXiv:2002.00021 (2020).
- [25]G. Négiar, M. W. Mahoney, and A. S. Krishnapriyan. Learning differentiable solvers for systems with hard constraints. In International Conference on Learning Representations (ICLR), 2023. URL https://arxiv.org/abs/2207.08675.
- [28]J. Richter-Powell, Y. Lipman, and R. T. Q. Chen. Neural conservation laws: A divergence-free perspective. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, 2022. URL https://arxiv.org/abs/2210.01741.
- [29]F. M. Rohrhofer, S. Posch, C. Göbnitzer, and B. C. Geiger. Data vs. physics: The apparent Pareto front of physics-informed neural networks. IEEE Access, 11:86252–86261, 2023. doi: 10.1109/ACCESS.2023.3302892. URL https://arxiv.org/abs/2105.00862.
- [31]K. Stachenfeld, D. B. Fielding, D. Kochkov, M. Cranmer, T. Pfaff, J. Godwin, C. Cui, S. Ho, P. Battaglia, and A. Sanchez-Gonzalez. Learned coarse models for efficient turbulence simulation. In International Conference on Learning Representations (ICLR), 2022. URL https://arxiv.org/abs/2112.15275.
- [32]U. Utkarsh, P. Cai, A. Edelman, R. Gomez-Bombarelli, and C. V. Rackauckas. Physics-constrained flow matching: Sampling generative models with hard constraints. In Advances in Neural Information Processing Systems (NeurIPS), 2025. URL https://arxiv.org/abs/2506.04171. arXiv:2506.04171.
- [33]C. Wang, B. Bonev, J. Berner, Z. Li, D. Zhou, J. Wang, T. Kurth, J. Bae, and A. Anandkumar. Coarse graining with neural operators for simulating chaotic systems. arXiv preprint arXiv:2408.05177, 2024. URL https://arxiv.org/abs/2408.05177. Earlier title: “Beyond Closure Models: Learning Chaotic Systems via Physics-Informed Neural Operators”.
- [34]X. Wang, M. Groom, R. Oliveira, H. Zhao, T. O’Kane, and E. V. Bonilla. Multi-scale wavelet transformers for operator learning of dynamical systems. arXiv preprint arXiv:2602.01486, 2026. URL https://arxiv.org/abs/2602.01486.
- [35]Q. Xu, J. L. Bamber, N. Thuerey, N. Boers, P. Bates, G. Camps-Valls, Y. Shi, and X. X. Zhu. Physically consistent and uncertainty-aware learning of spatiotemporal dynamics. arXiv preprint arXiv:2510.21023, 2025. URL https://arxiv.org/abs/2510.21023.
- [36]S. Zhao, Z. Li, B. Fan, Y. Wang, H. Yang, and J. Wang. LESnets (large-eddy simulation nets): Physics-informed neural operator for large-eddy simulation of turbulence. Journal of Computational Physics, 537:114125, 2025. doi: 10.1016/j.jcp.2025.114125. URL https://arxiv.org/abs/2411.04502. arXiv:2411.04502.






