WorksMathematics
Learning Autoregressive Neural Operators on Solver-Defined Physical Constraint Manifolds
Abstract
Autoregressive neural operators approximate time-dependent PDE solvers by repeatedly composing a learned transition, although the resulting states need not remain feasible under the discrete constraints of the numerical problem. We study this issue for incompressible flow, where each velocity state must satisfy solver-level incompressibility together with prescribed boundary values. We formulate autoregressive operator learning as a map between solver-defined affine feasible sets and introduce PCM-NO, a projected-residual realization of this feasible-transition principle. The neural backbone predicts a candidate state increment, a fixed tangent projection extracts its admissible component, and a metric retraction maps the advanced state to the prescribed feasible set. With exact affine constraint solves, the resulting transition is closed on the feasible set, while an affine identity separates admissible-update parameterization from state-level closure. With approximate solves, tangent-only residuals admit a cumulative upper bound, whereas per-step retraction resets feasibility to the residual achieved by the current solve. Experiments on periodic Navier–Stokes flow and lid-driven cavity flow demonstrate the superiority of PCM-NO in preserving solver-level constraints over long autoregressive rollouts and under parameter, boundary-value, and resolution shifts, while achieving lower rollout errors and better flow-specific diagnostics across both in-distribution and stress settings.
Introduction
Neural operators provide reusable function-to-function surrogates for parametric PDE families and have become an effective approach to accelerating repeated scientific simulation [18, 22, 28]. For time-dependent systems, an autoregressive neural operator learns a short-time transition and obtains long-horizon predictions by repeatedly composing that transition [2, 24, 29]. This formulation mirrors the stepwise structure of numerical solvers and supports flexible prediction horizons across varying physical conditions. However, it also makes every predicted state part of the input encountered by subsequent transitions. Constraint violations that survive one step can therefore enter the remainder of the rollout and progressively affect its physical validity and predictive stability.
This issue is especially acute in incompressible flow, where preservation of local fluid volume requires a divergence-free velocity field, and a fixed discretization encodes this requirement together with prescribed boundary values as algebraic constraints on admissible velocity states [12]. Yet solver-generated training trajectories alone do not guarantee that a learned transition preserves this admissible set from one step to the next [23, 29]. Existing physics-aware approaches address constraint violations through soft-constraint objectives based on residual or conservation penalties [17, 23, 32], structure-specific representations [4, 8, 34], or post-hoc state projection [20]. These approaches have distinct limitations in autoregressive rollout. Soft-constraint objectives do not guarantee feasibility at every step, so residual violations can enter subsequent predictions and accumulate over time. Structure-specific representations encode selected constraints but lack composability, requiring specialized designs for joint conditions such as incompressibility and nonhomogeneous boundary values. Post-hoc state projection like FinalProj [20] restores output feasibility only after an unconstrained next-state prediction, without constraining the learned update before state advancement.
Figure 1.
CCS Concepts • Computing methodologies → Machine learning; • Applied computing → Physics.
ACM Reference Format: Nannan Zhou, Guanmeng Xian, and Ning Yang. 2026. Learning Autoregressive Neural Operators on Solver-Defined Physical Constraint Manifolds. In . ACM, New York, NY, USA, 15 pages. https://doi.org/10.1145/nnnnnnn.nnnnnnn
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Conference’17, Washington, DC, USA © 2026 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-x-xxxx-xxxx-x/YYYY/MM https://doi.org/10.1145/nnnnnnn.nnnnnnn These limitations motivate the feasible-transition view of autoregressive operator learning. Rather than allowing the learned dynamics to evolve freely in the ambient state space, we treat the solver-defined feasible set as the state space of the learned transition, so that every autoregressive prediction remains physically feasible at the solver level. This set captures discrete kinematic requirements such as incompressibility and prescribed boundary values, while the neural operator learns the dynamics within the feasible set. Based on this idea, we propose Physical-Constraint-Manifold Neural Operators (PCM-NO), a unified transition-level formulation in which the solver-defined feasible set itself serves as the state space of the learned dynamics. PCM-NO realizes this formulation by coupling a backbone neural operator with two solver-defined components. The tangent projection filters the backbone’s candidate residual update before state advancement, so that only directions compatible with the homogeneous constraints enter the rollout. The metric retraction then maps the advanced state to the feasible set associated with the current constraint target, resetting numerical constraint residuals at each step. The advantages of PCM-NO are threefold. First, its projection–retraction factorization makes per-step solver-level feasibility a structural property of the transition, independent of loss balancing and of whether the backbone learns the constraints from data. Second, the projected-residual realization exposes the admissible component entering the learned dynamics and removes constraint-incompatible directions before state advancement, providing an interpretable inductive bias beyond state-level correction alone. Third, PCM-NO provides a unified transition formulation whose parameter-free, solver-derived layers accommodate periodic incompressibility, joint incompressibility–boundary constraints, and step-dependent affine targets.
Our analysis separates exact feasible-state closure, admissible-update parameterization, and finite-solve behavior. Under exact affine constraints, the direct-state and projected-residual realizations belong to the same feasible-transition family and map feasible inputs to the prescribed affine set. The tangent layer explicitly exposes the admissible component entering the learned dynamics and removes its orthogonal constraint-incompatible component before state advancement. Under approximate solves, tangent-only residuals can accumulate across rollout steps, whereas retraction resets the state residual to the accuracy achieved by the current solve and accommodates changes in the affine target. These results characterize the geometry and numerical behavior of the PCM-NO transition. The experiments then evaluate its predictive accuracy, rollout stability, and generalization under stress.
Our contributions can be summarized as follows:
We formulate autoregressive operator learning as a feasible-state transition and propose PCM-NO, which couples a neural candidate-update backbone with successive tangent projection and metric retraction to keep every rollout step feasible under solver-defined affine constraints.
We establish exact feasible-state closure, characterize the affine relationship between tangent projection and retraction, and analyze orthogonal update filtering together with residual accumulation and reset under approximate constraint solves.
Extensive experiments on periodic and boundary-constrained incompressible flows verify the superiority of PCM-NO.
Preliminaries
Time-Evolution Operator and Discrete Realization
At each time , the state is a spatial field function over the physical domain. The time- evolution operator maps the field function to the one at the next time level:
where denotes physical parameters and boundary or forcing data. At spatial resolution , these fields are represented by discrete states at , and the reference solver induces the discrete evolution operator
The learned transition is defined on this same discrete state representation, where the solver-level constraints are specified.
Feasible States
Let be the number of scalar constraints. At a fixed discretization and conditioning, a discrete state is called feasible if it satisfies the solver-defined constraint system , where is the discrete constraint operator, and contains the corresponding prescribed values. The set of all feasible states is
The operator may combine multiple solver-defined constraints. For incompressible flow, it includes the discrete divergence operator as the block encoding incompressibility, while the corresponding block of is zero. Consequently, every feasible state satisfies , the discrete counterpart of a divergence-free velocity field. Both and are determined by the numerical problem and are not learnable.
We assume that is linear and that is nonempty for every problem instance considered. Under these conditions, the feasible set is an affine translate of and therefore an embedded affine manifold. We call the algebraic object in Eq. (3) the solver-defined affine feasible set and its geometric interpretation the physical constraint manifold.
Problem formulation
Let the -th trajectory be generated by the reference solver under physical parameters , discretized boundary or forcing data , and the corresponding constraint target . The training set contains adjacent conditioned function pairs
with , where is the number of training trajectories and is the number of transitions in the -th trajectory.
The goal is to learn a neural operator that approximates the solver-induced transition while preserving the feasible state set:
Figure 1 (PDF p. 3). Overview of PCM-NO. An ambient autoregressive rollout can propagate constraint-incompatible components across steps. In PCM-NO, the backbone predicts a candidate update, retains its admissible component, and returns the advanced state to the prescribed affine feasible set. The right panel illustrates solver-defined constraints encoded by .
The first relation in Eq. (4) specifies a structural closure requirement for the learned transition, while the second relation defines the statistical approximation objective to accommodate constraints such as discrete incompressibility and prescribed boundary values.
Method
Overview
Figure 1 contrasts an ambient autoregressive rollout with the feasible-state transition of PCM-NO and shows how its fixed layers are derived from solver-defined constraints. In the ambient rollout, each predicted state becomes the input to the next transition, so any constraint-incompatible component can be propagated and amplified across steps. PCM-NO instead factorizes each learned residual transition into a candidate update, tangent projection, state advancement, and metric retraction. This construction separates data-driven dynamics from solver-level constraint enforcement, allowing the backbone to learn within the feasible state space defined by . The same formulation therefore supports both periodic incompressibility and joint incompressibility–boundary constraints.
Given a feasible state , the backbone predicts a candidate update in the ambient discrete space. The tangent projection retains its admissible component in before state advancement, and the metric retraction returns the advanced state to the affine feasible set associated with the current constraint target:
where denotes the identity operator on the discrete state space . Note that only is trainable. The projection filters the learned update, while the retraction resets state-level feasibility at every rollout step. Both layers are constructed from the discrete constraint system , allowing the same transition-level formulation to accommodate constraints such as discrete incompressibility and prescribed boundary values.
Tangent projection and retraction
Let define the discrete inner product , . For , let denote the tangent space of at , namely the set of admissible directions along which the constraint value remains unchanged. Because is linear,
Hence remains in precisely when . The tangent space is therefore independent of the base point and satisfies
Tangent projection. Given a candidate update , the tangent projection is the -orthogonal projection onto :
Thus the projected update , which can be added to a feasible state without changing the constraint value.
Metric retraction. For an intermediate state , we define the metric retraction as the -metric projection onto the affine feasible set:
For a feasible base state and a tangent increment , the corresponding local map
satisfies and .
PCM-NO Rollout Transition
We now specify the projected residual transition summarized in Figure 1. At time level , the backbone produces a candidate update
The tangent layer filters this update and advances the current state according to and . The complete PCM-NO transition is
The backbone determines a candidate update , while only enters the rollout. If and the tangent projection is exact, then the intermediate state is already feasible, i.e., . In this case, the final retraction acts as the identity, , leaving a feasible intermediate state unchanged. When the tangent solve is approximate or the affine target changes between steps, becomes active by resetting state-level feasibility and mapping the state to the prescribed next feasible set.
Relation to post-hoc state projection. The feasible-transition formulation subsumes post-hoc state projection [20] as a direct-state realization. Let be an unconstrained next-state predictor and define . For , the exact affine projection identity gives
Thus, post-hoc state projection is the direct-state realization of the affine feasible-transition principle, while PCM-NO is its projected-residual realization. Both inherit exact closure, while PCM-NO makes the admissible update explicit.
Changing affine targets. When the constraint operator remains fixed but its right-hand side changes from to , the same tangent projection can be combined with a retraction onto the next feasible set:
Thus, a state in is mapped to a state in , while the homogeneous admissible direction space remains .
Training objective
Training minimizes prediction error over autoregressive rollout windows. Let denote the training horizon and the starting index of a window. For trajectory , initialize , and recursively apply
The reported models are trained with
where is a discrete field-level prediction loss and controls the relative contribution of rollout step . Note that the feasibility is specified by the transition architecture in Eq. (9).
Structural Analysis
The analysis characterizes the solver-defined tangent projection and metric retraction, as well as the residual behavior of their numerical realizations. All proofs and derivations are given in Appendix A.
Proposition 1 (Exact feasible-state transition). Any feasible input satisfies
If , then the exact autoregressive rollout remains in for every . More generally, when the target changes between steps,
Remark 1 (Interpretation of exact closure). Proposition 1 shows that the exact PCM-NO transition maps a feasible state to the prescribed affine feasible set, including under step-dependent targets. This closure is determined by the fixed solver-defined constraint layers and is independent of the learned backbone and its parameters. Sections 5.2–5.4 will evaluate the corresponding numerical feasibility under in-distribution and shifted conditions.
Proposition 2 (Tangent filtering toward admissible increments). For any candidate increment , with in the PCM-NO transition, and any admissible target increment ,
Consequently,
The target may be the difference between two states in the same affine feasible set or the tangent projection of a reference increment.
Remark 2 (Interpretation of tangent filtering). Proposition 2 shows that removes exactly the -orthogonal component outside and cannot increase the distance to any admissible target increment. When two consecutive reference states satisfy the same affine target, , so their increment lies in up to the data-generation tolerance. Tangent filtering therefore preserves dynamics within the feasible set while removing constraint-incompatible components. Section 5.5 verifies this decomposition on realized model updates.
Assumption 1 (Approximate numerical constraint layers). At rollout step , the implemented tangent projection and metric retraction satisfy
Here, and denote the per-step residual bounds achieved by the numerical tangent-projection and metric-retraction solves, respectively, which may depend on the discretization, rollout step, and numerical solve budget.
Proposition 3 (Residual accumulation bound and per-step reset). Let be the residual for a fixed target . A tangent-only rollout satisfies
The full transition with a final approximate metric retraction satisfies
Remark 3 (Interpretation of residual propagation). Proposition 3 shows that residuals from approximate tangent projections can accumulate across rollout steps, whereas the metric retraction resets the state residual to the accuracy achieved by the current numerical solve. This behavior will be examined empirically in § 4.5.
Experiments
Experimental setup
Datasets and evaluation settings. We use two datasets governed by the 2-dimensional incompressible Navier–Stokes equations. Periodic NS (P-NS) is forced periodic Kolmogorov flow on , with and , where is the periodic spectral divergence operator. Lid-driven cavity flow (LC) is defined on with no-slip walls and a prescribed moving lid, using and , where extracts boundary degrees of freedom and stores the wall and lid velocities. On these datasets, we evaluate in-distribution rollout, long-horizon rollout, and stress settings that vary Reynolds number, lid velocity, resolution, data budget, or projection tolerance without changing the underlying dataset definitions. Appendix § 11.§ 1 describes the two constraint configurations and the construction of the stress conditions, while Table 5 in Appendix summarizes the data generation, discretization, and split statistics.
| Setting | P-NS | LC |
| Domain | ||
| Generator | Dealiased pseudo-spectral RK4 solver | MAC-grid projection solver |
| State representation | Velocity field | Staggered velocity field |
| Spatial resolution | ||
| Stored states per trajectory | 56 | 56 |
| One-step transitions per trajectory | 55 | 55 |
| Saved-state interval | 0.05 | 0.006 |
| Burn-in | 8 saved intervals | 4 saved intervals |
| Train/validation/test trajectories | 500/100/200 | 500/100/200 |
| Training Reynolds numbers | ||
| Training boundary data | Periodic | |
| Training rollout horizon | ||
| ID evaluation horizons |
Table 5. Dataset generation and evaluation statistics. Spatial resolution refers to the active computational grid, and stored states include the initial state of each trajectory. All methods use identical trajectory splits within each dataset.
Baselines. All methods use FNO-based backbones with matched capacity. FNO is the unconstrained autoregressive reference [22]. DivReg follows the penalty-based treatment of incompressibility in physics-informed operator learning [23], adding a divergence penalty on P-NS and both divergence and prescribed-boundary penalties on LC. PINO augments the rollout objective with a dataset-specific discrete PDE residual, without instance-wise test-time optimization [23]. ClawNO is an external structure-preserving baseline evaluated only on P-NS using its FNO-based clawFNO realization [25], since its periodic potential representation does not directly encode the stacked LC constraint . Its implementation details are provided in Appendix § 12.§ 0.1. FinalProj is a representative post-hoc state-projection baseline, following the direct-state correction principle [20]. On P-NS, it uses a spectral Leray projection; on LC, it enforces the joint affine constraint . PCM-NO instead predicts a candidate residual update, projects it onto , advances the state, and applies a final metric retraction.
Training and reporting. Unless stated otherwise, all methods use identical splits and autoregressive training windows of length on data with . LC additionally uses prescribed lid velocities in . Method-specific loss weights are fixed before test evaluation. The P-NS PINO residual weight is selected using validation rollouts, while the remaining soft-loss weights are fixed within each dataset. Complete settings are reported in Appendix C.3. Main test results are reported as mean standard deviation over three seeds. Constraint-solver diagnostics and end-to-end inference cost are reported in Appendices D.2 and D.3, respectively.
Metrics. Rollout accuracy is measured by RelL2, where denotes the rollout horizon. We quantify solver-level feasibility using Div, the base-10 logarithm of the discrete incompressibility residual. On LC, we additionally report BC, defined analogously from the boundary-condition residual. We additionally report the energy relative error on P-NS and the centerline profile error on LC. Formal definitions and the aggregation protocol are provided in Appendix § 11.§ 4.
In-Distribution Feasible Rollouts
On P-NS, FinalProj, ClawNO, and PCM-NO maintain divergence-free rollouts near the numerical floor of their constraint mechanisms, whereas the soft-constraint methods remain near and FNO becomes non-finite before 50 steps. PCM-NO achieves the lowest RelL2, RelL2, and RelL2. Although FinalProj reaches a nearly identical Div, the shared feasibility floor controls for output constraint satisfaction and isolates the effect of transition parameterization. The lower rollout errors of PCM-NO therefore support the empirical value of its projected-residual realization. The corresponding tangent-filtering mechanism is examined directly in § 4.5. Results in Appendix § 12.§ 0.1 further show that PCM-NO substantially improves accuracy and energy error over ClawNO, while the runtime comparison quantifies the cost of the projection–retraction transition.
On LC, where , PCM-NO maintains both incompressibility and the prescribed boundary values near the achieved solve floor while attaining the lowest rollout errors. The joint Div–BC behavior shows that the relevant feasible set is the complete affine constraint set and distinguishes hard solver-level feasibility from soft residual reduction.
Long-Horizon Rollout Stability
Figure 2 traces accuracy, feasibility, and flow-specific diagnostics across the rollout horizon. On P-NS, FinalProj and PCM-NO maintain projection-level incompressibility, while the unconstrained FNO rollout becomes non-finite. PCM-NO also gives the lowest long-horizon state and energy errors among the finite methods. On LC, it keeps both Div and BC near their numerical floors and substantially reduces the state and centerline-profile errors. The results show that the feasible transition remains effective under repeated composition and that its strongest predictive advantage appears when incompressibility and prescribed boundary values must be enforced jointly.
Figure 2. Long-rollout diagnostics. Shaded bands show the mean ± one standard deviation over three seeds. FNO curves are truncated once the corresponding rollout becomes non-finite in the P-NS accuracy and energy panels.
Generalization and Stress Diagnostics
We evaluate four stress settings: P-NS Reynolds-number extrapolation (P-NS-Re), LC Reynolds-number extrapolation (LC-Re), LC lid-velocity shift (LC-Lid), and P-NS resolution transfer (P-NS-Grid). Consistent with the pointwise closure property in Proposition 1, Table 2 shows that PCM-NO preserves near-numerical-floor feasibility beyond the training distribution. It maintains Div on
| Settings | Method | RelL ↓ | Div ↓ | BC ↓ | Energy/Profile Err. ↓ |
| P-NS-Re<br> | FNO | N.F. | N.F. | — | N.F. |
| P-NS-Re<br> | DivReg | 0.0999 ± 0.0006 | −1.9531 ± 0.0079 | — | 0.0091 ± 0.0051 |
| P-NS-Re<br> | PINO | 0.1091 ± 0.0055 | −1.8891 ± 0.0054 | — | 0.0105 ± 0.0010 |
| P-NS-Re<br> | FinalProj | 0.1161 ± 0.0103 | −5.5000 ± 0.0009 | — | 0.0275 ± 0.0205 |
| P-NS-Re<br> | PCM-NO | 0.0964 ± 0.0010 | −5.5006 ± 0.0005 | — | 0.0059 ± 0.0013 |
| LC-Re<br> | FNO | 0.0255 ± 0.0050 | −1.6761 ± 0.0374 | −3.1378 ± 0.1074 | 0.1493 ± 0.0257 |
| LC-Re<br> | DivReg | 0.0373 ± 0.0129 | −2.7939 ± 0.0618 | −3.1870 ± 0.0736 | 0.2134 ± 0.0387 |
| LC-Re<br> | PINO | 0.2799 ± 0.0288 | −1.5863 ± 0.1256 | −1.5279 ± 0.0522 | 0.8063 ± 0.1889 |
| LC-Re<br> | FinalProj | 0.0550 ± 0.0115 | −4.0374 ± 0.1134 | −6.0000 ± 0.0000 | 0.3059 ± 0.0762 |
| LC-Re<br> | PCM-NO | 0.0224 ± 0.0082 | −6.1617 ± 0.2751 | −6.0000 ± 0.0000 | 0.1025 ± 0.0324 |
| LC-Lid<br> | FNO | 0.0596 ± 0.0148 | −1.3433 ± 0.0264 | −2.4675 ± 0.1289 | 0.1415 ± 0.0287 |
| LC-Lid<br> | DivReg | 0.0839 ± 0.0135 | −2.1814 ± 0.0759 | −2.4631 ± 0.0400 | 0.1879 ± 0.0155 |
| LC-Lid<br> | PINO | 0.3579 ± 0.0115 | −1.4067 ± 0.0812 | −1.2386 ± 0.0266 | 0.6772 ± 0.0122 |
| LC-Lid<br> | FinalProj | 0.0601 ± 0.0363 | −3.3582 ± 0.1269 | −6.0000 ± 0.0000 | 0.1151 ± 0.0432 |
| LC-Lid<br> | PCM-NO | 0.0143 ± 0.0022 | −5.9020 ± 0.3165 | −6.0000 ± 0.0000 | 0.0311 ± 0.0031 |
| P-NS-Grid<br> | FNO | N.F. | N.F. | — | N.F. |
| P-NS-Grid<br> | DivReg | 0.0511 ± 0.0009 | −1.0807 ± 0.0588 | — | 0.0140 ± 0.0033 |
| P-NS-Grid<br> | PINO | 0.1002 ± 0.0052 | −1.2159 ± 0.0393 | — | 0.0080 ± 0.0009 |
| P-NS-Grid<br> | FinalProj | −5.1451 ± 0.0240 | — | ||
| P-NS-Grid<br> | PCM-NO | 0.0447 ± 0.0004 | −5.1798 ± 0.0009 | — | 0.0075 ± 0.0007 |
Table 2. Stress-set rollout results, using the in-distribution validation-selected checkpoints without adaptation. Reynolds-number rows pool the two test values. The last column reports energy error on P-NS and profile error on LC; N.F. denotes a non-finite rollout.
P-NS-Re, both Div and BC on LC-Re and LC-Lid, and near-feasibility after reconstructing the constraint operators on the P-NS grid. Across all four settings, PCM-NO also achieves the lowest RelL2 and flow-specific diagnostic error, whereas FinalProj reaches a comparable feasibility floor, so the lower state and flow-diagnostic errors of PCM-NO isolate the predictive advantage of the projected-residual realization under distribution shift. LC-Lid directly changes the affine target , and P-NS-Grid evaluates target-grid transfer under reconstructed constraint operators in a setting where grid transfer, unseen-frequency extrapolation, and aliasing are closely coupled [36].
Figure 3 further shows that PCM-NO better preserves the P-NS vorticity organization and LC cavity circulation after 50 steps, consistent with Table 2. Detailed protocols and metric definitions are provided in Appendices § 11.§ 1 and § 11.§ 4.
Figure 3. Representative 50-step stress predictions: P-NS Reynolds extrapolation at and the LC lid shift at . Rows share field and error color scales, respectively. Trajectories are selected by median ground-truth kinetic energy, independently of model error.
Mechanism Diagnostics
Tangent filtering. Proposition 2 states that projection onto the discrete tangent space cannot increase the distance from a candidate update to an admissible target. We test this property on realized updates from the trained PCM-NO models by comparing raw and projected candidate updates with the projected reference increment. As shown in Table 3, projection consistently reduces this error on both datasets, while the normalized Pythagorean gap remains near numerical precision. The removed normal component also accounts for a substantial fraction of the candidate-update energy, confirming that the tangent layer actively filters constraint-incompatible directions. This diagnostic verifies the local mechanism in Proposition 2, while § 4.6 evaluates its role in the complete transition.
| Dataset | Raw RelErr. ↓ | Projected RelErr. ↓ | Normal Frac. (%) | Pythag. Gap ↓ |
| P-NS | 0.8220 ± 0.0258 | 0.0607 ± 0.0014 | 39.75 ± 1.52 | |
| LC | 3.8898 ± 0.8964 | 0.0559 ± 0.0090 | 87.76 ± 3.19 |
Table 3. Tangent-filtering diagnostic. Errors are measured against the projected reference increment; Normal Frac. is the removed candidate-update energy fraction, and Pythag. Gap is the normalized residual of Proposition 2.
Residual accumulation and reset. Figure 4 examines the behavior predicted by Proposition 3. In a controlled LC replay, candidate updates generated along the full PCM-NO rollout are projected once using the evaluation solve budget and applied to two branches. TangentOnly omits the final retraction, whereas Full retracts the same update sequence after every step, so the branches differ only in the state-level reset. TangentOnly accumulates incompatibility error, while Full remains near the achieved constraint-solve floor. Both preserve the prescribed boundary values because the projected updates satisfy the homogeneous boundary constraint, and the tangent-only residual remains below the cumulative projection-residual bound. These results directly support the accumulation-versus-reset interpretation in Proposition 3.

Figure 4. Controlled LC replay with identical projected updates.
Component Ablations
Having directly verified tangent filtering and residual reset, we next examine how tangent projection, state retraction, and the complete affine constraint affect the learned rollout. We conduct the ablation on LC because it activates both incompressibility and prescribed boundary constraints. We consider three structural variants. TangentOnly retains the residual-update parameterization and the tangent projection, but omits the final retraction: . RetractOnly removes the tangent projection and retracts the unconstrained residual update: . DivOnly applies both fixed layers but the reduced constraint , thereby removing the prescribed boundary constraint . The full PCM-NO transition retains tangent projection, boundary-aware constraints, and state retraction.
Table 4 separates three numerical roles. TangentOnly retains the projected residual update but removes the state-level reset. Its increasing constraint residual is consistent with the accumulation mechanism in Proposition 3. RetractOnly applies the reset to an unconstrained residual update and therefore recovers much of the feasibility and accuracy. Relative to RetractOnly, PCM-NO reduces by 36% and by 58%, while improving by 3.35 log units. These gains quantify the empirical benefit of the explicit projected-update parameterization under finite constraint solves. DivOnly removes the prescribed-boundary component from , leading to substantial boundary violation and degraded profile accuracy. Together, the results support the complete affine constraint, the per-step feasibility reset, and the explicit admissible-update parameterization as complementary components of the implemented transition.
Computational implications. To place the accuracy–feasibility gains in a solver context, Appendix § 12.§ 2 compares complete learned transitions with the corresponding reference solvers over the same physical horizon. At batch size one, PCM-NO remains 7.6× faster than the P-NS reference solver and 6.4× faster than the LC reference solver over a 50-step rollout. Constraint enforcement costs 1.37× the FNO rollout on P-NS, but 41.53× on LC, where the two iterative constraint solves dominate inference. The computational overhead of the solver-defined constraint layers is therefore discretization-dependent, motivating warm starts, preconditioning, and adaptive solve budgets for boundary-aware settings.
Related Work
Neural operators and temporal PDE surrogates. Neural operators learn mappings between function spaces through DeepONet, FNO, and their factorized, geometry-adaptive, convolutional, localized, mesh-based, graph, and transformer variants [14, 18, 21, 22, 27, 28, 31, 33, 39, ?]. Foundation models and benchmarks broaden the task scope [15, 37], while autoregressive, refinement, memory, and stability-oriented methods target long-horizon accuracy [2, 3, 24, 29]. These works do not generally require every composed transition to remain in a solver-defined feasible set.
Physics-aware and constrained dynamics. Physics-informed, conservation aware, divergence-free, and boundary-aware operators incorporate PDE structure through residual objectives, data-free or boundary-integral formulations, and specialized parameterizations [4, 5, 8, 9, 17, 23, 25, 26, 32, 34, 41, 42]. EquiNO imposes equilibrium through divergence-free POD bases, while CompNO composes learned convection and diffusion modules with a Poisson solver [7, 16]. Related structure-preserving methods use Hodge decompositions, conservative transport, port–Hamiltonian factorizations, hard-constrained generative sampling, or classical projection methods [1, 6, 12, 19, 30, 38, 40, ?]. Neural–numerical and explicitly constrained models similarly combine learned mappings with analytical or iterative structure [10, 11, 13, 35], and post-hoc projection can enforce divergence-free outputs [20]. PCM-NO instead defines an autoregressive map between solver-defined affine feasible sets through a projected residual parameterization, supporting joint constraints and nonhomogeneous targets while enabling analysis of exact closure and approximate-solve residual propagation.
Conclusion
We introduced PCM-NO, an autoregressive neural-operator construction whose transition is closed on a solver-defined affine constraint manifold. The backbone predicts a candidate update direction, while fixed tangent projection and state-retraction layers determine the admissible increment and the feasible output state. The structural analysis establishes exact feasible-state closure, orthogonal filtering of constraint-incompatible increment components, and residual accumulation and reset bounds for approximate numerical solves. Across periodic and boundary-constrained incompressible-flow settings, PCM-NO maintains discrete incompressibility and prescribed boundary values over long rollouts and under controlled parameter, boundary-value, and resolution shifts. The experiments also show consistent improvements in rollout accuracy and flow-specific diagnostics. Extending the construction to nonlinear, state-dependent, and geometry-dependent constraint sets is a natural direction for future work.
Limitations and Ethical Considerations
The current formulation is limited to fixed linear discrete constraints and nonempty affine feasible sets. The experiments cover two-dimensional incompressible flows with an FNO backbone, while three-dimensional systems, irregular geometries, and nonlinear or state-dependent constraints remain to be studied. The cost of the constraint layers is also discretization dependent, with the iterative LC solver introducing substantially greater overhead than the spectral P-NS implementation.
The datasets are synthetic and contain no human or personally identifiable information. Learned CFD surrogates may nevertheless produce plausible but inaccurate predictions under untested conditions. Safety-critical use therefore requires independent validation, error estimation, and a verified numerical solver as fallback.
Generative AI Usage
Generative AI tools were used to assist language editing, structural revision, and code drafting. The authors reviewed all generated content and independently verified the mathematical derivations, implementations, experimental protocols, and reported results. Generative AI was not treated as an author and did not make autonomous scientific decisions.
Proofs and Approximate Constraint Solves
This appendix proves the structural statements in Section 4 and relates approximate constraint layers to the achieved residuals of their linear systems.
Proof of Proposition 1
Let
For every ,
Hence . Because is symmetric,
The KKT conditions for the tangent projection yield
Since ,
For the state projection,
The nonemptiness of implies , and therefore . It follows that
Finally, if , then
The subsequent state projection preserves membership in . Induction over proves the rollout statement.
Derivation of (10)
Define
For and any feasible target value ,
Taking gives
which proves (10).
Proof of Proposition 2
The projection is the -orthogonal projection onto . Hence , while is -orthogonal to . Since ,
Therefore, the decomposition
has -orthogonal components. Taking squared -norms gives the Pythagorean identity in Proposition 2. Dropping the nonnegative normal-component term yields the stated inequality.
Approximate Solves and Proof of Proposition 3
Let
The implementation may solve a damped normal equation
with . This includes the undamped case .
For tangent projection, set , define the achieved linear-system residual by
and let
Then
Thus one may take
or use the upper bound
For state projection, define
and
The resulting feasibility residual is
Consequently,
is valid. When , these expressions reduce to the undamped residual identities.
For the tangent-only rollout,
Repeated application of the triangle inequality gives
which proves Eq. (16). The full and retract-only transitions end each step with , so Eq. (17) follows directly from Assumption 1.
Discrete Constraint Operators
This appendix specifies the discrete operators used in the two constraint configurations. The same operators are used for data generation, projection or retraction, training-time rollout, and evaluation.
P-NS Spectral Constraint
For P-NS, is the periodic spectral divergence operator. Let denote the corresponding spectral gradient. Under the discrete periodic inner product,
so the Helmholtz projection
is the -orthogonal projection onto . It is implemented in Fourier space, with the zero mode left unchanged. The same spectral divergence operator is used by the constraint layer and by the reported P-NS feasibility metric.
LC MAC-Grid Constraint
LC states are represented on a MAC-style finite-volume grid. Let encode the no-slip walls and the moving lid, and let denote finite-volume incompressibility on the interior control volumes. Formally,
The implementation enforces this stacked constraint through boundary lifting and a reduced-coordinate projection. Let be the stored MAC field containing the prescribed wall and ghost values, and let
inject free-face variables into the full MAC state. Every state satisfying the boundary constraint has the unique representation
Define
Then
Thus the full constraint is equivalent to a reduced divergence constraint on the free velocity faces.
Let
The reduced tangent projection is
and the reduced state projection is
The current MAC implementation uses the Euclidean metric on the free faces, so . Candidate increments are first assigned homogeneous boundary values and then projected with Eq. (22). State retraction first applies the boundary lifting and then solves Eq. (23). The boundary constraint is therefore imposed by construction, while the remaining divergence residual is controlled by the reduced normal-equation solve.
The exact formulas correspond to a converged undamped solve. The implemented finite-iteration CG solver uses a small damping parameter, and its residual behavior is covered by Proposition 3 and Appendix A.4. The Div values reported for LC are evaluated on the interior MAC control volumes.
Experimental Details
Dataset Generation and Statistics
P-NS is generated as forced periodic incompressible flow on with . LC is generated on using a MAC-grid projection solver with no-slip walls and . Within each dataset, all methods use the same training, validation, and test trajectories. Table 5 summarizes the data generation, discretization, and evaluation settings.
Constraint configurations and stress protocol
The experiments use two constraint configurations as the organizing axis for the empirical study. This organization follows the central claim of PCM-NO: the learned autoregressive transition should map a solver-defined feasible set to itself. Both datasets are generated from two-dimensional incompressible Navier–Stokes dynamics, but they induce different affine constraint sets. This separation allows us to distinguish divergence preservation from boundary-aware feasibility. The continuous equations below define the data-generating families, while the reported feasibility metrics and the projection–retraction layers are defined using the corresponding discrete solver operators.
P-NS. P-NS denotes periodic Navier–Stokes, instantiated as forced periodic Kolmogorov flow. Let denote the two-dimensional torus. In continuous form, the P-NS data-generating family is
where . Equivalently, when represented on the periodic box , the velocity field satisfies
Here is the velocity field, is the pressure, in our nondimensionalized data generation, and is a fixed Kolmogorov forcing. At the discrete level, the feasible set is
so the only explicit constraint operator is the periodic spectral divergence . Periodicity is built into the discretization, and no separate BC residual is reported. This configuration provides a clean test of divergence-free rollout, energy-based diagnostics, Reynolds-number extrapolation, and resolution transfer because the spectral projection and the energy diagnostic are defined on the same periodic grid.
LC. LC denotes lid-driven cavity flow. The lid is the top wall of the unit-square cavity. The left, right, and bottom walls are stationary no-slip walls, while the top lid moves horizontally with prescribed speed . The continuous data-generating equation is
with boundary condition on the left, right, and bottom walls, on the top lid. The resulting discrete feasible set is
Equivalently, and . This setting is the main test of the general constraint formulation. It requires a rollout to preserve both incompressibility and prescribed boundary values, and it prevents the method from being interpreted as only a periodic Helmholtz projection. The profile error reported for LC uses the vertical and horizontal centerline velocity profiles, which are more informative for cavity flow than a spectral-energy diagnostic.
Evaluation-only stress diagnostics. The main stress diagnostics reuse the validation-selected checkpoints from the in-distribution experiments. Test data are regenerated under controlled changes in the physical or numerical condition, while the trained model, normalization statistics, backbone hyperparameters, and soft-loss weights are kept fixed. No stress-set tuning or instance-wise test-time fine-tuning is used. This protocol makes the stress results a direct test of out-of-distribution rollout behavior without additional adaptation.
Reynolds-number extrapolation. For P-NS, the training range is and the stress test uses . For LC, the stress test uses , while the lid velocity remains in the training range . Keeping the LC lid distribution fixed isolates the effect of Reynolds-number extrapolation from a boundary-value shift. Since higher Reynolds numbers correspond to smaller viscosity and stronger small-scale or boundary-layer effects, these tests probe whether the learned transition remains stable beyond the physical-parameter range seen during training.
Boundary-value shift on LC. The lid-shift experiment keeps and evaluates at , outside the training range. This condition changes the right-hand side of the constraint system through and therefore changes the feasible set itself. It directly tests whether boundary lifting and boundary-aware retraction continue to impose when the prescribed boundary value differs from those observed in training.
Resolution transfer. The resolution-transfer experiment is run on P-NS by training at and evaluating at . Ground-truth trajectories are generated directly by the solver, with no interpolation from coarse trajectories. The discrete operators , , and are reconstructed on the target grid. Because cross-resolution evaluation can conflate resampling with extrapolation to unseen frequencies and expose aliasing [36], we use this setting only to test whether the method remains meaningful after target-grid reconstruction; we do not interpret it as evidence of general resolution invariance. LC resolution transfer is kept out of the main table because it couples mesh refinement with MAC-grid boundary projection and would add a separate implementation factor to the stress comparison.
Low-data stress. The low-data experiment is reported in the appendix because it changes the training set size. We retrain the methods with fixed fractions of the P-NS training trajectories and keep the validation and test sets unchanged. This diagnostic measures the data-efficiency effect of hard feasible transitions and does not replace the evaluation-only stress tests reported in the main table.
FinalProj within the feasible-transition formulation. FinalProj is the closest structural control and instantiates the direct-state realization in Eq. (10), while PCM-NO instantiates the projected-residual realization. Both return the predicted state to the same solver-defined feasible set, so comparable Div or BC floors control for state-level closure. Differences in RelL and Energy/Profile error therefore quantify the empirical effect of transition parameterization, while Table 4 separately examines tangent projection and retraction under finite constraint solves.
| Variant | ||||
| TangentOnly | ||||
| RetractOnly | ||||
| DivOnly | ||||
| PCM-NO |
Table 4. LC component ablation with 50-step rollouts.
Hyperparameter Settings
Table 6 lists the settings used in the paper-scale experiments. Architecture and optimizer choices are shared across methods within each dataset, and all method-specific loss weights are fixed before test evaluation.
| Setting | P-NS | LC |
| Spatial resolution | ||
| Train/validation/test trajectories | 500/100/200 | 500/100/200 |
| FNO layers / width / Fourier modes | 4/48/16 | 4/48/16 |
| Optimizer | AdamW | AdamW |
| Learning rate / weight decay | ||
| Maximum epochs | 100 | 100 |
| Training / evaluation batch size | 16/8 | 8/8 |
| Training rollout horizon | ||
| Validation / test rollout horizon | 20/50 | 50/50 |
| Random seeds | ||
| DivReg/PINO divergence weight | 0.10 | 0.05 |
| Boundary penalty weight | – | 0.10 |
| PINO residual weight | ||
| Constraint solve | spectral FFT | CG: 30/120 train/eval iterations |
| Evaluation solve tolerance / damping | – |
Table 6. Hyperparameter settings for the paper-scale experiments. The split sizes are numbers of trajectories. A dash indicates that the corresponding term is absent for that constraint configuration.
For P-NS, the PINO residual weight is selected using validation rollouts from
The two strongest candidates are rerun over three seeds . Their validation errors are for and for , so we select . The LC soft-loss weights are fixed before test evaluation and shared across runs. PCM-NO uses only the rollout objective in Eq. (12) and introduces no auxiliary loss weight.
For LC, the iterative constraint solve uses 30 CG iterations during training and 120 iterations during evaluation, with evaluation tolerance and damping . P-NS uses the closed-form spectral Helmholtz projection implemented with FFTs.
Model and optimizer
All main experiments instantiate each method with the same four-layer FNO backbone, using width 48, 16 Fourier modes per spatial dimension, and GELU activations. The network predicts the next state directly for FNO, DivReg, PINO, and FinalProj, and predicts a candidate update for PCM-NO. We optimize all models with AdamW using learning rate , weight decay , and at most 100 epochs. The training batch size is 16 for P-NS and 8 for LC. All methods use autoregressive windows of length , and checkpoints are selected by validation rollout error. The complete training and method-specific loss settings are reported in Table 6.
Metric Definitions
This subsection defines the metrics used in Table 1, Figure 2, Table 2, and Table 4. For each random seed, a metric is first aggregated over all trajectories in the corresponding test set. The paper then reports the mean and standard deviation of the resulting per-seed values. When a stress setting pools multiple test conditions, such as two extrapolated Reynolds numbers, the per-seed metric is computed over the union of the corresponding test trajectories. Condition-wise results may additionally be reported in the appendix.
| Dataset | Method | RelL2 ↓ | RelL2 ↓ | RelL2 ↓ | Div ↓ | BC |
| P-NS | FNO | 0.0044 ± 0.0000 | 5.0484 ± 8.6469 | N.F. | N.F. | — |
| P-NS | DivReg | 0.0041 ± 0.0000 | 0.0290 ± 0.0015 | 0.0557 ± 0.0080 | −2.0171 ± 0.0089 | — |
| P-NS | PINO | 0.0047 ± 0.0000 | 0.0295 ± 0.0003 | 0.0477 ± 0.0011 | −1.9620 ± 0.0067 | — |
| P-NS | ClawNO | 0.0038 ± 0.0285 | 0.0266 ± 0.0015 | 0.9270 ± 0.3536 | −5.3557 ± 0.0179 | — |
| P-NS | FinalProj | 0.0037 ± 0.0001 | 0.0281 ± 0.0006 | 0.0503 ± 0.0017 | −5.4981 ± 0.0001 | — |
| P-NS | PCM-NO | 0.0033 ± 0.0001 | 0.0264 ± 0.0002 | 0.0429 ± 0.0005 | −5.4978 ± 0.0002 | — |
| LC | FNO | 0.0042 ± 0.0005 | 0.0049 ± 0.0001 | 0.0058 ± 0.0018 | −2.1344 ± 0.0524 | −3.8770 ± 0.0219 |
| LC | DivReg | 0.0167 ± 0.0030 | 0.0063 ± 0.0017 | 0.0050 ± 0.0001 | −3.0271 ± 0.0128 | −3.9249 ± 0.0160 |
| LC | PINO | 0.0433 ± 0.0042 | 0.2601 ± 0.0142 | 0.3305 ± 0.0127 | −1.6027 ± 0.0359 | −1.4254 ± 0.0441 |
| LC | FinalProj | 0.0121 ± 0.0062 | 0.0123 ± 0.0023 | 0.0162 ± 0.0019 | −4.0080 ± 0.1363 | −6.0000 ± 0.0000 |
| LC | PCM-NO | 0.0037 ± 0.0001 | 0.0043 ± 0.0002 | 0.0039 ± 0.0001 | −6.0625 ± 0.2304 | −6.0000 ± 0.0000 |
Table 1. In-distribution rollout accuracy and feasibility. Div uses spectral divergence on P-NS and interior MAC control volumes on LC; BC is reported only for LC. N.F. denotes a non-finite rollout before the stated horizon.
Unless otherwise stated, denotes the grid-weighted discrete Euclidean norm over the relevant state, constraint, or profile degrees of freedom. We use the fixed numerical stabilizer throughout.
Rollout accuracy. At rollout horizon , the relative state error is
Here, and denote the reference and predicted states, respectively, for test trajectory . Table 1 reports , while Table 2 reports . Figure 2 plots the same quantity as a function of the rollout horizon.
Constraint feasibility. Feasibility is evaluated using the discrete operators that define the corresponding affine constraint set. The incompressibility metric at rollout horizon is
where is the dataset-specific discrete divergence operator. For LC, the prescribed-boundary residual is
where extracts the constrained boundary degrees of freedom and contains the prescribed wall velocities for trajectory . For P-NS, is omitted because periodicity is built into the discretization and the explicit constraint is .
Equations (25) and (26) average the trajectory-wise logarithmic residuals, rather than taking the logarithm of an averaged residual. Lower and more negative values therefore indicate stronger constraint satisfaction. The logarithmic representation is used because FinalProj and PCM-NO can approach the numerical projection or retraction floor.
P-NS energy error. For P-NS, the flow-specific diagnostic is the relative kinetic-energy error. At rollout horizon , it is defined as
where the discrete kinetic energy is
LC centerline profile error. For LC, the flow-specific diagnostic compares the two standard centerline velocity profiles. Let
denote the vertical centerline samples used for the horizontal velocity component , and let
denote the horizontal centerline samples used for the vertical velocity component . The profile error at rollout horizon is
Figure 2 plots for P-NS and for LC across rollout horizons. Table 2 reports for P-NS settings and for LC settings. Table 4 additionally reports for the LC component variants.
Additional Experiment Results
External Structure-Preserving Baseline
ClawNО is included in the P-NS block of Table 1 as an external structure-preserving baseline [25]. We use its FFT-based clawFNO realization, which predicts a scalar potential and constructs
through fixed spectral differentiation. Hence up to numerical differentiation error. ClawFNO therefore enforces incompressibility through the output parameterization, whereas FinalProj projects a directly predicted state and PCM-NO projects a candidate residual update before advancement and retraction.
| Dataset | Batch | Method | Step ms/traj. ↓ | 50-step ms/traj. ↓ | Speedup vs. solver ↑ | Cost vs. FNO ↓ |
| P-NS | 1 | Reference solver | 26.70 ± 0.90 | 1297.33 ± 7.78 | 1.0× | – |
| P-NS | 1 | FNO | 2.62 ± 0.29 | 124.37 ± 0.86 | 10.4× | 1.00× |
| P-NS | 1 | FinalProj | 3.13 ± 0.29 | 147.71 ± 0.73 | 8.8× | 1.19× |
| P-NS | 1 | PCM-NO | 3.63 ± 0.59 | 170.21 ± 1.07 | 7.6× | 1.37× |
| P-NS | 8 | Reference solver | 3.55 ± 0.12 | 174.26 ± 2.21 | 1.0× | – |
| P-NS | 8 | FNO | 0.35 ± 0.04 | 15.75 ± 0.17 | 11.1× | 1.00× |
| P-NS | 8 | FinalProj | 0.39 ± 0.03 | 18.73 ± 0.24 | 9.3× | 1.19× |
| P-NS | 8 | PCM-NO | 0.45 ± 0.03 | 21.53 ± 0.14 | 8.1× | 1.37× |
| LC | 1 | Reference solver | 639.02 ± 13.57 | 31806.49 ± 68.21 | 1.0× | – |
| LC | 1 | FNO | 2.69 ± 0.49 | 120.11 ± 0.56 | 264.8× | 1.00× |
| LC | 1 | FinalProj | 53.05 ± 6.06 | 2512.74 ± 12.68 | 12.7× | 20.92× |
| LC | 1 | PCM-NO | 97.80 ± 0.92 | 4988.19 ± 81.61 | 6.4× | 41.53× |
| LC | 8 | Reference solver | 80.27 ± 0.64 | 4008.23 ± 3.08 | 1.0× | – |
| LC | 8 | FNO | 0.33 ± 0.06 | 16.16 ± 0.67 | 248.0× | 1.00× |
| LC | 8 | FinalProj | 6.40 ± 0.09 | 316.86 ± 0.26 | 12.6× | 19.61× |
| LC | 8 | PCM-NO | 12.54 ± 0.22 | 618.00 ± 4.39 | 6.5× | 38.24× |
Table 7. End-to-end computational cost on an NVIDIA A100-SXM4-40GB GPU. One learned transition and one reference-solver call advance the state over the same saved physical interval. P-NS uses ten pseudo-spectral RK4 micro-steps per saved step, while LC uses ten MAC micro-steps with the data-generation constraint-solve budget. Timings include all inference-time projection and retraction operations and exclude checkpoint loading and file I/O. Speedup is computed against the corresponding reference solver at the same batch size. Values are mean ± standard deviation over repeated synchronized timing runs.
The matched comparison uses the same P-NS splits, resolution, Reynolds-number range, training horizon , and seeds . ClawNO is trained from scratch with matched backbone capacity, while FinalProj and PCM-NO reuse their validation-selected checkpoints. All methods use the same 50-step evaluator; runtime is measured at batch size one. We restrict ClawNO to P-NS because its periodic potential representation does not directly encode the stacked LC constraint .
Table 8 supplements the main IID table with the P-NS kinetic-energy diagnostic and inference cost.
| Method | ↓ | ↓ | ↓ | Time/step (ms) ↓ |
| ClawNO | 0.9270 ± 0.3536 | −5.3557 ± 0.0179 | 1.1820 ± 0.8115 | 4.15 ± 0.07 |
| FinalProj | 0.0503 ± 0.0017 | −5.4981 ± 0.0001 | 0.0800 ± 0.0004 | 3.13 ± 0.29 |
| PCM-NO | 0.0429 ± 0.0005 | −5.4978 ± 0.0002 | 0.0086 ± 0.0058 | 3.63 ± 0.59 |
Table 8. P-NS comparison with ClawNO. ClawNO parameterizes the velocity field through a learned potential and fixed differentiation, while FinalProj and PCM-NO retain the velocity-space backbone used in the main experiments.
Since all three methods impose P-NS incompressibility structurally, mainly verifies feasibility. The discriminating quantities are , , and inference cost.
Constraint-solver diagnostics
P-NS uses the closed-form spectral Helmholtz projection implemented with FFTs, so its feasibility floor is determined by floating-point accuracy rather than an iterative stopping rule. LC uses a MAC-grid normal-equation projection with boundary lifting. We evaluate its numerical behavior separately because the linear-solve budget controls a cost–feasibility tradeoff that is distinct from neural-backbone hyperparameters.
Ground-truth consistency. Projecting held-out LC ground-truth states at rollout step 50 changes them by approximately in relative norm. The prescribed residual remains at on the reported scale. Thus the stored solver trajectories already lie close to the same discrete feasible set used by the constraint layer, and the evaluation does not compare predictions against a materially different post-processed target. Increasing the CG budget from 20 to 80 iterations reduces the interior Div residual from to , while 160 iterations yields only a marginal further change to . BC remains at throughout. At tolerance , runtime increases from 11.34 to 38.19 and 74.97 ms per batch for 20, 80, and 160 iterations, respectively. The paper-scale LC evaluation uses 120 iterations, which lies in the saturation regime indicated by the 80- and 160-iteration diagnostics. These results justify using an intermediate-to-high solve budget rather than the highest-cost setting. The observed saturation is consistent with Appendix § 9.§ 3, where feasibility is controlled by the achieved linear-system residual rather than by a nominal stopping tolerance alone.
Figure 5. Numerical sensitivity of the LC projection/retraction solve on step-50 ground-truth states. At a fixed CG iteration budget, tolerances in produce the same projected states, showing that the iteration cap is the effective control in this diagnostic.
End-to-End Computational Cost
Table 7 compares the complete learned transitions with the corresponding reference solvers on the same GPU and over the same saved physical interval. The measurements include all projection and retraction operations used during inference and exclude checkpoint loading and file I/O. DivReg and PINO share the FNO inference graph because their physical terms are used only during training.
On P-NS, the FFT-based constraint layers introduce moderate overhead: PCM-NO costs approximately 1.37× the FNO rollout while retaining a 7.6× speedup over the reference solver at batch size one. The result is similar at batch size eight. On LC, the iterative constraint solves dominate the learned-model cost. FinalProj, which applies one state retraction per step, costs approximately 21× FNO, while PCM-NO, which applies both tangent projection and state retraction, costs approximately 42× FNO. Nevertheless, the complete LC transition remains approximately 6.4× faster than the reference MAC solver.
These measurements expose a discretization-dependent cost–feasibility trade-off. Spectral constraints can be enforced with relatively small overhead, whereas boundary-aware iterative solves are substantially more expensive. The reported speedups characterize the implementations and hardware used in this study and should not be interpreted as universal comparisons with all optimized CFD solvers. Reducing the LC cost through warm starts, preconditioning, or adaptive solve budgets is an important direction for future work.
References
- [1]Douglas N. Arnold, Richard S. Falk, and Ragnar Winter. 2006. Finite element exterior calculus, homological techniques, and applications. Acta Numerica 15 (2006), 1–155.DOI
- [2]Johannes Brandstetter, Daniel E. Worrall, and Max Welling. 2022. Message Passing Neural PDE Solvers. In International Conference on Learning Representations.arxiv.org/abs/2202.03376
- [3]Ricardo Buitrago Ruiz, Tanya Marwah, Albert Gu, and Andrej Risteski. 2025. On the Benefits of Memory for Modeling Time-Dependent PDEs. In International Conference on Learning Representations.
- [4]Jinbao Cheng, Jianguo Huang, Haoqin Wang, and Tao Zhou. 2026. Decoupled Divergence-Free Neural Networks Basis Method for Incompressible Fluid Problems. arXiv preprint arXiv:2603.17906 (2026).arxiv.org/abs/2603.17906
- [5]Junho Choi, Teng-Yuan Chang, Namjun Kim, and Youngjoon Hong. 2025. A Data-Free Neural Operator Enabling Fast Inference of 2D and 3D Navier–Stokes Equations. arXiv preprint arXiv:2510.23936 (2025).arxiv.org/abs/2510.23936
- [6]Alexandre Joel Chorin. 1968. Numerical solution of the Navier–Stokes equations. Math. Comp. 22, 104 (1968), 745–762.DOI
- [7]Hamidreza Eivazi, Jendrik-Alexander Tröger, Stefan Wittek, Stefan Hartmann, and Andreas Rausch. 2025. EquiNO: A Physics-Informed Neural Operator for Multiscale Simulations. arXiv preprint arXiv:2504.07976 (2025).arxiv.org/abs/2504.07976
- [8]Luis Espath, Dmitry Kabanov, Jonas Kiessling, and Raúl Tempone. 2021. Statistical Learning for Fluid Flows: Sparse Fourier Divergence-Free Approximations. arXiv preprint arXiv:2107.07633 (2021).DOI
- [9]Zhiwei Fang, Sifan Wang, and Paris Perdikaris. 2023. Learning Only on Boundaries: A Physics-Informed Neural Operator for Solving Partial Differential Equations in Complex Geometries. arXiv preprint arXiv:2308.12939 (2023).arxiv.org/abs/2308.12939
- [10]Marc Finzi, Ke Alexander Wang, and Andrew Gordon Wilson. 2020. Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints. In Advances in Neural Information Processing Systems, Vol. 33.arxiv.org/abs/2010.13581
- [11]Samuel Greydanus, Misko Dzamba, and Jason Yosinski. 2019. Hamiltonian Neural Networks. In Advances in Neural Information Processing Systems.arxiv.org/abs/1906.01563
- [12]Jean-Luc Guermond, Peter Minev, and Jie Shen. 2006. An overview of projection methods for incompressible flows. Computer Methods in Applied Mechanics and Engineering 195, 44–47 (2006), 6011–6045.DOI
- [13]Wenrui Hao, Xinliang Liu, and Yanhong Yang. 2024. Newton Informed Neural Operator for Solving Nonlinear Partial Differential Equations. In Advances in Neural Information Processing Systems.
- [14]Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng, Jian Song, and Jun Zhu. 2023. GNOT: A General Neural Operator Transformer for Operator Learning. In Proceedings of the 40th International Conference on Machine Learning.
- [15]Maximilian Herde, Bogdan Raonic, Tobias Rohner, Roger Käppeli, Roberto Molinaro, Emmanuel de Bézenac, and Siddhartha Mishra. 2024. POSEIDON: Efficient Foundation Models for PDEs. In Advances in Neural Information Processing Systems.arxiv.org/abs/2405.19101
- [16]Hamda Hmida, Hsiu-Wen Chang, and Youssef Mesri. 2026. Compositional Neural Operators for Multi-Dimensional Fluid Dynamics. In International Conference on Learning Representations.arxiv.org/abs/2605.11691
- [17]George Em Karniadakis, Ioannis G Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. 2021. Physics-informed machine learning. Nature Reviews Physics 3, 6 (2021), 422–440.DOI
- [18]Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew M. Stuart, and Anima Anandkumar. 2023. Neural Operator: Learning Maps Between Function Spaces. Journal of Machine Learning Research 24, 89 (2023), 1–97.
- [19]Zishuo Lan, Junyi Li, Lei Wang, and Jincheng Wang. 2026. FluxNet: Learning Capacity-Constrained Local Transport Operators for Conservative and Bounded PDE Surrogates. In Proceedings of the 43rd International Conference on Machine Learning.arxiv.org/abs/2602.01941
- [20]Xigui Li, Hongwei Zhang, Ruoxi Jiang, Deshu Chen, Chensen Lin, Limei Han, Yuan Qi, Xin Guo, and Yuan Cheng. 2026. Project and Generate: Divergence-Free Neural Operators for Incompressible Flows. arXiv preprint arXiv:2603.24500 (2026).arxiv.org/abs/2603.24500
- [21]Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar. 2023. Fourier Neural Operator with Learned Deformations for PDEs on General Geometries. Journal of Machine Learning Research 24, 388 (2023), 1–26.DOI
- [22]Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew M. Stuart, and Anima Anandkumar. 2021. Fourier Neural Operator for Parametric Partial Differential Equations. In International Conference on Learning Representations.
- [23]Zongyi Li, Hongkai Zheng, Nikola Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Andrew M. Stuart, Kamyar Azizzadenesheli, and Anima Anandkumar. 2022. Physics-Informed Neural Operator for Learning Partial Differential Equations. In International Conference on Learning Representations.arxiv.org/abs/2111.03794
- [24]Phillip Lippe, Bastiaan S. Veeling, Paris Perdikaris, Richard E. Turner, and Johannes Brandstetter. 2023. PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers. In Advances in Neural Information Processing Systems.arxiv.org/abs/2308.05732
- [25]Ning Liu, Yiming Fan, Xianyi Zeng, Milan Klöwer, Lu Zhang, and Yue Yu. 2024. Harnessing the Power of Neural Operators with Automatically Encoded Conservation Laws. In Proceedings of the 41st International Conference on Machine Learning.arxiv.org/abs/2312.11176
- [26]Ning Liu, Yue Yu, Huaiqian You, and Neeraj Tatikola. 2023. INO: Invariant Neural Operators for Learning Complex Physical Systems with Momentum Conservation. In Proceedings of the 26th International Conference on Artificial Intelligence and Statistics.arxiv.org/abs/2212.14365
- [27]Miguel Liu-Schiaffini, Julius Berner, Boris Bonev, Thorsten Kurth, Kamyar Azizzadenesheli, and Anima Anandkumar. 2024. Neural Operators with Localized Integral and Differential Kernels. In Proceedings of the 41st International Conference on Machine Learning.
- [28]Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. 2021. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence 3, 3 (2021), 218–229.
- [29]Michael McCabe, Peter Harrington, Shashank Subramanian, and Jed Brown. 2023. Towards Stability of Autoregressive Neural Operators. Transactions on Machine Learning Research (2023).arxiv.org/abs/2306.10619
- [30]Satoshi Noguchi and Yoshinobu Kawahara. 2026. Mesh Field Theory: Port-Hamiltonian Formulation of Mesh-Based Physics. In Proceedings of the 43rd International Conference on Machine Learning.arxiv.org/abs/2605.00394
- [31]Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter W. Battaglia. 2021. Learning Mesh-Based Simulation with Graph Networks. In International Conference on Learning Representations.arxiv.org/abs/2010.03409
- [32]Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. 2019. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378 (2019), 686–707.DOI
- [33]Bogdan Raonic, Roberto Molinaro, Tim De Ryck, Tobias Rohner, Francesca Bartolucci, Rima Alairafi, Siddhartha Mishra, and Emmanuel de Bézenac. 2023. Convolutional Neural Operators for Robust and Accurate Learning of PDEs. In Advances in Neural Information Processing Systems.arxiv.org/abs/2302.01178
- [34]Alexander Richter-Powell, Yoran Lipman, and Ricky T. Q. Chen. 2022. Neural Conservation Laws: A Divergence-Free Perspective. In Advances in Neural Information Processing Systems.arxiv.org/abs/2210.01741
- [35]Alexander Rudikov, Vladimir Fanaskov, Ekaterina Muravleva, Yuri M. Laevsky, and Ivan Oseledets. 2024. Neural Operators Meet Conjugate Gradients: The FCG-NO Method for Efficient PDE Solving. In Proceedings of the 41st International Conference on Machine Learning.arxiv.org/abs/2402.05598
- [36]Mansi Sarkarvadia, Kareem Hegazy, Amin Toutounferush, Kyle Chard, Yaoqing Yang, Ian Foster, and Michael W. Mahoney. 2026. The False Promise of Zero-Shot Super-Resolution in Machine-Learned Operators. In The Fourteenth International Conference on Learning Representations.
- [37]Makoto Takamoto, Timothy Praditia, Raphael Leiteritz, Daniel MacKinlay, Francesco Alesiani, Dirk Pflüger, and Mathias Neidert. 2022. PDEBench: An Extensive Benchmark for Scientific Machine Learning. In Advances in Neural Information Processing Systems.arxiv.org/abs/2210.07182
- [38]Roger Temam. 1969. Sur l’approximation de la solution des équations de Navier-Stokes par la méthode des pas fractionnaires (II). Archive for Rational Mechanics and Analysis 33 (1969), 377–385.
- [39]Alasdair Tran, Alexander Mathews, Lexing Xie, and Cheng Soon Ong. 2023. Factorized Fourier Neural Operators. In International Conference on Learning Representations.
- [40]Utkarsh Utkarsh, Pengfei Cai, Alan Edelman, Rafael Gómez-Bombarelli, and Christopher Rackauckas. 2025. Physics-Constrained Flow Matching: Sampling Generative Models with Hard Constraints. In Advances in Neural Information Processing Systems, Vol. 38.arxiv.org/abs/2506.04171
- [41]Haixin Wang, Jiaxin Li, Anubhav Dwivedi, Kentaro Hara, and Tailin Wu. 2024. BENO: Boundary-Embedded Neural Operators for Elliptic PDEs. In International Conference on Learning Representations.arxiv.org/abs/2401.09323
- [42]Sifan Wang, Hanwen Wang, and Paris Perdikaris. 2021. Learning the Solution Operator of Parametric Partial Differential Equations with Physics-Informed DeepONets. Science Advances 7, 40 (2021), eabi8605.



