WorksComputer Science
TangentPDE: Learning Local Operator Responses with Sparse Finite-Difference Supervision
Abstract
Neural PDE operators are often optimized through state reconstruction, which largely constrains operator values at observed inputs but does not fully characterize their sensitivity to local perturbations. A more faithful approximation of operator behavior requires not only accurate state predictions, but also correct variations along local perturbation directions. We present TangentPDE, which uses finite differences to approximate directional derivatives and constructs task-relevant finite-step responses from the state outputs of a reference solver at sparse paired inputs. Its objective jointly constrains state fidelity, response-direction alignment, and response-magnitude calibration, thereby learning both operator values and their local variations. Because the response labels are constructed solely from state outputs at paired inputs, the method requires neither explicit derivative labels, linearized or adjoint PDE solves, nor full Jacobians, and remains applicable to nonlinear solvers without derivative interfaces. Building on this, we additionally propose a selectively executed adaptation mechanism to strengthen local operator representations under finite-difference supervision. Budget-matched and correspondence-perturbed controls isolate the effect of relational supervision from improvements attributable to additional state observations or increased data volume. Designed as a plug-and-play training framework, TangentPDE integrates with diverse neural-operator backbones. Across four PDE systems and 20 task-backbone settings, TangentPDE lowers the three-seed mean state error in 18 settings and the corresponding Rollout@20 error in 18, while intrinsic finite-step response gains are strongest on DiffReact and remain task- and backbone-dependent elsewhere.
Introduction
Neural operators learn mappings between function spaces and provide fast data-driven surrogates for partial differential equations (PDEs) [12, 13] (Li et al. 2020; Lu et al. 2021; Li et al. 2021; Lu et al. 2022). Standard supervision fits a predictor to a reference state over sampled input–output function pairs. State reconstruction remains indispensable because it controls operator values at the observed inputs and, for any fixed nonzero perturbation step, vanishing errors at both endpoints imply vanishing finite-step response error. Our focus is the finite-accuracy regime, where independent endpoint reconstruction leaves the cross-endpoint error increment implicit.
The central observation is simple but consequential. Let and let be a valid perturbation of . Then
where and are the reference and learned finite-step responses. Endpoint state errors therefore control a coarse upper bound on response error, but their aggregate magnitudes do not determine the realized response mismatch. Models with comparable state error can retain different cross-endpoint error increments and hence different local operator behavior. This distinction matters whenever a learned operator is used to compare nearby inputs, estimate sensitivity, support local design or control, or advance a system repeatedly.
Derivative- and Sobolev-informed studies show that local variation supervision can improve not only derivative fidelity but also function or state approximation [3, 21, 2, 23, 22]. This motivates an intermediate supervision regime: nearby admissible states can be queried from the reference system, while Jacobians, tangent solves, adjoints, or a differentiable solver interface may be unavailable or unnecessary at the task-defined perturbation scale. TangentPDE addresses this regime by converting correctly paired endpoint states into explicit finite-step relational supervision. The required reference information remains state-only, while ordinary backpropagation is used to train the neural operator.
We introduce TangentPDE-Core, which combines matched endpoint reconstruction with primary response-direction alignment and weaker response-magnitude calibration under sparse paired supervision. The novelty is not finite differencing itself, but the identification and direct optimization of the cross-endpoint error increment that remains implicit under independent state fitting. Core operates at the supervision level and preserves the backbone’s state-output semantics, making it architecture-orthogonal to heterogeneous neural-operator designs. As an optional extension, the compact Layer-Selective Tangent Head (LSTH) maps finite differences of selected hidden representations to an auxiliary response residual. Its correction never enters the state outputs, so the auxiliary estimate remains distinct from the intrinsic state-output response.
Our contributions are:
an exact error-increment characterization identifying cross-endpoint state-error variation as the quantity governing finite-step response mismatch;
TangentPDE-Core, a state-query-only objective that directly supervises intrinsic finite-step responses through direction alignment and weak magnitude calibration;
an architecture-orthogonal evaluation framework with matched controls for endpoint observations, query budget, correspondence, and sparse pair coverage; and
LSTH, a selectively executed auxiliary response learner that reuses selected hidden finite differences without changing standard single-input state inference.
Related Work
Neural PDE Operator Learning
DeepONet, Fourier, graph, message-passing, latent, recurrent, state-space, and attention-based neural operators mainly differ in representation, geometry handling, and information exchange [1, 5, 29, 24, 28, 26] (Li et al. 2020; Lu et al. 2021; Li et al. 2021; Li et al. 2023; Wu et al. 2024; Zheng et al. 2024; Ye, Zhang, and Wang 2025). Recent AAAI work further studies efficient kernels, locality, linearized physics-attention, and learning from partial observations [11, 33, 6] (Hu et al. 2026). TangentPDE is architecture-orthogonal: rather than redesigning the state predictor, it changes the supervision relation imposed on correctly paired operator evaluations while preserving the backbone’s output semantics.
Derivative-, Sensitivity-, and Sobolev-Informed Learning
Sobolev and derivative-informed learning supervise local variation beyond function values, including target gradients, full or reduced Jacobians, Fréchet derivatives, Jacobian–vector products, and sensitivities obtained from tangent or adjoint systems [3, 21, 2, 23, 22]. These methods directly constrain infinitesimal behavior when reliable derivative sources are available. TangentPDE occupies a distinct access regime: it learns task-scale finite-step responses from paired state queries rather than approximating a complete infinitesimal derivative operator. This replaces specialized derivative-interface requirements with ordinary state evaluations at admissible nearby inputs.
Finite-Difference, Augmentation, and Consistency Supervision
Finite differences also appear in data augmentation, numerical derivative approximation, and consistency objectives, but their supervision semantics differ from TangentPDE. Ordinary two-point augmentation treats the anchor and perturbed input as independent state examples and primarily enlarges input coverage. Consistency regularization commonly encourages invariance, equivariance, or agreement between related predictions. TangentPDE instead treats the anchor–perturbation correspondence itself as a supervised object and uses the reference endpoint difference as a generally nonzero target with both direction and scale. Although the target contains no reference information beyond the endpoint states, its coupled objective directly constrains the cross-endpoint error increment and is not reducible to independent endpoint reconstruction.
Physics-Informed Correction and Auxiliary Adaptation
Physics-informed training and rollout-stabilization methods impose governing-equation residuals, multi-step objectives, generative refinement, or inference-time numerical corrections [18, 4, 9] (Li et al. 2024). TangentPDE instead supervises the finite-step behavior exhibited by the reference operator and evaluates response fidelity separately from rollout stability. LSTH is related to auxiliary multi-task learning and compact adaptation modules [7], but its adapters consume pairwise hidden finite differences rather than a single hidden state and produce a dedicated auxiliary response estimate. Standard state inference remains head-free, while intrinsic and auxiliary responses are evaluated as separate objects.
Method
Problem Setup and Finite-Step Responses
Let denote the reference and learned operators. Each benchmark samples an admissible input and a perturbation protocol , producing
Ordinary and paired observations can be written compactly as
For each pair, and . The reference finite-step response and the intrinsic response induced by the state predictor are
These finite-step secants are the actual optimization objects. They approach directional derivatives only under the corresponding differentiability and small-step conditions. We assume neither a normalized direction nor recovery of a complete Jacobian, and all locality claims refer to the benchmark-defined perturbation neighborhood. The inner product and norm below are those used by the discrete response implementation.
Figure 1. Overview of TangentPDE. Core supervises the intrinsic state-output response ; optional LSTH produces a separate auxiliary estimate from selected hidden finite differences.
Finite-Accuracy Error Increment
With ,
The triangle inequality gives
Cross-endpoint error-increment characterization. (4) exactly characterizes finite-step response error through the cross-endpoint increment of state-prediction errors. (5) shows that endpoint accuracy limits the possible response error but does not determine the realized mismatch. In particular, endpoint-error pairs and have identical marginal norms but induce response errors and , respectively. TangentPDE-Core directly optimizes this relational quantity, which remains implicit under independent endpoint reconstruction.
Sparse Paired Observations
Pairs are constructed after data splitting. Their anchor labels are reused, perturbed endpoint labels are queried only for selected training anchors, and unpaired samples may continue to provide ordinary state supervision. Pair coverage is
Actual additional solver calls are reported from the pair-generation pipeline. MSE-2N-random tests whether additional state volume explains the gain; Two-point MSE matches the exact endpoint labels and query budget; Pair-shuffled Tangent tests whether preserving the implemented correspondence adds value beyond those matched observations.
TangentPDE-Core
For a paired sample, matched endpoint reconstruction is
Two-point MSE uses the same paired endpoints and reference queries but optimizes only their marginal state errors. TangentPDE-Core additionally constrains the relation between the two predictions through
Direction alignment is the primary structural term; the lower-weight magnitude term calibrates responses that are directionally correct but systematically too weak or too strong. The complete objective is
The hierarchy specifies modeling priority rather than a guaranteed ordering of gradient norms. The state objective controls endpoint fidelity, while the response terms control the cross-endpoint variation of prediction errors. Direction provides the primary geometric signal, and magnitude acts as secondary scale calibration. Core and LSTH use a common valid-response convention for their response objectives; its frozen implementation is specified with the experimental protocol.
Optional Layer-Selective Tangent Head
Let denote the state predictor after LSTH training. Its intrinsic response remains . For selected stages with pairwise aligned hidden representations ,
Each representation-specific adapter maps a hidden finite-step response to the output-response space. The final projections are zero-initialized and . The correction is not added to either state output, so is an auxiliary paired-response estimate rather than an intrinsic state-output secant.
LSTH confines response-specific parameters and computation to representation-specific depthwise/pointwise adapters and small output projections at selected stages. The adapters are bypassed during standard state and intrinsic-response inference and are activated only for auxiliary-response estimation.
In the four-block FNO instantiation, under one-based indexing, with
The selected features share a spatial grid, and maps hidden channels to response channels. The same interface extends to token-, sequence-, graph-, and multi-resolution backbones through correspondence-compatible adapters.
LSTH reuses the Core response losses and regularizes the correction through
The regularizer limits correction magnitude relative to the intrinsic response. The auxiliary estimate remains a dedicated paired-response output rather than a correction to either state prediction.
State-Dominant Optimization and Deployment
Stage I uses the first 20% of LSTH continuation updates to train only the adapters. Let denote . During the remaining 80%, early stages stay frozen while selected intermediate/late stages and the state-output head receive
with and . The two coefficients ramp linearly from zero during the first 20% of Stage II updates. Adapter, selected shared stages, and the output head use learning-rate multipliers 1.0, 0.02, and 0.05, respectively. This state-dominant parameter routing limits interference between auxiliary response learning and the primary state objective.
Standard single-input state inference requires one backbone forward, while intrinsic paired-response estimation uses two shared forwards and output differencing. Forward-path verification records zero adapter calls in both modes; auxiliary estimation alone evaluates the selected adapters. Offline pairing reuses anchor labels and queries perturbed endpoint states without differentiating through the reference solver.
Experiments
Experimental Setup
We evaluate four representative PDE regimes. DiffReact captures nonlinear reaction–diffusion dynamics with spatial smoothing and coupled local reactions; Kolmogorov flow represents forced nonlinear fluid dynamics with multi-scale interactions and long-horizon instability; Burgers isolates nonlinear conservation-law behavior with steep gradients and shock-like transport; and Advection isolates phase propagation and transport alignment. Together, these tasks span smoothing, multi-scale, sharp-gradient, and phase-sensitive regimes emphasized by modern scientific-ML benchmarks and operator-learning analyses [25, 10, 27].
The breadth study evaluates FNO, RNO (Ye, Zhang, and Wang 2025), CViT [28], MambaNO [32], and LaMO [26] under matched task-specific protocols; Table 3 provides additional budget-matched controls for FNO. Core and Two-point MSE share the same pairs, splits, seeds, and endpoint states. MSE-2N matches the state count, while Pair-shuffled retains endpoint supervision but breaks correspondence. We report state relative , response cosine, response relative , magnitude error, and Rollout@20. Unless noted, entries are three-seed means. Per-seed results, sample variances, standard deviations, 95% confidence intervals, effect sizes, paired tests, bootstrap analyses, and multiple-comparison-adjusted -values are reported in the appendix.
| Metric | MSE-1× | MSE-2N | Two-point | Core |
| DiffReact | ||||
| State ↓ | 0.009452 | 0.009445 | 0.009223 | 0.005908 |
| Resp. cos. ↑ | 0.2935 | 0.2944 | 0.2960 | 0.9308 |
| Resp. rel. ↓ | 0.9773 | 0.9768 | 0.9733 | 0.3758 |
| Mag. err. ↓ | 1.104 | 1.102 | 1.050 | 0.0997 |
| Roll.@20 ↓ | 0.004679 | 0.004687 | 0.004440 | 0.003573 |
| Kolmogorov | ||||
| State ↓ | 0.2810 | 0.2810 | 0.2810 | 0.2846 |
| Resp. cos. ↑ | 0.7204 | 0.7204 | 0.7205 | 0.7528 |
| Resp. rel. ↓ | 4.053 | 4.052 | 4.052 | 3.918 |
| Mag. err. ↓ | 1.108 | 1.108 | 1.108 | 1.095 |
| Roll.@20 ↓ | 0.2960 | 0.2960 | 0.2959 | 0.2945 |
Table 3. Budget-matched FNO controls. MSE-2N matches the state count; Two-point uses the same paired endpoints and reference-query budget as Core but omits response supervision.
Perturbation protocols. Each task uses a benchmark-specific admissible perturbation protocol. Table 2 reports the frozen perturbation step, valid train/validation/test pair counts, and the fraction of all training anchors receiving paired supervision. Training and evaluation use disjoint sampled pairs; unseen-direction and altered-step evaluations are reported separately from the in-distribution results.
| Task | Step | Pairs | ||
| DiffReact | 200/50/100 | 100% | ||
| Kolmogorov | 200/50/100 | 100% | ||
| Burgers | 1638/204/206 | 16.38% | ||
| Advection | 1638/204/206 | 16.38% |
Table 2. Benchmark-specific perturbation protocols. Pairs reports valid train/validation/test pair counts, and is the fraction of all training anchors receiving paired supervision. Constructing one perturbed endpoint per pair requires 350 additional reference-state queries for DiffReact and Kolmogorov and 2,048 for Burgers and Advection.
We organize the evaluation from breadth to mechanism.
Table 1 establishes transfer across systems and backbones. Table 3 controls observation count and independent two-endpoint fitting, while Table 4 separates direct response regression, direction–magnitude decomposition, and pair correspondence. Finally, Table 5 evaluates LSTH across shared-state, intrinsic-response, and auxiliary-response metrics. The control tables use their respective matched protocols, and their absolute values are interpreted within each table rather than directly across tables.
| Setting | State ↓ | Resp. cos. ↑ | Roll.@20 ↓ |
| Objective composition: DiffReact | |||
| State only | 0.008536 | 0.5360 | 0.0059 |
| State + Direct Resp. MSE | 0.008095 | 0.5953 | 0.0050 |
| State + Dir. | 0.007107 | 0.9640 | 0.0044 |
| State + Mag. | 0.008715 | 0.4311 | 0.0065 |
| Full Core | 0.006316 | 0.9630 | 0.0044 |
| Objective composition: Kolmogorov | |||
| State only | 0.223495 | 0.6775 | 0.3959 |
| State + Direct Resp. MSE | 0.179553 | 0.5073 | 0.3350 |
| State + Dir. | 0.179643 | 0.5089 | 0.3351 |
| State + Mag. | 0.179648 | 0.5077 | 0.3351 |
| Full Core | 0.179683 | 0.5090 | 0.3351 |
| Pair correspondence: DiffReact | |||
| State baseline | 0.008267 | 0.5349 | 0.0057 |
| Pair-shuffled | 0.006797 | 0.9297 | 0.0050 |
| True Core | 0.006316 | 0.9630 | 0.0044 |
| Pair correspondence: Kolmogorov | |||
| State baseline | 0.222942 | 0.6783 | 0.3936 |
| Pair-shuffled | 0.179625 | 0.5061 | 0.3350 |
| True Core | 0.179574 | 0.5068 | 0.3350 |
Table 4. RNO mechanism controls. Objective rows isolate response regression, direction, and magnitude terms; pair rows break or retain endpoint correspondence.
| Variant | State ↓ | Intr. resp. ↑ | Aux. resp. ↑ | Roll. ↓ |
| DiffReact | ||||
| Static | 0.0059 | 0.9433 | – | 0.0035 |
| State FT | 0.0032 | 0.9410 | – | 0.0018 |
| Weak FT | 0.0037 | 0.9440 | – | 0.0021 |
| LSTH | _0.0036_ | 0.9431 | 0.9779 | _0.0019_ |
| Kolmogorov | ||||
| Static | 0.2809 | 0.7572 | – | 0.2941 |
| State FT | 0.2749 | 0.7281 | – | 0.2887 |
| Weak FT | _0.2795_ | _0.7525_ | – | _0.2886_ |
| LSTH | 0.2749 | 0.7281 | 0.9071 | 0.2885 |
Table 5. Layer-selective response-aware fine-tuning. Entries are three-seed means; standard deviations and confidence intervals are reported in the appendix. Intrinsic response is obtained from the shared state predictor for every variant, while auxiliary response is defined only for LSTH.
Cross-System and Cross-Architecture Results
Table 1 compares Aug-only with Core across 20 task–backbone settings. Core lowers state error and Rollout@20 in 18 settings, showing broad but non-uniform transfer across architectures and PDE regimes. Intrinsic-response gains are strongest on DiffReact and remain heterogeneous across the other systems and backbones. The aggregate pattern is therefore not a claim that response supervision uniformly improves every measured quantity; rather, it shows that paired finite-step supervision is broadly compatible with different operator parameterizations while preserving the same state-query interface. The mixed response behavior on transport-dominated systems motivates the matched controls below, which separate endpoint coverage, pair correspondence, and explicit response geometry.
| Task | Backbone | Aug-only State ↓ | Aug-only Resp. cos. ↑ | Aug-only Roll.@20 ↓ | TangentPDE-Core State ↓ | TangentPDE-Core Resp. cos. ↑ | TangentPDE-Core Roll.@20 ↓ |
| DiffReact | FNO | 0.007529 | 0.4367 | 0.00413 | 0.005939 | 0.9358 | 0.00336 |
| DiffReact | CViT | 0.011589 | 0.1411 | 0.00690 | 0.009398 | 0.4721 | 0.00590 |
| DiffReact | LaMO | 0.007906 | 0.2455 | 0.00344 | 0.007188 | 0.4855 | 0.00348 |
| DiffReact | MambaNO | 0.012105 | 0.0722 | 0.00830 | 0.010433 | 0.9345 | 0.00740 |
| DiffReact | RNO | 0.008452 | 0.5431 | 0.00580 | 0.006316 | 0.9630 | 0.00440 |
| Kolmogorov | FNO | 0.237184 | 0.7392 | 0.22747 | 0.237143 | 0.7798 | 0.22030 |
| Kolmogorov | CViT | 0.414042 | 0.3075 | 0.43090 | 0.353682 | 0.3192 | 0.26900 |
| Kolmogorov | LaMO | 0.213051 | 0.4187 | 0.38450 | 0.200136 | 0.4000 | 0.36410 |
| Kolmogorov | MambaNO | 0.488124 | 0.6189 | 0.80600 | 0.483679 | 0.6493 | 0.79490 |
| Kolmogorov | RNO | 0.223250 | 0.6769 | 0.39700 | 0.179350 | 0.5074 | 0.33470 |
| Burgers | FNO | 0.156602 | 0.3901 | 0.15502 | 0.157755 | 0.4533 | 0.15108 |
| Burgers | CViT | 0.293270 | 0.2084 | 0.25000 | 0.265638 | 0.2191 | 0.24610 |
| Burgers | LaMO | 0.287690 | 0.2064 | 0.25420 | 0.286906 | 0.2056 | 0.25270 |
| Burgers | MambaNO | 0.683363 | 0.1263 | 0.72760 | 0.635693 | 0.1077 | 0.68070 |
| Burgers | RNO | 0.321615 | 0.1989 | 0.28290 | 0.289794 | 0.2288 | 0.25390 |
| Advection | FNO | 0.129336 | 0.0899 | 0.03090 | 0.135071 | 0.0881 | 0.06828 |
| Advection | CViT | 0.546122 | 0.0471 | 0.06160 | 0.483750 | 0.0454 | 0.05960 |
| Advection | LaMO | 0.557364 | 0.0492 | 0.09400 | 0.555104 | 0.0475 | 0.07730 |
| Advection | MambaNO | 0.787117 | 0.0455 | 0.12610 | 0.775974 | 0.0403 | 0.09050 |
| Advection | RNO | 0.670729 | 0.0135 | 0.65600 | 0.636664 | 0.0184 | 0.60530 |
Table 1. Cross-system and cross-architecture results under the matched evaluation protocol. State is relative error, response is intrinsic finite-step response cosine, and Roll.@20 is the 20-step rollout error. Bold marks the better value within each matched Aug-only/Core pair.
Same-data and budget controls. Table 3 separates gains from additional state observations, matched endpoint fitting, and explicit finite-step response supervision. On DiffReact, MSE-2N and Two-point provide only limited improvements over the state baseline, whereas Core substantially improves all five reported metrics. This establishes a clear contribution from the relational objective beyond state count and paired endpoint reconstruction. On Kolmogorov, the three state-only or endpoint-only controls attain nearly identical one-step state errors. Core slightly increases state error but improves response cosine, response relative , magnitude calibration, and Rollout@20, revealing a task-dependent state–response trade-off. The comparison also distinguishes the role of correspondence from the role of raw endpoint coverage: using the same two endpoint states without supervising their finite-step relation is not sufficient to reproduce the Core response gains. We therefore interpret the budget controls as a conservative check that the observed improvements are tied to relational supervision rather than to a larger state-fitting dataset.
Objective and correspondence analysis. Table 4 separates direct response regression, the direction–magnitude decomposition, and pair correspondence. State + Direct Resp. MSE is the undecomposed response-learning baseline and fits the full response vector with a single Euclidean objective. On DiffReact, direct response regression improves over state-only continuation but remains far below direction supervision in response cosine; Full Core gives the lowest state error while matching the best rollout. On Kolmogorov, Direct Resp. MSE gives the lowest state and rollout errors, whereas the response variants remain below the state-only response cosine. The pair controls further show that exact correspondence has clear marginal value on DiffReact but only a small incremental effect over the paired state preserving protocol on Kolmogorov.
Sparse paired supervision. Figure 2 jointly evaluates state and intrinsic-response pair efficiency on DiffReact. Two-point MSE changes only modestly with the paired-anchor budget. In contrast, TangentPDE-Core rapidly improves intrinsic response alignment and continues to reduce state error as more paired anchors are introduced. The distinct response and state trends show that the relational objective affects both local output variation and endpoint prediction, with different sensitivities to pair coverage. The response curve improves most sharply at small budgets, indicating that a limited number of paired secants can already provide a useful local direction signal. The state curve changes more gradually, which is consistent with state prediction depending on both endpoint coverage and the regularization induced by the response objective.
Figure 2. Pair efficiency on DiffReact. State relative error and intrinsic finite-step response cosine are reported as paired training anchors increase. Two-point MSE fits endpoint states independently, whereas TangentPDE-Core also supervises their finite-step relation.
Layer-selective tangent head. Table 5 separates changes in the shared state predictor from gains of the dedicated auxiliary response path. LSTH markedly improves auxiliary response alignment, while its intrinsic response remains task dependent. This confirms that the head primarily provides a specialized paired-response estimate rather than uniformly modifying the state operator’s intrinsic finite-step response.
Across the two FNO configurations, LSTH adds 8.77–26.52% parameters, while auxiliary-response evaluation adds 12.44–16.21% latency over intrinsic paired-response inference.
Figure 3. Response-field comparison for a representative paired query. Reference finite-step responses are shown against Two-point MSE and TangentPDE-Core predictions for the two output channels; cosine values are computed against the corresponding reference response field. This qualitative example complements the aggregate mechanism metrics in Table 4.
Discussion and Limitations
What TangentPDE learns. Taken together, the experiments establish TangentPDE as a finite-step response-learning framework whose relational supervision also regularizes state learning. Core improves state fidelity and rollout broadly, while the matched controls isolate contributions beyond additional state observations and independent endpoint reconstruction. Explicit response geometry provides the clearest additional value on sensitivity-relevant systems, and the response-MSE comparison shows when the direction–magnitude decomposition improves over undecomposed vector regression.
Task-dependent response behavior. Response supervision exhibits distinct behavior across the evaluated PDE systems. DiffReact shows the strongest and most consistent intrinsic-response gains, whereas Burgers and Advection retain state or rollout improvements in many settings but show weaker or architecture-dependent response changes. On Kolmogorov, the balance among state, intrinsic response, and rollout also varies across backbones. These results motivate task- and architecture-specific validation of response objectives.
LSTH semantics and deployment. LSTH improves a dedicated auxiliary response estimate by reusing selected hidden finite differences, while the intrinsic response remains defined by the shared state predictor’s output difference. The two quantities are therefore reported separately. The adapters are bypassed during standard state and intrinsic-response inference and execute only for the auxiliary estimate. Full correction remains task dependent, and transport settings can over-correct when the auxiliary residual dominates the intrinsic response.
Scope, solver interface, and reporting. TangentPDE learns finite-step secants along sampled perturbation families rather than a complete Jacobian. The state-query-only interface broadens applicability to solvers without derivative, tangent, or adjoint APIs, but wall-clock cost relative to JVP, tangent, or adjoint data generation remains solver dependent. When anchor labels are reused, each new pair typically requires one additional perturbed-state solve; the exact call count and measured runtime are reported from the pair-generation manifests. The appendix provides per-seed values, variances, confidence intervals, effect sizes, corrected significance tests, response masks, perturbation protocols, hyperparameters, and solver-query accounting. Claims about unseen directions and altered step sizes are restricted to the evaluated perturbation families.
Conclusion
TangentPDE identifies the cross-endpoint error increment as the missing relational quantity in finite-accuracy state fitting and turns paired state queries into direct finite-step response supervision. TangentPDE-Core preserves the state-prediction interface while improving state fidelity and rollout across diverse PDE regimes and backbones, and controlled comparisons isolate the value of finite-step geometry beyond additional endpoint observations. The direction–magnitude decomposition separates geometric alignment from scale calibration, while LSTH extends the framework with a dedicated auxiliary response learner whose output remains distinct from the intrinsic state-output response. Together, these results establish paired finite-step response learning as a practical supervision regime between state-only fitting and explicit derivative-informed training.
Reproducibility and Evidence Scope
This supplement distinguishes three evidence families that serve different purposes. First, the submitted task-specific results reproduce the point estimates used in the main paper. Second, independent uniform-protocol reruns test whether the qualitative behavior persists under one common training schedule, but they are not used to reconstruct uncertainty around the submitted point estimates. Third, matched controls and diagnostic studies isolate specific competing explanations or probe limited robustness properties. These evidence families are reported separately and should not be treated as interchangeable estimates of the same quantity.
All reported results use three random seeds, 1234, 2025, and 3407, unless stated otherwise. The controlled FNO and RNO experiments use Adam for 50 epochs with batch size 4 and an initial learning rate of . Model selection minimizes validation state relative error; test metrics are computed once from the selected checkpoint. TangentPDE-Core uses , , and , matching the frozen main-text objective. These controlled runs use zero weight decay, full-precision training, no learning-rate scheduler, and no gradient clipping. Loss warmup is disabled for Core and its matched controls. Any independently rerun protocol that uses a different continuation schedule is identified explicitly in its own subsection.
The breadth study compares FNO, CViT, LaMO, MambaNO, and RNO using the same task split and evaluation interface for Aug-only and TangentPDE-Core. Within each task–backbone pair, the two methods use the same state-predictor architecture and differ only in the training objective. Core and Two-point MSE receive identical paired endpoints. MSE-2N matches the number of state observations, while the pair-shuffled control retains the endpoint-state multiset and optimization budget but disrupts the declared endpoint correspondence.
Unless a table caption states otherwise, boldface identifies the better point estimate within the declared comparison and does not by itself imply statistical significance. A dash denotes an undefined or inapplicable quantity rather than a numerical zero.
Tables A1 and A2 give the architecture and optimization details for the uniform-protocol breadth study. The task-dependent output dimension is kept explicit because each model predicts the complete trajectory rather than a single shared-size output tensor.
Datasets and Perturbation Protocols
Pairs are constructed only after the anchor split, so training, validation, and test anchors remain disjoint before any perturbation is generated. This ordering prevents a perturbed endpoint derived from one split from being used as an anchor or label in another split. For each valid anchor , a perturbation protocol defines an admissible perturbed endpoint . Following the frozen main-text definition, when no task-specific transform is required. The reference solver is queried at while the existing label at is reused. Thus, one accepted pair adds exactly one perturbed-endpoint query rather than two new solver calls. Table A18 separates training-pair coverage from coverage over all available anchors.
| Task | Pairs | Overall | Calls | |
| DiffReact | 200/50/100 | 100% | 100% | 350 |
| Kolmogorov | 200/50/100 | 100% | 100% | 350 |
| Burgers | 1638/204/206 | 16.38% | 20.48% | 2,048 |
| Advection | 1638/204/206 | 16.38% | 20.48% | 2,048 |
Table A18. Perturbation and reference-query accounting. is the fraction of training anchors receiving paired supervision; Overall paired is the fraction over all available anchors.
DiffReact uses coupled reaction–diffusion perturbations at , and Kolmogorov uses admissible flow perturbations at . Burgers and Advection use normalized random Fourier perturbations with relative step ; their mean effective step is . These task-specific constructions define the finite-step response families evaluated in this work; conclusions about unseen directions or altered step sizes are restricted to the dedicated studies reported later.
Evaluation Metrics and Statistical Reporting
Let denote the learned state operator and let be a paired query. The intrinsic predicted and reference responses are
The term intrinsic refers to the response induced by the shared state predictor itself; it is distinct from the auxiliary response estimate introduced later for LSTH. These quantities are finite-step secants under the benchmark-defined perturbation protocol, not estimates of a complete Jacobian.
For clarity, the frozen TangentPDE-Core training objective is reproduced here:
The quantities below are reported evaluation metrics. They should not be conflated with the training losses even when they use the same response pair.
State error is the sample-wise relative norm
Response cosine, relative response error, and magnitude error are
Metrics are computed per sample and then averaged. Near-zero reference responses are excluded from directional metrics using the response-norm threshold . Rollout@20 is the state relative error after 20 autoregressive prediction steps. Where uncertainty is shown, entries are the mean and sample standard deviation across the three seeds.
| Backbone | Optimizer | LR | Batch | Epochs | Scheduler | Weight decay | Grad. clip | Precision | Checkpoint selection |
| FNO | Adam | 4 | 50 | None | 0 | None | FP32 | Lowest validation state relative | |
| CViT | Adam | 4 | 50 | None | 0 | None | FP32 | Lowest validation state relative | |
| LaMO | Adam | 4 | 50 | None | 0 | None | FP32 | Lowest validation state relative | |
| MambaNO | Adam | 4 | 50 | None | 0 | None | FP32 | Lowest validation state relative | |
| RNO | Adam | 4 | 50 | None | 0 | None | FP32 | Lowest validation state relative |
Table A2. Training configuration for the uniform-protocol breadth study. The same optimization and checkpoint-selection settings are used for Aug-only and TangentPDE-Core. “None” means that the corresponding operation is disabled.
Seed-level standard deviations describe variation due to training randomness under a fixed configuration; they are not interpreted as independent test-sample uncertainty. The across-setting analysis reported later instead treats a complete task–backbone setting as the paired resampling and testing unit. These two uncertainty summaries answer different questions and are therefore reported separately.
Table A3 reports the realized mask statistics on the frozen test pairs. No test response is removed at the fixed threshold. The mask is applied only to response cosine; response relative and magnitude error remain defined through the fixed stabilizer and are not removed by this directional mask. State and rollout metrics do not use this mask.
| Task | Total | Valid | Excluded | Valid ratio | Min | Median | P95 |
| DiffReact | 100 | 100 | 0 | 1.0000 | 102.7 | 155.3 | 260.8 |
| Kolmogorov | 100 | 100 | 0 | 1.0000 | 9.059 | 37.39 | 150 |
| Burgers | 206 | 206 | 0 | 1.0000 | 1.253 | 3.041 | 7.492 |
| Advection | 206 | 206 | 0 | 1.0000 | 15.7 | 53.24 | 217.7 |
Table A3. Sample-level response-mask statistics on the frozen test pairs. Directional response cosine excludes samples whose reference-response norm is below the fixed threshold . Min, median, and P95 summarize the sample-level distribution of .
Cross-System Results and Evidence Roles
Submitted Task-Wise Results
Tables A4–A7 reproduce the submitted cross-system comparison by task. These tables establish the primary task–backbone observations used in the main paper. They show broad state and rollout improvements while retaining the task- and architecture-dependent response behavior emphasized in the main text. DiffReact exhibits the most consistent response gains. Kolmogorov shows broader state and rollout improvements but a mixed intrinsic-response pattern across backbones. Burgers and Advection further show that response supervision is not uniformly beneficial for every architecture and metric.
The task-wise tables should therefore be read as evidence for a backbone-dependent training effect rather than as a claim of universal intrinsic-response improvement. In particular, a better state or rollout value does not imply that the same state predictor also has a better intrinsic finite-step response.
Supplementary Magnitude Diagnostics
Tables A8–A11 retain the submitted state, intrinsic finite-step response cosine, and Rollout@20 means, and append an independent three-seed magnitude-error diagnostic. The supplementary magnitude statistics describe only the added diagnostic and are not uncertainty estimates for the submitted state, response-cosine, or rollout values.
These tables therefore separate the identity of the main-paper quantities from the identity of the added diagnostic column. Negative and mixed cells are retained as task- and backbone-dependent evidence rather than converted into a claim that every metric persists under one common schedule.
Across-Setting Paired Analysis
Figure A4 summarizes the 20 matched means reported in the main paper. The comparison unit is one complete task–backbone setting, preserving the paired Aug-only/Core comparison within that setting. The reported changes use metric-specific improvement directions: , , and . Under these definitions, positive values consistently denote improvement by TangentPDE-Core. The underlying paired checks retain the same intervals and rank-based summaries as the source table, while the figure foregrounds the descriptive cross-setting trend.
Figure A4. Across-setting paired summary over the 20 submitted task–backbone settings. Points show mean metric-specific Core-over-Aug improvement; annotations report improved-setting counts. Positive values indicate improvement, and the figure is descriptive rather than a standalone inferential display.
Figure A1. Task-wise improvement coverage across the five submitted backbones. Bars count the task–backbone pairs for which TangentPDE-Core has the better point estimate than Aug-only for State, intrinsic response cosine, and Rollout@20. The figure summarizes the submitted task-wise tables and preserves the mixed response behavior; counts do not imply statistical significance.
Figure A2. Kolmogorov intrinsic-response fields for a representative canonical FNO test query. Signed responses share one zero-centered color scale, and the two absolute-error panels share a second scale. The query-selection rule is median Core-minus-Aug-only response-cosine change over the frozen seed-1234 test set. The heatmaps show the time slice with the largest reference-response norm. Annotated response cosine , relative response error , and magnitude error are full-trajectory sample metrics computed using the same definitions as the aggregate tables; they are not three-seed means.
This analysis summarizes whether improvements are systematic across the submitted settings. It does not treat grid points, time steps, or random seeds as independent observations, and it does not convert a mixed per-backbone response pattern into a universal claim.
Controlled Ablations and Competing Explanations
Table A13 tests whether the observed gains can be explained solely by seeing more endpoint states or by using a larger reference-query budget. MSE-2N matches the number of supervised state observations, while Two-point MSE uses the same paired endpoints and solver-query budget as Core but omits response supervision. On DiffReact, Core improves all five metrics beyond these controls. On Kolmogorov, the response and rollout gains are accompanied by a small state-error trade-off.
| Metric | MSE-1× | MSE-2N | Two-point | Core |
| DiffReact | ||||
| State | 0.009452 | 0.009445 | 0.009223 | 0.005908 |
| Resp. cos. | 0.2935 | 0.2944 | 0.2960 | 0.9308 |
| Resp. rel. | 0.9773 | 0.9768 | 0.9733 | 0.3758 |
| Mag. err. | 1.104 | 1.102 | 1.050 | 0.0997 |
| Roll.@20 | 0.004679 | 0.004687 | 0.004440 | 0.003573 |
| Kolmogorov | ||||
| State | 0.2810 | 0.2810 | 0.2810 | 0.2846 |
| Resp. cos. | 0.7204 | 0.7204 | 0.7205 | 0.7528 |
| Resp. rel. | 4.053 | 4.052 | 4.052 | 3.918 |
| Mag. err. | 1.108 | 1.108 | 1.108 | 1.095 |
| Roll.@20 | 0.2960 | 0.2960 | 0.2959 | 0.2945 |
Table A13. Budget-matched FNO controls. MSE-2N matches the state count; Two-point uses the same paired endpoints and reference-query budget as Core but omits response supervision.
Table A14 separates direct response-vector regression
| Setting | State | Resp. cos. | Roll.@20 |
| Objective composition: DiffReact | |||
| State only | 0.008536 | 0.5360 | 0.0059 |
| State + Direct Resp. MSE | 0.008095 | 0.5953 | 0.0050 |
| State + Dir. | 0.007107 | 0.9640 | 0.0044 |
| State + Mag. | 0.008715 | 0.4311 | 0.0065 |
| Full Core | 0.006316 | 0.9630 | 0.0044 |
| Objective composition: Kolmogorov | |||
| State only | 0.223495 | 0.6775 | 0.3959 |
| State + Direct Resp. MSE | 0.179553 | 0.5073 | 0.3350 |
| State + Dir. | 0.179643 | 0.5089 | 0.3351 |
| State + Mag. | 0.179648 | 0.5077 | 0.3351 |
| Full Core | 0.179683 | 0.5090 | 0.3351 |
| Pair correspondence: DiffReact | |||
| State baseline | 0.008267 | 0.5349 | 0.0057 |
| Pair-shuffled | 0.006797 | 0.9297 | 0.0050 |
| True Core | 0.006316 | 0.9630 | 0.0044 |
| Pair correspondence: Kolmogorov | |||
| State baseline | 0.222942 | 0.6783 | 0.3936 |
| Pair-shuffled | 0.179625 | 0.5061 | 0.3350 |
| True Core | 0.179574 | 0.5068 | 0.3350 |
Table A14. RNO mechanism controls. Objective rows isolate direct response regression, direction, and magnitude terms; pair rows break or retain endpoint correspondence.
@page 12/23 starts_mid=1 ends_mid=1
| Task | Total | Valid | Excluded | Valid ratio | Min | Median | P95 |
| DiffReact | 100 | 100 | 0 | 1.0000 | 102.7 | 155.3 | 260.8 |
| Kolmogorov | 100 | 100 | 0 | 1.0000 | 9.059 | 37.39 | 150 |
| Burgers | 206 | 206 | 0 | 1.0000 | 1.253 | 3.041 | 7.492 |
| Advection | 206 | 206 | 0 | 1.0000 | 15.7 | 53.24 | 217.7 |
Table A3 (PDF p. 12). Sample-level response-mask statistics on the frozen test pairs. Directional response cosine excludes samples whose reference-response norm is below the fixed threshold . Min, median, and P95 summarize the sample-level distribution of .
Figure A1 (PDF p. 12). Task-wise improvement coverage across the five submitted backbones. Bars count the task–backbone pairs for which TangentPDE-Core has the better point estimate than Aug-only for State, intrinsic response cosine, and Rollout@20. The figure summarizes the submitted task-wise tables and preserves the mixed response behavior; counts do not imply statistical significance.
Figure A2 (PDF p. 12). Kolmogorov intrinsic-response fields for a representative canonical FNO test query. Signed responses share one zero-centered color scale, and the two absolute-error panels share a second scale. The query-selection rule is median Core-minus-Aug-only response-cosine change over the frozen seed-1234 test set. The heatmaps show the time slice with the largest reference-response norm. Annotated response cosine , relative response error , and magnitude error are full-trajectory sample metrics computed using the same definitions as the aggregate tables; they are not three-seed means.
TangentPDE-Core. The underlying paired checks retain the same intervals and rank-based summaries as the source table, while the figure foregrounds the descriptive cross-setting trend.
This analysis summarizes whether improvements are systematic across the submitted settings. It does not treat grid points, time steps, or random seeds as independent observations, and it does not convert a mixed per-backbone response pattern into a universal claim.
Controlled Ablations and Competing Explanations
Table A13 tests whether the observed gains can be explained solely by seeing more endpoint states or by using a larger reference-query budget. MSE-2N matches the number of supervised state observations, while Two-point MSE uses the same paired endpoints and solver-query budget as Core but omits response supervision. On DiffReact, Core improves all five metrics beyond these controls. On Kolmogorov, the response and rollout gains are accompanied by a small state-error trade-off.
Table A14 separates direct response-vector regression
Figure A3. Space–time state trajectories for representative Burgers and Advection queries from the independent uniform-protocol FNO reruns. Rows use task-specific shared state and absolute-error color scales. The annotations report sample-level full-trajectory state relative error and Rollout@20 error using the same definitions as the aggregate tables. For each task, the displayed query has Core-minus-Aug-only state improvement closest to the median over the frozen seed-1234 test set. These diagnostic rerun fields do not alter the submitted task-wise point estimates.
| Model | Method | State ↓ | Resp. ↑ | Roll. ↓ |
| CViT | Aug-only | 0.011589 | 0.1411 | 0.00690 |
| CViT | TangentPDE-Core | 0.009398 | 0.4721 | 0.00590 |
| FNO | Aug-only | 0.007529 | 0.4367 | 0.00413 |
| FNO | TangentPDE-Core | 0.005939 | 0.9358 | 0.00336 |
| LaMO | Aug-only | 0.007906 | 0.2455 | 0.00344 |
| LaMO | TangentPDE-Core | 0.007188 | 0.4855 | 0.00348 |
| MambaNO | Aug-only | 0.012105 | 0.0722 | 0.00830 |
| MambaNO | TangentPDE-Core | 0.010433 | 0.9345 | 0.00740 |
| RNO | Aug-only | 0.008452 | 0.5431 | 0.00580 |
| RNO | TangentPDE-Core | 0.006316 | 0.9630 | 0.00440 |
Table A4. Expanded DiffReact results under the matched evaluation protocol. Bold marks the better value within each Aug-only/Core pair.
| Model | Method | State ↓ | Resp. ↑ | Roll. ↓ |
| CViT | Aug-only | 0.414042 | 0.3075 | 0.43090 |
| CViT | TangentPDE-Core | 0.353682 | 0.3192 | 0.26900 |
| FNO | Aug-only | 0.237184 | 0.7392 | 0.22747 |
| FNO | TangentPDE-Core | 0.237143 | 0.7798 | 0.22030 |
| LaMO | Aug-only | 0.213051 | 0.4187 | 0.38450 |
| LaMO | TangentPDE-Core | 0.200136 | 0.4000 | 0.36410 |
| MambaNO | Aug-only | 0.488124 | 0.6189 | 0.80600 |
| MambaNO | TangentPDE-Core | 0.483679 | 0.6493 | 0.79490 |
| RNO | Aug-only | 0.223250 | 0.6769 | 0.39700 |
| RNO | TangentPDE-Core | 0.179350 | 0.5074 | 0.33470 |
Table A5. Expanded Kolmogorov results under the matched evaluation protocol. Bold marks the better value within each Aug-only/Core pair.
| Model | Method | State ↓ | Resp. ↑ | Roll. ↓ |
| CViT | Aug-only | 0.293270 | 0.2084 | 0.25000 |
| TangentPDE-Core | 0.265638 | 0.2191 | 0.24610 | |
| FNO | Aug-only | 0.156602 | 0.3901 | 0.15502 |
| TangentPDE-Core | 0.157755 | 0.4533 | 0.15108 | |
| LaMO | Aug-only | 0.287690 | 0.2064 | 0.25420 |
| TangentPDE-Core | 0.286906 | 0.2056 | 0.25270 | |
| MambaNO | Aug-only | 0.683363 | 0.1263 | 0.72760 |
| TangentPDE-Core | 0.635693 | 0.1077 | 0.68070 | |
| RNO | Aug-only | 0.321615 | 0.1989 | 0.28290 |
| TangentPDE-Core | 0.289794 | 0.2288 | 0.25390 |
Table A6. Expanded Burgers results under the matched evaluation protocol. Bold marks the better value within each Aug-only/Core pair.
| Model | Method | State ↓ | Resp. ↑ | Roll. ↓ |
| CViT | Aug-only | 0.546122 | 0.0471 | 0.06160 |
| TangentPDE-Core | 0.483750 | 0.0454 | 0.05960 | |
| FNO | Aug-only | 0.129336 | 0.0899 | 0.03090 |
| TangentPDE-Core | 0.135071 | 0.0881 | 0.06828 | |
| LaMO | Aug-only | 0.557364 | 0.0492 | 0.09400 |
| TangentPDE-Core | 0.555104 | 0.0475 | 0.07730 | |
| MambaNO | Aug-only | 0.787117 | 0.0455 | 0.12610 |
| TangentPDE-Core | 0.775974 | 0.0403 | 0.09050 | |
| RNO | Aug-only | 0.670729 | 0.0135 | 0.65600 |
| TangentPDE-Core | 0.636664 | 0.0184 | 0.60530 |
Table A7. Expanded Advection results under the matched evaluation protocol. Bold marks the better value within each Aug-only/Core pair.
from the direction–magnitude decomposition. Direct Response MSE applies one Euclidean objective to the undecomposed finite-step response vector. The objective rows therefore test whether the decomposition itself matters, while the pair rows test whether exact endpoint correspondence contributes beyond observing the same endpoint states.
The pair-shuffled control is interpreted narrowly. It preserves the available endpoint states, pair count, and optimization budget while disrupting the declared relational correspondence. Consequently, it tests whether correct pair structure contributes beyond a response-shaped regularization effect; it does not establish that every mismatch is uninformative or that the same mechanism must hold for every task and backbone.
Figure A5 visualizes the seed-level objective composition rows from the same RNO mechanism study. The plot emphasizes the relative behavior of the objective variants on DiffReact and Kolmogorov, while the source values are kept unchanged.
Figure A5. Objective-composition study on DiffReact and Kolmogorov. Points show three-seed means with sample-standard-deviation error bars for State, intrinsic response cosine, and Rollout@20.
Layer-Selective Tangent Head: Semantics and Diagnostics
Table A16 distinguishes the intrinsic response of the shared state predictor from the auxiliary response estimate produced by LSTH. Let denote the shared state predictor after LSTH continuation. The frozen main-text semantics are
| Variant | State ↓ | Intr. resp. ↑ | Aux. resp. ↑ | Roll. ↓ |
| DiffReact | ||||
| Static | 0.0059 | 0.9433 | – | 0.0035 |
| State FT | 0.0032 | 0.9410 | – | 0.0018 |
| Weak FT | 0.0037 | 0.9440 | – | 0.0021 |
| LSTH | _0.0036_ | 0.9431 | 0.9779 | _0.0019_ |
| Kolmogorov | ||||
| Static | 0.2809 | 0.7572 | – | 0.2941 |
| State FT | 0.2749 | 0.7281 | – | 0.2887 |
| Weak FT | _0.2795_ | _0.7525_ | – | _0.2886_ |
| LSTH | 0.2749 | 0.7281 | 0.9071 | 0.2885 |
Table A16. Layer-selective response-aware fine-tuning. Intrinsic response is obtained from the shared state predictor, while auxiliary response is defined only for LSTH.
The correction is not added to either state output. The auxiliary metric is defined only for LSTH and is therefore not compared with an implicitly substituted intrinsic metric. Standard state prediction and intrinsic-response inference bypass the adapters; the selected hidden-difference adapters execute only when the auxiliary response estimate is requested.
LSTH reuses the Core response losses and regularizes the auxiliary correction through
The correction regularizer limits the auxiliary residual magnitude relative to the intrinsic response during LSTH continuation. It is part of the LSTH training path, whereas the over-correction value reported later is a separate supplementary diagnostic.
To diagnose whether the optional correction dominates the intrinsic response, we additionally report the implementation-defined over-correction statistic
This sample-level quantity is averaged over valid response samples, uses the same reference-response mask as the directional response metric, and is diagnostic only. It is not used for training, checkpoint selection, or the main-paper comparisons. Larger values indicate a larger auxiliary correction relative to the intrinsic state-output response.
LSTH continuation uses a 20% adapter-only warmup. During the remaining updates, the selected shared blocks and output head receive auxiliary-gradient weights and , with learning-rate multipliers 0.02 and 0.05, respectively. The two shared-parameter gradient coefficients ramp linearly from zero during the first 20% of the second stage. The adapter learning-rate multiplier is 1.0, and the residual scale is fixed at .
The LSTH study uses separate task-specific compact FNO configurations and is not parameterized by the width-32, three-block FNO reported for the breadth study in Table A1. Both LSTH configurations use two FNO layers with hidden width 8, and the two available hidden stages are used by the auxiliary head. Table A15 reports the parameter and optional auxiliary-response evaluation overhead for these two compact FNO configurations. The latency baseline is the intrinsic paired-response path of the same trained LSTM model, not state-only inference and not the intrinsic path of a separate static model. The latency measurements characterize only the optional auxiliary paired-response path and do not change the cost of standard state-only inference.
| Backbone | Input | Output | Blocks | Width | Backbone-specific setting | History | Prediction | Params. D/K (M) |
| FNO | 3 | 32 | Fourier modes | 1 | Parallel | 0.141/0.135 | ||
| CViT | 3 | 32 | Patch ; 4 heads | 1 | Parallel | 0.043/0.039 | ||
| LaMO | 3 | 32 | Latent width 32; stride 2 | 1 | Parallel | 0.078/0.075 | ||
| MambaNO | 3 | 32 | State size 16; convolution width 4 | 1 | Parallel | 0.014/0.011 | ||
| RNO | 3 | 32 | Three ConvGRU recurrent blocks | 1 | Recurrent | 0.176/0.176 |
Table A1. Backbone configurations for the uniform-protocol breadth study. All models use one input anchor, denoted by channels, and predict the complete -channel trajectory. For DiffReact, ; for Kolmogorov, . Parameter counts (in millions) are reported separately for these two output dimensions as D/K.
| Task | Base params | Added params | Total params | Param. overhead | Intrinsic (ms/batch) | Auxiliary (ms/batch) | Latency overhead |
| DiffReact | 7,526 | 1,996 | 9,522 | 26.52% | 3.7778 | 4.2479 | 12.44% |
| Kolmogorov | 6,133 | 538 | 6,671 | 8.77% | 3.6005 | 4.1841 | 16.21% |
Table A15. LSTH parameter and auxiliary-response evaluation overhead. Base parameters refer to the shared FNO state predictor. Added parameters count only newly introduced hidden-difference adapters and correction heads; shared backbone blocks and the output projection that participate in continuation training are not counted as added parameters. Latency compares intrinsic paired-response evaluation with auxiliary-response evaluation for the same LSTH model. Measurements use batch size 4 on one NVIDIA GeForce RTX 4080 SUPER, with 50 warmup iterations, 200 measured iterations, and five repeats. The displayed latencies are preserved aggregate means.
| Model | Method | State ↓ | Resp. ↑ | Supp. mag. ↓ | Roll. ↓ |
| CViT | Aug-only | 0.011589 | 0.1411 | 0.3249 ± 0.0305 | 0.00690 |
| TangentPDE-Core | 0.009398 | 0.4721 | 0.3281 ± 0.0111 | 0.00590 | |
| FNO | Aug-only | 0.007529 | 0.4367 | 0.8152 ± 0.0646 | 0.00413 |
| TangentPDE-Core | 0.005939 | 0.9358 | 0.3167 ± 0.3682 | 0.00336 | |
| LaMO | Aug-only | 0.007906 | 0.2455 | 1.0623 ± 0.0634 | 0.00344 |
| TangentPDE-Core | 0.007188 | 0.4855 | 0.3807 ± 0.6121 | 0.00348 | |
| MambaNO | Aug-only | 0.012105 | 0.0722 | 2.2330 ± 0.2587 | 0.00830 |
| TangentPDE-Core | 0.010433 | 0.9345 | 0.2033 ± 0.0044 | 0.00740 | |
| RNO | Aug-only | 0.008452 | 0.5431 | 0.8445 ± 0.0443 | 0.00580 |
| TangentPDE-Core | 0.006316 | 0.9630 | 0.0112 ± 0.0039 | 0.00440 |
Table A8. DiffReact results. State, response cosine, and Rollout@20 reproduce the main-paper means; supplementary magnitude error is mean ± standard deviation from the independent three-seed diagnostic.
| Model | Method | State ↓ | Resp. ↑ | Supp. mag. ↓ | Roll. ↓ |
| CViT | Aug-only | 0.414042 | 0.3075 | 1.7635 ± 0.0715 | 0.43090 |
| TangentPDE-Core | 0.353682 | 0.3192 | 0.6632 ± 0.3295 | 0.26900 | |
| FNO | Aug-only | 0.237184 | 0.7392 | 1.0413 ± 0.0058 | 0.22747 |
| TangentPDE-Core | 0.237143 | 0.7798 | 1.0397 ± 0.0043 | 0.22030 | |
| LaMO | Aug-only | 0.213051 | 0.4187 | 1.2073 ± 0.0397 | 0.38450 |
| TangentPDE-Core | 0.200136 | 0.4000 | 1.2221 ± 0.0345 | 0.36410 | |
| MambaNO | Aug-only | 0.488124 | 0.6189 | 1.0302 ± 0.0055 | 0.80600 |
| TangentPDE-Core | 0.483679 | 0.6493 | 1.0881 ± 0.0437 | 0.79490 | |
| RNO | Aug-only | 0.223250 | 0.6769 | 1.7261 ± 0.1585 | 0.39700 |
| TangentPDE-Core | 0.179350 | 0.5074 | 1.7674 ± 0.1716 | 0.33470 |
Table A9. Kolmogorov results. State, response cosine, and Rollout@20 reproduce the main-paper means; supplementary magnitude error is mean ± standard deviation from the independent three-seed diagnostic.
| Model | Method | State ↓ | Resp. ↑ | Supp. mag. ↓ | Roll. ↓ |
| CViT | Aug-only | 0.293270 | 0.2084 | 0.5475 ± 0.0229 | 0.25000 |
| TangentPDE-Core | 0.265638 | 0.2191 | 0.3839 ± 0.0098 | 0.24610 | |
| FNO | Aug-only | 0.156602 | 0.3901 | 0.4872 ± 0.0172 | 0.15502 |
| TangentPDE-Core | 0.157755 | 0.4533 | 0.2572 ± 0.0112 | 0.15108 | |
| LaMO | Aug-only | 0.287690 | 0.2064 | 0.8402 ± 0.0204 | 0.25420 |
| TangentPDE-Core | 0.286906 | 0.2056 | 0.4180 ± 0.0041 | 0.25270 | |
| MambaNO | Aug-only | 0.683363 | 0.1263 | 1.4627 ± 0.0515 | 0.72760 |
| TangentPDE-Core | 0.635693 | 0.1077 | 1.6136 ± 0.0398 | 0.68070 | |
| RNO | Aug-only | 0.321615 | 0.1989 | 0.7605 ± 0.0566 | 0.28290 |
| TangentPDE-Core | 0.289794 | 0.2288 | 0.5202 ± 0.0242 | 0.25390 |
Table A10. Burgers results. State, response cosine, and Rollout@20 reproduce the main-paper means; supplementary magnitude error is mean ± standard deviation from the independent three-seed diagnostic.
The parameter count uses unique nn.Parameter objects, includes biases, and excludes buffers, optimizer states, activations, and checkpoint metadata. The latency timing excludes data loading and host-to-device transfer. Raw per-repeat timing vectors and explicit precision metadata were not retained, so only aggregate mean latency is reported; median and P95 are not reported. These measurements characterize only the optional auxiliary paired-response path and do not change the cost of standard state-only inference.
The comparison among Static, State FT, Weak FT, and LSTH separates two questions. State and rollout metrics evaluate the shared state predictor, whereas the auxiliary response metric evaluates the dedicated LSTH execution path. An auxiliary-response gain must therefore not be interpreted as evidence that LSTH improves the intrinsic finite-step response of the same shared state operator.
| Model | Method | State | Resp. | Supp. mag. | Roll. |
| CViT | Aug-only | 0.546122 | 0.0471 | 2.4749 0.0164 | 0.06160 |
| TangentPDE-Core | 0.483750 | 0.0454 | 2.2433 0.0135 | 0.05960 | |
| FNO | Aug-only | 0.129336 | 0.0899 | 2.5620 0.0116 | 0.03090 |
| TangentPDE-Core | 0.135071 | 0.0881 | 2.5603 0.0095 | 0.06828 | |
| LaMO | Aug-only | 0.557364 | 0.0492 | 3.1964 0.0715 | 0.09400 |
| TangentPDE-Core | 0.555104 | 0.0475 | 1.5833 0.0871 | 0.07730 | |
| MambaNO | Aug-only | 0.787117 | 0.0455 | 1.7060 0.0369 | 0.12610 |
| TangentPDE-Core | 0.775974 | 0.0403 | 1.5748 0.0245 | 0.09050 | |
| RNO | Aug-only | 0.670729 | 0.0135 | 3.5990 0.2384 | 0.65600 |
| TangentPDE-Core | 0.636664 | 0.0184 | 2.1184 0.5829 | 0.60530 |
Table A11. Advection results. State, response cosine, and Rollout@20 reproduce the main-paper means; supplementary magnitude error is mean standard deviation from the independent three-seed diagnostic.
Table A12: Paired analysis over the 20 submitted task–backbone settings. Positive changes denote Core improvement; intervals use 10,000 paired bootstrap resamples, and values use Holm-corrected one-sided Wilcoxon signed-rank tests.
| Metric | Improved | Mean | Median | Bootstrap 95% CI | Raw | Holm | ||
| State | 18/20 | 0.017041 | 0.003352 | [0.007986, 0.026833] | 195.0 | 0.000131 | 0.000392 | 0.857 |
| Response cosine | 12/20 | 0.116285 | 0.011200 | [0.026504, 0.226892] | 160.0 | 0.020000 | 0.020000 | 0.524 |
| Rollout@20 | 18/20 | 0.020988 | 0.005555 | [0.006057, 0.039931] | 193.0 | 0.000197 | 0.000395 | 0.838 |
Table A12.
Supplementary LSTH Over-Correction Diagnostic
Table A17 reproduces the submitted state, intrinsic-response, auxiliary-response, and rollout means, and appends an independent three-seed over-correction diagnostic for LSTH. The added diagnostic does not reconstruct uncertainty around the submitted main metrics.
| Variant | State | Intr. resp. | Aux. resp. | Roll. | Supp. over-corr. | 95% CI |
| DiffReact | ||||||
| Static | 0.0059 | 0.9433 | – | 0.0035 | – | – |
| State FT | 0.0032 | 0.9410 | – | 0.0018 | – | – |
| Weak FT | 0.0037 | 0.9440 | – | 0.0021 | – | – |
| LSTH | <u>0.0036</u> | 0.9431 | 0.9779 | <u>0.0019</u> | 0.0000 ± 0.0000 | [0.0000, 0.0000] |
| Kolmogorov | ||||||
| Static | 0.2809 | 0.7572 | – | 0.2941 | – | – |
| State FT | 0.2749 | 0.7281 | – | 0.2887 | – | – |
| Weak FT | <u>0.2795</u> | <u>0.7525</u> | – | <u>0.2886</u> | – | – |
| LSTH | 0.2749 | 0.7281 | 0.9071 | 0.2885 | 0.3300 ± 0.0100 | [0.3052, 0.3548] |
Table A17. LSTH supplementary over-correction diagnostic. State, intrinsic response, auxiliary response, and rollout reproduce the submitted means. Supplementary over-correction is an independent three-seed diagnostic and is defined only for LSTH; dashes denote undefined or inapplicable quantities, and its uncertainty is a two-sided 95% Student- interval over three seeds.
Over-correction is defined only for LSTH because the other variants do not execute the auxiliary correction path. Dashes therefore denote undefined or inapplicable quantities rather than numerical zero. The reported standard deviation and confidence interval apply only to the supplementary over-correction diagnostic column.
Computational Resources and Query Accounting
These intervals are descriptive summaries over nine heterogeneous objective–seed runs and should not be interpreted as uncertainty intervals for one fixed objective. Table A20 reports measured response-pair generation cost. Each accepted pair adds one perturbed-endpoint solver call because its anchor label is reused. DiffReact and Kolmogorov are regenerated from their frozen perturbed inputs and numerically checked against the stored endpoints; Burgers and Advection use the original complete source-generation logs.
| Task | Source | Add. calls | Time (s) | Execution |
| Advection | 10000 | 10000 | 10.0 | 1 GPU |
| Burgers | 10000 | 10000 | 24.0 | 1 GPU |
| DiffReact | 350 | 350 | 25.1 | 4 CPU workers |
| Kolmogorov | 350 | 350 | 6.8 | 4 CPU workers |
Table A20. Measured response-pair generation cost. DiffReact and Kolmogorov regenerate the 350 frozen perturbed endpoints. Burgers and Advection report the original 10,000-pair source generation before selecting the submitted subset. Each entry is one wall-clock measurement including output writing.
Except for the separate LSTH latency microbenchmark in Table A15, experiments were executed on a Linux 6.8 system with two Intel Xeon Gold 6530 processors, approximately 512 GiB of system memory, and NVIDIA GeForce RTX 5090 GPUs with 32 GiB memory per device. The software environment used Python 3.10.12, PyTorch 2.12.0 with CUDA 13.0, NVIDIA driver 580.126.20, NumPy 2.2.6, and h5py 3.16.0. Independent single-GPU runs may be executed concurrently; the number of GPUs visible to the host is not treated as the number used by one training run.
The reported costs refer to different stages and should not be conflated. Pair-generation time measures reference-solver data construction, whereas the unseen-direction cost measures execution of a diagnostic workflow. Neither quantity is a proxy for state-predictor training time or model inference latency.
Additional Burgers Objective Analysis
Figure A6 presents the Burgers objective study as an additional matched RNO ablation. It is separate from the cross-architecture comparison and tests whether adding a frequency-response term changes the direction–magnitude objective on nonlinear transport.

Figure A6. Additional RNO objective ablation on Burgers. Points show three-seed means for State, response cosine, and Rollout@20 across the objective variants; error bars denote sample standard deviation. The plot is intended to compare objective trends rather than to foreground inferential statistics.
The frequency-response row is interpreted as a local objective comparison within this Burgers configuration. It does not establish that frequency supervision is uniformly beneficial across the other tasks or backbones.
Robustness to Perturbation Changes
The unseen-direction study separates perturbation families between training and testing while preserving the original anchor splits, optimization budget, seeds, and validation-state checkpoint criterion. Table A21 therefore measures transfer across the tested perturbation families rather than in-distribution response fitting. It does not evaluate arbitrary directions outside the declared construction.
| Task | Method | State | Resp. cos. | Rel. resp. | Mag. err. | Roll.@20 |
| DiffReact | Aug-only | 0.010514 0.000405 | 0.1667 0.0784 | 1.0227 0.0643 | 1.5896 0.6075 | 0.00580 0.00133 |
| DiffReact | Two-point MSE | 0.010411 0.000500 | 0.1707 0.0680 | 1.0208 0.0652 | 1.5491 0.5881 | 0.00564 0.00146 |
| DiffReact | TangentPDE-Core | 0.008211 0.000776 | 0.8834 0.0034 | 0.6393 0.0108 | 0.2504 0.0095 | 0.00491 0.00034 |
| Kolmogorov | Aug-only | 0.286750 0.003532 | 0.5059 0.0021 | 9.8993 0.0603 | 2.3386 0.0056 | 0.31424 0.01693 |
| Kolmogorov | Two-point MSE | 0.286668 0.003640 | 0.5059 0.0021 | 9.9005 0.0588 | 2.3387 0.0055 | 0.31396 0.01725 |
| Kolmogorov | TangentPDE-Core | 0.286205 0.005883 | 0.5419 0.0016 | 9.7912 0.0321 | 2.3318 0.0031 | 0.30824 0.02328 |
| Task | Method | State CI | Resp. CI | Rel. resp. CI | Mag. CI | Roll. CI |
| DiffReact | Aug-only | 0.001007 | 0.1948 | 0.1596 | 1.5091 | 0.00331 |
| DiffReact | Two-point MSE | 0.001243 | 0.1690 | 0.1620 | 1.4610 | 0.00363 |
| DiffReact | TangentPDE-Core | 0.001926 | 0.0086 | 0.0269 | 0.0235 | 0.00085 |
| Kolmogorov | Aug-only | 0.008773 | 0.0052 | 0.1498 | 0.0140 | 0.04205 |
| Kolmogorov | Two-point MSE | 0.009042 | 0.0051 | 0.1460 | 0.0137 | 0.04284 |
| Kolmogorov | TangentPDE-Core | 0.014615 | 0.0039 | 0.0797 | 0.0078 | 0.05783 |
Table A21. FNO evaluation on perturbation families excluded from training. The upper panel reports mean sample standard deviation over three seeds; the lower panel gives the half-width of the corresponding two-sided 95% Student- interval. Bold marks the best mean per task.
On DiffReact, Core improves all five reported metrics and raises response cosine from 0.1667 for Aug-only and 0.1707 for Two-point MSE to 0.8834. On Kolmogorov, Core provides a smaller response-cosine gain, from approximately 0.506 to 0.542; the state and rollout differences are small relative to their across-seed variation. The transfer benefit is therefore task dependent.
The altered-step experiment holds the DiffReact anchors, perturbation directions, normalization, and check-points fixed. The reaction–diffusion solver regenerates each endpoint at , , and . Table A22 and Figure A7 show that the decomposed objective retains high response alignment across all three test steps, whereas Direct Response MSE remains substantially less aligned. Because the perturbation family and checkpoints are fixed, this experiment tests same-source step-size transfer only; it does not establish step-size invariance for the other PDE systems.
Figure A7. Same-source DiffReact altered-step response evaluation using fixed checkpoints. Points show the three-seed mean and error bars show the sample standard deviation. The comparison evaluates transfer across the three tested step sizes within the same perturbation family.
| Objective | Test step | Resp. cos. | Mag. err. |
| Full Core | 0.9537 0.0041 | 0.0220 0.0051 | |
| Full Core | 0.9630 0.0035 | 0.0146 0.0051 | |
| Full Core | 0.9665 0.0036 | 0.0191 0.0056 | |
| Direct Resp. MSE | 0.5768 0.0616 | 0.8388 0.0452 | |
| Direct Resp. MSE | 0.5953 0.0610 | 0.8716 0.0735 | |
| Direct Resp. MSE | 0.6034 0.0599 | 0.8858 0.0878 |
| Objective | Test step | Resp. 95% CI hw | Mag. 95% CI hw |
| Full Core | 0.0102 | 0.0127 | |
| Full Core | 0.0086 | 0.0127 | |
| Full Core | 0.0088 | 0.0140 | |
| Direct Resp. MSE | 0.1531 | 0.1123 | |
| Direct Resp. MSE | 0.1514 | 0.1826 | |
| Direct Resp. MSE | 0.1489 | 0.2181 |
Table A22. DiffReact same-source altered-step evaluation. Fixed checkpoints are evaluated on solver-regenerated endpoints at each test step. The upper panel reports mean sample standard deviation over three seeds; the lower panel gives the corresponding two-sided 95% Student- CI half-width.
Scope and Additional Limitations
TangentPDE learns finite-step secants along sampled perturbation families, not a complete Jacobian or a uniformly accurate local linearization in every direction. Improvements in state, intrinsic response, and rollout are related but distinct outcomes; improvement in one metric does not logically imply improvement in the others. The submitted results and the independent reruns therefore retain their negative and mixed cells rather than collapsing them into one universal claim.
Response gains remain task and backbone dependent. The unseen-direction study covers only the declared held-out perturbation families, and the altered-step evidence is limited to the evaluated same-source DiffReact construction. Linear rescaling of an existing Kolmogorov pair is not a new PDE solve and is excluded from the altered-step evidence. These boundaries are stated to keep the appendix aligned with the implemented diagnostics rather than to broaden the empirical claim.
LSTH provides a dedicated auxiliary response estimate rather than a universal intrinsic-response improvement. Its auxiliary correction must be validated per task because a residual that is useful for one system can over-correct another. The LSTH auxiliary metric is consequently reported only on the execution path for which it is defined. In particular, the auxiliary head is treated as a separate diagnostic output, not as a direct correction to the reported state operator. This distinction is kept explicit because the appendix contains both shared-state metrics and auxiliary-response quantities.
Finally, seed-level summaries are based on three training seeds and should be read as descriptive optimization variability rather than high-powered single-configuration significance tests. Pair-generation time, diagnostic-run time, training time, and inference latency measure different computational stages and are not interchangeable. The resource tables therefore report accounting for the stage named in each caption: query construction, diagnostic evaluation, and auxiliary-head inference are intentionally separated. The visual summaries preserve the metric directions and mixed outcomes of the underlying submitted results. Exact protocol and accounting quantities remain reported in tables. These conventions make the appendix a reproducibility and audit companion to the main paper rather than an additional source of broader claims.
| Task | Time (min) | 95% CI hw | Peak (GiB) | 95% CI hw | GPUs |
| DiffReact | 6.4 ± 2.1 | [4.8, 8.0] | 0.18 ± 0.00 | [0.18, 0.18] | 1 |
| Kolmogorov | 7.9 ± 2.2 | [6.2, 9.6] | 0.09 ± 0.00 | [0.09, 0.09] | 1 |
Table A19. Measured cost of the unseen-direction runs. Time and peak allocated GPU memory are mean ± sample standard deviation per run, with two-sided 95% Student- intervals over the nine objective–seed runs for each task.
The ordering of the appendix follows the evidence type rather than a single performance ranking. Configuration and budget tables establish comparability, mechanism tables separate objective components, resource tables document the query and execution accounting, and the final robustness tables record perturbation-family and step-size checks. This order is intended to make clear which quantities are directly comparable and which are protocol-specific.
Figure A8. Early-time DiffReact state fields for the first output channel. The columns show the reference field, Aug-only and TangentPDE-Core predictions, and their absolute errors at five fixed saved indices before the physical field decays. TangentPDE-Core better preserves the localized structures and reduces the dominant error on this median-improvement trajectory.
Figure A9. Early-time Burgers one-dimensional state fields at saved rollout indices 0, 2, 4, 6, 8 from the independent uniform-protocol FNO rerun. TangentPDE-Core more closely follows the transported structures and reduces the dominant errors on this median-improvement trajectory. The color strips are expanded vertically for readability.
Figure A10. Early-time Advection one-dimensional state fields at saved rollout indices 0, 2, 4, 6, 8 from the independent uniform-protocol FNO rerun. Aug-only and TangentPDE-Core remain visually similar, consistent with the small and mixed transport-task differences in the quantitative results. The color strips are expanded vertically for readability.
References
- [1]Brandstetter, J.; Worrall, D. E.; and Welling, M. 2022. Message Passing Neural PDE Solvers. In International Conference on Learning Representations.arxiv.org/abs/2202.03376
- [2]Cao, L.; O’Leary-Roseberry, T.; and Ghattas, O. 2025. Derivative-Informed Neural Operator Acceleration of Geometric MCMC for Infinite-Dimensional Bayesian Inverse Problems. Journal of Machine Learning Research, 26(78): 1–68.arxiv.org/abs/2403.08220
- [3]Czarnecki, W. M.; Osindero, S.; Jaderberg, M.; Swirszcz, G.; and Pascanu, R. 2017. Sobolev Training for Neural Networks. In Advances in Neural Information Processing Systems, volume 30.arxiv.org/abs/1706.04859
- [4]Feng, H.; Wang, Y.; and Fan, D. 2025. How to Re-enable PDE Loss for Physical Systems Modeling Under Partial Observation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, 182–190.arxiv.org/abs/2412.09116
- [5]Hao, Z.; Wang, Z.; Su, H.; Ying, C.; Dong, Y.; Liu, S.; Cheng, Z.; Song, J.; and Zhu, J. 2023. GNOT: A General Neural Operator Transformer for Operator Learning. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, 12556–12569. PMLR.arxiv.org/abs/2302.14376
- [6]Hou, J.; Wang, H.; Xu, P.; Gao, C.; Liu, H.; and Jing, L. 2026. Learning Neural Operators from Partial Observations via Latent Autoregressive Modeling. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 390–398.arxiv.org/abs/2601.15547
- [7]Houlsby, N., Gjur̆giu, A., Jastrzebski, S., Morrone, B., de Laroussilhe, Q., Guestrin, C., Attarian, M., & Gelly, S. (2019). Parameter-Efficient Transfer Learning for NLP. In International Conference on Machine Learning (pp. 2790–2799).
- [9]Huang, X.; and Perdikaris, P. 2026. PhysicsCorrect: A Training-Free Approach for Stable Neural PDE Simulations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 22057–22065.arxiv.org/abs/2507.02227
- [10]Koehler, F., Niedermayer, S., Westermann, R., & Thuerey, N. (2024). APBench: A Benchmark for Autoregressive Neural Emulators of PDEs. In Advances in Neural Information Processing Systems (Vol. 37).
- [11]Koren, N.; Mackenbach, R. J. J.; van Sloun, R. J. G.; Radinsky, K.; and Freedman, D. 2026. SVD-NO: Learning PDE Solution Operators with SVD Integral Kernels. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 525–533.arxiv.org/abs/2511.10025
- [12]Kovachki, N.; Lanthaler, S.; and Mishra, S. 2021. On Universal Approximation and Error Bounds for Fourier Neural Operators. Journal of Machine Learning Research, 22(290): 1–76.arxiv.org/abs/2107.07562
- [13]Kovachki, N., Li, Z., Liu, B., Azizzadenesheli, K., Bhattacharya, K., Stuart, A., & Anandkumar, A. (2023). Neural Operator: Learning Maps Between Function Spaces with Applications to PDEs. Journal of Machine Learning Research, 24(89), 1–97.
- [18]Lippe, P.; Veeling, B. S.; Perdikaris, P.; Turner, R. E.; and Brandstetter, J. 2023. PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers. In Advances in Neural Information Processing Systems, volume 36, 67398–67433.arxiv.org/abs/2308.05732
- [21]O’Leary-Roseberry, T.; Chen, P.; Villa, U.; and Ghattas, O. 2024. Derivative-Informed Neural Operator: An Efficient Framework for High-Dimensional Parametric Derivative Learning. Journal of Computational Physics, 496: 112555.arxiv.org/abs/2206.10745
- [22]Peyvan, A.; Kumar, V.; and Karniadakis, G. E. 2025. Fusion-DeepONet: A Data-Efficient Neural Operator for Geometry-Dependent Hypersonic and Supersonic Flows. arXiv preprint arXiv:2501.01934.arxiv.org/abs/2501.01934
- [23]Qiu, Y.; Bridges, N.; and Chen, P. 2024. Derivative-Enhanced Deep Operator Network. In Advances in Neural Information Processing Systems, volume 37, 20945–20981.arxiv.org/abs/2402.19242
- [24]Rahman, M. A.; George, R. J.; Elleithy, M.; Leibovici, D.; Li, Z.; Bonev, B.; White, C.; Berner, J.; Yeh, R. A.; Kossaifi, J.; Azizzadenesheli, K.; and Anandkumar, A. 2024. Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs. In Advances in Neural Information Processing Systems, volume 37.arxiv.org/abs/2403.12553
- [25]Takamoto, M., Praditia, T., Leiteritz, R., MacKinlay, D., Alesiani, F., Pflueger, D., & Niepert, M. (2022). PDEBench: An Extensive Benchmark for Scientific Machine Learning. Advances in Neural Information Processing Systems, 35, 1596–1611.
- [26]Tiwari, K., Dutta, N., Krishnan, N. M. A., & A. P., P. (2025). Latent Mamba Operator for Partial Differential Equations. Proceedings of the 42nd International Conference on Machine Learning, 267, 59639–59667.
- [27]Wang, C., & Townsend, A. (2025). Operator Learning for Hyperbolic PDEs. Journal of Machine Learning Research, 26(199), 1–44.
- [28]Wang, S.; Seidman, J. H.; Sankaran, S.; Wang, H.; Pappas, G. J.; and Perdikaris, P. 2025. CViT: Continuous Vision Transformer for Operator Learning. In The Thirteenth International Conference on Learning Representations.arxiv.org/abs/2405.13998
- [29]Wang, T.; and Wang, C. 2024. Latent Neural Operator for Solving Forward and Inverse PDE Problems. In Advances in Neural Information Processing Systems, volume 37.arxiv.org/abs/2406.03923
- [32]Zheng, J.; Li, W.; Xu, N.; Zhu, J.; Lin, X.; and Zhang, X. 2024. Alias-Free Mamba Neural Operator. In Advances in Neural Information Processing Systems, volume 37, 52962–52995.DOI
- [33]Zhou, C.; Chen, J.; and Yang, Z. 2026. SAOT: An Enhanced Quality-Aware Spectral Transformer for Solving PDEs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 28928–28936.arxiv.org/abs/2511.18777











