betaFeedback

WorksComputer Science

TangentPDE: Learning Local Operator Responses with Sparse Finite-Difference Supervision

Abstract

Neural PDE operators are often optimized through state reconstruction, which largely constrains operator values at observed inputs but does not fully characterize their sensitivity to local perturbations. A more faithful approximation of operator behavior requires not only accurate state predictions, but also correct variations along local perturbation directions. We present TangentPDE, which uses finite differences to approximate directional derivatives and constructs task-relevant finite-step responses from the state outputs of a reference solver at sparse paired inputs. Its objective jointly constrains state fidelity, response-direction alignment, and response-magnitude calibration, thereby learning both operator values and their local variations. Because the response labels are constructed solely from state outputs at paired inputs, the method requires neither explicit derivative labels, linearized or adjoint PDE solves, nor full Jacobians, and remains applicable to nonlinear solvers without derivative interfaces. Building on this, we additionally propose a selectively executed adaptation mechanism to strengthen local operator representations under finite-difference supervision. Budget-matched and correspondence-perturbed controls isolate the effect of relational supervision from improvements attributable to additional state observations or increased data volume. Designed as a plug-and-play training framework, TangentPDE integrates with diverse neural-operator backbones. Across four PDE systems and 20 task-backbone settings, TangentPDE lowers the three-seed mean state error in 18 settings and the corresponding Rollout@20 error in 18, while intrinsic finite-step response gains are strongest on DiffReact and remain task- and backbone-dependent elsewhere.

Introduction

Neural operators learn mappings between function spaces and provide fast data-driven surrogates for partial differential equations (PDEs) [12, 13] (Li et al. 2020; Lu et al. 2021; Li et al. 2021; Lu et al. 2022). Standard supervision fits a predictor Gθ(z)G_\theta(z) to a reference state S(z)S(z) over sampled input–output function pairs. State reconstruction remains indispensable because it controls operator values at the observed inputs and, for any fixed nonzero perturbation step, vanishing errors at both endpoints imply vanishing finite-step response error. Our focus is the finite-accuracy regime, where independent endpoint reconstruction leaves the cross-endpoint error increment implicit.

The central observation is simple but consequential. Let eθ(z)=Gθ(z)−S(z)e_\theta(z)=G_\theta(z)-S(z) and let z+z^+ be a valid perturbation of zz. Then

rop−r∗=eθ(z+)−eθ(z)ϵ,r_{\mathrm{op}}-r^*=\frac{e_\theta(z^+)-e_\theta(z)}{\epsilon},

where r∗r^* and ropr_{\mathrm{op}} are the reference and learned finite-step responses. Endpoint state errors therefore control a coarse upper bound on response error, but their aggregate magnitudes do not determine the realized response mismatch. Models with comparable state error can retain different cross-endpoint error increments and hence different local operator behavior. This distinction matters whenever a learned operator is used to compare nearby inputs, estimate sensitivity, support local design or control, or advance a system repeatedly.

Derivative- and Sobolev-informed studies show that local variation supervision can improve not only derivative fidelity but also function or state approximation [3, 21, 2, 23, 22]. This motivates an intermediate supervision regime: nearby admissible states can be queried from the reference system, while Jacobians, tangent solves, adjoints, or a differentiable solver interface may be unavailable or unnecessary at the task-defined perturbation scale. TangentPDE addresses this regime by converting correctly paired endpoint states into explicit finite-step relational supervision. The required reference information remains state-only, while ordinary backpropagation is used to train the neural operator.

We introduce TangentPDE-Core, which combines matched endpoint reconstruction with primary response-direction alignment and weaker response-magnitude calibration under sparse paired supervision. The novelty is not finite differencing itself, but the identification and direct optimization of the cross-endpoint error increment that remains implicit under independent state fitting. Core operates at the supervision level and preserves the backbone’s state-output semantics, making it architecture-orthogonal to heterogeneous neural-operator designs. As an optional extension, the compact Layer-Selective Tangent Head (LSTH) maps finite differences of selected hidden representations to an auxiliary response residual. Its correction never enters the state outputs, so the auxiliary estimate remains distinct from the intrinsic state-output response.

Our contributions are:

  • an exact error-increment characterization identifying cross-endpoint state-error variation as the quantity governing finite-step response mismatch;

  • TangentPDE-Core, a state-query-only objective that directly supervises intrinsic finite-step responses through direction alignment and weak magnitude calibration;

  • an architecture-orthogonal evaluation framework with matched controls for endpoint observations, query budget, correspondence, and sparse pair coverage; and

  • LSTH, a selectively executed auxiliary response learner that reuses selected hidden finite differences without changing standard single-input state inference.

Neural PDE Operator Learning

DeepONet, Fourier, graph, message-passing, latent, recurrent, state-space, and attention-based neural operators mainly differ in representation, geometry handling, and information exchange [1, 5, 29, 24, 28, 26] (Li et al. 2020; Lu et al. 2021; Li et al. 2021; Li et al. 2023; Wu et al. 2024; Zheng et al. 2024; Ye, Zhang, and Wang 2025). Recent AAAI work further studies efficient kernels, locality, linearized physics-attention, and learning from partial observations [11, 33, 6] (Hu et al. 2026). TangentPDE is architecture-orthogonal: rather than redesigning the state predictor, it changes the supervision relation imposed on correctly paired operator evaluations while preserving the backbone’s output semantics.

Derivative-, Sensitivity-, and Sobolev-Informed Learning

Sobolev and derivative-informed learning supervise local variation beyond function values, including target gradients, full or reduced Jacobians, Fréchet derivatives, Jacobian–vector products, and sensitivities obtained from tangent or adjoint systems [3, 21, 2, 23, 22]. These methods directly constrain infinitesimal behavior when reliable derivative sources are available. TangentPDE occupies a distinct access regime: it learns task-scale finite-step responses from paired state queries rather than approximating a complete infinitesimal derivative operator. This replaces specialized derivative-interface requirements with ordinary state evaluations at admissible nearby inputs.

Finite-Difference, Augmentation, and Consistency Supervision

Finite differences also appear in data augmentation, numerical derivative approximation, and consistency objectives, but their supervision semantics differ from TangentPDE. Ordinary two-point augmentation treats the anchor and perturbed input as independent state examples and primarily enlarges input coverage. Consistency regularization commonly encourages invariance, equivariance, or agreement between related predictions. TangentPDE instead treats the anchor–perturbation correspondence itself as a supervised object and uses the reference endpoint difference as a generally nonzero target with both direction and scale. Although the target contains no reference information beyond the endpoint states, its coupled objective directly constrains the cross-endpoint error increment and is not reducible to independent endpoint reconstruction.

Physics-Informed Correction and Auxiliary Adaptation

Physics-informed training and rollout-stabilization methods impose governing-equation residuals, multi-step objectives, generative refinement, or inference-time numerical corrections [18, 4, 9] (Li et al. 2024). TangentPDE instead supervises the finite-step behavior exhibited by the reference operator and evaluates response fidelity separately from rollout stability. LSTH is related to auxiliary multi-task learning and compact adaptation modules [7], but its adapters consume pairwise hidden finite differences rather than a single hidden state and produce a dedicated auxiliary response estimate. Standard state inference remains head-free, while intrinsic and auxiliary responses are evaluated as separate objects.

Method

Problem Setup and Finite-Step Responses

Let S,Gθ:Zadm→U\mathcal{S},G_{\theta}:\mathcal{Z}_{\mathrm{adm}}\to\mathcal{U} denote the reference and learned operators. Each benchmark samples an admissible input zz and a perturbation protocol (v,ϵ)(v,\epsilon), producing

z+=T(z,v,ϵ)∈Zadm,z^{+}=T(z,v,\epsilon)\in\mathcal{Z}_{\mathrm{adm}},
T(z,v,ϵ)=z+ϵvwhen no transform is required.(1)T(z,v,\epsilon)=z+\epsilon v \quad\text{when no transform is required.} \tag*{(1)}

Ordinary and paired observations can be written compactly as

Ds={(zi,ui):ui=S(zi)},\mathcal{D}_{s}=\{(z_i,u_i):u_i=\mathcal{S}(z_i)\},
Dp={(zj,zj+,uj,uj+,ϵj)}.(2)\mathcal{D}_{p}=\{(z_j,z_j^{+},u_j,u_j^{+},\epsilon_j)\}. \tag*{(2)}

For each pair, u=S(z)u=\mathcal{S}(z) and u+=S(z+)u^{+}=\mathcal{S}(z^{+}). The reference finite-step response and the intrinsic response induced by the state predictor are

r∗=u+−uϵ,rop=Gθ(z+)−Gθ(z)ϵ.(3)r^{*}=\frac{u^{+}-u}{\epsilon},\qquad r_{\mathrm{op}}=\frac{G_{\theta}(z^{+})-G_{\theta}(z)}{\epsilon}. \tag*{(3)}

These finite-step secants are the actual optimization objects. They approach directional derivatives only under the corresponding differentiability and small-step conditions. We assume neither a normalized direction nor recovery of a complete Jacobian, and all locality claims refer to the benchmark-defined perturbation neighborhood. The inner product and norm below are those used by the discrete response implementation.

Figure 1. Overview of TangentPDE. Core supervises the intrinsic state-output response ropr_{\mathrm{op}}; optional LSTH produces a separate auxiliary estimate rauxr_{\mathrm{aux}} from selected hidden finite differences.

Finite-Accuracy Error Increment

With eθ(z)=Gθ(z)−S(z)e_{\theta}(z)=G_{\theta}(z)-S(z),

rop−r∗=eθ(z+)−eθ(z)ϵ.(4)r_{\mathrm{op}}-r^{*}=\frac{e_{\theta}(z^{+})-e_{\theta}(z)}{\epsilon}. \tag*{(4)}

The triangle inequality gives

∥rop−r∗∥2≤∥eθ(z+)∥2+∥eθ(z)∥2ϵ.(5)\left\|r_{\mathrm{op}}-r^{*}\right\|_{2}\leq\frac{\left\|e_{\theta}(z^{+})\right\|_{2}+\left\|e_{\theta}(z)\right\|_{2}}{\epsilon}. \tag*{(5)}

Cross-endpoint error-increment characterization. (4) exactly characterizes finite-step response error through the cross-endpoint increment of state-prediction errors. (5) shows that endpoint accuracy limits the possible response error but does not determine the realized mismatch. In particular, endpoint-error pairs (a,a)(a,a) and (a,−a)(a,-a) have identical marginal norms but induce response errors 00 and −2a/ϵ-2a/\epsilon, respectively. TangentPDE-Core directly optimizes this relational quantity, which remains implicit under independent endpoint reconstruction.

Sparse Paired Observations

Pairs are constructed after data splitting. Their anchor labels are reused, perturbed endpoint labels are queried only for selected training anchors, and unpaired samples may continue to provide ordinary state supervision. Pair coverage is

ρ=Npaired anchorsNtraining anchors.(6)\rho=\frac{N_{\mathrm{paired\ anchors}}}{N_{\mathrm{training\ anchors}}}. \tag*{(6)}

Actual additional solver calls are reported from the pair-generation pipeline. MSE-2N-random tests whether additional state volume explains the gain; Two-point MSE matches the exact endpoint labels and query budget; Pair-shuffled Tangent tests whether preserving the implemented correspondence adds value beyond those matched observations.

TangentPDE-Core

For a paired sample, matched endpoint reconstruction is

Lstate=MSE⁡(Gθ(z),u)+MSE⁡(Gθ(z+),u+).(7)\mathcal{L}_{\mathrm{state}} = \operatorname{MSE}(G_\theta(z),u) + \operatorname{MSE}(G_\theta(z^{+}),u^{+}). \tag*{(7)}

Two-point MSE uses the same paired endpoints and reference queries but optimizes only their marginal state errors. TangentPDE-Core additionally constrains the relation between the two predictions through

Ldir(a,b)=1−⟨a,b⟩∥a∥2∥b∥2+δ(8)\mathcal{L}_{\mathrm{dir}}(a,b) = 1 - \frac{\langle a,b\rangle}{\lVert a\rVert_{2}\lVert b\rVert_{2}+\delta} \tag*{(8)}
Lmag(a,b)=∣log⁡(∥a∥2+δ)−log⁡(∥b∥2+δ)∣.\mathcal{L}_{\mathrm{mag}}(a,b) = \left\lvert\log(\lVert a\rVert_{2}+\delta)-\log(\lVert b\rVert_{2}+\delta)\right\rvert.

Direction alignment is the primary structural term; the lower-weight magnitude term calibrates responses that are directionally correct but systematically too weak or too strong. The complete objective is

LCore=Lstate+λdirLdir(rop,r∗)+λmagLmag(rop,r∗),λdir≫λmag>0.(9)\begin{aligned} \mathcal{L}_{\mathrm{Core}} &= \mathcal{L}_{\mathrm{state}}+\lambda_{\mathrm{dir}}\mathcal{L}_{\mathrm{dir}}(r_{\mathrm{op}},r^{*}) \\ &\quad+\lambda_{\mathrm{mag}}\mathcal{L}_{\mathrm{mag}}(r_{\mathrm{op}},r^{*}),\\ \lambda_{\mathrm{dir}} &\gg\lambda_{\mathrm{mag}} > 0. \tag*{(9)} \end{aligned}

The hierarchy specifies modeling priority rather than a guaranteed ordering of gradient norms. The state objective controls endpoint fidelity, while the response terms control the cross-endpoint variation of prediction errors. Direction provides the primary geometric signal, and magnitude acts as secondary scale calibration. Core and LSTH use a common valid-response convention for their response objectives; its frozen implementation is specified with the experimental protocol.

Optional Layer-Selective Tangent Head

Let Gθ′G_{\theta^{\prime}} denote the state predictor after LSTH training. Its intrinsic response remains rop=[Gθ′(z+)−Gθ′(z)]/ϵr_{\mathrm{op}}=[G_{\theta^{\prime}}(z^{+})-G_{\theta^{\prime}}(z)]/\epsilon. For selected stages with pairwise aligned hidden representations k∈Kk \in\mathcal{K},

Δhk=hk(z+)−hk(z)ϵ,cϕ=γ∑k∈KAk(Δhk),raux=rop+cϕ.(10)\begin{aligned} \Delta h_k &= \frac{h_k(z^{+})-h_k(z)}{\epsilon},\\ c_\phi&= \gamma\sum_{k\in\mathcal{K}} A_k(\Delta h_k),\\ r_{\mathrm{aux}} &= r_{\mathrm{op}}+c_\phi. \tag*{(10)} \end{aligned}

Each representation-specific adapter AkA_k maps a hidden finite-step response to the output-response space. The final projections are zero-initialized and γ=0.05\gamma=0.05. The correction is not added to either state output, so rauxr_{\mathrm{aux}} is an auxiliary paired-response estimate rather than an intrinsic state-output secant.

LSTH confines response-specific parameters and computation to representation-specific depthwise/pointwise adapters and small output projections at selected stages. The adapters are bypassed during standard state and intrinsic-response inference and are activated only for auxiliary-response estimation.

In the four-block FNO instantiation, K={2,4}\mathcal{K}=\{2,4\} under one-based indexing, with

Ak=Pk∘PWC⁡1×1∘GELU⁡∘DWConv⁡3×3.(11)A_k=P_k\circ\operatorname{PWC}_{1\times1}\circ\operatorname{GELU}\circ\operatorname{DWConv}_{3\times3}. \tag*{(11)}

The selected features share a spatial grid, and PkP_k maps hidden channels to response channels. The same interface extends to token-, sequence-, graph-, and multi-resolution backbones through correspondence-compatible adapters.

LSTH reuses the Core response losses and regularizes the correction through

Lcorr=1B∑i=1B(∥cϕ,i∥2∥rop,i∥2+δ)2,(12)\mathcal{L}_{\mathrm{corr}}=\frac{1}{B}\sum_{i=1}^{B}\left(\frac{\lVert c_{\phi,i}\rVert_{2}}{\lVert r_{\mathrm{op},i}\rVert_{2}+\delta}\right)^{2}, \tag*{(12)}
Ladapter=0.1Ldir(raux,r∗)+0.01Lmag(raux,r∗)+10−4Lcorr.(13)\mathcal{L}_{\mathrm{adapter}}=0.1\mathcal{L}_{\mathrm{dir}}(r_{\mathrm{aux}},r^{*}) +0.01\mathcal{L}_{\mathrm{mag}}(r_{\mathrm{aux}},r^{*}) +10^{-4}\mathcal{L}_{\mathrm{corr}}. \tag*{(13)}

The regularizer limits correction magnitude relative to the intrinsic response. The auxiliary estimate remains a dedicated paired-response output rather than a correction to either state prediction.

State-Dominant Optimization and Deployment

Stage I uses the first 20% of LSTH continuation updates to train only the adapters. Let Laux\mathcal{L}_{\mathrm{aux}} denote Laux-response\mathcal{L}_{\mathrm{aux\text{-}response}}. During the remaining 80%, early stages stay frozen while selected intermediate/late stages and the state-output head receive

gmid/late=∇θmid/lateLstate+μblock∇θmid/lateLaux,gout=∇θoutLstate+μhead∇θoutLaux,gϕ=∇ϕLadapter.(14)\begin{aligned} g_{\mathrm{mid/late}} &= \nabla_{\theta_{\mathrm{mid/late}}}\mathcal{L}_{\mathrm{state}}+\mu_{\mathrm{block}}\nabla_{\theta_{\mathrm{mid/late}}}\mathcal{L}_{\mathrm{aux}},\\ g_{\mathrm{out}} &= \nabla_{\theta_{\mathrm{out}}}\mathcal{L}_{\mathrm{state}}+\mu_{\mathrm{head}}\nabla_{\theta_{\mathrm{out}}}\mathcal{L}_{\mathrm{aux}},\\ g_{\phi} &= \nabla_{\phi}\mathcal{L}_{\mathrm{adapter}}. \tag*{(14)} \end{aligned}

with μblock=0.05\mu_{\mathrm{block}}=0.05 and μhead=0.02\mu_{\mathrm{head}}=0.02. The two coefficients ramp linearly from zero during the first 20% of Stage II updates. Adapter, selected shared stages, and the output head use learning-rate multipliers 1.0, 0.02, and 0.05, respectively. This state-dominant parameter routing limits interference between auxiliary response learning and the primary state objective.

Standard single-input state inference requires one backbone forward, while intrinsic paired-response estimation uses two shared forwards and output differencing. Forward-path verification records zero adapter calls in both modes; auxiliary estimation alone evaluates the selected adapters. Offline pairing reuses anchor labels and queries perturbed endpoint states without differentiating through the reference solver.

Experiments

Experimental Setup

We evaluate four representative PDE regimes. DiffReact captures nonlinear reaction–diffusion dynamics with spatial smoothing and coupled local reactions; Kolmogorov flow represents forced nonlinear fluid dynamics with multi-scale interactions and long-horizon instability; Burgers isolates nonlinear conservation-law behavior with steep gradients and shock-like transport; and Advection isolates phase propagation and transport alignment. Together, these tasks span smoothing, multi-scale, sharp-gradient, and phase-sensitive regimes emphasized by modern scientific-ML benchmarks and operator-learning analyses [25, 10, 27].

The breadth study evaluates FNO, RNO (Ye, Zhang, and Wang 2025), CViT [28], MambaNO [32], and LaMO [26] under matched task-specific protocols; Table 3 provides additional budget-matched controls for FNO. Core and Two-point MSE share the same (z,z+,u,u+)(z, z^+, u, u^+) pairs, splits, seeds, and endpoint states. MSE-2N matches the state count, while Pair-shuffled retains endpoint supervision but breaks correspondence. We report state relative L2L_2, response cosine, response relative L2L_2, magnitude error, and Rollout@20. Unless noted, entries are three-seed means. Per-seed results, sample variances, standard deviations, 95% confidence intervals, effect sizes, paired tests, bootstrap analyses, and multiple-comparison-adjusted pp-values are reported in the appendix.

MetricMSE-1×MSE-2NTwo-pointCore
DiffReact
State ↓0.0094520.0094450.0092230.005908
Resp. cos. ↑0.29350.29440.29600.9308
Resp. rel. L2L_2 ↓0.97730.97680.97330.3758
Mag. err. ↓1.1041.1021.0500.0997
Roll.@20 ↓0.0046790.0046870.0044400.003573
Kolmogorov
State ↓0.28100.28100.28100.2846
Resp. cos. ↑0.72040.72040.72050.7528
Resp. rel. L2L_2 ↓4.0534.0524.0523.918
Mag. err. ↓1.1081.1081.1081.095
Roll.@20 ↓0.29600.29600.29590.2945

Table 3. Budget-matched FNO controls. MSE-2N matches the state count; Two-point uses the same paired endpoints and reference-query budget as Core but omits response supervision.

Perturbation protocols. Each task uses a benchmark-specific admissible perturbation protocol. Table 2 reports the frozen perturbation step, valid train/validation/test pair counts, and the fraction of all training anchors receiving paired supervision. Training and evaluation use disjoint sampled pairs; unseen-direction and altered-step evaluations are reported separately from the in-distribution results.

TaskStepPairsρtr\rho_{\mathrm{tr}}
DiffReactϵ=2.391×10−3\epsilon= 2.391 \times10^{-3}200/50/100100%
Kolmogorovϵ=1.0×10−2\epsilon= 1.0 \times10^{-2}200/50/100100%
ϵrel=0.05\epsilon_{\mathrm{rel}} = 0.05
Burgersϵeff=2.387×10−2\epsilon_{\mathrm{eff}} = 2.387 \times10^{-2}1638/204/20616.38%
ϵrel=0.05\epsilon_{\mathrm{rel}} = 0.05
Advectionϵˉeff=2.387×10−2\bar{\epsilon}_{\mathrm{eff}} = 2.387 \times10^{-2}1638/204/20616.38%

Table 2. Benchmark-specific perturbation protocols. Pairs reports valid train/validation/test pair counts, and ρtr\rho_{\mathrm{tr}} is the fraction of all training anchors receiving paired supervision. Constructing one perturbed endpoint per pair requires 350 additional reference-state queries for DiffReact and Kolmogorov and 2,048 for Burgers and Advection.

We organize the evaluation from breadth to mechanism.

Table 1 establishes transfer across systems and backbones. Table 3 controls observation count and independent two-endpoint fitting, while Table 4 separates direct response regression, direction–magnitude decomposition, and pair correspondence. Finally, Table 5 evaluates LSTH across shared-state, intrinsic-response, and auxiliary-response metrics. The control tables use their respective matched protocols, and their absolute values are interpreted within each table rather than directly across tables.

SettingState ↓Resp. cos. ↑Roll.@20 ↓
Objective composition: DiffReact
State only0.0085360.53600.0059
State + Direct Resp. MSE0.0080950.59530.0050
State + Dir.0.0071070.96400.0044
State + Mag.0.0087150.43110.0065
Full Core0.0063160.96300.0044
Objective composition: Kolmogorov
State only0.2234950.67750.3959
State + Direct Resp. MSE0.1795530.50730.3350
State + Dir.0.1796430.50890.3351
State + Mag.0.1796480.50770.3351
Full Core0.1796830.50900.3351
Pair correspondence: DiffReact
State baseline0.0082670.53490.0057
Pair-shuffled0.0067970.92970.0050
True Core0.0063160.96300.0044
Pair correspondence: Kolmogorov
State baseline0.2229420.67830.3936
Pair-shuffled0.1796250.50610.3350
True Core0.1795740.50680.3350

Table 4. RNO mechanism controls. Objective rows isolate response regression, direction, and magnitude terms; pair rows break or retain endpoint correspondence.

VariantState ↓Intr. resp. ↑Aux. resp. ↑Roll. ↓
DiffReact
Static0.00590.9433–0.0035
State FT0.00320.9410–0.0018
Weak FT0.00370.9440–0.0021
LSTH_0.0036_0.94310.9779_0.0019_
Kolmogorov
Static0.28090.7572–0.2941
State FT0.27490.7281–0.2887
Weak FT_0.2795__0.7525_–_0.2886_
LSTH0.27490.72810.90710.2885

Table 5. Layer-selective response-aware fine-tuning. Entries are three-seed means; standard deviations and confidence intervals are reported in the appendix. Intrinsic response is obtained from the shared state predictor for every variant, while auxiliary response is defined only for LSTH.

Cross-System and Cross-Architecture Results

Table 1 compares Aug-only with Core across 20 task–backbone settings. Core lowers state error and Rollout@20 in 18 settings, showing broad but non-uniform transfer across architectures and PDE regimes. Intrinsic-response gains are strongest on DiffReact and remain heterogeneous across the other systems and backbones. The aggregate pattern is therefore not a claim that response supervision uniformly improves every measured quantity; rather, it shows that paired finite-step supervision is broadly compatible with different operator parameterizations while preserving the same state-query interface. The mixed response behavior on transport-dominated systems motivates the matched controls below, which separate endpoint coverage, pair correspondence, and explicit response geometry.

TaskBackboneAug-only State ↓Aug-only Resp. cos. ↑Aug-only Roll.@20 ↓TangentPDE-Core State ↓TangentPDE-Core Resp. cos. ↑TangentPDE-Core Roll.@20 ↓
DiffReactFNO0.0075290.43670.004130.0059390.93580.00336
DiffReactCViT0.0115890.14110.006900.0093980.47210.00590
DiffReactLaMO0.0079060.24550.003440.0071880.48550.00348
DiffReactMambaNO0.0121050.07220.008300.0104330.93450.00740
DiffReactRNO0.0084520.54310.005800.0063160.96300.00440
KolmogorovFNO0.2371840.73920.227470.2371430.77980.22030
KolmogorovCViT0.4140420.30750.430900.3536820.31920.26900
KolmogorovLaMO0.2130510.41870.384500.2001360.40000.36410
KolmogorovMambaNO0.4881240.61890.806000.4836790.64930.79490
KolmogorovRNO0.2232500.67690.397000.1793500.50740.33470
BurgersFNO0.1566020.39010.155020.1577550.45330.15108
BurgersCViT0.2932700.20840.250000.2656380.21910.24610
BurgersLaMO0.2876900.20640.254200.2869060.20560.25270
BurgersMambaNO0.6833630.12630.727600.6356930.10770.68070
BurgersRNO0.3216150.19890.282900.2897940.22880.25390
AdvectionFNO0.1293360.08990.030900.1350710.08810.06828
AdvectionCViT0.5461220.04710.061600.4837500.04540.05960
AdvectionLaMO0.5573640.04920.094000.5551040.04750.07730
AdvectionMambaNO0.7871170.04550.126100.7759740.04030.09050
AdvectionRNO0.6707290.01350.656000.6366640.01840.60530

Table 1. Cross-system and cross-architecture results under the matched evaluation protocol. State is relative L2L_2 error, response is intrinsic finite-step response cosine, and Roll.@20 is the 20-step rollout error. Bold marks the better value within each matched Aug-only/Core pair.

Same-data and budget controls. Table 3 separates gains from additional state observations, matched endpoint fitting, and explicit finite-step response supervision. On DiffReact, MSE-2N and Two-point provide only limited improvements over the state baseline, whereas Core substantially improves all five reported metrics. This establishes a clear contribution from the relational objective beyond state count and paired endpoint reconstruction. On Kolmogorov, the three state-only or endpoint-only controls attain nearly identical one-step state errors. Core slightly increases state error but improves response cosine, response relative L2L_2, magnitude calibration, and Rollout@20, revealing a task-dependent state–response trade-off. The comparison also distinguishes the role of correspondence from the role of raw endpoint coverage: using the same two endpoint states without supervising their finite-step relation is not sufficient to reproduce the Core response gains. We therefore interpret the budget controls as a conservative check that the observed improvements are tied to relational supervision rather than to a larger state-fitting dataset.

Objective and correspondence analysis. Table 4 separates direct response regression, the direction–magnitude decomposition, and pair correspondence. State + Direct Resp. MSE is the undecomposed response-learning baseline and fits the full response vector with a single Euclidean objective. On DiffReact, direct response regression improves over state-only continuation but remains far below direction supervision in response cosine; Full Core gives the lowest state error while matching the best rollout. On Kolmogorov, Direct Resp. MSE gives the lowest state and rollout errors, whereas the response variants remain below the state-only response cosine. The pair controls further show that exact correspondence has clear marginal value on DiffReact but only a small incremental effect over the paired state preserving protocol on Kolmogorov.

Sparse paired supervision. Figure 2 jointly evaluates state and intrinsic-response pair efficiency on DiffReact. Two-point MSE changes only modestly with the paired-anchor budget. In contrast, TangentPDE-Core rapidly improves intrinsic response alignment and continues to reduce state error as more paired anchors are introduced. The distinct response and state trends show that the relational objective affects both local output variation and endpoint prediction, with different sensitivities to pair coverage. The response curve improves most sharply at small budgets, indicating that a limited number of paired secants can already provide a useful local direction signal. The state curve changes more gradually, which is consistent with state prediction depending on both endpoint coverage and the regularization induced by the response objective.

Figure 2. Pair efficiency on DiffReact. State relative L2L_2 error and intrinsic finite-step response cosine are reported as paired training anchors increase. Two-point MSE fits endpoint states independently, whereas TangentPDE-Core also supervises their finite-step relation.

Layer-selective tangent head. Table 5 separates changes in the shared state predictor from gains of the dedicated auxiliary response path. LSTH markedly improves auxiliary response alignment, while its intrinsic response remains task dependent. This confirms that the head primarily provides a specialized paired-response estimate rather than uniformly modifying the state operator’s intrinsic finite-step response.

Across the two FNO configurations, LSTH adds 8.77–26.52% parameters, while auxiliary-response evaluation adds 12.44–16.21% latency over intrinsic paired-response inference.

Figure 3. Response-field comparison for a representative paired query. Reference finite-step responses are shown against Two-point MSE and TangentPDE-Core predictions for the two output channels; cosine values are computed against the corresponding reference response field. This qualitative example complements the aggregate mechanism metrics in Table 4.

Discussion and Limitations

What TangentPDE learns. Taken together, the experiments establish TangentPDE as a finite-step response-learning framework whose relational supervision also regularizes state learning. Core improves state fidelity and rollout broadly, while the matched controls isolate contributions beyond additional state observations and independent endpoint reconstruction. Explicit response geometry provides the clearest additional value on sensitivity-relevant systems, and the response-MSE comparison shows when the direction–magnitude decomposition improves over undecomposed vector regression.

Task-dependent response behavior. Response supervision exhibits distinct behavior across the evaluated PDE systems. DiffReact shows the strongest and most consistent intrinsic-response gains, whereas Burgers and Advection retain state or rollout improvements in many settings but show weaker or architecture-dependent response changes. On Kolmogorov, the balance among state, intrinsic response, and rollout also varies across backbones. These results motivate task- and architecture-specific validation of response objectives.

LSTH semantics and deployment. LSTH improves a dedicated auxiliary response estimate by reusing selected hidden finite differences, while the intrinsic response remains defined by the shared state predictor’s output difference. The two quantities are therefore reported separately. The adapters are bypassed during standard state and intrinsic-response inference and execute only for the auxiliary estimate. Full correction remains task dependent, and transport settings can over-correct when the auxiliary residual dominates the intrinsic response.

Scope, solver interface, and reporting. TangentPDE learns finite-step secants along sampled perturbation families rather than a complete Jacobian. The state-query-only interface broadens applicability to solvers without derivative, tangent, or adjoint APIs, but wall-clock cost relative to JVP, tangent, or adjoint data generation remains solver dependent. When anchor labels are reused, each new pair typically requires one additional perturbed-state solve; the exact call count and measured runtime are reported from the pair-generation manifests. The appendix provides per-seed values, variances, confidence intervals, effect sizes, corrected significance tests, response masks, perturbation protocols, hyperparameters, and solver-query accounting. Claims about unseen directions and altered step sizes are restricted to the evaluated perturbation families.

Conclusion

TangentPDE identifies the cross-endpoint error increment as the missing relational quantity in finite-accuracy state fitting and turns paired state queries into direct finite-step response supervision. TangentPDE-Core preserves the state-prediction interface while improving state fidelity and rollout across diverse PDE regimes and backbones, and controlled comparisons isolate the value of finite-step geometry beyond additional endpoint observations. The direction–magnitude decomposition separates geometric alignment from scale calibration, while LSTH extends the framework with a dedicated auxiliary response learner whose output remains distinct from the intrinsic state-output response. Together, these results establish paired finite-step response learning as a practical supervision regime between state-only fitting and explicit derivative-informed training.

Reproducibility and Evidence Scope

This supplement distinguishes three evidence families that serve different purposes. First, the submitted task-specific results reproduce the point estimates used in the main paper. Second, independent uniform-protocol reruns test whether the qualitative behavior persists under one common training schedule, but they are not used to reconstruct uncertainty around the submitted point estimates. Third, matched controls and diagnostic studies isolate specific competing explanations or probe limited robustness properties. These evidence families are reported separately and should not be treated as interchangeable estimates of the same quantity.

All reported results use three random seeds, 1234, 2025, and 3407, unless stated otherwise. The controlled FNO and RNO experiments use Adam for 50 epochs with batch size 4 and an initial learning rate of 10−310^{-3}. Model selection minimizes validation state relative L2L_2 error; test metrics are computed once from the selected checkpoint. TangentPDE-Core uses λdir=0.1\lambda_{\mathrm{dir}} = 0.1, λmag=0.01\lambda_{\mathrm{mag}} = 0.01, and δ=10−8\delta= 10^{-8}, matching the frozen main-text objective. These controlled runs use zero weight decay, full-precision training, no learning-rate scheduler, and no gradient clipping. Loss warmup is disabled for Core and its matched controls. Any independently rerun protocol that uses a different continuation schedule is identified explicitly in its own subsection.

The breadth study compares FNO, CViT, LaMO, MambaNO, and RNO using the same task split and evaluation interface for Aug-only and TangentPDE-Core. Within each task–backbone pair, the two methods use the same state-predictor architecture and differ only in the training objective. Core and Two-point MSE receive identical paired endpoints. MSE-2N matches the number of state observations, while the pair-shuffled control retains the endpoint-state multiset and optimization budget but disrupts the declared endpoint correspondence.

Unless a table caption states otherwise, boldface identifies the better point estimate within the declared comparison and does not by itself imply statistical significance. A dash denotes an undefined or inapplicable quantity rather than a numerical zero.

Tables A1 and A2 give the architecture and optimization details for the uniform-protocol breadth study. The task-dependent output dimension is kept explicit because each model predicts the complete trajectory rather than a single shared-size output tensor.

Datasets and Perturbation Protocols

Pairs are constructed only after the anchor split, so training, validation, and test anchors remain disjoint before any perturbation is generated. This ordering prevents a perturbed endpoint derived from one split from being used as an anchor or label in another split. For each valid anchor zz, a perturbation protocol (v,ϵ)(v,\epsilon) defines an admissible perturbed endpoint z+=T(z,v,ϵ)∈Zadmz^{+} = T(z,v,\epsilon) \in\mathcal{Z}_{\mathrm{adm}}. Following the frozen main-text definition, T(z,v,ϵ)=z+ϵvT(z,v,\epsilon) = z + \epsilon v when no task-specific transform is required. The reference solver is queried at z+z^{+} while the existing label at zz is reused. Thus, one accepted pair adds exactly one perturbed-endpoint query rather than two new solver calls. Table A18 separates training-pair coverage from coverage over all available anchors.

TaskPairsρtr\rho_{\mathrm{tr}}OverallCalls
DiffReact200/50/100100%100%350
Kolmogorov200/50/100100%100%350
Burgers1638/204/20616.38%20.48%2,048
Advection1638/204/20616.38%20.48%2,048

Table A18. Perturbation and reference-query accounting. ρtr\rho_{\mathrm{tr}} is the fraction of training anchors receiving paired supervision; Overall paired is the fraction over all available anchors.

DiffReact uses coupled reaction–diffusion perturbations at ϵ=2.391×10−3\epsilon= 2.391 \times10^{-3}, and Kolmogorov uses admissible flow perturbations at ϵ=10−2\epsilon= 10^{-2}. Burgers and Advection use normalized random Fourier perturbations with relative step 0.050.05; their mean effective step is 2.387×10−22.387 \times10^{-2}. These task-specific constructions define the finite-step response families evaluated in this work; conclusions about unseen directions or altered step sizes are restricted to the dedicated studies reported later.

Evaluation Metrics and Statistical Reporting

Let GG denote the learned state operator and let (z,z+,u,u+)(z,z^{+},u,u^{+}) be a paired query. The intrinsic predicted and reference responses are

rop=G(z+)−G(z)ϵ,r⋆=u+−uϵ.(15)r_{\mathrm{op}} = \frac{G(z^{+}) - G(z)}{\epsilon}, \qquad r^{\star} = \frac{u^{+} - u}{\epsilon}. \tag*{(15)}

The term intrinsic refers to the response induced by the shared state predictor itself; it is distinct from the auxiliary response estimate introduced later for LSTH. These quantities are finite-step secants under the benchmark-defined perturbation protocol, not estimates of a complete Jacobian.

For clarity, the frozen TangentPDE-Core training objective is reproduced here:

Lstate=MSE⁡(Gθ(z),u)+MSE⁡(Gθ(z+),u+),(16)\mathcal{L}_{\mathrm{state}} = \operatorname{MSE}(G_{\theta}(z),u) + \operatorname{MSE}(G_{\theta}(z^{+}),u^{+}), \tag*{(16)}
Ldir(a,b)=1−⟨a,b⟩∥a∥2∥b∥2+δ,(17)\mathcal{L}_{\mathrm{dir}}(a,b) = 1 - \frac{\langle a,b\rangle}{\lVert a\rVert_{2}\lVert b\rVert_{2}+\delta}, \tag*{(17)}
Lmag(a,b)=∣log⁡(∥a∥2+δ)−log⁡(∥b∥2+δ)∣,(18)\mathcal{L}_{\mathrm{mag}}(a,b) = \left\lvert\log(\lVert a\rVert_{2}+\delta) - \log(\lVert b\rVert_{2}+\delta) \right\rvert, \tag*{(18)}
LCore=Lstate+λdirLdir(rop,r⋆)+λmagLmag(rop,r⋆).(19)\begin{aligned} \mathcal{L}_{\mathrm{Core}} &= \mathcal{L}_{\mathrm{state}} + \lambda_{\mathrm{dir}}\mathcal{L}_{\mathrm{dir}}(r_{\mathrm{op}},r^{\star}) \\ &\quad+ \lambda_{\mathrm{mag}}\mathcal{L}_{\mathrm{mag}}(r_{\mathrm{op}},r^{\star}). \tag*{(19)} \end{aligned}

The quantities below are reported evaluation metrics. They should not be conflated with the training losses even when they use the same response pair.

State error is the sample-wise relative norm

Estate=∥G(z)−u∥2∥u∥2+δ.(20)E_{\mathrm{state}} = \frac{\lVert G(z)-u\rVert_{2}}{\lVert u\rVert_{2}+\delta}. \tag*{(20)}

Response cosine, relative response error, and magnitude error are

Cresp=⟨rop,r⋆⟩∥rop∥2∥r⋆∥2+δ,(21)C_{\mathrm{resp}} = \frac{\langle r_{\mathrm{op}},r^{\star}\rangle}{\lVert r_{\mathrm{op}}\rVert_{2}\lVert r^{\star}\rVert_{2}+\delta}, \tag*{(21)}
Eresp=∥rop−r⋆∥2∥r⋆∥2+δ,(22)E_{\mathrm{resp}} = \frac{\lVert r_{\mathrm{op}}-r^{\star}\rVert_{2}}{\lVert r^{\star}\rVert_{2}+\delta}, \tag*{(22)}
Emag=∣log⁡(∥rop∥2+δ)−log⁡(∥r⋆∥2+δ)∣.(23)E_{\mathrm{mag}} = \left\lvert\log(\lVert r_{\mathrm{op}}\rVert_{2}+\delta) - \log(\lVert r^{\star}\rVert_{2}+\delta) \right\rvert. \tag*{(23)}

Metrics are computed per sample and then averaged. Near-zero reference responses are excluded from directional metrics using the response-norm threshold δ\delta. Rollout@20 is the state relative L2L_2 error after 20 autoregressive prediction steps. Where uncertainty is shown, entries are the mean and sample standard deviation across the three seeds.

BackboneOptimizerLRBatchEpochsSchedulerWeight decayGrad. clipPrecisionCheckpoint selection
FNOAdam10−310^{-3}450None0NoneFP32Lowest validation state relative L2L_2
CViTAdam10−310^{-3}450None0NoneFP32Lowest validation state relative L2L_2
LaMOAdam10−310^{-3}450None0NoneFP32Lowest validation state relative L2L_2
MambaNOAdam10−310^{-3}450None0NoneFP32Lowest validation state relative L2L_2
RNOAdam10−310^{-3}450None0NoneFP32Lowest validation state relative L2L_2

Table A2. Training configuration for the uniform-protocol breadth study. The same optimization and checkpoint-selection settings are used for Aug-only and TangentPDE-Core. “None” means that the corresponding operation is disabled.

Seed-level standard deviations describe variation due to training randomness under a fixed configuration; they are not interpreted as independent test-sample uncertainty. The across-setting analysis reported later instead treats a complete task–backbone setting as the paired resampling and testing unit. These two uncertainty summaries answer different questions and are therefore reported separately.

Table A3 reports the realized mask statistics on the frozen test pairs. No test response is removed at the fixed threshold. The mask is applied only to response cosine; response relative L2L_2 and magnitude error remain defined through the fixed stabilizer δ\delta and are not removed by this directional mask. State and rollout metrics do not use this mask.

TaskTotalValidExcludedValid ratioMinMedianP95
DiffReact10010001.0000102.7155.3260.8
Kolmogorov10010001.00009.05937.39150
Burgers20620601.00001.2533.0417.492
Advection20620601.000015.753.24217.7

Table A3. Sample-level response-mask statistics on the frozen test pairs. Directional response cosine excludes samples whose reference-response norm ∥r⋆∥2\lVert r^\star\rVert_2 is below the fixed threshold δ=10−8\delta= 10^{-8}. Min, median, and P95 summarize the sample-level distribution of ∥r⋆∥2\lVert r^\star\rVert_2.

Cross-System Results and Evidence Roles

Submitted Task-Wise Results

Tables A4–A7 reproduce the submitted cross-system comparison by task. These tables establish the primary task–backbone observations used in the main paper. They show broad state and rollout improvements while retaining the task- and architecture-dependent response behavior emphasized in the main text. DiffReact exhibits the most consistent response gains. Kolmogorov shows broader state and rollout improvements but a mixed intrinsic-response pattern across backbones. Burgers and Advection further show that response supervision is not uniformly beneficial for every architecture and metric.

The task-wise tables should therefore be read as evidence for a backbone-dependent training effect rather than as a claim of universal intrinsic-response improvement. In particular, a better state or rollout value does not imply that the same state predictor also has a better intrinsic finite-step response.

Supplementary Magnitude Diagnostics

Tables A8–A11 retain the submitted state, intrinsic finite-step response cosine, and Rollout@20 means, and append an independent three-seed magnitude-error diagnostic. The supplementary magnitude statistics describe only the added diagnostic and are not uncertainty estimates for the submitted state, response-cosine, or rollout values.

These tables therefore separate the identity of the main-paper quantities from the identity of the added diagnostic column. Negative and mixed cells are retained as task- and backbone-dependent evidence rather than converted into a claim that every metric persists under one common schedule.

Across-Setting Paired Analysis

Figure A4 summarizes the 20 matched means reported in the main paper. The comparison unit is one complete task–backbone setting, preserving the paired Aug-only/Core comparison within that setting. The reported changes use metric-specific improvement directions: Δstate=Estate(Aug)−Estate(Core)\Delta_{\mathrm{state}} = E_{\mathrm{state}}(\mathrm{Aug}) - E_{\mathrm{state}}(\mathrm{Core}), Δresp=Cresp(Core)−Cresp(Aug)\Delta_{\mathrm{resp}} = C_{\mathrm{resp}}(\mathrm{Core}) - C_{\mathrm{resp}}(\mathrm{Aug}), and Δroll=Eroll(Aug)−Eroll(Core)\Delta_{\mathrm{roll}} = E_{\mathrm{roll}}(\mathrm{Aug}) - E_{\mathrm{roll}}(\mathrm{Core}). Under these definitions, positive values consistently denote improvement by TangentPDE-Core. The underlying paired checks retain the same intervals and rank-based summaries as the source table, while the figure foregrounds the descriptive cross-setting trend.

Figure A4. Across-setting paired summary over the 20 submitted task–backbone settings. Points show mean metric-specific Core-over-Aug improvement; annotations report improved-setting counts. Positive values indicate improvement, and the figure is descriptive rather than a standalone inferential display.

Figure A1. Task-wise improvement coverage across the five submitted backbones. Bars count the task–backbone pairs for which TangentPDE-Core has the better point estimate than Aug-only for State, intrinsic response cosine, and Rollout@20. The figure summarizes the submitted task-wise tables and preserves the mixed response behavior; counts do not imply statistical significance.

Figure A2. Kolmogorov intrinsic-response fields for a representative canonical FNO test query. Signed responses share one zero-centered color scale, and the two absolute-error panels share a second scale. The query-selection rule is median Core-minus-Aug-only response-cosine change over the frozen seed-1234 test set. The heatmaps show the time slice with the largest reference-response norm. Annotated response cosine CC, relative response error ErE_r, and magnitude error EmE_m are full-trajectory sample metrics computed using the same definitions as the aggregate tables; they are not three-seed means.

This analysis summarizes whether improvements are systematic across the submitted settings. It does not treat grid points, time steps, or random seeds as independent observations, and it does not convert a mixed per-backbone response pattern into a universal claim.

Controlled Ablations and Competing Explanations

Table A13 tests whether the observed gains can be explained solely by seeing more endpoint states or by using a larger reference-query budget. MSE-2N matches the number of supervised state observations, while Two-point MSE uses the same paired endpoints and solver-query budget as Core but omits response supervision. On DiffReact, Core improves all five metrics beyond these controls. On Kolmogorov, the response and rollout gains are accompanied by a small state-error trade-off.

MetricMSE-1×MSE-2NTwo-pointCore
DiffReact
State ↓\downarrow0.0094520.0094450.0092230.005908
Resp. cos. ↑\uparrow0.29350.29440.29600.9308
Resp. rel. L2L_{2} ↓\downarrow0.97730.97680.97330.3758
Mag. err. ↓\downarrow1.1041.1021.0500.0997
Roll.@20 ↓\downarrow0.0046790.0046870.0044400.003573
Kolmogorov
State ↓\downarrow0.28100.28100.28100.2846
Resp. cos. ↑\uparrow0.72040.72040.72050.7528
Resp. rel. L2L_{2} ↓\downarrow4.0534.0524.0523.918
Mag. err. ↓\downarrow1.1081.1081.1081.095
Roll.@20 ↓\downarrow0.29600.29600.29590.2945

Table A13. Budget-matched FNO controls. MSE-2N matches the state count; Two-point uses the same paired endpoints and reference-query budget as Core but omits response supervision.

Table A14 separates direct response-vector regression

SettingState ↓\downarrowResp. cos. ↑\uparrowRoll.@20 ↓\downarrow
Objective composition: DiffReact
State only0.0085360.53600.0059
State + Direct Resp. MSE0.0080950.59530.0050
State + Dir.0.0071070.96400.0044
State + Mag.0.0087150.43110.0065
Full Core0.0063160.96300.0044
Objective composition: Kolmogorov
State only0.2234950.67750.3959
State + Direct Resp. MSE0.1795530.50730.3350
State + Dir.0.1796430.50890.3351
State + Mag.0.1796480.50770.3351
Full Core0.1796830.50900.3351
Pair correspondence: DiffReact
State baseline0.0082670.53490.0057
Pair-shuffled0.0067970.92970.0050
True Core0.0063160.96300.0044
Pair correspondence: Kolmogorov
State baseline0.2229420.67830.3936
Pair-shuffled0.1796250.50610.3350
True Core0.1795740.50680.3350

Table A14. RNO mechanism controls. Objective rows isolate direct response regression, direction, and magnitude terms; pair rows break or retain endpoint correspondence.

@page 12/23 starts_mid=1 ends_mid=1

TaskTotalValidExcludedValid ratioMinMedianP95
DiffReact10010001.0000102.7155.3260.8
Kolmogorov10010001.00009.05937.39150
Burgers20620601.00001.2533.0417.492
Advection20620601.000015.753.24217.7

Table A3 (PDF p. 12). Sample-level response-mask statistics on the frozen test pairs. Directional response cosine excludes samples whose reference-response norm ∥r⋆∥2\lVert r^\star\rVert_2 is below the fixed threshold δ=10−8\delta= 10^{-8}. Min, median, and P95 summarize the sample-level distribution of ∥r⋆∥2\lVert r^\star\rVert_2.

Figure A1 (PDF p. 12). Task-wise improvement coverage across the five submitted backbones. Bars count the task–backbone pairs for which TangentPDE-Core has the better point estimate than Aug-only for State, intrinsic response cosine, and Rollout@20. The figure summarizes the submitted task-wise tables and preserves the mixed response behavior; counts do not imply statistical significance.

Figure A2 (PDF p. 12). Kolmogorov intrinsic-response fields for a representative canonical FNO test query. Signed responses share one zero-centered color scale, and the two absolute-error panels share a second scale. The query-selection rule is median Core-minus-Aug-only response-cosine change over the frozen seed-1234 test set. The heatmaps show the time slice with the largest reference-response norm. Annotated response cosine CC, relative response error ErE_r, and magnitude error EmE_m are full-trajectory sample metrics computed using the same definitions as the aggregate tables; they are not three-seed means.

TangentPDE-Core. The underlying paired checks retain the same intervals and rank-based summaries as the source table, while the figure foregrounds the descriptive cross-setting trend.

This analysis summarizes whether improvements are systematic across the submitted settings. It does not treat grid points, time steps, or random seeds as independent observations, and it does not convert a mixed per-backbone response pattern into a universal claim.

Controlled Ablations and Competing Explanations

Table A13 tests whether the observed gains can be explained solely by seeing more endpoint states or by using a larger reference-query budget. MSE-2N matches the number of supervised state observations, while Two-point MSE uses the same paired endpoints and solver-query budget as Core but omits response supervision. On DiffReact, Core improves all five metrics beyond these controls. On Kolmogorov, the response and rollout gains are accompanied by a small state-error trade-off.

Table A14 separates direct response-vector regression

Figure A3. Space–time state trajectories for representative Burgers and Advection queries from the independent uniform-protocol FNO reruns. Rows use task-specific shared state and absolute-error color scales. The annotations report sample-level full-trajectory state relative L2L_2 error EsE_s and Rollout@20 error R20R_{20} using the same definitions as the aggregate tables. For each task, the displayed query has Core-minus-Aug-only state improvement closest to the median over the frozen seed-1234 test set. These diagnostic rerun fields do not alter the submitted task-wise point estimates.

ModelMethodState ↓Resp. ↑Roll. ↓
CViTAug-only0.0115890.14110.00690
CViTTangentPDE-Core0.0093980.47210.00590
FNOAug-only0.0075290.43670.00413
FNOTangentPDE-Core0.0059390.93580.00336
LaMOAug-only0.0079060.24550.00344
LaMOTangentPDE-Core0.0071880.48550.00348
MambaNOAug-only0.0121050.07220.00830
MambaNOTangentPDE-Core0.0104330.93450.00740
RNOAug-only0.0084520.54310.00580
RNOTangentPDE-Core0.0063160.96300.00440

Table A4. Expanded DiffReact results under the matched evaluation protocol. Bold marks the better value within each Aug-only/Core pair.

ModelMethodState ↓Resp. ↑Roll. ↓
CViTAug-only0.4140420.30750.43090
CViTTangentPDE-Core0.3536820.31920.26900
FNOAug-only0.2371840.73920.22747
FNOTangentPDE-Core0.2371430.77980.22030
LaMOAug-only0.2130510.41870.38450
LaMOTangentPDE-Core0.2001360.40000.36410
MambaNOAug-only0.4881240.61890.80600
MambaNOTangentPDE-Core0.4836790.64930.79490
RNOAug-only0.2232500.67690.39700
RNOTangentPDE-Core0.1793500.50740.33470

Table A5. Expanded Kolmogorov results under the matched evaluation protocol. Bold marks the better value within each Aug-only/Core pair.

ModelMethodState ↓Resp. ↑Roll. ↓
CViTAug-only0.2932700.20840.25000
TangentPDE-Core0.2656380.21910.24610
FNOAug-only0.1566020.39010.15502
TangentPDE-Core0.1577550.45330.15108
LaMOAug-only0.2876900.20640.25420
TangentPDE-Core0.2869060.20560.25270
MambaNOAug-only0.6833630.12630.72760
TangentPDE-Core0.6356930.10770.68070
RNOAug-only0.3216150.19890.28290
TangentPDE-Core0.2897940.22880.25390

Table A6. Expanded Burgers results under the matched evaluation protocol. Bold marks the better value within each Aug-only/Core pair.

ModelMethodState ↓Resp. ↑Roll. ↓
CViTAug-only0.5461220.04710.06160
TangentPDE-Core0.4837500.04540.05960
FNOAug-only0.1293360.08990.03090
TangentPDE-Core0.1350710.08810.06828
LaMOAug-only0.5573640.04920.09400
TangentPDE-Core0.5551040.04750.07730
MambaNOAug-only0.7871170.04550.12610
TangentPDE-Core0.7759740.04030.09050
RNOAug-only0.6707290.01350.65600
TangentPDE-Core0.6366640.01840.60530

Table A7. Expanded Advection results under the matched evaluation protocol. Bold marks the better value within each Aug-only/Core pair.

from the direction–magnitude decomposition. Direct Response MSE applies one Euclidean objective to the undecomposed finite-step response vector. The objective rows therefore test whether the decomposition itself matters, while the pair rows test whether exact endpoint correspondence contributes beyond observing the same endpoint states.

The pair-shuffled control is interpreted narrowly. It preserves the available endpoint states, pair count, and optimization budget while disrupting the declared relational correspondence. Consequently, it tests whether correct pair structure contributes beyond a response-shaped regularization effect; it does not establish that every mismatch is uninformative or that the same mechanism must hold for every task and backbone.

Figure A5 visualizes the seed-level objective composition rows from the same RNO mechanism study. The plot emphasizes the relative behavior of the objective variants on DiffReact and Kolmogorov, while the source values are kept unchanged.

Figure A5. Objective-composition study on DiffReact and Kolmogorov. Points show three-seed means with sample-standard-deviation error bars for State, intrinsic response cosine, and Rollout@20.

Layer-Selective Tangent Head: Semantics and Diagnostics

Table A16 distinguishes the intrinsic response of the shared state predictor from the auxiliary response estimate produced by LSTH. Let Gθ′G_{\theta^{\prime}} denote the shared state predictor after LSTH continuation. The frozen main-text semantics are

VariantState ↓Intr. resp. ↑Aux. resp. ↑Roll. ↓
DiffReact
Static0.00590.9433–0.0035
State FT0.00320.9410–0.0018
Weak FT0.00370.9440–0.0021
LSTH_0.0036_0.94310.9779_0.0019_
Kolmogorov
Static0.28090.7572–0.2941
State FT0.27490.7281–0.2887
Weak FT_0.2795__0.7525_–_0.2886_
LSTH0.27490.72810.90710.2885

Table A16. Layer-selective response-aware fine-tuning. Intrinsic response is obtained from the shared state predictor, while auxiliary response is defined only for LSTH.

rop=Gθ′(z+)−Gθ′(z)ϵ,(24)r_{\mathrm{op}} = \frac{G_{\theta^{\prime}}(z^{+}) - G_{\theta^{\prime}}(z)}{\epsilon}, \tag*{(24)}
Δhk=hk(z+)−hk(z)ϵ,(25)\Delta h_k = \frac{h_k(z^{+}) - h_k(z)}{\epsilon}, \tag*{(25)}
cϕ=γ∑k∈KAk(Δhk),(26)c_{\phi} = \gamma\sum_{k \in\mathcal{K}} A_k(\Delta h_k), \tag*{(26)}
raux=rop+cϕ.(27)r_{\mathrm{aux}} = r_{\mathrm{op}} + c_{\phi}. \tag*{(27)}

The correction is not added to either state output. The auxiliary metric is defined only for LSTH and is therefore not compared with an implicitly substituted intrinsic metric. Standard state prediction and intrinsic-response inference bypass the adapters; the selected hidden-difference adapters execute only when the auxiliary response estimate is requested.

LSTH reuses the Core response losses and regularizes the auxiliary correction through

Lcorr=1B∑i=1B(∥cϕ,i∥2∥rop,i∥2+δ)2,(28)\mathcal{L}_{\mathrm{corr}} = \frac{1}{B} \sum_{i=1}^{B} \left(\frac{\lVert c_{\phi,i} \rVert_2}{\lVert r_{\mathrm{op},i} \rVert_2 + \delta}\right)^2, \tag*{(28)}
Ladapter=0.1Ldir(raux,r⋆)+0.01Lmag(raux,r⋆)+10−4Lcorr.(29)\mathcal{L}_{\mathrm{adapter}} = 0.1\mathcal{L}_{\mathrm{dir}}(r_{\mathrm{aux}},r^{\star}) + 0.01\mathcal{L}_{\mathrm{mag}}(r_{\mathrm{aux}},r^{\star}) + 10^{-4}\mathcal{L}_{\mathrm{corr}}. \tag*{(29)}

The correction regularizer limits the auxiliary residual magnitude relative to the intrinsic response during LSTH continuation. It is part of the LSTH training path, whereas the over-correction value reported later is a separate supplementary diagnostic.

To diagnose whether the optional correction dominates the intrinsic response, we additionally report the implementation-defined over-correction statistic

Eover=1Nvalid∑i:∥ri⋆∥2≥δ∥cϕ,i∥2∥rop,i∥2+δ.(30)E_{\mathrm{over}} = \frac{1}{N_{\mathrm{valid}}} \sum_{i:\lVert r_i^{\star}\rVert_2 \ge\delta} \frac{\lVert c_{\phi,i}\rVert_2}{\lVert r_{\mathrm{op},i}\rVert_2 + \delta}. \tag*{(30)}

This sample-level quantity is averaged over valid response samples, uses the same reference-response mask as the directional response metric, and is diagnostic only. It is not used for training, checkpoint selection, or the main-paper comparisons. Larger values indicate a larger auxiliary correction relative to the intrinsic state-output response.

LSTH continuation uses a 20% adapter-only warmup. During the remaining updates, the selected shared blocks and output head receive auxiliary-gradient weights μblock=0.05\mu_{\mathrm{block}} = 0.05 and μhead=0.02\mu_{\mathrm{head}} = 0.02, with learning-rate multipliers 0.02 and 0.05, respectively. The two shared-parameter gradient coefficients ramp linearly from zero during the first 20% of the second stage. The adapter learning-rate multiplier is 1.0, and the residual scale is fixed at γ=0.05\gamma= 0.05.

The LSTH study uses separate task-specific compact FNO configurations and is not parameterized by the width-32, three-block FNO reported for the breadth study in Table A1. Both LSTH configurations use two FNO layers with hidden width 8, and the two available hidden stages are used by the auxiliary head. Table A15 reports the parameter and optional auxiliary-response evaluation overhead for these two compact FNO configurations. The latency baseline is the intrinsic paired-response path of the same trained LSTM model, not state-only inference and not the intrinsic path of a separate static model. The latency measurements characterize only the optional auxiliary paired-response path and do not change the cost of standard state-only inference.

BackboneInputOutputBlocksWidthBackbone-specific settingHistoryPredictionParams. D/K (M)
FNOCzC_zTCuTC_u332Fourier modes 8×88 \times81Parallel0.141/0.135
CViTCzC_zTCuTC_u332Patch 4×44 \times4; 4 heads1Parallel0.043/0.039
LaMOCzC_zTCuTC_u332Latent width 32; stride 21Parallel0.078/0.075
MambaNOCzC_zTCuTC_u332State size 16; convolution width 41Parallel0.014/0.011
RNOCzC_zTCuTC_u332Three ConvGRU recurrent blocks1Recurrent0.176/0.176

Table A1. Backbone configurations for the uniform-protocol breadth study. All models use one input anchor, denoted by CzC_z channels, and predict the complete TCuTC_u-channel trajectory. For DiffReact, (Cz,T,Cu)=(2,51,2)(C_z,T,C_u)=(2,51,2); for Kolmogorov, (Cz,T,Cu)=(1,21,1)(C_z,T,C_u)=(1,21,1). Parameter counts (in millions) are reported separately for these two output dimensions as D/K.

TaskBase paramsAdded paramsTotal paramsParam. overheadIntrinsic (ms/batch)Auxiliary (ms/batch)Latency overhead
DiffReact7,5261,9969,52226.52%3.77784.247912.44%
Kolmogorov6,1335386,6718.77%3.60054.184116.21%

Table A15. LSTH parameter and auxiliary-response evaluation overhead. Base parameters refer to the shared FNO state predictor. Added parameters count only newly introduced hidden-difference adapters and correction heads; shared backbone blocks and the output projection that participate in continuation training are not counted as added parameters. Latency compares intrinsic paired-response evaluation with auxiliary-response evaluation for the same LSTH model. Measurements use batch size 4 on one NVIDIA GeForce RTX 4080 SUPER, with 50 warmup iterations, 200 measured iterations, and five repeats. The displayed latencies are preserved aggregate means.

ModelMethodState ↓Resp. ↑Supp. mag. ↓Roll. ↓
CViTAug-only0.0115890.14110.3249 ± 0.03050.00690
TangentPDE-Core0.0093980.47210.3281 ± 0.01110.00590
FNOAug-only0.0075290.43670.8152 ± 0.06460.00413
TangentPDE-Core0.0059390.93580.3167 ± 0.36820.00336
LaMOAug-only0.0079060.24551.0623 ± 0.06340.00344
TangentPDE-Core0.0071880.48550.3807 ± 0.61210.00348
MambaNOAug-only0.0121050.07222.2330 ± 0.25870.00830
TangentPDE-Core0.0104330.93450.2033 ± 0.00440.00740
RNOAug-only0.0084520.54310.8445 ± 0.04430.00580
TangentPDE-Core0.0063160.96300.0112 ± 0.00390.00440

Table A8. DiffReact results. State, response cosine, and Rollout@20 reproduce the main-paper means; supplementary magnitude error is mean ± standard deviation from the independent three-seed diagnostic.

ModelMethodState ↓Resp. ↑Supp. mag. ↓Roll. ↓
CViTAug-only0.4140420.30751.7635 ± 0.07150.43090
TangentPDE-Core0.3536820.31920.6632 ± 0.32950.26900
FNOAug-only0.2371840.73921.0413 ± 0.00580.22747
TangentPDE-Core0.2371430.77981.0397 ± 0.00430.22030
LaMOAug-only0.2130510.41871.2073 ± 0.03970.38450
TangentPDE-Core0.2001360.40001.2221 ± 0.03450.36410
MambaNOAug-only0.4881240.61891.0302 ± 0.00550.80600
TangentPDE-Core0.4836790.64931.0881 ± 0.04370.79490
RNOAug-only0.2232500.67691.7261 ± 0.15850.39700
TangentPDE-Core0.1793500.50741.7674 ± 0.17160.33470

Table A9. Kolmogorov results. State, response cosine, and Rollout@20 reproduce the main-paper means; supplementary magnitude error is mean ± standard deviation from the independent three-seed diagnostic.

ModelMethodState ↓Resp. ↑Supp. mag. ↓Roll. ↓
CViTAug-only0.2932700.20840.5475 ± 0.02290.25000
TangentPDE-Core0.2656380.21910.3839 ± 0.00980.24610
FNOAug-only0.1566020.39010.4872 ± 0.01720.15502
TangentPDE-Core0.1577550.45330.2572 ± 0.01120.15108
LaMOAug-only0.2876900.20640.8402 ± 0.02040.25420
TangentPDE-Core0.2869060.20560.4180 ± 0.00410.25270
MambaNOAug-only0.6833630.12631.4627 ± 0.05150.72760
TangentPDE-Core0.6356930.10771.6136 ± 0.03980.68070
RNOAug-only0.3216150.19890.7605 ± 0.05660.28290
TangentPDE-Core0.2897940.22880.5202 ± 0.02420.25390

Table A10. Burgers results. State, response cosine, and Rollout@20 reproduce the main-paper means; supplementary magnitude error is mean ± standard deviation from the independent three-seed diagnostic.

The parameter count uses unique nn.Parameter objects, includes biases, and excludes buffers, optimizer states, activations, and checkpoint metadata. The latency timing excludes data loading and host-to-device transfer. Raw per-repeat timing vectors and explicit precision metadata were not retained, so only aggregate mean latency is reported; median and P95 are not reported. These measurements characterize only the optional auxiliary paired-response path and do not change the cost of standard state-only inference.

The comparison among Static, State FT, Weak FT, and LSTH separates two questions. State and rollout metrics evaluate the shared state predictor, whereas the auxiliary response metric evaluates the dedicated LSTH execution path. An auxiliary-response gain must therefore not be interpreted as evidence that LSTH improves the intrinsic finite-step response of the same shared state operator.

ModelMethodState ↓\downarrowResp. ↑\uparrowSupp. mag. ↓\downarrowRoll. ↓\downarrow
CViTAug-only0.5461220.04712.4749 ±\pm 0.01640.06160
TangentPDE-Core0.4837500.04542.2433 ±\pm 0.01350.05960
FNOAug-only0.1293360.08992.5620 ±\pm 0.01160.03090
TangentPDE-Core0.1350710.08812.5603 ±\pm 0.00950.06828
LaMOAug-only0.5573640.04923.1964 ±\pm 0.07150.09400
TangentPDE-Core0.5551040.04751.5833 ±\pm 0.08710.07730
MambaNOAug-only0.7871170.04551.7060 ±\pm 0.03690.12610
TangentPDE-Core0.7759740.04031.5748 ±\pm 0.02450.09050
RNOAug-only0.6707290.01353.5990 ±\pm 0.23840.65600
TangentPDE-Core0.6366640.01842.1184 ±\pm 0.58290.60530

Table A11. Advection results. State, response cosine, and Rollout@20 reproduce the main-paper means; supplementary magnitude error is mean ±\pm standard deviation from the independent three-seed diagnostic.

Table A12: Paired analysis over the 20 submitted task–backbone settings. Positive changes denote Core improvement; intervals use 10,000 paired bootstrap resamples, and pp values use Holm-corrected one-sided Wilcoxon signed-rank tests.

MetricImprovedMean Δ\DeltaMedian Δ\DeltaBootstrap 95% CIWWRaw ppHolm pprrbr_{\rm rb}
State18/200.0170410.003352[0.007986, 0.026833]195.00.0001310.0003920.857
Response cosine12/200.1162850.011200[0.026504, 0.226892]160.00.0200000.0200000.524
Rollout@2018/200.0209880.005555[0.006057, 0.039931]193.00.0001970.0003950.838

Table A12.

Supplementary LSTH Over-Correction Diagnostic

Table A17 reproduces the submitted state, intrinsic-response, auxiliary-response, and rollout means, and appends an independent three-seed over-correction diagnostic for LSTH. The added diagnostic does not reconstruct uncertainty around the submitted main metrics.

VariantState ↓\downarrowIntr. resp. ↑\uparrowAux. resp. ↑\uparrowRoll. ↓\downarrowSupp. over-corr. ↓\downarrow95% CI
DiffReact
Static0.00590.9433–0.0035––
State FT0.00320.9410–0.0018––
Weak FT0.00370.9440–0.0021––
LSTH<u>0.0036</u>0.94310.9779<u>0.0019</u>0.0000 ± 0.0000[0.0000, 0.0000]
Kolmogorov
Static0.28090.7572–0.2941––
State FT0.27490.7281–0.2887––
Weak FT<u>0.2795</u><u>0.7525</u>–<u>0.2886</u>––
LSTH0.27490.72810.90710.28850.3300 ± 0.0100[0.3052, 0.3548]

Table A17. LSTH supplementary over-correction diagnostic. State, intrinsic response, auxiliary response, and rollout reproduce the submitted means. Supplementary over-correction is an independent three-seed diagnostic and is defined only for LSTH; dashes denote undefined or inapplicable quantities, and its uncertainty is a two-sided 95% Student-tt interval over three seeds.

Over-correction is defined only for LSTH because the other variants do not execute the auxiliary correction path. Dashes therefore denote undefined or inapplicable quantities rather than numerical zero. The reported standard deviation and confidence interval apply only to the supplementary over-correction diagnostic column.

Computational Resources and Query Accounting

These intervals are descriptive summaries over nine heterogeneous objective–seed runs and should not be interpreted as uncertainty intervals for one fixed objective. Table A20 reports measured response-pair generation cost. Each accepted pair adds one perturbed-endpoint solver call because its anchor label is reused. DiffReact and Kolmogorov are regenerated from their frozen perturbed inputs and numerically checked against the stored endpoints; Burgers and Advection use the original complete source-generation logs.

TaskSourceAdd. callsTime (s)Execution
Advection100001000010.01 GPU
Burgers100001000024.01 GPU
DiffReact35035025.14 CPU workers
Kolmogorov3503506.84 CPU workers

Table A20. Measured response-pair generation cost. DiffReact and Kolmogorov regenerate the 350 frozen perturbed endpoints. Burgers and Advection report the original 10,000-pair source generation before selecting the submitted subset. Each entry is one wall-clock measurement including output writing.

Except for the separate LSTH latency microbenchmark in Table A15, experiments were executed on a Linux 6.8 system with two Intel Xeon Gold 6530 processors, approximately 512 GiB of system memory, and NVIDIA GeForce RTX 5090 GPUs with 32 GiB memory per device. The software environment used Python 3.10.12, PyTorch 2.12.0 with CUDA 13.0, NVIDIA driver 580.126.20, NumPy 2.2.6, and h5py 3.16.0. Independent single-GPU runs may be executed concurrently; the number of GPUs visible to the host is not treated as the number used by one training run.

The reported costs refer to different stages and should not be conflated. Pair-generation time measures reference-solver data construction, whereas the unseen-direction cost measures execution of a diagnostic workflow. Neither quantity is a proxy for state-predictor training time or model inference latency.

Additional Burgers Objective Analysis

Figure A6 presents the Burgers objective study as an additional matched RNO ablation. It is separate from the cross-architecture comparison and tests whether adding a frequency-response term changes the direction–magnitude objective on nonlinear transport.

Additional RNO objective ablation on Burgers

Figure A6. Additional RNO objective ablation on Burgers. Points show three-seed means for State, response cosine, and Rollout@20 across the objective variants; error bars denote sample standard deviation. The plot is intended to compare objective trends rather than to foreground inferential statistics.

The frequency-response row is interpreted as a local objective comparison within this Burgers configuration. It does not establish that frequency supervision is uniformly beneficial across the other tasks or backbones.

Robustness to Perturbation Changes

The unseen-direction study separates perturbation families between training and testing while preserving the original anchor splits, optimization budget, seeds, and validation-state checkpoint criterion. Table A21 therefore measures transfer across the tested perturbation families rather than in-distribution response fitting. It does not evaluate arbitrary directions outside the declared construction.

TaskMethodState ↓\downarrowResp. cos. ↑\uparrowRel. resp. ↓\downarrowMag. err. ↓\downarrowRoll.@20 ↓\downarrow
DiffReactAug-only0.010514 ±\pm 0.0004050.1667 ±\pm 0.07841.0227 ±\pm 0.06431.5896 ±\pm 0.60750.00580 ±\pm 0.00133
DiffReactTwo-point MSE0.010411 ±\pm 0.0005000.1707 ±\pm 0.06801.0208 ±\pm 0.06521.5491 ±\pm 0.58810.00564 ±\pm 0.00146
DiffReactTangentPDE-Core0.008211 ±\pm 0.0007760.8834 ±\pm 0.00340.6393 ±\pm 0.01080.2504 ±\pm 0.00950.00491 ±\pm 0.00034
KolmogorovAug-only0.286750 ±\pm 0.0035320.5059 ±\pm 0.00219.8993 ±\pm 0.06032.3386 ±\pm 0.00560.31424 ±\pm 0.01693
KolmogorovTwo-point MSE0.286668 ±\pm 0.0036400.5059 ±\pm 0.00219.9005 ±\pm 0.05882.3387 ±\pm 0.00550.31396 ±\pm 0.01725
KolmogorovTangentPDE-Core0.286205 ±\pm 0.0058830.5419 ±\pm 0.00169.7912 ±\pm 0.03212.3318 ±\pm 0.00310.30824 ±\pm 0.02328
TaskMethodState CIResp. CIRel. resp. CIMag. CIRoll. CI
DiffReactAug-only0.0010070.19480.15961.50910.00331
DiffReactTwo-point MSE0.0012430.16900.16201.46100.00363
DiffReactTangentPDE-Core0.0019260.00860.02690.02350.00085
KolmogorovAug-only0.0087730.00520.14980.01400.04205
KolmogorovTwo-point MSE0.0090420.00510.14600.01370.04284
KolmogorovTangentPDE-Core0.0146150.00390.07970.00780.05783

Table A21. FNO evaluation on perturbation families excluded from training. The upper panel reports mean ±\pm sample standard deviation over three seeds; the lower panel gives the half-width of the corresponding two-sided 95% Student-tt interval. Bold marks the best mean per task.

On DiffReact, Core improves all five reported metrics and raises response cosine from 0.1667 for Aug-only and 0.1707 for Two-point MSE to 0.8834. On Kolmogorov, Core provides a smaller response-cosine gain, from approximately 0.506 to 0.542; the state and rollout differences are small relative to their across-seed variation. The transfer benefit is therefore task dependent.

The altered-step experiment holds the DiffReact anchors, perturbation directions, normalization, and check-points fixed. The reaction–diffusion solver regenerates each endpoint at 0.5ϵ0.5\epsilon, ϵ\epsilon, and 2ϵ2\epsilon. Table A22 and Figure A7 show that the decomposed objective retains high response alignment across all three test steps, whereas Direct Response MSE remains substantially less aligned. Because the perturbation family and checkpoints are fixed, this experiment tests same-source step-size transfer only; it does not establish step-size invariance for the other PDE systems.

Figure A7. Same-source DiffReact altered-step response evaluation using fixed checkpoints. Points show the three-seed mean and error bars show the sample standard deviation. The comparison evaluates transfer across the three tested step sizes within the same perturbation family.

ObjectiveTest stepResp. cos. ↑\uparrowMag. err. ↓\downarrow
Full Core0.5ϵ0.5\epsilon0.9537 ±\pm 0.00410.0220 ±\pm 0.0051
Full Coreϵ\epsilon0.9630 ±\pm 0.00350.0146 ±\pm 0.0051
Full Core2ϵ2\epsilon0.9665 ±\pm 0.00360.0191 ±\pm 0.0056
Direct Resp. MSE0.5ϵ0.5\epsilon0.5768 ±\pm 0.06160.8388 ±\pm 0.0452
Direct Resp. MSEϵ\epsilon0.5953 ±\pm 0.06100.8716 ±\pm 0.0735
Direct Resp. MSE2ϵ2\epsilon0.6034 ±\pm 0.05990.8858 ±\pm 0.0878
ObjectiveTest stepResp. 95% CI hwMag. 95% CI hw
Full Core0.5ϵ0.5\epsilon0.01020.0127
Full Coreϵ\epsilon0.00860.0127
Full Core2ϵ2\epsilon0.00880.0140
Direct Resp. MSE0.5ϵ0.5\epsilon0.15310.1123
Direct Resp. MSEϵ\epsilon0.15140.1826
Direct Resp. MSE2ϵ2\epsilon0.14890.2181

Table A22. DiffReact same-source altered-step evaluation. Fixed checkpoints are evaluated on solver-regenerated endpoints at each test step. The upper panel reports mean ±\pm sample standard deviation over three seeds; the lower panel gives the corresponding two-sided 95% Student-tt CI half-width.

Scope and Additional Limitations

TangentPDE learns finite-step secants along sampled perturbation families, not a complete Jacobian or a uniformly accurate local linearization in every direction. Improvements in state, intrinsic response, and rollout are related but distinct outcomes; improvement in one metric does not logically imply improvement in the others. The submitted results and the independent reruns therefore retain their negative and mixed cells rather than collapsing them into one universal claim.

Response gains remain task and backbone dependent. The unseen-direction study covers only the declared held-out perturbation families, and the altered-step evidence is limited to the evaluated same-source DiffReact construction. Linear rescaling of an existing Kolmogorov pair is not a new PDE solve and is excluded from the altered-step evidence. These boundaries are stated to keep the appendix aligned with the implemented diagnostics rather than to broaden the empirical claim.

LSTH provides a dedicated auxiliary response estimate rather than a universal intrinsic-response improvement. Its auxiliary correction must be validated per task because a residual that is useful for one system can over-correct another. The LSTH auxiliary metric is consequently reported only on the execution path for which it is defined. In particular, the auxiliary head is treated as a separate diagnostic output, not as a direct correction to the reported state operator. This distinction is kept explicit because the appendix contains both shared-state metrics and auxiliary-response quantities.

Finally, seed-level summaries are based on three training seeds and should be read as descriptive optimization variability rather than high-powered single-configuration significance tests. Pair-generation time, diagnostic-run time, training time, and inference latency measure different computational stages and are not interchangeable. The resource tables therefore report accounting for the stage named in each caption: query construction, diagnostic evaluation, and auxiliary-head inference are intentionally separated. The visual summaries preserve the metric directions and mixed outcomes of the underlying submitted results. Exact protocol and accounting quantities remain reported in tables. These conventions make the appendix a reproducibility and audit companion to the main paper rather than an additional source of broader claims.

TaskTime (min)95% CI hwPeak (GiB)95% CI hwGPUs
DiffReact6.4 ± 2.1[4.8, 8.0]0.18 ± 0.00[0.18, 0.18]1
Kolmogorov7.9 ± 2.2[6.2, 9.6]0.09 ± 0.00[0.09, 0.09]1

Table A19. Measured cost of the unseen-direction runs. Time and peak allocated GPU memory are mean ± sample standard deviation per run, with two-sided 95% Student-tt intervals over the nine objective–seed runs for each task.

The ordering of the appendix follows the evidence type rather than a single performance ranking. Configuration and budget tables establish comparability, mechanism tables separate objective components, resource tables document the query and execution accounting, and the final robustness tables record perturbation-family and step-size checks. This order is intended to make clear which quantities are directly comparable and which are protocol-specific.

Figure A8. Early-time DiffReact state fields for the first output channel. The columns show the reference field, Aug-only and TangentPDE-Core predictions, and their absolute errors at five fixed saved indices before the physical field decays. TangentPDE-Core better preserves the localized structures and reduces the dominant error on this median-improvement trajectory.

Figure A9. Early-time Burgers one-dimensional state fields at saved rollout indices 0, 2, 4, 6, 8 from the independent uniform-protocol FNO rerun. TangentPDE-Core more closely follows the transported structures and reduces the dominant errors on this median-improvement trajectory. The color strips are expanded vertically for readability.

Figure A10. Early-time Advection one-dimensional state fields at saved rollout indices 0, 2, 4, 6, 8 from the independent uniform-protocol FNO rerun. Aug-only and TangentPDE-Core remain visually similar, consistent with the small and mixed transport-task differences in the quantitative results. The color strips are expanded vertically for readability.

References

  1. [1]Brandstetter, J.; Worrall, D. E.; and Welling, M. 2022. Message Passing Neural PDE Solvers. In International Conference on Learning Representations.arxiv.org/abs/2202.03376
  2. [2]Cao, L.; O’Leary-Roseberry, T.; and Ghattas, O. 2025. Derivative-Informed Neural Operator Acceleration of Geometric MCMC for Infinite-Dimensional Bayesian Inverse Problems. Journal of Machine Learning Research, 26(78): 1–68.arxiv.org/abs/2403.08220
  3. [3]Czarnecki, W. M.; Osindero, S.; Jaderberg, M.; Swirszcz, G.; and Pascanu, R. 2017. Sobolev Training for Neural Networks. In Advances in Neural Information Processing Systems, volume 30.arxiv.org/abs/1706.04859
  4. [4]Feng, H.; Wang, Y.; and Fan, D. 2025. How to Re-enable PDE Loss for Physical Systems Modeling Under Partial Observation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, 182–190.arxiv.org/abs/2412.09116
  5. [5]Hao, Z.; Wang, Z.; Su, H.; Ying, C.; Dong, Y.; Liu, S.; Cheng, Z.; Song, J.; and Zhu, J. 2023. GNOT: A General Neural Operator Transformer for Operator Learning. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, 12556–12569. PMLR.arxiv.org/abs/2302.14376
  6. [6]Hou, J.; Wang, H.; Xu, P.; Gao, C.; Liu, H.; and Jing, L. 2026. Learning Neural Operators from Partial Observations via Latent Autoregressive Modeling. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 390–398.arxiv.org/abs/2601.15547
  7. [7]Houlsby, N., Gjur̆giu, A., Jastrzebski, S., Morrone, B., de Laroussilhe, Q., Guestrin, C., Attarian, M., & Gelly, S. (2019). Parameter-Efficient Transfer Learning for NLP. In International Conference on Machine Learning (pp. 2790–2799).
  8. [9]Huang, X.; and Perdikaris, P. 2026. PhysicsCorrect: A Training-Free Approach for Stable Neural PDE Simulations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 22057–22065.arxiv.org/abs/2507.02227
  9. [10]Koehler, F., Niedermayer, S., Westermann, R., & Thuerey, N. (2024). APBench: A Benchmark for Autoregressive Neural Emulators of PDEs. In Advances in Neural Information Processing Systems (Vol. 37).
  10. [11]Koren, N.; Mackenbach, R. J. J.; van Sloun, R. J. G.; Radinsky, K.; and Freedman, D. 2026. SVD-NO: Learning PDE Solution Operators with SVD Integral Kernels. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 525–533.arxiv.org/abs/2511.10025
  11. [12]Kovachki, N.; Lanthaler, S.; and Mishra, S. 2021. On Universal Approximation and Error Bounds for Fourier Neural Operators. Journal of Machine Learning Research, 22(290): 1–76.arxiv.org/abs/2107.07562
  12. [13]Kovachki, N., Li, Z., Liu, B., Azizzadenesheli, K., Bhattacharya, K., Stuart, A., & Anandkumar, A. (2023). Neural Operator: Learning Maps Between Function Spaces with Applications to PDEs. Journal of Machine Learning Research, 24(89), 1–97.
  13. [18]Lippe, P.; Veeling, B. S.; Perdikaris, P.; Turner, R. E.; and Brandstetter, J. 2023. PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers. In Advances in Neural Information Processing Systems, volume 36, 67398–67433.arxiv.org/abs/2308.05732
  14. [21]O’Leary-Roseberry, T.; Chen, P.; Villa, U.; and Ghattas, O. 2024. Derivative-Informed Neural Operator: An Efficient Framework for High-Dimensional Parametric Derivative Learning. Journal of Computational Physics, 496: 112555.arxiv.org/abs/2206.10745
  15. [22]Peyvan, A.; Kumar, V.; and Karniadakis, G. E. 2025. Fusion-DeepONet: A Data-Efficient Neural Operator for Geometry-Dependent Hypersonic and Supersonic Flows. arXiv preprint arXiv:2501.01934.arxiv.org/abs/2501.01934
  16. [23]Qiu, Y.; Bridges, N.; and Chen, P. 2024. Derivative-Enhanced Deep Operator Network. In Advances in Neural Information Processing Systems, volume 37, 20945–20981.arxiv.org/abs/2402.19242
  17. [24]Rahman, M. A.; George, R. J.; Elleithy, M.; Leibovici, D.; Li, Z.; Bonev, B.; White, C.; Berner, J.; Yeh, R. A.; Kossaifi, J.; Azizzadenesheli, K.; and Anandkumar, A. 2024. Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs. In Advances in Neural Information Processing Systems, volume 37.arxiv.org/abs/2403.12553
  18. [25]Takamoto, M., Praditia, T., Leiteritz, R., MacKinlay, D., Alesiani, F., Pflueger, D., & Niepert, M. (2022). PDEBench: An Extensive Benchmark for Scientific Machine Learning. Advances in Neural Information Processing Systems, 35, 1596–1611.
  19. [26]Tiwari, K., Dutta, N., Krishnan, N. M. A., & A. P., P. (2025). Latent Mamba Operator for Partial Differential Equations. Proceedings of the 42nd International Conference on Machine Learning, 267, 59639–59667.
  20. [27]Wang, C., & Townsend, A. (2025). Operator Learning for Hyperbolic PDEs. Journal of Machine Learning Research, 26(199), 1–44.
  21. [28]Wang, S.; Seidman, J. H.; Sankaran, S.; Wang, H.; Pappas, G. J.; and Perdikaris, P. 2025. CViT: Continuous Vision Transformer for Operator Learning. In The Thirteenth International Conference on Learning Representations.arxiv.org/abs/2405.13998
  22. [29]Wang, T.; and Wang, C. 2024. Latent Neural Operator for Solving Forward and Inverse PDE Problems. In Advances in Neural Information Processing Systems, volume 37.arxiv.org/abs/2406.03923
  23. [32]Zheng, J.; Li, W.; Xu, N.; Zhu, J.; Lin, X.; and Zhang, X. 2024. Alias-Free Mamba Neural Operator. In Advances in Neural Information Processing Systems, volume 37, 52962–52995.DOI
  24. [33]Zhou, C.; Chen, J.; and Yang, Z. 2026. SAOT: An Enhanced Quality-Aware Spectral Transformer for Solving PDEs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 28928–28936.arxiv.org/abs/2511.18777

Paper details

vlnc:W_hv7gj4yb
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 5

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents