Introduction
Engineering design and qualification rest on repeated numerical simulation, evaluating a system’s governing equations across a range of parameters and conditions (Winsberg & Alvarado, 2026; Lavin et al., 2021). In steady-state problems such as thermal management, structural loading and aerodynamic pressures, the quantity of interest is a field defined over an unstructured finite-element or finite-volume discretisation of a three-dimensional domain, determined by a small set of design and operating parameters (Perreman et al., 2020; Weller et al., 1998). Design sweeps, sensitivity studies and uncertainty quantification each require this map to be evaluated thousands of times, while a single high-fidelity solve can occupy a compute cluster for hours (Keyes et al., 2013). The cost of the solver therefore limits how much of a design space can be explored (Gopakumar et al., 2025). Safety-critical settings such as fusion devices also require guaranteed prediction ranges for mesh nodes under uncertain operating conditions.
Neural operators provide a route past this bound, learning mappings between function spaces with parameters shared across discretisations and approximating solution operators of partial differential equations (PDEs) at a fraction of the cost of the solver that generated their training data [3 ] (Kovachki et al. 2023; Azizzadenesheli et al. 2024). The Fourier neural operator (FNO) (Li et al., 2021) has been deployed as a surrogate across fluid dynamics (Wen et al., 2022), heat transfer (Yuan et al., 2025) and plasma modelling [5 ] (Pamela et al. 2025), reporting speed-ups of several orders of magnitude while retaining discretisation convergence and zero-shot super-resolution (Gopakumar et al., 2024). This efficiency follows from the fast Fourier transform, which assumes the field is sampled on a uniform rectangular grid. However, engineering meshes are often irregular, feature-refined and vary across designs, rarely satisfying this assumption. Existing approaches either process the unstructured mesh directly, which can become memory-intensive at large node counts, or preserve structured processing by transferring the field onto a latent grid. Retaining information from the original mesh while keeping the representation memory-efficient is therefore a key challenge in neural PDEs for large unstructured grids. Existing learned transfers use global coordinate deformations (Li et al., 2023a) or kernel integrals over a radius-limited neighbourhood graph (Li et al., 2023b), but neither preserves structure by construction (§ 2 ).
Our approach: We build the transfer as a sparse, differentiable and structurally constrained linear map from tensor-product B-splines (Piegl & Tiller, 1996; Prautzsch et al., 2002). Compact support gives each mesh point a local stencil whose size is set by the spline degree, not by a search radius. Partition of unity preserves constants. Spline smoothness gives closed-form evaluation and differentiation at any off-grid query point. We call a transfer with all three properties structure-preserving . The same structure gives a non-negative Bernstein basis on each decoder cell, so the prediction lies within the convex hull of its coefficients; propagating a Taylor model of the parameters through the network extends this to rigorous bounds over continuous space and uncertain parameters. The resulting sparse transfer preserves physical locality and gives a memory-efficient link between unstructured meshes and structured Fourier processing.
To further address the parametric nature of engineering problems, we incorporate global design and operating parameters into the operator. With no initial condition to propagate in steady-state problems, the geometry Ω \Omega Ω and a conditioning vector μ \boldsymbol{\mu} μ of design variables, loads, material properties, and operating conditions carry all sample-specific information. How μ \boldsymbol{\mu} μ should enter an operator is unsettled, and its effect is confounded by a redundancy that is rarely controlled for (Koupaï et al., 2024). When μ \boldsymbol{\mu} μ restates geometric quantities already visible in the point cloud, a model can ignore the conditioning pathway and still appear conditioned. Isolating this pathway therefore requires settings in which μ \boldsymbol{\mu} μ carries information that the mesh cannot provide, up to the limiting case of a fixed mesh on which μ \boldsymbol{\mu} μ is the only sample-specific signal. Our benchmarks span this range, from the heat sink, where μ \boldsymbol{\mu} μ restates the geometry, to HIVE, whose fixed mesh leaves μ \boldsymbol{\mu} μ as the only sample-specific signal (§ 4 ). We combine this conditioning with the learnable spline transfer to form a complete conditional L-SplineFNO for parametric fields on complex unstructured meshes. Our contributions are:
Learnable spline transfer for unstructured neural operators. We introduce L-SplineFNO, which couples an FNO backbone with a learnable, structure-preserving spline transformation from unstructured meshes to a structured latent grid. This replaces the geometry-dependent linear solve of analytic SplineFNO with a learnable , structure-preserving B-spline scatter onto a latent FNO grid, with analytic reconstruction at arbitrary query points.
An end-to-end conditional operator framework. We condition the operator on global design and operating parameters through an affine transformation (Perez et al., 2018). The FiLM-style conditioning produces a per-channel scale and shift, applied to the encoded geometry features before the processor backbone. The conditioning method sits in its own module with interchangeable backbones.
Comprehensive evaluation on complex engineering problems. Across six three-dimensional thermal, structural, and aerodynamic benchmarks, we compare L-SplineFNO against representative mesh-native, latent-attention, and gridded neural operators under a common conditioning pipeline and at matched parameter counts, demonstrating strong predictive accuracy while retaining the efficiency of structured latent processing. We further demonstrate linear scaling with mesh size, inexpensive analytic spatial derivatives, and deployment on a 29.2-million-node mesh of the MAST-U tokamak.
Background and Related Work
Conditional operator learning. Let U \mathcal{U} U denote a space of solution fields on a domain Ω \Omega Ω , and let μ ∈ P ⊂ R k \boldsymbol{\mu}\in\mathcal{P}\subset\mathbb{R}^{k} μ ∈ P ⊂ R k collect design variables, material properties, boundary conditions, or operating conditions. For the steady problems considered here, with no input field, the solution operator maps the geometry and parameters to the solution,
N : ( μ , Ω ) ⟼ y ∈ U , L μ , Ω ( y ) = f , (1) \mathcal{N}:(\boldsymbol{\mu},\Omega)\longmapsto y\in\mathcal{U},\qquad\mathcal{L}_{\boldsymbol{\mu},\Omega}(y)=f, \tag*{(1)} N : ( μ , Ω ) ⟼ y ∈ U , L μ , Ω ( y ) = f , (1)
where L μ , Ω \mathcal{L}_{\boldsymbol{\mu},\Omega} L μ , Ω is the governing operator and f f f the forcing, if any. All sample-specific information enters through Ω \Omega Ω and μ \boldsymbol{\mu} μ , so the learning task is both geometric and conditional.
Neural operators. Neural operators approximate maps such as Equation (1) using architectures whose parameters are shared across discretisations, so that a trained model converges across mesh discretisations rather than to a fixed resolution [3 ] (Kovachki et al. 2023). The FNO parametrises the kernel integral as a global convolution in Fourier space ([12 ] (Li et al. 2021)): for a lifted feature field v t v_t v t , a layer computes
v t + 1 ( x ) = σ ( W v t ( x ) + F − 1 ( R ϕ ⋅ F ( v t ) ) ( x ) ) , (2) v_{t+1}(x)=\sigma\left(Wv_t(x)+\mathcal{F}^{-1}\left(R_{\phi}\cdot\mathcal{F}(v_t)\right)(x)\right), \tag*{(2)} v t + 1 ( x ) = σ ( W v t ( x ) + F − 1 ( R ϕ ⋅ F ( v t ) ) ( x ) ) , (2)
where F \mathcal{F} F is the Fourier transform, R ϕ R_{\phi} R ϕ is a learned multiplier on a truncated band of modes, and W W W is a pointwise linear map. Physics-informed neural operators further constrain the learned solution family by penalising PDE residuals ((Li et al., 2023c; Ganeshram et al., 2026)). Evaluating F \mathcal{F} F with the fast Fourier transform offers only O ( N log N ) O(N\log N) O ( N log N ) complexity, making the Equation (2) efficient and FNOs a popular choice ([11 ] ), but restricts its operation to uniform rectangular grids.
Operator learning on unstructured geometries. Extensions to unstructured grids fall into three groups. Mesh-native models attend over mesh points or learned groupings of them, as in GNOT (Hao et al., 2023), Transolver (Wu et al., 2024) and its parallel successor Transolver++ (Luo et al., 2025), or compress the point set into latent tokens, as in UPT ([1 ] ); they avoid gridding, but cost and fidelity depend on how aggressively the mesh is compressed. Latent-grid models keep a structured backbone by transferring the data onto a regular domain, through learned deformations in Geo-FNO (Li et al., 2023a), balanced non-uniform partitions in NUNO (Liu et al., 2023), graph-kernel encoders in GINO (Li et al., 2023b), multi-scale attentional encoders with a vision-transformer processor in GAOT (Wen et al., 2026), or optimal-transport maps for surfaces (Li et al., 2025); here the transfer becomes part of the approximation, since coarse grids lose thin features and interfaces, while dense grids and large neighbourhood graphs cost memory. Precomputed representations such as PGD-NO build geometry tokens ahead of time, saving memory at very large mesh sizes but removes the representation from end-to-end training (Zhong et al., 2026). Industry-scale comparisons find no representation uniformly best across accuracy, cost, memory and deployment (Zhong et al., 2025).
B-splines in scientific machine learning. Tensor-product B-splines are compactly supported, smooth, and form a partition of unity (Piegl & Tiller, 1996). Their sparse Jacobians have been used as differentiable layers for geometry reconstruction and finite-element learning ([7 ] ); physics-informed B-spline networks predict control coefficients directly, so that derivatives and boundary conditions can be imposed analytically (Wang et al., 2026); and Neural Spline Operators extend this coefficient-learning view from finite-dimensional PDE parameters to functional inputs (Wang et al., 2025). In each, B-splines serve as the output representation, rather than as a parameter-efficient, learnable transfer across an unstructured mesh and a latent structured grid.
Conditioning on physical parameters. The parameters μ \mu μ either restate information already visible in the geometry or encode quantities the geometry cannot reveal, such as loads, materials and operating conditions. Broadcasting μ \mu μ as an extra input channel is simple, but asks the backbone to wastefully preserve and propagate a signal that is spatially constant across the domain (Takamoto et al., 2023). Alternatives inject μ \mu μ into the network directly: feature-wise linear modulation (FiLM) applies parameter-dependent, channel-wise affine transformations (Perez et al., 2018); adaptive normalisation and cross-attention modulate intermediate features in a similar spirit (Peebles & Xie, 2023; Rebain et al., 2022); and hypernetworks generate the operator’s weights from μ \mu μ (Jing et al., 2026).
Spline-Gridded Fourier Neural Operators
Figure 1 summarises the method: sparse spline operators carry mesh features to a uniform latent grid for a standard FNO, and reconstruct the output at arbitrary query points. We first introduce conditional input lifting, then formulate analytically SplineFNO, and finally develop L-SplineFNO by replacing its geometry-dependent spline transform with a local learnable scatter.
Figure 1. End-to-end conditional L-SplineFNO architecture. The unstructured geometry and global conditioning vector are first mapped to conditioned nodal features through coordinate normalisation, positional encoding, and FiLM modulation. A learnable C p − 1 C^{p-1} C p − 1 spline scatter then transforms these features from the unstructured mesh to a structured latent grid, where they are processed by a Fourier neural operator (FNO) backbone. The resulting grid representation is mapped back to the physical mesh using analytic spline reconstruction and a pointwise projection to produce the predicted field. This construction provides a sparse, structure-preserving, differentiable mesh-to-grid-to-mesh mapping with the structured Fourier processing.
Notation. Let { w i } i = 1 N ⊂ Ω \{w_i\}_{i=1}^{N}\subset\Omega { w i } i = 1 N ⊂ Ω denote the nodes of an unstructured input mesh in a d d d -dimensional domain Ω ⊂ R d \Omega\subset\mathbb{R}^{d} Ω ⊂ R d . We introduce a uniform tensor-product control grid with multi-index set G = ∏ ℓ = 1 d { 0 , … , S ℓ − 1 } \mathcal{G}=\prod_{\ell=1}^{d}\{0,\ldots,S_{\ell}-1\} G = ∏ ℓ = 1 d { 0 , … , S ℓ − 1 } ; a multi-index is written g = ( g 1 , … , g d ) ∈ G g=(g_1,\ldots,g_d)\in\mathcal{G} g = ( g 1 , … , g d ) ∈ G , where g ℓ g_{\ell} g ℓ is the grid index along coordinate axis ℓ \ell ℓ . The grid contains G : = ∣ G ∣ = ∏ ℓ = 1 d S ℓ G := \lvert G\rvert= \prod_{\ell=1}^{d} S_{\ell} G := ∣ G ∣ = ∏ ℓ = 1 d S ℓ control points, with coordinates ξ g ( ℓ ) = ξ 0 ( ℓ ) + g ℓ Δ ℓ \xi_{g}^{(\ell)} = \xi_{0}^{(\ell)} + g_{\ell}\Delta_{\ell} ξ g ( ℓ ) = ξ 0 ( ℓ ) + g ℓ Δ ℓ , ℓ = 1 , … , d \ell= 1,\ldots,d ℓ = 1 , … , d , where ξ 0 ( ℓ ) \xi_{0}^{(\ell)} ξ 0 ( ℓ ) and Δ ℓ > 0 \Delta_{\ell} > 0 Δ ℓ > 0 are the origin and spacing along axis ℓ \ell ℓ . The spacing may differ between coordinate axes, but is constant along each axis.
Conditional Input Lifting
Many computational physics problems depend on both geometry and global parameters such as material properties, boundary conditions, source terms, or operating conditions. We introduce conditioning in our framework at the input lifting stage to incorporate these problem-specific parameters into the nodal features supplied to the operator. The model receives an unstructured set of geometry coordinates { ω i } i = 1 N \{\omega_i\}_{i=1}^{N} { ω i } i = 1 N , a global conditioning vector μ ∈ R k \mu\in\mathbb{R}^{k} μ ∈ R k , and a set of output query points { q j } j = 1 N q \{q_j\}_{j=1}^{N_q} { q j } j = 1 N q . No input field is supplied, and the sample-specific signal is carried by the geometry and μ \mu μ .
In the feature path, each normalised coordinate ω ˉ i ∈ [ 0 , 1 ] d \bar{\omega}_i \in[0,1]^d ω ˉ i ∈ [ 0 , 1 ] d is encoded by integrated sinusoidal features φ ( ω ˉ i ) ∈ R 2 d n f \varphi(\bar{\omega}_i) \in\mathbb{R}^{2dn_f} φ ( ω ˉ i ) ∈ R 2 d n f at n f n_f n f frequencies. The global condition is embedded as e μ = E μ ( μ ) e_{\mu} = E_{\mu}(\mu) e μ = E μ ( μ ) and applied through FiLM, the most efficient of the conditioning methods we compared (Section G),
f ~ i = γ μ ⊙ φ ( ω ˉ i ) + β μ , h i = P i n [ f ~ i ; ω ˉ i ] , (3) \tilde{f}_i = \gamma_{\mu} \odot\varphi(\bar{\omega}_i) + \beta_{\mu}, \qquad h_i = P_{\mathrm{in}}\left[\tilde{f}_i;\bar{\omega}_i\right], \tag*{(3)} f ~ i = γ μ ⊙ φ ( ω ˉ i ) + β μ , h i = P in [ f ~ i ; ω ˉ i ] , (3)
where γ μ = H γ ( e μ ) \gamma_{\mu} = H_{\gamma}(e_{\mu}) γ μ = H γ ( e μ ) and β μ = H β ( e μ ) \beta_{\mu} = H_{\beta}(e_{\mu}) β μ = H β ( e μ ) are broadcast across all mesh nodes. The modulation acts only on the encoded features φ \varphi φ ; the normalised coordinates bypass it and are concatenated back before the pointwise lift. We collect the lifted nodal features as X = [ h 1 , … , h N ] T ∈ R N × c X = [h_1,\ldots,h_N]^{\mathsf{T}} \in\mathbb{R}^{N \times c} X = [ h 1 , … , h N ] T ∈ R N × c . In the geometry path, the same normalised coordinates ω ˉ i \bar{\omega}_i ω ˉ i , unmodulated, determine the compact mesh-to-grid spline stencils (Figure 1 ).
SplineFNO
After constructing the conditioned nodal features, we first introduce SplineFNO, which uses fixed analytic spline operators to transform from the unstructured mesh to a latent uniform grid and to reconstruct at the output coordinates, with a standard FNO operating on the latent grid. Writing t i ( ℓ ) = ( ω i ( ℓ ) − ξ 0 ( ℓ ) ) / Δ ℓ t_i^{(\ell)} = (\omega_i^{(\ell)} - \xi_0^{(\ell)})/\Delta_{\ell} t i ( ℓ ) = ( ω i ( ℓ ) − ξ 0 ( ℓ ) ) / Δ ℓ for the continuous grid coordinate of mesh node ω i \omega_i ω i along axis ℓ \ell ℓ , and letting B p B_p B p denote the centred cardinal B-spline of degree p p p introduced in Section B.1, we use p = 3 p = 3 p = 3 by default and write B : = B p B := B_p B := B p for conciseness.
The unnormalised tensor product basis matrix Φ ~ ∈ R N × G \widetilde{\Phi} \in\mathbb{R}^{N \times G} Φ ∈ R N × G is defined as
Φ ~ i , g = ∏ ℓ = 1 d B ( t i ( ℓ ) − g ℓ ) , i = 1 , … , N , g ∈ G . (4) \widetilde{\Phi}_{i,\mathbf{g}}=\prod_{\ell=1}^{d}B\left(t_i^{(\ell)}-g_\ell\right),\qquad i=1,\ldots,N,\quad\mathbf{g}\in\mathcal{G}. \tag*{(4)} Φ i , g = ℓ = 1 ∏ d B ( t i ( ℓ ) − g ℓ ) , i = 1 , … , N , g ∈ G . (4)
Because the degree p p p spline has compact support over p + 1 p+1 p + 1 grid points, each row of Φ ~ \widetilde{\Phi} Φ contains at most ( p + 1 ) d (p+1)^d ( p + 1 ) d nonzero entries. To preserve constants near the boundary, contributions outside G \mathcal{G} G are discarded and the remaining basis values are normalised row wise as Φ i , g = Φ ~ i , g / ∑ k ∈ G Φ ~ i , k \Phi_{i,\mathbf{g}}=\widetilde{\Phi}_{i,\mathbf{g}}/\sum_{\mathbf{k}\in\mathcal{G}}\widetilde{\Phi}_{i,\mathbf{k}} Φ i , g = Φ i , g / ∑ k ∈ G Φ i , k , so that partition of unity is maintained: ∑ g ∈ G Φ i , g = 1 \sum_{\mathbf{g}\in\mathcal{G}}\Phi_{i,\mathbf{g}}=1 ∑ g ∈ G Φ i , g = 1 , whenever the spline support intersects the control grid.
The forward pass consists of three stages. (i) Spline transformation: unstructured to structured. We seek grid coefficients X G ∈ R G × c \mathcal{X}_{\mathcal{G}}\in\mathbb{R}^{G\times c} X G ∈ R G × c whose spline reconstruction approximates the nodal features,
X G = ( Φ T Φ + λ I G ) − 1 Φ T X , (5) \mathcal{X}_{\mathcal{G}}=\left(\Phi^{\mathsf{T}}\Phi+\lambda I_{\mathcal{G}}\right)^{-1}\Phi^{\mathsf{T}}\mathcal{X}, \tag*{(5)} X G = ( Φ T Φ + λ I G ) − 1 Φ T X , (5)
where λ > 0 \lambda>0 λ > 0 ensures that Φ T Φ + λ I G \Phi^{\mathsf{T}}\Phi+\lambda I_{\mathcal{G}} Φ T Φ + λ I G is positive definite. (ii) Grid-space neural operator. The coefficients are reshaped according to the tensor-product grid and processed by a standard FNO, Y G = F θ ( X G ) \mathcal{Y}_{\mathcal{G}}=\mathcal{F}_{\theta}(\mathcal{X}_{\mathcal{G}}) Y G = F θ ( X G ) . (iii) Spline reconstruction: structured to unstructured. The grid features are evaluated using the same analytic basis,
Y = Φ F θ ( ( Φ T Φ + λ I G ) − 1 Φ T X ) . (6) \mathcal{Y}=\Phi\mathcal{F}_{\theta}\left(\left(\Phi^{\mathsf{T}}\Phi+\lambda I_{\mathcal{G}}\right)^{-1}\Phi^{\mathsf{T}}\mathcal{X}\right). \tag*{(6)} Y = Φ F θ ( ( Φ T Φ + λ I G ) − 1 Φ T X ) . (6)
The spline matrices are sparse, analytic functions of the sampling coordinates and contain no learned parameters. Learning occurs in F θ \mathcal{F}_{\theta} F θ , while gradients propagate through both spline transfers.
Learnable SplineFNO (L-SplineFNO)
The spline transformation in Equation (5) requires a geometry-dependent linear solve, whose inverse is generally dense. L-SplineFNO replaces this solve with a local, learnable scatter that directly transfers nodal features to the latent grid. The learned quantities are the transfer kernel shapes, while the grid nodes and analytic spline basis remain fixed. Along each coordinate axis ℓ \ell ℓ , we use
κ θ ( ℓ ) ( t ) = B ( t ) + ∑ j = 1 J ζ j ( ℓ ) B ( t − τ j h ) , ζ ( ℓ ) = V 0 α θ ( ℓ ) . (7) \kappa_{\theta}^{(\ell)}(t)=B(t)+\sum_{j=1}^{J}\zeta_{j}^{(\ell)}B\left(\frac{t-\tau_j}{h}\right),\qquad\zeta^{(\ell)}=V_0\alpha_{\theta}^{(\ell)}. \tag*{(7)} κ θ ( ℓ ) ( t ) = B ( t ) + j = 1 ∑ J ζ j ( ℓ ) B ( h t − τ j ) , ζ ( ℓ ) = V 0 α θ ( ℓ ) . (7)
that is, the analytic spline B B B plus J J J narrower B-spline corrections with fixed centres τ j \tau_j τ j and relative width h ∈ ( 0 , 1 ) h\in(0,1) h ∈ ( 0 , 1 ) . The columns of V 0 V_0 V 0 span the null space of the partition-of-unity constraints, so only the unconstrained coordinates α θ ( ℓ ) \alpha_{\theta}^{(\ell)} α θ ( ℓ ) are trained. The constrained correction preserves compact support, partition of unity to numerical precision, and C p − 1 C^{p-1} C p − 1 regularity. Details are given in Section B.2.
The transfer weights are tensor products of the per-axis kernels, normalised over the valid stencil of node i i i : S ℓ ( i ) \mathcal{S}_{\ell}(i) S ℓ ( i ) holds the grid indices on axis ℓ \ell ℓ within the kernel support of t i ( ℓ ) t_i^{(\ell)} t i ( ℓ ) , and S ( i ) = ∏ ℓ S ℓ ( i ) \mathcal{S}(i)=\prod_{\ell}\mathcal{S}_{\ell}(i) S ( i ) = ∏ ℓ S ℓ ( i ) the at most ( p + 1 ) d (p+1)^d ( p + 1 ) d grid points of its tensor-product stencil (Equation (29) ),
( W θ ) i , g = ∏ ℓ = 1 d κ θ ( ℓ ) ( t i ( ℓ ) − g ℓ ) ∑ g ′ ∈ S ℓ ( i ) κ θ ( ℓ ) ( t i ( ℓ ) − g ℓ ′ ) , g ∈ S ( i ) , (8) (W_{\theta})_{i,\mathbf{g}}=\prod_{\ell=1}^{d}\frac{\kappa_{\theta}^{(\ell)}\left(t_i^{(\ell)}-g_\ell\right)}{\sum_{\mathbf{g}'\in\mathcal{S}_{\ell}(i)}\kappa_{\theta}^{(\ell)}\left(t_i^{(\ell)}-g_\ell'\right)},\qquad\mathbf{g}\in\mathcal{S}(i), \tag*{(8)} ( W θ ) i , g = ℓ = 1 ∏ d ∑ g ′ ∈ S ℓ ( i ) κ θ ( ℓ ) ( t i ( ℓ ) − g ℓ ′ ) κ θ ( ℓ ) ( t i ( ℓ ) − g ℓ ) , g ∈ S ( i ) , (8)
and zero otherwise. Each grid node then forms a normalised weighted accumulation, with n g = ∑ i = 1 N ( W θ ) i , g X i n_{\mathbf{g}}=\sum_{i=1}^{N}(W_{\theta})_{i,\mathbf{g}}\mathcal{X}_i n g = ∑ i = 1 N ( W θ ) i , g X i , d g = ∑ i = 1 N ( W θ ) i , g d_{\mathbf{g}}=\sum_{i=1}^{N}(W_{\theta})_{i,\mathbf{g}} d g = ∑ i = 1 N ( W θ ) i , g , and X ~ G [ g ] = n g / d ~ g \widetilde{\mathcal{X}}_{\mathbf{G}}[\mathbf{g}]=n_{\mathbf{g}}/\widetilde{d}_{\mathbf{g}} X G [ g ] = n g / d g , where d ~ g \widetilde{d}_{\mathbf{g}} d g is the sign preserving safeguarded version of d g d_{\mathbf{g}} d g defined in Section B.3. Defining D θ = diag ( d ~ g ) D_{\theta}=\operatorname{diag}(\widetilde{d}_{\mathbf{g}}) D θ = diag ( d g ) , the learnable spline transformation can then be written compactly as
X ~ G = D θ − 1 W θ T X , (9) \widetilde{\mathcal{X}}_{\mathcal{G}}=D_{\theta}^{-1}W_{\theta}^{\mathsf{T}}\mathcal{X}, \tag*{(9)} X G = D θ − 1 W θ T X , (9)
which provides a local approximation to the spline transformation ( Φ T Φ + λ I G ) − 1 Φ T X (\Phi^{\mathsf{T}}\Phi+\lambda I_{\mathcal{G}})^{-1}\Phi^{\mathsf{T}}\mathcal{X} ( Φ T Φ + λ I G ) − 1 Φ T X without requiring a geometry-dependent linear solve.
Table 1: Relative L 2 L_2 L 2 error (%, mean ± \pm ± std over five seeds; top line of each entry) and training time (hh:mm on a single NVIDIA A100; grey line beneath) for each architecture under FiLM conditioning. The best mean error for each case is shown in bold. N N N is the number of spatial training nodes. Heat Sink, Heated Duct and HIVE predict absolute temperature, so their relative errors are small by construction. L-SplineFNO outperforms baselines on most experiments, while remaining significantly compute-efficient.
Table 1.
To retain mesh coverage information removed by normalisation, we augment the scattered features with an analytic density feature ρ \rho ρ computed from the fixed spline basis and form the FNO input
X G = P ρ [ X ~ G ; log ( 1 + ρ ) ] , Y G = F θ ( X G ) , (10) \mathcal{X}_{G}=P_{\rho}\left[\widetilde{\mathcal{X}}_{G};\log(1+\rho)\right],\qquad\mathcal{Y}_{G}=F_{\theta}(\mathcal{X}_{G}), \tag*{(10)} X G = P ρ [ X G ; log ( 1 + ρ ) ] , Y G = F θ ( X G ) , (10)
where P ρ P_{\rho} P ρ is a learned pointwise projection. The construction of ρ \rho ρ is detailed in Section B.3. Finally, the output is reconstructed analytically at each query point and projected to the output channels
z j = ∑ g ∈ G Φ g ( q j ) Y G [ g ] , y ^ j = P o u t z j , (11) \mathbf{z}_{j}=\sum_{\mathbf{g}\in\mathcal{G}}\Phi_{\mathbf{g}}(q_{j})\mathcal{Y}_{G}[\mathbf{g}],\qquad\widehat{\mathbf{y}}_{j}=P_{\mathrm{out}}\mathbf{z}_{j}, \tag*{(11)} z j = g ∈ G ∑ Φ g ( q j ) Y G [ g ] , y j = P out z j , (11)
where P o u t P_{\mathrm{out}} P out is a learned pointwise output projection.
Region-specific transfers. On meshes with several materials or regions, each node carries a region label and can be routed through its own lift, scatter kernels and output projection, around one shared latent grid and FNO. The per-region scatter sums are added before the division in Equation (9) , so constants are still reproduced. A per-region readout lets each region take its own combination of the shared latent features, so the field can remain continuous across an interface while its normal derivative jumps. Only the lift, projection and kernel weights are duplicated; the FNO is shared. We use this whenever the mesh is naturally partitioned by regions or materials.
Experiments
Data: We evaluate on six steady-state, three-dimensional engineering benchmarks, including a CPU-board heat sink ((Zhong et al., 2026)), a structurally loaded bracket ((Zhong et al., 2025)), aircraft flow fields ((Luo et al., 2025)), a building fire simulated with FDS ((McGrattan et al., 2013)), conjugate heat transfer through a multi-region heated duct (Weller et al., 1998), and the HIVE induction-heating benchmark (Nuno Nobre, Josh Williams and Karthikeyan Chockalingam, 2024; Horne-Jones et al., 2026). Together they span thermal, structural, and aerodynamic physics, sample-dependent and shared meshes, and single- and multi-region domains. The suite also varies how much information the conditioning vector μ \boldsymbol{\mu} μ provides beyond the observed geometry. It is largely redundant for the heat sink; partly non-redundant for the bracket and for the building fire and essential for Aircraft, heated duct and HIVE, where operating conditions cannot be recovered from the geometry alone. Further details on the physics, data, and preprocessing are provided in Section D.
Models: We compare L-SplineFNO with six representative neural operators for unstructured domains, including mesh-based attention models Transolver (Wu et al., 2024) and GNOT (Hao et al., 2023), latent attention models UPT [1 ] and MSPT [9 ] , and gridded FNO-based models NUNO (Liu et al., 2023) and GINO (Li et al., 2023b). We also include the analytic SplineFNO to isolate the effect of learning the spline transfer. All models use the same outer conditional pipeline in Figure 1 and match in parameter count (2.3 to 2.5 million parameters), so we replace only the backbone and its associated spatial representation. Full architectural and implementation details are provided in Section F.
Training Objective: Given a dataset D = { ( μ ( i ) , { ( ω n ( i ) , y ( i ) ( ω n ( i ) ) ) } n = 1 N i ) } i = 1 M \mathcal{D} = \left\{\left(\boldsymbol{\mu}^{(i)}, \left\{\left(\boldsymbol{\omega}_{n}^{(i)}, y^{(i)}\left(\boldsymbol{\omega}_{n}^{(i)}\right)\right)\right\}_{n=1}^{N_i}\right)\right\}_{i=1}^{M} D = { ( μ ( i ) , { ( ω n ( i ) , y ( i ) ( ω n ( i ) ) ) } n = 1 N i ) } i = 1 M of conditioning vectors, mesh nodes and target fields, with no input field supplied, we minimise the mean unweighted discrete relative L 2 L_2 L 2 error over the sampled mesh nodes (Equation (12) ).
Here y ( i ) ∈ R N i \mathbf{y}^{(i)} \in\mathbb{R}^{N_i} y ( i ) ∈ R N i is the target field at the nodes { ω n ( i ) } n = 1 N i \left\{\boldsymbol{\omega}_{n}^{(i)}\right\}_{n=1}^{N_i} { ω n ( i ) } n = 1 N i of sample i i i , and ∥ ⋅ ∥ 2 \lVert\cdot\rVert_2 ∥ ⋅ ∥ 2 is the unweighted discrete Euclidean norm over those nodes. We use the same metric for training and its percentage formulation for the final error evaluation in Table 1.
All models use the same training recipe: Adam with a starting learning rate of 10 − 3 10^{-3} 1 0 − 3 , halved every 50 epochs, for 250 epochs.
J ( θ ) = 1 M ∑ i = 1 M ∥ y ^ ( i ) − y ( i ) ∥ 2 ∥ y ( i ) ∥ 2 (12) \mathcal{J}(\theta) = \frac{1}{M} \sum_{i=1}^{M} \frac{\left\lVert\hat{\mathbf{y}}^{(i)} - \mathbf{y}^{(i)} \right\rVert_2}{\left\lVert\mathbf{y}^{(i)} \right\rVert_2} \tag*{(12)} J ( θ ) = M 1 i = 1 ∑ M y ( i ) 2 y ^ ( i ) − y ( i ) 2 (12)
Benchmarks
As shown in Table 1, L-SplineFNO occupies a favourable accuracy–cost regime across the benchmark suite: it provides the strongest predictive accuracy on the heat sink, building fire, heated duct, and HIVE cases, and remains close to the leading attention-based models on Aircraft while requiring substantially less training time than MSPT and GNOT. The bracket case is the principal exception, where direct mesh-based attention retains a clear accuracy advantage. The bracket’s peak stresses sit in small, sharp concentrations, which the latent grid’s averaging and C 2 C^2 C 2 decode flatten, whereas attention acts on the mesh nodes directly (Figure 11 ). Compared with the analytic SplineFNO, however, learning the spline transfer consistently improves accuracy across all six cases with only a modest change in model size. The faster gridded baselines sacrifice considerably more accuracy, whereas the more accurate attention-based alternatives incur higher training cost; L-SplineFNO therefore combines the efficiency of a structured latent grid with predictive performance competitive with, and often better than, several more computationally demanding unstructured operators. Figure 2 demonstrates how L-SplineFNO achieves near-perfect accuracy in modelling the heat distribution across a heat sink. Figure 3 shows the error maps produced by each model for the HIVE monoblock case, with L-SplineFNO modelling the heat distribution with 99.92% accuracy. We show further ablations on spline degree and multi-scale scattering in § 1 .
Figure 2. Error maps produced by baseline architectures in modelling heat distribution across the heat sink. L-SplineFNO produces the most accurate predictions, with minimal errors concentrated at the edges of the heat sink where the temperature gradient is highest.
Figure 3. Error maps produced by baselines in modelling heat distribution across the HIVE monoblock. L-SplineFNO models the temperature across the structure with 99.9% accuracy (sample-specific error shown).
Figure 11. Error maps produced by various architectures in modelling stress across the bracket.
Analytic Gradients
The analytic decoder of L-SplineFNO yields spatial derivatives without an automatic-differentiation graph over the query coordinates. The decoded field of Equation (11) is y ^ ( q ) = P o u t ∑ g ∈ G Y g [ g ] Φ g ( q ) \hat{y}(q)=P_{\mathrm{out}}\sum_{g\in G}\mathcal{Y}_g[g]\Phi_g(q) y ^ ( q ) = P out ∑ g ∈ G Y g [ g ] Φ g ( q ) , where Y g [ g ] \mathcal{Y}_g[g] Y g [ g ] are the latent-grid coefficients, Φ g \Phi_g Φ g is the normalised basis (Equation (4) ), and P o u t z = W o u t z + b o u t P_{\mathrm{out}}z=W_{\mathrm{out}}z+b_{\mathrm{out}} P out z = W out z + b out is an affine projection. The bias b o u t b_{\mathrm{out}} b out vanishes under differentiation, so
∂ n y ^ ( q ) = W o u t ∑ g ∈ G Y g [ g ] ∂ n Φ g ( q ) , 1 ≤ ∣ n ∣ ≤ 2. (13) \partial^n\hat{y}(q)=W_{\mathrm{out}}\sum_{g\in G}\mathcal{Y}_g[g]\partial^n\Phi_g(q),\qquad1\leq\lvert n\rvert\leq2. \tag*{(13)} ∂ n y ^ ( q ) = W out g ∈ G ∑ Y g [ g ] ∂ n Φ g ( q ) , 1 ≤ ∣ n ∣ ≤ 2. (13)
The derivatives ∂ n Φ g \partial^n\Phi_g ∂ n Φ g are evaluated in closed form from the first and second derivatives of the degree-p p p B-spline, including those of the boundary partition-of-unity normalisation. Each query touches at most $(p+1)^d control points, so N N N queries cost O ( N ( p + 1 ) d ) O(N(p+1)^d) O ( N ( p + 1 ) d ) with no reverse-mode coordinate tape, and the result stays differentiable in Y G \mathcal{Y}_G Y G , keeping derivative-based physics losses trainable.
Full-model autograd instead returns the coordinate sensitivity ∂ ( ∑ i y ^ i ) / ∂ ω j \partial(\sum_i \hat{y}_i)/\partial\omega_j ∂ ( ∑ i y ^ i ) / ∂ ω j , which for globally coupled operators differs from ∇ y ^ ( ω j ) \nabla\hat{y}(\omega_j) ∇ y ^ ( ω j ) because moving node j j j can alter predictions elsewhere; we therefore compare against autograd through the same decoder with the latent grid fixed. Figure 4 a reports runtime and memory across all benchmark experiments.
Figure 4. (a) Gradient-evaluation costs for L-SplineFNO. (b) Mesh capacity of models across a single GPU. Derivative-evaluation cost and scalability. (a) Mean derivative runtime (left) and additional peak GPU allocation (right), averaged across the benchmark experiments. Forward preparation is excluded; memory is measured above the prepared live state, not total GPU use. Full-model autograd VJP measures coordinate sensitivity, whereas decoder autograd and analytic spline differentiation evaluate pointwise field gradients. Closed-form spline derivatives give exact gradients at a fraction of the cost. (b) Mean largest successful node count across three seeds on an A100 80GB GPU in FP32 with batch size one. Training includes a complete Adam step. L-SplineFNO accomodates 26.6M nodes in training and 46.0M in inference, demonstrating superior scalability against the baselines.
Scalability
We benchmark all architectures on synthetic three-dimensional point clouds with the HIVE settings, on one NVIDIA A100 80GB GPU in FP32 with batch size one (Figure 4 b). Runtime grows near-linearly with node count for every model over its feasible range, so memory decides capacity. L-SplineFNO trains on up to 26.6M nodes and runs inference on 46.0M, well beyond the capacity of existing methods. Details are in Section H.
Virtual Tokamak of MAST-U
Beyond benchmarks, we apply L-SplineFNO to a whole-device structural modelling of the Mega-Ampere Spherical Tokamak Upgrade (MAST-U), a fusion experiment that informs compact spherical-tokamak power plants [6 ] (Milnes et al. 2015; Morris et al. 2018; Harrison et al. 2024). Such simulations, referred to as virtual tokamaks load the vessel, centre column, coils and supports together, so each component sees boundary conditions set by the rest of the assembly (Ellis et al., 2025). A single static solve of the MAST-U model costs about 500+ CPU-hours on 800 cores, challenging the many-query workloads a structural digital twin needs (Pamela et al., 2025).
To alleviate this bottleneck, we learn the static displacement field u = ( u x , u y , u z ) \mathbf{u} = (u_x,u_y,u_z) u = ( u x , u y , u z ) of the full MAST-U structural model under thermal expansion, given the thermal expansion coefficient and wall temperature. The task stresses unstructured operators on several axes at once. The mesh has 29.2M nodes and 134.6M tetrahedra; merging 3,938 CAD parts leaves element sizes spanning more than four orders of magnitude. Steel, copper and graphite meet at over a million shared interface nodes. L-SplineFNO, with material-specific spline transfers and a shared FNO backbone, is trained on the full mesh with 29.2M nodes and reaches a relative vector L 2 L_2 L 2 error of 1.8% on a single H100 GPU (Figure 5 ). Section E details the device, physics, data, and training.
Figure 5. MAST-U virtual tokamak: Displacement magnitude (mm) from the MOOSE simulation (ground truth) and the L-SplineFNO (prediction) and the absolute error across them (mm).
Spatial bounds and parameter uncertainty. The analytic spline decoder enables post-training bounds on the predicted displacement over continuous spatial regions and uncertain operating parameters, without needing additional reference solutions. These bounds provide a rigorous guarantee that the model prediction lies within a specified range, which is critical for safety-critical applications such as nuclear fusion devices.
Spatial bounds. At fixed parameters, each displacement component on a spline cell can be expressed in Bernstein form (Romani & Sabin, 2004) as
u ^ j ( x ) = ∑ k Y j , k b k ( x ) ∑ k D k b k ( x ) , min k Y j , k D k ≤ u ^ j ( x ) ≤ max k Y j , k D k (14) \widehat{u}_j(x) = \frac{\sum_k Y_{j,k}b_k(x)}{\sum_k D_kb_k(x)}, \qquad\min_k \frac{Y_{j,k}}{D_k} \le\widehat{u}_j(x) \le\max_k \frac{Y_{j,k}}{D_k} \tag*{(14)} u j ( x ) = ∑ k D k b k ( x ) ∑ k Y j , k b k ( x ) , k min D k Y j , k ≤ u j ( x ) ≤ k max D k Y j , k (14)
Here Y j , k Y_{j,k} Y j , k and D k D_k D k are the Bernstein coefficients of the decoder numerator and normalisation denominator, respectively, and b k b_k b k are the spatial Bernstein basis functions. Since b k ( x ) ≥ 0 b_k(x) \ge0 b k ( x ) ≥ 0 and D k > 0 D_k > 0 D k > 0 , the prediction is a convex combination of the coefficient ratios Y j , k / D k Y_{j,k}/D_k Y j , k / D k (Pieg̈l & Tiller, 1996). Their extrema therefore enclose every point within the cell, including points between mesh nodes. The bound is loose when a cell is large compared with the variation of the field, since the Bernstein coefficients then lie well away from the function values. We tighten it by spatial subdivision. The de Casteljau algorithm splits a cell exactly in half along its longest side and gives each half its own Bernstein coefficients, describing the same spline on a smaller box (Boehm & Müller, [4 ] ). After n n n such splits, each cell is covered by 2 n 2^n 2 n sub-boxes. Equation (14) is applied to each sub-box, and as the sub-boxes shrink their coefficients converge to the function values (see Section B.4 for more details).
Parameter uncertainty. The same enclosure extends to an uncertain parameter box B \mathcal{B} B , where μ = ( α , T w ) ∈ B \boldsymbol{\mu} = (\alpha,T_w) \in\mathcal{B} μ = ( α , T w ) ∈ B . Only the spline coefficients change: at fixed parameters each is a number, whereas over B \mathcal{B} B each is a function of μ \boldsymbol{\mu} μ . We obtain these functions by propagating a second-order Taylor model through the fixed trained network (Makino & Berz, 1999), so that each physical spline coefficient takes the form
c i ( μ ) = c i 0 + a i T δ + 1 2 δ T H i δ + ε i , c_i(\boldsymbol{\mu}) = c_i^0 + \mathbf{a}_i^{\mathsf{T}}\boldsymbol{\delta} + \frac{1}{2}\boldsymbol{\delta}^{\mathsf{T}}H_i\boldsymbol{\delta} + \varepsilon_i, c i ( μ ) = c i 0 + a i T δ + 2 1 δ T H i δ + ε i ,
∣ ε i ∣ ≤ r i , δ = μ − μ 0 . (15) |\varepsilon_i| \le r_i,\qquad\boldsymbol{\delta} = \boldsymbol{\mu} - \boldsymbol{\mu}_0. \tag*{(15)} ∣ ε i ∣ ≤ r i , δ = μ − μ 0 . (15)
Here c i 0 c_i^0 c i 0 , a i \mathbf{a}_i a i , and H i H_i H i are the coefficient value, parameter gradient, and parameter Hessian at the expansion centre μ 0 \boldsymbol{\mu}_0 μ 0 , while r i r_i r i bounds the remainder uniformly over B \mathcal{B} B . The parameters enter only through the input normalisation of Equation (3); the affine operations that follow (lifting, scatter, Fourier layers and readout) carry the signed polynomial terms exactly, and only the nonlinear activations add to r i r_i r i .
The Bernstein transformation in Equation (14) is linear in the spline coefficients, so applying it to these polynomials gives each numerator coefficient as a quadratic in δ \delta δ plus a bounded remainder,
Y j , k ( μ ) = Y j , k 0 + π j , k ( δ ) + η j , k , ∣ η j , k ∣ ≤ r j , k . (16) Y_{j,k}(\mu) = Y^0_{j,k} + \pi_{j,k}(\delta) + \eta_{j,k}, \qquad\lvert\eta_{j,k}\rvert\le r_{j,k}. \tag*{(16)} Y j , k ( μ ) = Y j , k 0 + π j , k ( δ ) + η j , k , ∣ η j , k ∣ ≤ r j , k . (16)
The transformation acts on the polynomial coefficients before any extremum is taken, so all spline coefficients share the same δ \delta δ and their parameter contributions can cancel. Minimising and maximising π j , k \pi_{j,k} π j , k over B \mathcal{B} B and subtracting or adding r j , k r_{j,k} r j , k gives Y j , k − Y^-_{j,k} Y j , k − and Y j , k + Y^+_{j,k} Y j , k + . The denominator coefficients D k D_k D k depend only on geometry, so the convex-hull argument of Equation (14) applies unchanged:
min k Y j , k − D k ≤ u ^ j ( x , μ ) ≤ max k Y j , k + D k for all x in the cell and all μ ∈ B . (17) \min_k \frac{Y^-_{j,k}}{D_k} \le\hat{u}_j(x,\mu) \le\max_k \frac{Y^+_{j,k}}{D_k}
\qquad\text{for all } x \text{ in the cell and all } \mu\in\mathcal{B}. \tag*{(17)} k min D k Y j , k − ≤ u ^ j ( x , μ ) ≤ k max D k Y j , k + for all x in the cell and all μ ∈ B . (17)
Spatial subdivision tightens this enclosure exactly as before. Parameter subdivision is the complementary refinement. It partitions B \mathcal{B} B and re-centres the Taylor model on each part, which shrinks the nonlinear remainders, and the final enclosure is the union over all parts. In Figure 6 we show the displacement bounds over a parameter box B \mathcal{B} B such that α ∈ [ 1.4228 , 1.4428 ] × 10 − 5 K − 1 \alpha\in[1.4228, 1.4428] \times10^{-5}\,\mathrm{K}^{-1} α ∈ [ 1.4228 , 1.4428 ] × 1 0 − 5 K − 1 and T w ∈ [ 859 , 869 ] K T_w \in[859, 869]\,\mathrm{K} T w ∈ [ 859 , 869 ] K .
Figure 6. Rigorous enclosure of L-SplineFNO over the uncertain parameter box B \mathcal{B} B , obtained combining a Taylor-remainder bound with the convex-hull property of the spline.
These intervals describe the range of the surrogate prediction, including spatial variation, parameter variation and enclosure conservatism. They do not quantify prediction error against a reference numerical solution. Here we present the validated binary64 (FP64) results, which carry an end-to-end rounding certificate: outward rounding accounts for arithmetic error, so the enclosure contains the real-arithmetic surrogate defined by the stored weights. The same enclosure formulas can also be evaluated in ordinary FP32 or FP64, which is cheaper but gives no rounding certificate. All three are compared in Table 5 , and Section B.4 gives the construction and arithmetic assumptions.
Table 5. Parameter-box arithmetic comparison for the evaluation case at 1 × 1 \times 1 × , using the optimized validated binary64 implementation. Times are median per-box costs over three repetitions in the same A100 allocation with cached constants, excluding setup and warmup. Widths are displacement-magnitude enclosure widths at fixed face centres. The ordinary FP32/FP64 rows have no end-to-end rounding certificate; validated binary64 encloses the real surrogate defined by the stored weights.
Discussion
Strengths. L-SplineFNO constructs a structure-preserving learnable transfer that maintains compact support, partition of unity and C p − 1 C^{p-1} C p − 1 smoothness. This allows the model to scale linearly in both memory and compute for mesh size. L-SplineFNO achieves better performance on most benchmarks while remaining training-compute efficient and maintaining accuracy on super-resolution tasks. Because the transfer is parameter-efficient, it can be specialised per material or region at negligible cost. The explicit spline reconstruction provides inexpensive analytic gradients, useful for sensitivity analysis and physics-based objectives. The same reconstruction also yields rigorous bounds on the prediction over whole cells, not just at mesh nodes, and over boxes of uncertain parameters. Mesh-native and attention-based operators have no continuous reconstruction, so they provide no comparable guarantee.
Limitations. The spline transfer can smooth features in the target, such as kinks at material interfaces and stress concentrations around small geometric features, where mesh-native attention remains more accurate. Per-region output projections recover interface kinks, but the scatter still mixes regions on a shared latent grid; interface-aware or deformed transfers and local graph corrections near thin features could address this (Li et al., 2023a; 2025; 2023b). The fixed linear decoder also limits each output to the span of the spline basis, which sets an accuracy floor that no encoder or backbone can overcome, most visibly at sharp interfaces and on surfaces embedded in a volumetric grid. Finally, stencil cost grows rapidly with spline degree in three dimensions, so higher degrees exhaust memory on large meshes without relieving the smoothness bias (see § 1 ). Adaptive refinement of latent cells only where geometry or error demands it, as in NUNO (Liu et al., 2023), could ease both the accuracy floor and the memory cost while keeping cubic stencils. The enclosures bound the surrogate, not its error, and are conservative on account of the rigour. They loosen with cell size, parameter-box width and network depth; subdivision tightens them at proportional cost for a fixed geometry.
Future work. Several extensions follow directly from the spline structure. First, interface kinks could be represented exactly. Our decoder is a cubic B-spline basis, which is C p − 1 C^{p-1} C p − 1 at every knot; repeating a knot lowers the continuity at that location alone, so three coincident knots permit a jump in gradient and four a jump in value [10 ] (Piegl & Tiller 1996). Isogeometric analysis, which solves PDEs directly in a spline basis, uses exactly this to resolve material interfaces (Hughes et al., 2005). Second, the closed-form derivatives make physics-informed training inexpensive (Raissi et al., 2019; Li et al., 2023c); on multi-region problems, PDE-residual losses could be paired with penalties on flux continuity across interfaces, as in domain-decomposed physics-informed networks (Jagtap et al., 2020). Third, the parameter enclosures supply a natural acquisition function for active learning, placing new simulations where the enclosure is widest (Musekamp et al., 2025; Polanska et al., 2026). Fourth, because FiLM conditions on a global vector, time can enter as one more component of μ \mu μ , as in time-conditioned PDE foundation models (Herde et al., 2024; Takamoto et al., 2023); this would extend the framework from steady states to transient fields, complementing an operator-split formulation (Gopakumar et al., 2026). Finally, the enclosures currently bound the surrogate rather than the numerical simulation it approximates. Widening them by a calibrated bound on the surrogate-to-solver error, for instance through conformal prediction on held-out simulations (Gopakumar et al., 2025), would give intervals that contain the simulated field over the whole parameter box with a stated probability.
Acknowledgments
The Fusion Futures Programme partially funds Vignesh Gopakumar and Ander Gray. As announced by the UK Government in October 2023, Fusion Futures aims to provide holistic support for the development of the fusion sector. Ander Gray is partially funded by the Academic and Research Chaire “Architecture des Systèmes Complexes” Dassault Aviation, Naval Group, Dassault Systèmes, KNDS France, Agence de l’Innovation de Défense, Institut Polytechnique de Paris. Anima Anandkumar is supported in part by Bren endowed chair, ONR (MURI grant N00014-23-1-2654), and the AI2050 senior fellow program at Schmidt Sciences. The authors acknowledge funding from the Fusion Computing Lab collaboration between the Hartree Centre and UKAEA. This work was performed using resources provided by The Cambridge Service for Data Driven Discovery (CSD3) operated by the University of Cambridge Research Computing Service (www.csd3.cam.ac.uk), provided by Dell EMC and Intel using Tier-2 funding from the Engineering and Physical Sciences Research Council (capital grant EP/T022159/1), and DiRAC funding from the Science and Technology Facilities Council (www.dirac.ac.uk).
Notation
Table 2 lists the symbols used across the paper, grouped by topic, with a pointer to where each is defined. Bold lowercase denotes a vector at one node, query point or sample (h i \mathbf{h}_i h i , μ \boldsymbol{\mu} μ ); calligraphic letters denote collections over nodes or grid points (X \mathcal{X} X , G \mathcal{G} G , N \mathcal{N} N ); a subscript G G G marks a latent-grid quantity, and a hat marks a prediction (y ^ \hat{\mathbf{y}} y ^ ). A parenthesised superscript indexes a coordinate axis on coordinates (ω i ( ℓ ) \omega_i^{(\ell)} ω i ( ℓ ) ) and a sample on dataset quantities (y ( i ) \mathbf{y}^{(i)} y ( i ) ); it never denotes a power. Symbols local to one derivation, such as the intermediate Taylor-model terms in Section B.4 and the internals of individual baselines in Section F, are defined where they appear. Symbols with more than one meaning are collected in Table 3 .
Table 2. Notation.
Table 3. Symbols with more than one meaning. The context, given in the last column, always fixes which is meant.
Table 1 (PDF p. 22).
Learnable-Spline Construction
The spline transfers used in this work are built from one-dimensional cardinal B-spline kernels and lifted to d d d dimensions by tensor products. SplineFNO uses the fixed analytic spline B p B_p B p of degree p p p , while L-SplineFNO augments this kernel with a constrained learnable correction. For clarity, the main text presents the default cubic case p = 3 p=3 p = 3 , while the construction below is stated for a general spline degree p p p . The higher-degree variants considered in the spline-degree ablation follow the same construction with the corresponding support and smoothness.
This appendix proceeds in three parts. Section B.1 establishes the tensor-product properties of the analytic degree-p p p basis. Section B.2 introduces the constrained learned kernel and shows how compact support, partition of unity, and spline smoothness are retained. Non-negativity is not imposed on the learned correction, and Section B.3 introduces the safeguarded normalised scatter used when signed weights cancel.
From 1D to d d d -D: Tensor-Product Construction
Let B p : R → R B_p:\mathbb{R}\to\mathbb{R} B p : R → R denote the centred cardinal B-spline of degree p p p . We write B : = B p B:=B_p B := B p when the degree is fixed. The one-dimensional basis satisfies
(P1) compact support, supp B ⊆ [ − R p , R p ] \operatorname{supp} B\subseteq[-R_p,R_p] supp B ⊆ [ − R p , R p ] with R p = ( p + 1 ) / 2 R_p=(p+1)/2 R p = ( p + 1 ) /2 ;
(P2) partition of unity, ∑ n ∈ Z B ( t − n ) = 1 \sum_{n\in\mathbb{Z}}B(t-n)=1 ∑ n ∈ Z B ( t − n ) = 1 for all t t t ;
(P3) smoothness, B ∈ C p − 1 ( R ) B\in C^{p-1}(\mathbb{R}) B ∈ C p − 1 ( R ) ;
(P4) non-negativity, B ( t ) ≥ 0 B(t)\geq0 B ( t ) ≥ 0 .
For the default cubic case p = 3 p=3 p = 3 , R p = 2 R_p=2 R p = 2 and B ∈ C 2 B\in C^2 B ∈ C 2 , with
B 3 ( t ) = { 1 6 ( 4 − 6 t 2 + 3 ∣ t ∣ 3 ) , 0 ≤ ∣ t ∣ < 1 , 1 6 ( 2 − ∣ t ∣ ) 3 , 1 ≤ ∣ t ∣ < 2 , 0 , ∣ t ∣ ≥ 2. (18) B_3(t)=
\begin{cases}
\frac{1}{6}\left(4-6t^2+3\lvert t\rvert^3\right), & 0\leq\lvert t\rvert<1,\\
\frac{1}{6}\left(2-\lvert t\rvert\right)^3, & 1\leq\lvert t\rvert<2,\\
0, & \lvert t\rvert\geq2.
\end{cases} \tag*{(18)} B 3 ( t ) = ⎩ ⎨ ⎧ 6 1 ( 4 − 6 t 2 + 3 ∣ t ∣ 3 ) , 6 1 ( 2 − ∣ t ∣ ) 3 , 0 , 0 ≤ ∣ t ∣ < 1 , 1 ≤ ∣ t ∣ < 2 , ∣ t ∣ ≥ 2. (18)
Writing { ξ g ℓ ( ℓ ) } g ℓ = 0 S ℓ − 1 \{\xi_{g_\ell}^{(\ell)}\}_{g_\ell=0}^{S_\ell-1} { ξ g ℓ ( ℓ ) } g ℓ = 0 S ℓ − 1 for the control points along axis ℓ \ell ℓ , define
φ g ℓ ( ℓ ) ( x ( ℓ ) ) = B ( x ( ℓ ) − ξ g ℓ ( ℓ ) Δ ℓ ) , Φ g ( x ) = ∏ ℓ = 1 d φ g ℓ ( ℓ ) ( x ( ℓ ) ) . (19) \varphi_{g_\ell}^{(\ell)}(x^{(\ell)})=B\left(\frac{x^{(\ell)}-\xi_{g_\ell}^{(\ell)}}{\Delta_\ell}\right),\qquad
\boldsymbol{\Phi}_g(\mathbf{x})=\prod_{\ell=1}^{d}\varphi_{g_\ell}^{(\ell)}(x^{(\ell)}). \tag*{(19)} φ g ℓ ( ℓ ) ( x ( ℓ ) ) = B ( Δ ℓ x ( ℓ ) − ξ g ℓ ( ℓ ) ) , Φ g ( x ) = ℓ = 1 ∏ d φ g ℓ ( ℓ ) ( x ( ℓ ) ) . (19)
Row i i i of Φ \boldsymbol{\Phi} Φ stores Φ g ( ω i ) \boldsymbol{\Phi}_g(\boldsymbol{\omega}_i) Φ g ( ω i ) over the active multi-indices.
Compact support and sparsity. A product vanishes whenever any factor does, so Φ g ( x ) = 0 \boldsymbol{\Phi}_g(\mathbf{x})=0 Φ g ( x ) = 0 unless
∣ x ( ℓ ) − ξ g ℓ ( ℓ ) Δ ℓ ∣ < R p \left\lvert\frac{x^{(\ell)}-\xi_{g_\ell}^{(\ell)}}{\Delta_\ell}\right\rvert<R_p Δ ℓ x ( ℓ ) − ξ g ℓ ( ℓ ) < R p
along every coordinate axis. Since a degree-p p p cardinal spline has at most p + 1 p+1 p + 1 active shifts along each axis,
# { g : Φ g ( x ) ≠ 0 } ≤ ( p + 1 ) d . (20) \#\{g:\boldsymbol{\Phi}_g(\mathbf{x})\ne0\}\leq(p+1)^d. \tag*{(20)} # { g : Φ g ( x ) = 0 } ≤ ( p + 1 ) d . (20)
The default cubic case therefore gives a 4 d 4^d 4 d stencil.
Partition of unity. Because each axis sum is independent of the remaining indices,
∑ g Φ g ( x ) = ∏ ℓ = 1 d ( ∑ g ℓ φ g ℓ ( ℓ ) ( x ( ℓ ) ) ) = 1. (21) \sum_g\boldsymbol{\Phi}_g(\mathbf{x})=\prod_{\ell=1}^{d}\left(\sum_{g_\ell}\varphi_{g_\ell}^{(\ell)}(x^{(\ell)})\right)=1. \tag*{(21)} g ∑ Φ g ( x ) = ℓ = 1 ∏ d ( g ℓ ∑ φ g ℓ ( ℓ ) ( x ( ℓ ) ) ) = 1. (21)
The reconstruction therefore reproduces constant control-grid fields exactly in the interior. Near a domain boundary, part of the infinite-grid stencil falls outside G \mathcal{G} G . Renormalising the surviving basis values restores partition of unity over the valid stencil.
Smoothness. Since the tensor-product factors depend on disjoint coordinates,
∂ n Φ g ( x ) = ∏ ℓ = 1 d d n ℓ φ g ( ℓ ) d ( x ( ℓ ) ) n ℓ ( x ( ℓ ) ) . (22) \partial^n \Phi_{\mathbf{g}}(\mathbf{x}) = \prod_{\ell=1}^{d} \frac{d^{n_\ell}\varphi_{g}^{(\ell)}}{d(x^{(\ell)})^{n_\ell}}(x^{(\ell)}). \tag*{(22)} ∂ n Φ g ( x ) = ℓ = 1 ∏ d d ( x ( ℓ ) ) n ℓ d n ℓ φ g ( ℓ ) ( x ( ℓ ) ) . (22)
Each factor is continuous whenever n ℓ ≤ p − 1 n_\ell\le p-1 n ℓ ≤ p − 1 , so the tensor-product basis inherits the C p − 1 C^{p-1} C p − 1 smoothness of the one-dimensional spline.
Non-negativity. A product of non-negative factors is non-negative. Together with Equation (21) , the active basis values form a convex combination, so analytic spline reconstruction cannot overshoot the range of the active control-point values.
Implication for the transformation. Partition of unity makes a constant field exactly representable, Φ ( c 1 G ) = c 1 N \Phi(c\mathbf{1}_G)=c\mathbf{1}_N Φ ( c 1 G ) = c 1 N . Provided Φ \Phi Φ has full column rank, the regularised transformation in Equation (5) recovers this constant in the limit λ → 0 + \lambda\to0^+ λ → 0 + . The compact support in Equation (20) also bounds the number of basis evaluations per mesh point by ( p + 1 ) d (p+1)^d ( p + 1 ) d , independently of G G G .
From Analytic to Learned Kernels
L-SplineFNO augments the fixed degree-p p p spline with compact spline corrections. Along coordinate axis ℓ \ell ℓ ,
κ θ ( ℓ ) ( t ) = B ( t ) + ∑ j = 1 J ζ j ( ℓ ) B ( t − τ j h ) , ζ ( ℓ ) = V 0 α θ ( ℓ ) . (23) \kappa_{\boldsymbol{\theta}}^{(\ell)}(t)=B(t)+\sum_{j=1}^{J}\zeta_j^{(\ell)}B\left(\frac{t-\tau_j}{h}\right),\qquad\zeta^{(\ell)}=V_0\alpha_{\boldsymbol{\theta}}^{(\ell)}. \tag*{(23)} κ θ ( ℓ ) ( t ) = B ( t ) + j = 1 ∑ J ζ j ( ℓ ) B ( h t − τ j ) , ζ ( ℓ ) = V 0 α θ ( ℓ ) . (23)
where the correction centres τ j \tau_j τ j and relative width h h h are fixed and only α θ ( ℓ ) \alpha_{\boldsymbol{\theta}}^{(\ell)} α θ ( ℓ ) is trained. The cubic construction shown in the main text corresponds to p = 3 p=3 p = 3 .
Compact support and smoothness (P1, P3). Let R p = ( p + 1 ) / 2 R_p=(p+1)/2 R p = ( p + 1 ) /2 . We choose the correction centres and width to satisfy
∣ τ j ∣ ≤ R p ( 1 − h ) , 0 < h < 1. (24) |\tau_j| \le R_p(1-h),\qquad0<h<1. \tag*{(24)} ∣ τ j ∣ ≤ R p ( 1 − h ) , 0 < h < 1. (24)
The support of each correction is then contained within [ − R p , R p ] [-R_p,R_p] [ − R p , R p ] , so κ θ ( ℓ ) \kappa_{\boldsymbol{\theta}}^{(\ell)} κ θ ( ℓ ) retains the same bounded stencil as the analytic spline. Since both the base spline and every correction are degree-p p p splines, the learned kernel remains C p − 1 C^{p-1} C p − 1 . These two properties hold independently of the learned coefficients.
Partition of unity in the interior (P2). For a fractional cell position u ∈ [ 0 , 1 ) u\in[0,1) u ∈ [ 0 , 1 ) , let
O p ( u ) = { o ∈ Z : B ( u − o ) ≠ 0 } (25) \mathcal{O}_p(u)=\{o\in\mathbb{Z}:B(u-o)\ne0\} \tag*{(25)} O p ( u ) = { o ∈ Z : B ( u − o ) = 0 } (25)
denote the active analytic stencil. Its cardinality is at most p + 1 p+1 p + 1 , and
∑ o ∈ O p ( u ) B ( u − o ) = 1. \sum_{o\in\mathcal{O}_p(u)}B(u-o)=1. o ∈ O p ( u ) ∑ B ( u − o ) = 1.
For p = 3 p=3 p = 3 , this reduces to the familiar four-point stencil O 3 = { − 1 , 0 , 1 , 2 } \mathcal{O}_3=\{-1,0,1,2\} O 3 = { − 1 , 0 , 1 , 2 } for u ∈ [ 0 , 1 ) u\in[0,1) u ∈ [ 0 , 1 ) . The learned kernel retains partition of unity when its correction has zero stencil sum,
∑ j = 1 J ζ j ( ℓ ) ∑ o ∈ O p ( u ) B ( u − o − τ j h ) = 0. (26) \sum_{j=1}^{J}\zeta_j^{(\ell)}\sum_{o\in\mathcal{O}_p(u)}B\left(\frac{u-o-\tau_j}{h}\right)=0. \tag*{(26)} j = 1 ∑ J ζ j ( ℓ ) o ∈ O p ( u ) ∑ B ( h u − o − τ j ) = 0. (26)
Given collocation points { u q } q = 1 Q ⊂ [ 0 , 1 ) \{u_q\}_{q=1}^{Q}\subset[0,1) { u q } q = 1 Q ⊂ [ 0 , 1 ) , define
A q j = ∑ o ∈ O p ( u q ) B ( u q − o − τ j h ) , A ∈ R Q × J . (27) A_{qj}=\sum_{o\in\mathcal{O}_p(u_q)}B\left(\frac{u_q-o-\tau_j}{h}\right),\qquad A\in\mathbb{R}^{Q\times J}. \tag*{(27)} A q j = o ∈ O p ( u q ) ∑ B ( h u q − o − τ j ) , A ∈ R Q × J . (27)
Let V 0 ∈ R J × r V_0 \in\mathbb{R}^{J \times r} V 0 ∈ R J × r contain the right singular vectors spanning the numerical null space of A A A . We parameterise
ζ ( ℓ ) = V 0 α θ ( ℓ ) , α θ ( ℓ ) ∈ R r , (28) \zeta^{(\ell)} = V_0\alpha_{\theta}^{(\ell)}, \qquad\alpha_{\theta}^{(\ell)} \in\mathbb{R}^{r}, \tag*{(28)} ζ ( ℓ ) = V 0 α θ ( ℓ ) , α θ ( ℓ ) ∈ R r , (28)
which gives A ζ ( ℓ ) ≃ 0 A\zeta^{(\ell)} \simeq0 A ζ ( ℓ ) ≃ 0 to numerical precision for every value of the learned parameters. The matrix V 0 V_0 V 0 is computed once and remains fixed throughout training. We additionally verify Equation (26) on a dense set of fractional coordinates.
Partition of unity at the boundary (P2). For mesh point ω i \omega_i ω i , define the valid one-dimensional stencil
S ℓ ( i ) = { g ℓ ∈ { 0 , … , S ℓ − 1 } : ∣ t i ( ℓ ) − g ℓ ∣ < R p } . (29) S_\ell(i) = \left\{g_\ell\in\{0,\ldots,S_\ell-1\} : \left|t_i^{(\ell)} - g_\ell\right| < R_p\right\}. \tag*{(29)} S ℓ ( i ) = { g ℓ ∈ { 0 , … , S ℓ − 1 } : t i ( ℓ ) − g ℓ < R p } . (29)
The tensor-product transfer weights are
( W θ ) i , g = ∏ ℓ = 1 d κ θ ( ℓ ) ( t i ( ℓ ) − g ℓ ) ∑ q ∈ S ℓ ( i ) κ θ ( ℓ ) ( t i ( ℓ ) − q ) , g ∈ S ( i ) , (30) (W_\theta)_{i,\mathbf{g}} = \prod_{\ell=1}^{d} \frac{\kappa_{\theta}^{(\ell)}\left(t_i^{(\ell)} - g_\ell\right)}{\sum_{q\in S_\ell(i)} \kappa_{\theta}^{(\ell)}\left(t_i^{(\ell)} - q\right)}, \qquad\mathbf{g} \in S(i), \tag*{(30)} ( W θ ) i , g = ℓ = 1 ∏ d ∑ q ∈ S ℓ ( i ) κ θ ( ℓ ) ( t i ( ℓ ) − q ) κ θ ( ℓ ) ( t i ( ℓ ) − g ℓ ) , g ∈ S ( i ) , (30)
and zero otherwise, where S ( i ) = ∏ ℓ = 1 d S ℓ ( i ) S(i) = \prod_{\ell=1}^{d} S_\ell(i) S ( i ) = ∏ ℓ = 1 d S ℓ ( i ) . The explicit normalisation restores partition of unity over the surviving entries whenever the denominator is nonzero. Each mesh point therefore interacts with at most ( p + 1 ) d (p+1)^d ( p + 1 ) d latent-grid points.
Non-negativity is surrendered (P4). The null-space parameterisation does not constrain the sign of the learned correction. Consequently, κ θ ( ℓ ) \kappa_{\theta}^{(\ell)} κ θ ( ℓ ) may contain negative lobes. The learned transfer retains compact support, C p − 1 C^{p-1} C p − 1 smoothness, and partition of unity to numerical precision, but no longer has the convex-combination property of the analytic spline. Signed contributions may therefore cancel when they are accumulated at a latent-grid point.
Stable Normalised Scatter and Density Feature
The main text defines
n g = ∑ i = 1 N ( W θ ) i , g X i , d g = ∑ i = 1 N ( W θ ) i , g . n_{\mathbf{g}} = \sum_{i=1}^{N}(W_\theta)_{i,\mathbf{g}}\mathcal{X}_i,\qquad d_{\mathbf{g}} = \sum_{i=1}^{N}(W_\theta)_{i,\mathbf{g}}. n g = i = 1 ∑ N ( W θ ) i , g X i , d g = i = 1 ∑ N ( W θ ) i , g .
Because the learned weights may be signed, d g d_{\mathbf{g}} d g can approach zero through cancellation even when the corresponding grid point is well covered. We therefore also accumulate the total unsigned weight
a g = ∑ i = 1 N ∣ ( W θ ) i , g ∣ (31) a_{\mathbf{g}} = \sum_{i=1}^{N}\left|(W_\theta)_{i,\mathbf{g}}\right| \tag*{(31)} a g = i = 1 ∑ N ∣ ( W θ ) i , g ∣ (31)
and use the sign preserving safeguarded denominator
d ~ g = sgn + ( d g ) max { ∣ d g ∣ , η a g , ε } , (32) \widetilde{d}_{\mathbf{g}} = \operatorname{sgn}_{+}(d_{\mathbf{g}})\max\left\{\left|d_{\mathbf{g}}\right|,\eta a_{\mathbf{g}},\varepsilon\right\}, \tag*{(32)} d g = sgn + ( d g ) max { ∣ d g ∣ , η a g , ε } , (32)
where sgn + ( z ) = − 1 \operatorname{sgn}_{+}(z)=-1 sgn + ( z ) = − 1 if z < 0 z<0 z < 0 and + 1 +1 + 1 otherwise, η ∈ ( 0 , 1 ] \eta\in(0,1] η ∈ ( 0 , 1 ] , and ε > 0 \varepsilon>0 ε > 0 . The normalised feature is then
X ~ g = n g d ~ g . \widetilde{\mathcal{X}}_{\mathbf{g}} = \frac{n_{\mathbf{g}}}{\widetilde{d}_{\mathbf{g}}}. X g = d g n g .
Equivalently, defining D θ = diag ( d ~ g ) D_\theta=\operatorname{diag}(\widetilde{d}_{\mathbf{g}}) D θ = diag ( d g ) gives the compact form in Equation (9) .
The relative floor bounds the amplification caused by cancellation,
∥ X ~ g ∥ ∞ ≤ 1 η max i : ( W θ ) i , g ≠ 0 ∥ X i ∥ ∞ (33) \left\|\widetilde{\mathcal{X}}_{\mathbf{g}}\right\|_{\infty} \leq\frac{1}{\eta}\max_{i:(W_\theta)_{i,\mathbf{g}}\neq0}\left\|\mathcal{X}_i\right\|_{\infty} \tag*{(33)} X g ∞ ≤ η 1 i : ( W θ ) i , g = 0 max ∥ X i ∥ ∞ (33)
whenever a g > 0 a_{\mathbf{g}}>0 a g > 0 . The absolute floor ε \varepsilon ε handles uncovered or numerically degenerate grid points. For non-negative weights, d g = a g d_{\mathbf{g}}=a_{\mathbf{g}} d g = a g , so the relative safeguard is inactive for η ≤ 1 \eta\leq1 η ≤ 1 .
Sampling-density feature. Normalised scatter reduces variations caused solely by local point density, but it also removes information about mesh coverage. We retain this information through the fixed analytic spline basis,
ρ g = ∑ i = 1 N Φ i , g , X G = P ρ [ X ~ G ; log ( 1 + ρ ) ] . (34) \rho_{\mathrm{g}} = \sum_{i=1}^{N} \Phi_{i,\mathrm{g}}, \qquad\mathcal{X}_{\mathrm{G}} = P_{\rho}\left[\widetilde{\mathcal{X}}_{\mathrm{G}};\log(1+\rho)\right]. \tag*{(34)} ρ g = i = 1 ∑ N Φ i , g , X G = P ρ [ X G ; log ( 1 + ρ ) ] . (34)
Because Φ \Phi Φ is non-negative, ρ \rho ρ provides a non-negative analytic measure of local mesh coverage that is unaffected by cancellation in the learned scatter. The pointwise projection P ρ P_{\rho} P ρ is initialised to preserve the scattered feature channels and assign zero weight to the density feature, so adding ρ \rho ρ does not change the initial model output.
Implementation. All transfers remain local. For degree p p p , each mesh point contributes to at most ( p + 1 ) d (p+1)^d ( p + 1 ) d latent-grid points, and the products in Equation (9) are implemented as indexed scatter-add operations over these active mesh-to-grid edges, as detailed in Section C.
Continuous spatial bounds and parameter uncertainty
Setup. All bounds are computed post-training, using the checking across the 29.2M nodal points. Geometry, model weights, preprocessing, and material assignments remain fixed. We consider three local material regions around held-out sample, one each in steel, copper, and graphite. No reference displacement values enter the enclosure construction.
Continuous spatial bounds. For fixed parameters, each physical displacement component on a decoder cell is a rational tensor-product cubic spline,
u ^ j ( x ) = Y j ( x ) D ( x ) . \hat{u}_j(x) = \frac{Y_j(x)}{D(x)}. u ^ j ( x ) = D ( x ) Y j ( x ) .
The numerator contains the physical output coefficients, including the affine readout, while the denominator accounts for the normalisation of spline support truncated at the decoder boundary. In the interior, this denominator is one.
We cover the requested material region with restricted decoder-cell boxes. On each box, a linear basis transformation expresses the numerator and denominator in tensor-product Bernstein form ((Piegl & Tiller, 1996)):
Y j ( x ) = ∑ k Y j , k b k ( x ) , D ( x ) = ∑ k D k b k ( x ) . Y_j(x) = \sum_k Y_{j,k} b_k(x), \qquad D(x) = \sum_k D_k b_k(x). Y j ( x ) = k ∑ Y j , k b k ( x ) , D ( x ) = k ∑ D k b k ( x ) .
Here k k k is a multi-index over the spatial Bernstein coefficients. Since the basis functions are nonnegative and sum to one, positive denominator coefficients give
u ^ j ( x ) = ∑ k w k ( x ) Y j , k D k , w k ( x ) = D k b k ( x ) ∑ ℓ D ℓ B ℓ ( x ) , w k ( x ) ≥ 0 , ∑ k w k ( x ) = 1. \hat{u}_j(x) = \sum_k w_k(x)\frac{Y_{j,k}}{D_k}, \qquad w_k(x) = \frac{D_k b_k(x)}{\sum_{\ell}D_\ell B_\ell(x)}, \qquad w_k(x) \ge0, \qquad\sum_k w_k(x) = 1. u ^ j ( x ) = k ∑ w k ( x ) D k Y j , k , w k ( x ) = ∑ ℓ D ℓ B ℓ ( x ) D k b k ( x ) , w k ( x ) ≥ 0 , k ∑ w k ( x ) = 1.
Thus the prediction is a convex combination of the coefficient ratios, giving the continuous enclosure
L j = min k Y j , k D k , U j = max k Y j , k D k . L_j = \min_k \frac{Y_{j,k}}{D_k}, \qquad U_j = \max_k \frac{Y_{j,k}}{D_k}. L j = k min D k Y j , k , U j = k max D k Y j , k .
These are the spatial bounds in eq. (14) . Their validity follows from the basis representation, rather than evaluations at sampled points.
We tighten the enclosure using de Casteljau subdivision, which restricts the numerator and denominator together to smaller spatial boxes. Each step splits a box at the midpoint of its longest side, so depth n n n covers every original decoder cell with 2 n 2^n 2 n sub-boxes. The split is an exact change of basis: each sub-box carries Bernstein coefficients for the same rational spline, and these approach the function values as the box shrinks. Taking the smallest lower endpoint and largest upper endpoint across all covering boxes gives a regional enclosure. Each box uses the decoder head associated with its material. Table 4 reports the resulting trade-off: the median width halves between depths zero and three, while the cost roughly doubles with every further level.
Table 4. MAST-U spatial subdivision ablation at nominal parameters. Depth zero uses no subdivision within each original spline cell; each subsequent depth doubles the number of sub-boxes per cell. Widths are displacement-magnitude enclosure widths over the same 4,077,312 displayed triangles, with equal weight per triangle. Times are medians of three warm, synchronised bound calculations on one NVIDIA A100 GPU, excluding model loading, source encoding and reporting. Depth three is the current subdivision setting.
Taylor propagation through the network. For the prescribed parameter box, we propagate a second-order polynomial plus a bounded interval remainder through the unchanged network (Makino & Berz, 1999). Writing δ = μ − μ 0 \boldsymbol{\delta}=\boldsymbol{\mu}-\boldsymbol{\mu}_0 δ = μ − μ 0 , an intermediate scalar quantity is represented as
z ( μ ) = z 0 + a T δ + 1 2 δ T H δ + e , ∣ e ∣ ≤ R . z(\boldsymbol{\mu})=z_0+\mathbf{a}^{\mathsf{T}}\boldsymbol{\delta}+\frac{1}{2}\boldsymbol{\delta}^{\mathsf{T}}H\boldsymbol{\delta}+e,\qquad|e|\le R. z ( μ ) = z 0 + a T δ + 2 1 δ T H δ + e , ∣ e ∣ ≤ R .
The parameter endpoints are converted using the model’s fixed normalisation before propagation. With two parameters, the retained polynomial consists of six coefficient fields: a constant, two linear terms, and three quadratic terms.
FiLM conditioning introduces the parameter dependence. Affine lifting, fixed-geometry scatter, Fourier processing, and physical readout propagate the signed polynomial coefficient fields. Remainders are enclosed using absolute-weight or norm bounds; Fourier remainder propagation uses Parseval-based norm envelopes. The same parameter symbols are shared across source points, channels, and spline coefficients, rather than treating their perturbations as independent intervals.
Nonlinear activations require an additional remainder. For a scalar activation input z 0 + L + Q + e z_0+L+Q+e z 0 + L + Q + e , where L L L and Q Q Q are the retained linear and quadratic terms, let ∣ L ∣ ≤ s |L|\le s ∣ L ∣ ≤ s , ∣ Q ∣ ≤ t |Q|\le t ∣ Q ∣ ≤ t , and ∣ e ∣ ≤ R |e|\le R ∣ e ∣ ≤ R . The retained second-order activation polynomial is
g ( z 0 ) + g ′ ( z 0 ) ( L + Q ) + 1 2 g ′ ′ ( z 0 ) L 2 . g(z_0)+g'(z_0)(L+Q)+\frac{1}{2}g''(z_0)L^2. g ( z 0 ) + g ′ ( z 0 ) ( L + Q ) + 2 1 g ′′ ( z 0 ) L 2 .
A valid bound on the discarded terms is
R n e w ≤ L g R + ∣ g ′ ′ ( z 0 ) ∣ ( s t + 1 2 t 2 ) + 1 6 M 3 ( z 0 , s + t ) ( s + t ) 3 , R_{\mathrm{new}}\le L_gR+|g''(z_0)|\left(st+\frac{1}{2}t^2\right)+\frac{1}{6}M_3(z_0,s+t)(s+t)^3, R new ≤ L g R + ∣ g ′′ ( z 0 ) ∣ ( s t + 2 1 t 2 ) + 6 1 M 3 ( z 0 , s + t ) ( s + t ) 3 ,
where L g L_g L g bounds ∣ g ′ ∣ |g'| ∣ g ′ ∣ and M 3 ( z 0 , v ) M_3(z_0,v) M 3 ( z 0 , v ) bounds ∣ g ′ ′ ′ ∣ |g'''| ∣ g ′′′ ∣ on [ z 0 − v , z 0 + v ] [z_0-v,z_0+v] [ z 0 − v , z 0 + v ] . This accounts both for the incoming remainder and for higher-order products discarded during composition. For exact GELU, g ( z ) = z Φ ( z ) g(z)=z\Phi(z) g ( z ) = z Φ ( z ) ,
g ′ ′ ′ ( z ) = ( z 3 − 4 z ) φ ( z ) , g'''(z)=(z^3-4z)\varphi(z), g ′′′ ( z ) = ( z 3 − 4 z ) φ ( z ) ,
so the third-derivative envelope is obtained analytically from interval endpoints and contained stationary points. The implementation also evaluates a direct composition remainder bound and takes the smaller of the two valid envelopes.
Combining parameter and spatial bounds. The joint bound reuses the spatial construction above unchanged; only the coefficients it is applied to differ. After physical readout, every spline coefficient has a quadratic parameter polynomial and a bounded remainder. We transform the signed polynomial coefficient fields into spatial Bernstein form before computing their parameter extrema. This preserves cancellations that would be lost by first replacing each spline coefficient with an independent interval.
On each spatial patch, write
Y j , k ( μ ) = Y j , k 0 + π j , k ( δ ) + η j , k , ∣ η j , k ∣ ≤ r j , k . Y_{j,k}(\boldsymbol{\mu})=Y^0_{j,k}+\boldsymbol{\pi}_{j,k}(\boldsymbol{\delta})+\eta_{j,k},\qquad|\eta_{j,k}|\le r_{j,k}. Y j , k ( μ ) = Y j , k 0 + π j , k ( δ ) + η j , k , ∣ η j , k ∣ ≤ r j , k .
where π j , k \pi_{j,k} π j , k contains the linear and quadratic parameter terms. We obtain its extrema π j , k ± \pi_{j,k}^{\pm} π j , k ± over the parameter-offset box by checking corners, admissible stationary points on its edges, and any admissible interior stationary point. For a singular quadratic Hessian, boundary candidates suffice; degenerate parameter boxes are handled in their remaining dimensions.
The lower and upper numerator-coefficient bounds are
Y j , k − = Y j , k 0 + π j , k − − r j , k , Y j , k + = Y j , k 0 + π j , k + + r j , k . Y_{j,k}^{-}=Y_{j,k}^{0}+\pi_{j,k}^{-}-r_{j,k}, \qquad Y_{j,k}^{+}=Y_{j,k}^{0}+\pi_{j,k}^{+}+r_{j,k}. Y j , k − = Y j , k 0 + π j , k − − r j , k , Y j , k + = Y j , k 0 + π j , k + + r j , k .
Since the denominator is parameter-independent and its Bernstein coefficients are positive, the spatial convex-hull argument gives
min k Y j , k − D k ≤ u ^ j ( x , μ ) ≤ max k Y j , k + D k \min_k \frac{Y_{j,k}^{-}}{D_k}\le\hat{u}_j(x,\mu)\le\max_k \frac{Y_{j,k}^{+}}{D_k} k min D k Y j , k − ≤ u ^ j ( x , μ ) ≤ k max D k Y j , k +
for every position and parameter setting in the covered product domain.
Refinement and magnitude ranges. Spatial subdivision tightens the Bernstein enclosure. Parameter subdivision reduces the local nonlinear remainders: we partition the complete original parameter box, recenter the Taylor model on every leaf, and take the union of all leaf enclosures. Every leaf must be included to retain coverage of the original uncertainty range.
For component intervals [ L j , U j ] [L_j,U_j] [ L j , U j ] , a displacement-magnitude enclosure follows from the nearest and furthest points of the component rectangle from the origin:
m − = ( ∑ j dist ( 0 , [ L j , U j ] ) 2 ) 1 / 2 , m + = ( ∑ j max { L j 2 , U j 2 } ) 1 / 2 . m^{-}=\left(\sum_j \operatorname{dist}(0,[L_j,U_j])^2\right)^{1/2}, \qquad m^{+}=\left(\sum_j \max\{L_j^2,U_j^2\}\right)^{1/2}. m − = ( j ∑ dist ( 0 , [ L j , U j ] ) 2 ) 1/2 , m + = ( j ∑ max { L j 2 , U j 2 } ) 1/2 .
Magnitude enclosures are computed before combining parameter leaves, avoiding the unnecessary combination of component extrema from different leaves.
At spatial subdivision depth three, with physical parameter half-widths 10 − 7 K − 1 10^{-7}\ \mathrm{K}^{-1} 1 0 − 7 K − 1 in α \alpha α and 5 K 5\ \mathrm{K} 5 K in T w T_w T w , the reported second-order results are
Table 28.
All parameter leaves cover the same original uncertainty box; these results do not arise from shrinking that box.
Interval arithmetic and rounding certificates. We compare arithmetic implementations on the held-out case as shown in Figure 5 and Figure 6 , using the same frozen checkpoint and the 1× parameter box with physical half-widths 10 − 7 K − 1 10^{-7}\ \mathrm{K}^{-1} 1 0 − 7 K − 1 in α \alpha α and 5 K 5\ \mathrm{K} 5 K in T w T_w T w . Every evaluation encodes all 29,183,754 native source nodes and computes displacement-magnitude intervals at the same 4,077,312 display face centres. These are parameter bounds at fixed positions; they differ from the continuous spatial-subdivision experiment in Table 4 and the partitioned local-region ranges above. No parameter subdivision is used here.
The original formulation implements the real-arithmetic Taylor and spline inequalities using ordinary FP32 scatter, FFT, nonlinear and SVD calculations. The updated ordinary FP32/FP64 formulation retains the same second-order construction but replaces numerical norm estimates by verified upper bounds: SVD proposals are checked using outward-enclosed Hermitian LDL pivots, with conservative norm fallbacks. Quadratic parameter extrema are evaluated with outward binary64 intervals, including stationary values and conservative fallbacks when the Hessian determinant cannot be resolved safely. These checks validate those individual calculations; the remaining ordinary propagation is not an end-to-end rounding certificate.
Validated binary64 instead propagates outward-enclosed arithmetic through parameter normalisation, coordinate/positional preprocessing, FiLM, encoder accumulation, Fourier/Taylor propagation, nonlinear remainders and the surface decoder. Arithmetic uncertainty is included alongside the Taylor remainder. This encloses the real surrogate defined by the exact stored weight values under the checked arithmetic assumptions. It does not certify every output of the existing vendor FP32/TF32 inference program.
The validated implementation specializes interval products by exact stored weights and fuses spline gathers and transcendental series recurrences into CUDA kernels. These kernels retain explicit directed binary64 rounding, the original accumulation order and analytic remainder bounds. For the measured box, the saved magnitude endpoints are identical to those from the earlier validated implementation at every face centre.
Table 5 reports medians of three CUDA-synchronised repetitions in one NVIDIA A100-SXM4-80GB allocation. The original and updated implementations ran in separate allocations of the same GPU model, with matched geometry, parameter boxes, chunk sizes and thread counts. Timed calls include source/query preprocessing and transfers, FiLM conditioning, encoding, FNO propagation and surface queries. They exclude model restoration, mesh loading, constant preparation, warmup, statistics, archive writes and plotting. Width statistics use the first measured repetition, with equal weight per face centre. Every call rebuilds its Taylor state and output bounds; only loaded fixed geometry and weight/grid-dependent constants are reused. Source features and spline stencils are recomputed.
The original and updated FP32 widths agree at the precision of Table 5 : their largest pointwise width difference is 7.97 × 10 − 6 7.97 \times10^{-6} 7.97 × 1 0 − 6 mm and their largest endpoint difference is 3.41 × 10 − 5 3.41 \times10^{-5} 3.41 × 1 0 − 5 mm. Thus this case shows negligible change in enclosure width from verifying norms and extrema. The updated FP32 surface-query stage costs about 40.07 s versus 8.45 s for the original, consistent with its more expensive enclosed extremum calculation. Validated binary64 increases the median width by approximately 1.53% and the maximum by 1.72%. It also encloses real preprocessing, whereas ordinary FP64 promotes the captured FP32 processed features, so this increase is not solely a final rounding pad. Agreement of computed widths and sampled containment checks are empirical diagnostics; they do not establish the enclosure guarantee of ordinary arithmetic. Caching reduces repeated constant-construction cost in both formulations and is independent of rounding validation.
Effect of parameter-box size in FP32. Table 6 varies both physical parameter half-widths together for the same evaluation case, using the updated formulation with verified norms/extrema and ordinary FP32 propagation. The checkpoint, native source nodes and face centres remain fixed. Width statistics again use the first measured repetition, with no parameter subdivision. Larger boxes increase both the allowed field variation and enclosure conservatism.
Table 6. FP32 displacement-magnitude enclosure widths for eval_02 across three parameter-box sizes. Both half-widths scale by the box factor about the same nominal parameters. Median and maximum widths are computed over all 4,077,312 fixed face centres. These are ordinary-arithmetic enclosure calculations, without an end-to-end rounding certificate.
Setup, warmup and constant reuse. Model restoration and geometry loading cost 10.69 s for the original run and 12.25 s shared across the three updated methods. Table 7 gives the additional constant preparation/check and warmup costs. The original formulation builds ordinary spectral constants in 2.64 s. The updated run first builds verified Fourier and skip norm constants in FP32 mode (336.39 s); FP64 and validated binary64 reuse the same exact promoted weight/grid keys, with cache checks of 0.46 s and 0.41 s, respectively. The aggregate verified-constant preparation/check cost is 337.25 s. An independent cold start of an updated method must build the verified constants; the later modes’ lookup times do not represent cold-start setup costs. Each method receives one full-workload warmup on eval_02 at 0.5 × 0.5 \times 0.5 × before any timed repetitions.
Table 7. One-time constant preparation/checks and one full warmup per method, excluding the separately reported restore/geometry cost. The updated methods ran in order FP32, FP64, validated binary64 in one process: FP32 builds the shared verified constants, while the last two rows report cache checks. Warmup uses the 0.5× box; the measured comparison in Table 5 uses 1×.
Figure 8. FP32 displacement-magnitude enclosure widths for eval_02 over the 0.5×, 1× and 2× parameter boxes in Table 6 . Both parameter half-widths scale together about the same nominal parameters, with no parameter subdivision. The geometry and camera are identical to Figure 7 . A shared linear colour scale spanning the largest width in these three panels shows the growth of field variation and enclosure conservatism without clipping or independent panel rescaling. Widths are evaluated at fixed face centres using the updated ordinary FP32 formulation.
Figure 7. Displacement-magnitude enclosure widths at fixed face centres for eval_02 over the 1 × 1 \times 1 × parameter box, using ordinary FP32, ordinary FP64 and validated binary64. The FP32/FP64 calculations use verified norms and quadratic extrema with ordinary propagation; validated binary64 encloses the real stored-weight surrogate. All panels use the manuscript camera, the same geometry and a shared linear colour scale without clipping. The distributions correspond to Table 5 ; panel annotations give the median and maximum widths over all 4,077,312 face centres.
These bounds enclose the surrogate prediction. They do not quantify prediction error relative to a reference solution or bound PDE discretisation, solver convergence, or physical modelling error.
L-SplineFNO Implementation
L-SplineFNO scatters point features onto a regular grid with learned spline weights, processes the grid with an FNO, and gathers the result at query points with the fixed analytic spline. The listings describe these cubic transfers in PyTorch-style pseudocode using ordinary autograd. We describe the single-scale, single-region path with an affine readout, followed by the extensions and memory optimisations.
Notation. We use B b a t c h B_{\mathrm{batch}} B batch samples per batch, C C C feature channels (the width c c c of the main text), N N N input points and N q N_q N q output query points in d d d spatial dimensions. The regular grid has S ℓ S_\ell S ℓ nodes along axis ℓ \ell ℓ and G = ∏ ℓ = 1 d S ℓ G = \prod_{\ell=1}^{d} S_\ell G = ∏ ℓ = 1 d S ℓ nodes in total. Each point interacts with at most K = 4 d K = 4^d K = 4 d grid nodes. The listings abbreviate B b a t c h B_{\mathrm{batch}} B batch as B B B ; all tensors and kernels must have compatible devices and floating-point dtypes.
Step 1: neighbouring grid nodes and their weights. The stencil of a point is the set of grid nodes it contributes to. Along axis ℓ \ell ℓ , with grid origin ξ 0 ( ℓ ) \xi_0^{(\ell)} ξ 0 ( ℓ ) and spacing Δ ℓ \Delta_\ell Δ ℓ , the coordinate t i ( ℓ ) = ( ω i ( ℓ ) − ξ 0 ( ℓ ) ) / Δ ℓ t_i^{(\ell)} = (\omega_i^{(\ell)}-\xi_0^{(\ell)})/\Delta_\ell t i ( ℓ ) = ( ω i ( ℓ ) − ξ 0 ( ℓ ) ) / Δ ℓ gives the position of point i i i in grid cells. Its four candidate neighbours are g ℓ = ⌊ t i ( ℓ ) ⌋ + o g_\ell= \lfloor t_i^{(\ell)} \rfloor+ o g ℓ = ⌊ t i ( ℓ ) ⌋ + o with o ∈ { − 1 , 0 , 1 , 2 } o \in\{-1,0,1,2\} o ∈ { − 1 , 0 , 1 , 2 } , at distances t i ( ℓ ) − g ℓ ∈ [ − 2 , 2 ) t_i^{(\ell)}-g_\ell\in[-2,2) t i ( ℓ ) − g ℓ ∈ [ − 2 , 2 ) . Each neighbour is weighted by the kernel evaluated at that distance. Neighbours that fall outside the grid receive zero weight, and the remaining weights are normalised along each axis before being multiplied across axes:
v i , g ℓ ( ℓ ) = 1 0 ≤ g ℓ < S ℓ κ θ ( ℓ ) ( t i ( ℓ ) − g ℓ ) sgn + ( s i ℓ ) max ( ∣ s i ℓ ∣ , δ ) , ( W θ ) i , g = ∏ ℓ = 1 d v i , g ℓ ( ℓ ) (35) v_{i,g_\ell}^{(\ell)} = \frac{\mathbf{1}_{0\leq g_\ell<S_\ell}\kappa_\theta^{(\ell)}(t_i^{(\ell)}-g_\ell)}{\operatorname{sgn}_{+}(s_{i\ell})\max(|s_{i\ell}|,\delta)}, \qquad(W_\theta)_{i,\mathbf{g}} = \prod_{\ell=1}^{d}v_{i,g_\ell}^{(\ell)} \tag*{(35)} v i , g ℓ ( ℓ ) = sgn + ( s i ℓ ) max ( ∣ s i ℓ ∣ , δ ) 1 0 ≤ g ℓ < S ℓ κ θ ( ℓ ) ( t i ( ℓ ) − g ℓ ) , ( W θ ) i , g = ℓ = 1 ∏ d v i , g ℓ ( ℓ ) (35)
where s i ℓ s_{i\ell} s i ℓ is the sum of the valid kernel weights of point i i i along axis ℓ \ell ℓ , sgn + ( s ) = − 1 \operatorname{sgn}_{+}(s)=-1 sgn + ( s ) = − 1 for s < 0 s<0 s < 0 and + 1 +1 + 1 otherwise, and δ \delta δ prevents division by a near-zero sum. Whenever δ \delta δ is inactive, each axis’s weights sum to one, and hence so does each row of W θ W_\theta W θ . We use δ = 10 − 3 \delta=10^{-3} δ = 1 0 − 3 for the learned kernel. The analytic kernel uses δ = 10 − 12 \delta=10^{-12} δ = 1 0 − 12 ; this floor is inactive for points inside the grid bounds.
The learned kernel follows Equation (7) , with J = 9 J=9 J = 9 correction centres uniformly spaced on [ − 1 , 1 ] [-1,1] [ − 1 , 1 ] and width h = 1 / 2 h=1/2 h = 1/2 by default. The coefficients c = V 0 α c=V_0\alpha c = V 0 α preserve the interior partition of unity to numerical precision. A float64 SVD of the 257 × 9 257\times9 257 × 9 collocation matrix in Section B.2 gives six free parameters per axis; V 0 V_0 V 0 is cached per kernel configuration. Initialising α = 0 \alpha=0 α = 0 recovers the analytic cubic kernel, not the least-squares encoder of SplineFNO. The implementation evaluates the correction using a piecewise-cubic table and Horner’s rule. Listing 1 stores the four neighbours per axis separately; their tensor product is formed only for the point chunk being processed.
import torch
from torch import nn
# Assumed helpers: C2SplineKernel1D is the learned kernel;
# cubic_bspline is the analytic kernel; _scatter_axis builds
# the per-axis weights; _FactoredStencil stores their factors.
# One trainable kernel per axis; create these once in the model.
d = 3 # use 2 for planar grids
kernels = nn.ModuleList([C2SplineKernel1D() for _ in range(d)])
def stencil(coords, axes, kernels, floor=1e-3):
# coords: [B, d, N]; axes[ax]: grid-node coordinates along axis ax.
# Returns, per axis, [B, N, 4] neighbour indices and weights.
pairs = [_scatter_axis(coords[:, ax], axes[ax], kernels[ax],
pou_floor=floor) for ax in range(len(axes))]
indices, weights = zip(*pairs)
return _FactoredStencil(indices, weights, [len(a) for a in axes])
# Learned weights at the source points; fixed analytic weights at the queries.
# src = stencil(source_coords, axes, kernels)
# qry = stencil(query_coords, axes, [cubic_bspline] * d, floor=1e-12)
Listing 1: Build a stencil: the four neighbouring grid nodes and their normalised weights along each axis, stored per axis.
Step 2: scatter to the grid and gather back. Scatter adds each input point’s features to its stencil nodes, weighted by W θ W_\theta W θ . For features H ∈ R B b a t c h × N × C H \in\mathbb{R}^{B_{\mathrm{batch}}\times N\times C} H ∈ R B batch × N × C , each grid node g \mathbf{g} g accumulates three sums (batch index omitted):
n g = ∑ i ( W θ ) i , g H i , d g = ∑ i ( W θ ) i , g , a g = ∑ i ∣ ( W θ ) i , g ∣ , (36) n_g = \sum_i (W_\theta)_{i,g}H_i,\qquad d_g = \sum_i (W_\theta)_{i,g},\qquad a_g = \sum_i |(W_\theta)_{i,g}|, \tag*{(36)} n g = i ∑ ( W θ ) i , g H i , d g = i ∑ ( W θ ) i , g , a g = i ∑ ∣ ( W θ ) i , g ∣ , (36)
the weighted feature sum, the signed weight sum and the absolute weight sum. Without safeguards, the normalised value n g / d g n_g/d_g n g / d g reproduces constant features wherever d g ≠ 0 d_g \ne0 d g = 0 . Signed weights can, however, cancel. We therefore use Equation (32) with η = 0.1 \eta= 0.1 η = 0.1 and ε = 10 − 6 \varepsilon= 10^{-6} ε = 1 0 − 6 . This bounds each output channel by 1 / η 1/\eta 1/ η times the largest absolute input value contributing to that node. For non-negative weights, a g = d g a_g = d_g a g = d g , so only the absolute floor can be active. Uncovered nodes remain zero; constant features are reproduced wherever neither floor changes the denominator.
After the FNO produces grid features Z ∈ R B b a t c h × G × C Z \in\mathbb{R}^{B_{\mathrm{batch}}\times G\times C} Z ∈ R B batch × G × C , gather evaluates them at each query point q b , j q_{b,j} q b , j as a weighted sum of that point’s stencil nodes, using the analytic weights Φ \Phi Φ :
H ^ b , j = ∑ g Φ ( q b , j ) g Z b , g . (37) \widehat{H}_{b,j} = \sum_g \Phi(q_{b,j})_g Z_{b,g}. \tag*{(37)} H b , j = g ∑ Φ ( q b , j ) g Z b , g . (37)
Scatter and gather build separate stencils, so the source and query clouds may differ in both location and size. The output projection then maps H ^ \widehat{H} H to the physical output variables.
Listing 2 processes chunk points at a time. The method taps returns flattened grid indices and weights of shape [ B b a t c h , h i − l o , K ] [B_{\mathrm{batch}}, h_i-l_o, K] [ B batch , h i − l o , K ] , or batch dimension one for a shared mesh. The first grid axis varies slowest. Adding b G bG b G to sample b b b ’s indices lets indexed additions and gathers process the batch together. Gradients propagate to features, grid values and learned kernel coefficients.
def scatter(H, src, G, chunk, eta=0.1, eps=1e-6):
# H: [B, N, C] point features -> [B, G, C] grid features.
B, N, C = H.shape
offset = (torch.arange(B, device=H.device) * G)[:, None, None]
num = H.new_zeros(B * G, C) # n_g
mass = H.new_zeros(B * G, 2) # (d_g, a_g)
for lo in range(0, N, chunk):
hi = min(lo + chunk, N)
col, w = src.taps(node_start=lo, node_stop=hi)
w = w.expand(B, -1, -1) # a shared-mesh stencil has batch 1
idx = (col + offset).reshape(-1)
values = H[:, lo:hi, None, :] * w[..., None]
num.index_add_(0, idx, values.reshape(-1, C))
mass.index_add_(0, idx,
torch.stack((w, w.abs()), -1).reshape(-1, 2))
# Normalise only after every point has contributed.
den, absolute = mass.unbind(-1)
sign = torch.where(den < 0, -1., 1.)
safe = sign * torch.maximum(den.abs(), (eta * absolute).clamp_min(eps))
return (num / safe[:, None]).view(B, G, C)
def gather(Z, qry, chunk):
# Z: [B, G, C] grid features -> [B, N_q, C] features at the queries.
B, G, C = Z.shape
offset = (torch.arange(B, device=Z.device) * G)[:, None, None]
flat = Z.reshape(B * G, C)
out = Z.new_empty(B, qry.n_points, C)
for lo in range(0, qry.n_points, chunk):
hi = min(lo + chunk, qry.n_points)
col, w = qry.taps(node_start=lo, node_stop=hi)
idx = (col + offset).reshape(-1)
values = flat.index_select(0, idx).view(B, hi - lo, -1, C)
out[:, lo:hi] = (values * w[..., None]).sum(2)
return out
Listing 2: Scatter features from points to the grid with guarded normalisation, and gather grid features at query points. chunk is the number of points processed at once.
Step 3: the full model. The pointwise lift forms H i = P i n [ x i ; ω ˉ i ] H_i = P_{\mathrm{in}}[x_i;\bar{\omega}_i] H i = P in [ x i ; ω ˉ i ] , as in Equation (3) . For the FiLM-conditioned benchmarks, x i = γ μ ⊙ φ ( ω ˉ i ) + β μ x_i = \gamma_\mu\odot\varphi(\bar{\omega}_i) + \beta_\mu x i = γ μ ⊙ φ ( ω ˉ i ) + β μ contains 96 sinusoidal features (d = 3 d = 3 d = 3 , n f = 16 n_f = 16 n f = 16 ), modulated in the outer wrapper. The three normalised coordinates bypass modulation, so P i n : R 99 → R C P_{\mathrm{in}} : \mathbb{R}^{99} \to\mathbb{R}^{C} P in : R 99 → R C .
Normalisation removes the overall weight magnitude at each node. We retain coverage information through ρ g = ∑ i Φ ( ω ˉ i ) g \rho_g = \sum_i \Phi(\bar{\omega}_i)_g ρ g = ∑ i Φ ( ω ˉ i ) g , computed on the fine grid from the source coordinates. This non-negative feature depends on the sampled geometry but not on the learned kernels. The channel log ( 1 + ρ ) \log(1 + \rho) log ( 1 + ρ ) is appended to the fused features and mixed back to C C C channels by a 1 × 1 1 \times1 1 × 1 convolution P ρ P_\rho P ρ , initialised as the identity on the features and zero on the density, so that the density has no effect at initialisation. The full forward pass is
X s = D θ , s − 1 W θ , s ⊤ H , X f i n e = Fuse ( { Up s X s } s ∈ S ) , Z = F θ ( P ρ [ X f i n e ; log ( 1 + ρ ) ] ) , Y ^ ( q ) = P o u t ( Φ ( q ) Z ) , (38) \begin{aligned}
X_s &= D_{\theta,s}^{-1}W_{\theta,s}^{\top}H, \qquad&X_{\mathrm{fine}} &= \operatorname{Fuse}(\{ \operatorname{Up}_s X_s\}_{s\in S}),\\
Z &= F_\theta(P_\rho[X_{\mathrm{fine}};\log(1+\rho)]), \qquad&\widehat{Y}(q) &= P_{\mathrm{out}}(\Phi(q)Z), \tag*{(38)}
\end{aligned} X s Z = D θ , s − 1 W θ , s ⊤ H , = F θ ( P ρ [ X fine ; log ( 1 + ρ )]) , X fine Y ( q ) = Fuse ({ Up s X s } s ∈ S ) , = P out ( Φ ( q ) Z ) , (38)
where D θ , s D_{\theta,s} D θ , s contains the guarded denominators of Step 2, S S S is the set of scales, Up s \operatorname{Up}_s Up s interpolates scale s s s to the fine grid (Up 1 \operatorname{Up}_1 Up 1 is the identity) and Fuse is the 1 × 1 1 \times1 1 × 1 fusion; both are omitted with a single scale. When spectral modulation is selected, a condition embedding is also passed to every FNO layer. Region-specific variants may use separate lifts, output projections or kernel banks. With separate kernels, the three scatter sums are added across regions before division; constant reproduction has the same coverage and safeguard conditions as in Step 2. Region routing uses node labels supplied alongside the coordinates.
The two halves are exposed separately. encode runs the lift, scatter and FNO and returns the grid state Z Z Z , and decode gathers Z Z Z at a query cloud and applies P o u t P_{\mathrm{out}} P out . Evaluating at new query points therefore needs only a new decode in the path described here. Region-specific readouts additionally require query-region labels, and the optional node skip connection requires lifted features at the query nodes.
Cost and memory. At fixed width and stencil size, both transfers are linear in the number of points: writing B = B b a t c h B = B_{\mathrm{batch}} B = B batch , scatter costs O ( B N K C ) O(BNKC) O ( B N K C ) and gather O ( B N q K C ) O(BN_qKC) O ( B N q K C ) , plus O ( B G C ) O(BGC) O ( B GC ) to initialise the grid buffers. The encoder needs no G × G G \times G G × G Gram matrix and no factorisation. Each FNO layer costs O ( B C G log G ) O(BCG\log G) O ( B C G log G ) for its real FFTs, and dense spectral channel mixing costs O ( B C 2 M f ) O(BC^2M_f) O ( B C 2 M f ) for M f M_f M f retained modes, with a further O ( B C 2 G ) O(BC^2G) O ( B C 2 G ) for pointwise channel mixing. The main transfer optimisations are:
Recomputed backward passes. Custom autograd functions save the per-axis stencil factors and the source features or grid state. They reconstruct tensor-product indices and weights in chunks during backward, avoiding retention of every chunk’s expanded intermediates as ordinary autograd would do for the reference listings.
Shared geometry. When every sample in a batch has the same coordinates and these do not require gradients, one stencil is built and broadcast across the batch, and the weight sums d g d_g d g and a g a_g a g , which depend only on the geometry, are accumulated once. Samples with different meshes keep their own stencils.
The parameter scatter_max_temp_mb (default 128 MiB) sets an approximate temporary-buffer budget for both transfers on every device. It controls chunk size, not total model or training memory. Gather reduces each query’s K K K contributions within a single chunk; scatter reductions may differ in floating-point rounding with chunk size or device. For source clouds too large to lift in one pass, streamed single-sample inference reads and lifts the inputs in chunks, accumulates the scatter and density sums over all chunks before normalisation, and runs the FNO once. This matches the complete-cloud computation up to floating-point rounding.
Physics and Data
All six benchmarks are steady-state, conditioned operator-learning problems posed on unstructured meshes. They were chosen to span two axes that matter for a conditional neural operator. The first is geometric : shared-mesh problems, where a single node ordering is valid for every case (Aircraft, HIVE, Heated duct), against per-sample-mesh problems, where node index i i i refers to unrelated physical locations in different cases (Heat sink, Bracket, Building fire). The second is the informativeness of μ \mu μ : how much of what μ \mu μ says is already recoverable from the per-node input.
Partly non-redundant. Bracket, Building fire — on the bracket the three geometry parameters restate the mesh, but the applied pressure load is invisible in it, so two designs can share a shape and differ only in μ \mu μ . The fire splits the same way: the refinement box that follows the burner places the two fire coordinates in the point cloud exactly, while the five ventilation parameters move no node at all. These are the only cases here where μ \mu μ splits into an inferable half and a hidden one, and both loaders expose each half separately so the two can be ablated apart.
Non-redundant. Aircraft, HIVE, Heated duct — the five flight conditions of one airframe have identical coordinates, so μ \mu μ is the only signal separating Mach 2 from Mach 7. HIVE and the heated duct are the same in kind: one shared mesh across every run, so nothing in the geometry distinguishes them.
A third property is carried by two cases. The heated duct is multi-region : its domain is three separate meshes that share no cells and couple only across two conjugate interfaces, so temperature is continuous there while its gradient is not. All other benchmarks here are single structures, and the gridded operators reduce to a single latent grid without loss; on the heated duct and building fire the architecture deploys three different branches for mesh encoding and decoding with a shared latent backbone.
Conventions common to all datasets
Each data loader returns a single tensor F ∈ R N × ( c i n + c o u t ) × N \mathbf{F} \in\mathbb{R}^{N\times(c_{\mathrm{in}}+c_{\mathrm{out}})\times N} F ∈ R N × ( c in + c out ) × N together with μ ∈ R N × k \mu\in\mathbb{R}^{N\times k} μ ∈ R N × k and Δ t = 0 \Delta t = 0 Δ t = 0 , the latter flagging the steady-state path. The first c i n c_{\mathrm{in}} c in channels of F \mathbf{F} F are inputs and the remainder are targets.
Where the node count varies case to case (Heat sink, Bracket), every case is drawn down to one fixed node budget N N N so the cases stack into a dense tensor. The budget is a free parameter of the sweep rather than a property of the corpus; we set it to the power of two nearest the median mesh, so a majority of cases are drawn without replacement and the remainder are duplicate-padded with their own real nodes. Padding with repeats rather than with zeros or a sentinel matters for the gridded backbones: a repeated node adds weight to a knot that a real node already occupies, which the scatter and its density normalisation absorb exactly, whereas an artificial coordinate would invent geometry. The cost is a reduced effective sample — at the budgets used here, 432 of the 625 heat-sink cases and 878 of the 3,000 bracket designs are padded, leaving on average 76% and 97% of their slots occupied by distinct nodes (Table 8 ). The per-sample geometry path then reads each case’s own coordinates out of its input channels rather than broadcasting a cross-case mean mesh, which for independently drawn meshes would collapse towards the domain centroid. The three shared-mesh cases need none of this: Aircraft draws all 150 cases at the same node indices, and HIVE and the heated duct use every node of their one mesh.
Table 8. Overview of the six benchmarks. “Native nodes” denotes the original mesh size, whereas “used nodes” is the spatial budget presented to the model. Variable-size meshes are sampled or duplicate-padded to this budget.
All six benchmarks are geometry-only : no case carries a field input channel, so the per-node input is the processed coordinates and nothing else. Where node positions differ between cases, the three leading channels of F \mathbf{F} F hold that case’s own ( x , y , z ) (x,y,z) ( x , y , z ) and c i n = 3 c_{\mathrm{in}}=3 c in = 3 . This happens in two ways: either each case has its own mesh (Heat sink, Bracket, Building fire), or every case is a deformation of one shared template, so node k k k is the same template node in every case, but its position changes with the airframe (Aircraft). The per-sample geometry path consumes these channels as geometry rather than as features, so they are not passed to the backbone as channels. Where the mesh is genuinely fixed (HIVE, Heated duct), the coordinates live in the mesh metadata instead and c i n = 0 c_{\mathrm{in}}=0 c in = 0 . Either way, the coordinates reach the backbone only through the coordinate processor, where an integrated sinusoidal positional encoding at 16 frequencies gives them bandwidth, mapping R 3 → R 3 + 2 ⋅ 3 ⋅ 16 = R 99 \mathbb{R}^{3}\to\mathbb{R}^{3+2\cdot3\cdot16}=\mathbb{R}^{99} R 3 → R 3 + 2 ⋅ 3 ⋅ 16 = R 99 . The network therefore consumes X ∈ R B b a t c h × N × 99 \mathcal{X}\in\mathbb{R}^{B_{\mathrm{batch}}\times N\times99} X ∈ R B batch × N × 99 and emits R B b a t c h × N × c o u t \mathbb{R}^{B_{\mathrm{batch}}\times N\times c_{\mathrm{out}}} R B batch × N × c out throughout. The first 3 of these 99 channels are the normalised coordinates and the other 96 are the sinusoidal features φ ( ω ⃗ ) \varphi(\vec{\omega}) φ ( ω ) . Under the field-space conditioning methods (FiLM, AdaLN-Zero, cross-attention), μ \boldsymbol{\mu} μ modulates only the 96 sinusoidal channels, and the 3 coordinates pass through unmodulated (Equation (3) ).
Field normalisation is per-variable and standardised throughout; conditioning parameters are min–max normalised per variable. Train/test splits are random permutations under a fixed seed of 42, except on the heated duct, whose designed split is honoured instead (Section D.6). The field normaliser, the parameter normaliser and the coordinate processor are all fitted on the pooled corpus before the train–test split. The Building fire departs from these conventions in one respect: a cell-size-aware positional encoding, described in Section D.5.
Reported errors are the percentage relative L 2 L_2 L 2 error, computed per sample in physical (denormalised) units and averaged over the test set,
E = 100 M t e s t ∑ i = 1 M t e s t ∥ y ^ ( i ) − y ( i ) ∥ 2 ∥ y ( i ) ∥ 2 , (39) \mathcal{E} = \frac{100}{M_{\mathrm{test}}} \sum_{i=1}^{M_{\mathrm{test}}} \frac{\lVert\hat{\mathbf{y}}^{(i)}-\mathbf{y}^{(i)}\rVert_2}{\lVert\mathbf{y}^{(i)}\rVert_2}, \tag*{(39)} E = M test 100 i = 1 ∑ M test ∥ y ( i ) ∥ 2 ∥ y ^ ( i ) − y ( i ) ∥ 2 , (39)
which equals 100 × 100 \times 100 × NRMSE under the definition used in § 4 . Because Equation (39) normalises by the magnitude of the target rather than by its variation, it is generous to fields carrying a large constant offset; all three thermal cases sit on a pedestal — ∼ 300 \sim300 ∼ 300 K for the heat sink and HIVE, 280 K for the heated duct — and the per-dataset notes below give the scale of that effect.
Heat sink
Governing equation. Steady conduction through a finned aluminium heat sink cooling a CPU,
− ∇ ⋅ ( κ ∇ T ) = Q , − κ ∂ T ∂ n = h ( T − T ∞ ) on Γ l a t ∪ Γ t o p , (40) -\nabla\cdot(\kappa\nabla T) = Q,\qquad-\kappa\frac{\partial T}{\partial n} = h(T-T_{\infty})\ \text{on }\Gamma_{\mathrm{lat}}\cup\Gamma_{\mathrm{top}}, \tag*{(40)} − ∇ ⋅ ( κ ∇ T ) = Q , − κ ∂ n ∂ T = h ( T − T ∞ ) on Γ lat ∪ Γ top , (40)
with forced convection on the lateral surfaces, free convection on the top surface and the chip heat load applied on the baseplate underside. The material conductivity κ \kappa κ , the load Q Q Q , the film coefficient h h h and the ambient T ∞ T_{\infty} T ∞ are held fixed across the whole dataset. Only the geometry varies, so μ \boldsymbol{\mu} μ fully determines the solution and the benchmark is a direct test of whether a conditional operator can read a 3D shape.
Parameter scan. A complete 5 4 = 625 5^4 = 625 5 4 = 625 factorial over four design variables at five levels each (Table 9 ). The baseplate is a fixed 26 mm along x x x (widened only when W > 26 W > 26 W > 26 mm), so the fin pitch is 26 m m / n f i n ∈ [ 3.25 , 6.5 ] 26\ \mathrm{mm}/n_{\mathrm{fin}} \in[3.25, 6.5] 26 mm / n fin ∈ [ 3.25 , 6.5 ] mm and is not an independent variable. This is the source of the benchmark’s difficulty for grid-based methods: fins 1.5–2.5 mm thick at a 3.25–6.5 mm pitch alias badly on any uniform latent grid, and at least 16 cells along x x x are needed for eight fins to separate at all.
Table 9. Heat sink: the 5 4 5^4 5 4 factorial design. All 625 combinations are present exactly once.
Mesh and provenance. The dataset is the heat-sink benchmark of Zhong et al. (2026), distributed as a MATLAB struct of nodal coordinates and temperatures with no connectivity. Every case carries its own tetrahedral mesh, from 4,671 to 32,645 nodes (median 12,673, with 613 distinct node counts among 625 cases), so node index i i i is an unrelated physical location in different cases. At the node budget used here, N = 16384 N = 16384 N = 16384 , the 193 cases with larger meshes are drawn without replacement and the remaining 432 are duplicate-padded; the padding factor reaches 3.5× on the smallest mesh, and averaged over the corpus 76% of the budget is distinct nodes. The pooled bounding box is x ∈ [ 0 , 30 ] x \in[0,30] x ∈ [ 0 , 30 ] , y ∈ [ − 30 , 0 ] y \in[-30,0] y ∈ [ − 30 , 0 ] , z ∈ [ − 2.5 , 18.5 ] z \in[-2.5,18.5] z ∈ [ − 2.5 , 18.5 ] mm; each design fills its own sub-box of that domain, and that occupancy is the primary signal, which is why coordinates are min–max scaled per axis over the pooled cloud rather than per sample.
Learning setup. F ∈ R 625 × 4 × 16384 \mathbf{F} \in\mathbb{R}^{625 \times4 \times16384} F ∈ R 625 × 4 × 16384 with channels ( x , y , z , T ) (x,y,z,T) ( x , y , z , T ) and c i n = 3 c_{\mathrm{in}}=3 c in = 3 , so inputs are R N × 3 × 16384 \mathbb{R}^{N \times3 \times16384} R N × 3 × 16384 and targets R N × 1 × 16384 \mathbb{R}^{N \times1 \times16384} R N × 1 × 16384 ; μ ∈ R 625 × 4 \boldsymbol{\mu} \in\mathbb{R}^{625 \times4} μ ∈ R 625 × 4 . Coordinates are dropped as features and re-enter only through the 99-channel integrated encoding, so the network sees geometry and μ \boldsymbol{\mu} μ and nothing else. The split is 70/30 (438 train, 187 test) at batch size 8, matching the test share used by the source paper (Zhong et al. (2026)).
Data statistics. T ∈ [ 296.46 , 343.76 ] T \in[296.46,343.76] T ∈ [ 296.46 , 343.76 ] K with mean 307.25 K and standard deviation 5.79 K (Skew = 1.25, Kurt = 5.60). Case-mean temperatures span [ 300.07 , 322.86 ] [300.07,322.86] [ 300.07 , 322.86 ] K. The measurements are taken on the temperature rise above ambient, T − T min T-T_{\min} T − T m i n with T min = 296.46 T_{\min}=296.46 T m i n = 296.46 K.
!Heat sink temperature error maps for Ground Truth, Transolver, UPT, MSPT, GNOT, NUNO, SplineFNO, and L-SplineFNO on held-out test case 95; Ground Truth: T T T [K] from 306 to 318; predicted maps: ∣ e r r o r ∣ |error| ∣ er r or ∣ [K] from 0 to 0.434; errors: Transolver 0.04%, UPT 0.05%, MSPT 0.04%, GNOT 0.05%, NUNO 0.06%, SplineFNO 0.04%, L-SplineFNO 0.03% bbox=0.20,0.13,0.80,0.41
Figure 10. Error maps produced by various architectures in modelling heat distribution across the heat sink.
Bracket
Governing equations. Steady elasto-plastic deformation of a lug, i.e. a hollow beam with a bolt hole which is fixed at one end and loaded by a pressure over a marked area:
∇ ⋅ σ = 0 , σ = C ( ε − ε p ) , ε = 1 2 ( ∇ u + ∇ u T ) , (41) \nabla\cdot\sigma= 0,\qquad\sigma= \mathsf{C}\left(\varepsilon-\varepsilon^{\mathrm{p}}\right),\qquad\varepsilon= \frac{1}{2}\left(\nabla u+\nabla u^{\mathsf{T}}\right), \tag*{(41)} ∇ ⋅ σ = 0 , σ = C ( ε − ε p ) , ε = 2 1 ( ∇ u + ∇ u T ) , (41)
with an elastic–plastic constitutive law under isotropic hardening and the small-deformation assumption, solved with quadratic hexahedral elements. The prediction target is the nodal stress field. This is the only non-thermal, non-fluid problem in the set.
Why this dataset earns its place. μ \mu μ splits cleanly into a redundant half and a non-redundant one: the three geometry parameters restate what the node coordinates already show, while the applied pressure is invisible in the mesh — two designs can share a geometry and differ only in load. It is the only benchmark here where half of μ \mu μ is inferable from the input and half is not, and the loader exposes each half separately so the two contributions can be ablated independently.
Parameter scan and normalisation. N = 3000 N = 3000 N = 3000 designs over four continuous parameters: length, thickness and hole radius (geometry) plus the applied pressure (loading), each realised at 3000 distinct values. The published release (Zhong et al., 2025) is already min–max normalised by its authors: coordinates, design parameters and stress all arrive on [ 0 , 1 ] [0,1] [ 0 , 1 ] , and the scaling factors for coordinates and parameters are not distributed, so physical units cannot be recovered for those channels. The normalisation is global rather than per sample, so each design occupies its own sub-box of the unit cube — mean per-design spans and relative scale between designs is preserved. The latent grids used for this dataset are consequently isotropic: the pooled cloud fills the unit cube exactly, and the true physical aspect ratio is unknowable.
Mesh and learning setup. Every design has its own hexahedral mesh, 5,054 to 15,210 nodes (median 9,405), with no node correspondence between designs. At the node budget used here, N = 8192 N = 8192 N = 8192 , the 2,122 designs with larger meshes are drawn without replacement and the remaining 878 are duplicate-padded, at worst 1.6 × 1.6\times 1.6 × ; averaged over the corpus 97% of the budget is distinct nodes, so padding is a marginal effect on this dataset in a way it is not on the heat sink. F ∈ R 3000 × 4 × 8192 \mathbf{F}\in\mathbb{R}^{3000\times4\times8192} F ∈ R 3000 × 4 × 8192 with channels ( x , y , z , σ ^ ) (x,y,z,\hat{\sigma}) ( x , y , z , σ ^ ) and c i n = 3 c_{\mathrm{in}}=3 c in = 3 ;
μ ∈ R 3000 × 4 \boldsymbol{\mu} \in\mathbb{R}^{3000 \times4} μ ∈ R 3000 × 4 . Split 70/30 (2100 train, 900 test) at batch size 8. The random split is sound here: 3000 independent designs with no grouping structure to leak across, unlike the aircraft case.
Data statistics. σ ^ \hat{\sigma} σ ^ fills [ 0 , 1 ] [0,1] [ 0 , 1 ] by construction of the authors’ normalisation, with mean 0.197 and standard deviation 0.121 (Skew = 1.19, Kurt = 3.92); the 99th percentile is 0.534, so the upper half of the normalised range is populated only by stress concentrations at the hole and the fixed end. Case-mean stresses span [ 0.086 , 0.383 ] [0.086,0.383] [ 0.086 , 0.383 ] . The metric is not degenerate here, so unlike the two temperature datasets, the stress field carries spatial structure that no constant can fake. See Figure 9 for the distribution.
Figure 9. Distribution of the target field over all nodes of all cases, one panel per benchmark (pressure shown for aircraft and temperature change for building fire). Counts are normalised to a fraction of nodes and plotted on a log ordinate; the horizontal axis is trimmed to [ p 0.05 , p 99.95 ] [p_{0.05},p_{99.95}] [ p 0.05 , p 99.95 ] and the lighter shading marks the tails outside [ p 1 , p 99 ] [p_{1},p_{99}] [ p 1 , p 99 ] . The dashed line is the median. Sample skewness (γ 1 \gamma_{1} γ 1 ) and kurtosis (β 2 \beta_{2} β 2 ) are annotated (a Gaussian has Skew = 0, Kurt = 3).
Aircraft
Governing equations and flight envelope. Steady compressible external flow over a parameterised airframe, with the solution sampled on the surface triangulation. The released fields are the freestream-normalised density, the three velocity components and the pressure. The envelope is supersonic to hypersonic, with shocks present on the surface: 30 airframes are each solved at five flight conditions,
( M a , α , β ) ∈ { ( 2 , 0 , 0 ) , ( 7 , 0 , 0 ) , ( 7 , 0 , 2 ) , ( 7 , 7 , 0 ) , ( 7 , 7 , 2 ) } , (\mathrm{Ma}, \alpha, \beta) \in\{(2,0,0),\ (7,0,0),\ (7,0,2),\ (7,7,0),\ (7,7,2)\}, ( Ma , α , β ) ∈ {( 2 , 0 , 0 ) , ( 7 , 0 , 0 ) , ( 7 , 0 , 2 ) , ( 7 , 7 , 0 ) , ( 7 , 7 , 2 )} ,
with α \alpha α the angle of attack and β \beta β the sideslip angle, both in degrees, giving N = 150 N = 150 N = 150 cases.
Why this dataset earns its place. All five flight conditions of one airframe share bit-identical coordinates, so the point cloud carries no information about the flight condition: only μ \boldsymbol{\mu} μ distinguishes them. This is the opposite extreme from the heat sink, where μ \boldsymbol{\mu} μ merely restates the shape. Unlike HIVE and the heated duct, where the mesh is fixed outright, the geometry here still varies across airframes as we subsample them, so the model must use both: the shape to tell airframes apart, and μ \boldsymbol{\mu} μ to tell flight conditions apart.
Mesh and provenance. The data ships with Transolver++ (Luo et al. (2025)) as Tecplot ASCII FEPOINT blocks and is used as one of the six industrial benchmarks of Zhong et al. (2026). Every case has the same 331,971 surface nodes and 663,938 triangles; the airframes are deformations of a shared template (mean node displacement between the two extreme geometries is 1.3% of the bounding-box diagonal, maximum 13%), so node index i i i is the same semantic vertex everywhere. We subsample all 150 cases at the same 131,072 node indices — 39% of the surface. The pooled bounding box over the sampled nodes is x ∈ [ 0 , 5000 ] x \in[0,5000] x ∈ [ 0 , 5000 ] , y ∈ [ − 298 , 774 ] y \in[-298,774] y ∈ [ − 298 , 774 ] , z ∈ [ − 2320 , 2320 ] z \in[-2320,2320] z ∈ [ − 2320 , 2320 ] : a thin sheet in y y y , which is why coordinates are min–max scaled per axis; a global scaling would leave the y y y axis of a latent grid nearly empty. Per-case force coefficients ( C A , C N , C Z , C l , C n , C m ) (C_A,C_N,C_Z,C_l,C_n,C_m) ( C A , C N , C Z , C l , C n , C m ) are carried through, with the lift coefficient C L = C N cos α − C A sin α C_L=C_N\cos\alpha-C_A\sin\alpha C L = C N cos α − C A sin α precomputed, so the aggregate quantity of interest needs no surface integration.
Learning setup. F ∈ R 150 × 8 × 131072 \mathbf{F} \in\mathbb{R}^{150\times8\times131072} F ∈ R 150 × 8 × 131072 with channels ( x , y , z , ρ , u , v , w , p ) (x,y,z,\rho,u,v,w,p) ( x , y , z , ρ , u , v , w , p ) and c i n = 3 c_{\mathrm{in}}=3 c in = 3 , giving inputs R N × 3 × 131072 \mathbb{R}^{N\times3\times131072} R N × 3 × 131072 and targets R N × 5 × 131072 \mathbb{R}^{N\times5\times131072} R N × 5 × 131072 ; μ ∈ R 150 × 3 \boldsymbol{\mu} \in\mathbb{R}^{150\times3} μ ∈ R 150 × 3 . Split 70/30 (105 train, 45 test) at batch size 4.
Split caveat. The split is a random permutation over the 150 cases, so the same airframe appears in train and test under different flight conditions. With only 30 airframes, almost every test geometry has been seen. The reported numbers therefore measure flight-condition interpolation on known shapes, not geometry generalisation.
Data statistics. Alongside the histogram in Figure 9 , per-channel statistics are given in Table 10 . The channels are strikingly unequal. The stream-wise velocity u u u is nearly determined by the flight condition alone whereas the transverse components v v v and w w w are largely geometry responses (67.8% and 61.3%). The pressure is the most heavy-tailed target in the whole benchmark set (Kurt = 108, reaching 46 p ∞ 46p_\infty 46 p ∞ at the shock), and it is the channel that drives the joint score.
Table 10. Aircraft: per-channel statistics over all 150 cases at the 131,072 sampled nodes. All quantities are freestream-normalised and dimensionless. The last column is the relative L 2 L_2 L 2 error of predicting the mean field of the matching flight condition, i.e. the score of a predictor that uses μ \boldsymbol{\mu} μ perfectly and ignores the geometry; a model must beat 23.3% on the joint field before it has learned anything from the airframe shape.
Building fire: smoke and heat transport in a building
Governing equations. A 500 kW methane fire in a ventilated three-storey building, computed with Fire Dynamics Simulator (FDS 6) (McGrattan et al., 2013) in large-eddy simulation mode. FDS solves the low-Mach-number form of the reacting Navier–Stokes equations for thermally driven flow,
∂ t ρ + ∇ ⋅ ( ρ u ) = 0 , ∂ t ( ρ Z α ) + ∇ ⋅ ( ρ Z α u ) = ∇ ⋅ ( ρ D α ∇ Z α ) + m ˙ α ′ ′ ′ , ∂ t ( ρ u ) + ∇ ⋅ ( ρ u u ) + ∇ p ~ = ( ρ − ρ 0 ) g + ∇ ⋅ τ , ∂ t ( ρ h s ) + ∇ ⋅ ( ρ h s u ) = D p ˉ D t + q ˙ ′ ′ ′ − ∇ ⋅ q ˙ ′ ′ , p ˉ = ρ R T ∑ α Z α W α . (42) \begin{aligned}
\partial_t \rho+\nabla\cdot(\rho\mathbf{u})&=0, &
\partial_t(\rho Z_\alpha)+\nabla\cdot(\rho Z_\alpha\mathbf{u})&=\nabla\cdot(\rho D_\alpha\nabla Z_\alpha)+\dot{m}^{\prime\prime\prime}_\alpha,\\
\partial_t(\rho\mathbf{u})+\nabla\cdot(\rho\mathbf{u}\mathbf{u})+\nabla\tilde{p}&=(\rho-\rho_0)\mathbf{g}+\nabla\cdot\boldsymbol{\tau},\\
\partial_t(\rho h_s)+\nabla\cdot(\rho h_s\mathbf{u})&=\frac{\mathrm{D}\bar{p}}{\mathrm{D}t}+\dot{q}^{\prime\prime\prime}-\nabla\cdot\dot{\mathbf{q}}^{\prime\prime}, &
\bar{p}&=\rho RT\sum_\alpha\frac{Z_\alpha}{W_\alpha}. \tag*{(42)}
\end{aligned} ∂ t ρ + ∇ ⋅ ( ρ u ) ∂ t ( ρ u ) + ∇ ⋅ ( ρ uu ) + ∇ p ~ ∂ t ( ρ h s ) + ∇ ⋅ ( ρ h s u ) = 0 , = ( ρ − ρ 0 ) g + ∇ ⋅ τ , = D t D p ˉ + q ˙ ′′′ − ∇ ⋅ q ˙ ′′ , ∂ t ( ρ Z α ) + ∇ ⋅ ( ρ Z α u ) p ˉ = ∇ ⋅ ( ρ D α ∇ Z α ) + m ˙ α ′′′ , = ρR T α ∑ W α Z α . (42)
The three lumped species are α ∈ { a i r , f u e l , p r o d u c t s } \alpha\in\{\mathrm{air},\mathrm{fuel},\mathrm{products}\} α ∈ { air , fuel , products } , with mass fractions Z α Z_\alpha Z α . Here, h s h_s h s is sensible enthalpy, q ˙ ′ ′ ′ \dot{q}^{\prime\prime\prime} q ˙ ′′′ is the volumetric heat-release rate, and q ˙ ′ ′ \dot{\mathbf{q}}^{\prime\prime} q ˙ ′′ contains conductive, diffusive and radiative heat fluxes. Combustion is a single-step, mixing-controlled reaction (soot yield 0.01, CO yield 0.005) and radiation is a grey-gas solve. FDS uses the sensible-enthalpy balance to constrain ∇ ⋅ u \nabla\cdot\mathbf{u} ∇ ⋅ u rather than advancing it explicitly, and recovers temperature from the equation of state.
Figure 12. Error maps produced by various architectures in modelling the pressure distribution across the aircraft.
The flow is turbulent and unsteady, so we predict a late-time average rather than an instantaneous field,
ϕ ˉ ( x ) = 1 120 s ∫ 480 s 600 s ϕ ( x , t ) d t , (43) \bar{\phi}(\mathbf{x}) = \frac{1}{120\,\mathrm{s}}\int_{480\,\mathrm{s}}^{600\,\mathrm{s}} \phi(\mathbf{x},t)\,\mathrm{d}t, \tag*{(43)} ϕ ˉ ( x ) = 120 s 1 ∫ 480 s 600 s ϕ ( x , t ) d t , (43)
taken over a window beginning 450 s after the burner reaches full output. The averaged field is treated as a static learning target (Δ t = 0 \Delta t = 0 Δ t = 0 ).
Target fields. We predict three cell-centred averages: the temperature rise above ambient, Δ T = T − 293.15 K \Delta T = T - 293.15\,\mathrm{K} Δ T = T − 293.15 K , and the partial densities ( ρ a i r , ρ p r o d ) (\rho_{\mathrm{air}}, \rho_{\mathrm{prod}}) ( ρ air , ρ prod ) of the lumped air and combustion-products species. These partial densities measure the mass of each mixture per unit volume; ρ p r o d \rho_{\mathrm{prod}} ρ prod tracks smoke transport through the building.
Scenario. The building measures 12 × 12 × 10 m 12 \times12 \times10\,\mathrm{m} 12 × 12 × 10 m and comprises three levels: a car park at 0–4 m, a middle level at 4.25–7 m, and an upper level at 7.25–10 m. Concrete slabs separate the levels, with four openings connecting the car park to the middle level and a 6 × 6 m 6 \times6\,\mathrm{m} 6 × 6 m atrium void connecting the middle and upper levels. The upper floor is partitioned into adjoining rooms surrounding the atrium void. A floor-mounted methane burner has a 0.5 × 0.5 m 0.5 \times0.5\,\mathrm{m} 0.5 × 0.5 m burning surface at z = 0.25 m z = 0.25\,\mathrm{m} z = 0.25 m above the car-park floor. Its prescribed heat-release rate increases linearly over the first 30 s to 2000 k W m − 2 2000\,\mathrm{kW}\,\mathrm{m}^{-2} 2000 kW m − 2 (500 k W 500\,\mathrm{kW} 500 kW ). Two 1.5 × 1.0 m 1.5 \times1.0\,\mathrm{m} 1.5 × 1.0 m wall vents on opposite faces provide push–pull ventilation: for Q > 0 Q > 0 Q > 0 , air enters through the vent at x = 0 x = 0 x = 0 and exits through the vent at x = 12 m x = 12\,\mathrm{m} x = 12 m ; Q < 0 Q < 0 Q < 0 reverses the direction. The enclosure is otherwise closed to gas flow. Ambient conditions are 20 ∘ C 20\,^{\circ}\mathrm{C} 20 ∘ C with no stratification. μ \boldsymbol{\mu} μ contains all prescribed case-to-case variation.
Parameter scan. Seven randomly sampled parameters control the burner position, the signed ventilation rate, and the position of each wall vent (Table 11 ). The six positional parameters are restricted to a 0.25 m lattice to maintain alignment with the computational mesh, whereas the signed ventilation rate is sampled continuously. The corpus contains 547 cases.
Table 11. Building fire: the seven conditioning parameters, their sampling ranges, and positional discretisation for the 547-case corpus. Left and right denote the vents at x = 0 x = 0 x = 0 and x = 12 x = 12 x = 12 m, respectively. The last column indicates whether each parameter is recoverable from the node coordinates; see the redundancy discussion below.
Mesh. FDS discretises the domain into 11 rectilinear blocks containing 116,928 Cartesian cells in three cell classes (Table 12 ). Of these, 111,237 are gas cells and 5,691 lie inside solid obstructions. Solid cells are excluded from the learning point clouds, targets, field statistics and error metrics. The counts and index ranges below describe the full Cartesian arrays, before either solid-cell masking or refinement-box removal. A 24 × 24 × 32 24 \times24 \times32 24 × 24 × 32 refinement block, centred horizontally on the burner and extending from z = 0 z = 0 z = 0 to 4 m, uses cubic cells of side 0.125 m. Outside this block, four vertically refined blocks with flattened 0.25 × 0.25 × 0.125 m 0.25 \times0.25 \times0.125\,\mathrm{m} 0.25 × 0.25 × 0.125 m cells form a layer at z = 3 z = 3 z = 3 –4 m, immediately below the intermediate floor. The remaining six blocks use cubic cells of side 0.25 m. We use cell centres as nodes; their pooled bounding box is x , y ∈ [ 0.125 , 11.875 ] x, y \in[0.125, 11.875] x , y ∈ [ 0.125 , 11.875 ] m and z ∈ [ 0.0625 , 9.875 ] z \in[0.0625, 9.875] z ∈ [ 0.0625 , 9.875 ] m. The refinement box follows the burner, resizing the surrounding car-park blocks and producing 415 distinct point clouds, all with the same node count. These cases therefore require per-sample coordinates, as with the heat sink and bracket cases. The middle and upper meshes are bit-identical across cases, and the concatenated ordering preserves fixed storey ranges: car park [ 0 , 61632 ) [0, 61632) [ 0 , 61632 ) , middle [ 61632 , 89280 ) [61632, 89280) [ 61632 , 89280 ) and upper [ 89280 , 116928 ) [89280, 116928) [ 89280 , 116928 ) .
Table 12. Building fire: the three cell classes of the 116,928-cell mesh (1440 m3 ^3 3 in total, including solid obstructions), before solid-cell masking and refinement-box removal. On this full Cartesian mesh, an unweighted node average over-weights the refinement box by ∼ 6.3 × \sim6.3\times ∼ 6.3 × relative to its volume, so integral quantities of interest over the gas domain must exclude solid cells and be volume-weighted; the loader carries the cell volume per node for this purpose.
Why this dataset earns its place. Its conditioning vector splits in a way no other benchmark here does. The refinement box is centred on the burner, so the centre of the finest cells reproduces ( x f , y f ) (x_{\mathrm{f}}, y_{\mathrm{f}}) ( x f , y f ) exactly. The five ventilation parameters, by contrast, move no cell centre at all. Two dimensions of μ \boldsymbol{\mu} μ are therefore recoverable from the geometry, and five are invisible in it. The unconditioned arm is consequently not a geometry-only model; it knows exactly where the fire is and nothing about the ventilation. As on the bracket, the loader exposes the fire and ventilation parameter groups of μ \boldsymbol{\mu} μ separately so that they can be ablated apart.
The case is also multi-region , in a different sense from the heated duct. The three storeys are separated by slabs that the flow crosses only through the openings, so the fields change near-discontinuously in value across a slab rather than kinking in gradient across a material interface — but the difficulty for a single C 2 C^2 C 2 basis on a single latent grid is the same. The storeys serve as the region set for the per-region encoding and decoding branches described for the heated duct. The 11 FDS blocks cannot be used for this, because four of their boundaries move with the fire and block membership is therefore a per-sample property.
Prediction domain: the smoke, not the flame. A fire-safety engineer does not run this simulation to learn what happens inside the flame. The design fire is prescribed: the burner’s position and output are inputs to the study, not results of it. The question is whether the rest of the building stays tenable, meaning how hot the gas gets, how far the smoke spreads and how much combustion product reaches the car park and the upper storeys, where occupants are and where the escape routes run. The error metric pulls the other way. The 18,400 gas nodes of the refinement box around the burner are 16.5% of the gas-cell cloud but carry roughly two thirds of the error energy, because they contain the flame itself (Δ T \Delta T Δ T up to 1002 K there, against 189 K everywhere else). With them left in, every joint metric on this case would score the flame, which is the one region the engineer does not need predicted.
We therefore remove the box’s gas nodes from the prediction domain, leaving 92,837 nodes: 43,200 in the car park and 49,637 in the middle and upper storeys. The task becomes predicting heat and combustion-product transport into the occupied building, away from the source. Removing the box does not hide where the fire is, because the surrounding mesh blocks still abut it and the retained cloud still moves with the burner. It does make that information far less accessible. The two fire-position dimensions of μ \mu μ can still be recovered from the geometry, but they are no longer cheap to extract ((Panayis et al., 2025)).
Learning setup. F ∈ R 547 × 6 × 98496 \mathbf{F} \in\mathbb{R}^{547 \times6 \times98496} F ∈ R 547 × 6 × 98496 with channels ( x , y , z , Δ T , ρ a i r , ρ p r o d ) (x, y, z, \Delta T, \rho_{\mathrm{air}}, \rho_{\mathrm{prod}}) ( x , y , z , Δ T , ρ air , ρ prod ) and c i n = 3 c_{\mathrm{in}} = 3 c in = 3 , giving inputs R N × 3 × 98496 \mathbb{R}^{N \times3 \times98496} R N × 3 × 98496 and targets R N × 3 × 98496 \mathbb{R}^{N \times3 \times98496} R N × 3 × 98496 ; μ ∈ R 547 × 7 \mu\in\mathbb{R}^{547 \times7} μ ∈ R 547 × 7 . The coordinates are consumed as geometry by the per-sample path, so the network again sees the 99-channel encoding and μ \mu μ only. One departure from the common convention: because this mesh has two cell sizes, the integrated positional encoding is given a per-node footprint, attenuating frequency band f f f by exp ( − 1 2 f 2 σ 2 ) \exp\left(-\frac{1}{2}f^{2}\sigma^{2}\right) exp ( − 2 1 f 2 σ 2 ) with σ = h / 12 \sigma= h/\sqrt{12} σ = h / 12 the standard deviation of a uniform distribution over the node’s cell of side h h h (in normalised units); on every other benchmark σ = 0 \sigma= 0 σ = 0 and the encoding reduces to the plain sinusoidal one. The storey index is carried alongside the mesh for the region branches and is not an input. The split is a random 70/30 permutation (383 train, 164 test) at batch size 4. It is sound here, unlike on the aircraft: the cases were submitted in batches, but the within-batch parameter spread matches the corpus spread on all seven dimensions, so the batches are not design clusters.
Data statistics. Per-channel statistics over the retained gas nodes are given in Table 13 . The three targets sit at very different scales, so a joint relative L 2 L_2 L 2 over the raw channels (Equation (39) ) comes down to the error in Δ T \Delta T Δ T alone. We therefore report the per-variable normalised error. Unlike the other thermal cases, the temperature target is a rise over ambient and carries no pedestal. The cases differ substantially from one another: the case-mean temperature rise and the case-mean product density each vary by roughly a factor of two across the corpus. Air density varies least, only a few percent about its mean, and that small spread makes it the hardest of the three targets to predict in normalised terms.
Table 13. Building fire: per-channel statistics over all 547 cases at the 92,837 retained gas nodes (solid cells and refinement box removed).
Figure 13. Error maps produced by various architectures in modelling temperature change across the building due to a fire in the lower floor.
Heated duct: conjugate heat transfer
Governing equations. Steady conjugate heat transfer over three materials — a laminar water duct, the metal shell around it, and a volumetrically heated block beneath — coupled only through two interfaces:
− ∇ ⋅ ( k f ∇ T ) + ρ c p U ⋅ ∇ T = 0 (44) -\nabla\cdot(k_f \nabla T) + \rho c_p \mathbf{U} \cdot\nabla T = 0 \tag*{(44)} − ∇ ⋅ ( k f ∇ T ) + ρ c p U ⋅ ∇ T = 0 (44)
− ∇ ⋅ ( k m ∇ T ) = 0 (45) -\nabla\cdot(k_m \nabla T) = 0 \tag*{(45)} − ∇ ⋅ ( k m ∇ T ) = 0 (45)
− ∇ ⋅ ( k h ∇ T ) = q h (46) -\nabla\cdot(k_h \nabla T) = q_h \tag*{(46)} − ∇ ⋅ ( k h ∇ T ) = q h (46)
subject at each interface to continuity of temperature and of normal heat flux,
T a = T b , k a ∂ T a ∂ n = k b ∂ T b ∂ n . (47) T_a = T_b,\qquad k_a \frac{\partial T_a}{\partial n} = k_b \frac{\partial T_b}{\partial n}. \tag*{(47)} T a = T b , k a ∂ n ∂ T a = k b ∂ n ∂ T b . (47)
The flow is laminar (Re = 21 \operatorname{Re} = 21 Re = 21 –83 83 83 ) and, with constant properties and gravity switched off, the momentum equation does not depend on temperature — a fact with sharp consequences for the parameter scan, taken up below. Data were generated with the OpenFOAM Foundation v13 multiRegion/CHT/heatedDuct tutorial geometry (Weller et al., 1998), run to steady state with foamMultiRun.
Why this dataset earns its place. It is the only multi-material case in the set. The three regions are three separate meshes that share no cells: 40,560 fluid, 67,440 metal and 16,800 heater cell centres, which the loader concatenates into one 124,800-node cloud so a single operator spans all of them. They occupy disjoint sub-volumes of one bounding box — the heater sits below the duct at y < 0 y < 0 y < 0 , the fluid inside it, the metal in the shell between — so the concatenated cloud is a partition of the domain rather than an overlay, and region membership is a function of position. That is why the region label can be withheld: the default configuration feeds coordinates alone and the model must read the material off the geometry, while a one-hot variant hands it over explicitly and measures what naming the material is worth.
The interfaces are conformal to machine precision — 15,360 matched metal–fluid face pairs and 3,360 heater–metal pairs, with a maximum pair separation of 8 × 10 − 17 m 8 \times10^{-17}\ \mathrm{m} 8 × 1 0 − 17 m — so Equation (47) is imposed on coincident points and the kink in ∇ T \nabla T ∇ T is a property of the solution rather than of the discretisation.
Geometry. The pooled bounding box is x ∈ [ 0.5 , 49.5 ] x \in[0.5,49.5] x ∈ [ 0.5 , 49.5 ] , y ∈ [ − 9.5 , 49.5 ] y \in[-9.5,49.5] y ∈ [ − 9.5 , 49.5 ] , z ∈ [ 0.5 , 149.5 ] z \in[0.5,149.5] z ∈ [ 0.5 , 149.5 ] mm. The structure is cross-sectional and the geometry is prismatic in z z z : the fluid occupies x , y ∈ [ 5.6 , 44.4 ] x,y \in[5.6,44.4] x , y ∈ [ 5.6 , 44.4 ] mm, leaving a 5.0 mm metal shell, and the heater spans x ∈ [ 8.0 , 42.0 ] x \in[8.0,42.0] x ∈ [ 8.0 , 42.0 ] , y ∈ [ − 9.5 , − 0.6 ] y \in[-9.5,-0.6] y ∈ [ − 9.5 , − 0.6 ] mm. Under the per-axis min–max scaling this puts the fluid–metal wall at 0.103 and 0.897 along x x x , and at 0.255 and 0.915 along y y y , so a latent grid must resolve roughly a tenth of its own extent to see the wall at all. This is why the latent grids for this case are sized to feature scale in x x x and y y y rather than to the 49 : 59 : 149 bounding-box aspect, which would put the resolution along the one axis that carries no structure.
Parameter scan. Five parameters, drawn on a Sobol sequence and listed in Table 14 . Of 544 proposed designs, 146 converged and were accepted; the rest were rejected for an estimated peak temperature above 900 K or for failing the convergence gate. Acceptance is tight: heater power spans 131–1050 W, matched by the advected power to an energy imbalance of at most 9.6 × 10 − 4 9.6 \times10^{-4} 9.6 × 1 0 − 4 , with a maximum solver residual of 2.3 × 10 − 5 2.3 \times10^{-5} 2.3 × 1 0 − 5 .
Table 14. Heated duct: the five conditioning parameters. μ \mu μ is the dimensionless transform, not the raw value — only the log on q h q_h q h , which spans a factor of eight, is non-trivial. The last column records whether the parameter changes the shape of the temperature field or only rescales it; see the degeneracy noted below.
A degeneracy in two of the five dimensions. Because the momentum equation decouples from temperature, the coupled energy problem is linear in T T T and admits the exact separation
T ( x ; q h , U i n , T i n , k h , k m ) = T i n + q h Θ ( x ; U i n , k h , k m ) . (48) T(\mathbf{x};q_h,U_{\mathrm{in}},T_{\mathrm{in}},k_h,k_m) = T_{\mathrm{in}} + q_h \Theta(\mathbf{x};U_{\mathrm{in}},k_h,k_m). \tag*{(48)} T ( x ; q h , U in , T in , k h , k m ) = T in + q h Θ ( x ; U in , k h , k m ) . (48)
Two of the five parameters therefore carry no shape information at all: they are an exact affine rescaling of the field, and only U i n U_{\mathrm{in}} U in , k h k_h k h and k m k_m k m change Θ \Theta Θ . This has to be read into any conditioning ablation on this case, because a method that captures the q h / T i n q_h/T_{\mathrm{in}} q h / T in affinity exactly and gets Θ \Theta Θ wrong will still post a respectable relative L 2 L_2 L 2 .
The interface is where the gridded family is stressed. Temperature is continuous across a conjugate interface but its gradient is not, and the jump is set by the conductivity ratio in Equation (47) —here k h ∈ [ 40 , 120 ] k_h \in[40,120] k h ∈ [ 40 , 120 ] against k m ∈ [ 100 , 300 ] k_m \in[100,300] k m ∈ [ 100 , 300 ] W m− 1 ^{-1} − 1 K− 1 ^{-1} − 1 . Every gridded backbone in the comparison scatters all three regions onto one latent grid through one C 2 C^2 C 2 basis, and no number of retained modes lets a C 2 C^2 C 2 function reproduce a kink. The effect is measurable rather than notional: on the [ 48 , 48 , 32 ] [48,48,32] [ 48 , 48 , 32 ] latent grid, 24.0% of covered knots receive mass from more than one region, and on a coarser [ 32 , 32 , 24 ] [32,32,24] [ 32 , 32 , 24 ] grid that rises to 37.8%. Each such knot holds a single number serving two materials, at exactly the location where the kink should be. We treat this as a property of the benchmark to be measured, not a defect to be hidden.
Split. The corpus ships a designed split — 85 train, 18 validation, 19 interpolation test and 24 extrapolation corners — and the loader honours it by permuting the samples so the pipeline’s own seeded split lands exactly on the design. The default mode trains on train plus validation (103 cases) and tests on the 19 interpolation cases, dropping the corners; that is the N = 122 N = 122 N = 122 recorded in Table 8 . An extrapolation mode instead trains on all 122 and tests on the 24 corners, answering the harder question of performance outside the sampled parameter box. A plain random split over all 146 is available but leaks the corner design into training and is not used.
Learning setup. F ∈ R 122 × 1 × 124800 \mathbf{F} \in\mathbb{R}^{122 \times1 \times124800} F ∈ R 122 × 1 × 124800 carrying T T T alone, with c i n = 0 c_{\mathrm{in}} = 0 c in = 0 : there are no field channels whatsoever, so inputs are R N × 0 × 124800 \mathbb{R}^{N \times0 \times124800} R N × 0 × 124800 and the network sees only the 99-channel encoded coordinates, R B b a t c h × 124800 × 99 \mathbb{R}^{B_{\mathrm{batch}} \times124800 \times99} R B batch × 124800 × 99 , with targets R N × 1 × 124800 \mathbb{R}^{N \times1 \times124800} R N × 1 × 124800 . μ ∈ R 122 × 5 \mu\in\mathbb{R}^{122 \times5} μ ∈ R 122 × 5 ; batch size 4. The per-node region index is carried alongside the mesh for masked losses and per-region metrics, but is not an input under the default configuration.
Data statistics. T ∈ [ 280.00 , 880.63 ] T \in[280.00, 880.63] T ∈ [ 280.00 , 880.63 ] K with mean 461.0 K and standard deviation 118.1 K. With Skew = 0.87 and Kurt = 3.01 this is much the most nearly Gaussian target in the set (Figure 9 ), which is a direct consequence of Equation (48) : the field is a smooth Θ \Theta Θ scaled and shifted by two broadly sampled parameters, with none of the shock or stress-concentration tails that make the aircraft and bracket targets heavy. Case-mean temperatures span [ 319.5 , 732.2 ] [319.5, 732.2] [ 319.5 , 732.2 ] K, a factor of 2.3, so the case-to-case variation is far larger than on any other benchmark here.
Figure 14. Error maps produced by various architectures in modelling the temperature distribution from the conjugate heat transfer across the heated duct.
HIVE monoblock
Physics. HIVE (Heating by Induction to Verify Extremes) is a UKAEA facility for testing fusion divertor components under reactor-relevant heat loads (Hancock, 2018). The simulation is a coupled multiphysics MOOSE model (Permann et al., 2020; Nuno Nobre, Josh Williams and Karthikeyan Chockalingam, 2024): an induction coil drives eddy currents in a divertor monoblock, and the resulting Joule heating is the source term of a conduction problem (Horne-Jones et al., 2026),
ρ c p ∂ T ∂ t = ∇ ⋅ ( κ ∇ T ) + q E M ( x ) , − κ ∂ T ∂ n = h ( T − T c o o l ) on Γ p i p e , (49) \rho c_p \frac{\partial T}{\partial t} = \nabla\cdot(\kappa\nabla T) + q_{\mathrm{EM}}(\mathbf{x}), \qquad-\kappa\frac{\partial T}{\partial n} = h\left(T-T_{\mathrm{cool}}\right)\ \text{on }\Gamma_{\mathrm{pipe}}, \tag*{(49)} ρ c p ∂ t ∂ T = ∇ ⋅ ( κ ∇ T ) + q EM ( x ) , − κ ∂ n ∂ T = h ( T − T cool ) on Γ pipe , (49)
run forward to convergence. The coolant-side film coefficient is set by a Nusselt correlation of the standard form
N u = C Re a Pr b , h = N u κ f D h , (50) \mathrm{Nu}=C\operatorname{Re}^{a}\operatorname{Pr}^{b}, \qquad h=\frac{\mathrm{Nu}\,\kappa_f}{D_h}, \tag*{(50)} Nu = C Re a Pr b , h = D h Nu κ f , (50)
whose three coefficients ( C , a , b ) (C,a,b) ( C , a , b ) are the epistemically uncertain quantities of the study, alongside the three components of the coil-to-target offset.
Parameter scan. Latin hypercube sampling over the six independent uniform priors of Table 15 . The launched ensemble contains some failed runs; we use the 186 that completed, listed in the release’s completeness manifest. The two groups act on the physics very differently — the coil offsets move where the induced heating is deposited, while the Nusselt coefficients rescale the coolant-side film coefficient.
Table 15. HIVE: the six uncertain parameters, their sampling priors and the ranges realised over the 186 completed runs.
Mesh and a node-ordering subtlety. The solid meshes (coil and target) are identical across runs, with the coil displaced relative to a stationary monoblock. A separate tool then builds a vacuum mesh around the two bodies, and that step permutes the point ordering on the monoblock. The release resolves this by taking the first sample’s mesh as a template and storing, per run, the permutation that matches it; we apply those maps, after which all 186 runs share a single 194,691-node ordering. The preprocessed archive retains only that template’s coordinates — and the coil is in any case not part of the stored mesh — so the per-run coil displacement is not visible in the geometry presented to the operator. It reaches the model through μ \mu μ alone, as does everything else that distinguishes one run from another. The bounding box is x ∈ [ − 11.5 , 11.5 ] x \in[-11.5, 11.5] x ∈ [ − 11.5 , 11.5 ] , y ∈ [ − 75 , 75 ] y \in[-75, 75] y ∈ [ − 75 , 75 ] , z ∈ [ − 11.5 , 19.5 ] z \in[-11.5, 19.5] z ∈ [ − 11.5 , 19.5 ] mm. Unlike the other 3D datasets, no spatial subsampling is applied: all 194,691 nodes are used.
Learning setup. F ∈ R 186 × 1 × 194691 \mathbf{F} \in\mathbb{R}^{186 \times1 \times194691} F ∈ R 186 × 1 × 194691 carrying the converged T f i n a l T_{\mathrm{final}} T final alone, with c i n = 0 c_{\mathrm{in}} = 0 c in = 0 : inputs are R N × 0 × 194691 \mathbb{R}^{N \times0 \times194691} R N × 0 × 194691 , the network sees only the 99-channel encoded coordinates, R B b a t c h × 194691 × 99 \mathbb{R}^{B_{\mathrm{batch}} \times194691 \times99} R B batch × 194691 × 99 , and targets are R N × 1 × 194691 \mathbb{R}^{N \times1 \times194691} R N × 1 × 194691 . μ ∈ R 186 × 6 \boldsymbol{\mu} \in\mathbb{R}^{186 \times6} μ ∈ R 186 × 6 ; split 80/20 (149 train, 37 test) at batch size 5. No spatial subsampling is applied, so this is the largest per-sample node count in the set.
Figure 15. Error maps produced by various architectures in modelling the temperature distribution from inductive heating in a monoblock in the HIVE case.
With one shared mesh and no field input, the model is fitting a 6 → R 194691 6 \to\mathbb{R}^{194691} 6 → R 194691 map from roughly 148 training points in a six-dimensional design space. The conditioning module, not the backbone, is expected to be the bottleneck — which is precisely what makes this case a measurement of the conditioning pathway rather than of the operator. The corpus also ships the first stored timestep T i n i t i a l T_{\mathrm{initial}} T initial ; a companion configuration feeds it in as a field channel at the same seed and split, so the pair measures directly what an initial field is worth against the parameter path.
Data statistics. T f i n a l ∈ [ 300.00 , 557.23 ] T_{\mathrm{final}} \in[300.00, 557.23] T final ∈ [ 300.00 , 557.23 ] K with mean 341.43 K and standard deviation 36.67 K (Skew = 1.30, Kurt = 4.78). Case-mean temperatures span [ 310.98 , 414.67 ] [310.98, 414.67] [ 310.98 , 414.67 ] K. As on the heat sink, the ∼ 300 \sim300 ∼ 300 K pedestal flatters Equation (39) : a single global constant scores 9.3% and a per-case constant 6.5%. A linear fit of μ \mu μ to the case mean explains R 2 = 0.89 R^2 = 0.89 R 2 = 0.89 of the case-to-case variation and scores 6.9%, so the six parameters set the bulk thermal level well and what the operator must actually supply is the spatial structure. For scale, the stored initial field T i n i t i a l T_{\mathrm{initial}} T initial sits at [ 294.06 , 405.23 ] [294.06, 405.23] [ 294.06 , 405.23 ] K and the rise to convergence spans [ 0.03 , 162.58 ] [0.03, 162.58] [ 0.03 , 162.58 ] K with mean 39.34 K — the field the operator predicts is far from the state the simulation started in.
MAST-U: full-device structural reconstruction
The device. MAST-U is the upgraded successor to MAST, the Mega-Ampere Spherical Tokamak at UKAEA’s Culham site (Milnes et al., 2015). A spherical tokamak has a low aspect ratio: the plasma wraps closely around a slender centre column, so magnets, vessel, and in-vessel structures are packed into a compact volume. MAST-U places many of its coils inside the main vacuum vessel [8 ] , and adds closed upper and lower divertors that can run configurations from conventional to the long-leg Super-X (Morris et al., 2018). Its experimental programme targets the integration of core, pedestal, and exhaust physics (Harrison et al., 2024), and it informs the design of the Spherical Tokamak for Energy Production (STEP) [6 ] . Structurally, the machine is a 6.8 m tall, 6.2 m wide assembly of roughly 110 tonnes (Ellis et al., 2025), held on four support legs.
Why a whole-device model. Fusion components are usually analysed one at a time, with loads from the rest of the machine prescribed as boundary conditions. For a tightly coupled assembly those boundary conditions are not known in advance: temperature, stress, and magnetic loads vary with position and are related through the operating state of the whole device, so a component-scale view makes the mechanical equilibrium problem difficult even to pose. Ellis et al. (2025) demonstrates a single mechanical model of the complete MAST-U structure from 3,938 CAD volumes, using a conformal tetrahedral mesh and the solid-mechanics module of MOOSE (Permann et al., 2020), as a first step towards a structural digital twin. The paper describes approximately 127 million elements, whereas the archived mesh used here contains 134,609,743 four-node tetrahedra and 29,183,754 unique nodes. Each static solve takes about 500 CPU-hours on 600–800 cores (Ellis et al., 2025). A digital twin must be queried far more often than that cost allows, across load cases, operating scenarios, material uncertainty, and design changes, which is the role a trained surrogate would fill (Pamela et al., 2025). We take the first step of that programme: a surrogate conditioned on the two parameters of a thermal-load parameter study.
Governing equations. For small-strain linear elasticity, displacement u : Ω → R 3 \mathbf{u} : \Omega\to\mathbb{R}^{3} u : Ω → R 3 satisfies
∇ ⋅ σ + ρ g = 0 , σ = C ( ω ) : ( ε ( u ) − ε t h ) , ε t h = α ( T ( x ) − T s f ) I , (51) \nabla\cdot\boldsymbol{\sigma} + \rho\mathbf{g} = 0,\qquad
\boldsymbol{\sigma} = \mathsf{C}(\omega) : \left(\boldsymbol{\varepsilon}(\mathbf{u}) - \boldsymbol{\varepsilon}^{\mathrm{th}}\right),\qquad
\boldsymbol{\varepsilon}^{\mathrm{th}} = \alpha\left(T(\mathbf{x}) - T_{\mathrm{sf}}\right)\mathbf{I}, \tag*{(51)} ∇ ⋅ σ + ρ g = 0 , σ = C ( ω ) : ( ε ( u ) − ε th ) , ε th = α ( T ( x ) − T sf ) I , (51)
with ε ( u ) = 1 2 ( ∇ u + ( ∇ u ) T ) \boldsymbol{\varepsilon}(\mathbf{u}) = \frac{1}{2}\left(\nabla\mathbf{u} + (\nabla\mathbf{u})^{\mathsf{T}}\right) ε ( u ) = 2 1 ( ∇ u + ( ∇ u ) T ) , C \mathsf{C} C the piecewise isotropic stiffness tensor, α \alpha α the thermal expansion coefficient and T s f T_{\mathrm{sf}} T sf the stress-free temperature. Displacement is fixed at the four support-leg footprints. Stainless steel, copper and graphite have Young’s moduli ( 210 , 125 , 11 ) (210, 125, 11) ( 210 , 125 , 11 ) GPa, densities ( 7800 , 8960 , 2260 ) (7800, 8960, 2260) ( 7800 , 8960 , 2260 ) kg m− 3 ^{-3} − 3 and a common Poisson ratio of 0.3 (Ellis et al., 2025). Merged material interfaces share nodes, enforcing displacement continuity while strain jumps across the stiffness discontinuity. We predict nodal ( u x , u y , u z ) (u_x, u_y, u_z) ( u x , u y , u z ) in metres; element-wise stresses are not targets.
Parameter scan. The corpus contains a stochastic parameter sampling study giving 70 simulations on the same device. Each solve uses 1,120 MPI ranks and samples the two parameters of Table 16 over a rectangle. Each sub-application writes its own partitioned Nemesis/Exodus output. The sampled inputs are read from the corresponding study’s parameter table, with the study identifier and sub-application index cross-checked against the row; the output files themselves carry no inputs. The product α T w \alpha T_w α T w , which sets the thermal-loading scale, spans a factor of 3.36 (5.32 × 10 − 3 5.32 \times10^{-3} 5.32 × 1 0 − 3 to 1.79 × 10 − 2 1.79 \times10^{-2} 1.79 × 1 0 − 2 ).
Table 16. MAST-U: the two conditioning parameters over the pooled 70 solves. Both are min–max normalised per variable using the pooled corpus ranges before entering the conditioning module.
Mesh. All 70 solves share one mesh with 29,183,754 nodes and 134,609,743 four-node tetrahedra; cache construction checks each new solve’s coordinates against the reference mesh node for node. Comparison with the single archived solve of Ellis et al. (2025) confirms exactly matching coordinates, per-node material membership and lumped control volumes. The 1,120-rank decomposition does, however, renumber nodes relative to that archived solve, so the two must be matched by coordinate rather than global ID. The mesh is strongly multiscale: merging thousands of parts leaves sliver elements, with edge lengths from 1.4 × 10 − 6 1.4 \times10^{-6} 1.4 × 1 0 − 6 to 4.7 × 10 − 2 4.7 \times10^{-2} 4.7 × 1 0 − 2 m around a 3 mm median. Node count is therefore a poor proxy for volume: the lumped nodal volume spans eleven orders of magnitude. Steel, copper and graphite occupy 64.4%, 22.3% and 13.3% of the 13.11 m 3 13.11\ \mathrm{m}^{3} 13.11 m 3 of solid and meet at 1,290,161 shared interface nodes (1,164,457 steel/copper, 125,704 steel/graphite, none copper/graphite).
Split. The split contains 60 training solves and 10 named test solves. All ten lie inside the training parameter ranges, so the task measures interpolation in μ \boldsymbol{\mu} μ .
Learning setup. The full corpus is F ∈ R 70 × 3 × 29,183,754 \mathbf{F} \in\mathbb{R}^{70 \times3 \times29{,}183{,}754} F ∈ R 70 × 3 × 29 , 183 , 754 , carrying ( u x , u y , u z ) (u_x,u_y,u_z) ( u x , u y , u z ) in metres, with c i n = 0 c_{\mathrm{in}} = 0 c in = 0 and μ ∈ R 70 × 2 \boldsymbol{\mu} \in\mathbb{R}^{70 \times2} μ ∈ R 70 × 2 . Training and the ten-case evaluation uses the entire 29.2M nodes with material specific routing for the mesh transfer. The shared mesh allows positional encodings and spline stencils to be computed once and reused. Native-mesh inference for Figure 5 uses all 29,183,754 nodes as both sources and queries.
Data statistics. Over all 70 solves and all native nodes, u x ∈ [ − 3.70 , 3.77 ] u_x \in[-3.70,3.77] u x ∈ [ − 3.70 , 3.77 ] cm and u y ∈ [ − 3.71 , 3.73 ] u_y \in[-3.71,3.73] u y ∈ [ − 3.71 , 3.73 ] cm are approximately centred (means −0.004 and 0.003 cm, standard deviations 0.92 and 0.98 cm), while u z ∈ [ − 0.18 , 5.43 ] u_z \in[-0.18,5.43] u z ∈ [ − 0.18 , 5.43 ] cm has mean 2.10 cm and standard deviation 0.92 cm: heating expands the machine upward and outward from its supports. With equal weight per node, u z u_z u z carries 74.3% of ∥ u ∥ 2 2 \lVert\mathbf{u}\rVert_2^2 ∥ u ∥ 2 2 , compared with 12.1% and 13.7% for u x u_x u x and u y u_y u y , so the joint relative L 2 L_2 L 2 is weighted towards the vertical. Displacement magnitude peaks at 5.89 cm and the case-mean magnitude spans [ 1.52 , 3.54 ] [1.52,3.54] [ 1.52 , 3.54 ] cm, a factor of 2.3. Displacement is pinned to zero at the supports; predicting zero scores exactly 100% under Equation (39) .
Model and optimisation. The model used for Figure 5 is FiLM-conditioned L-SplineFNO with a 120 × 120 × 173 120 \times120 \times173 120 × 120 × 173 latent grid, four dense FNO layers of width 32, 12 Fourier modes, spline scatter scales ( 1 , 2 , 4 ) (1,2,4) ( 1 , 2 , 4 ) and material-specific lifting, projection and spline kernels around a shared spectral backbone; 28,359,627 trainable parameters. A local bypass carries the conditioned lifted node features directly to the decoder, where they are concatenated with the decoded grid features and passed through a material-specific GELU readout with hidden width 64. Coordinates are min–max normalised per axis and encoded with integrated positional encoding at 16 frequencies; displacement components are standardised separately. We train for 500 epochs with Adam, an initial learning rate of 10 − 3 10^{-3} 1 0 − 3 , cosine decay to zero, gradient clipping at 1.0, the relative L 2 L_2 L 2 loss on standardised targets and deterministic kernels. The training takes 18.7 h on one Nvidia H100 with 96 GB.
Native-mesh reconstruction. Figure 5 shows displacement magnitude and absolute vector error for the held-out solve with ( α , T w ) = ( 1.43283 × 10 − 5 K − 1 , 863.979 K ) (\alpha,T_w) = (1.43283 \times10^{-5}\ \mathrm{K}^{-1}, 863.979\ \mathrm{K}) ( α , T w ) = ( 1.43283 × 1 0 − 5 K − 1 , 863.979 K ) , on the undeformed device with the front quarter removed. All native source nodes contribute to a single global encoding and FNO evaluation, and all 29,183,754 query nodes are decoded in chunks. This full-mesh evaluation reaches a relative vector L 2 L_2 L 2 error of 3.40% on this case, compared with 1.60% for its saved prediction on the sampled nodes. The ten-case mean above therefore describes the sampled-node evaluation; the native-mesh result additionally changes the source set used to construct the latent field. The figure uses millimetres and caps the error colour scale at the 99th percentile, 2.87 mm; the maximum nodal vector error is 13.02 mm.
Models
This appendix describes the six baselines against which the spline-gridded operators are compared. They fall into three families: point-set transformers that never leave the mesh (Transolver, GNOT), latent-token transformers that compress the mesh into a fixed-size bottleneck (UPT, MSPT), and gridded neural operators that transfer the mesh onto a uniform latent grid so that an FNO can be applied (NUNO, GINO). The last family is the one our method belongs to, and the transfer operators used by NUNO and GINO — nearest-cell scatter and a radius-graph kernel integral, respectively — are exactly the components the spline encode/decode pair replaces.
All six consume the same per-node tensor produced by the pipeline,
x i = [ ω i ∥ u 0 ( ω i ) ] ∈ R d + c i n , i = 1 , … , N , (52) \mathbf{x}_i = [\boldsymbol{\omega}_i \mathbin{\|} u_0(\boldsymbol{\omega}_i)] \in\mathbb{R}^{d+c_{\mathrm{in}}}, \qquad i=1,\ldots,N, \tag*{(52)} x i = [ ω i ∥ u 0 ( ω i )] ∈ R d + c in , i = 1 , … , N , (52)
where ω i ∈ Ω \boldsymbol{\omega}_i \in\Omega ω i ∈ Ω is the coordinate of mesh node i i i and u 0 ( ω i ) u_0(\boldsymbol{\omega}_i) u 0 ( ω i ) the input field features at that node, and all six emit Y ∈ R N × c o u t \mathbf{Y} \in\mathbb{R}^{N \times c_{\mathrm{out}}} Y ∈ R N × c out at the same nodes. In all six benchmarks there is no input field, and the tensor is the positional encoding of Equation (3) , x i = [ ω ˉ i ∥ φ ( ω ˉ i ) ] ∈ R 3 + 96 \mathbf{x}_i = [\bar{\boldsymbol{\omega}}_i \mathbin{\|} \varphi(\bar{\boldsymbol{\omega}}_i)] \in\mathbb{R}^{3+96} x i = [ ω ˉ i ∥ φ ( ω ˉ i )] ∈ R 3 + 96 . Conditioning on μ ∈ P \mu\in\mathcal{P} μ ∈ P enters through the shared conditioning module (FiLM, concatenation, or cross-attention) applied to the encoded features φ ( ω ˉ i ) \varphi(\bar{\boldsymbol{\omega}}_i) φ ( ω ˉ i ) before each model’s input embedding, with the leading d d d coordinate channels passed through unmodulated, except where a model has its own native mechanism — GINO’s adaptive instance normalisation, noted below. Throughout, c c c denotes the model width, LN layer normalisation, and MHA ( Q , K , V ) = Softmax ( Q K T / c h ) V \operatorname{MHA}(Q,K,V)=\operatorname{Softmax}(QK^{\mathsf{T}}/\sqrt{c_h})V MHA ( Q , K , V ) = Softmax ( Q K T / c h ) V standard softmax attention with head dimension c h c_h c h ; MHSA ( X ) \operatorname{MHSA}(X) MHSA ( X ) is the self-attention special case MHA ( X , X , X ) \operatorname{MHA}(X,X,X) MHA ( X , X , X ) .
Table 17. The six baselines by encode–process–decode role. N N N is the number of mesh nodes, n s n_s n s the number of slices or latent tokens, n p n_p n p the number of patches and n q n_q n q the supernodes per patch, c c c the model width. The two gridded operators have no attention term; their cost is dominated by the FFT on the latent grid, O ( G log G ) O(G\log G) O ( G log G ) for G G G grid points.
Transolver
Transolver (Wu et al., 2024) is a transformer solver for PDEs on general geometries. Applying a standard transformer to a discretised 3D mesh is quadratic in the node count and therefore infeasible at the 10 4 10^4 1 0 4 –10 6 10^6 1 0 6 nodes these benchmarks carry. Transolver’s answer is to stop attending between nodes altogether: it learns a soft partition of the nodes into a small number n s ≪ N n_s \ll N n s ≪ N of slices , attends only among the n s n_s n s slice tokens, and broadcasts the result back. Crucially the partition is a function of the node features , not of spatial proximity, so nodes in physically similar states — a boundary layer distributed across a wing, say — are grouped into the same token regardless of where they sit in Ω \Omega Ω . This is the Physics-Attention mechanism, and it is what lets the model be geometry-general without a graph. Transolver++ (Luo et al., 2025) scales the same idea to million-node meshes.
The mesh is treated as an unordered point set. There is no adjacency structure, no grid, and no requirement that nodes be in correspondence across samples, so the model is permutation-equivariant in the node index and consumes per-sample geometries without modification.
Encoding. Node features are lifted, then assigned to slices by a softmax with a per-node adaptive temperature s i s_i s i , which controls how sharply a node commits to a single slice:
h i ( 0 ) = MLP l i f t ( x i ) ∈ R c , s i = Softplus ( MLP s ( h i ) ) + ε , (53) h_i^{(0)} = \operatorname{MLP}_{\mathrm{lift}}(x_i) \in\mathbb{R}^{c}, \qquad s_i = \operatorname{Softplus}(\operatorname{MLP}_s(h_i)) + \varepsilon, \tag*{(53)} h i ( 0 ) = MLP lift ( x i ) ∈ R c , s i = Softplus ( MLP s ( h i )) + ε , (53)
w i j = exp ( ( W s h i ) j / s i ) ∑ j ′ = 1 n s exp ( ( W s h i ) j ′ / s i ) , W s ∈ R M × c , ∑ j = 1 n s w i j = 1. (54) w_{ij} = \frac{\exp\left((W_s h_i)_j/s_i\right)}{\sum_{j'=1}^{n_s} \exp\left((W_s h_i)_{j'}/s_i\right)}, \qquad W_s \in\mathbb{R}^{M \times c}, \qquad\sum_{j=1}^{n_s} w_{ij} = 1. \tag*{(54)} w ij = ∑ j ′ = 1 n s exp ( ( W s h i ) j ′ / s i ) exp ( ( W s h i ) j / s i ) , W s ∈ R M × c , j = 1 ∑ n s w ij = 1. (54)
The n s n_s n s slice tokens are the weighted means of their member nodes,
z j = ∑ i = 1 N w i j h i ∑ i = 1 N w i j + ε ∈ R c . (55) z_j = \frac{\sum_{i=1}^{N} w_{ij} h_i}{\sum_{i=1}^{N} w_{ij} + \varepsilon} \in\mathbb{R}^{c}. \tag*{(55)} z j = ∑ i = 1 N w ij + ε ∑ i = 1 N w ij h i ∈ R c . (55)
Processing. Full softmax self-attention is applied among the n s n_s n s tokens only, which is where the N 2 N^2 N 2 cost is avoided:
Z ′ = Z + MHSA ( LN ( Z ) ) , Z = [ z 1 , … , z M ] ⊤ ∈ R M × c . (56) Z' = Z + \operatorname{MHSA}(\operatorname{LN}(Z)), \qquad Z = [z_1,\ldots,z_M]^{\top} \in\mathbb{R}^{M \times c}. \tag*{(56)} Z ′ = Z + MHSA ( LN ( Z )) , Z = [ z 1 , … , z M ] ⊤ ∈ R M × c . (56)
Decoding. Tokens are scattered back to nodes with the same assignment weights (“deslicing”), followed by a pointwise feed-forward block; L L L such blocks are stacked and a final projection produces the output field:
h i ← h i + ∑ j = 1 n s w i j z j ′ , h i ← h i + FFN ( LN ( h i ) ) , Y i = MLP p r o j ( h i ( L ) ) . (57) h_i \leftarrow h_i + \sum_{j=1}^{n_s} w_{ij} z'_j, \qquad h_i \leftarrow h_i + \operatorname{FFN}(\operatorname{LN}(h_i)), \qquad Y_i = \operatorname{MLP}_{\mathrm{proj}}\left(h_i^{(L)}\right). \tag*{(57)} h i ← h i + j = 1 ∑ n s w ij z j ′ , h i ← h i + FFN ( LN ( h i )) , Y i = MLP proj ( h i ( L ) ) . (57)
Each block costs O ( N n s c + n s 2 c ) O(Nn_s c + n_s^2 c) O ( N n s c + n s 2 c ) , linear in N N N . The LinearNO variant used in some of our sweeps replaces the token-level softmax attention with separate key and query slicing weights and a spectrally-normalised value projection, z j = ∑ i w i j k ( W v h i ) / ∑ i w i j k z_j = \sum_i w_{ij}^{k}(W_v h_i)/\sum_i w_{ij}^{k} z j = ∑ i w ij k ( W v h i ) / ∑ i w ij k and h ^ i = ∑ j w i j q LN ( z j ) \hat{h}_i = \sum_j w_{ij}^{q}\operatorname{LN}(z_j) h ^ i = ∑ j w ij q LN ( z j ) , removing the n s 2 n_s^2 n s 2 term.
GNOT (Hao et al., 2023) is a transformer built for the case where the operator consumes heterogeneous inputs — a geometry, one or more input functions, and a set of scalar parameters — which is precisely the conditional setting of this paper. It keeps a trunk stream of query points and one branch stream per input function, with separate key/value projections for each branch, so streams of different physical meaning are never forced into a shared embedding. Two design choices make it usable at mesh scale: all attention is normalised linear attention, O ( N c 2 ) O(Nc^2) O ( N c 2 ) rather than O ( N 2 c ) O(N^2c) O ( N 2 c ) , and the feed-forward blocks are mixtures of experts gated by the point’s coordinates, which lets different regions of the domain (boundary layer against far field) be handled by different experts — a soft, learned analogue of domain decomposition.
Inputs are point clouds. Query points are arbitrary and need not coincide with the points at which the input functions are sampled, so the model imposes no mesh correspondence across samples.
Encoding. The trunk and each of the n b n_b n b branches are embedded by residual MLPs,
h i ( 0 ) = MLP t r u n k ( x i ) ∈ R c , y t ( s ) = MLP b r a n c h ( s ) ( b t ( s ) ) ∈ R c , s = 1 , … , S . (58) h_i^{(0)} = \operatorname{MLP}_{\mathrm{trunk}}(x_i) \in\mathbb{R}^{c}, \qquad y_t^{(s)} = \operatorname{MLP}_{\mathrm{branch}}^{(s)}\left(b_t^{(s)}\right) \in\mathbb{R}^{c}, \qquad s = 1,\ldots,S. \tag*{(58)} h i ( 0 ) = MLP trunk ( x i ) ∈ R c , y t ( s ) = MLP branch ( s ) ( b t ( s ) ) ∈ R c , s = 1 , … , S . (58)
In our single-input-function configuration n b = 1 n_b = 1 n b = 1 and the branch is fed the same node tensor as the trunk.
Processing. Write ς \varsigma ς for the softmax taken over the feature dimension. The normalised linear attention kernel is
LinAttn ( Q , K , V ) i = ς ( q i ) ⊤ ( ∑ t ς ( k t ) v t ⊤ ) ς ( q i ) ⊤ ∑ t ς ( k t ) , (59) \operatorname{LinAttn}(Q,K,V)_i = \frac{\varsigma(q_i)^{\top}\left(\sum_t \varsigma(k_t)v_t^{\top}\right)}{\varsigma(q_i)^{\top}\sum_t \varsigma(k_t)}, \tag*{(59)} LinAttn ( Q , K , V ) i = ς ( q i ) ⊤ ∑ t ς ( k t ) ς ( q i ) ⊤ ( ∑ t ς ( k t ) v t ⊤ ) , (59)
which never forms the N × N N \times N N × N attention matrix: the context ∑ t ς ( k t ) v t ⊤ ∈ R c h × c h \sum_t \varsigma(k_t)v_t^{\top} \in\mathbb{R}^{c_h \times c_h} ∑ t ς ( k t ) v t ⊤ ∈ R c h × c h is accumulated first, giving O ( N c 2 ) O(Nc^2) O ( N c 2 ) cost. Cross-attention sums the per-branch contributions, and the geometry gate selects among n e n_e n e experts,
CrossAttn ( h , { y ( s ) } ) i = ∑ s = 1 n b LinAttn ( W q h , W k ( s ) y ( s ) , W v ( s ) y ( s ) ) i , (60) \operatorname{CrossAttn}\left(h,\{y^{(s)}\}\right)_i = \sum_{s=1}^{n_b} \operatorname{LinAttn}\left(W_q h, W_k^{(s)} y^{(s)}, W_v^{(s)} y^{(s)}\right)_i, \tag*{(60)} CrossAttn ( h , { y ( s ) } ) i = s = 1 ∑ n b LinAttn ( W q h , W k ( s ) y ( s ) , W v ( s ) y ( s ) ) i , (60)
g i = Softmax ( MLP g a t e ( ω i ) ) ∈ Δ n e − 1 , MoE ( h i ) = ∑ e = 1 n e g i e FFN e ( h i ) . (61) g_i = \operatorname{Softmax}\left(\operatorname{MLP}_{\mathrm{gate}}(\omega_i)\right) \in\Delta^{n_e-1}, \qquad\operatorname{MoE}(h_i) = \sum_{e=1}^{n_e} g_{ie}\operatorname{FFN}_e(h_i). \tag*{(61)} g i = Softmax ( MLP gate ( ω i ) ) ∈ Δ n e − 1 , MoE ( h i ) = e = 1 ∑ n e g i e FFN e ( h i ) . (61)
The gate sees the coordinates alone, so its specialisation is spatial while the attention remains feature-driven. One GNOT layer applies both attention types, each followed by a gated mixture:
h ← h + CrossAttn ( LN 1 ( h ) , { LN ( s ) ( y ( s ) ) } ) , h ← h + LN 3 ( MoE 1 ( h ) ) , h \leftarrow h + \operatorname{CrossAttn}(\operatorname{LN}_{1}(h), \{\operatorname{LN}^{(s)}(y^{(s)})\}), \qquad h \leftarrow h + \operatorname{LN}_{3}(\operatorname{MoE}_{1}(h)), h ← h + CrossAttn ( LN 1 ( h ) , { LN ( s ) ( y ( s ) )}) , h ← h + LN 3 ( MoE 1 ( h )) ,
h ← h + LinSelfAttn ( LN 4 ( h ) ) , h ← h + LN 5 ( MoE 2 ( h ) ) . (62) h \leftarrow h + \operatorname{LinSelfAttn}(\operatorname{LN}_{4}(h)), \qquad h \leftarrow h + \operatorname{LN}_{5}(\operatorname{MoE}_{2}(h)). \tag*{(62)} h ← h + LinSelfAttn ( LN 4 ( h )) , h ← h + LN 5 ( MoE 2 ( h )) . (62)
Decoding. A residual MLP maps the final trunk tokens to the output channels, Y i = MLP o u t ( h i ( L ) ) \mathcal{Y}_{i} = \operatorname{MLP}_{\mathrm{out}}(h_{i}^{(L)}) Y i = MLP out ( h i ( L ) ) . Because the trunk is evaluated pointwise, the operator can be queried at coordinates that appear in no training mesh.
The Universal Physics Transformer [1 ] is an encode–process–decode architecture whose defining property is that the processor never sees the discretisation. A Perceiver-style pooling stage (Jaegle et al., 2022) compresses a variable-length point cloud into a fixed number n s n_{s} n s of latent tokens; the physics is modelled entirely among those tokens; and a decoder cross-attends from arbitrary query positions back into the latent set. Cost in the processor is therefore O ( n s 2 c ) O(n_{s}^{2}c) O ( n s 2 c ) and independent of N N N — the model scales with the complexity of the solution, not the density of the mesh — and the encoder and decoder discretisations are decoupled, so the field can be read out on a mesh other than the one it was encoded from.
Meshes are handled as raw point clouds: no padding, no graph construction, no correspondence between samples. Our steady-state configuration drops the latent time-propagation stage of the original, which advances the latent tokens between time steps; the encoder here pools directly over all nodes rather than over the message-passed supernodes of the reference implementation.
Encoding. Coordinates are embedded with geometrically-spaced sinusoids, PE ( ω ) = [ sin ( ω ( ℓ ) f r ) , cos ( ω ( ℓ ) f r ) ] ℓ , r \operatorname{PE}(\omega) = [\sin(\omega^{(\ell)}f_{r}), \cos(\omega^{(\ell)}f_{r})]_{\ell,r} PE ( ω ) = [ sin ( ω ( ℓ ) f r ) , cos ( ω ( ℓ ) f r ) ] ℓ , r with f r = 10000 − r / F f_{r} = 10000^{-r/F} f r = 1000 0 − r / F , and added to the lifted features:
h i = MLP l i f t ( x i ) + W p PE ( ω i ) ∈ R c . (63) h_{i} = \operatorname{MLP}_{\mathrm{lift}}(x_{i}) + W_{p}\operatorname{PE}(\omega_{i}) \in\mathbb{R}^{c}. \tag*{(63)} h i = MLP lift ( x i ) + W p PE ( ω i ) ∈ R c . (63)
n s n_{s} n s learnable query tokens Q ∈ R n s × c Q \in\mathbb{R}^{n_{s} \times c} Q ∈ R n s × c pool the mesh:
Z ( 0 ) = Q + MHA ( LN ( Q ) , LN ( H ) , LN ( H ) ) ∈ R M × c , H = [ h 1 , … , h N ] ⊤ . (64) Z^{(0)} = Q + \operatorname{MHA}(\operatorname{LN}(Q), \operatorname{LN}(H), \operatorname{LN}(H)) \in\mathbb{R}^{M \times c}, \qquad H = [h_{1}, \ldots, h_{N}]^{\top}. \tag*{(64)} Z ( 0 ) = Q + MHA ( LN ( Q ) , LN ( H ) , LN ( H )) ∈ R M × c , H = [ h 1 , … , h N ] ⊤ . (64)
Processing. L L L pre-norm transformer blocks act on the latent set alone:
Z ← Z + MHSA ( LN ( Z ) ) , Z ← Z + FFN ( LN ( Z ) ) . (65) Z \leftarrow Z + \operatorname{MHSA}(\operatorname{LN}(Z)), \qquad Z \leftarrow Z + \operatorname{FFN}(\operatorname{LN}(Z)). \tag*{(65)} Z ← Z + MHSA ( LN ( Z )) , Z ← Z + FFN ( LN ( Z )) . (65)
Decoding. Any query position ω q \omega_{q} ω q forms a query token from the same lifting and positional embedding, and cross-attends into the processed latents:
Y ( ω q ) = MLP p r o j ( h q + MHA ( LN ( h q ) , LN ( Z ( L ) ) , LN ( Z ( L ) ) ) ) . (66) \mathcal{Y}(\omega_{q}) = \operatorname{MLP}_{\mathrm{proj}}\left(h_{q} + \operatorname{MHA}(\operatorname{LN}(h_{q}), \operatorname{LN}(Z^{(L)}), \operatorname{LN}(Z^{(L)}))\right). \tag*{(66)} Y ( ω q ) = MLP proj ( h q + MHA ( LN ( h q ) , LN ( Z ( L ) ) , LN ( Z ( L ) )) ) . (66)
MSPT [9 ] attacks the same quadratic-attention problem from the opposite direction to UPT: instead of compressing the whole mesh into a global bottleneck, it partitions the nodes into n p n_{p} n p patches, runs full attention inside each patch, and couples the patches through a small set of supernodes — n q n_{q} n q pooled summaries per patch — that every patch also attends to. Local detail is therefore resolved at full resolution while global communication is carried by n p n q n_{p}n_{q} n p n q tokens, giving the two scales the name refers to. Supernodes persist across blocks by residual accumulation, so the global state is refined depth-wise rather than recomputed.
Partitioning is either sequential over the node ordering or, for spatially coherent patches, a ball-tree partition of the coordinates; the latter is what makes patches correspond to compact regions of Ω \Omega Ω on an unstructured mesh. Rotary positional embeddings may be applied within a patch sequence. Node counts that are not a multiple of n p n_{p} n p are zero-padded and the padding is stripped on output.
Encoding. Tokens are formed from coordinates (optionally replaced by a unified position embedding: distances to a fixed refd ^{d} d reference lattice, which gives every sample a common frame) concatenated with field features, and permuted into patches { C v } v = 1 n p \{C_v\}_{v=1}^{n_p} { C v } v = 1 n p by the partition π \pi π :
h i = MLP p r e ( [ ϕ ( ω i ) ∥ u 0 ( ω i ) ] ) + p , { C v } v = 1 n p = π ( { ω i } ) , ∣ C v ∣ = N / V . (67) h_i = \operatorname{MLP}_{\mathrm{pre}}\left([\phi(\omega_i)\mathbin{\|}u_0(\omega_i)]\right)+p,\qquad\{C_v\}_{v=1}^{n_p}=\pi(\{\omega_i\}),\qquad|C_v|=N/V. \tag*{(67)} h i = MLP pre ( [ ϕ ( ω i ) ∥ u 0 ( ω i )] ) + p , { C v } v = 1 n p = π ({ ω i }) , ∣ C v ∣ = N / V . (67)
Processing. Each block pools n q n_q n q supernodes per patch and adds the previous block’s,
s v , q ( ℓ ) = pool q ( { h i ( ℓ ) } i ∈ C v ) + s v , q ( ℓ − 1 ) , S ( ℓ ) = [ s v , q ( ℓ ) ] ∈ R V Q × c , (68) s_{v,q}^{(\ell)}=\operatorname{pool}_q\left(\{h_i^{(\ell)}\}_{i\in C_v}\right)+s_{v,q}^{(\ell-1)},\qquad S^{(\ell)}=[s_{v,q}^{(\ell)}]\in\mathbb{R}^{VQ\times c}, \tag*{(68)} s v , q ( ℓ ) = pool q ( { h i ( ℓ ) } i ∈ C v ) + s v , q ( ℓ − 1 ) , S ( ℓ ) = [ s v , q ( ℓ ) ] ∈ R V Q × c , (68)
with pool a mean, top-n q n_q n q max, or learned linear pooling. Attention then runs on each patch concatenated with the full supernode set , so every node sees a summary of every other patch:
H ~ v = [ H C v ∥ S ( ℓ ) ] ∈ R ( N / V + V Q ) × c , H ~ v ← H ~ v + MHSA ( LN ( H ~ v ) ) , (69) \widetilde{H}_v=[H_{C_v}\mathbin{\|}S^{(\ell)}]\in\mathbb{R}^{(N/V+VQ)\times c},\qquad\widetilde{H}_v\leftarrow\widetilde{H}_v+\operatorname{MHSA}(\operatorname{LN}(\widetilde{H}_v)), \tag*{(69)} H v = [ H C v ∥ S ( ℓ ) ] ∈ R ( N / V + V Q ) × c , H v ← H v + MHSA ( LN ( H v )) , (69)
H ~ v ← H ~ v + FFN ( LN ( H ~ v ) ) , H C v = H ~ v [ 1 : N / V ] , S ( ℓ ) ← 1 V ∑ v = 1 V H ~ v [ − V Q : ] . (70) \widetilde{H}_v\leftarrow\widetilde{H}_v+\operatorname{FFN}(\operatorname{LN}(\widetilde{H}_v)),\qquad H_{C_v}=\widetilde{H}_v[1:N/V],\qquad S^{(\ell)}\leftarrow\frac{1}{V}\sum_{v=1}^{V}\widetilde{H}_v[-VQ:]. \tag*{(70)} H v ← H v + FFN ( LN ( H v )) , H C v = H v [ 1 : N / V ] , S ( ℓ ) ← V 1 v = 1 ∑ V H v [ − V Q : ] . (70)
The cost is O ( n p ( N / n p + n p n q ) 2 c ) = O ( ( N 2 / n p + N n p n q ) c ) O\left(n_p(N/n_p+n_pn_q)^2c\right)=O\left((N^2/n_p+Nn_pn_q)c\right) O ( n p ( N / n p + n p n q ) 2 c ) = O ( ( N 2 / n p + N n p n q ) c ) , minimised at n p ∼ N / n q n_p\sim\sqrt{N/n_q} n p ∼ N / n q .
Decoding. The final block projects to the output channels and the patch permutation is inverted, Y = π − 1 ( W o u t LN ( H ( L ) ) ) \mathcal{Y}=\pi^{-1}(W_{\mathrm{out}}\operatorname{LN}(H^{(L)})) Y = π − 1 ( W out LN ( H ( L ) )) .
NUNO ((Liu et al., 2023)) extends grid-based operator learning to non-uniform data. Fast neural operators such as the FNO ((Li et al., 2021)) rest on the FFT, which requires a uniform grid; applying one to a tetrahedral mesh means interpolating globally onto a single grid, and the resulting error is set by the worst-resolved region of the domain. NUNO’s remedy is decomposition: a K-D tree recursively splits Ω \Omega Ω into n c n_c n c subdomains of roughly equal node count, so that dense regions receive small cells and sparse regions large ones, and the local interpolation error is bounded within each subdomain rather than globally. The subdomain grids are then processed by an FNO, retaining FFT speed.
Encoding. The tree splits each node set P P P at the median along its widest axis,
ℓ ∗ = arg max ℓ ∈ { 1 , … , d } ( max i ∈ P ω i ( ℓ ) − min i ∈ P ω i ( ℓ ) ) , \ell^*=\arg\max_{\ell\in\{1,\ldots,d\}}\left(\max_{i\in P}\omega_i^{(\ell)}-\min_{i\in P}\omega_i^{(\ell)}\right), ℓ ∗ = arg ℓ ∈ { 1 , … , d } max ( i ∈ P max ω i ( ℓ ) − i ∈ P min ω i ( ℓ ) ) ,
P ↦ { i ∈ P : ω i ( ℓ ∗ ) ≤ m ℓ ∗ } ∪ { i ∈ P : ω i ( ℓ ∗ ) > m ℓ ∗ } , (71) P\mapsto\{i\in P:\omega_i^{(\ell^*)}\le m_{\ell^*}\}\cup\{i\in P:\omega_i^{(\ell^*)}>m_{\ell^*}\}, \tag*{(71)} P ↦ { i ∈ P : ω i ( ℓ ∗ ) ≤ m ℓ ∗ } ∪ { i ∈ P : ω i ( ℓ ∗ ) > m ℓ ∗ } , (71)
recursed until n c n_c n c leaves remain. Lifted features are transferred to the uniform grid G \mathcal{G} G of shape S 1 × ⋯ × S d S_1\times\cdots\times S_d S 1 × ⋯ × S d by nearest-cell scatter,
X G [ g ] = ∑ i : γ ( ω i ) = g X i , γ ( ω i ) ( ℓ ) = ⌊ ω i ( ℓ ) − min j ω j ( ℓ ) max j ω j ( ℓ ) − min j ω j ( ℓ ) ( S ℓ − 1 ) ⌋ , (72) \mathcal{X}_{\mathcal{G}}[g]=\sum_{i:\gamma(\omega_i)=g}\mathcal{X}_i,\qquad\gamma(\omega_i)^{(\ell)}=\left\lfloor\frac{\omega_i^{(\ell)}-\min_j\omega_j^{(\ell)}}{\max_j\omega_j^{(\ell)}-\min_j\omega_j^{(\ell)}}(S_\ell-1)\right\rfloor, \tag*{(72)} X G [ g ] = i : γ ( ω i ) = g ∑ X i , γ ( ω i ) ( ℓ ) = ⌊ max j ω j ( ℓ ) − min j ω j ( ℓ ) ω i ( ℓ ) − min j ω j ( ℓ ) ( S ℓ − 1 ) ⌋ , (72)
after which the normalised grid coordinates are concatenated as channels and lifted. Equation (72) is the piecewise-constant limit of the transfer operator our spline encoder generalises: it is order-zero, so it assigns a node’s whole contribution to one cell and provides no gradient with respect to ω i \omega_i ω i .
Processing. Standard Fourier layers act on the grid, truncated to ∣ k ℓ ∣ ≤ m ℓ |k_\ell|\le m_\ell ∣ k ℓ ∣ ≤ m ℓ modes per axis:
v ( ℓ + 1 ) = σ ( W v ( ℓ ) + F − 1 ( R ( ℓ ) ⋅ F ( v ( ℓ ) ) ) ) . (73) v^{(\ell+1)}=\sigma\left(Wv^{(\ell)}+\mathcal{F}^{-1}\left(R^{(\ell)}\cdot\mathcal{F}(v^{(\ell)})\right)\right). \tag*{(73)} v ( ℓ + 1 ) = σ ( W v ( ℓ ) + F − 1 ( R ( ℓ ) ⋅ F ( v ( ℓ ) ) ) ) . (73)
Decoding. A pointwise MLP expands the grid features into n c n_c n c per-subdomain copies, and each subdomain’s nodes are read out by evaluating the truncated Fourier series directly at their true coordinates, rather than by interpolating off the grid a second time:
Y i = MLP o u t ( ∑ ∥ k ∥ ∞ ≤ m v ^ k e 2 π i ⟨ k , ω i ⟩ ) , v ^ k = ∑ g ∈ G v [ g ] e − 2 π i ⟨ k , ξ g ⟩ . (74) \mathcal{Y}_i = \operatorname{MLP}_{\mathrm{out}}\left(\sum_{\lVert k\rVert_\infty\le m} \hat{v}_k e^{2\pi i\langle k,\omega_i\rangle}\right), \qquad\hat{v}_k = \sum_{g\in\mathcal{G}} v[g]e^{-2\pi i\langle k,\xi_g\rangle}. \tag*{(74)} Y i = MLP out ∥ k ∥ ∞ ≤ m ∑ v ^ k e 2 π i ⟨ k , ω i ⟩ , v ^ k = g ∈ G ∑ v [ g ] e − 2 π i ⟨ k , ξ g ⟩ . (74)
This is the same non-uniform forward/inverse transform pair used by Geo-FNO Li et al., 2023a. In three dimensions our implementation falls back to trilinear grid_sample for the read-out, since the explicit O ( N ∏ ℓ 2 m ℓ ) O(N\prod_{\ell}2m_\ell) O ( N ∏ ℓ 2 m ℓ ) basis becomes the dominant memory cost.
Geometry-Informed Neural Operator (GINO)
GINO Li et al., 2023b is the other gridded baseline, and the closest architectural comparison to our method: it is an FNO on a uniform latent grid, bracketed by learned transfer operators that move features between the mesh and that grid. Where we use a tensor-product spline scatter and its analytic adjoint, GINO uses a Graph Neural Operator on both sides — a radius-graph kernel integral that aggregates, for each latent grid point, the mesh nodes within a ball of radius r r r , and then the reverse. The encoder and decoder are therefore Monte-Carlo/Riemann approximations of a continuous kernel integral, learned as a neural network of the relative displacement.
The radius graph is built between the mesh and the latent grid, so the mesh needs no fixed connectivity and per-sample geometries are admitted. Two practical points follow from the construction. First, r r r is measured in the normalised [ 0 , 1 ] d [0,1]^d [ 0 , 1 ] d coordinates of the latent grid and must be at least the largest grid spacing, or latent points collect no neighbours and their features are identically zero. Second, because the neighbourhood structure differs per sample, the reference implementation batches only when the geometry is shared, and our wrapper therefore loops over samples.
Encoding. For each latent grid point ξ g \xi_g ξ g , g ∈ G g\in\mathcal{G} g ∈ G , with neighbourhood N r ( ξ g ) = { i : ∥ ω i − ξ g ∥ 2 ≤ r } \mathcal{N}_r(\xi_g)=\{i:\lVert\omega_i-\xi_g\rVert_2\le r\} N r ( ξ g ) = { i : ∥ ω i − ξ g ∥ 2 ≤ r } ,
X G [ g ] = 1 ∣ N r ( ξ g ) ∣ ∑ i ∈ N r ( ξ g ) κ G ( ξ g , ω i , X i ) X i ≈ ∫ B r ( ξ g ) κ G ( ξ g , ω ) u ( ω ) d ω , (75) X_{\mathcal{G}}[g] = \frac{1}{\lvert\mathcal{N}_r(\xi_g)\rvert}\sum_{i\in\mathcal{N}_r(\xi_g)}\kappa_{\mathcal{G}}(\xi_g,\omega_i,\mathcal{X}_i)\mathcal{X}_i \approx\int_{B_r(\xi_g)}\kappa_{\mathcal{G}}(\xi_g,\omega)u(\omega)\,\mathrm{d}\omega, \tag*{(75)} X G [ g ] = ∣ N r ( ξ g )∣ 1 i ∈ N r ( ξ g ) ∑ κ G ( ξ g , ω i , X i ) X i ≈ ∫ B r ( ξ g ) κ G ( ξ g , ω ) u ( ω ) d ω , (75)
with κ G \kappa_{\mathcal{G}} κ G an MLP of the sinusoidally-embedded relative displacement ξ g − ω i \xi_g-\omega_i ξ g − ω i .
Processing. An FNO acts on the latent grid as in Equation (73) . Conditioning is native here: when k > 0 k>0 k > 0 the normalisation layers are adaptive instance norms driven by μ \mu μ ,
AdaIN ( v ; μ ) = γ G ( e ( μ ) ) v − E [ v ] Var [ v ] + ε + β G ( e ( μ ) ) , e ( μ ) = [ sin ( f r μ ) , cos ( f r μ ) ] r = 1 F , (76) \operatorname{AdaIN}(v;\mu)=\gamma_{\mathcal{G}}(e(\mu))\frac{v-\mathbb{E}[v]}{\sqrt{\operatorname{Var}[v]+\varepsilon}}+\beta_{\mathcal{G}}(e(\mu)), \qquad e(\mu)=\left[\sin(f_r\mu),\cos(f_r\mu)\right]_{r=1}^{F}, \tag*{(76)} AdaIN ( v ; μ ) = γ G ( e ( μ )) Var [ v ] + ε v − E [ v ] + β G ( e ( μ )) , e ( μ ) = [ sin ( f r μ ) , cos ( f r μ ) ] r = 1 F , (76)
where γ G , β G \gamma_{\mathcal{G}},\beta_{\mathcal{G}} γ G , β G are MLPs. Their width scales with k k k , so the parameter count of GINO grows with the conditioning dimension — at k = 40 k=40 k = 40 the norm MLPs alone add roughly 1.5M parameters, which is why F F F (ada_in_frequencies) is kept small in our configurations.
Decoding. A second GNO integrates back from the grid to the query nodes,
Y i = 1 ∣ N r ( ω i ) ∣ ∑ g ∈ N r ( ω i ) κ G ′ ( ω i , ξ g ) Y G [ g ] , N r ( ω i ) = { g : ∥ ξ g − ω i ∥ 2 ≤ r } . (77) \mathcal{Y}_i=\frac{1}{\lvert\mathcal{N}_r(\omega_i)\rvert}\sum_{g\in\mathcal{N}_r(\omega_i)}\kappa'_{\mathcal{G}}(\omega_i,\xi_g)\mathcal{Y}_{\mathcal{G}}[g], \qquad\mathcal{N}_r(\omega_i)=\{g:\lVert\xi_g-\omega_i\rVert_2\le r\}. \tag*{(77)} Y i = ∣ N r ( ω i )∣ 1 g ∈ N r ( ω i ) ∑ κ G ′ ( ω i , ξ g ) Y G [ g ] , N r ( ω i ) = { g : ∥ ξ g − ω i ∥ 2 ≤ r } . (77)
Like the spline decoder of Equation (11) , Equation (77) is linear in the grid values: once trained, the prediction at node i i i is a combination of the ∣ N r ( ω i ) ∣ \lvert\mathcal{N}_r(\omega_i)\rvert ∣ N r ( ω i )∣ grid values inside the ball, with weights that depend only on the positions of the node and the grid points, never on the field itself. The ball N r \mathcal{N}_r N r plays the role our spline stencil plays, with two differences: its size is set by the radius r r r rather than by the spline degree, and its weights κ G ′ \kappa'_{\mathcal{G}} κ G ′ are learned without any constraint, whereas the analytic spline weights form a partition of unity.
Conditioning Methods
All conditioning methods receive the physical parameter vector μ ∈ R k \boldsymbol{\mu} \in\mathbb{R}^{k} μ ∈ R k and act at the same place in the pipeline: on the per-node encoded coordinate features f = φ ( ω ˉ ) ∈ R B b a t c h × N × c f \mathbf{f} = \varphi(\bar{\boldsymbol{\omega}}) \in\mathbb{R}^{B_{\mathrm{batch}} \times N \times c_f} f = φ ( ω ˉ ) ∈ R B batch × N × c f , c f = 96 c_f = 96 c f = 96 , before the backbone’s input lift Equation (3) . The three normalised coordinates ω ˉ \bar{\boldsymbol{\omega}} ω ˉ are excluded from conditioning for every method and every backbone; the conditioned features f ~ \tilde{\mathbf{f}} f ~ are concatenated with ω ˉ \bar{\boldsymbol{\omega}} ω ˉ and lifted, h i = P i n [ f ~ i ; ω ˉ i ] \mathbf{h}_i = P_{\mathrm{in}}[\tilde{\mathbf{f}}_i;\bar{\boldsymbol{\omega}}_i] h i = P in [ f ~ i ; ω ˉ i ] . No method acts on the width-c c c lifted features or on intermediate backbone states in any reported run.
FiLM
Feature-wise Linear Modulation (FiLM) (Perez et al., 2018) is a lightweight conditioning strategy that affinely transforms each channel of the field representation based on the conditioning parameters. An MLP H μ : R k → R 2 c f H_{\boldsymbol{\mu}} : \mathbb{R}^{k} \to\mathbb{R}^{2c_f} H μ : R k → R 2 c f , composed of the embedding E μ E_{\boldsymbol{\mu}} E μ and the two heads H γ , H β H_{\gamma}, H_{\beta} H γ , H β of § 3.1 , maps μ \boldsymbol{\mu} μ to per-channel scale and shift parameters:
( γ μ , β μ ) = H μ ( μ ) , γ μ , β μ ∈ R c f (78) (\gamma_{\boldsymbol{\mu}},\beta_{\boldsymbol{\mu}})=H_{\boldsymbol{\mu}}(\boldsymbol{\mu}), \qquad\gamma_{\boldsymbol{\mu}},\beta_{\boldsymbol{\mu}}\in\mathbb{R}^{c_f} \tag*{(78)} ( γ μ , β μ ) = H μ ( μ ) , γ μ , β μ ∈ R c f (78)
The conditioned field is then given by an element-wise affine transformation applied uniformly across all spatial locations:
f ~ = γ μ ⊙ f + β μ (79) \tilde{\mathbf{f}}=\gamma_{\boldsymbol{\mu}}\odot\mathbf{f}+\beta_{\boldsymbol{\mu}} \tag*{(79)} f ~ = γ μ ⊙ f + β μ (79)
where ⊙ \odot ⊙ denotes element-wise multiplication broadcast over the spatial dimensions. The FiLM layer preserves the channel count c f c_f c f and does not alter the spatial structure of the representation. The scale and shift heads have zero-initialised weights and biases 1 1 1 and 0 0 0 , so γ μ = 1 \gamma_{\boldsymbol{\mu}}=1 γ μ = 1 and β μ = 0 \beta_{\boldsymbol{\mu}}=0 β μ = 0 exactly at initialisation and the unconditioned backbone behaviour is recovered.
Concatenation
The simplest conditioning strategy directly augments the input field by appending the conditioning parameters as additional spatial channels. Each scalar in μ \boldsymbol{\mu} μ is broadcast uniformly across all mesh nodes or grid locations and concatenated to the field:
f ~ = [ f ∥ 1 Ω ⊗ μ ] ∈ R c f + k (80) \tilde{\mathbf{f}}=\left[\mathbf{f}\mathbin{\|}\mathbf{1}_{\Omega}\otimes\boldsymbol{\mu}\right]\in\mathbb{R}^{c_f+k} \tag*{(80)} f ~ = [ f ∥ 1 Ω ⊗ μ ] ∈ R c f + k (80)
where 1 Ω ⊗ μ \mathbf{1}_{\Omega}\otimes\boldsymbol{\mu} 1 Ω ⊗ μ denotes the spatial broadcast of μ \boldsymbol{\mu} μ to every node or grid point. This increases the number of feature channels from c f c_f c f to c f + k c_f+k c f + k , so the lift becomes P i n : R c f + k + 3 → R c P_{\mathrm{in}}:\mathbb{R}^{c_f+k+3}\to\mathbb{R}^{c} P in : R c f + k + 3 → R c . Concatenation conditioning places no constraint on how the model uses the parameter information, leaving the entire burden of interpreting μ \boldsymbol{\mu} μ to the backbone architecture.
Cross-Attention
Cross-attention conditioning enables the model to selectively query physical parameter information at each spatial location through an attention mechanism. The conditioning parameters μ \boldsymbol{\mu} μ are first embedded into a set of n τ n_{\tau} n τ learnable parameter tokens:
T = reshape ( W τ μ ) + E τ ∈ R n τ × c τ (81) \mathcal{T}=\operatorname{reshape}(W_{\tau}\boldsymbol{\mu})+E_{\tau}\in\mathbb{R}^{n_{\tau}\times c_{\tau}} \tag*{(81)} T = reshape ( W τ μ ) + E τ ∈ R n τ × c τ (81)
where W τ ∈ R ( n τ ⋅ c τ ) × k W_{\tau}\in\mathbb{R}^{(n_{\tau}\cdot c_{\tau})\times k} W τ ∈ R ( n τ ⋅ c τ ) × k is a learnable projection and E τ ∈ R n τ × c τ E_{\tau}\in\mathbb{R}^{n_{\tau}\times c_{\tau}} E τ ∈ R n τ × c τ are learnable positional embeddings. The field features at each spatial location serve as queries, attending to the parameter tokens as keys and values:
Q = W Q f ∈ R N × c τ , (82) Q = W_Q f \in\mathbb{R}^{N \times c_{\tau}}, \tag*{(82)} Q = W Q f ∈ R N × c τ , (82)
K = V = LayerNorm ( T ) ∈ R n τ × c τ , (83) K = V = \operatorname{LayerNorm}(\mathcal{T}) \in\mathbb{R}^{n_{\tau} \times c_{\tau}}, \tag*{(83)} K = V = LayerNorm ( T ) ∈ R n τ × c τ , (83)
A = softmax ( Q K T c τ ) V ∈ R N × c τ (84) A = \operatorname{softmax}\left(\frac{QK^{\mathsf{T}}}{\sqrt{c_{\tau}}}\right)V \in\mathbb{R}^{N \times c_{\tau}} \tag*{(84)} A = softmax ( c τ Q K T ) V ∈ R N × c τ (84)
where N N N is the number of spatial locations and multi-head attention is used in practice. The output is projected back to the c f c_f c f feature channels and added as a residual:
f ~ = f + W o u t A (85) \widetilde{f} = f + W_{\mathrm{out}} A \tag*{(85)} f = f + W out A (85)
The output projection W o u t W_{\mathrm{out}} W out is initialised to zero so that the module acts as an identity at the start of training. This preserves the original backbone dynamics and allows the conditioning signal to be introduced gradually.
This section presents tables showcasing full scale ablations across all of the experiments across all models and conditioning methods. We find that the learnable Spline-FNO, on average offers the best performance and compute-efficiency. Similarly FiLM conditioning method across the spatial features of the input performs remarkably well compared to concatenation and cross-attention.
Table 18. Performance metrics of various models and conditioning methods for the Heat Sink (3D, steady-state, thermal) case. Best value per metric within each architecture in bold.
Table 19. Performance metrics of various models and conditioning methods for the Bracket (3D, steady-state, elasto-plastic) case. Best value per metric within each architecture in bold.
Table 20. Performance metrics of various models and conditioning methods for the Aircraft (3D, steady-state, compressible flow, 5 fields) case. Best value per metric within each architecture in bold. R 2 R^2 R 2 is the unweighted mean of the per-variable field R 2 R^2 R 2 over the five outputs.
Table 21. Performance metrics of various models and conditioning methods for the FDS Building Fire (3D, time-averaged LES, three output fields) case. Single run per entry (seed 42); training times on a single NVIDIA H100; N = 92,837 N = 92{,}837 N = 92 , 837 nodes. Best value per metric within each architecture in bold.
Table 22. Performance metrics of various models and conditioning methods for the Heated Duct (3D, steady-state, multi-region conjugate heat transfer) case. Best value per metric within each architecture in bold.
Table 23. Performance metrics of various models and conditioning methods for the HIVE Monoblock (3D, steady-state, thermal) case. Best value per metric within each architecture in bold.
Scalability
We extend the comparison in § 4.3 using synthetic three-dimensional point clouds with increasing node count, scalar FiLM conditioning, and one output channel. All runs use a single A100 80GB GPU, FP32 with TF32 disabled, and batch size one. We retain the HIVE architecture settings and 99 coordinate features, holding each model’s latent resolution fixed as the point cloud becomes denser. We measure inference without an autograd graph, a forward pass with autograd enabled, a backward pass, and a complete Adam training step. Backward timing excludes the prerequisite forward pass; its memory includes the retained graph. Measurements use five repetitions after warm-up for each of three seeds. Timing curves report medians of per-seed timing medians, and memory curves report medians of per-seed peak PyTorch allocations, including resident tensors but excluding CUDA-context and non-PyTorch allocations. Model and data setup are outside the timed regions.
Figure 16 shows that L-SplineFNO’s runtime and memory grow approximately linearly over the observed range, while its lower memory cost allows execution on substantially larger point clouds. Several baselines also exhibit near-linear trends over their feasible ranges; NUNO is faster in the successful steady-state trials, so runtime and capacity capture different aspects of scalability. The approximately level L-SplineFNO curves in Figure 17 make its stable cost per node more apparent, with fixed overhead contributing more at smaller sizes. These results along with section 4.3 describe the tested implementations at fixed latent resolution; they do not establish accuracy scaling or a unique asymptotic advantage.
Figure 16. Runtime (top) and peak allocated GPU memory (bottom) as node count increases, separated by execution phase. Both axes are logarithmic. Points summarise successful seeds only, and lines connect measured sizes without extrapolation. NUNO’s training result at ten million nodes includes two successful seeds; the remaining seed runs out of memory. Missing points indicate unsuccessful scaling rather than zero cost.
Figure 17. Runtime and peak allocated memory per million nodes, using the same measurements and aggregation as Figure 16 . Both axes are logarithmic. A nearly horizontal curve indicates approximately linear growth of the total cost over the measured range; a decreasing curve can reflect amortisation of fixed overhead. L-SplineFNO maintains a broadly stable cost per node as the point cloud grows.
Ablations
Zero-shot super-resolution
We evaluate L-SplineFNO without fine-tuning on held-out native meshes containing at least 95% of the maximum available nodes, which is 1.91×, 1.79–1.86×, and 2.53× the training resolution for Heat Sink, Bracket, and Aircraft, respectively. We chose these tasks to demonstrate super-resolution because of the inherent mesh variability across their datasets. As shown in Table 24 , the model retains high predictive agreement at each resolution. This zero-shot transfer is enabled by the coordinate-dependent spline scatter and decoder, which operate independently of the number and ordering of mesh nodes: L-SplineFNO can therefore be deployed on substantially finer meshes without retraining while largely preserving its predictive accuracy.
Table 24. Zero-shot super-resolution with FiLM-conditioned L-SplineFNO.
Multiscale depth and spline degree
We isolate two properties of the spline transfer while holding the backbone, FiLM conditioning pathway, optimiser, and training budget fixed: the range of geometric information supplied to the FNO, and the polynomial degree, smoothness, and compact support of the spline basis. Every configuration is trained on all six benchmark cases at three seeds; a seed redraws the train/test split and the initialisation while the sampled point cloud stays fixed, and the heated duct and building fire use region mode basis. Because error and runtime differ substantially between benchmarks, Figure 18 a and Figure 18 b report changes relative to the single-scale cubic configuration used for the main results, paired by seed so that every configuration is compared with the baseline on the same test set; the corresponding absolute errors, runtimes, parameter counts, and R 2 R^2 R 2 values are given in Table 25 and Table 26 .
Figure 18. Accuracy–cost trade-offs for multiscale scattering and spline degree. Values are percentage changes relative to the single-scale and cubic configurations, respectively, computed against the baseline trained at the same seed. Points are means over three seeds and error bars one standard deviation; spline orders 6 and 8 were run at a single seed and carry no bar. Spline orders 2, 4, 6, and 8 correspond to degrees p = 3 p = 3 p = 3 , 5, 7, and 9, i.e. a C p − 1 C^{p-1} C p − 1 basis. By increasing spline order and the scatter length, the model only makes marginal gains against a linear climb of computational costs. Absolute results are reported in Table 25 and Table 26 .
Table 25. Multiscale L-SplineFNO (FiLM conditioning) across the benchmark cases at increasing scatter-scale depth, all at spline degree 3. Heated-duct and FDS run under region mode basis. Errors, L 2 / L 1 L_2/L_1 L 2 / L 1 and R 2 R^2 R 2 are mean ± s.d. over seeds 42, 43 and 44 where a cell has several (train/test split and initialisation vary with the seed); training time is the seed mean. For the multi-variable cases (Aircraft, FDS) R 2 R^2 R 2 is the mean of the per-variable scores.
Table 26: Spline degree on single-scale L-SplineFNO (FiLM conditioning) across the benchmark cases. Heated-duct and FDS run under region mode basis. Errors, L 2 / L 1 L_2/L_1 L 2 / L 1 and R 2 R^2 R 2 are mean ± \pm ± s.d. over seeds 42, 43 and 44 where a cell has several (train/test split and initialisation vary with the seed); training time is the seed mean. For the multi-variable cases (Aircraft, FDS) R 2 R^2 R 2 is the mean of the per-variable scores. Degrees 7 and 9 are a single seed (42), so their absolute errors are not comparable with the seed means above them; the bracketed figure is the change against degree 3 at the same seed.
Table 26.
Multiscale scatter. The single-scale L-SplineFNO scatters the unstructured nodal features onto one fine latent grid. A scatter scale s > 1 s > 1 s > 1 adds a second branch over the same physical domain with approximately s s s times fewer grid intervals along each axis, so its compact spline stencil covers a wider physical neighbourhood; the coarse representation is interpolated back to the fine grid and fused with the other branches by a learned pointwise projection before the FNO blocks. We compare a fine branch alone (1), fine and medium branches (1–2), and fine, medium, and coarse branches (1–2–4), with the analytic cubic decoder unchanged in all three configurations. The additional branches add at most 0.13% to the parameter count.
Multiscale depth gives no systematic accuracy gain. Aircraft is the only case it helps: the 1–2 hierarchy lowers its relative L 2 L_2 L 2 error in every seed, by 5.2 ± 5.7%, while 1–2–4 gives 3.7 ± 11.0% with no consistent sign. The heat sink is consistently harmed, with errors 6.1 ± 4.1% and 6.9 ± 0.9% higher under the two hierarchies. On the bracket and building fire the paired changes stay below 2%, and on the heated duct and HIVE they are smaller than the seed-to-seed spread. The cost, by contrast, is systematic: training time rises by 21–44% for 1–2 and by 40–89% for 1–2–4. The aircraft results suggest that its data form a thin surface inside a volumetric latent grid, where a fine stencil leaves most knots unreached and a coarser branch carries surface information further; on volumetric meshes the fine branch already covers the grid, and the wider branches mainly add smoothing. Additional scatter scales are therefore not a reliable source of accuracy at this budget, which is why the main results use a single scale.
Spline degree. Increasing the spline degree p p p raises the continuity of the analytic basis from C 2 C^2 C 2 for the cubic basis to C 4 C^4 C 4 , C 6 C^6 C 6 , and C 8 C^8 C 8 for degrees five, seven, and nine, and widens the compact support from p + 1 = 4 p+1 = 4 p + 1 = 4 knots per coordinate axis to six, eight, and ten. We vary this degree while retaining a single scatter scale, so the ablation changes the scatter and decoder bases but not the FNO backbone; the parameter count moves by at most nine, through the rank of the partition-of-unity subspace of the learnable kernel. Higher degrees do not improve accuracy. Degree five, measured at all three seeds, lowers the aircraft error and leaves the building fire unchanged, but raises the error on the bracket, heated duct, HIVE, and heat sink. At degrees seven and nine, run at a single seed, the error is never meaningfully below the cubic one and grows with degree on the bracket, aircraft, and heated duct. Aircraft is again the case that benefits from a moderately wider support, in line with the multiscale result, but the extra smoothing of degrees seven and nine outweighs it there too. This is consistent with a wider, smoother basis offering little for fields governed by local geometric detail and, for the heated duct, by sharp changes in gradient across material interfaces, where excess smoothness mixes information across these features.
The computational penalty is substantially larger and systematic. In three dimensions, a degree-p p p tensor-product basis contains ( p + 1 ) 3 (p+1)^3 ( p + 1 ) 3 stencil entries per point — 64, 216, 512, and 1000 for degrees three, five, seven, and nine — so the scatter and decode work scales as O ( N ( p + 1 ) 3 ) O(N(p+1)^3) O ( N ( p + 1 ) 3 ) , where N N N is the number of mesh nodes. Training time accordingly rises exponentially with the degree. Because the stencil is streamed in memory bounded chunks, every degree completes on every mesh, and the cost appears as training time rather than memory. With the parameter count effectively fixed, this cost comes from basis construction alone, which leaves the cubic basis as the most reliable accuracy–efficiency compromise across the benchmark suite.
Analytic Gradient
With the encoded latent grid held fixed, the L-SplineFNO decoder defines a continuous field over the query coordinates. For a selected scalar output, its affine readout can be applied to the latent coefficients before
interpolation, giving
u ^ ( q ) = ∑ g ∈ G c g Φ g ( q ) + b , ∇ q u ^ ( q ) = ∑ g ∈ G c g ∇ q Φ g ( q ) , c g = w T h g , (86) \hat{u}(q)=\sum_{g\in\mathcal{G}} c_g\Phi_g(q)+b,\qquad\nabla_q\hat{u}(q)=\sum_{g\in\mathcal{G}}c_g\nabla_q\Phi_g(q),\qquad c_g=w^{\mathsf{T}}h_g, \tag*{(86)} u ^ ( q ) = g ∈ G ∑ c g Φ g ( q ) + b , ∇ q u ^ ( q ) = g ∈ G ∑ c g ∇ q Φ g ( q ) , c g = w T h g , (86)
where h g h_g h g is a latent-grid coefficient and Φ g \Phi_g Φ g is the normalized cubic tensor-product basis. The readout bias contributes no derivative. Boundary normalization must also be differentiated: writing Φ g = B g / S \Phi_g=B_g/S Φ g = B g / S with S = ∑ h B h > 0 S=\sum_h B_h>0 S = ∑ h B h > 0 on the covered query domain gives ∇ Φ g = ( S ∇ B g − B g ∇ S ) / S 2 \nabla\Phi_g=(S\nabla B_g-B_g\nabla S)/S^2 ∇ Φ g = ( S ∇ B g − B g ∇ S ) / S 2 . Thus the calculation differentiates the actual normalized decoder, including queries near grid boundaries.
Each query touches at most 4 d 4^d 4 d grid points in d d d dimensions. After projecting the latent coefficients onto the requested output channel, the implementation gathers each local coefficient once and reuses it for all first-derivative components. Axis-basis evaluation and material-specific readouts are batched to reduce repeated operations and temporary arrays. This gives O ( N 4 d ) O(N4^d) O ( N 4 d ) query-evaluation work for a fixed number of output channels, in addition to the grid projection. For material-specific readouts, region membership is held fixed, so derivatives are evaluated within each material. Coordinate and output normalization scales convert the results to physical units. These expressions remain differentiable with respect to the latent coefficients and readout weights, allowing first-derivative losses to train the model.
The two pointwise methods in Table 27 evaluate this same fixed-grid field. Full-model autograd VJP instead differentiates the sum of the selected output over all nodes with respect to the input coordinates. Moving an input node can change the encoded grid and hence predictions elsewhere, so this coordinate sensitivity is a different quantity from the pointwise field gradient. Decoder autograd starts from a prepared forward graph, whereas the analytic method constructs its basis derivatives and projects the coefficients during the timed call; these operations are not cached outside the measurement.
Table 27. L-SplineFNO spatial-derivative cost: time and additional peak memory. Decoder autograd and the analytic spline compute the same pointwise gradient of field u u u with the encoded latent grid fixed. Autograd VJP measures full-model coordinate sensitivity. Timings cover only derivative evaluation and unit conversion, excluding encoding and prediction, with a fresh forward state per call. Values are medians of 20 runs after 3 warm-ups on an A100 80GB (batch 1, FP32, TF32 off). Memory is the extra peak CUDA allocation above the prepared state, including any saved autograd graph, not total GPU memory.
Across the six cases, the analytic evaluator is 1.84–3.67× \times × faster than decoder autograd and uses 52–68% less additional peak memory, using the unrounded measurements. The largest relative L 2 L_2 L 2 disagreement between their physical gradients is 3.04 × 10 − 6 3.04\times10^{-6} 3.04 × 1 0 − 6 . Figure 19 shows the corresponding per-case costs. These results concern first-derivative evaluation after forward preparation; they do not measure end-to-end inference or the memory needed to retain that prepared state.
Figure 19. Gradient comparisons across autograd vector Jacobian products across the entire model, Decoder queries and the analytic spline across the six benchmark cases.