WorksComputer Science
ISLA: A Coordinate-Free, Measure-Weighted Transformer for Boundary-Driven PDEs
Abstract
A foundational principle of physics is that all coordinate systems are equally valid: the behavior of a PDE-governed system does not depend on the origin or orientation of the frame, nor on the units or mesh used to write it down, and neither should a surrogate model. Yet the neural surrogates with the lowest errors on today's aerodynamics benchmarks lack these invariances and fail under simple transformations of the same problem: for models trained on PDEs of cars in one pose (i.e., translation and rotation) and from one mesher, rotating the full PDE raises the surface-pressure error from 4% to more than 100%, and re-sampling its boundary raises it to about 30%. We introduce ISLA (Invariant Slice Attention), a slice-attention transformer for steady, boundary-driven PDEs in which no learned operation receives a coordinate and every sum over the boundary is a quadrature. Positions enter only as distances and relative angles to learned slice anchors, lengths are measured in a unit computed from the boundary, and interior points query the boundary through a tree-accelerated learned kernel without influencing one another. With every boundary cell a token and an exact boundary read, ISLA is equivariant to rigid motions and reflections, invariant to the units of its declared data, and consistent under mesh refinement, while at every setting each query is independent of all other queries. On AeroZoo, 2,816 flows that vary in shape, size, speed, and boundary conditions, its mean reference-scaled velocity and pressure errors are at least 3.1 and 3.3 times lower than any baseline's. On AirfRANS its errors are second only to those of a boundary-integral model (GLOBE) and at least 1.6 times lower than those of Transolver and GeoTransolver.
Introduction
Neural surrogates now predict the flow around a car in seconds, with errors of a few percent of the reference simulation [5, 3] (Wu et al. 2024). The models with the lowest errors on aerodynamics benchmarks are transformers that make global attention affordable on meshes of to points, often by routing points through a small set of learned slices, which are soft groups of points [60, 99] (Wu et al. 2024).
A boundary-value problem is a physical statement: a boundary, the conditions imposed on it, and the properties of the medium. Writing it down for a computer takes choices that are not part of the physics: an origin and axes, units, a mesh of the boundary, the cells of that mesh that are kept, and the points at which the solution is requested. We call these choices the presentation of the problem, and the solution does not depend on them. The surrogates with the lowest errors on today’s benchmarks do Figure 1, because the presentation is their input: coordinates enter as features, and sums over points ignore the area that each point represents. Transolver and GeoTransolver also pool interior query points into the shared slices, so a prediction at one point changes when other points are queried (a coupling that anchor-based cross-attention models such as AB-UPT avoid; [5]). For tuned benchmark networks of Transolver and GeoTransolver, trained on DrivAerML [8] as stored, rotating its validation PDE problems at random raises the surface-pressure error from 4% to more than 100% (worse than predicting zero), and uniform area sampling raises it to about 30%.
Figure 1. ISLA’s error does not depend on how a problem is presented. Top: the reference surface pressure coefficient on a DrivAerML evaluation case in three presentations: as stored, with the full PDE problem rotated by and translated, and with surface cells drawn by area (dots: subset of cells given; insets: rear wheel). Below: each model’s error in the car’s own frame, with its relative error on this case. Both models were trained without augmentation (GeoTransolver: width 256, Section C.3); the text quotes tuned DrivAerML ensemble members over 34 validation cases (Table 21).
This dependence limits practical use. A training set fixes one convention: every car points along , in meters, from one mesher. A model that depends on the convention fails when the same problem is written in another frame, sampled by another rule, or stated in other units, and rotation augmentation multiplies the stored-case errors of Transolver and GeoTransolver by 1.04 and 1.03 (§ 5.2). On AeroZoo [81], where every case has its own pose (i.e., translation and rotation), size, speed, and boundary conditions, GeoTransolver needs boundary-condition tokens and per-case scales, both added by hand, to improve on predicting the inlet state everywhere.
Exact symmetry can be built into an architecture layer by layer, but at a cost in training time or accuracy. Layerwise equivariant networks constrain every linear map and nonlinearity ([91, 21, 35, 53, 22, 84]), and AB-GATr, an equivariant transformer on a subset of the points, takes two to eleven times as long to train as its non-equivariant counterpart ([73]). GLOBE ([80]), a boundary-integral architecture, is equivariant and independent of units, and it uses the boundary measure. It has the lowest errors on two-dimensional airfoil flows, and higher errors than slice transformers on three-dimensional automotive flows. We ask whether the accuracy and linear cost of slice attention can be had together with exact independence of the frame, the units, and the query set, and with consistency under remeshing.
ISLA (Invariant Slice Attention) follows three rules, one for each part of the presentation. (i) No learned operation receives a coordinate. Each slice has an anchor, the mean position and mean normal of its tokens (boundary cells) weighted by measure,1 and a token’s position enters the network only as distances and relative angles to the anchors. Lengths are measured in a unit computed from the boundary, and vectors are assembled at the output from directions that rotate with the body. (ii) Every sum over the boundary is a quadrature. Cell measures weight every attention over boundary tokens, so a mesh changes a prediction only by a quadrature error. (iii) Interior points are read-only, to give query independence. A point in the domain receives its exact distance and direction to the boundary, and attends to the slices and, through a tree-accelerated learned kernel, to every boundary token; because it contributes to no sum, the prediction there depends on the point and the boundary alone.
Contributions. First, we introduce ISLA (Section 3), to our knowledge the first slice-attention surrogate that is exactly equivariant to the Euclidean group. Second, we prove its invariances (§ 4) and show that measure-weighted slice attention is a ratio of two quadratures, so that with every cell a token its outputs converge under refinement. Third, we show (§ 5) that ISLA’s errors are the lowest on AeroZoo, second only to a boundary-integral model (GLOBE) on AirfRANS, and changed by at most 1.1% by a change of presentation. An open-source implementation of ISLA will be available in NVIDIA PhysicsNeMo ([67]) at https://github.com/NVIDIA/physicsnemo.
What a Solution Cannot Depend On
Setting. We consider steady boundary-value problems on a domain whose boundary is a union of pieces, , one per boundary condition:
Here collects the unknown fields and its spatial partial derivatives up to order (including divergence, curl, strain rate, and Laplacian), is condition ’s prescribed data, and holds material constants and any vector attached to no finite boundary, such as an asymptotic far-field velocity as on an unbounded domain (a Dirichlet state at a finite outer boundary enters through on that boundary). Given the posed problem alone, a surrogate approximates from examples.
A linear model problem. For a linear problem with constant coefficients, the field at an interior point is an integral over the boundary, , of a density that the boundary data determine [41, 70, 62, 52]. The formula has three features: every boundary point influences every field point, its influence is weighted by the surface measure, and it depends on relative positions alone. ISLA keeps all three and does not assume linearity.
Presentations. A model receives a presentation of : the coordinates of in some frame and units, a mesh of or a sample of one, and a list of points at which is requested. Infinitely many presentations describe one problem. If the problems of interest are closed under rigid motions, reflections, and changes of units, each with one solution, and the equations hold in every frame and system of units (Assumption 1), then the solution operator (i) is covariant under rigid motions and reflections, (ii) rescales each output by the factor that its units dictate, and (iii) depends on as an oriented surface, and not on how is meshed or sampled, on the order of its cells, or on the other query points (Proposition 1, stated and proved in Section A).
The proof uses nothing about the form of . We call a surrogate presentation-independent if it satisfies exactly (i), (ii), and the parts of (iii) that concern the order of the cells and the query set. We call it consistent if its dependence on the mesh and sample vanishes under refinement [82, 77]; ISLA is both under § 4.
The ISLA Architecture
ISLA keeps what makes slice attention effective: a learned soft partition through which every token exchanges information with every other at linear cost. It changes what the network receives and how it sums. Figure 2 follows one block from boundary cells to outputs on an airfoil, and Figure 3 shows the slices of a trained model on a car (pseudocode in Section B).
Figure 2. One block of ISLA, piece by piece, on an airfoil with 96 cells and six slices (hand-made scores). (a) A token is a cell with a centroid, a normal, and a measure; lengths are in units of . (b) The routing weights , a softmax over tokens with the measure ( is the same with the refined scores ): each short leading-edge cell gets a small weight; colors show each cell’s highest-scoring slice. (c) Each slice’s anchor (star and arrow) and the six scalars () that relate one token to one anchor. (d) The slice states of one channel (squares), the read-back weights , a softmax over slices, and the read-back (black). (e) A vector output in two poses, with the same coefficients. (f) A query point’s distance and direction to the boundary, and the BVH clusters and tokens of its boundary read at .
Figure 3. The slices of a trained ISLA on one DrivAerML case. (a) Each surface token colored by its dominant slice, the slice with the largest read-back weight , in layers 3, 6, and 12, with the number of dominant slices; pale tints only separate slices and are unrelated between layers; the three saturated colors of layer 6 are the slices of (b). (b) The three slices of layer 6 dominant at the most tokens, with their area and token shares (finer mesh there): from the side and below, with routing anchor (star) and mean normal (arrow). On this nearly symmetric car, each slice’s weights are nearly left–right symmetric as well.
Tokens with a Measure and a Scale
Inputs. A case is one mesh per boundary condition (an inlet, an outlet, a no-slip wall). A token is a cell with its centroid , its unit normal , its measure (area for 2D boundaries, or length for 1D boundaries), and any data
that its condition prescribes. The global data are unit directions and scalars . A surface task asks for fields on the cells of a boundary, and an interior task for fields at query points in the domain (§ 3.4). A token’s first state is computed from invariants only (the cosines , norms and inner products of its condition’s data, and ) by a per-condition layer; all later layers are shared (Figure 2a).
An intrinsic unit of length. Coordinates enter only after division by a length that the boundary itself defines, the root-mean-square distance between two points of the surface,
evaluated exactly for the meshed surface in operations. needs no origin, is unchanged when a cell is split, and scales with the geometry. A problem may declare which of its data define other scales, such as a speed from the inlet and the fluid’s density. Every dimensional input is divided by its scales and every output multiplied by its own (a pressure by the density times ). By the Buckingham theorem [25, 11], physical laws relate dimensionless groups alone; feeding only dimensionless numbers lets the network apply arbitrary nonlinear maps without breaking unit invariance. We write .
Measure weights. Each token has a weight , the surface measure that it represents: when every cell is a token. When the boundary has more cells than a budget of tokens (a car has ; we use ), a cell kept with probability has the Horvitz–Thompson weight [43]. On interior tasks the model draws the sample itself, with , systematically along a space-filling curve [61, 83]. On surface tasks the data pipeline draws cells uniformly. Each condition is normalized by its own measure and given a learned log share , so that a far-field boundary’s share of every sum does not grow with its truncation radius. With ( with one declared condition), every softmax over tokens is
that is, the measure enters as a bias on the score [17, 95]. A sum is then a Nadaraya–Watson ratio of two quadratures [64, 90], of the same two surface integrals for every mesh. A short cell near a leading edge stands for little surface, so it carries little weight (Figure 2b), whatever the density of the mesher.
The Invariant Slice Block
A block updates the token states in four steps (Figure 2b–Figure 2d), with a layer normalization over the channels.
Route. Each of slices has a prototype , a learned vector plus a measure-weighted attention pooling of the case’s tokens (Section B.3). A token is scored by a scaled cosine similarity, , and routed by , a softmax over the tokens. Each slice spreads one unit of weight over the boundary, in proportion to the surface that each token represents, so a slice is a quadrature average over a soft subset.
Anchor. From the same weights each slice forms its anchor, a position and a facing direction,
Both are weighted means of vectors that move with the body, with weights that do not change, so the anchors move with the body and give each layer adaptive reference points (stars in Figure 2c).
Relate. With and its direction, scalars relate token to anchor ,
how far the anchor is, whether it lies along each global direction, whether the token and the anchor each lie in front of the other’s face, and whether the two faces agree. Each scalar is a norm or an inner product of vectors that rotate together, so it is unchanged by every rotation, reflection, and translation. These scalars are the only positional information that a boundary token receives (Figure 2c). A linear map of them refines the routing, .
Mix and read back. The refined scores give over tokens, which forms the slice states, and over slices (unweighted, since slices have no measure), which returns them to the tokens. With biases omitted,
followed by a token MLP; the last entry is the token’s mean relation to the slices it reads. The refined scores are thus normalized twice (as in Figure 2b and d): over tokens with the measure, so that a slice state is a quadrature average of its tokens, and over slices without it, so that each token reads a convex mixture of slice states. Transolver normalizes once, over slices, and pools by soft counts, so its slice states weight every point equally, whatever surface it represents. ISLA has no attention among slices; Transolver does not need its own [92] (Hu et al. 2026). A slice state reaches the rest of the body through the tokens that read it, at the next block’s routing. In a trained model the partition refines with depth (Figure 3a): a few slices share the car in a shallow layer and almost all of the in the last one, while single slices pick out regions such as the rear wheels and the floor between them (Figure 3b); consistent with reflection equivariance (Theorem 1), their weights are nearly left–right symmetric on this nearly symmetric car. The cost is per block: linear in the number of tokens, with no pairwise term.
Outputs of Any Tensor Rank
The states are invariant, so a vector or tensor output is assembled at the end, from invariant coefficients and the token’s own directions [98, 55, 87]:
plus isotropic terms, with and basis vectors built from the global directions , the normal , and the direction to the token from its soft anchor, a mixture of anchors under one more routing (Section B.5). A pressure is one coefficient, and a wall shear stress combines the basis vectors (Figure 2e).
Read-Only Interior Points
A query point in the domain has no normal and no measure (Figure 2f). The interior form of ISLA gives it two geometric quantities, computed inside the model from the full-resolution surface. The first is its signed distance to the boundary, from an exact closest-point search whose sign comes from the generalized winding number [45, 12]. The second is the direction in which that distance increases, which replaces the normal in every invariant.
With no measure , a query enters no sum over tokens. At each layer it performs the read-back of (7) with the boundary’s slice states, and it attends to every boundary token directly, as in a boundary-integral method:
for one head and without biases and layer normalizations (Algorithm 1), where and are keys and values of the boundary states and is the query vector of , scaled by for its channels. is a learned kernel of the pair’s invariants : a distance in units of , its logarithm, the logarithm of its ratio to , and cosines (Section B.7). A tree makes this boundary read affordable, as in GLOBE (Sharpe,
and panel-clustering and -body codes (Rokhlin, [72]; Barnes & Hut [13]; Hackbusch [38]): a distant cluster enters as one virtual token, with the measure-weighted means of its members, when its size is below times its distance and its normals agree, where is the tree’s opening angle. We use , about 680 sources per query in place of . At the read is exact.
Since queries contribute to no sum, the boundary side runs once per case, and a prediction at is a function of and the boundary alone, so large fields are evaluated in chunks.
Configuration. We use one configuration on every dataset: width , 12 blocks, and slices, with 66.3 M parameters (67.5 M for the interior form).
Guarantees
The three rules make ISLA presentation-independent (§ 2) at every valid parameter value and consistent under remeshing (proofs in Section A).
Theorem 1 (Exact invariances). In exact arithmetic, with every boundary cell a token and no cell’s weight below of the case’s total measure, the outputs of ISLA are (i) equivariant under rotations, reflections, and translations of the cells, their normals and data, the global directions, and the query points; (ii) multiplied by the factors of their units before target standardization, with unchanged internal states, under a rescaling of all lengths and of the units of the declared scales, at fixed data without declared units; (iii) unchanged by a permutation of the cells, by a common rescaling of the measures given with them, and by replacing a cell with coincident copies that share its measure; and (iv) at each query point, independent of the other query points; this clause needs neither hypothesis and holds for every and every fixed draw of tokens. For the interior form, (i) to (iii) hold with the exact boundary read () when the boundaries form a closed surface, at every query point that has a unique closest point on it.
The proof is an induction over the layers: a measure-weighted softmax of invariant scores has invariant weights that sum to one, so the anchors of Equation (4) are equivariant, the relations of Equation (5) are invariant, and so is the next state. Under these hypotheses, clause (i) holds for symmetric bodies as well. Clause (ii) matches Proposition 1(ii) when every dimensional constant is a declared scale or enters a global scalar; without a Reynolds number, ISLA predicts one dimensionless field at all sizes.
Consistency. For remeshing the statement is weaker, convergence in place of invariance (Proposition 2, Section A). Along meshes that refine a surface with cell normals that converge to its own, every measure-weighted sum in ISLA converges to a ratio of surface integrals. With every cell a token, , and no weight floor, the outputs converge to a mesh-independent limit; without the measure, the limit follows the mesher’s density.
Approximations. Large interior cases use two approximations outside the theorem: a draw of tokens from cells and three at , both ordered along a principal-axis curve that can tie on symmetric boundaries (Section A). In float32 a rigid motion moves the predicted velocity on DrivAerML by , below another draw’s (Section D.5).
Experiments
Setup. AeroZoo (Sharpe et al., [81]) contains 3,815 time-averaged RANS solutions, each with its own geometry, in eight enclosure configurations, at Reynolds numbers from to ; we use the 2,816 whose averages are converged. AirFRANS (Bonnet et al., [18]) contains RANS solutions of airfoil flows, and DrivAerML (Ashton et al., [8]), WindsoRML (Ashton et al., [6]), and AhmedML (Ashton et al., [7]) time-averaged scale-resolving simulations of flows around road vehicles; on these four we use the splits of FluidsBench (FluidisBench, [32]). We compare ISLA with Transolver (Wu et al., 2024) and GeoTransolver (Adams et al., [3]), and on AirFRANS and the automotive datasets also with GLOBE (Section C).
AeroZoo
AeroZoo varies every part of a problem’s presentation at once: geometry, size, speed, pose, and boundary-condition layout (Figure 4 and Table 1; medians in Table 8).
| Model | ||||||
| Free-stream reference | 0.440 | 0.442 | 1.809 | 0.389 | 1.000 | — |
| Zero predictor | 1.203 | 1.193 | 1.809 | 1.000 | 1.000 | — |
| Transolver, SI units | 1.202 | 1.193 | 1.810 | 0.999 | 1.009 | 1% |
| Transolver, scaled | 0.617 | 0.621 | 1.419 | 0.539 | 2.192 | 41% |
| GeoTransolver, SI units | 1.184 | 1.180 | 1.811 | 0.981 | 1.015 | 1% |
| GeoTransolver, scaled | 0.898 | 0.899 | 1.839 | 0.753 | 1.935 | 13% |
| GeoTransolver + BC, SI units | 0.475 | 0.479 | 1.809 | 0.401 | 1.002 | 20% |
| GeoTransolver + BC, scaled | 0.294 | 0.306 | 1.292 | 0.257 | 0.838 | 82% |
| ISLA | 0.082 | 0.114 | 0.298 | 0.112 | 0.315 | 98% |
| ISLA + boundary targets | 0.078 | 0.110 | 0.171 | 0.107 | 0.289 | 98% |
Table 1. Table 1. AeroZoo, full.: means over 515 test cases (bold: best per column). , (within of body), and : reference-scaled; , : relative ; “”: share of cases with below the reference’s; “BC”: boundary tokens; “scaled”: per-case scales; last row adds boundary targets (Table 8).
Figure 4. On AeroZoo, ISLA has the lowest velocity error in every enclosure configuration. Top: one median-error test case per configuration: reference velocity magnitude on a plane through the body and one ISLA model’s prediction (duct: around its contraction). Bottom: median by configuration on in-distribution test cases; ISLA’s is 1.3–7.8× below the best baseline’s.
Table 8. AeroZoo, full split (515 canonical test cases; lower is better). “BC”: with tokens that carry the boundary conditions; without them a baseline has no input for the inlet conditions. The two indented rows are the other settings of the row above them. “Med. ref.” is the median over cases of a model’s divided by the free-stream reference’s, and “Share ref.” is the share of cases on which the model’s is lower. Baseline rows are from Sharpe et al. [81] (one seed each); both ISLA rows average 4 seeds, and “ISLA + boundary targets” also supervises boundary targets in training (Section C.2). Bold: the best value in each column. Dash: no value.
| Model | Units | Parameters | Ref.-scaled median | Ref.-scaled mean | Ref.-scaled median | Ref.-scaled mean | Med. ref. | Share ref. |
| Free-stream reference | – | – | 0.110 | 0.440 | 0.059 | 1.809 | – | – |
| Zero predictor | – | – | 1.005 | 1.203 | 0.059 | 1.809 | – | – |
| Transolver | SI units | 9.0 M | 1.007 | 1.202 | 0.059 | 1.810 | 9.19 | 1% |
| Transolver | per-case scales | 9.0 M | 0.345 | 0.617 | 0.056 | 1.419 | 1.15 | 41% |
| GeoTransolver | SI units | 27.6 M | 1.004 | 1.184 | 0.062 | 1.811 | 8.88 | 1% |
| GeoTransolver | per-case scales | 27.6 M | 0.769 | 0.898 | 0.081 | 1.839 | 3.90 | 13% |
| GeoTransolver + BC | SI units | 27.6 M | 0.173 | 0.475 | 0.059 | 1.809 | 1.12 | 20% |
| GeoTransolver + BC | per-case scales | 27.6 M | 0.084 | 0.294 | 0.047 | 1.292 | 0.84 | 82% |
| float32, step schedule | per-case scales | 27.6 M | 0.096 | 0.277 | 0.047 | 1.160 | 0.84 | 80% |
| bfloat16, step schedule | per-case scales | 27.6 M | 0.101 | 0.253 | 0.050 | 0.987 | 0.86 | 76% |
| ISLA | computed from the boundary | 67.5 M | 0.053 | 0.082 | 0.023 | 0.298 | 0.50 | 98% |
| ISLA + boundary targets | computed from the boundary | 67.5 M | 0.052 | 0.078 | 0.021 | 0.171 | 0.47 | 98% |
Table 8.
First, Transolver and GeoTransolver as released have no input for the inlet conditions, and in SI units their velocity errors are within 2% of predicting zero ( and 0.981). Second, with tokens that carry the boundary conditions (BC), GeoTransolver in SI units still has a higher mean than the free-stream reference, which predicts the inlet state everywhere (0.475 and 0.440; ), on cases whose sizes span a factor of and speeds 200. With per-case scales as well, it reaches (, ) and has a lower error than the reference on 82% of the cases. Third, ISLA reaches (), (), and 98%: its mean and are at least 3.1 and 3.3 times lower than any baseline’s in any setting (Table 8). The baselines are single seeds from Sharpe et al. [81] (9.0 and 27.6 M parameters; ISLA averages 4 seeds), and size does not explain the margin: on the same step schedule, a 28.2 M ISLA with half the boundary tokens has mean 3% above a 67.5 M one (Table 12). Supervising boundary targets too (1.46× step time) lowers to 0.171 () and to 0.078 ().
| ISLA, step schedule | Parameters | Ref.-scaled median | Ref.-scaled mean | Ref.-scaled median | Ref.-scaled mean | Med. ref. | Share < ref. |
| 12 blocks, 128 slices | 67.5 M | 0.054 | 0.083 | 0.022 | 0.321 | 0.49 | 98% |
| 8 blocks, 64 slices | 28.2 M | 0.054 | 0.085 | 0.022 | 0.316 | 0.49 | 98% |
Table 12. An ISLA of 28.2 M parameters against one of 67.5 M on AeroZoo’s full split (515 canonical test cases), both trained on the same step schedule, 2 seeds each. The smaller model also uses half the MLP width and half the boundary tokens per case. Columns as in Table 8.
Held-out families and layouts. With a geometry family held out of training (blobs, or CAD parts), ISLA’s median on cases with a body is 0.059 and 0.079 (reference: 0.099 and 0.138). With Reynolds numbers above held out it is 0.067, close to the reference’s 0.076. With the crossflow or dual-inlet layout held out, it is only 31% and 11% below the reference’s, and with non-box enclosures held out it exceeds the reference’s (Table 11): presentation independence does not extrapolate to unseen topologies.
Table 11. ISLA on AeroZoo: the full split and six splits that hold a region out of training, with the free-stream reference on the same canonical test cases (mean of 4 seeds on full, 2 on the other splits). “Med. ref.” and “Share ref.” are as in Table 8. The two splits that hold out a geometry family are scored on cases with a body.
| Split | Held out of training | Cases train | Cases test | ISLA Ref. median | ISLA Ref. mean | ISLA Ref. median | ISLA Ref. mean | ISLA Med. ref. | ISLA Share ref. | Free-stream reference Ref. median | Free-stream reference Ref. mean | Free-stream reference Ref. median | Free-stream reference Ref. mean |
full | nothing | 2,011 | 515 | 0.053 | 0.082 | 0.023 | 0.298 | 0.50 | 98% | 0.110 | 0.440 | 0.059 | 1.809 |
bank_blobs | blob bodies | 1,662 | 872 | 0.059 | 0.085 | 0.024 | 0.294 | 0.61 | 99% | 0.099 | 0.409 | 0.060 | 1.655 |
bank_zero_to_cad | CAD-part bodies | 2,250 | 224 | 0.079 | 0.117 | 0.041 | 0.362 | 0.59 | 99% | 0.138 | 0.466 | 0.083 | 2.004 |
reynolds | 1,577 | 1,001 | 0.067 | 0.125 | 0.030 | 0.576 | 0.75 | 82% | 0.076 | 0.379 | 0.046 | 1.640 | |
family_non_box | non-box enclosures | 1,850 | 692 | 0.103 | 0.257 | 0.134 | 0.747 | 1.37 | 23% | 0.075 | 0.238 | 0.039 | 0.595 |
profile_crossflow | crossflow layout | 2,298 | 173 | 0.614 | 0.656 | 2.341 | 2.906 | 0.72 | 97% | 0.886 | 0.947 | 2.624 | 3.490 |
profile_dual_inlet | dual-inlet layout | 2,044 | 459 | 1.397 | 1.450 | 6.780 | 8.379 | 0.88 | 97% | 1.563 | 1.659 | 7.150 | 9.134 |
Table 11.
Pose and Discretization Tests
We give each trained model the same evaluation problems in another presentation and measure its error again, on 8 tasks (Figure 8 in Section D): AirfRANS, DrivAerML, HiLiftAeroML (Ashton et al., [9]), SHIFT-SUV (Lumi-Table 2).
Figure 8. Changing how a problem is presented raises the baselines’ errors and leaves ISLA’s unchanged. Each model’s error on the evaluation cases as stored (open markers) and on the same cases rotated and translated (top) or re-sampled by area (bottom); means over 2 to 4 seeds (Section C.3). The pipelines of the automotive, wing, and DeepJEB++ datasets center each case, so there the top panel tests rotation alone.
nary Cloud, 2025), and three solid-mechanics datasets (Section C). In the pose test, every case is rotated at random and translated by one radius of the body, together with its global directions and vector targets. In the re-sampling test, the surface cells that a model receives are drawn with probability proportional to their area, in place of the mesher’s refinement pattern, and errors are weighted by measure for every model. The models of Figure 8 were trained without augmentation, so the tests isolate invariance.
A rotated case lies outside everything that Transolver and GeoTransolver were trained on, and their errors rise to between 0.8 and 2.0 times that of predicting zero. Re-sampling by area changes only the density of the points that they receive, yet it raises their surface-pressure errors on DrivAerML from 4% to 30 and 14%, and on HiLiftAeroML and SHIFT-SUV to 5 to 18%. ISLA’s error changes by at most 1.1% of its value in every test. On DrivAerML’s 34 validation cases, rotating the full PDE problem multiplies the surface-pressure errors of tuned Transolver and GeoTransolver networks by 26 to 31 and changes ISLA’s by at most 0.2% (Table 21). Moving each case into the vehicle’s principal-axis frame at inference still multiplies the baselines’ errors by 5.8 and 5.4, or by 1.09 and 1.18 after a fixed rotation fitted on the training cases. Augmentation covers only its trained subgroup: vertical-axis augmentation removes the vertical-axis rise (1.001×) while leaving SO(3) at 23× (Table 21), whereas uniform SO(3) augmentation holds rotation changes within 0.2% and multiplies stored-case errors by 1.04 and 1.03 (Table 22).
Table rows:
| Presentation | Transolver | Transolver | GeoTransolver | GeoTransolver | GLOBE | GLOBE | ISLA | ISLA |
| As stored | 4.05 | 6.20 | 3.92 | 5.93 | 8.87 | 9.67 | 3.91 | 6.40 |
| Rotated about the vertical axis | 105 | 112 | 104 | 125 | 11.0 | 11.5 | 3.91 | 6.41 |
| Rotated on SO(3) | 110 | 115 | 122 | 130 | 11.1 | 11.8 | 3.91 | 6.42 |
| in principal-axis frame | 23.3 | 31.5 | 21.2 | 28.8 | – | – | – | – |
| in aligned principal-axis frame | 4.40 | 6.91 | 4.61 | 7.08 | – | – | – | – |
| As stored, on the sample | 3.91 | 6.15 | 3.78 | 5.88 | 8.77 | 9.64 | 3.78 | 6.36 |
| Re-sampled by area, on the sample | 29.9 | 39.2 | 33.8 | 40.0 | 9.35 | 10.6 | 3.77* | 6.35* |
| Retrained with vertical-axis rotation and translation (1 seed) | ||||||||
| As stored | 4.59 | 6.67 | 4.82 | 6.84 | – | – | – | – |
| Rotated about the vertical axis | 4.59 | 6.66 | 4.83 | 6.84 | – | – | – | – |
| Rotated on SO(3) | 107 | 98.1 | 113 | 100 | – | – | – | – |
| Re-sampled by area, on the sample | 31.4 | 41.4 | 32.7 | 42.8 | – | – | – | – |
Table 21. Pose and discretization tests on DrivAerML’s validation cases: relative error in percent of one network of each family’s DrivAerML full entry, averaged over the 34 validation cases of that split (: surface pressure; : wall shear stress). Top block: no network was trained with rotations or translations. In each rotated row each case has one pose, the same for every model; every pipeline centers the case, so the translation reaches no model. The first five rows are scored on every cell of the stored mesh (the two indented rows apply the vehicle’s principal-axis canonical frame at inference, without and with a fixed alignment rotation fitted on the training cases), and the last two on one sample of about cells per case drawn in proportion to area, with Horvitz–Thompson weights, the same cells for every model. *: ISLA’s network here is its interior form, a joint surface and volume network, which draws its own tokens in proportion to area (Section B.2), so for it the last row is a second draw of the same law and not a change of law. The networks of Transolver, GeoTransolver, and GLOBE are at the checkpoints of their entries, chosen on these cases; ISLA’s is its final checkpoint.
| Model | Seeds | Surface pressure (rel. ) ↓ Stored [%] | Surface pressure (rel. ) ↓ Rotated [%] | Surface pressure (rel. ) ↓ Rot. ÷ st. | Wall shear stress (rel. ) ↓ Stored [%] | Wall shear stress (rel. ) ↓ Rotated [%] | Wall shear stress (rel. ) ↓ Rot. ÷ st. |
| Transolver | 4 | 4.13 | 118 | 29 | 6.67 | 141 | 21 |
| Transolver, trained on rotations | 2 | 4.32 | 4.32 | 1.001 | 6.99 | 6.99 | 1.001 |
| GeoTransolver | 4 | 4.00 | 123 | 31 | 6.55 | 133 | 20 |
| GeoTransolver, trained on rotations | 2 | 4.13 | 4.14 | 1.002 | 6.75 | 6.74 | 0.999 |
| ISLA | 4 | 4.02 | 4.02 | 1.000 | 6.48 | 6.48 | 1.000 |
Table 22. Rotation augmentation on the DrivAerML surface task of the pose and discretization tests: relative error in percent on the evaluation cases as stored and rotated, and the ratio of the two (geometric mean over seeds). The two augmented models are trained on rotations drawn uniformly over all orientations, the law of the rotated test; the other models are trained on cases as stored.
AirfRANS
Each family’s entry is an ensemble on FluidsBench’s splits (FluidsBench, [32]; Section C). GLOBE (learned boundary kernels) has the lowest error in every field (Table 2); ISLA’s are 1.0–2.0× GLOBE’s and at least 1.6× below Transolver’s and GeoTransolver’s. On aoa_extrapolation (test angles outside training), every family’s errors grow by 4–15× from validation to test, and ISLA’s stay at least 2.4× below Transolver’s and GeoTransolver’s.
| Field | Transolver | GeoTransolver | GLOBE | ISLA |
| Relative error [%] | ||||
| full (720 training cases), test cases | ||||
| 1.17 | 1.20 | 0.34 | 0.55 | |
| 1.76 | 1.46 | 0.55 | 0.57 | |
| 0.53 | 0.47 | 0.15 | 0.30 | |
| 1.14 | 1.09 | 0.28 | 0.57 | |
| aoa_extrapolation (724 training cases), test cases | ||||
| 8.31 | 9.29 | 2.14 | 2.58 | |
| 6.44 | 6.60 | 2.14 | 2.67 | |
| 3.84 | 3.91 | 0.98 | 1.37 | |
| 12.21 | 13.02 | 3.87 | 4.55 |
Table 2. AirfRANS test cases: relative error in percent (, : surface pressure and wall shear; , : volume velocity and pressure; bold: lowest in row; Section C).
Automotive Aerodynamics Datasets
We trained and tuned all four model families on the FluidsBench splits of DrivAerML, WindsorML, and AhmedML (FluidsBench, [32]; 3–20 networks per entry; Section C). Because each dataset sweeps one vehicle family in a fixed pose and boundary layout, full-split errors are already near saturation for existing architectures (Sharpe et al., [81]); we report them here for direct comparison with prior work. On the full splits (validation cases on DrivAerML and WindsorML), ISLA’s surface errors lie 12% below to 5% above the lowest other family’s, and its volume errors 7% below to 28% above (Table 14). With 67 training cases (DrivAerML, AhmedML), errors lie 4% below to 13% above; with DrivAerML’s 11 training cases (validation errors; 50,000 steps per network for Transolver, GeoTransolver, and ISLA), ISLA’s errors are 8–13% below the lowest other family’s in all four fields (schedule check in Section D.3).
| Dataset | Split (train) | Cases | Rel. error: ISLA ÷ lowest other ↓ | ||||
| DrivAerML | full (400) | val. | 1.02 | 1.05 | 1.09 | 1.05 | |
| scarce (67) | val. | 1.03 | 1.04 | 1.13 | 1.12 | ||
| super_scarce (11) | val. | 0.87 | 0.92 | 0.91 | 0.91 | ||
| WindsorML | full (280) | val. | 0.94 | 0.99 | 0.95 | 0.93 | |
| super_scarce (8) | val. | 1.19 | 1.24 | 0.97 | 1.06 | ||
| AhmedML | full (400) | test | 0.88 | 0.97 | 1.28 | 1.00 | |
| scarce (67) | test | 1.06 | 1.06 | 1.05 | 0.96 | ||
| super_scarce (11) | test | 1.00 | 1.05 | 0.93 | 1.12 |
Table 14. Automotive aerodynamics datasets: ISLA’s relative error divided by the lowest of Transolver’s, GeoTransolver’s, and GLOBE’s, for each field ( and : surface pressure and wall shear stress; u and : velocity and pressure in the volume), on test cases, or on validation cases (val.) where not all four families have test results. Bold, below 1: ISLA’s error is the lowest. Every error is in Table 15 to Table 17. On DrivAerML, Transolver (all splits) and GeoTransolver (all splits) use lineups retrained to at least 50,000 steps per network (matching ISLA’s 50,000-step lineups), while GLOBE shows its earlier lineups; on WindsorML, Transolver (all splits), GeoTransolver (all splits), and ISLA (full) use lineups retrained to at least 50,000 steps per network, while the other cells show the earlier lineups.
Ablations and Cost
We remove one rule at a time and retrain (Table 26 in Section D.7; the DrivAerML surface task unless stated). The first two variants keep their accuracy on cases as stored and lose the one invariance that their rule provides. With counts in place of measures, the surface-pressure error on cells re-sampled by area is 11 times higher. With coordinates as inputs, it is about 4% lower on stored cases and 21 times higher on rotated cases. Without anchor relations it is 71% in every presentation; on AirfRANS, dropping the boundary read raises velocity error 10% and lowers eddy-viscosity error 8%, while training at (2.7× step time) matches within 0.9% (2 seeds).
Table 26. Ablations. Error of a variant divided by the error of ISLA on cases as stored, for cases as stored, rotated, and re-sampled by area (seed means; DrivAerML surface pressure, and AirfRANS velocity for the last three rows). On DrivAerML the reference is the mean of 4 seeds of ISLA. Dash: no value.
| Variant | Rel. error ÷ ISLA (stored) | ||
| ISLA | 1.00 | 1.00 | 1.00 |
| Counts in place of measures | 0.99 | 0.99 | 11 |
| Coordinates as inputs | 0.96 | 21 | 0.97 |
| No relations to anchors | 18 | 18 | 18 |
| Body alone as the boundary | 0.98 | 0.98 | 0.98 |
| ISLA (AirfRANS) | 1.00 | 1.00 | – |
| No boundary read | 1.10 | 1.10 | – |
| Exact boundary read () | 1.01 | 1.01 | – |
Table 26.
ISLA’s cost is linear in the boundary tokens and the query points (Section D.6). One float32 prediction for surface cells of a car takes 0.37 s on one GPU, between the 9 M GeoTransolver’s and Transolver’s.
Related Work
Slice transformers route mesh points into learned slices (Wu et al., 2024; Luo et al., [60]; Zhou et al., [99]; Adams et al., [3]), while anchor models compress them into a subset (Alkin et al., [4, 5]); AB-UPT and SMART also decouple read-only queries (Alkin et al., [5]; Hagnberger & Niepert, [39]) and Wen & Mishra [92] find slice means stable across same-law subsamples. Where layerwise equivariant networks constrain intermediate features (Thomas et al., [86]; Geiger & Smidt, [35]; Liao & Smidt, [53]; Brehmer et al., [22]; Rygiel et al., [73]), ISLA maps invariant scalars to covariant outputs (Villar et al., [87]; Ling et al., [55]) with anchors like Invariant Slot Attention (Biza et al., [16]). Its log-measure bias adapts proportional attention (Bolya et al., [17]; Yang et al., [95]) per condition with Horvitz–Thompson draws, and its boundary read builds on panel and Green’s-operator methods (Hess & Smith, [41]; Schäfer & Owczarek, [78]; Boullé & Townsend, [19]; Sharpe, [80]) (Section F).
Limitations and Conclusion
Limitations. Presentation independence covers pose, units, and meshing rather than unseen flow topologies: on boundary layouts absent from training, ISLA’s error is near or above the free-stream reference’s (§ 5.1). As a deterministic equivariant map, ISLA cannot represent a single symmetry-breaking mode such as an instantaneous bistable wake (Grandemange et al., [36]), and its length unit (Equation 2) scales with the outer boundary when a far field is truncated (59.5 m with DrivAerML’s tunnel vs. 2 m on the car alone). Three-dimensional interior training uses more than 80 GiB per GPU (Section C.6), and volume-velocity error is 9–13% above tuned GeoTransolver’s on DrivAerML (≥ 67 cases) and 27–30% on full AhmedML.
Conclusion. ISLA has, by construction, the invariances of a boundary-integral model: changing a problem’s presentation moves predictions only by rounding, quadrature, and interior token sampling and tree clustering. It has the lowest errors on AeroZoo and is second only to GLOBE on AirfRANS; on full automotive training sets its surface errors lie 12% below to 5% above the lowest other family’s, and on DrivAerML’s 11-case split 8–13% below them.
Proofs
Throughout, a rigid motion with acts on a posed problem as follows. It moves every point, . It rotates every vector datum, , and every rank-2 datum, , and it leaves scalars unchanged. A pseudotensor takes the further factor . Under a reflection () the orientation of the boundary is carried along, so that a cell’s normal maps as a true vector, . A cell’s normal follows its vertex order, so reflecting the coordinates alone would give , which exchanges the two sides of the boundary and poses a different problem.
What the Solution Operator Cannot Depend On
Assumption 1. (a) Well-posedness: the problems of interest form a class that is closed under rigid motions, reflections, and changes of units, and each has exactly one solution. (b) Frame indifference: and the commute with every rigid motion and reflection that acts on , the , and by tensor type, and every vector, tensor, or dimensional constant that they involve is part of . (c) Dimensional homogeneity: and the hold in every system of units.
Proposition 1 (What the solution operator cannot depend on). Under Assumption 1, for every posed problem :
(i) Euclidean covariance. For and a translation acting on the oriented boundary, its data, and the tensors in by tensor type, for a rank- output (times if a pseudotensor).
(ii) Unit covariance. Rewriting in other units rescales each output by the factor that its dimensions dictate.
(iii) Independence of the discretization and of the query set. depends on as an oriented surface, and not on how is meshed or sampled, on the order of its cells, or on the other query points.
Proof of Proposition 1. (i) Let , let be the moved problem, with global data , and define on the moved domain, componentwise for each field of rank (with the factor for a pseudotensor). By frame indifference, and each commute with the action of : is the rotated image of , and is the rotated image of , which is the datum of on the moved piece . Hence solves , which is again a problem of the class. By well-posedness it is the only solution, so . This is the statement that a symmetry of an equation maps solutions to solutions (Olver, [65]).
(ii) A change of units multiplies every quantity by , where rescales the -th base dimension and is the exponent of that dimension in . By dimensional homogeneity, if solves then the rescaled field solves the rescaled problem, and by well-posedness it is that problem’s solution. The theorem (Buckingham, [25]) follows: choosing the to normalize a maximal set of dimensionally independent data of shows that the dimensionless solution is a function of the remaining dimensionless groups.
(iii) is a function of , and of and . A mesh of , a sample of that mesh, an ordering of its cells, and a list of other points are not part of , so the solution cannot depend on them.
What well-posedness excludes. Part (a) of Assumption 1 fails where a symmetric problem has several stable solutions, such as the bistable wake of a bluff body (Grandemange et al., [36]). There the solution operator is set-valued, and clause (i) of Proposition 1 states only that a rigid motion maps the solution set of onto that of the moved problem.
Proof of Theorem 1
Full statement. Consider ISLA in exact arithmetic, with any parameters for which it is defined, when every boundary cell is a token and no weight is below its floor.
(i) Euclidean equivariance. Let and act on the cells, their normals and data, the global directions, and the query points, with the orientation of the cells carried along under a reflection, as above. Every output of rank is mapped to times itself at the moved points (times for an output declared with odd parity).
(ii) Similarity. Multiply every coordinate by and every datum with declared units , on the boundary or global, by , with for each declared scale (a global value that defines is multiplied by ), and leave every datum without declared units as it is. Then the internal states are unchanged, and each output, before any standardization, is multiplied by the factor of its declared units. (iii) Measure. The outputs are unchanged by a permutation of the cells, by a common rescaling of the measures given with the cells, and by replacing a cell with coincident copies that share its measure.
(iv) Query independence. The prediction at a query point is a function of that point, the boundary, and the global data only. This clause needs neither of the two hypotheses above: it holds for every opening angle and, with a token budget, for every fixed draw.
For the interior form, (i) to (iii) hold with the exact boundary read () when the boundaries form a closed surface, at every query point that has a unique closest point on it.
Write for the transformations of clauses (i) to (iii): rigid motions, uniform rescalings of length (under which measures scale by ), changes of units of the declared scales, a common rescaling of the measures given with the cells, and permutations of the cells. A quantity attached to a token is invariant if it is unchanged by every element of other than a permutation and is permuted with the cells by a permutation.
Hypotheses. The model is defined when no normalization divides by zero, for which a positive gain in every prototype map (Section B.3) suffices. Each weight has a floor of times the case’s total raw measure over all conditions, applied before the per-condition normalization, which guards the logarithm, and the theorem assumes that no weight is below it and that every cell remains a token, before and after a cell is replaced by copies. The other floors keep every normalization and logarithm defined, and a clamp on a norm commutes with . Boundary data are true tensors of rank at most two.
Lemma 1. Let be invariant scores and let the weights be invariant up to a common positive factor. Then is invariant and . Consequently is invariant for invariant ; it maps to for vectors ; and it maps to for points .
Proof. The common factor cancels in the ratio, so is invariant, and the sum to one by construction. A sum of invariant terms is then invariant. The claim for vectors holds because is linear, and the claim for points uses to take the translation out of the sum.
Inputs. The length unit of Equation (2) is a function of pairwise distances and of ratios of measures. It is therefore unchanged by rigid motions, permutations, and a common rescaling of the measures, and it is multiplied by under . Hence maps to under a rigid motion and is unchanged by a rescaling of length. The normalized weight is a ratio of measures of the same kind, hence invariant; with a single declared condition the model uses , which is invariant up to a common factor. A declared scale is a measure-weighted root-mean-square magnitude of data, or a declared global value. A change of its unit multiplies it and every datum that declares it by the same factor, so dividing each datum by its scales gives numbers that do not change with the units. The mean direction of a field that defines a scale, which the model adds to the global directions, is a normalized sum , so it maps to times itself and is unchanged by the other elements of . The invariant input of a token, the vector from which its first state is computed (line 7 of Algorithm 1), consists of inner products of unit vectors that rotate together (, ), of norms and inner products of its dimensionless data with those vectors, and of dimensionless scalars. All are invariant, including under reflections, because only true vectors enter. The first state, a per-condition map of the invariant input followed by a shared one, is therefore invariant.
Induction over the blocks. Suppose the states entering a block are invariant. The prototypes are sums of the form of Lemma 1 over invariant quantities, so they and the scores are invariant. By the lemma the routing weights are invariant and sum to one, so the anchors satisfy and . Then : the translation cancels exactly, with no centering in any input of a learned map. Every component of is the norm of , a function of that norm, or an inner product of two of the vectors , , , (or a contraction of a boundary vector or tensor with ), all of which rotate together. So is invariant, and with , , , , and the updated state. A permutation of the cells permutes per-token quantities and leaves sums over tokens unchanged. By induction the final states are invariant.
Outputs. The coefficients are an affine map of the normalized final state, hence invariant. The head’s anchors (Section B.5) are of the form of Equation (4), and a token’s soft anchor is with invariant weights that sum to one over the slices, so it maps to . The translation cancels in the difference between the token and its soft anchor, so , and every basis vector, being one of , , or a combination of two of them with invariant coefficients, maps as . The output of Equation (8) therefore maps to times itself; the isotropic terms (the identity and its products) are fixed by . An odd-parity output replaces exactly one factor by a cross product of two basis directions (in two dimensions, a perpendicular), which maps as , giving the extra sign; an odd output of rank also has the Levi-Civita tensor, which multiplies by . Multiplying each output by the scales of its units gives clause (ii). This proves (i) and (ii), and the permutation and rescaling parts of (iii).
Coincident copies. If a cell is replaced by copies with the same vertices and data and measure each, the copies have equal invariant inputs and, by induction, equal states in every layer, while every sum is unchanged. The length unit is unchanged as well. Every other cell therefore keeps its output, and each copy has the output of the cell that it replaces, which proves the last part of (iii).
Queries. A query has zero weight, so it contributes to no sum over tokens: the boundary-side states, prototypes, anchors, and slice states are those of the boundary alone. Each operation on a query (its embedding, its read-back from the slices, its boundary read, its token MLP, the head) uses only the query’s own state and geometry and boundary-side quantities. In particular the list of sources of a query in the tree read is a function of the tree and of that query’s position. This proves (iv) for every , and, with a token budget, for every fixed start of the draw, as in evaluation.
The interior form with the exact boundary read. The signed distance of a query is invariant under rigid motions and scales with length; where the closest boundary point is unique, the direction is the unit gradient of the signed distance and maps as . The sign of comes from a winding number, which is exact for a closed surface. (We take the winding number as evaluated exactly; in three dimensions the implementation uses a far-field expansion with a measured error of at most 0.03, where the margin is .) The query’s invariant input and its relations to the anchors are then invariant as above. The pair invariants are norms and inner products of , , , and , functions of those norms and of , and contractions of the boundary vectors with and , and the exact boundary read of Equation (9) is a sum of the form of Lemma 1 over all boundary tokens, so the query states remain invariant. The set of points with more than one closest boundary point (the medial set) has measure zero; ISLA uses the least-norm element of the convex hull of the tied directions, which is again equivariant wherever the tie is exact.
The approximations. Large interior cases use two approximations that the theorem does not cover: a draw of tokens and a tree. Both depend on a frame. With a token budget below the number of cells, tokens are drawn along a Morton curve laid out in the frame of the boundary’s principal axes (signed by third moments), and with the boundary tokens are grouped by a tree built in the same kind of frame. The frame moves with the boundary, and both the curve and the tree are laid out in the boundary’s own units. The draw and the tree are therefore unchanged by rigid motions and rescalings whenever the principal variances are distinct, no third moment along a principal axis vanishes, and no cell lies on a threshold of a discrete decision. Where the frame is degenerate, as for a boundary with equal principal moments or a symmetry that reverses a principal axis (a mirror plane, two-fold rotation axis, or center of symmetry), a rigid motion can regroup cells; the prediction then changes by the difference between two quadratures of the same surface, or between two tree approximations of the same sum (and on an open boundary, whose winding number is not integer-valued, the BVH far-field approximation of the winding number can also flip the sign of near the threshold ).
The tree adds an approximation of its own. A cluster replaces over its members by at the cluster’s mean position, normal, key, and value, which is exact when the score is constant over the cluster. The score depends on learned states as well as on geometry, so limits how much the geometry varies over a cluster and does not limit how much the learned states vary. A model should be evaluated at the angle it was trained at.
Consistency under Remeshing and Re-sampling
A new mesh of the same surface is not one of the transformations of Theorem 1 (Figure 5), so the statement for remeshing is one of convergence, along sequences of meshes of the following kind. Call a sequence of meshes of a compact, oriented, piecewise surface admissible if
Figure 5. Measure-weighted slice pooling converges under mesh refinement; count-normalized pooling does not. (a) A Joukowski airfoil in potential flow, discretized into boundary cells that are about nine times shorter at the leading edge, with six soft slices. Bars show the slice states of the exact pooled by soft counts (gray) and by cell measure (charcoal, ISLA) beside the exact ratio of surface integrals (crimson ticks). (b) Largest error across the six slice states as the same grading law is refined from to : count-normalized pooling stays near 0.12, its mesh-density-weighted limit, whereas ISLA’s measure-weighted pooling falls by five decades as .
(a) every cell has its vertices on the closure of one smooth piece of one condition, on which the data are continuous;
(b) the closest-point projection onto that piece is one-to-one on the cells and onto the piece, up to sets whose measure tends to zero;
(c) the largest cell diameter tends to zero; and
(d) , where is the unit normal of and is the point of the cell’s piece closest to the centroid .
Shape-regular triangulations that satisfy (a) to (c) and are oriented as satisfy (d) [30]; in two dimensions (d) follows from (a) to (c) and the orientation. Condition (d) is needed: inscribed triangulations of a cylinder can have diameters that tend to zero and areas that diverge.
Proposition 2 (Consistency under remeshing and re-sampling). Let be compact, oriented, and piecewise , and let and be bounded on and continuous on each smooth piece of each condition.
(i) Along an admissible sequence of meshes, , with on (and with one declared condition). Without the measure, if the empirical distribution of the centroids converges to a density, the same sum converges to a ratio of integrals against that density, which depends on the mesher.
(ii) Let a design keep cell with probability and weight . For scores and values that do not depend on the sample, the numerator and the denominator of the sampled sum are unbiased for their full-mesh values. This uses first-order inclusion probabilities only, so it holds for the systematic design of Section B.2 with a random start. Here is the measure of the full mesh, which the model computes before the draw.
(iii) Fix parameters for which the model is defined, with a positive gain in every prototype map, and take the model without the floor on the weights. Let be closed, let the data of cell be the values at of data that are continuous on each smooth piece of each condition, let every boundary cell be a token, and let the boundary read be exact (). Then there is a field on with along every admissible sequence, and the output at every fixed query point off that has a unique closest point on converges. The limits depend on , its data, the global data, and the parameters, and not on the sequence.
Proof. (i) Let be bounded on and continuous on each smooth piece of each condition. For an admissible sequence, and [30], so is a Riemann sum and converges to . Apply this on each condition to and to , whose integral is positive; the ratio converges because division is continuous away from zero. Without the measure, converges to whenever the empirical distribution of the points converges to a density , and is set by the mesher. (ii) Let indicate that cell is kept. Then for every fixed [43]; this holds for the numerator () and the denominator (). Their ratio is the Hájek estimator [40, 75], which is not unbiased. The hypothesis that the scores and values do not depend on the sample holds in the prototype attention of the first block; elsewhere they are functions of sampled sums. For the systematic design and every start , including the fixed start of evaluation, the error of each sum is bounded without any assumption of independence. With for in the interval of cell on the line on which the cells are laid, and the estimate is , so
where is the oscillation of over the -th unit interval. Within a condition equals the threshold of Section B.2 for every cell with , so on those cells , and its oscillation over a unit interval is times that of over the cells that the interval contains. The error of each sum, relative to , therefore vanishes when and vary little over those cells in all but a vanishing share of the intervals, as when the mesh and the budget are refined together. For every interval is one cell and the error is zero, but the bound does not tend to zero as on a fixed mesh. We claim no rate.
(iii) Every map in a block is continuous: each normalization and each logarithm has a positive floor on its argument, and the remaining operations are affine maps, GELU, exponentials, and ratios with positive denominators. Suppose that at the input of a block there is a bounded field , continuous on each smooth piece of each condition, with as . For the first block is the embedding of the invariant input, a continuous function of the normal field and the data, and condition (d) gives the convergence. By (i) and uniform continuity on bounded sets, every sum over tokens in the block converges to the corresponding ratio of surface integrals of , the anchors converge, and the updated states converge to the values of a new field of the same kind at the points . By induction the outputs at the tokens converge to the values of a field determined by , its data, the global data, and the parameters. The length unit of the mesh converges to that of by (i), and the signed distance and the direction of a fixed query point off the closed surface converge wherever its closest point is unique, so the output at the query converges too.
With a fixed token budget, or with , the outputs converge only up to the sampling error of the draw and the error of the tree, which Section D.5 measures. With the floor of of the total weight, the model equals that of clause (iii) on every mesh whose smallest weight exceeds the floor (fewer than about cells of comparable size), and not in the limit.
Order of convergence under a geometric split. If a flat cell with centroid is split into cells that tile it, then , so for a twice-differentiable summand the first-order terms cancel and : the token sums are second-order consistent. The length unit is exactly unchanged, because Equation (2) is evaluated with each cell’s own second moment (Section B). This is a statement about splitting the flat cells of one polyhedral surface; for two meshes of a curved surface, Proposition 2 gives convergence and no rate.
Architecture in Full
This appendix specifies the forward pass in full. Figure 6 maps the dataflow and tensor shapes across the boundary, slice, and interior-query paths, Algorithm 1 gives the computation step by step for boundary data of rank zero, the subsections define the pieces that the main text only names, and Section B.4 adds boundary vectors and tensors. Throughout, LN is a layer normalization over the channels of one token, is the cosine similarity of two vectors, is the measure-weighted softmax over tokens of Equation (3), and is an unweighted softmax over slices. Logarithms and normalizations have small positive floors, relative to the case’s own scales, which the equations omit.
Figure 6. The forward pass of ISLA, with the shape of every tensor. Left: the boundary is scaled once, tokens are drawn with Horvitz–Thompson weights, and initial states are embedded from invariants without coordinates. Center: each of 12 blocks passes states from the boundary tokens (blue) to the slices (amber) and back in four steps: route, anchor, relate, and mix and read back. Gray: the per-block states and token trees stored once per case. Bottom: interior query points (), an independent input, read the stored slices and, through the tree, the boundary tokens without contributing to any sum over tokens, so arbitrary query sets run in chunks. Right: the equivariant tensor-basis head.
The Length Unit and the Scales
For a mesh with cells (segments or triangles) of measure , centroid , and vertices , with , let and . The length unit of Equation (2), the root-mean-square distance between two points of the surface, is evaluated exactly for the piecewise-flat surface by
where is the second moment of cell about its own centroid per unit measure. The identity gives the form, and the term makes exactly unchanged when a cell is split into cells that tile it. is times the radius of gyration of the surface. On the surface tasks of this paper the cells given to the model are the data pipeline’s sample with Horvitz–Thompson measures, so there and the are estimates from that sample. The mean position appears inside this one scalar and in no input of a learned map.
The other scales come from the declarations of a problem’s fields. The declaration of a field, an input or an output, may state its units as a monomial in named scales, and may state that the field defines one of them. A scale defined by a boundary vector field, such as an inlet’s prescribed velocity, its measure-weighted root-mean-square magnitude over the cells that carry it, and the measure-weighted mean direction of that field becomes one of the global directions . A scale may also be given directly as a global value, such as a density. Scales are computed in float64 from the full-resolution boundary, before any token is drawn. Each input with units is divided by the monomial of its scales and each output is multiplied by its own. A Reynolds number, where a problem has one, is an ordinary global scalar formed from the problem’s own scales and viscosity; the model itself names no physical quantity. In this paper AeroZoo declares a velocity and a density scale, and AirfRANS a velocity scale. The automotive datasets hold one flow condition each and declare none: their data pipeline makes the targets dimensionless with the dataset’s free-stream constants, and only depends on the size of the geometry. A global vector contributes only its direction, because the outputs do not scale as any fixed power of its magnitude; a magnitude that matters enters as a scale or as a scalar.
Drawing Boundary Tokens
When an interior task’s boundary has more cells than the budget , tokens are drawn as follows.
Shares. The budget is divided among the conditions by water filling: equal shares, except that a condition with fewer cells than its share is kept whole and its unused share is passed to the others.
Inclusion probabilities. Within a condition with share , cell is kept with probability , where the
Algorithm 1 The forward pass of ISLA for one case. The index runs over boundary tokens, over conditions, over slices, over blocks, over query points, over the sources of a boundary read, over its heads, and over output tokens. Lines 25 to 39 and the steps marked “(interior)” run only for interior tasks.
Require: boundary cells grouped by condition ; global directions and scalars ; query points (interior task)
root-mean-square pairwise distance of the surface, Equation (11)
declared scales from the data; divide every datum by its scale
(interior) if the cells outnumber the budget : draw tokens, (Section B.2); else
with invariant input
for do
prototypes from (Section B.3)
(over tokens )
relational invariants of , Equation (5)
(over tokens )
(over slices )
(interior) store and the tokens’ keys and values
end for
exact signed distance and direction to the full surface (Section B.6)
sources tokens and accepted clusters of the tree (Section B.7)
for
for do
relational invariants of
(over slices )
for each head , with
end for
output tokens : the cells of the requested boundaries (surface task) or the queries (interior task)
direction from the token’s soft anchor (Section B.5)
, per declared field
multiply each output by the scales of its units
threshold makes .
Order
Within each condition, the cells are sorted along a Morton (Z-order) curve (Morton, 1966) of their centroids, and the conditions are laid one after another. The curve lies in the frame of the boundary’s measure-weighted principal axes, each axis signed so that the third moment along it is positive, and in units of the boundary’s own extent. The frame moves with the boundary, so in exact arithmetic the same cells are kept in every pose and at every scale, except where two principal variances are equal, a third moment vanishes (as for a mirror-symmetric boundary), or a centroid lies on a grid line of the curve. In float32, rounding moves cells across such lines: a rigid motion changes 9% of the drawn tokens on DrivAerML (Section D.5).
Systematic draw. The cells are laid end to end as intervals of length , and the cells that contain the points , , , . . . are kept [61], with the start uniform on in training and in evaluation, so that inference is deterministic.
Weights. A kept cell has the weight .
The number of tokens is exactly . In training every cell is kept with probability exactly , so every sum with independent of the draw is an unbiased estimate of the sum over all cells, and it is exact for a condition that is kept whole. Ordering along a space-filling curve spreads the sample over the surface [83]. Since large cells are kept with certainty and small cells in proportion to their area, the tokens of a finely meshed region do not crowd out the rest of the boundary.
A surface task uses every cell that it is given, with the measures that it is given. On the surface tasks of this paper the data pipeline draws 200,000 cells of the body uniformly, multiplies their measures by the inverse of the sampling fraction, and passes the tunnel’s boundaries whole.
Prototypes Adapted to the Case
Each block has learned base prototypes , , and adapts them to the case by one measure-weighted attention,
where is the summary of the case that the attention forms for slice , rms is the root mean square over all components of its argument, and is a learned gain, initialized to 0.1. Centering removes the part of the summaries that is common to all slices, and for the last factor keeps the root-mean-square size of the adaptation below a fraction of that of the base prototypes. The cosine-similarity score is bounded, , so the first routing cannot sharpen without limit as the weights grow. The refinement adds a learned linear function of the invariants, without a bias. It contains the distance and its logarithm, so it is not bounded.
Boundary Data of Rank One and Two
A condition may prescribe vectors on its cells, such as an inlet’s velocity, and a rank-2 tensor . With the frame vectors and , a token’s invariant input contains the invariants of these data: , , , , the trace of , and its Frobenius norm. Each condition also maps its vectors (and the vectors ) onto a set of vector channels shared by all conditions, and its tensors onto shared tensor channels, by learned scalar matrices initialized to the identity. A combination of vectors with scalar weights commutes with every rotation and reflection, so the channels and remain true vectors and tensors. They enter each block through the contractions and , which join the relational invariants of (5), and the boundary read through and , which join the pair invariants of Section B.7. Query points have zeros in these channels. On the DrivAerML tasks, which have no boundary data and one global direction, the relational invariants number six. The model accepts global scalars and unit directions only, so a problem whose global data hold a rank-2 material tensor or a pseudovector, such as a rotation rate, cannot be posed to it in full. The interior form accepts boundary data of rank zero and one only.
Output Basis
The basis of (8) contains the direction to each output token from a soft anchor of its own. To form it, one more routing map, with its own prototypes and without the geometric refinement, is applied to the final states; its prototypes are adapted to the case as in Section B.3, with the final states in place of :
The soft anchor of token is , and is the direction from it to the token. The anchors are formed by the boundary tokens alone. With global directions the true basis vectors are
in all, where is minus the part of that is perpendicular to . For a query, takes the place of . An even-parity output of rank is the contraction of coefficients with the products , plus the isotropic tensors of that rank (the identity at rank 2). A symmetric output uses one coefficient per unordered multi-index. An odd-parity output replaces exactly one factor by a pseudovector: in three dimensions a cross product of two of , , , and in two dimensions the perpendicular of one of them. An odd output of rank also has the Levi-Civita tensor. With one global direction in three dimensions a vector uses 5 coefficients, a symmetric rank-2 tensor 16, and a pseudovector 3.
The span of the true basis is that of , of dimension at most . With at least one global direction it is all of except where the three directions are coplanar. With none () in three dimensions, every even vector output lies in the plane of and . For general vector outputs such a problem needs a declared direction, or a boundary vector field that defines a scale, whose mean direction then becomes a global direction.
Exact Geometry of a Query Point
A query point receives two geometric quantities: its signed distance to the surface, and the direction in which that distance increases. The surface is the union of all cells of all given boundaries at full resolution, never the drawn tokens. For a query an exact point-to-simplex search (on a bounding-volume hierarchy in three dimensions) finds the closest point of the surface, and
with the sign positive on the side that the cells’ normals point to. The sign is taken from the generalized winding number of the oriented cells [45], evaluated with a hierarchical far-field expansion in three dimensions [12]. It is therefore defined at folds of the surface, where cells meet back to back and a sign taken from the nearest cell’s normal would depend on rounding. For a closed surface the sign is exact; behind an isolated open sheet it is approximate near the level at which the winding number changes sign. Where the closest point lies inside a cell, is that cell’s normal, including at a point on the surface itself. At a point with several closest features, such as the mid-plane of a thin plate, the gradient of the distance is not defined. There ISLA takes the least-norm point of the convex hull of the tied directions (the midpoint of two, zero where they cancel), which is a function of the tied features alone and not of the order of the mesh. Both and are functions of the surface: they are the same for every draw of the tokens and are unchanged when a cell is split. This is why ISLA takes no stored signed-distance field. The channel is logarithmic for distances above , so that the viscous sublayer and the far wake are both resolved, and are linear through the wall.
The Boundary Read and Its Tree
The boundary read of Equation (9) scores a query and a source by a learned kernel of the pair’s invariants plus an inner product of the query’s vector and the source’s key. For a source with position and normal , let . The pair invariants are
together with the contractions of any boundary vector data with and . A small MLP maps them once per forward pass to 16 basis functions , which all layers share. Each layer and each of its 8 heads has its own kernel, the linear combination , and its own keys and values of the boundary tokens’ states (4 key and 8 value channels per head). The output projection of each layer is initialized to zero.
The tokens of each condition are first grouped by the face of a cube that their normal points to in the boundary’s principal frame. Each group forms a binary radix tree over a Morton order of its tokens (Karras, [49]), with leaves of at most four tokens. A node stores the sum of its tokens’ weights and the measure-weighted means of their positions, normals (renormalized to unit length), keys, and values. A query traverses each tree from the root and accepts a node as one virtual token when both
that is, under the -matrix admissibility criterion (Hackbusch, [38]; Sauter & Schwab, [77]) together with normal coherence; otherwise it opens the node, down to individual tokens. Here is the diagonal of the box of the node’s centroids, widened by a bound on rounding so that no tie of the criterion is decided by rounding, and the distance from the query to that box. Within one tree all normals point to the same face of the cube, so in three dimensions the second condition always holds for , and at only the first is active. Each query’s list covers every token exactly once, as itself or inside one accepted node. Each query traverses the trees independently: queries never form clusters with one another. At a query of a DrivAerML case has about 680 sources for tokens. Only zeroth moments are stored, so the approximation is that of a monopole expansion whose “charges” are learned states. No gradient is propagated through the pair invariants, and a model is trained and evaluated at one . With randomly initialized weights in float64, the outputs at deviate from those of the exact boundary read by 1.4 to 3.5% on two synthetic geometries, a sphere and a random point cloud. Trained models depend on their angle more strongly: evaluated with the exact boundary read, AirfRANS models trained at have 2.0 to 5.1 times their error (Section D.7).
Sizes and Training
Table 3 lists the reference configuration and the training recipe of the pose and discretization tests; the optimizers are Muon (Jordan et al., [47]) and AdamW (Loshchilov & Hutter, [57]). Each block has 5,449,991 parameters, of which the token MLP and the slice MLP account for 77%. The interior form adds the query input layer, the kernel basis, and twelve boundary reads (1,214,336 parameters in all). Under bfloat16 autocast, the query geometry, the tree, and the anchors’ mean normals are computed in float32, and so is the head when scales or output statistics are declared; the anchor positions follow autocast.
Table 3. Top: the reference configuration of ISLA, used on every dataset in this paper. Bottom: the training recipe of the pose and discretization tests. The recipes of AeroZoo, AirfRANS, and the automotive datasets differ from it in the loss, the learning rate, and the length of training (Section C).
Cost. Per block, the boundary side costs about multiply-adds for tokens: the dense maps dominate, the slice terms are a tenth of the total, and there is no term in . A query costs per layer for its slice read and MLPs, plus 224 multiply-adds per source for the boundary read. The boundary side runs once per case, and the query side is linear in the number of queries with no interaction among them.
Experimental Setup
Datasets
Table 4. The datasets. “Cases” counts the cases with released fields (the dataset papers describe 500 for DrivAerML and 355 for WindsorML), and the cell counts are per case, from the released files. DrivAerML stores polygons: split into triangles for the interior form of ISLA, the surface of a car has about cells.
| Dataset | Cases | What varies | Simulation | Surface | Volume |
| AeroZoo ([81]) | 3,815 | shape, enclosure, size, speed, ( to ), orientation | unsteady RANS, transition SST | 0.43 M (median) | 5.0 M (median) |
| AirfRANS ([18]) | 1,000 | airfoil, angle of attack, | RANS (2-D) | about 1,000 | about 179,000 |
| DrivAerML ([8]) | 484 | car shape (16 parameters) | hybrid RANS-LES | 7.4 to 10.1 M | about 142 M |
| WindsorML ([6]) | 350 | Windsor-body shape (7 parameters) | wall-modeled LES | 2.0 to 2.6 M | 271 to 330 M |
| AhmedML ([7]) | 500 | Ahmed-body shape (6 parameters) | hybrid RANS-LES | 0.7 to 1.5 M | 20 to 23 M |
Table 4 lists the datasets. AeroZoo varies the shape (seven geometry families, no body used twice) and the enclosure, and with it the layout of the boundary conditions (eight configurations; 161 of the cases are ducts without a body). It also varies the size (0.02 to 20 m), the inlet speed (0.25 to 50 m/s), the Reynolds number, and the orientation of every case, which is random. Its inputs are the four boundary meshes of a case, by condition type (inlet, outlet, no-slip, slip), the values that each prescribes, and the fluid’s viscosity and density ([81]). AeroZoo calls a solved and released case a sample; we say “case” for every dataset and keep “sample” for a draw of cells. AirfRANS varies airfoil geometry, inlet direction, and speed in two dimensions, while the three road-vehicle datasets (DrivAerML, WindsorML, and AhmedML) hold one flow condition each, in one orientation, from one mesher, so their cases differ in shape only.
AeroZoo
Data. We use the interior task of AeroZoo release v1.2 (catalog SHA-256 c8e9508e8a9d8291...) ([81]), on its full split and on six of its splits that hold out a region (Table 11). Training, validation, and test use the canonical cases of a split, those whose time averages are converged (2,816 of the 3,815); on full these are 2,011 training, 290 validation, and 515 test cases. The flag that marks a case as canonical is computed from the solution and is not an input. As AeroZoo specifies, the two splits that hold out a geometry family are scored on test cases with a body.
Metrics. With cell volumes , reference speed , and reference dynamic pressure of a case,
over every interior cell ([81]). The reference scales and the cell volumes are used by the scorer only. Results are medians and means over the canonical test cases; the mean is dominated by crossflow and dual-inlet cases. The free-stream reference predicts the mean prescribed inlet state everywhere and calibrates the metrics case by case; the zero predictor predicts zero velocity and gauge pressure. We also report the median over cases of the ratio of a model’s to the reference’s, and the share of cases on which the model’s is lower.
ISLA on AeroZoo. The four conditions are declared by name: the inlet with its prescribed velocity and turbulence quantities, the outlet with its pressure and reverse-flow values, and the no-slip and slip walls without data. The inlet velocity defines the velocity scale and the one global direction, the density is a declared scale, and the length unit is that of Equation (2). The only global scalar is minus its mean over the training cases, so the length by which coordinates are scaled reaches the model only through a Reynolds number built on it. Other dimensionless inputs and outputs are divided by constants fitted on the training cases. ISLA predicts all seven fields of the task (velocity, pressure, and five turbulence quantities). Training uses 200,000 interior points and 200,000 boundary tokens per case, for 250 epochs at the reference recipe’s default learning rate () and schedule (5 epochs of linear warm-up and a cosine decay to ), with a per-case relative loss per field in which velocity has weight 3, four cases per step, and the final checkpoint. The same settings are used on all seven splits. The two models of Table 12 use a step schedule instead, with the learning rate divided by 10 after epochs 100 and 200. For scoring, the model predicts every interior cell in float32, with the one deterministic draw of boundary tokens that evaluation uses. The interior-only ISLA rows are means over 4 seeds on full and 2 seeds on the other six splits. The row “ISLA + boundary targets” (4 seeds, 67.5 M parameters: 67,493,495, adding three wall-shear output channels) is ISLA’s default AeroZoo training recipe: each step adds 200,000 boundary faces (queried off the face on the fluid side) and supervises boundary pressure, boundary velocity, and wall shear stress where no Dirichlet condition fixes them, at 1.46 times the training step time and unchanged interior inference.
Baselines on AeroZoo. The baseline rows of Table 8 are those of Sharpe et al. [81]: Transolver (9.0 M parameters) and GeoTransolver (27.6 M), both with 12 layers of width 256. Each appears without and with tokens that carry the boundary conditions, and with inputs in either SI units or per-case scales (a length, a speed, and a Reynolds number computed from each posed problem). As released, the two models take the coordinates of a point, its distance and direction to the no-slip faces, and the viscosity, so no inlet or outlet value reaches them. The boundary tokens are the faces of all four boundaries with their condition type and prescribed values. Each row is one seed, scored at its final checkpoint, on the same 515 test cases as ISLA. That paper trains every row in 3 settings (float32 with a cosine schedule, float32 with a step schedule, and bfloat16 with a step schedule) and prints the first for all rows, as Table 8 does. Over the settings, the best configuration’s median is 0.084 to 0.101 and its mean 0.253 to 0.294. With boundary tokens in SI units, the median is 0.173 in the printed setting and 0.98 to 0.99 in the other two. ISLA is larger (67.5 M parameters) and is the mean of 4 seeds.
Pose and Discretization Tests
Models. The tests of § 5.2 use single models, each trained on a dataset’s stored presentation without augmentation: ISLA with the configuration and recipe of Table 3; GeoTransolver with 12 layers of width 1,024 on the three surface flow tasks (143 M parameters, trained with 100,000 cells) and width 256 elsewhere (9 M parameters; 28 M with its local features on the DrivAerML volume task); and Transolver with 16 layers of width 512 (47 M parameters). Figure 1 and Table 7, Table 23 and Table 24 use the 9 M width-256 GeoTransolver on the surface task. The baselines use the same optimizer, batch, number of epochs, warm-up, and cosine schedule as ISLA (GeoTransolver uses a step schedule on the bracket lug and, in float32 with tuned field weights, on DrivAerML volume), a learning rate of , and bfloat16. They take the coordinates of the sampled cells or points, centered on the case, the normals, the unit free-stream direction, and for volume points the stored signed distance and its gradient. ISLA takes the named boundaries of the whole domain: on DrivAerML, HiLiftAeroML, and SHIFT-SUV it therefore receives the tunnel or far-field boundaries as well as the body, which the baselines do not. Each model has four seeds on the three surface flow tasks and on DrivAerML volume (and ISLA on Tensile2d), and two seeds elsewhere. All models are evaluated in float32 at their final checkpoint.
| Setting | Value |
| Reference architecture | |
| Width , blocks, slices | 512, 12, 128 |
| MLP expansion; activation | 4; GELU |
| Routing temperature | 10 |
| Boundary token budget (interior) | 200,000 |
| Boundary read | 8 heads, 16 kernel basis functions, opening angle , leaves of 4 |
| Parameters | 66,261,600 (surface form); 67,476,449 (interior; 67,490,930 on AeroZoo) |
| Training recipe | |
| Optimizer | Muon for matrices, AdamW for all other parameters; weight decay |
| Learning rate | ; warm-up over 5 epochs, then cosine decay to |
| Length | 500 epochs (AeroZoo 250, DeepJEB++ 40) |
| Batch | four cases per step, one on each of four GPUs |
| Loss | Huber () on standardized targets, measure-weighted on surface tasks |
| Points per case and step | 200,000 surface cells; or 200,000 tokens and 200,000 query points |
| Precision | bfloat16 autocast in training; float32 in evaluation |
| Checkpoint | the final one |
| Augmentation, gradient clipping, weight averaging | none |
Table 3.
| Task | Model | Step time [s] | Peak memory [GiB] | GPU-hours / run |
| AirfRANS | Transolver | 0.30 | 62 | 36 |
| AirfRANS | GeoTransolver | 0.21 | 39 | 27 |
| AirfRANS | ISLA | 0.32 | 25 | 38 |
| DrivAerML surface | Transolver | 0.27 | 62 | 22 |
| DrivAerML surface | GeoTransolver | 0.39 | 78 | 28 |
| DrivAerML surface | ISLA | 0.16 | 56 | 11 |
| DrivAerML volume | Transolver | 0.27 | 62 | 31 |
| DrivAerML volume | GeoTransolver | 0.98 | 118 | 67 |
| DrivAerML volume | ISLA | 1.06 | 129 | 71 |
Table 7. Training cost of single models (Section C.3; GeoTransolver at width 256): time per optimizer step, peak allocated memory per GPU, and GPU-hours per run. ISLA’s blocks are compiled; the baselines run uncompiled, as released. Step time and memory are from earlier runs of the same models with another learning-rate schedule. The models differ in size (Section C.3).
| Relative change of prediction | ||||||
| Surface task (DrivAerML, float32) | ||||||
| Change of presentation | Transolver | Transolver | GeoTransolver | GeoTransolver | ISLA | ISLA |
| Rotation and translation | 1.2 | 1.6 | 1.2 | 1.4 | ||
| rotation alone | 1.1 | 1.6 | 1.1 | 1.4 | ||
| translation alone | ||||||
| Reflection | 1.2 | 1.5 | 1.1 | 1.3 | ||
| Lengths in millimeters, all inputs converted | ||||||
| Size at the same | 0.65 | 0.66 | 0.62 | 0.65 | ||
| Order of the cells | ||||||
| Every cell split in two coincident halves | ||||||
| Cells drawn by area (on shared cells) | 0.28 | 0.33 | 0.29 | 0.35 | 0.010 | |
| redrawn by the same law (on shared cells) | 0.010 | |||||
| Last-bit move of every coordinate | ||||||
| Interior tasks (ISLA, float32) | ||||||
| Change of presentation | DrivAerML, interior | DrivAerML, interior | AeroZoo | AeroZoo | ||
| Rotation and translation | ||||||
| with the token draw held fixed | ||||||
| Reflection | – | – | ||||
| Lengths in millimeters, all inputs converted | ||||||
| Size at the same | ||||||
| Order of the cells and of the query points | ||||||
| Every cell split in two coincident halves | 0.014 | 0.011 | 0.023 | |||
| Another token draw of the same budget | 0.014 | 0.011 | 0.010 | 0.025 | ||
| Last-bit move of every coordinate | 0.022 | |||||
| 10,000 points alone or among 200,000 | 0 | 0 | 0 | 0 | ||
| 10,000 points in ten chunks |
Table 23. Measured invariances in float32: relative change of each trained model’s own prediction under each change of presentation (: pressure; : wall shear stress; : velocity), as medians over 8 cases, where 1 is a change as large as the prediction. The last two rows test query independence. Dash: not measured.
| Model | Parameters | Inference, float32 time [s] | Inference, float32 peak mem. [GiB] | Inference, bfloat16 time [s] | Inference, bfloat16 peak mem. [GiB] |
| Transolver | 47.2 M | 0.561 | 6.3 | 0.136 | 5.8 |
| GeoTransolver | 9.0 M | 0.213 | 5.4 | 0.092 | 5.1 |
| ISLA | 66.3 M | 0.368 | 6.4 | 0.084 | 4.0 |
| ISLA, blocks compiled | 66.3 M | 0.321 | 3.4 | 0.050 | 2.7 |
Table 24. Time and memory of one prediction for 200,000 surface cells of a DrivAerML car on one GPU (median of five calls; peak allocated memory). The baselines are the Transolver of § 5.2 and the 9 M width-256 GeoTransolver of Figure 1 (Section C.3), uncompiled, as released. ISLA is shown uncompiled and with each of its blocks compiled, as it is trained and evaluated.
Data and metric. On AirFRANS we use the 800 training and 200 evaluation cases of its full task, with 100,000 points per case. On DrivAerML we use 435 training and 48 evaluation cases (one of its 484 cases is in neither set), with 200,000 cells or points per case (Transolver is trained with 100,000 and evaluated with 200,000). We also report the tests on HiLiftAeroML (Ashton et al., [9]) and SHIFT-SUV (Luminary Cloud, [59]), and on three solid-mechanics datasets to which the same models apply unchanged: Tensile2d (Casenave et al., [27]), the bracket lug of Liu et al. [56], and DeepJEB++ (Yoo et al., [96]). Figure 8 shows one field per task: velocity on the two interior flow tasks, pressure on the three surface tasks, stress on Tensile2d and the lug, and displacement on DeepJEB++. Table 19 gives every field. The metric is the relative error of a field over the scored cells or points of a case, weighted by cell measure on surfaces and unweighted over the sampled points of an interior task, averaged over cases and then over seeds. Every model is scored on the same cells of each case. The tests compare each model with itself in another presentation; the baselines here are single networks rather than the ensembles of § 5.4 (Table 21 repeats the DrivAerML tests on tuned ensemble members).
| Task | Field | Relative error as stored [%] ↓ Transolver | Relative error as stored [%] ↓ GeoTransolver | Relative error as stored [%] ↓ ISLA | Error (rotated, translated) ÷ stored Transolver | Error (rotated, translated) ÷ stored GeoTransolver | Error (rotated, translated) ÷ stored ISLA |
| AirfRANS | pressure | 0.888 | 0.979 | 0.831 | 134 | 158 | 1.004 |
| velocity | 2.82 | 3.55 | 0.411 | 44 | 32 | 1.004 | |
| eddy viscosity | 1.70 | 1.68 | 1.58 | 87 | 86 | 1.000 | |
| DrivAerML surface | pressure | 4.13 | 4.00 | 4.02 | 29 | 31 | 1.000 |
| wall shear | 6.67 | 6.55 | 6.48 | 21 | 20 | 1.000 | |
| DrivAerML volume | pressure | 4.68 | 3.49 | 3.82 | 17 | 24 | 1.000 |
| velocity | 9.09 | 4.74 | 5.27 | 14 | 24 | 1.000 | |
| eddy viscosity | 8.34 | 8.07 | 7.76 | 18 | 24 | 1.001 | |
| HiLiftAeroML surface | pressure | 5.29 | 5.16 | 5.37 | 19 | 21 | 1.011 |
| wall shear | 8.96 | 8.77 | 9.09 | 14 | 15 | 1.007 | |
| SHIFT-SUV surface | pressure | 4.01 | 3.83 | 4.03 | 30 | 37 | 1.000 |
| wall shear | 6.26 | 6.16 | 6.28 | 18 | 19 | 1.000 | |
| Tensile2d | stress | 0.615 | 0.454 | 0.517 | 155 | 241 | 1.004 |
| displacement | 0.460 | 0.327 | 0.412 | 262 | 402 | 1.003 | |
| Bracket lug | von Mises stress | 0.198 | 0.224 | 0.193 | 504 | 415 | 1.005 |
| DeepJEB++ surface | von Mises stress | 39.9 | 41.7 | 37.9 | 3.2 | 4.8 | 1.000 |
| displacement | 14.8 | 15.4 | 14.6 | 9.7 | 13 | 1.000 |
Table 19. Pose and discretization tests, rigid motion, on every task and field: each model’s relative error in percent on the cases as stored, and its error on the rotated and translated cases divided by that error (seed means, float32; Section C.3).
Pose. Each case receives one rotation, uniform on (in the plane for two-dimensional problems), and a translation by one root-mean-square radius of its points, both fixed by the case’s identifier. Positions, normals, global vectors, and vector and tensor targets are transformed together, and predictions are compared with the transformed targets. The data pipelines of the automotive, wing, and DeepJEB++ datasets center each case on the mean of its points, which removes the translation before any model receives it; on AirFRANS, Tensile2d, and the bracket lug the translation reaches every model.
Re-sampling. From a pool of cells drawn uniformly, each cell is kept independently with probability proportional to its area, for an expected 200,000 cells, and is weighted by its measure divided by its inclusion probability. ISLA uses the cells with these measures; the baselines use the same cells’ centroids and normals. The metric uses the same weights for every model. This test applies where a surface has more cells than a model receives (DrivAerML, HiLiftAeroML, SHIFT-SUV).
AirfRANS
On AirfRANS we use the three splits and node-level evaluator defined by FluidsBench [32] (Table 5): full and scarce, with the same 200 test cases, and aoa_extrapolation, whose test angles of attack lie outside the training range. AirfRANS defines 800 official training cases on full and 804 on aoa_extrapolation, with no separate validation list; we hold out 80 official training cases for validation (using the same 80 on scarce, whose 200 training cases do not contain them, and 80 of its own on aoa_extrapolation), and 6 of the 17 members of ISLA’s full test entry (and 6 of 13 on aoa_extrapolation) train on all 800 or 804 official training cases, with a fixed schedule and the final checkpoint (excluded from the 80-case validation rows of Table 13).
| Split | AirfRANS | DrivAerML | WindsorML | AhmedML |
| full | 720 / 80 / 200 | 400 / 34 / 50 | 280 / 35 / 35 | 400 / 50 / 50 |
| medium | — | 133 / 34 / 50 | 93 / 35 / 35 | 133 / 50 / 50 |
| scarce | 200 / 80 / 200 | 67 / 34 / 50 | 47 / 35 / 35 | 67 / 50 / 50 |
| super_scarce | — | 11 / 34 / 50 | 8 / 35 / 35 | 11 / 50 / 50 |
| out of distribution | 724 / 80 / 196 (one) | 339 / 48 / 97 | 243–244 / 35–36 / 70–71 | 350 / 50 / 100 |
Table 5. Splits of AirfRANS and the automotive aerodynamics datasets (defined by FluidsBench [32]): numbers of training / validation / test cases. The last row gives the counts of each out-of-distribution split (one on AirfRANS, four on each automotive dataset). Dash: the dataset has no such split.
| Split (train) | Cases | Model | |||||||
| full (720) | validation | 80 | Transolver | 1.22 | 1.84 | 0.56 | 1.26 | 0.974 | 1.000 |
| GeoTransolver | 1.34 | 1.52 | 0.50 | 1.27 | 0.970 | 1.000 | |||
| GLOBE | 0.37 | 0.57 | 0.16 | 0.34 | 0.997 | 1.000 | |||
| ISLA | 0.57 | 0.61 | 0.27 | 0.57 | 0.993 | 1.000 | |||
| full (720) | test | 200 | Transolver | 1.17 | 1.76 | 0.53 | 1.14 | 0.899 | 1.000 |
| GeoTransolver | 1.20 | 1.46 | 0.47 | 1.09 | 0.924 | 1.000 | |||
| GLOBE | 0.34 | 0.55 | 0.15 | 0.28 | 0.991 | 1.000 | |||
| ISLA | 0.55 | 0.57 | 0.30 | 0.57 | 0.958 | 1.000 | |||
| scarce (200) | validation | 80 | Transolver | 3.46 | 3.04 | 1.24 | 3.91 | 0.100 | 0.999 |
| GeoTransolver | 3.18 | 2.80 | 1.11 | 3.46 | 0.594 | 0.999 | |||
| GLOBE | 1.03 | 1.04 | 0.38 | 1.05 | 0.896 | 1.000 | |||
| ISLA | 1.55 | 1.71 | 0.76 | 2.13 | 0.743 | 1.000 | |||
| scarce (200) | test | 200 | Transolver | 2.99 | 2.84 | 1.11 | 3.25 | 0.470 | 0.999 |
| GeoTransolver | 2.87 | 2.67 | 1.04 | 2.91 | 0.557 | 0.999 | |||
| GLOBE | 0.86 | 0.98 | 0.32 | 0.75 | 0.919 | 1.000 | |||
| ISLA | 1.54 | 1.70 | 0.76 | 1.96 | 0.509 | 0.999 | |||
| aoa_extrapolation (724) | validation | 80 | Transolver | 0.82 | 1.20 | 0.39 | 0.95 | 0.830 | 1.000 |
| GeoTransolver | 0.98 | 1.27 | 0.43 | 1.03 | 0.786 | 1.000 | |||
| GLOBE | 0.30 | 0.54 | 0.12 | 0.25 | 0.992 | 1.000 | |||
| ISLA | 0.52 | 0.62 | 0.21 | 0.46 | 0.953 | 1.000 | |||
| aoa_extrapolation (724) | test | 196 | Transolver | 8.31 | 6.44 | 3.84 | 12.21 | −0.067 | 0.998 |
| GeoTransolver | 9.29 | 6.60 | 3.91 | 13.02 | 0.355 | 0.996 | |||
| GLOBE | 1.94 | 2.14 | 0.98 | 3.87 | 0.793 | 1.000 | |||
| ISLA | 2.58 | 2.67 | 1.37 | 4.55 | 0.652 | 1.000 |
Table 13. AirfRANS: every split (training cases in parentheses), on validation and test cases. : evaluated cases. and : surface pressure and wall shear stress; and : volume velocity and pressure; and : the of the drag and lift coefficients. Bold: the lowest error and the highest among the four families where all four are evaluated. Dash: not evaluated.
An entry predicts velocity and pressure at every mesh node of each test case, and the evaluator derives the wall quantities and integrates the drag and lift coefficients ( over cases). Each family’s entry is an ensemble (Table 6): on full, ISLA’s entry averages 17 networks of 67 M parameters and GLOBE’s averages five of 1.1 M; Transolver and GeoTransolver blend two kinds of member by wall distance. ISLA takes the airfoil and the free stream and predicts a wall-scaled velocity, Transolver and GeoTransolver predict through a form that vanishes on the wall, and GLOBE predicts a disturbance of the free stream.
| Setting | Transolver | GeoTransolver | GLOBE | ISLA |
| AirfRANS (full, aoa_extrapolation, scarce) | ||||
| Networks | 3, 20, 15 | 9, 9, 9 | 5, 4, 7 | 17, 13, 6 |
| Parameters | 9 M | 9 M and larger | 1.1 M (two of 0.3 M) | 67 M |
| DrivAerML | ||||
| Networks () | 5+5 (full: 5+10) | 5+5 (full: 10+5) | 3+3 | 5 (full: 9), each a joint surface and volume network |
| Layers width | 16 768 | 12 768 or 12 512 (), 12 256 () | – | 12 512 |
| Parameters | 106 M | 80 M or 36 M (), 28 M () | 12 M | 67 M |
| Training points | 100,000 | 200,000 | 50,000 | 200,000 () and 200,000 () |
| Learning rate | ||||
| Optimizer steps | 50,000 on every split | 50,000 on every split | 60,000 / 28,800 / 4,800 | 50,000 on every split |
| WindsorML | ||||
| Networks () | 6+6 (super_scarce: 3+3) | 3+3 | 3+3 | 2 to 5 (), 2 to 5 () |
| Parameters | 7 M | 18 M | 4 M | 66 M (), 67 M () |
| Training points | 200,000 | 200,000 | 40,000 | 200,000 |
| Learning rate | or | (), () | ||
| AhmedML | ||||
| Networks () | 3+3 or 3+4 | 3+3 or 3+4 | 3+3, 4+3, or 3+4 | 2+2 to 3+8 |
| Parameters | 47 M | 36 M (), 28 M () | 1 M | 66 M (), 67 M () |
| Training points | 100,000 | 200,000 (), 100,000 () | 80,000 | 200,000 |
| Learning rate | (), () | |||
| Optimizer steps | 50,000 on every split | 50,000 on every split | 50,000 on every split | 50,000 on every split |
Table 6. Ensemble entries on AirfRANS and the automotive aerodynamics datasets (Table 2 and Table 14). “” is the number of surface and volume networks in an entry; where entries differ between the splits of a dataset, the table lists the sizes that occur. Parameters are per network. Learning rates are peak values. Optimizer steps are those scheduled, for full / scarce / super_scarce. A dash: does not apply.
Automotive Aerodynamics Datasets
On DrivAerML, WindsorML, and AhmedML we use the splits and evaluators defined by FluidsBench [32]. We report the evaluator’s field errors and values.
Splits. Each road-vehicle dataset has eight splits (Table 5): four share one validation and one test list and differ in the number of training cases (full, medium, scarce, and super_scarce), and four hold out a region for testing: the cases farthest from the others in shape (geometry), the highest or lowest drag (high_drag, low_drag), and the largest wake or rear separation, measured on published images (rear_separation on DrivAerML, image_wake on WindsorML and AhmedML). On AhmedML the validation cases of these four splits are drawn from the training-side population, so validation errors there do not show the shift of the test cases.
Entries and metrics. An entry predicts pressure and wall shear stress at every surface cell and velocity and pressure at every volume cell of the reference mesh of each test case; force coefficients are integrated by the evaluator from the predicted fields. All entries are trained from scratch on the official training cases of their split. For each case the evaluator computes the relative error of each field over all evaluated entities, weighted by face area on the surface and equally per cell in the volume, and averages it over cases, alongside the of drag and lift.
Model families. We trained all four families ourselves, using the released PhysicsNeMo implementations of Transolver, GeoTransolver, and GLOBE (PhysicsNeMo Contributors, [67]) and the ISLA architecture of Table 3. Each entry is the equal-weight mean of independently trained member networks, taken separately for the surface and volume (Table 6), with member count, recipe, and inputs tuned per family and dataset:
Size. On DrivAerML one ISLA network predicts the surface and the volume; on its full training set the entries hold 0.61 (ISLA, nine networks), 0.72 (GeoTransolver, fifteen), and 1.59 (Transolver, fifteen) billion parameters. On WindsorML
and AhmedML every family trains separate surface and volume networks, and ISLA’s networks and entries are the largest.
Inputs. Transolver and GeoTransolver take the coordinates and normals of the sampled cells, the free-stream direction, and for volume points a stored signed distance to the body. GLOBE takes the body and, on DrivAerML and WindsorML, patches of the floor and tunnel as boundary sources. ISLA takes the named boundaries of the whole domain (the body and the tunnel’s inlet, outlet, floor, and walls) and no coordinates, computing boundary distance itself (Section B.6).
Loss. ISLA trains on a per-case relative loss, and the baselines on a Huber loss of standardized fields.
Budget and checkpoint. Every Transolver, GeoTransolver, and ISLA network trains for at least 50,000 optimizer steps with a cosine schedule (as does GLOBE on AhmedML; GLOBE on DrivAerML and WindsorML uses its earlier shorter-schedule runs on the smaller splits, Table 6). On DrivAerML the baselines are entered at the saved checkpoint with the lowest validation error, and ISLA (and all AhmedML lineups) at the final checkpoint.
Augmentation and precision. No family was trained with rotations or translations. Transolver and GeoTransolver are trained in float32 on DrivAerML and in bfloat16 elsewhere; GLOBE and ISLA are trained in bfloat16.
Dataset notes. WindsorML. No model has test results, because the official evaluator does not read the released volume files. The validation results use our reader of those files with the official reduction. WindsorML stores an absolute volume pressure of about 37 kPa, so the relative error of volume pressure is only 0.05 to 0.22% for every model. Its official lists contain six pairs of cases with identical geometry, some of which lie on both sides of a split. On WindsorML, Transolver (all splits), GeoTransolver (all splits), and ISLA (full) use lineups retrained to at least 50,000 steps per network, while the other cells show the earlier lineups.
AhmedML. On image_wake and high_drag, the field errors of Transolver, GeoTransolver, and GLOBE on test cases are 3.0 to 11.1 and 1.6 to 3.1 times those on validation cases; this is the shift that those splits are built to contain. ISLA’s full entry has three surface and eight volume networks, the latter trained with five variants of the recipe; its validation entries on high_drag and low_drag were not evaluated per field.
Compute
All models were trained on nodes with four NVIDIA GB300 or GB200 GPUs, with one case per GPU per step on the three-dimensional datasets. One ISLA model on AeroZoo full takes 105 GPU-hours, at 0.66 s per step and 142 GiB per GPU; Table 7 gives the training cost of single models on AirfRANS and DrivAerML (Section C.3). A later version of the code recomputes some activations of the compiled blocks in the backward pass instead of storing them. On AeroZoo (measured over the second and third epochs), it lowers the peak allocated memory per GPU from 120.5 to 83.3 GiB and the time per iteration from 0.65 to 0.56 s (over a full 250-epoch AeroZoo run, the largest-epoch peak drops from 142.2 to 105.4 GiB and the median step time from 0.66 to 0.56 s). Over epochs 2 and 3, it lowers peak memory from 25.3 to 17.6 GiB on AirfRANS, from 55.9 to 41.7 GiB on the DrivAerML surface, and from 129.0 to 93.3 GiB on its volume, while iteration time changes by at most 2% on these tasks. The joint surface-and-volume DrivAerML networks of Section C.5 cost more.
Additional Results
AeroZoo: Every Model and Split
Table 8 gives every model on full, and Table 9 the medians by enclosure configuration. ISLA’s median relative errors over the interior points, counted equally, are 0.103 for velocity and 0.321 for gauge pressure, and its median errors within one reference length of the body are 0.091 and 0.043. The mean pressure error of every model is set by a few cases: ISLA’s mean is 0.298 and its median 0.023. Supervising boundary targets as well lowers mean to 0.171 (median 0.021, and relative pressure error 8% lower), while turbulence metrics remain unchanged within noise. ISLA also predicts the five turbulence fields of the task (Table 10). For the intermittency and the transition Reynolds number we report its median relative errors, 0.016 and 0.085, in place of AeroZoo’s own metrics.
Table 9. AeroZoo, full: median by enclosure configuration, the numbers behind Figure 4. “BC”: with boundary tokens; “scaled”: in per-case scales. Baseline rows are from Sharpe et al. [81]; its rows in SI units without boundary tokens are at the zero predictor’s value in every configuration and are omitted. Bold: the lowest error in each column.
| Model (test cases) | Sphere (63) | Open box (95) | Ground effect (107) | Confined box (59) | Hemisphere (29) | Duct (41) | Crossflow (34) | Dual inlet (87) |
| Free-stream reference | 0.034 | 0.043 | 0.074 | 0.159 | 0.194 | 0.510 | 0.969 | 1.513 |
| Transolver, scaled | 1.236 | 0.083 | 0.091 | 0.162 | 1.578 | 0.443 | 0.972 | 1.118 |
| GeoTransolver, scaled | 1.076 | 0.429 | 0.390 | 0.427 | 1.280 | 0.531 | 1.200 | 1.607 |
| GeoTransolver + BC, SI | 0.078 | 0.091 | 0.113 | 0.202 | 0.204 | 0.542 | 0.947 | 1.516 |
| GeoTransolver + BC, scaled | 0.031 | 0.057 | 0.058 | 0.117 | 0.114 | 0.310 | 0.396 | 1.124 |
| ISLA | 0.023 | 0.028 | 0.043 | 0.090 | 0.052 | 0.138 | 0.124 | 0.143 |
Table 9.
Table 10. AeroZoo, full: median errors of three turbulence fields, in the units that AeroZoo defines (, , and ). The reference’s errors give their scale. Bold: the lowest error in each column.
| Model | Median ref.-scaled error | Median ref.-scaled error | Median ref.-scaled error |
| Free-stream reference | 0.0057 | 35.6 | 0.0013 |
| GeoTransolver + BC, per-case scales | 0.0043 | 19.6 | 0.00070 |
| ISLA | 0.0030 | 5.81 | 0.00036 |
Table 10.
Table 11 gives ISLA’s errors on every AeroZoo split that we trained. With a geometry family held out, the median velocity error on cases with a body is 0.059 (blobs) and 0.079 (CAD parts), compared with 0.053 over all test cases of full. With the Reynolds numbers above held out, the median velocity error (0.067) is close to the reference’s (0.076), while the mean (0.125) is three times lower than the reference’s (0.379). With a layout of the boundary conditions held out (profile_crossflow, profile_dual_inlet), ISLA’s median is 31 and 11% below the reference’s. With every non-box enclosure held out, ISLA is as little less accurate than the reference in both fields. We have no baseline with per-case scales on these splits.
Table 12 compares an ISLA of about GeoTransolver’s size with one of the reference size, both trained on the same step schedule (Section C.2). The smaller one has 8 blocks, 64 slices, an MLP ratio of 2, and boundary tokens per case in place of , so it reduces the token budget as well as the parameters. On full its errors are within 3% of the larger model’s, and its mean is 3% higher. With every non-box enclosure held out (family_non_box), its mean is 1.17 times the larger model’s.
AirfRANS
Table Table 13 gives every field error and value on all three AirfRANS splits (defined by FluidsBench; [32]), on validation and test cases. Bold marks the lowest error and the highest among the four families, and a dash a value that does not exist.
Table 13. AirfRANS: every split (training cases in parentheses), on validation and test cases. : evaluated cases. and : surface pressure and wall shear stress; and : volume velocity and pressure; and : the of the drag and lift coefficients. Bold: the lowest error and the highest among the four families where all four are evaluated. Dash: not evaluated.
Automotive Aerodynamics Datasets
Table Table 14 compares ISLA with the lowest of the other three families on the main splits of DrivAerML, WindsorML, and AhmedML (using the splits defined by FluidsBench; [32]), and Table 15 to Table 17 give every field error and value for every split. With WindsorML’s 8 training cases ISLA’s errors differ from the lowest other’s by −3 to +24%, and with AhmedML’s 11 by −7 to +12%. Figure 7 plots the surface-pressure and volume-velocity errors as functions of the number of training cases on the three automotive datasets.
Figure 7. Surface-pressure and volume-velocity errors as functions of the number of training cases on the three automotive datasets (relative , logarithmic axes). Lines and filled markers: validation cases. Open markers: test cases, where an entry has test results; none has on WindsorML.
| Split (train) | Cases | Model | |||||||
| full (400) | validation | 34 | Transolver | 3.72 | 5.92 | 4.56 | 3.21 | 0.990 | 0.992 |
| GeoTransolver | 3.60 | 5.70 | 4.43 | 3.11 | 0.992 | 0.992 | |||
| GLOBE | 7.67 | 8.67 | 7.36 | 5.89 | 0.959 | 0.980 | |||
| ISLA | 3.67 | 5.98 | 4.83 | 3.28 | 0.993 | 0.993 | |||
| full (400) | test | 50 | Transolver | – | – | – | – | – | – |
| GeoTransolver | – | – | – | – | – | – | |||
| GLOBE | 7.34 | 8.31 | 7.37 | 5.82 | 0.965 | 0.970 | |||
| ISLA | 3.36 | 5.58 | 4.80 | 3.23 | 0.988 | 0.982 | |||
| medium (133) | validation | 34 | Transolver | 4.31 | 6.78 | 5.42 | 3.98 | 0.982 | 0.982 |
| GeoTransolver | 4.30 | 6.77 | 5.19 | 3.79 | 0.984 | 0.981 | |||
| GLOBE | 8.90 | 9.89 | 8.41 | 6.79 | 0.958 | 0.944 | |||
| ISLA | 4.41 | 7.03 | 5.74 | 4.10 | 0.980 | 0.982 | |||
| medium (133) | test | 50 | Transolver | – | – | – | – | – | – |
| GeoTransolver | – | – | – | – | – | – | |||
| GLOBE | 8.51 | 9.40 | 8.39 | 6.58 | 0.954 | 0.924 | |||
| ISLA | 4.12 | 6.63 | 6.00 | 4.11 | 0.980 | 0.967 | |||
| scarce (67) | validation | 34 | Transolver | 5.16 | 7.85 | 6.55 | 4.96 | 0.969 | 0.948 |
| GeoTransolver | 5.03 | 7.66 | 6.22 | 4.64 | 0.962 | 0.957 | |||
| GLOBE | 9.39 | 11.28 | 9.35 | 7.56 | 0.930 | 0.924 | |||
| ISLA | 5.21 | 7.98 | 7.03 | 5.18 | 0.955 | 0.945 | |||
| scarce (67) | test | 50 | Transolver | – | – | – | – | – | – |
| GeoTransolver | – | – | – | – | – | – | |||
| GLOBE | 9.07 | 10.81 | 9.43 | 7.41 | 0.968 | 0.912 | |||
| ISLA | 4.97 | 7.66 | 7.33 | 5.09 | 0.968 | 0.917 | |||
| super_scarce (11) | validation | 34 | Transolver | 14.11 | 16.77 | 14.51 | 12.62 | 0.500 | 0.188 |
| GeoTransolver | 14.47 | 17.38 | 14.59 | 12.86 | 0.546 | 0.334 | |||
| GLOBE | 17.80 | 21.26 | 16.80 | 14.70 | 0.505 | 0.449 | |||
| ISLA | 12.29 | 15.44 | 13.23 | 11.45 | 0.743 | 0.530 | |||
| super_scarce (11) | test | 50 | Transolver | – | – | – | – | – | – |
| GeoTransolver | – | – | – | – | – | – | |||
| GLOBE | 17.14 | 20.11 | 16.41 | 14.06 | 0.694 | -0.015 | |||
| ISLA | 11.29 | 14.03 | 12.65 | 10.45 | 0.729 | 0.186 | |||
| geometry (339) | validation | 48 | Transolver | 3.24 | 5.34 | 4.52 | 3.11 | 0.992 | 0.978 |
| GeoTransolver | 3.14 | 5.22 | 4.37 | 2.99 | 0.994 | 0.983 | |||
| GLOBE | 7.28 | 8.32 | 7.45 | 6.00 | 0.976 | 0.964 | |||
| ISLA | 3.30 | 5.55 | 4.91 | 3.31 | 0.992 | 0.980 | |||
| geometry (339) | test | 97 | Transolver | – | – | – | – | – | – |
| GeoTransolver | – | – | – | – | – | – | |||
| GLOBE | 8.11 | 9.65 | 8.34 | 6.66 | 0.953 | 0.978 | |||
| ISLA | 4.20 | 6.85 | 6.20 | 4.28 | 0.987 | 0.989 | |||
| high_drag (339) | validation | 48 | Transolver | 3.50 | 5.61 | 4.39 | 3.01 | 0.989 | 0.988 |
| GeoTransolver | 3.46 | 5.53 | 4.33 | 2.96 | 0.990 | 0.985 | |||
| GLOBE | 7.66 | 8.61 | 7.58 | 5.94 | 0.966 | 0.968 | |||
| ISLA | 3.55 | 5.72 | 4.74 | 3.13 | 0.991 | 0.986 | |||
| high_drag (339) | test | 97 | Transolver | – | – | – | – | – | – |
| GeoTransolver | – | – | – | – | – | – | |||
| GLOBE | 8.44 | 9.84 | 8.43 | 6.86 | 0.868 | 0.941 | |||
| ISLA | 4.08 | 6.57 | 5.72 | 4.15 | 0.943 | 0.975 | |||
| low_drag (339) | validation | 48 | Transolver | 3.53 | 5.75 | 4.57 | 3.09 | 0.990 | 0.984 |
| GeoTransolver | 3.49 | 5.69 | 4.45 | 2.99 | 0.988 | 0.983 | |||
| GLOBE | 7.71 | 8.75 | 7.32 | 5.84 | 0.958 | 0.958 | |||
| ISLA | 3.57 | 5.90 | 4.93 | 3.34 | 0.991 | 0.983 | |||
| low_drag (339) | test | 97 | Transolver | – | – | – | – | – | – |
| GeoTransolver | – | – | – | – | – | – | |||
| GLOBE | 8.20 | 9.62 | 8.59 | 6.68 | 0.964 | 0.952 | |||
| ISLA | 4.33 | 7.07 | 7.04 | 4.91 | 0.974 | 0.974 | |||
| rear_separation (339) | validation | 48 | Transolver | 3.58 | 5.63 | 4.55 | 3.23 | 0.991 | 0.983 |
| GeoTransolver | 3.55 | 5.54 | 4.41 | 3.14 | 0.992 | 0.985 | |||
| GLOBE | 7.74 | 8.65 | 7.56 | 6.07 | 0.967 | 0.950 | |||
| ISLA | 3.69 | 5.88 | 4.92 | 3.46 | 0.991 | 0.986 | |||
| rear_separation (339) | test | 97 | Transolver | – | – | – | – | – | – |
| GeoTransolver | – | – | – | – | – | – | |||
| GLOBE | 7.66 | 8.78 | 7.91 | 6.24 | 0.963 | 0.943 | |||
| ISLA | 3.78 | 6.13 | 5.59 | 3.81 | 0.988 | 0.960 |
Table 15. Table 15. DrivAerML: every split (training cases in parentheses). Columns, bold, and dashes as in Table 13. On DrivAerML, Transolver (all splits) and GeoTransolver (all splits) use lineups retrained to at least 50,000 steps per network (matching ISLA’s 50,000-step lineups), while GLOBE shows its earlier lineups.
| Split (train) | Cases | Model | |||||||
| full (400) | validation | 50 | Transolver | 3.59 | 5.57 | 1.80 | 3.25 | 0.961 | 0.951 |
| full (400) | validation | 50 | GeoTransolver | 3.34 | 5.36 | 1.79 | 3.22 | 0.963 | 0.975 |
| full (400) | validation | 50 | GLOBE | 3.75 | 5.27 | 2.52 | 4.04 | 0.842 | 0.892 |
| full (400) | validation | 50 | ISLA | 3.07 | 5.15 | 2.32 | 3.55 | 0.968 | 0.960 |
| full (400) | test | 50 | Transolver | 3.41 | 5.35 | 1.71 | 3.23 | 0.921 | 0.919 |
| full (400) | test | 50 | GeoTransolver | 3.35 | 5.36 | 1.73 | 3.21 | 0.932 | 0.925 |
| full (400) | test | 50 | GLOBE | 3.68 | 5.13 | 2.43 | 4.04 | 0.867 | 0.896 |
| full (400) | test | 50 | ISLA | 2.95 | 4.97 | 2.18 | 3.20 | 0.927 | 0.915 |
| medium (133) | validation | 50 | Transolver | 3.76 | 5.56 | 2.10 | 3.66 | 0.974 | 0.947 |
| medium (133) | validation | 50 | GeoTransolver | 3.55 | 5.37 | 2.15 | 3.71 | 0.982 | 0.954 |
| medium (133) | validation | 50 | GLOBE | 3.53 | 5.14 | 2.57 | 4.25 | 0.933 | 0.952 |
| medium (133) | validation | 50 | ISLA | 3.13 | 5.23 | 2.39 | 3.54 | 0.976 | 0.959 |
| medium (133) | test | 50 | Transolver | 4.08 | 5.33 | 2.16 | 4.06 | 0.698 | 0.823 |
| medium (133) | test | 50 | GeoTransolver | 3.73 | 5.13 | 2.16 | 4.00 | 0.775 | 0.869 |
| medium (133) | test | 50 | GLOBE | 3.73 | 5.20 | 2.67 | 4.47 | 0.871 | 0.881 |
| medium (133) | test | 50 | ISLA | 3.43 | 5.25 | 2.37 | 3.57 | 0.892 | 0.919 |
| scarce (67) | validation | 50 | Transolver | 4.78 | 6.22 | 2.90 | 5.00 | 0.892 | 0.907 |
| scarce (67) | validation | 50 | GeoTransolver | 4.74 | 6.19 | 2.97 | 5.06 | 0.877 | 0.897 |
| scarce (67) | validation | 50 | GLOBE | 4.75 | 5.35 | 3.04 | 5.40 | 0.904 | 0.902 |
| scarce (67) | validation | 50 | ISLA | 4.16 | 5.82 | 2.91 | 4.86 | 0.941 | 0.941 |
| scarce (67) | test | 50 | Transolver | 4.86 | 5.90 | 2.43 | 4.27 | 0.416 | 0.584 |
| scarce (67) | test | 50 | GeoTransolver | 5.01 | 5.90 | 2.48 | 4.41 | 0.424 | 0.578 |
| scarce (67) | test | 50 | GLOBE | 4.08 | 5.21 | 2.72 | 4.67 | 0.851 | 0.856 |
| scarce (67) | test | 50 | ISLA | 4.33 | 5.54 | 2.56 | 4.11 | 0.631 | 0.720 |
| super_scarce (11) | validation | 50 | Transolver | 9.20 | 8.19 | 5.35 | 11.01 | 0.560 | 0.754 |
| super_scarce (11) | validation | 50 | GeoTransolver | 8.25 | 7.73 | 5.31 | 11.46 | 0.694 | 0.756 |
| super_scarce (11) | validation | 50 | GLOBE | 9.05 | 7.32 | 6.30 | 10.59 | 0.721 | 0.675 |
| super_scarce (11) | validation | 50 | ISLA | 8.10 | 7.61 | 4.48 | 9.43 | 0.753 | 0.798 |
| super_scarce (11) | test | 50 | Transolver | 9.66 | 9.05 | 5.19 | 10.27 | 0.096 | 0.370 |
| super_scarce (11) | test | 50 | GeoTransolver | 8.96 | 8.58 | 5.29 | 10.75 | 0.326 | 0.384 |
| super_scarce (11) | test | 50 | GLOBE | 8.46 | 7.76 | 5.65 | 8.78 | 0.537 | 0.504 |
| super_scarce (11) | test | 50 | ISLA | 8.49 | 8.12 | 4.80 | 9.84 | 0.550 | 0.604 |
| geometry (350) | validation | 50 | Transolver | 3.25 | 5.08 | 1.80 | 3.42 | 0.933 | 0.951 |
| geometry (350) | validation | 50 | GeoTransolver | 3.26 | 5.13 | 1.80 | 3.47 | 0.919 | 0.936 |
| geometry (350) | validation | 50 | GLOBE | 4.00 | 5.11 | 2.60 | 4.52 | 0.723 | 0.814 |
| geometry (350) | validation | 50 | ISLA | 3.21 | 5.08 | 2.38 | 3.64 | 0.903 | 0.921 |
| geometry (350) | test | 100 | Transolver | 3.92 | 5.50 | 2.18 | 3.69 | 0.938 | 0.868 |
| geometry (350) | test | 100 | GeoTransolver | 3.88 | 5.44 | 2.17 | 3.68 | 0.941 | 0.876 |
| geometry (350) | test | 100 | GLOBE | 4.00 | 5.27 | 2.97 | 4.60 | 0.941 | 0.870 |
| geometry (350) | test | 100 | ISLA | – | – | – | – | – | – |
| high_drag (350) | validation | 50 | Transolver | 2.85 | 4.81 | 1.68 | 3.22 | 0.852 | 0.947 |
| high_drag (350) | validation | 50 | GeoTransolver | 2.73 | 4.88 | 1.68 | 3.21 | 0.890 | 0.963 |
| high_drag (350) | validation | 50 | GLOBE | 2.81 | 4.54 | 2.17 | 3.62 | 0.888 | 0.946 |
| high_drag (350) | validation | 50 | ISLA | – | – | – | – | – | – |
| high_drag (350) | test | 100 | Transolver | 8.74 | 7.64 | 4.71 | 9.51 | -1.518 | 0.569 |
| high_drag (350) | test | 100 | GeoTransolver | 8.42 | 7.67 | 4.91 | 10.10 | -1.389 | 0.622 |
| high_drag (350) | test | 100 | GLOBE | 8.55 | 7.25 | 5.31 | 10.50 | -1.403 | 0.610 |
| high_drag (350) | test | 100 | ISLA | – | – | – | – | – | – |
| low_drag (350) | validation | 50 | Transolver | 3.86 | 5.64 | 1.96 | 3.37 | 0.552 | 0.806 |
| low_drag (350) | validation | 50 | GeoTransolver | 3.72 | 5.46 | 1.98 | 3.42 | 0.669 | 0.846 |
| low_drag (350) | validation | 50 | GLOBE | 3.46 | 4.89 | 2.65 | 4.10 | 0.916 | 0.938 |
| low_drag (350) | validation | 50 | ISLA | – | – | – | – | – | – |
| low_drag (350) | test | 100 | Transolver | 3.01 | 4.74 | 1.40 | 2.84 | 0.534 | 0.913 |
| low_drag (350) | test | 100 | GeoTransolver | 2.85 | 4.58 | 1.36 | 2.74 | 0.643 | 0.943 |
| low_drag (350) | test | 100 | GLOBE | 3.33 | 4.62 | 2.29 | 4.37 | 0.320 | 0.882 |
| low_drag (350) | test | 100 | ISLA | – | – | – | – | – | – |
| image_wake (350) | validation | 50 | Transolver | 2.44 | 4.98 | 1.36 | 2.33 | 0.959 | 0.869 |
| image_wake (350) | validation | 50 | GeoTransolver | 2.38 | 4.96 | 1.33 | 2.27 | 0.959 | 0.880 |
| image_wake (350) | validation | 50 | GLOBE | 2.21 | 4.39 | 2.01 | 2.80 | 0.954 | 0.878 |
| image_wake (350) | validation | 50 | ISLA | 2.04 | 4.81 | 1.89 | 2.26 | 0.968 | 0.899 |
| image_wake (350) | test | 100 | Transolver | 22.10 | 15.02 | 11.42 | 25.87 | -0.552 | -10.153 |
| image_wake (350) | test | 100 | GeoTransolver | 22.01 | 15.08 | 11.07 | 25.22 | -0.535 | -9.917 |
| image_wake (350) | test | 100 | GLOBE | 21.87 | 15.00 | 10.52 | 23.94 | -0.630 | -9.600 |
| image_wake (350) | test | 100 | ISLA | – | – | – | – | – | – |
Table 17. AhmedML: every split (training cases in parentheses). Columns, bold, and dashes as in Table 13.
One network per family. Table 18 gives the field errors of one network of each family, the first seed of its entry, beside those of the entry, on validation cases (Table 6 lists network sizes and schedules). A single network’s errors differ from its entry’s by −2% to +13%.
Table 18. One network per family beside the family’s entry: relative errors in percent on validation cases (first seed of each entry; ISLA on DrivAerML is one joint surface-and-volume network, evaluated in bfloat16 on full). On DrivAerML the baseline networks use the saved checkpoint with lowest validation error, and all others use their final checkpoint.
| Relative error [%] | ||||||||||
| Dataset | Split (train) | Field | ||||||||
| One network | Entry (ensemble) | |||||||||
| Dataset | Split (train) | Field | Transolver | GeoTransolver | GLOBE | ISLA | Transolver | GeoTransolver | GLOBE | ISLA |
| AhmedML | full (400) | 3.72 | 3.43 | 3.97 | 3.27 | 3.59 | 3.34 | 3.75 | 3.07 | |
| AhmedML | full (400) | 5.76 | 5.56 | 5.39 | 5.35 | 5.57 | 5.36 | 5.27 | 5.15 | |
| AhmedML | full (400) | 1.78 | 1.88 | 2.78 | 2.50 | 1.80 | 1.79 | 2.52 | 2.32 | |
| AhmedML | full (400) | 3.20 | 3.42 | 4.59 | 3.87 | 3.25 | 3.22 | 4.04 | 3.55 |
Table 18. One network per family beside the family’s entry: relative errors in percent on validation cases (first seed of each entry; ISLA on DrivAerML is one joint surface-and-volume network, evaluated in bfloat16 on full). On DrivAerML the baseline networks use the saved checkpoint with lowest validation error, and all others use their final checkpoint.
On DrivAerML’s 11-case split, before the full entries were retrained to 50,000 steps, single networks of Transolver and GeoTransolver trained on an earlier 4,000-step schedule were also retrained on a schedule 12.5 times longer (50,000 steps scheduled, as for ISLA’s networks): stopped at 30,002 to 30,502 steps by that recipe’s learning-rate rule, with their best checkpoints at 4,502 to 10,002. Against controls trained beside them at the original length, the four field errors on validation cases fell by 31 to 36% and 12 to 26%, each with a 90% case-bootstrap interval below 1. Their errors then lay between 3% below and 11% above those of ISLA’s single network, against 22 to 72% above before, with surface pressure still 9 to 11% higher (the retrained networks’ checkpoints are chosen on these validation cases, while ISLA uses its final checkpoint).
Table 16. WindsorML: every split (training cases in parentheses), on validation cases only. Columns, bold, and dashes as in Table 13.
| Split (train) | Cases | Model | |||||||
| relative error [%] | |||||||||
| Force | |||||||||
| (280) | validation | 35 | Transolver | 7.12 | 6.51 | 5.65 | 0.06 | 0.582 | 0.939 |
| GeoTransolver | 7.00 | 6.74 | 5.65 | 0.06 | 0.583 | 0.941 | |||
| GLOBE | 7.36 | 6.52 | 6.18 | 0.06 | 0.571 | 0.944 | |||
| ISLA | 6.61 | 6.46 | 5.38 | 0.06 | 0.657 | 0.953 | |||
| (93) | validation | 35 | Transolver | 8.09 | 7.12 | 6.70 | 0.07 | 0.436 | 0.921 |
| GeoTransolver | 8.09 | 7.21 | 6.96 | 0.08 | 0.489 | 0.916 | |||
| GLOBE | 8.46 | 7.07 | 7.32 | 0.08 | 0.352 | 0.916 | |||
| ISLA | 7.70 | 7.04 | 6.73 | 0.07 | 0.566 | 0.930 | |||
| (47) | validation | 35 | Transolver | 9.29 | 8.11 | 7.82 | 0.10 | 0.169 | 0.880 |
| GeoTransolver | 9.86 | 8.44 | 7.79 | 0.10 | -0.153 | 0.817 | |||
| GLOBE | 9.23 | 7.73 | 8.00 | 0.09 | -0.180 | 0.879 | |||
| ISLA | 9.25 | 8.35 | 7.28 | 0.08 | -0.061 | 0.867 | |||
| (8) | validation | 35 | Transolver | 22.39 | 16.22 | 14.04 | 0.22 | -5.705 | 0.343 |
| GeoTransolver | 22.16 | 15.96 | 14.30 | 0.22 | -9.865 | 0.435 | |||
| GLOBE | 17.70 | 12.68 | 13.52 | 0.18 | -5.683 | 0.220 | |||
| ISLA | 21.09 | 15.67 | 13.14 | 0.20 | -7.480 | 0.425 | |||
| (243) | validation | 36 | Transolver | 7.78 | 6.63 | 5.70 | 0.06 | 0.676 | 0.873 |
| GeoTransolver | 7.88 | 7.09 | 5.74 | 0.06 | 0.610 | 0.864 | |||
| GLOBE | 7.89 | 6.62 | 6.17 | 0.07 | 0.442 | 0.873 | |||
| ISLA | 7.42 | 6.82 | 5.45 | 0.05 | 0.796 | 0.946 | |||
| (244) | validation | 36 | Transolver | 6.85 | 5.91 | 5.76 | 0.05 | 0.768 | 0.970 |
| GeoTransolver | 6.93 | 5.92 | 6.10 | 0.06 | 0.732 | 0.973 | |||
| GLOBE | 7.00 | 6.15 | 6.05 | 0.06 | 0.632 | 0.951 | |||
| ISLA | 6.81 | 6.00 | 5.99 | 0.06 | 0.750 | 0.972 | |||
| (244) | validation | 36 | Transolver | 7.51 | 7.04 | 5.69 | 0.06 | -0.101 | 0.948 |
| GeoTransolver | 7.54 | 7.05 | 5.74 | 0.06 | -0.027 | 0.953 | |||
| GLOBE | 7.77 | 7.05 | 6.55 | 0.07 | -0.240 | 0.953 | |||
| ISLA | 7.08 | 6.97 | 5.86 | 0.06 | 0.094 | 0.958 | |||
| (244) | validation | 35 | Transolver | 8.06 | 6.97 | 6.48 | 0.08 | 0.822 | 0.834 |
| GeoTransolver | 8.02 | 7.19 | 6.64 | 0.08 | 0.852 | 0.847 | |||
| GLOBE | 7.74 | 6.56 | 6.67 | 0.07 | 0.713 | 0.832 | |||
| ISLA | 7.48 | 6.58 | 6.48 | 0.07 | 0.858 | 0.848 |
Table 16. WindsorML: every split (training cases in parentheses), on validation cases only. Columns, bold, and dashes as in Table 13.
Pose and Discretization Tests: Every Field
Tables 19 and 20 give the numbers behind Figure 8, with every field of every task. A rotated case lies outside the training set of a model that takes coordinates, and the model’s error on it is then between 0.8 and 2.0 times that of predicting zero. The ratio in Table 19 is therefore of the order of one over the model’s error as stored, and it is largest on the tasks that are fitted best (the bracket lug, to 0.2%).
Table 21 repeats the DrivAerML surface tests with one network of each family’s DrivAerML full entry, on its validation cases. A rotation changes two inputs of Transolver and GeoTransolver at once: the geometry and the free-stream velocity, which is the same in every training case. The geometry causes the rise: with the free-stream input left as stored, the pressure error is still 27 to 29 times that on the stored cases. Moving each rotated case at inference into the vehicle’s area-weighted principal-axis frame (signs fixed by third central moments and right-handedness) lowers the pressure ratio only to 5.8 and 5.4, because a car’s principal axes tilt by about from the stored frame. Adding one fixed alignment rotation fitted on the training cases reduces the residual rise to 9 and 18% in pressure and 12 to 19% in wall shear (inference only; canonicalization leaves re-sampling, units, query sets, and symmetric bodies untouched). Re-sampling by area multiplies their pressure errors by 7.6 to 9.0, and as many cells drawn uniformly leave them unchanged, so the rise comes from the law of the draw and not from the number of cells. GLOBE is equivariant by construction, yet the errors of its network rise by a factor of 1.19 to 1.26 under rotation and 1.07 to 1.10 under re-sampling. We have not identified the cause of either rise. The bottom block of Table 21 retrains one surface network each of Transolver and GeoTransolver with the pipelines’ vertical-axis rotation and m horizontal translation enabled (one seed; one random pose per four-GPU step; stopped near epoch 300 by the learning-rate rule). That augmentation removes the vertical-axis rotation rise (pressure ratio at most 1.001), leaves SO(3) rotation at 23 times stored and area re-sampling at 7.0 times, and multiplies stored-case errors by 1.13 and 1.23 in pressure (1.08 and 1.15 in wall shear).
Table 22 adds Transolver and GeoTransolver trained on rotations drawn uniformly over all orientations, the law of the rotated test, with the pipelines and schedules of the unaugmented seeds (2 seeds per augmented model). Their surface-pressure errors change by 0.1 and 0.2% under rotation, and are 4.5 and 3.3% higher on the stored cases than without augmentation. On the rotated cases they are 1.08 times ISLA’s (90% interval 1.03 to 1.12) and 1.03 times (0.99 to 1.07).
| Task | Field | Transolver | GeoTransolver | ISLA |
| DrivAerML surface | pressure | 7.2 | 3.6 | 1.005 |
| wall shear | 5.6 | 3.3 | 1.010 | |
| HiLiftAeroML surface | pressure | 2.8 | 3.4 | 1.000 |
| wall shear | 2.8 | 3.1 | 1.000 | |
| SHIFT-SUV surface | pressure | 1.51 | 1.37 | 0.999 |
| wall shear | 1.39 | 1.23 | 1.000 |
Table 20. Pose and discretization tests, re-sampling by area: each model’s error on the re-sampled cases divided by its error on the cases as stored (seed means, float32), on the three tasks whose surfaces have more cells than a model receives.
Measured Invariances
Table 23 gives the relative change of a trained model ’s own float32 prediction when the same problem is given in another presentation . The norm is the measure-weighted norm over the evaluated cells or points, and “back” undoes on the output: vectors are rotated back, rows are put back in order, and units are converted back. Entries are medians over 8 evaluation cases of DrivAerML or test cases of AeroZoo, with 3 poses per case for rigid motions. acts on a case as the dataset stores it, before the data pipeline; the pipeline centers each DrivAerML case, so no model receives a pure translation there. With weights, inputs, and pipeline cast to float64, every exact invariance of Theorem 1 holds to at most on the surface task, and query independence to at most in the interior.
The surface task. In float32 a rigid motion moves its pressure by and its wall shear stress by . A move of every coordinate by one unit in its last place, at random up or down, moves them by as much ( and ), and a rescaling that is exact in binary arithmetic (doubling) by . The float32 rows therefore measure the rounding of the transformed input: where cells of less than a millimeter lie at coordinates of meters, their normals and areas change in the fourth digit. Transolver and GeoTransolver respond to the last-place move by and . A conversion of units in which every dimensional input and the dataset’s reference length are converted together leaves the baselines unchanged to the same rounding, because their data pipeline divides by that reference length. Doubling the geometry at a fixed Reynolds number, with the dataset’s constants left as they are, changes their predictions by 0.65 and 0.62.
When the cells are drawn by another law, the predictions can be compared on the cells that both draws contain. Between two draws from the dataset’s own law that share half of their cells, the predictions on the shared cells differ by for Transolver, for GeoTransolver, and for ISLA. No model is exactly consistent between two samples, and ISLA is somewhat less so than the baselines. Between a draw from the dataset’s law and a draw by area, the differences are 0.28, 0.29, and : ISLA’s is no larger than between two draws from the dataset’s law.
The interior tasks. Query independence is exact: a point predicted alone, or in other chunks, receives the same value to rounding. The other rows include the model’s own draw of boundary tokens. On DrivAerML, a rigid motion in float32 changes 9% of the drawn tokens, through the rounding of the frame in which the cells are ordered, and the predicted velocity moves by ; with the draw held at that of the first presentation it moves by . A last-bit move of every coordinate moves the predicted pressure by , more than a rigid motion does: it flips the computed side of the boundary for about 710 of the 200,000 query points and turns the direction to the boundary by more than 0.1 rad at 0.4% of them. Splitting every cell changes which tokens are drawn (91% of them), because the budget is shared among the conditions by cell count, and moves the velocity by , in float64 as in float32. This is the difference between two draws of the same budget (), which is the sampling error of tokens for cells. With a fixed budget this error does not vanish under refinement, so the interior form as run is consistent only up to it (Proposition 2). On AeroZoo, whose boundaries are smaller, a rigid motion changes few tokens and moves the velocity by ; across all 538 evaluation cases (290 of AeroZoo full, 200 of AirfRANS, and 48 of DrivAerML; Section C.3), zero cases hit an exact frame tie in float64, and on 128 near-degenerate AeroZoo validation cases across 5 float32 poses, mean and change by at most 0.05% (median per case 0.01%).
Cost and Scaling
Figure 9 and Table 24 and Table 25 give the cost of a prediction. Time and memory are linear in the number of surface cells for all three models above about cells. Uncompiled, ISLA’s time lies between those of GeoTransolver and Transolver in float32 and below both in bfloat16 (Table 24 and Section C.3). For the interior task, the boundary side (the token draw, the twelve blocks, and the tree) takes 0.58 s in float32 and 0.32 s in bfloat16, once per case, and each million query points adds 3.3 s or 1.2 s. Query points are processed in chunks, so memory grows slowly: points take 50 GiB. In a separate run, points of one DrivAerML case, whose volume mesh has cells, were predicted in 7 minutes at 72 GiB, and all 2.2 million cells of an AeroZoo case in 4.8 s. Table 7 in Section C.6 gives the cost of training.
Figure 9. Cost of one prediction as a function of problem size, on one GPU for one DrivAerML case. (a, b) The surface task, with every model uncompiled. (c) ISLA’s interior task with boundary tokens: the boundary side is a fixed cost per case, and the rest is linear in the number of query points.
| Query points | Inference, float32 time [s] | Inference, float32 peak mem. [GiB] | Inference, bfloat16 time [s] | Inference, bfloat16 peak mem. [GiB] |
| 0.63 | 16 | 0.35 | 16 | |
| 0.96 | 16 | 0.47 | 16 | |
| 3.98 | 48 | 1.57 | 31 | |
| 33.75 | 50 | 11.99 | 32 |
Table 25. Time and memory of ISLA’s interior prediction for one DrivAerML case on one GPU, by the number of query points ( boundary tokens drawn from boundary cells; blocks compiled).
Ablations
For Table 26 we remove one design rule at a time and retrain the reference configuration with nothing else changed, on the DrivAerML surface task of the pose and discretization tests, and on AirfRANS for the boundary read (3 seeds for the first three variants and 2 for the last two). On DrivAerML each ratio divides by the mean of the reference’s 4 seeds, which lie within 3.3% of it; a fresh run of its first seed differs from that seed by 0.2%.
Counts in place of measures: within each boundary condition, every cell is given the condition’s mean measure, so that sums over tokens weight cells equally, as a model without quadrature weights does; the loss and the metric keep the true measures. The error on cells as stored is 0.5% lower, well inside the spread of the reference’s seeds. On cells re-sampled by area it is 43%, 11 times the reference’s.
Coordinates as inputs: the three coordinates of each cell’s centroid, centered and scaled as the baselines’ pipeline does, are added to the token’s input. The error on cases as stored is 3.6% lower than the reference’s (3 seeds), a difference near the spread of the reference’s seeds. On rotated cases it is 83%.
No relations to anchors: the relational invariants of Equation (5) are set to zero in every block, which removes all positional information from the boundary tokens. The error is 71% in every presentation.
Body alone as the boundary: the boundaries of the tunnel are removed from the input, so the model receives the car’s surface alone, and its length unit (Equation (2)) becomes that of the car. The error is 2% lower than the reference’s in every presentation, and both seeds of this variant lie within the range of the reference’s seeds. On this task the tunnel’s boundaries do not lower the error.
No boundary read: the read of Equation (9) is removed from the interior form, which leaves the slices as a query’s only access to the boundary. On AirfRANS the velocity error is 10% higher than the reference’s and the pressure error 8% higher, and the eddy-viscosity error is 8% lower. The reference’s two seeds lie within 2%, 4%, and 6% of their mean in these three fields. Both seeds of the variant lie outside that spread in velocity (ratios 1.07 and 1.13) and in eddy viscosity (0.92 and 0.93), but not in pressure (1.01 and 1.15). We have no fresh run of the AirfRANS reference.
Exact boundary read (): the AirfRANS interior model is trained and evaluated with the exact boundary read () in place of the tree (2 seeds). Its errors differ from the reference’s by at most 0.9% across pressure, velocity, and eddy viscosity (well within the reference’s two-seed spread), at 2.7 times the training step time and 75 vs. 26 GiB per GPU.
The opening angle at inference. The trained weights depend on the tree. The two AirfRANS models of the pose and discretization tests, trained at , have 1.5 to 2.4 times their error when evaluated at , and 2.0 to 5.1 times with the exact boundary read (): 2.3 times in pressure, 2.0 in velocity, and 5.1 in eddy viscosity. On AeroZoo (8 validation cases of full, points each), the two ISLA models trained at deviate from their own exact boundary read () by 2.5 and 2.8% in velocity and 7.2 and 7.3% in pressure (up to 14% on a single case, and 1.7% at ; models trained and evaluated at deviate by 1.4 and 1.6%), while their pressure error relative to the targets moves only from 27.3 and 28.3% at to 28.4 and 29.1% at . A model must be evaluated at the angle at which it was trained, and the guarantees of the exact boundary read do not transfer to a model trained at another angle.
Additional Figures
Figure 10 shows what the slices of a trained model hold, layer by layer. Figure 11 and Figure 12 show interior predictions on AirfRANS and DrivAerML on a regular grid in place of the cells of the mesh. Figure 13 and Figure 14 show the fields of the ensemble entries on WindsorML and AhmedML.

Figure 10. What the slices of a trained model hold. All 12 layers of a trained ISLA on one DrivAerML case (run 171 of the dataset); Figure 3 shows layers 3, 6, and 12 larger. Each surface point has the color of the slice that receives the largest share of its weight. In each panel the eight slices that hold the most area have colors of their own, and all other slices are shades of gray; colors are unrelated between panels. The number in each panel counts the slices that hold some point. From layer 3 to layer 12 the slices go from broad bands to compact patches to sets scattered over the car (Section E). Anchors are not drawn: an anchor is a mean over its slice’s tokens and need not lie in or near the slice’s territory.
Figure 11. An AirfRANS evaluation case. Top: pressure coefficient, formed with the inlet dynamic pressure. Bottom: velocity magnitude over the inlet speed. Left: the reference, which exists only at mesh nodes, so it is drawn as a triangulation of the 100,000 scored nodes of this case and never across the airfoil; the nodes are a random subset of the mesh, which makes the thin wake look jagged. Middle: ISLA, evaluated on a regular grid of 1,000 by 600 points. Right: the difference, taken at the scored nodes, with its own and much finer color scale. The case has an inlet speed of 44.6 m/s, an angle of attack of , and a Reynolds number of . Over its scored nodes, the relative error is 0.66% for pressure and 0.31% for velocity.
Figure 12. Figure 12. An interior field on a plane. ISLA predicts volume fields from the boundary surfaces alone (the car, the ground, and the tunnel), at any requested points. Here the points are a regular grid of 1,200 by 500 on the symmetry plane of one DrivAerML evaluation case (run 129 of the dataset), in place of the cells of its mesh. Rows: velocity magnitude over the free-stream speed in the reference (the stored cells within 16 mm of the plane, each grid point showing its nearest cell) and from ISLA, and the magnitude of their difference at the stored cells. The car’s section is gray, and the ticks give the streamwise coordinate. Bottom: the same three fields in the near wake (the outlined window). Over the 200,000 scored points of this case, the relative error is 5.5% for velocity and 4.0% for pressure.
Figure 13. Figure 13. WindsorML fields. What the ensemble entry predicts on a case it was not trained on: run 8 of WindsorML, a validation case of the full split. Top: the surface pressure coefficient on the body, seen from behind. Bottom: the velocity magnitude over the reference speed (40 m/s) on the body’s center plane , with the body’s outline; the window is three body lengths wide. The inflow is yawed by about 2.5 degrees, so this plane is not a symmetry plane of the flow. Left: the dataset’s time-averaged simulation. Right: the ISLA entry, which is the equal-weight mean of five surface networks and of five volume networks (Section C). ISLA predicts both fields from the boundary alone. The two columns share one color scale in each row.
Figure 14. Figure 14. AhmedML fields. What the ensemble entry predicts on a case it was not trained on: run 24 of AhmedML, a validation case of the full split. Top: the surface pressure coefficient on the body, seen from behind, so that the slant and the base are in view. Bottom: the velocity magnitude over the inlet speed on the center plane , with the body’s outline; the window is three body lengths wide. Left: the dataset’s time-averaged simulation. Right: the ISLA entry, which is the equal-weight mean of three surface networks and of eight volume networks (Section C). ISLA predicts both fields from the boundary alone. The two columns share one color scale in each row.
Slices by layer. The slices of Figure 10 change character with depth. The first two layers pool the whole car: in layer 1, three slices share every point almost equally (weights between 0.32 and 0.34), and in layer 2 one slice holds 98% of the weight. Layer 3 has 5 broad bands, and the middle layers have tens of compact patches, which begin to recur in separate places by layer 9. In the last layer the slices are many and each is scattered over the car: there a slice groups tokens by their state and no longer by their place. The mean radius of a slice’s territory (the root-mean-square distance of its points from their centroid on one side of the car, averaged over slices by area) is 0.4 m in layer 6 and 1.1 m in layer 12, compared with 1.3 m for that side of the car as a whole. No coordinate enters the routing, only invariant relations to the anchors and the learned state.
Extended Related Work
Transformers for PDEs on general geometry. Operator learning on sensors and graphs [66] (Lu et al. 2021) came before transformers. Transformers reach industrial meshes by compressing points into a small latent set: supernodes and anchor tokens [4, 5], or learned slices [60, 99, 3] (Wu et al. 2024). For slices, Wen & Mishra [92] read the slice average as a quadrature, and with Hu et al. (2026) show that Transolver does not need its attention among slices. ISLA has none. These models take coordinates as features, and their sums over points or anchors carry no measure, so their limit follows the density of the points that they are given. ISLA keeps slice routing and changes its inputs, which are invariants in place of coordinates, and its weights, which are surface measures.
Symmetry. An invariance that a model lacks must come from training examples, at a cost in data that is strict for linear models [31] and known exactly for kernel regression [85]. Two classical routes to geometric symmetry [24] constrain the layers or the inputs. The first constrains every layer through steerable or Clifford representations [86, 91, 33, 76, 35, 53, 22], and it has been applied to PDEs and flow on surfaces and in volumes [21, 84, 73]. The closest such model to ISLA in purpose is AB-GATr [73], an equivariant transformer on a subset of the points with read-only queries, which its authors train in two to eleven times the time of its non-equivariant counterpart. Even with specialized tensor-product kernels (Geiger & Smidt, 2022; cuEquivariance Contributors, 2025), carrying steerable representations through every layer adds memory and compute at industrial token counts. ISLA takes the second route: standard scalar matrix multiplications run throughout the trunk, invariant scalars go in, and invariant coefficients times covariant vectors come out at the head, following the classical representation of isotropic tensor functions [68, 98, 55, 87].
The invariants are formed relative to anchors, which have two close relatives. In Invariant Slot Attention [16], each slot of an image model has a position and a scale, and its inputs are expressed in the slot’s frame. In equivariant neural fields [93], pose-conditioned latents are related to query points through pair invariants of the kind that Bekkers et al. [15] use for message passing. ISLA relates tokens to anchors by rotation-invariant scalars, uses the mean normal as the anchor’s orientation, and forms anchors with the surface measure. No learned map receives a frame, unlike in frame averaging and learned canonicalization [71, 48]. A canonical frame cannot be both continuous and equivariant [29]; ISLA uses one only to group cells for its tree (Section B.7).
Invariants of distances and angles are not a complete set [69], so we claim no universality. Nor do we claim that built-in symmetry always helps: whether it is better than augmentation depends on the data [23, 73], and § 5.2 adds evidence from both regimes.
Units and similarity. Dimensional analysis and similarity scaling [25, 11] have been built into learning by nondimensional features [10, 88], and scale equivariance into mesh networks through the local mesh spacing [42]. GLOBES is invariant to units whenever supplies reference lengths [80]. ISLA instead computes its length unit from the boundary measure, so no reference length is an input. That unit, times the radius of gyration of the surface, is the size measure of statistical shape analysis.
Discretization. Neural operators are defined to converge under refinement [51], but analyses and measurements show that the property often fails [14, 34] (Sakavardia et al. 2026). DiffusionNet tests agreement between meshes of one surface [79]. For current surrogates, sensitivity to the sampling of the inputs has been observed for anchor tokens [5]. Wen & Mishra [92] find that the error of a trained Transolver has a relative standard deviation of at most 0.18% over six random subsamples drawn by one law; the tests of § 5.2 draw by another law.
ISLA makes every sum over the boundary a quadrature, an idea with several precedents. With surface measures, a weighted softmax takes the self-normalized kernel-regression form of Nadaraya [64] and Watson [90]; reading unweighted attention as a quadrature of an integral operator goes back to Cao [26]. The log-measure bias that we use is the proportional attention of token merging [17]. Yang et al. [95] apply it, with surface area as the weight, to a softmax over surface points into latent tokens, and compare it with a variant without the weight under non-uniform re-sampling. We apply the bias to the routing softmax of slice attention with one normalization per boundary condition, and draw tokens inside the forward pass as a Hájek ratio estimator with Horvitz–Thompson weights from a systematic design [61] [43, 40, 83].
From the boundary to the field. The interior form of ISLA draws on three lines of work. The first is boundary integral equations and aerodynamic panel methods: representation formulas and Steklov–Poincaré (Calderón) operators for elliptic systems and viscous flows [70, 62, 82, 52], panel and boundary-element discretizations [41, 50, 77], rigorous sample-complexity results for learning elliptic Green’s operators from their off-diagonal low-rank structure [78, 19, 20], and neural boundary-integral surrogates [54, 89]. GLOBE [80] superposes learned kernels from boundary faces, with a tree for distant faces, and ISLA’s boundary read follows it. ISLA adds learned slices for global mixing and exact distances to the boundary for near-wall resolution, which GLOBE omits by design. The second is decoding at independent query points, which is standard [46] (Lu et al. 2021). Queries that receive a state and contribute to none are used by AB-UPT [5] and SMART [39], and Transolver-3 [99] decodes full meshes from cached slice states. ISLA’s queries receive states that the boundary alone forms, and also attend to every boundary token. The third is the hierarchical summary of distant sources, from elliptic-integral treecodes, -body methods, -matrices, and parallel Morton BVHs [72, 13, 37, 38, 49] to hierarchical transformers [97].
References
- [3]Adams, C., Ranade, R., Nidhan, S., Nabian, M. A., Akhare, D., Cherukuri, R., and Choudhry, S. GeoTransolver: Learning physics on irregular domains using multi-scale geometry aware physics attention transformer. In Advances in Neural Information Processing Systems, 2026. URL https://arxiv.org/abs/2512.20399. To appear.
- [4]Alkin, B., Fürst, A., Schmid, S., Gruber, L., Holzleitner, M., and Brandstetter, J. Universal physics transformers: A framework for efficiently scaling neural operators. In Advances in Neural Information Processing Systems, 2024. URL https://arxiv.org/abs/2402.12365.
- [5]Alkin, B., Bleeker, M., Kurle, R., Kronlachner, T., Sonnleitner, R., Dorfer, M., and Brandstetter, J. AB-UPT: Scaling neural CFD surrogates for high-fidelity automotive aerodynamics simulations via anchored-branched universal physics transformers. Transactions on Machine Learning Research, 2025. URL https://arxiv.org/abs/2502.09692.
- [6]Ashton, N., Angel, J. B., Ghate, A. S., Kenway, G. K. W., Wong, M. L., Kiris, C., Walle, A., Maddix, D. C., and Page, G. WindsorML: High-fidelity computational fluid dynamics dataset for automotive aerodynamics. In Advances in Neural Information Processing Systems, volume 37, pp. 37823–37835, 2024a. doi: 10.52202/079017-1194. URL https://arxiv.org/abs/2407.19320. Datasets and Benchmarks Track.
- [7]Ashton, N., Maddix, D. C., Gundry, S., and Shabestari, P. M. AhmedML: High-fidelity computational fluid dynamics dataset for incompressible, low-speed bluff body aerodynamics. arXiv preprint arXiv:2407.20801, 2024b. URL https://arxiv.org/abs/2407.20801.
- [8]Ashton, N., Mockett, C., Fuchs, M., Fliessbach, L., Hetmann, H., Knacke, T., Schonwald, N., Skaperdas, V., Fotiadis, G., Walle, A., Hupertz, B., and Maddix, D. DrivAerML: High-fidelity computational fluid dynamics dataset for road-car external aerodynamics. arXiv preprint arXiv:2408.11969, 2024c. URL https://arxiv.org/abs/2408.11969.
- [9]Ashton, N., Clark, A., Goc, K., Heidt, L., Ivey, C., Agrawal, R., Bose, S. T., Adams, C., Sharpe, P., Leibovici, D., Akkurt, S., Nidhan, S., Ranade, R., and Kossaifi, J. HiLiftAeroML: A high-fidelity computational fluid dynamics dataset for high-lift aircraft aerodynamics. arXiv preprint arXiv:2605.19565, 2026. URL https://arxiv.org/abs/2605.19565.
- [10]Bakarji, J., Callaham, J., Brunton, S. L., and Kutz, J. N. Dimensionally consistent learning with Buckingham Pi. Nature Computational Science, 2(12):834–844, 2022. doi: 10.1038/s43588-022-00355-5.DOI
- [11]Barenblatt, G. I. Scaling, Self-similarity, and Intermediate Asymptotics. Cambridge University Press, Cambridge, 1996. doi: 10.1017/CBO9781107050242.DOI
- [12]Barill, G., Dickson, N. G., Schmidt, R., Levin, D. I. W., and Jacobson, A. Fast winding numbers for soups and clouds. ACM Transactions on Graphics, 37(4):1–12, 2018. doi: 10.1145/3197517.3201337.DOI
- [13]Barnes, J. and Hut, P. A hierarchical O(N log N) force-calculation algorithm. Nature, 324(6096):446–449, 1986. doi: 10.1038/324446a0.DOI
- [14]Bartolucci, F., de Bézenac, E., Raonić, B., Molinaro, R., Mishra, S., and Alaifari, R. Representation equivalent neural operators: a framework for alias-free operator learning. In Advances in Neural Information Processing Systems, volume 36, 2023. URL https://proceedings.neurips.cc/paper_files/paper/2023/hash/dc35c593e61f6dfc6d5b411b976d09dcf-Abstract-Conference.html.proceedings.neurips.cc/paper_files/paper/2023/hash/dc35c593e61f6dfc6d5b411b976d09dcf-Abstract-Conference.html
- [15]Bekkers, E. J., Vadgama, S., Hesselink, R. D., van der Linden, P. A., and Romero, D. W. Fast, expressive SE(n) equivariant networks through weight-sharing in position-orientation space. In International Conference on Learning Representations, 2024. URL https://arxiv.org/abs/2310.02970.
- [16]Biza, O., van Steenkiste, S., Sajjadi, M. S. M., Elsayad, G. F., Mahendran, A., & Kipf, T. (2023). Invariant slot attention: Object discovery with slot-centric reference frames. In Proceedings of the 40th International Conference on Machine Learning (Vol. 202, pp. 2507–2527). https://arxiv.org/abs/2302.04973
- [17]Bolya, D., Fu, C.-Y., Dai, X., Zhang, P., Feichtenhofer, C., and Hoffman, J. Token merging: Your ViT but faster. In International Conference on Learning Representations, 2023. URL https://arxiv.org/abs/2210.09461.
- [18]Bonnet, F., Mazari, A. J., Cinnella, P., and Gallinari, P. AirFRANS: High fidelity computational fluid dynamics dataset for approximating Reynolds-averaged Navier–Stokes solutions. In Advances in Neural Information Processing Systems, 2022. URL https://arxiv.org/abs/2212.07564. Datasets and Benchmarks Track.
- [19]Boullé, N. and Townsend, A. Learning elliptic partial differential equations with randomized linear algebra. Foundations of Computational Mathematics, 23:709–739, 2023. doi: 10.1007/s10208-022-09556-w.DOI
- [20]Boullé, N., Halikias, D., and Townsend, A. Elliptic PDE learning is provably data-efficient. Proceedings of the National Academy of Sciences, 120(39):e2303904120, 2023. doi: 10.1073/pnas.2303904120.DOI
- [21]Brandstetter, J., Hesselink, R., van der Pol, E., Bekkers, E. J., and Welling, M. Geometric and physical quantities improve E(3) equivariant message passing. In International Conference on Learning Representations, 2022. URL https://arxiv.org/abs/2110.02905.
- [22]Brehmer, J., de Haan, P., Behrends, S., and Cohen, T. Geometric algebra transformer. In Advances in Neural Information Processing Systems, 2023. URL https://arxiv.org/abs/2305.18415.
- [23]Brehmer, J., Behrends, S., de Haan, P., and Cohen, T. Does equivariance matter at scale? Transactions on Machine Learning Research, 2025. URL https://arxiv.org/abs/2410.23179.
- [24]Bronstein, M. M., Bruna, J., Cohen, T., and Veličković, P. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13478, 2021. URL https://arxiv.org/abs/2104.13478.
- [25]Buckingham, E. On physically similar systems; illustrations of the use of dimensional equations. Physical Review, 4(4):345–376, 1914. doi: 10.1103/PhysRev.4.345.DOI
- [26]Cao, S. Choose a transformer: Fourier or Galerkin. In Advances in Neural Information Processing Systems, 2021. URL https://arxiv.org/abs/2105.14995.
- [29]Dym, N., Lawrence, H., and Siegel, J. W. Equivariant frames and the impossibility of continuous canonicalization. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 12228–12267, 2024. URL https://arxiv.org/abs/2402.16077.
- [30]Dziuk, G. and Elliott, C. M. Finite element methods for surface PDEs. Acta Numerica, 22:289–396, 2013. doi: 10.1017/S0962492913000056.DOI
- [31]Elesedy, B., & Zaidi, S. (2021). Provably strict generalisation benefit for equivariant models. Proceedings of the 38th International Conference on Machine Learning, 139, 2959–2969. https://arxiv.org/abs/2101.10333
- [32]FluidsBench. (2026). FluidsBench: Open benchmarks for scientific machine learning. https://fluidsbench.org/fluidsbench.org
- [33]Fuchs, F. B., Worrall, D. E., Fischer, V., and Welling, M. SE(3)-transformers: 3D roto-translation equivariant attention networks. In Advances in Neural Information Processing Systems, 2020. URL https://arxiv.org/abs/2006.10503.
- [34]Gao, W., Xu, R., Deng, Y., & Liu, Y. (2025). Discretization-invariance? On the discretization mismatch errors in neural operators. International Conference on Learning Representations, 19360–19384. https://proceedings.iclr.cc/paper_files/paper/2025/hash/313829757739365201b5adb3a1cbd9bd-Abstract-Conference.htmlproceedings.iclr.cc/paper_files/paper/2025/hash/313829757739365201b5adb3a1cbd9bd-Abstract-Conference.html
- [35]Geiger, M. and Smidt, T. e3nn: Euclidean neural networks. arXiv preprint arXiv:2207.09453, 2022. URL https://arxiv.org/abs/2207.09453.
- [36]Grandemange, M., Gohlke, M., and Cadot, O. Turbulent wake past a three-dimensional blunt body. Part 1. Global modes and bi-stability. Journal of Fluid Mechanics, 722:51–84, 2013. doi: 10.1017/jfm.2013.83.DOI
- [37]Greengard, L. and Rokhlin, V. A fast algorithm for particle simulations. Journal of Computational Physics, 73(2):325–348, 1987. doi: 10.1016/0021-9991(87)90140-9.DOI
- [38]Hackbusch, W. (1989). A sparse matrix arithmetic based on $\mathcal{H}$-matrices. Part I: Introduction to $\mathcal{H}$-matrices. Computing, 62(2), 89–108. https://doi.org/10.1007/s00607050015
- [39]Hagnberger, J. and Niepert, M. SMART: Scalable mesh-free aerodynamic simulations from raw geometries using a transformer-based surrogate model. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pp. 39376–39420, 2026. URL https://arxiv.org/abs/2601.18707.
- [40]Hájek, J. (1971). Comment on “An essay on the logical foundations of survey sampling, part one”. In Foundations of Statistical Inference (p. 236).
- [41]Hess, J. L. and Smith, A. M. O. Calculation of potential flow about arbitrary bodies. Progress in Aerospace Sciences, 8:1–138, 1967. doi: 10.1016/0376-0421(67)90003-6.DOI
- [42]Horie, M. and Mitsume, N. Graph neural PDE solvers with conservation and similarity-equivariance. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 18785–18814, 2024. URL https://arxiv.org/abs/2405.16183.
- [43]Horvitz, D. G. and Thompson, D. J. A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association, 47(260):663–685, 1952. doi: 10.1080/01621459.1952.10483446.DOI
- [45]Jacobson, A., Kavan, L., and Sorkine-Hornung, O. Robust inside-outside segmentation using generalized winding numbers. ACM Transactions on Graphics, 32(4):1–12, 2013. doi: 10.1145/2461912.2461916.DOI
- [46]Jaegle, A., Borgeaud, S., Alayrac, J.-B., Doersch, C., Ionescu, C., Ding, D., Koppula, S., Zoran, D., Brock, A., Shelhamer, E., Hénaff, O., Botvinick, M. M., Zisserman, A., Vinyals, O., and Carreira, J. Perceiver IO: A general architecture for structured inputs & outputs. In International Conference on Learning Representations, 2022. URL https://arxiv.org/abs/2107.14795.
- [47]Jordan, K., Jin, Y., Boza, V., You, J., Cesista, F., Newhouse, L., & Bernstein, J. (2024). Muon: An optimizer for hidden layers in neural networks. https://kellerjordan.github.io/posts/muon/kellerjordan.github.io/posts/muon
- [48]Kaba, S.-O., Mondal, A. K., Zhang, Y., Bengio, Y., and Ravanbakhsh, S. Equivariance with learned canonicalization functions. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 15546–15566, 2023. URL https://arxiv.org/abs/2211.06489.
- [49]Karras, T. (2012). Maximizing parallelism in the construction of BVHs, octrees, and k-d trees. Proceedings of the Fourth ACM SIGGRAPH / Eurographics Conference on High-Performance Graphics, 33–37. https://dl.acm.org/doi/abs/10.5555/2383795.2383801dl.acm.org/doi/abs/10.5555/2383795.2383801
- [50]Katz, J. and Plotkin, A. Low-Speed Aerodynamics. Cambridge University Press, Cambridge, 2 edition, 2001. doi: 10.1017/CBO9780511810329.DOI
- [51]Kovachki, N., Li, Z., Liu, B., Azizzadenesheli, K., Bhattacharya, K., Stuart, A., & Anandkumar, A. (2023). Neural operator: Learning maps between function spaces with applications to PDEs. Journal of Machine Learning Research, 24(89), 1 ….
- [52]Kress, R. Linear Integral Equations. Applied Mathematical Sciences. Springer, New York, 3 edition, 2014. doi: 10.1007/978-1-4614-9593-2.DOI
- [53]Liao, Y.-L. and Smidt, T. Equiformer: Equivariant graph attention transformer for 3D atomistic graphs. In International Conference on Learning Representations, 2023. URL https://arxiv.org/abs/2206.11990.
- [54]Lin, G., Chen, F., Hu, P., Chen, X., Chen, J., Wang, J., and Shi, Z. BI-GreenNet: Learning Green’s functions by boundary integral network. Communications in Mathematics and Statistics, 11(1):103–129, 2023. doi: 10.1007/s40304-023-00338-6.DOI
- [55]Ling, J., Kurzawski, A., and Templeton, J. Reynolds averaged turbulence modelling using deep neural networks with embedded invariance. Journal of Fluid Mechanics, 807:155–166, 2016. doi: 10.1017/jfm.2016.615.DOI
- [56]Liu, Q., Zhong, W., Meidani, H., Abueidda, D., Koric, S., and Geubelle, P. Geometry-informed neural operator transformer. arXiv preprint arXiv:2504.19452, 2025. URL https://arxiv.org/abs/2504.19452.
- [57]Loshchilov, I. and Hutter, F. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019. URL https://arxiv.org/abs/1711.05101.
- [59]Cloud, L. (2025). SHIFT-SUV: High-fidelity computational fluid dynamics dataset for SUV external aerodynamics. https://huggingface.co/datasets/luminary-shift/SUVhuggingface.co/datasets/luminary-shift/SUV
- [60]Luo, H., Wu, H., Zhou, H., Xing, L., Di, Y., Wang, J., and Long, M. Transolver++: An accurate neural solver for PDEs on million-scale geometries. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pp. 41432–41449. PMLR, 2025. URL https://proceedings.mlr.press/v267/luo25o.html.proceedings.mlr.press/v267/luo25o.html
- [61]Madow, W. G. On the theory of systematic sampling, II. The Annals of Mathematical Statistics, 20(3):333–354, 1949. doi: 10.1214/aoms/1177729988.DOI
- [62]McLean, W. Strongly Elliptic Systems and Boundary Integral Equations. Cambridge University Press, Cambridge, 2000.
- [64]Nadaraya, E. A. On estimating regression. Theory of Probability and Its Applications, 9(1):141–142, 1964. doi: 10.1137/1109020.DOI
- [65]Olver, P. J. Applications of Lie Groups to Differential Equations, volume 107 of Graduate Texts in Mathematics. Springer, New York, 2 edition, 1993. doi: 10.1007/978-1-4612-4350-2.DOI
- [66]Pfaff, T., Fortunato, M., Sanchez-Gonzalez, A., and Battaglia, P. W. Learning mesh-based simulation with graph networks. In International Conference on Learning Representations, 2021. URL https://arxiv.org/abs/2010.03409.
- [67]Contributors, P. (2023). NVIDIA PhysicsNeMo: An open-source framework for physics-based deep learning in science and engineering. <a href="https://doi.org/https://github.com/NVIDIA/physicsnemo" rel="noopener noreferrer" target="_blank">https://doi.org/https://github.com/NVIDIA/physicsnemo</a>
- [68]Pope, S. B. A more general effective-viscosity hypothesis. Journal of Fluid Mechanics, 72(2):331–340, 1975. doi: 10.1017/S0022112075003382.DOI
- [69]Pozdnyakov, S. N., Willatt, M. J., Bartók, A. P., Ortner, C., Csányi, G., and Ceriotti, M. Incompleteness of atomic structure representations. Physical Review Letters, 125:166001, 2020. doi: 10.1103/PhysRevLett.125.166001.DOI
- [70]Pozrikidis, C. Boundary Integral and Singularity Methods for Linearized Viscous Flow. Cambridge University Press, Cambridge, 1992. doi: 10.1017/CBO9780511624124.DOI
- [71]Puny, O., Atzmon, M., Ben-Hamu, H., Misra, I., Grover, A., Smith, E. J., and Lipman, Y. Frame averaging for invariant and equivariant network design. In International Conference on Learning Representations, 2022. URL https://arxiv.org/abs/2110.03336.
- [72]Rokhlin, V. Rapid solution of integral equations of classical potential theory. Journal of Computational Physics, 60 (2):187–207, 1985. doi: 10.1016/0021-9991(85)90002-6.DOI
- [73]Rygiel, P., Suk, J., Yeung, K. K., Brune, C., and Wolterink, J. M. When does equivariance help? canonical alignment in neural fluid surrogates. arXiv preprint arXiv:2605.18816, 2026. URL https://arxiv.org/abs/2605.18816.
- [75]Särndal, C.-E., Swensson, B., and Wretman, J. Model Assisted Survey Sampling. Springer Series in Statistics. Springer, New York, 1992. doi: 10.1007/978-1-4612-4378-6.DOI
- [76]Satorras, V. G., Hoogeboom, E., and Welling, M. E(n) equivariant graph neural networks. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 9323–9332, 2021. URL https://arxiv.org/abs/2102.09844.
- [77]Sauter, S. A. and Schwab, C. Boundary Element Methods. Springer Series in Computational Mathematics. Springer, Berlin, Heidelberg, 2011. doi: 10.1007/978-3-540-68093-2.DOI
- [78]Schäfer, F. and Owhadi, H. Sparse recovery of elliptic solvers from matrix-vector products. SIAM Journal on Scientific Computing, 46(2):A998–A1025, 2024. doi: 10.1137/22M154226X.DOI
- [79]Sharp, N., Attaiki, S., Crane, K., and Ovsjanikov, M. DiffusionNet: Discretization agnostic learning on surfaces. ACM Transactions on Graphics, 41(3):1–16, 2022. doi: 10.1145/3507905.DOI
- [80]Sharpe, P. GLOBE: Accurate surrogates for boundary-driven PDEs via domain-inspired architectures and equivariance. In Advances in Neural Information Processing Systems, 2026. URL https://arxiv.org/abs/2511.15856. To appear.
- [81]Sharpe, P., Gonzales, C., Adams, C., Ataei, M., and Choudhry, S. AeroZoo: A benchmark of well-posed flow problems for foundation models of aerodynamics, 2026. arXiv preprint.arxiv.org/abs/2611.17152
- [82]Steinbach, O. Numerical Approximation Methods for Elliptic Boundary Value Problems. Springer, New York, 2008. doi: 10.1007/978-0-387-68805-3.DOI
- [83]Stevens, D. L. and Olsen, A. R. Spatially balanced sampling of natural resources. Journal of the American Statistical Association, 99(465):262–278, 2004. doi: 10.1198/016214504000000250.DOI
- [84]Suk, J., Imre, B., and Wolterink, J. M. LaB-GATr: geometric algebra transformers for large biomedical surface and volume meshes. In Medical Image Computing and Computer Assisted Intervention, pp. 185–195, 2024. doi: 10.1007/978-3-031-72390-2_18. URL https://arxiv.org/abs/2403.07536.
- [85]Tahmasebi, B. and Jegelka, S. The exact sample complexity gain from invariances for kernel regression. In Advances in Neural Information Processing Systems, 2023. URL https://arxiv.org/abs/2303.14269.
- [86]Thomas, N., Smidt, T., Kearnes, S., Yang, L., Li, L., Kohlhoff, K., and Riley, P. Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds. arXiv preprint arXiv:1802.08219, 2018. URL https://arxiv.org/abs/1802.08219.
- [87]Villar, S., Hogg, D. W., Storey-Fisher, K., Yao, W., & Blum-Smith, B. (2021). Scalars are universal: Equivariant machine learning, structured like classical physics. Advances in Neural Information Processing Systems, 34, 28848–28863. https://arxiv.org/abs/2106.06610
- [88]Villar, S., Yao, W., Hogg, D. W., Blum-Smith, B., and Dumitrascu, B. Dimensionless machine learning: Imposing exact units equivariance. Journal of Machine Learning Research, 24:1–32, 2023. URL https://arxiv.org/abs/2204.00887.
- [89]Wang, H., Li, J., Dwivedi, A., Hara, K., and Wu, T. BENO: Boundary-embedded neural operators for elliptic PDEs. In International Conference on Learning Representations, 2024. URL https://arxiv.org/abs/2401.09323.
- [90]Watson, G. S. Smooth regression analysis. Sankhya: The Indian Journal of Statistics, Series A, 26:359–372, 1964.
- [91]Weiler, M., Geiger, M., Welling, M., Boomsma, W., and Cohen, T. 3D steerable CNNs: Learning rotationally equivariant features in volumetric data. In Advances in Neural Information Processing Systems, 2018. URL https://arxiv.org/abs/1807.02547.
- [92]Wen, S. and Mishra, S. Does Transolver really need a transformer? arXiv preprint arXiv:2609.32525, 2026. URL https://arxiv.org/abs/2609.32525.
- [93]Wessels, D. R., Knigge, D. M., Papa, S., Valperga, R., Vadgama, S., Gavves, E., and Bekkers, E. J. Grounding continuous representations in geometry: Equivariant neural fields. In International Conference on Learning Representations, 2025. URL https://arxiv.org/abs/2406.05753.
- [95]Yang, Z., Du, T., and Liu, L. Learning Laplacian eigenspace with mass-aware neural operators on point clouds. In ACM SIGGRAPH 2026 Conference Papers, pp. 1–11, 2026. doi: 10.1145/3799902.3811185.DOI
- [96]Yoo, S., Jeong, L., Ra, J., Lee, D., Yang, S., Jeong, H., and Kang, N. DeepJEB++: Foundation model-driven large-scale 3D engineering dataset via 2D latent space augmentation. arXiv preprint arXiv:2606.12994, 2026. URL https://arxiv.org/abs/2606.12994.
- [97]Zhdanov, M., Welling, M., and van de Meent, J.-W. Erwin: A tree-based hierarchical transformer for large-scale physical systems. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pp. 78023–78040, 2025. URL https://arxiv.org/abs/2502.1719.
- [98]Zheng, Q.-S. Theory of representations for tensor functions—A unified invariant approach to constitutive equations. Applied Mechanics Reviews, 47(11):545–587, 1994. doi: 10.1115/1.3111066.DOI
- [99]Zhou, H., Wu, H., Shangguan, H., Ma, Y., Weng, H., Wang, J., and Long, M. Transolver-3: Scaling up transformer solvers to industrial-scale geometries. arXiv preprint arXiv:2602.04940, 2026. URL https://arxiv.org/abs/2602.04940.












