WorksComputer Science
Enforcing Symmetries on Pre-trained ML Models for Physics
Abstract
Post-hoc group averaging (or test-time augmentation) is a simple way to impose symmetry on a trained predictor without changing its architecture or retraining it. For finite and compact groups, averaging over a normalized Haar measure on the group gives the Reynolds operator: a canonical projection onto invariant or equivariant predictors with known non-increasing risk guarantees under certain assumptions. This paper provides numerical experiments on physics emulation models that demonstrate the effectiveness of the approach in this setting. We then extend this framework to non-compact group symmetries. For non-compact groups, the Haar measure cannot be normalized, so group averaging cannot be defined in the same way. We replace it with three alternatives: the Weyl unitary trick (which requires complexifying the space and extending the predictor), a data-dependent orthogonal projection (which involves a non-trivial optimization on the space of invariant or equivariant functions), and a frame averaging approach (which involves mapping each point to a set of orbit representatives). We illustrate the constructions theoretically and empirically on the unit-rescaling group and the Lorentz group. Our results show that in the non-compact case, the Weyl trick and frame averaging do not provide a consistent improvement over the baseline, except where the test data is out-of-distribution (an orders-of-magnitude improvement in this case) and, for the Weyl trick, in low-data regimes. The orthogonal projection can provide improvements in-distribution and out-of-distribution. We also observe that models trained directly on invariant features matched or beat the projection of an unconstrained one. We conclude that post-hoc symmetrization extends to non-compact groups, but it does not consistently outperform training a symmetric model.
Introduction
A common guiding principle in machine learning for the physical sciences is that predictive models should be consistent with the laws of physics. These laws are often manifestly symmetric, which constrains admissible observables and determines how physical quantities transform. When learning from data generated by a physical system, respecting these symmetries is not only conceptually natural, but can also improve robustness.
One way to incorporate symmetry into learning models is through invariant or equivariant architectures, which enforce symmetries by construction. Although such approaches have been successful in many settings, they are not always used in practice. Implementing equivariant models typically requires nontrivial representation-theoretic or invariant-theoretic machinery, and existing equivariant methods are not always compatible with state-of-the-art architectures. As a result, many modern machine learning models for physics prioritize well-understood, symmetry-agnostic architectures, relying on data to learn the physics rather than explicit symmetry constraints.
An alternative approach is to impose symmetry post hoc by projecting a trained model onto the space of symmetry-preserving functions [1, 2, 3, 4]. In this framework, a pretrained predictor is treated as an arbitrary function, and a new predictor is obtained by averaging over group orbits. For compact groups, this procedure is canonically implemented by Reynolds averaging with respect to the normalized Haar measure of the group (a.k.a. the invariant volume measure). Theoretical work shows that this projection yields a provably nonnegative, and in some cases strictly positive, improvement in expected risk under invariance assumptions on the data distribution, without requiring retraining or architectural modification [5]. However, this result applies only to compact groups, without results for non-compact ones. Yet, many physical symmetries are non-compact. For instance, the Lorentz group plays a central role in particle physics.
In this work, we first present empirical evidence showing that the group averaging approach works in the compact case. To this end, we take pretrained surrogate models released with The Well [6] and compare each with its group-averaged version. In many cases, this mere act of averaging, even when the group is small, already improves the results significantly. We then adapt the methodology to non-compact groups. We first argue why the approach and its guarantees fail, and we then investigate alternatives.
The remainder of the paper is organized as follows. § 2 reviews related work on invariant and equivariant learning, data augmentation, and post-hoc symmetrization. § 3 introduces Reynolds averaging for compact groups, recalls its risk-improvement guarantee, and evaluates it on pretrained foundation models for physics. § 4 explains the obstruction posed by non-compact groups and develops projection methods for unit rescalings and Lorentz transformations, together with numerical experiments. § 5 discusses the limitations and concludes.
Related work
Symmetry has long served as an inductive bias in machine learning for physical systems, where predictors are constrained by transformation laws; building symmetries into network structure dates back at least to [7]. A large body of work enforces these constraints directly in the architecture: group-equivariant convolutional networks [8], their generalization to arbitrary compact group actions [9, 10], steerable and irreducible-representation-based layers [11, 12, 13], equivariant multilayer perceptrons for arbitrary matrix groups, including the Lorentz group [14], and further exactly or approximately equivariant designs [15, 16, 17, 18]. A related group-averaging construction appears in variational quantum machine learning, where twirling projects gate generators onto the commutant of a unitary symmetry representation to construct equivariant quantum circuits [19]. A complementary invariant-theoretic viewpoint characterizes equivariant maps by generating scalar invariants and tensorial output transformation laws [20, 21, 22, 23]. This perspective is especially natural in physics, where scalar contractions often determine the allowed form of a model. All of these methods fix the symmetry constraint before training, by modifying the model class or the input representation.
A different route is to impose symmetry after training. Averaging a predictor over the group orbit is the classical symmetrization device, used, for example, to establish universality of invariant models [24] and to define permutation-invariant pooling [25]. For compact groups, averaging with respect to the normalized Haar measure gives the Reynolds operator, which projects an arbitrary predictor onto the invariant or equivariant class. When the data distribution and target are -invariant, this projection cannot increase squared risk and produces a strict improvement whenever the predictor has a nonzero non-equivariant component [5]. Related theoretical work quantifies the generalization and sample-complexity benefits of enforcing symmetry [26, 27, 28, 29].
On the empirical side, inexpensive post-hoc group averaging has been shown to improve already-trained surrogate models at test time, without retraining or architectural modification [30]. Several related methods impose symmetry without averaging over the entire group. Frame averaging evaluates a non-equivariant backbone over a small, input-dependent set of canonical frames and averages the resulting predictions [31, 32]. Learned canonicalization instead learns a transformation that maps each input to a canonical representative, applies a non-equivariant backbone, and transforms the output back [33]. Lie-algebra-based canonicalization extends this paradigm to continuous non-compact symmetry groups and can be applied to pre-trained neural operators [34]. Reynolds networks build the projection directly into the network architecture [4]. Probabilistic symmetrization instead learns an input-dependent probability distribution over group transformations. It evaluates the backbone in transformations sampled from this distribution, transforms the predictions back, and averages them. The distribution is designed to shift consistently when the input is transformed, making the exact expected prediction equivariant [35]. Finite sampled averages over transformations are closely related to data augmentation and test-time ensembling [2], and are used for non-compact groups as well: multi-crop and multi-scale evaluation average a trained network over translations and rescalings [36, 37], and Lorentz-boost augmentation has been used to train jet taggers [38]. Unlike compact Reynolds projection, such procedures depend on the chosen sampling distribution and need not define an exact projection.
Our compact-group experiments concern surrogate models for spatiotemporal dynamical systems. Learned surrogates have demonstrated strong performance across many physical domains [39, 40, 41, 42, 43, 44, 45, 46], and these successes have motivated large simulation benchmarks [47, 48], including the Well [6], which provides the datasets and pretrained models used in § 3.3.
Lorentz symmetry is central in particle physics and has motivated a substantial literature on symmetry-aware collider learning. Classical invariant theory implies that scalar polynomial invariants are generated by pairwise Minkowski products [49], this viewpoint has been developed for Lorentz- and permutation-invariant particle features and invariant-polynomial machine learning [50, 51, 20]. Early successful jet taggers represented jets as sets or clouds of particles, including energy-flow and particle-flow networks [52], ParticleNet [53], and transformer-based models such as the Particle Transformer [54]. Other methods incorporate relativistic structure explicitly: the Lorentz Layer introduced Lorentz-invariant combinations into top tagging [55]; the Lorentz Group Network builds equivariant layers from finite-dimensional Lorentz representations and tensor products [56]; LorentzNet uses Minkowski products and four-vector message passing [57]; the Covariant Particle Transformer gives a partially Lorentz-covariant transformer for top-quark reconstruction [58]; and PELICAN constructs permutation-equivariant, Lorentz-invariant or covariant aggregators [59]. More recent Clifford- and geometric-algebra architectures, including CGENN and L-GATr, use multivector representations to build Lorentz-aware networks for high-energy physics [60, 61, 62]. These works impose Lorentz structure by architecture design, whereas our focus is on post-hoc symmetrization of a given black-box predictor.
Since the Lorentz group is non-compact, the compact Reynolds-averaging argument does not apply directly. Similar issues arise for other non-compact physical symmetries, such as dimensional rescalings. In [63] the authors use the Buckingham–Π theorem to replace dimensional inputs by dimensionless invariant features [64] and they explain how to project functions onto the space of invariants using the Weyl unitarian trick. In § 4.6 we show how to apply the Weyl trick to the Lorentz group. Our Weyl unitarian Lorentz average can therefore be viewed as a post-hoc invariant projection for polynomial or holomorphic predictors.
A further complication is that collider tasks often exhibit only partial Lorentz symmetry. The beam axis, detector frame, fiducial cuts, jet definitions, and preprocessing choices can introduce preferred directions or frames. Recent Lorentz-equivariant architectures address this through controlled symmetry breaking, for example, by adding reference multivectors associated with the beam or detector frame [62]. This is in part related to broader work showing that augmentation or equivariance can be harmful when transformed examples are not representative of the relevant data distribution [65]. Our jet-tagging experiments should therefore not be interpreted as constructing a fully Lorentz-invariant predictor. Averaging over a compact subgroup enforces invariance only under that subgroup, while averaging over a bounded boost window depends on the chosen window and is not a group projection. Instead, these experiments test whether averaging over selected physically relevant transformations improves prediction.
Compact groups
Compact group averaging and Reynolds projection
We begin with the setting in which post-hoc symmetrization is canonical. Let be a group acting on a vector space , and write the action as
where , with the group of invertible linear maps , and is a matrix that specifies how elements in transform under . In particular is a group representation, and therefore it satisfies the transformation rules and .
We consider another real vector space equipped with a representation . A predictor is -equivariant if
In the special case where , this reduces to invariance: . In physical applications, equivariance expresses the requirement that transforming the input physical state and then predicting gives the same result as predicting first and then transforming the output.
Equivariance can be written as an invariance condition on the space of predictors. Define the induced action
Then is -equivariant if and only if
Thus the equivariant predictors are precisely the fixed points of the induced action .
Now assume that is compact. Then admits a normalized Haar probability measure . That is, a measure satisfying and the invariance property
for every measurable set and every . This allows us to define the Reynolds operator as a generalization of the concept of averaging over the group orbits
For a finite group, this is simply
In words, evaluate the trained model on transformed versions of the same input, transform the outputs back, and average.
The averaged predictor is exactly -equivariant. Indeed, for any ,
where the final equality follows from right invariance of Haar measure. Hence is fixed by every , and therefore satisfies the equivariance condition (1).
The statistical benefit of Reynolds averaging follows from its relation to orthogonal projections. Let be a -invariant probability distribution on . Here carries the inner product
If we further assume that the induced group action is unitary on we have that for all and . We define the equivariant subspace
The -orthogonal projection onto this subspace is
For a compact group, if is -invariant, the Reynolds operator coincides with this projection [5],
Consequently, under squared loss with a -equivariant target , the orthogonal decomposition gives
Thus under the assumptions above, Reynolds averaging cannot increase the expected risk and decreases it strictly whenever has a nonzero non-equivariant component, equivalently whenever in .
Application to surrogate models
Surrogate models estimate the state variables of a spatiotemporal dynamical system, such as a parameterized family of partial differential equations (PDEs) with fixed parameters or the observations of a physical system. The solution to a system of state variables over a spatial domain and for all times is represented as . The domain is typically a subset of two- or three-dimensional space, such as or . We employ a standard uniform discretization of the system in time and space and denote by the number of elements in the discretized domain . A snapshot represents the values of all state variables at all points in the discretized domain at a fixed time .
The surrogate model is an autoregressive prediction problem that maps a sequence of snapshots to the snapshot of the following time step such that
The goal of the model is to minimize some distance between its predictions and the truth . The initial conditions at are given by snapshots , so that the estimated next sequence is updated according to . The model is applied repeatedly to generate estimates for all desired times.
Our goal is to improve already-trained models that do not inherently respect some particular symmetry of the problem, by group averaging by the use of (3). In [5] it is proven that this averaging procedure reduces the error of the predictions. The assumptions to obtain provable improvements are:
The underlying physical problem is -invariant. This often holds because governing equations satisfy symmetries.
The distribution of the training data is -invariant.
Our goal is to predict the dynamics of a system with a known symmetry. We can improve a surrogate model as in (6) with initial sequence by group averaging
When the symmetry group is continuous, our approach approximates the continuous group average (2) using a Monte Carlo method with random samples from its elements. In the minimal case, for , the Monte Carlo approximation reduces to
where each is a randomly sampled group element of .
We provide numerical examples for surrogate models in the next Section. We argue that this simple procedure provides an efficient, zero-shot method that allows us to inject physical knowledge into a model after it has been trained.
Numerical experiments
We use the simulations and models published in “The Well” [6]. The project provides 15 TB of simulation data across 16 datasets, ranging from biological systems, fluid dynamics, acoustic scattering, as well as magneto-hydrodynamic simulations of extra-galactic fluids and supernova explosions. We chose to analyze models corresponding to datasets defined on a Cartesian 2D grid with symmetric initial and boundary conditions. For an overview of the groups used in this work, see Appendix § 6. The dataset properties and state variables are summarized in Appendix § 7.
The active matter dataset [6] consists of simulations of a continuum theory describing the dynamics of rod-like active particles immersed in a Stokes fluid. The simulations were run at constant particle density and varying values for the parameters dimensionless active dipole strength and strength of particle alignment through steric interactions. The initial conditions of the quantities are plane-wave perturbations about the isotropic state. The gray_scott_reaction_diffusion dataset (from now on abbreviated as gray_scott) contains numerical solutions of the Gray-Scott equations. They are a set of coupled reaction-diffusion equations that describe two chemical species, and , whose concentrations vary in space and time. Two parameters control the “feed” and “kill” rates in the reaction. The initial conditions are either a random Fourier series or random clusters of Gaussians. The boundary conditions are periodic for both datasets. The underlying physical problems are sampled on a square grid and have the symmetry of the torus . Toroidal symmetry allows all variables to translate periodically in both directions. Additionally, the problem is invariant under rotations and inversions, which is the discrete symmetry of the square .
Additionally we analyze two datasets on a rectangular, non-square grid. The boundary conditions are periodic in one direction and Neumann in the other. The rayleigh_benard dataset [67] contains simulations of convection between a hot and cold fluid layer. Initial random Gaussian noise creates density variations which causes fluid motion. The underlying problem has the symmetry of the circle as well as inversions . The turbulent_radiative_layer_2D dataset [68] contains simulations of cold dense gas moving relative to hot dilute gas. Once turbulence is induced due to the Kelvin Helmholtz instability, mixing of the two layers occurs. While the equations governing this process are invariant under translations and inversions, the initial conditions are not sampled from a distribution that is invariant under inversions. In all samples of the training set, the gas moves preferred in one direction. Consequently, the surrogate model does not respect this symmetry either.
The predictive power of the trained models vary greatly between the different datasets and models. Group averaging provides no improvement for the shear_flow dataset because the specific initial conditions break the system’s underlying symmetry, even though the theory itself and the boundary conditions are symmetric.
The CNextU-Net [69, 70] was chosen in the analysis of the datasets due to its consistent performance. The predictions and generated by (6) and (7) must be compared to the simulations . To measure how well a prediction fits the simulation, we employ the variance scaled root mean squared error (VRMSE)
where denotes the spatial mean operator, and to prevent division by zero in case . Lower VRMSE indicates higher agreement. signifies poor performance, as it indicates a prediction that is less accurate than a naively predicting the constant mean.
Results
We evaluate the performance improvement of the group averaging method by comparing the errors of the baseline prediction and of the equivariant prediction . Figure 1 provides illustrations. The results are summarized in Table 1 and Figure 2.

Figure 1. Example comparisons of time evolutions of the concentration of two datasets. (a) After steps we visually start to observe the benefits of group averaging. The equivariant model has a lower loss than the baseline at all times and is closer to the simulation (truth). At later times significant deviations between the approaches become apparent. (b) The equivariant model captures the features of the simulation relatively well for long times, while the baseline model develops several fictitious clusters.
Figure 2. Losses for the concentration predictions. In both cases the translations of are performed with a single () random translation per time step.
| Panel | Model | Start | 1 | 5 | 10 | 20 | 50 | Rollout |
| active_matter | Baseline | 0.063 | 0.386 | 0.938 | 10.81 | |||
| active_matter | 10 | 0.035 | 0.236 | 0.594 | 7.012 | |||
| active_matter | 0.066 | 0.319 | 0.803 | 9.277 | ||||
| active_matter | Baseline | 50 | 0.038 | 0.279 | 0.663 | 7.659 | ||
| active_matter | 0.026 | 0.193 | 0.497 | 5.821 | ||||
| active_matter | 0.038 | 0.202 | 0.507 | 5.976 | ||||
| gray_scott | Baseline | 0.008 | 0.154 | 0.742 | 1.833 | |||
| gray_scott | 10 | 0.007 | 0.146 | 0.680 | 1.730 | |||
| gray_scott | 0.008 | 0.126 | 0.598 | 1.511 | ||||
| gray_scott | Baseline | 50 | 0.004 | 0.073 | 0.530 | 0.879 | ||
| gray_scott | 0.003 | 0.064 | 0.453 | 0.764 | ||||
| gray_scott | 0.004 | 0.051 | 0.357 | 0.608 | ||||
| rayleigh_benard | Baseline | 0.065 | 0.346 | 0.627 | 4.246 | |||
| rayleigh_benard | 50 | 0.054 | 0.311 | 0.589 | 3.795 | |||
| rayleigh_benard | 0.065 | 0.294 | 0.526 | 3.664 | ||||
| rayleigh_benard | Baseline | 100 | 0.035 | 0.243 | 0.436 | 2.908 | ||
| rayleigh_benard | 0.032 | 0.216 | 0.394 | 2.586 | ||||
| rayleigh_benard | 0.034 | 0.198 | 0.363 | 2.395 | ||||
| turbulent_radiative_layer_2D | Baseline | 0.142 | 0.350 | 0.579 | 7.834 | |||
| turbulent_radiative_layer_2D | 0 | 0.171 | 0.355 | 0.537 | 7.813 | |||
| turbulent_radiative_layer_2D | 0.144 | 0.306 | 0.487 | 6.911 | ||||
| turbulent_radiative_layer_2D | Baseline | 50 | 0.201 | 0.499 | 0.744 | 9.128 | ||
| turbulent_radiative_layer_2D | 0.204 | 0.485 | 0.730 | 8.939 | ||||
| turbulent_radiative_layer_2D | 0.174 | 0.449 | 0.722 | 8.605 |
Table 1. Average loss values for the datasets. The different starting positions in the trajectory and the used groups are given. The loss values are the averages of the over all trajectories and state variables in the respective test sets. The loss is evaluated at the time steps indicated above each column. The rollout loss is the sum of the first 15 errors after the initial sequence. The continuous symmetries and are approximated with random group elements, except in (d), as indicated.
We experimented with the number of random Monte Carlo samples, , used to approximate the continuous groups. We observe significant improvements even at using Eq. (8), demonstrating an enhancement over the baseline surrogate models at no additional computational cost. Table 1d shows how the group average improves as increases.
The group averaging procedure outperforms the baseline prediction on all time frames, even if we sum over only one random representative of a continuous group per time step. At sufficiently late times, the models lose predictive power and the loss approaches 1. The rate of loss growth and the optimal averaging group are model-dependent.
Non-compact groups
For non-compact groups, the averaging construction encounters a few obstructions. We first describe the issues, then introduce the Weyl unitary trick and frame averaging as the alternatives, and finally study unit rescaling and Lorentz symmetry numerically.
Why the compact argument fails for non-compact groups
The first obstruction concerns the measure. If is locally compact, then the group invariant (Haar) measure exists, but it has finite total mass if and only if is compact. Thus, for non-compact , the formal expression
is not a normalized average and typically diverges. One may introduce a weight or restrict the integral to a finite window,
but this no longer gives a canonical Reynolds projection. The weight selects a preferred region of the group, which is shifted by the group action. Consequently, windowed averaging may be a useful numerical heuristic, but it does not generally produce an exactly -equivariant predictor. A related localized sampling scheme is used for Lorentz data augmentation in [71], where boosts are sampled from a distribution concentrated near the identity. Although this procedure is used during training rather than for post-hoc averaging, it similarly selects a preferred region of the non-compact group.
The second obstruction concerns the data distribution. Group averaging evaluates the predictor along the orbit
For a compact group, this orbit is compact, whereas for a non-compact group it may be unbounded. A trained model is typically reliable only on or near the support of its training distribution. Since the training data is finite, the training distribution is bounded, and non-compact transformations can move an input far outside this region.
The third obstruction is statistical. As discussed in § 3.1, for a compact group with a -invariant probability distribution , the Reynolds operator Eq. (2) coincides with the -orthogonal projection Eq. (4) onto the equivariant subspace. For a non-compact group, we may not have a probability distribution that is -invariant (due to the non-compactness of orbits discussed above). We may be able to extend it to arbitrary invariant distributions, and still define whenever that subspace is closed, but this projection depends on , and there is generally no group average that coincides with it. For the unit-rescaling group, Appendix § 6 of [63] shows explicitly that the Weyl-type Reynolds projection on Laurent monomials is not, in general, the orthogonal projection for a nontrivial real data distribution. Therefore the compact-group risk-improvement identity does not extend automatically to the non-compact setting.
The Weyl unitarian trick
In this section, we explain an analogous group averaging method for non-compact groups: the Weyl unitarian trick [72]. It extends the group averaging construction to any group arising as a real form of a complex reductive group (for example, scalings, Lorentz, symplectic, and many other groups of relevance for physics applications) at the cost of passing through the complex numbers.
We first explain how the projection is defined for a simple example. A Laurent polynomial is a function of the form
which allows both positive and negative integer powers and finite. Consider the scaling action for and . Of the functions of the form (11), only the constant function is invariant. To apply the average to this scaling group, the Weyl unitarian trick first extends from to using the same Laurent-polynomial formula. The acting group is likewise complexified to the multiplicative group , which contains the compact unit circle with normalized measure . Averaging over this compact subgroup gives
Two properties of this example hold in general. The first is the configuration of subgroups. Inside sit two one-dimensional real subgroups: the scaling group whose symmetry we wished to impose, and the compact circle over which we averaged. Each has real dimension equal to the complex dimension of , and each has as its complexification; such subgroups are the real forms of a complex group. Real forms of one group can be very different, one non-compact and another compact, and this is what the trick exploits: the symmetry is carried by one form and the averaging is carried out over the other. For connected the compact form is its maximal compact subgroup and is unique up to conjugacy. For the Lorentz group the two forms are and , each of real dimension six, which become real forms of a single complex group only after the coordinate change that is described in § 4.6, which identifies , with . The second is rigidity. Holomorphic functions (i.e. complex differentiable) are rigid, whereas smooth ones (i.e. real infinitely differentiable) are not. A classic complex analysis theorem states that holomorphic function on a connected domain are determined by their values on a suitable slice.
Passing to complex inputs buys this rigidity, and rigidity carries an invariance imposed over the compact subgroup alone to the rest of complexified group.
We now state the general construction. Let be a complex reductive algebraic group with non-compact real form and compact real form . Assume the input and output actions extend to algebraic representations and ; and let be a finite-dimensional, -stable space of algebraic maps containing an extension of the real predictor . The induced action of § 3.1 carries over unchanged, with equivariant maps again its fixed points, and the compact carries a normalized Haar measure , so we may define
which is the Reynolds operator (2) with in place of .
We can understand this construction in two different perspectives:
Group-average view. Similar to the compact-group average, this trick evaluates the complexified predictor at transformed inputs, transforms the outputs back, and averages against a probability measure. It is the same group averaging that has been done for the compact group, but we take a route that does the average which avoids the issues of non-compactness.
Representation-theoretic view. The same operator has an equivalent description in the language of representations: projects onto the trivial component of the induced representation of on , that is, onto the -fixed subspace, and rigidity reads
Retaining the trivial isotypic piece for is therefore the same as retaining it for the whole complex group.
Limitations. The construction is narrower than compact-group averaging in two respects. First, it requires a common complex domain on which both the predictor and the Weyl unitarian projection are defined: polynomial, Laurent-polynomial, and entire holomorphic predictors qualify, whereas a network with non-analytic activations such as ReLU may not. Second, it also loses the risk guarantee. For a compact group, we have the results in Eq. (5); for a non-compact group they do not necessarily hold. Here is canonical, closed-form, and blind to the data distribution, whereas carries the risk interpretation but depends on and has no closed form. Which of the two yields lower risk is therefore an empirical question, taken up in Section 4.5.2 and § 4.6.3.
Data-dependent projection
The other way to avoid averaging over a non-compact group is to define projection relative to the input distribution. Let be a probability distribution on the input space , and let be an inner-product space. For predictors , define
where is the space of predictors with finite norm.
Let be the set of -equivariant predictors, namely those that transform according to the prescribed group actions on and . We assume that is a closed linear subspace, which guarantees that every has a unique closest equivariant predictor. This predictor is the -orthogonal projection
Its residual is orthogonal to every equivariant predictor:
Let be the regression function for squared loss. If the prediction problem respects the symmetry, then . Orthogonality therefore gives
Here is the excess squared-loss risk of . Hence cannot increase squared risk under .
For compact and -invariant , the group average is self-adjoint on and coincides with , so (14) recovers the compact result of Elesedy and Zaidi [5]. For non-compact , Haar measure has infinite mass and is not -invariant, so an averaging operator would not be self-adjoint in even where one can be defined. Averaging and projection come apart, and closedness of must be assumed rather than inherited from a bounded averaging operator.
Frame averaging
Frame averaging provides a third route to post-hoc symmetrization. Intuitively, it assigns each input a finite set of input-dependent changes of coordinates that carry it to one or more canonical representatives of its group orbit. The original predictor is evaluated at these canonicalized inputs, after which each output is transported back to the coordinates of the original input and the results are averaged. Averaging over all admissible choices removes dependence on any finite ambiguity in the canonicalization, while the consistency condition below ensures that the resulting predictor is equivariant.
Using the input and output actions and introduced in § 3.1, a finite frame assigns each a nonempty finite subset satisfying
where [31]. Given an arbitrary predictor , define
Indeed, writing in the frame at gives
Thus is exactly -equivariant; when , it is invariant.
Because the average is finite, this construction does not require a normalized Haar measure and can be applied to a non-compact group whenever an equivariant finite frame is available. The concrete frames used below are obtained by bringing input-dependent quantities to canonical forms and averaging over any remaining finite ambiguity; their groups and regularity conditions are stated in the corresponding application sections. Although fixes equivariant predictors and is therefore idempotent, it is generally not the Reynolds/Haar projection or the -orthogonal projection and hence does not automatically inherit their risk-reduction guarantees. A further limitation concerns regularity: unweighted frame averaging can turn a continuous predictor into a discontinuous equivariant one, motivating continuity preserving weighted-frame constructions [73].
Example: Unit-rescaling group
Consider predicting the drag force on a body moving through a fluid from the fluid density , the flow speed , and the body diameter . In SI base units a configuration might read , , , with drag force . Recorded in CGS units, with masses in grams and lengths in centimeters, the same configuration reads
Each value is rescaled by the combination of conversion factors dictated by its physical dimensions. The physical state is unchanged; only the numbers representing it are.
A change of units is specified by one positive factor per base unit, here one each for mass, length, and time. These factors multiply under composition and are invertible, so the admissible conversions form the group , acting on numerical data by rescaling each coordinate by the monomial in the conversion factors determined by its dimensions.
We ask that a predictor commute with this action: converting the inputs to CGS units, predicting, and converting the predicted force back to newtons should return the SI prediction. A model fitted to raw numerical values need not have this property, since nothing prevents it from combining coordinates in dimensionally inconsistent ways.
We now state this in general, following [63]. Fix base units and consider an input . The units of each coordinate are encoded by a unit vector , where is the exponent of the -th base unit in the units of . Collect these vectors as the columns of the unit matrix
For and , write . The unit-rescaling group is
An element rescales the -th base unit by and acts on the numerical input coordinates by
the exponent being negative because a larger unit gives a smaller numerical value. In the example above , the base units are , and the passage to CGS units is ; the density has unit vector , and (17) returns the factor used there.
Together with the assigned units vectors , the input space is a units-typed space, which we denote by . A function is unit-equivariant if
The drag force has , so , the newton-to-dyne factor. An output with is dimensionless, and (18) reduces to unit invariance.
Weyl projection for the unit-rescaling group
Although is not itself an algebraic real form, it is the identity component of , the real points of a split algebraic torus. The corresponding complex algebraic group and a compact real form are
Both and are Zariski dense in . Consequently, a Laurent polynomial satisfies a given units-equivariant transformation law under if and only if its complexification satisfies the corresponding transformation law under .
Let
be a Laurent-polynomial predictor on . The complex torus acts by the extension of equation (17), Appendix A of [63] defines the corresponding torus projection for invariant, dimensionless outputs. For a scalar output with units vector , the equivariant extension is
where is normalized Haar measure. The factor is the inverse output action: since the output transforms as , transforming the output back contributes .
Proposition 4.1 (Units Weyl unitary projection). For , write
If
then
Thus is the projection onto the Laurent-polynomial predictors satisfying
In particular, retains exactly the dimensionless monomials.
Proof. Since
we have
The integrand is a character of , whose Haar average is 1 when and 0 otherwise. The result follows by linearity.
If has real coefficients, then so does . Its restriction to is therefore a real-valued units-equivariant predictor.
In words, creates no equivariant features; it only deletes the ones that are not. Each monomial is a joint eigenvector of the scaling action, with eigenvalue , so is diagonal in the monomial basis: it keeps the monomials with and annihilates the rest. The cost is expressivity: a predictor whose dictionary contains no monomial of weight projects to zero.
Numerical experiments with the unit-rescaling group
We consider force regression and Reynolds-number classification tasks. The regression inputs are density , dynamic viscosity , speed , area , and length , with units , , , , , respectively. Classification uses the same variables except . The regression target is
Since is dimensionless, has units of force, , corresponding to the unit-weight vector .
For classification, the decision boundary is : the regime is inertia dominated, whereas is viscosity dominated. We define the true logit and class probability by
We compare the following methods:
Baseline: following the experiments from [63] we fit a ridge regression or -penalized logistic regression on a set of features, of which 10 are units-invariant and 250 are not invariant.
Weyl-projection: the projection of the baseline to the space of invariant/equivariant functions using the Weyl trick in this case gives us the restriction of the baseline model to the invariant features only. That is, zeroing out the coefficients of the non-invariant features.
Empirical projection: the projection of the baseline to the space of invariant/equivariant functions, obtained by minimizing Eq. (4), as discussed in Section 4.3.
Frame average: A change of mass, length, and time units has three scaling factors. Each of the following triples uniquely determines a change of units that makes its three variables equal to one:
For , let contain the powers of , , in the units of the selected variables, and define
Then ; the remaining variables are rescaled consistently but are not generally equal to one. Moreover, and , so the inverse transformations form an equivariant frame.
We average the frozen baseline over these four canonicalized inputs:
where . We use for regression and for the classification logit, applying the sigmoid after averaging. The factor converts the prediction back to the original units. This is the “Frame average ” reported below. It reuses the fitted coefficients and original RMS scales without refitting.
Invariant regression: an invariant regression using only the invariant terms as aforementioned.
We train our model with two different training set,
Fixed units: all the training data is in one fixed unit system.
Transformed units (small shift): the training data are transformed with random units transformations sampled from the distribution
We further evaluate the performance in the following three scenarios:
Fixed units: the test data is in one fixed unit system as above.
Transformed units (small shift): the test data are transformed with random units transformations sampled from Eq. (21).
Transformed units (larger shift): the test data are transformed with random units transformations, sampled from
We also verify that the numerical group averaging produces an invariant/equivariant output. We compare the integral over a fixed grid with a Monte Carlo approximation. This, combined with further experiments studying the dependency on the number of samples are described in Appendix Appendix § 8.
The regression risk is dimensionless
The classification risk is the sample mean Bernoulli Kullback–Leibler (KL) divergence from the true probability to the fitted probability :
All risks are nonnegative and lower is better. For each test set with transformed units, the risk is first averaged over 16 held-out unit systems within each seed. The tables report mean sample standard deviation over ten independently generated and fitted seeds unless stated otherwise.
For classification, the baseline model outputs a logit . We apply the projection to this logit and then convert the projected logit into a probability:
Regression training and validation targets contain one-percent multiplicative noise; tests use the noiseless target.
| Task | Training | Model | Fixed units | Transformed units (small shift) | Transformed units (large shift) |
| Regression (NRMSE) | Fixed units | Baseline | 0.0164 ± 0.00647 | ||
| Regression (NRMSE) | Fixed units | Weyl-Projected | 0.0806 ± 0.0641 | 0.0806 ± 0.0637 | 0.0814 ± 0.0656 |
| Regression (NRMSE) | Fixed units | Empirical | 0.00936 ± 0.00474 | 0.0100 ± 0.00413 | 0.0101 ± 0.00379 |
| Regression (NRMSE) | Fixed units | Invariant regression | 0.00487 ± 0.00125 | 0.00532 ± 0.00157 | 0.00525 ± 0.00163 |
| Regression (NRMSE) | Fixed units | Frame average | 2.11 ± 1.26 | 2.02 ± 1.14 | 2.31 ± 1.48 |
| Regression (NRMSE) | Transformed units (small shift) | Baseline | 0.0325 ± 0.0152 | 0.703 ± 1.96 | |
| Regression (NRMSE) | Transformed units (small shift) | Weyl-Projected | 0.0398 ± 0.0206 | 0.0390 ± 0.0200 | 0.0391 ± 0.0200 |
| Regression (NRMSE) | Transformed units (small shift) | Empirical | 0.0402 ± 0.0174 | 0.0409 ± 0.0175 | 0.0430 ± 0.0243 |
| Regression (NRMSE) | Transformed units (small shift) | Invariant regression | 0.00487 ± 0.00125 | 0.00532 ± 0.00157 | 0.00525 ± 0.00163 |
| Regression (NRMSE) | Transformed units (small shift) | Frame average | 0.0366 ± 0.0180 | 0.0362 ± 0.0173 | 0.0374 ± 0.0194 |
| Classification (KL) | Fixed units | Baseline | 0.0156 ± 0.00233 | 10.26 ± 1.38 | 14.72 ± 1.04 |
| Classification (KL) | Fixed units | Weyl-Projected | 0.137 ± 0.0514 | 0.138 ± 0.0516 | 0.138 ± 0.0520 |
| Classification (KL) | Fixed units | Empirical | |||
| Classification (KL) | Fixed units | Invariant regression | |||
| Classification (KL) | Fixed units | Frame average | 0.0185 ± 0.0112 | 0.0186 ± 0.0113 | 0.0186 ± 0.0113 |
| Classification (KL) | Transformed units (small shift) | Baseline | 0.0240 ± 0.0189 | 0.0675 ± 0.0627 | 6.01 ± 2.10 |
| Classification (KL) | Transformed units (small shift) | Weyl-Projected | 0.0258 ± 0.0194 | 0.0259 ± 0.0194 | 0.0257 ± 0.0193 |
| Classification (KL) | Transformed units (small shift) | Empirical | 0.0147 ± 0.0142 | 0.0147 ± 0.0141 | 0.0146 ± 0.0140 |
| Classification (KL) | Transformed units (small shift) | Invariant regression | |||
| Classification (KL) | Transformed units (small shift) | Frame average | 0.0247 ± 0.0192 | 0.0248 ± 0.0193 | 0.0247 ± 0.0192 |
Table 2. Regression uses NRMSE and classification uses expected Bernoulli KL; lower is better. Here, training a new invariant model gives the best result in all cases.
The frame average is nearly insensitive to the test unit system, as expected from its exact equivariance. With fixed-unit training, it prevents catastrophic failures under unit shifts but can substantially reduce accuracy: the regression NRMSE remains near 2.1, compared with 0.0164 for the baseline in fixed units, while the classification KL remains near 0.0186 across all unit systems. With transformed-unit training, its performance improves considerably, with regression NRMSE near 0.037 and classification KL near 0.025 across all test conditions. Thus, frame averaging guarantees the correct transformation law, but its accuracy still depends on how well the frozen predictor performs on the canonicalized inputs.
The Weyl projection satisfies the prescribed units scaling law to numerical precision, so its risk is unchanged across the three unit conditions. It nevertheless increased the in-distribution risk for both tasks. When the baseline models were tested in unit systems with transformed units, the projection reduced the regression NRMSE from to 0.0821 and the classification KL from 10.25 to 0.138. These large gains arose because the baseline models’ non-invariant terms transformed incorrectly. Using transformed units during training reduced, but did not eliminate, this sensitivity, especially for transformations out-of-distribution (such as the large shifts).
Conclusion. The Weyl projection can greatly improve predictions when test examples are expressed in unit systems not seen by an already trained baseline model, because it removes dependence on the non-invariant terms. It is not a universal accuracy improvement: it worsened the risk for the fixed units experiment. Its demonstrated benefit is its robustness to unit transformations. Frame averaging showed the same robustness with accuracy tradeoff: it prevented the catastrophic failures under unseen unit transformations, but it did not provide a uniform in-distribution improvement and performed particularly poorly for the fixed-unit regression baseline.
Example: Lorentz group
We now specialize the construction of § 4.2 to scalar-valued predictors that take Lorentz vectors as inputs. Invariance under means that applying the same Lorentz transformation to every input vector leaves the prediction unchanged. Unlike the unit-rescaling example, the compact real form is not contained in the real Lorentz group in the original coordinates. After Wick-rotating the time coordinate, however, the noncompact form and the compact form become real forms of the same complex orthogonal group. The projection therefore has three steps: pass to Wick coordinates, average over , and restrict back to the real slice.
Throughout, let and use the sign convention . For , with , define the metric and the Minkowski invariant
The full Lorentz group is
and its complexification is
Let
Since , conjugation by gives an isomorphism
In these Wick coordinates, is a compact real form. We define the Wick embedding
It intertwines the Lorentz action with the complex orthogonal action:
On , let
For ,
Thus converts Lorentz transformations into complex orthogonal transformations and Minkowski contractions into complex orthogonal contractions. We can therefore average in Wick coordinates and then restrict back.
Lorentz invariant functions with a single four-vector input
Let be a polynomial predictor, and let denote its coefficientwise complexification. Define the corresponding predictor in Wick coordinates by
so that . Let act complex-linearly on , and define
where is normalized Haar probability measure. The corresponding predictor on the Lorentz real slice is
Haar averaging is a linear idempotent projection onto the -invariant polynomials. Therefore , and hence , is a linear idempotent projection on the corresponding polynomial function space. Equation (32) is the scalar specialization of the average (12).
Proposition 4.2 (Single-vector Weyl unitary projection). Let be polynomial, and let be its Wick-coordinate complexification defined by (31). Then there exists such that
Consequently,
and is a real-valued, -invariant polynomial.
Proof. Since is polynomial, is polynomial. Haar invariance implies that is -invariant. The compact real form is Zariski dense in , so is -invariant. By the first fundamental theorem for the complex orthogonal group,
Hence for some . Equation (30) then gives
which is -invariant.
It remains to verify that the restriction is real-valued. Define the antiholomorphic involution
Its set of fixed points is . Because has real coefficients, the extension (31) satisfies
Moreover,
and the automorphism preserves Haar measure on . Therefore
The averaged function is thus real-valued on the set of fixed points.
The Wick embedding is essential. For example, let and . Averaging directly over gives
which is Euclidean rather than Lorentz invariant. In Wick coordinates, however,
and Weyl averaging gives
Restricting to yields
which depends only on the Lorentz invariant .
Lorentz invariant functions with several vector inputs
We can extend the results above to the case of multiple input vectors. Let
and write
The compact group acts diagonally by
For a real polynomial predictor
let denote its coefficientwise complexification and define
Then . Define
The complex bilinear Gram map is
By equation (30),
The image of is the Gram variety
Thus when , whereas for the Gram entries satisfy the corresponding determinantal relations.
Proposition 4.3 (Multi-vector Weyl unitary projection). Let be polynomial, and let be its Wick-coordinate complexification defined above. Then there exists a polynomial
such that
Consequently,
so is real-valued and invariant under the diagonal action of . When , the ambient polynomial is determined only up to polynomials that vanish on .
Proof of the polynomial statement. The average is polynomial and is -invariant by Haar invariance. Since is Zariski dense in , is also -invariant. The first fundamental theorem for the complex orthogonal group states that invariant polynomials on vectors are generated by the pairwise contractions . Hence factors polynomially through , proving (39). Equation (38) gives the restriction (40). Applying the real-structure argument from the single-vector case diagonally to shows that this restriction is real-valued.
In particular, if has total degree at most two, then its Weyl unitary projection has the form
for suitable real coefficients and .
Numerical experiments for the Lorentz group
We now test the Lorentz Weyl construction from Section 4. We consider three settings. The first is a synthetic regression problem whose target is exactly Lorentz invariant by construction. This allows us to test the effect of Weyl unitary averaging with respect to different sizes of the training data. The second is jet tagging, a realistic classification task from collider physics in which the inputs are particle four-momenta. Analogous to improving the time-series prediction of Section 3, we also apply the discussed methods to a relativistic dynamic system, the driven covariant double well.
Synthetic experiments We first evaluate the Weyl projection on a controlled regression problem with an exactly Lorentz-invariant conditional mean. Each input consists of four-vectors
Writing and , we generate
where is sampled once per seed and held fixed across training-set sizes. Since the Gram matrix is unchanged by a simultaneous Lorentz transformation of all input vectors, it is Lorentz invariant.
We consider two input distributions. In the Euclidean-isotropic setting, . In the time–space anisotropic setting, the components are independent with
We compare four different models.
Baseline model is a ridge regression on all coordinate monomials of degree at most two.
Spatially averaged model post-processes the fitted predictor using a Monte Carlo average with samples.
Weyl-projected model applies the Weyl projection from Section 4.6.2. For degree-two polynomials, this projection eliminates the linear component and maps the quadratic component into the span of the Minkowski products , yielding a Lorentz-invariant predictor.
Invariant baseline is trained directly on the 21 distinct Gram entries that are invariant with respect to unit-rescaling group actions.
We use training sizes between 100 and 5000, five random seeds, and 5000 independent test examples per seed. Figure 3 reports the test mean-squared error; points and error bars denote the mean and one standard deviation across seeds.

Figure 3. Test MSE for the baseline polynomial, its exact Weyl projection, the finite spatial average, and the Gram-feature invariant baseline. Both axes are logarithmic.
The invariant baseline is already close to the irreducible noise level with 100 training examples. In the Euclidean-isotropic setting, post-training Weyl projection also substantially improves sample efficiency: at , it reduces the MSE from to .
This improvement is not universal. In the time–space anisotropic setting with , Weyl projection increases the MSE from to , whereas the spatial average gives . Hence, the exact dataset distribution is important, and which specific projection used depends on the distribution.
We also evaluate prediction variance under common Lorentz boosts. The Weyl-projected and Gram-feature models remain invariant to numerical precision, whereas the baseline and spatially averaged predictors do not. The diagnostic and complete results are reported in Appendix § 8.
Conclusion. With limited training data, an unconstrained model may fail to recover the symmetry of the regression function. The Weyl projection removes non-invariant components improves test error for the Euclidean-isotropic distribution, which is invariant under the compact group used in the projection. However, exact Lorentz invariance does not by itself guarantee lower prediction error: under time–space anisotropy, the Weyl projection can increase the MSE. Thus, while the projection always enforces the desired symmetry, its statistical benefit depends on the input distribution.
Chaotic relativistic dynamics We now analyze a spatiotemporal chaotic relativistic dynamical system, analogous to § 3. As a toy model for a continuous, Lorentz-equivariant model we choose the driven covariant double well in dimensions.
System. The state of the particle consists of the space-time event and the momentum , with metric and . All physical variables are functions of the Lorentz-invariant evolution parameter . The dynamics follows from the Lagrangian
where dots denote derivatives with respect to and is the drive phase. The equations of motion are
we choose and drive amplitude . The force is a Lorentz scalar times a Lorentz vector, so the flow of (44) is exactly equivariant under the full Lorentz group , generated by the boosts, parity, and time reversal
Due to the driving force, the total energy is not conserved, , which permits chaotic motion.
Data. We generate trajectories by numerical integration and split them by trajectory into 1400 training, 300 validation, and 300 test trajectories. Since the external drive breaks time-translation symmetry, we include the drive phase as a state variable, encoded as to avoid the discontinuity at . The phase is a Lorentz scalar and is advanced analytically, so the state is
We restrict to trajectories with vanishing boost charge . is conserved by (44), and the sector , in which and are parallel, is invariant under both the flow and . Trajectories in this sector remain bounded, whereas generic trajectories with escape to infinity. All trajectories start at the space-time origin, , with , so that holds automatically and is timelike. Mass, rapidity, and phase are drawn independently with , truncated at , and . The first of each trajectory are discarded as burn-in, and the following are recorded at intervals , i.e. 1500 states per trajectory. The finite-time Lyapunov exponent of the chaotic trajectories has a median of 0.15, a Lyapunov time of ; 7% of the trajectories are regular.
Reduced coordinates. On the sector both vectors lie on a fixed timelike ray , and . The rapidity is a constant of motion, and
are Lorentz invariants, the position and momentum in the rest frame of the ray. In these coordinates (44) reduces to a parametrically driven Duffing double well,
with wells at separated by a barrier at . To generate trajectories, we integrate (48) and map the result to . The prediction model only ever sees the physical coordinates.
Surrogate model. As surrogate model (6) we use a fully connected network that maps a window of consecutive states (46) to the increment of the two Lorentz vectors. The input is the flattened window of dimension , the network has three hidden layers of width 128 with ReLU activations (39 812 parameters), and the output is the residual . Inputs and outputs are normalized by one scalar per Lorentz vector , computed on the training set, so that the normalization commutes with the group action. The network is trained on all windows of the training trajectories with a mean-squared loss on the normalized increments, using AdamW with batch size 1024 for 100 epochs, a linear warm-up over steps to a learning rate of , and cosine decay to zero; we keep the checkpoint with the lowest validation loss. Predictions are autoregressive rollouts from the first states of each test trajectory. We test this network with three types of averages:
Boosts average (not equivariant):
Inversions average (equivariant):
and the rest-frame average (53) below. averages over Lorentz boosts around the frame of the input. Since the boosts are non-compact and carry no normalizable Haar measure, the Gaussian weight is a choice and is not exactly equivariant. averages over the finite subgroup of time and space inversions; it is an exact group average. Each group element acts on both Lorentz vectors of every state in the window and leaves the phase unchanged.
Rest-frame average. To compute a frame average as in § 4.4, we apply a Lorentz boost that transforms the particle into its rest frame. The rapidity can be computed from a short window of data: for the input window ,
which is exact on the sector , where it transforms with the input, . Following [31], we use the frame , which on this sector satisfies for all . The frame average
is then exactly -equivariant on the sector for any network . Here we extend this procedure by applying, in the rest frame, additional boosts in addition to the inversions.
Extended frame average:
which requires 16 network evaluations. Because the nodes are symmetric and for , the weighted frame is still equivariant, and remains exactly equivariant. Every rollout of stays on the ray of its initial window.
The Gaussian averages in (49) and (53) are evaluated by Gauss–Hermite quadrature,
where are the roots of the Hermite polynomial and the corresponding weights. We use for and for . The widths 0.3 and 0.4 were selected on the validation set.
Error. The error is measured by the total deviation in the rest frame of the true trajectory. Both the prediction and the truth are transformed by , with from (51) applied to the whole true trajectory, and . Then the error (or loss) is
where denotes prediction minus truth and is evaluated on the prediction. The first two terms measure the error within the dynamics and the departure from the ray of the true trajectory. The distance is invariant under a joint Lorentz transformation of prediction and truth. The reported errors are averages over the test set at a given time . Trajectories that erroneously escape to infinity are excluded from the test set averages.
Group averaging can here too significantly increase the predictive power of a model without pre-training. The results are shown in Figure 4 and Table 3. We find that for this example a combination of the Monte-Carlo method for average boosts together with a frame-average gives the best results.

Figure 4. (a) The rollout test-set average error . (b) The phase plot of a sample trajectory. The averaging procedures on average decrease the rollout loss. We compare 3 different averages to the baseline prediction (the model evaluated only once). On the right, we see with an example how group averaging improves prediction: The averaged trajectory (red) stay closer to the truth (black) for longer.
| Model | 1 | 10 | 100 | Rollout |
| Baseline | 0.0018 | 0.0126 | 0.2321 | 0.1497 |
| Boosts | 0.0014 | 0.0100 | 0.1911 | 0.1201 |
| Inversions | 0.0010 | 0.0065 | 0.1298 | 0.0786 |
| Extended frame | 0.0003 | 0.0027 | 0.0674 | 0.0313 |
Table 3. Average invariant loss values for the test-set ( test trajectories) after autoregressive steps with per step. The rollout loss is the sum of the first 15 errors.
Jet tagging We next study quark–gluon jet tagging, a standard classification problem in collider physics. A jet is represented as an unordered collection of constituent four-momenta,
We use the two-class Pythia8 quark–gluon sample distributed with EnergyFlow [74, 52], with quark jets treated as signal and gluon jets as background. The split contains training jets, validation jets, and test jets, with equal numbers of the two classes in each split. We evaluate two complementary classifiers. The first is ParticleNet-Lite [53], a realistic high-capacity jet tagger. The second is a degree-three polynomial classifier whose logit is a polynomial in the Cartesian components of the constituent momenta, built from the permutation-invariant aggregates , and , which supply features once the symmetry of is used, and fitted by -penalized logistic regression. It serves two purposes. Because it retains explicit dependence on the azimuthal coordinate, it isolates the effect of post-training averaging on a predictor whose fitted coefficients are not already azimuthally invariant. And because its logit is polynomial in the four-momenta, it lies in the function class on which the Weyl projection of § 4.6 is defined.
Across the two models, we consider the following post-hoc averaging procedures:
Beam-axis averaging. We average over rotations about the beam axis. This is a compact-group average aligned with the azimuthal symmetry of the proton–proton collision sample.
averaging. We average over spatial rotations while holding the laboratory beam axis fixed. Because these transformations do not preserve the saved-jet distribution, this provides a deliberate symmetry-mismatch control.
Bounded boost averaging. We separately average over boosts parallel and transverse to the beam axis, with rapidities drawn from a fixed bounded law, rejection-truncated to . Since boosts form a non-compact group, these are windowed test-time averages rather than Reynolds projections.
Weyl-unitarian projection. For degree-three polynomial classifier, we use the formula derived and applied to the polynomial to act as the classifier.
Collider-aligned frame averaging. We also apply the frame average from § 4.4 to ParticleNet-Lite. Let
We remove the jet azimuth and longitudinal rapidity through
so that . We then average over the residual reflection sets
where and reverse the corresponding momentum components. For a scalar predictor , the resulting -view predictor is
The construction enforces invariance under rotations about and boosts along the beam axis, together with the selected reflections, and is well defined for and .
For each probability-valued classifier , we form
Each transformation acts on every constituent four-momentum in the jet before all derived kinematic features are recomputed. For , we use the deterministic angles . For , we use a fixed list of Haar-sampled proper rotations while holding the laboratory beam axis fixed. The ParticleNet-Lite results below use , average the two class probabilities, and use the same transformation list for every jet and every trained model.
To resolve changes hidden by aggregate accuracy, let be the number of test jets whose prediction changes from incorrect before averaging to correct after averaging, and let count changes in the opposite direction. We report
which is exactly the change in the number of correctly classified test jets. We also report the background rejection at quark efficiencies and , where is the gluon acceptance at the threshold chosen to obtain the stated quark efficiency.
| Predictor | Accuracy | |||||
| Baseline (ParticleNet-Lite) | — | — | — | |||
| Collider-aligned frame average () | ||||||
| average | ||||||
| average | ||||||
| Longitudinal boost average | ||||||
| Transverse boost average |
Table 4. ParticleNet-Lite performance before and after spatial-rotation, bounded-boost, and input-dependent collider-aligned frame averaging. Entries are the mean sample standard deviation over nine independently trained models evaluated on the same 200,000-jet test set. The spatial-rotation and bounded-boost entries use 128 frames and average class probabilities. The collider-aligned entry uses four deterministic canonical views and averages pre-softmax binary log-odds. The boost rapidities are sampled from a rejection-truncated Gaussian , with . Here counts incorrect-to-correct changes and counts correct-to-incorrect changes.
averaging leaves the ParticleNet predictions unchanged to numerical precision, as a result of the construction of the model that extract numerical values of a jet making it invariant with respect to changes. Fixed-lab averaging, however, changes many decisions in both directions. This shows that the exact Lorentz symmetry is not present in the Quark-gluon dataset as the particles are recorded with a fixed beam axis. The collider-aligned frame average is the method which showed consistent improvements across all seeds.
To probe the finite-sample effect of beam-axis averaging and also attempt to apply an exact Weyl unitary trick, we also train the degree-three polynomial classifier. For each size and each of ten seeds, all post-hoc methods are applied to the best checkpoint.
Figure 5 shows that beam-axis averaging improves AUC throughout the training-size sweep, with the largest gain in the low-data regime and a diminishing advantage as the training set grows. This is consistent with compact-group projection. Averaging removes non-invariant components that become less important as the baseline learner sees more data [5].
Figure 5. ROC AUC and background rejection at quark efficiencies of 30% and 50% for the degree-three polynomial classifier.
The Exact Weyl projection supplies a stronger low-data inductive bias. The saved-jet distribution is not strictly Lorentz invariant, for the beam-axis reason described above, yet restricting the classifier to the fixed 18-dimensional invariant subspace already saturates its achievable performance within a few hundred examples. That subspace is spanned by the products of total degree at most three in , the Minkowski traces , , and of , and the contractions and . The projection collapses the 816-coefficient hypothesis class onto these 18 directions, and the surviving coefficients converge almost immediately: the projected discriminant direction at 256 examples lies within one degree of its counterpart, while its norm grows by a factor of 23. Because a positive rescaling leaves the ordering of scores unchanged, AUC and background rejection are flat across the entire sweep. The projected model is therefore the strongest predictor in the low-data regime, but it stops improving, but it stops improving: the unprojected baseline overtakes it in AUC and at 50% quark efficiency between 8,192 and 16,384 examples, while at 30% it stays at or above the baseline across the sweep. The ROC curves cross, so the projection remains competitive at the high-purity working points.
Overall, performance follows the alignment between the imposed transformation and the collider setup. Beam-axis preserves that setup, whereas fixed-laboratory averaging is detrimental because arbitrary spatial rotations respect neither the beam direction nor the selected-jet distribution. The bounded longitudinal-boost average stays close to the raw model, while transverse averaging is generally mildly detrimental. We therefore read the boost curves only as bounded-window sensitivity checks rather than canonical projections over the noncompact Lorentz group; their uncertainty intervals additionally condition on a single fixed transformation bank.
Discussion and conclusion
This work asks how a known physical symmetry can be imposed on an already trained predictor, and when doing so improves prediction. For compact groups, these objectives are closely connected. As shown in § 3.1, normalized Haar averaging defines the Reynolds projection onto the invariant or equivariant subspace. When the input data distribution is invariant, the induced group action is unitary, and the squared-loss regression function is equivariant, this operator coincides with the -orthogonal projection in Eq. (5) and therefore cannot increase population squared risk. The surrogate-model experiments of § 3.3 are consistent with this theory: averaging generally reduced prediction errors when the implemented transformations were genuine symmetries of the complete learning problem.
For non-compact groups, this correspondence breaks down. There is no normalized Haar probability measure, so the compact Reynolds construction cannot be applied directly. More fundamentally, the algebraic projection that enforces a transformation law need not coincide with the projection that minimizes error under a specified data distribution. The data-dependent projection of Section 4.3 retains the latter statistical interpretation, but its implementation requires to solve an optimization problem akin to training an equivariant model. The Weyl construction of § 4.2 instead provides a distribution-independent algebraic projection for the polynomial, Laurent-polynomial, and suitably holomorphic function classes covered by the construction. It enforces the desired symmetry exactly, but it does not inherit the compact-group risk guarantee.
Frame averaging, in § 4.4, is a third route. It replaces the integral over the group with a finite average over an input-dependent set of transformations, so it requires neither a normalized measure nor an analytic predictor and applies to an arbitrary black-box model. Like the Weyl construction it is exact, in that the averaged predictor satisfies the prescribed transformation law identically, and like the Weyl construction it is in general neither the Haar projection nor the projection, so it carries no risk guarantee either.
The non-compact-group experiments give four main empirical conclusions. First, the Weyl projection did not consistently improve in-distribution accuracy, but it produced substantial gains with limited training data and when test inputs were expressed in unit systems not seen during training. Second, frame averaging gives the same exactness without the function-class restriction: it is units-equivariant by construction and applies to an arbitrary black-box predictor, and it prevented the catastrophic failures the baseline suffers under unseen unit transformations. Its in-distribution cost was severe for the fixed-unit-trained regression. Third, the empirical projection improved on the baseline under most tested unit shift and in-distribution task and training-regime combinations, illustrating the benefit of adapting the projection to the data distribution; it was also the least stable of the three, degrading sharply at small training-set sizes (Appendix § 8). Fourth, when retraining was possible, models trained directly using symmetry-adapted features matched or outperformed the post-hoc projections in our controlled unit-rescaling and Lorentz experiments. These comparisons position post-hoc symmetrization as a complement to symmetry-aware training, particularly when retraining or architectural modification is impractical.
Several questions remain open. Theoretically, it would be useful to characterize the distributions and predictor classes for which the Weyl projection reduces risk, and to quantify its discrepancy from the projection in terms of the geometry of the data distribution. Computationally, extending the construction beyond polynomial or holomorphically extendable predictors would be necessary for its direct application to modern black-box neural networks.
Acknowledgements
SV and WZ were partially funded by NSF DMS 2031985. SV is also funded by NSF CAREER 2339682, NSF BSF 2430292, and a Sloan Fellowship. We thank Lara Knezevic for drawing our attention to related work on twirling in quantum machine learning.
Data availability statement
The data that support the findings of this study are openly available. The quark–gluon jet dataset is available at https://doi.org/10.5281/zenodo.3164691. The Well dataset used in § 3 is part of the Well collection [6] and is available at https://polymathic-ai.org/the_well/. The synthetic unit-rescaling and Lorentz datasets, together with the code used to generate them and to reproduce all experiments reported here, are available at https://github.com/wenjie271/Post-hoc-group-averaging.
Conflicts of interest statement
The authors have no conflicts of interest.
Ethics statement
Not applicable.
Groups
Dihedral groups
For auto-regressive prediction problems, we are interested in datasets on a square or rectangular Cartesian grid. Datasets defined on a square grid with appropriate initial and boundary conditions have the largest possible symmetry.
The symmetry group of the square is and contains elements. For datasets defined on a rectangle, the biggest symmetry group is , containing elements. Inversions correspond to with elements.
Generally, the dihedral groups refer to the symmetry groups of the -gon, groups of order . We can think of these as rotational symmetries and reflection symmetries.
The group representation of is
where its members can be thought of as rotations and inversions. One representation is given by the matrices
and the group elements can be constructed by
can be constructed as above using only even powers of , and by using only the inversion.
Circle groups
Translation invariant problems with periodic boundary conditions obey the symmetry of the circle. The circle group is an example of a compact continuous Lie group. It is known by several names, such as . The irreducible representations are one-dimensional maps to the unit circle. If the problem is translation invariant and periodic in two directions, it is defined on a 2-torus. The torus group is the continuous Abelian group formed from the direct product of two circle groups .
Since numerical simulation data is always given on a discrete grid of dimension , we approximate the continuous groups with discrete cyclic groups , that is with and with .
Experimental details
Compact group averaging
“The Well”1 [6] is published under CC BY 4.0. Each test set contains of the trajectories. We excluded 9 trajectories from the gray_scott dataset that converge to a steady state in our analysis.
| Dataset | Size (GB) | ||||
| active_matter | 51.3 | 81 | 360 | 11 | |
| gray_scott | 153.8 | 1001 | 1200 | 2 | |
| rayleigh_benard | 342 | 200 | 1750 | 4 | |
| turbulent_radiative_layer_2D | 6.9 | 101 | 90 | 4 |
Table 5. Dataset specifications. (a) Total size, resolution , number of time steps per trajectory , number of trajectories in the data set , number of state variables .
| Dataset | Quantity | Type | Components |
| active_matter | Concentration | Scalar | 1 |
| active_matter | Velocity | Vector | 2 |
| active_matter | Orientation | Tensor | 4 |
| active_matter | Rate-of-strain | Tensor | 4 |
| gray_scott | Concentration | Scalar | 1 |
| gray_scott | Concentration | Scalar | 1 |
| rayleigh_benard | Buoyancy | Scalar | 1 |
| rayleigh_benard | Pressure | Scalar | 1 |
| rayleigh_benard | Velocity | Vector | 2 |
| turbulent_radiative_layer_2D | Buoyancy | Scalar | 1 |
| turbulent_radiative_layer_2D | Pressure | Scalar | 1 |
| turbulent_radiative_layer_2D | Velocity | Vector | 2 |
Table 5b. (b) The state variables of the datasets.
Jet-tagging dataset
The quark–gluon jet dataset [52] is published under CC BY 4.0.2 The sample was generated with Pythia 8.226 at TeV, with quark jets produced via and gluon jets via ; no detector simulation or pileup is included. Final-state particles are clustered with the anti- algorithm (, FastJet 3.3.0), and jets with GeV and are retained. We use the default sample, which excludes charm and bottom quark jets at the level of the hard process. The dataset contains jets with equal numbers of the two classes; we use jets for training, for validation, and for testing, preserving the class balance in each split. Each jet constituent is stored as , from which we reconstruct the Cartesian four-momenta of equation (56).
Model and preprocessing. Our primary classifier is ParticleNet-Lite [53]. Each jet retains at most the 100 constituents with largest original transverse momentum, and the model is trained on the standard ParticleNet input features derived from the constituent four-momenta. The degree-three polynomial classifier of § 4.6.3 is built from coordinate-dependent jet aggregates of the same constituent four-momenta.
| Generator | Pythia 8.226, |
| Processes | (quark), (gluon) |
| Clustering | anti-, (FastJet 3.3.0) |
| Selection | , |
| Constituent features | |
| Split (train/val/test) | |
| Classes | quark (signal) vs. gluon (background), balanced |
Table 6. Specifications of the quark–gluon jet dataset [52].
Additional experiments of unit-rescaling group
Numerical realization of the projection. Proposition 4.1 evaluates in closed form by deleting the coefficients of monomials in the wrong units sector. Outside the Laurent-polynomial class that closed form is unavailable and the projection must instead be computed as an average over the compact torus, so we first check that such an average reproduces the algebraic operator.
We compare three evaluations of the same operator. Exact coefficient filtering applies by deleting wrong-sector coefficients. Deterministic torus quadrature averages over a finite roots-of-unity grid. Monte Carlo projection averages random phases , where denotes the total number of model evaluations. The comparison uses 20 frozen fitted predictors: five seeds for each combination of force regression or Reynolds classification and fixed-unit or unit-randomized training. Writing for a frozen fitted predictor, the error of a numerical approximation is over the evaluation set.
| Operator | Mean error | Maximum error |
| Deterministic | ||
| 0.0681 | 0.449 | |
| 0.0325 | 0.203 | |
| 0.0206 | 0.109 |
Table 7. Relative prediction error against exact coefficient filtering. Monte Carlo rows aggregate combinations of task, training regime, seed, and independent phase sequence.
For every tested fit, deterministic quadrature reproduced coefficient filtering to machine precision. Monte Carlo error decreased as increased but remained heterogeneous across predictors: at , the mean was 0.0206 and the maximum was 0.109.
Hence, averaging over the compact torus realizes the algebraic Weyl projection: the deterministic quadrature is exact up to floating-point error. The Monte Carlo realization converges slowly enough that its maximum error over predictors is still above at 256 evaluations, which bounds what a sampled average can be expected to deliver for predictors outside the Laurent class. This experiment concerns the numerical operator, not its predictive benefit.
Robustness under unit changes. Section 4.5.2 reports a single training-set size and a single dictionary size. Here we vary both, to test how the tradeoff between fixed-unit accuracy and robustness under unit changes depends on them. For drag-force regression, we vary the number of labeled training examples (Table 8) and the number of non-invariant features (Table 9), that is, dictionary features whose units weight does not match the output weight and which therefore deletes. We compare raw ridge, its exact coefficient projection, the frame average, and a direct invariant regression fitted on the invariant features alone. We also test an empirical approximation, where is an input distribution: a ridge regression on the invariant features is fitted to predictions of the frozen raw model on a separate unlabelled calibration sample. Unlike exact projection, this refit may change the coefficients of the invariant features.
| Training | Model, | Fixed units | Transformed units (small shift) | Transformed units (large shift) | |
| Fixed units | 25 | Baseline | |||
| Weyl-Projected | |||||
| Empirical | |||||
| Invariant regression | |||||
| Frame average | |||||
| 250 | Baseline | ||||
| Weyl-Projected | |||||
| Empirical | |||||
| Invariant regression | |||||
| Frame average | |||||
| Transformed units (small shift) | 25 | Baseline | |||
| Weyl-Projected | |||||
| Empirical | |||||
| Invariant regression | |||||
| Frame average | |||||
| 250 | Baseline | ||||
| Weyl-Projected | |||||
| Empirical | |||||
| Invariant regression | |||||
| Frame average |
Table 8. Test NRMSE at for two training-set sizes. Rows give the training regime and the model; columns give the unit system of the test data. Entries are the mean sample standard deviation over seeds.
| Training | Model, | Fixed units | Transformed units (small shift) | Transformed units (large shift) | |
| Fixed units | 25 | Baseline | |||
| Weyl-Projected | |||||
| Empirical | |||||
| Invariant regression | |||||
| Frame average | |||||
| 250 | Baseline | ||||
| Weyl-Projected | |||||
| Empirical | |||||
| Invariant regression | |||||
| Frame average | |||||
| Transformed units (small shift) | 25 | Baseline | |||
| Weyl-Projected | |||||
| Empirical | |||||
| Invariant regression | |||||
| Frame average | |||||
| 250 | Baseline | ||||
| Weyl-Projected | |||||
| Empirical | |||||
| Invariant regression | |||||
| Frame average |
Table 9. Test NRMSE at for two non-invariant feature counts. Rows give the training regime and the model; columns give the unit system of the test data. Entries are the mean sample standard deviation over seeds.
Weyl unitary trick with Lorentz group on vector valued function
In this section, we consider vector-valued outputs. Let
The Weyl unitarian projection equivariant Reynolds operator is
| Training | Model, | Fixed units | Transformed units (small shift) | Transformed units (large shift) | |
| Fixed units | 50 | Baseline | |||
| Weyl-Projected | |||||
| Empirical | |||||
| Invariant regression | |||||
| Frame average | |||||
| 3000 | Baseline | ||||
| Weyl-Projected | |||||
| Empirical | |||||
| Invariant regression | |||||
| Frame average | |||||
| Transformed units (small shift) | 50 | Baseline | |||
| Weyl-Projected | |||||
| Empirical | |||||
| Invariant regression | |||||
| Frame average | |||||
| 3000 | Baseline | ||||
| Weyl-Projected | |||||
| Empirical | |||||
| Invariant regression | |||||
| Frame average |
Table 10.
This is the specialization of (12) with and output representation . By the same fixed-point argument used in the compact case,
Proposition D.1 (Vector-valued Weyl unitarian projection average). Let be polynomial, or more generally entire holomorphic. Then the Weyl unitarian projection average is a -equivariant holomorphic map. Moreover, it has the form
where the coefficients are invariant scalar functions of the Gram data. For polynomial , the may be chosen polynomial in the entries of ; for entire , they may be chosen holomorphic on the relevant Gram strata.
Proof. Equation (65) gives -equivariance. Since the averaged map is holomorphic, -equivariance extends to equivariance under the complexified group . The first fundamental theorem for orthogonal covariants states that -equivariant vector-valued polynomial maps from standard vectors to one standard vector are generated, as a module over the invariant scalar functions, by the input vectors . Therefore the output has the form (66). The holomorphic case follows by applying the same argument to the homogeneous Taylor pieces and summing locally uniformly.
Thus the Weyl unitarian projection equivariant average always lies in the complex span of the input vectors:
This is the vector-valued analogue of the Gram-matrix statement for scalar invariants.
On the locus where the Gram matrix is nondegenerate, the coefficients can be computed directly. Let also denote the matrix whose -th column is , and set
Define
Since preserves the complex bilinear form,
If is invertible and
then
and hence
More generally, a pseudoinverse formula is valid on strata where , equivalently where the bilinear form is nondegenerate on . On degenerate strata, especially when null directions are present, the representation (66) still holds, but the coefficients are not uniquely determined by the pairings . In such cases, a Moore–Penrose pseudoinverse may be used as a numerical regularization, but it should not be interpreted as a canonical algebraic formula.
Finally, suppose the original real output is Lorentz-vector-valued,
and the complexified output is chosen so that
If the Weyl unitarian projection average restricts back to the Lorentz real slice, then (66) becomes
Thus a Lorentz-equivariant vector output is built from the input four-vectors with Lorentz-invariant scalar coefficients. This is the equivariant analogue of (40).
References
- [1]E. L. Lehmann and George Casella. Theory of Point Estimation. 2nd ed. Springer Texts in Statistics. New York: Springer, 1998. DOI: 10.1007/b98854.DOI
- [2]Shuxiao Chen, Edgar Dobriban, and Jane H. Lee. “A Group-Theoretic Framework for Data Augmentation”. In: Advances in Neural Information Processing Systems. Vol. 33. 2020, pp. 21321–21333.arxiv.org/abs/1907.10905
- [3]Clare Lyle et al. “On the Benefits of Invariance in Neural Networks”. In: arXiv preprint arXiv:2005.00178 (2020). DOI: 10.48550/arXiv.2005.00178. arXiv: 2005.00178 [cs.LG].DOI
- [4]Akiyoshi Sannai, Makoto Kawano, and Wataru Kumagai. “Invariant and Equivariant Reynolds Networks”. In: Journal of Machine Learning Research 25.42 (2024), pp. 1–36. URL: https://jmlr.org/papers/v25/22-0891.html.
- [5]Bryn Elesedy and Sheheryar Zaidi. “Provably Strict Generalisation Benefit for Equivariant Models”. In: Proceedings of the 38th International Conference on Machine Learning. PMLR, 2021, pp. 2959–2969.arxiv.org/abs/2102.10333
- [6]Ruben Ohana et al. “The Well: A large-scale collection of diverse physics simulations for machine learning”. In: Advances in Neural Information Processing Systems 37 (2024), pp. 44989–45037.arxiv.org/abs/2412.00568
- [7]John Shawe-Taylor. “Building symmetries into feedforward networks”. In: 1989 First IEEE International Conference on Artificial Neural Networks, (Conf. Publ. No. 313). IET. 1989, pp. 158–162.
- [8]Taco Cohen and Max Welling. “Group equivariant convolutional networks”. In: International Conference on Machine Learning. PMLR. 2016, pp. 2990–2999.arxiv.org/abs/1602.07576
- [9]Risi Kondor and Shubhendu Trivedi. “On the generalization of equivariance and convolution in neural networks to the action of compact groups”. In: International conference on machine learning. PMLR. 2018, pp. 2747–2755.arxiv.org/abs/1802.03690
- [10]Taco S Cohen, Mario Geiger, and Maurice Weiler. “A general theory of equivariant cnns on homogeneous spaces”. In: Advances in Neural Information Processing Systems 32 (2019).arxiv.org/abs/1811.02017
- [11]Taco S Cohen and Max Welling. “Steerable cnns”. In: arXiv preprint arXiv:1612.08498 (2016).arxiv.org/abs/1612.08498
- [12]Risi Kondor. “N-body networks: a covariant hierarchical neural network architecture for learning atomic potentials”. In: arXiv preprint arXiv:1803.01588 (2018).arxiv.org/abs/1803.01588
- [13]Mario Geiger and Tess Smidt. “e3nn: Euclidean neural networks”. In: arXiv preprint arXiv:2207.09453 (2022).arxiv.org/abs/2207.09453
- [14]Marc Finzi, Max Welling, and Andrew Gordon Wilson. “A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups”. In: Proceedings of the 38th International Conference on Machine Learning. Vol. 139. Proceedings of Machine Learning Research. PMLR, 2021, pp. 3318–3328. arXiv: 2104.09459 [cs.LG].arxiv.org/abs/2104.09459
- [15]Daniel Worrall and Max Welling. “Deep scale-spaces: Equivariance over scale”. In: Advances in Neural Information Processing Systems 32 (2019).arxiv.org/abs/1905.11697
- [16]Rui Wang, Robin Walters, and Rose Yu. “Incorporating symmetry into deep dynamics models for improved generalization”. In: arXiv preprint arXiv:2002.03061 (2020).arxiv.org/abs/2002.03061
- [17]Ningyuan Huang, Ron Levie, and Soledad Villar. “Approximately equivariant graph networks”. In: Advances in Neural Information Processing Systems 36 (2023), pp. 34627–34660.arxiv.org/abs/2308.10436
- [18]Wilson G Gregory, Kaze WK Wong, and Soledad Villar. “Robust Emulator for Compressible Navier-Stokes using Equivariant Geometric Convolutions”. In: Machine Learning and the Physical Sciences Workshop. NeurIPS. 2024.
- [19]Johannes Jakob Meyer et al. “Exploiting Symmetry in Variational Quantum Machine Learning”. In: PRX Quantum 4 (1 Mar. 2023), p. 010328. DOI: 10.1103/PRXQuantum.4.010328. URL: https://link.aps.org/doi/10.1103/PRXQuantum.4.010328.DOI
- [20]Soledad Villar et al. “Scalars are universal: Equivariant machine learning, structured like classical physics”. In: Advances in neural information processing systems 34 (2021), pp. 28848–28863.
- [21]Ben Blum-Smith and Soledad Villar. “Machine learning and invariant theory”. In: Notices of the American Mathematical Society 70.08 (2023), pp. 1–1.
- [22]Ben Blum-Smith et al. “A Galois theorem for machine learning: Functions on symmetric matrices and point clouds via lightweight invariant features”. In: arXiv preprint arXiv:2405.08097 (2024).
- [23]Wilson G Gregory et al. “Learning equivariant tensor functions with applications to sparse vector recovery”. In: arXiv preprint arXiv:2406.01552 (2024).
- [24]Dmitry Yarotsky. “Universal approximations of invariant maps by neural networks”. In: Constructive Approximation 55.1 (2022), pp. 407–474.arxiv.org/abs/1804.10306
- [25]Ryan Murphy et al. “Relational pooling for graph representations”. In: International Conference on Machine Learning. PMLR. 2019, pp. 4663–4673.arxiv.org/abs/1903.02541
- [26]Mircea Petrache and Shubhendu Trivedi. “Approximation-generalization trade-offs under (approximate) group equivariance”. In: Advances in Neural Information Processing Systems 36 (2023), pp. 61936–61959.arxiv.org/abs/2305.17592
- [27]Behrooz Tahmasebi and Stefanie Jegelka. “The exact sample complexity gain from invariances for kernel regression”. In: Advances in Neural Information Processing Systems 36 (2023), pp. 55616–55646.arxiv.org/abs/2303.14269
- [28]Alberto Bietti, Luca Venturi, and Joan Bruna. On the Sample Complexity of Learning under Invariance and Geometric Stability. 2021. arXiv: 2106.07148 [stat.ML]. URL: https://arxiv.org/abs/2106.07148.
- [29]Behrooz Tahmasebi and Melanie Weber. Achieving Approximate Symmetry Is Exponentially Easier than Exact Symmetry. 2026. arXiv: 2512.11855 [cs.LG]. URL: https://arxiv.org/abs/2512.11855.
- [30]Valentino F. Foit, David W. Hogg, and Soledad Villar. Group Averaging for Physics Applications: Accuracy Improvements at Zero Training Cost. 2025. arXiv: 2511.09573 [cs.LG]. URL: https://arxiv.org/abs/2511.09573.
- [31]Omri Puny et al. Frame Averaging for Invariant and Equivariant Network Design. 2022. arXiv: 2110.03336 [cs.LG]. URL: https://arxiv.org/abs/2110.03336.
- [32]Yuchao Lin et al. Equivariance via Minimal Frame Averaging for More Symmetries and Efficiency. 2026. arXiv: 2406.07598 [cs.LG]. URL: https://arxiv.org/abs/2406.07598.
- [33]Sékou-Oumar Kaba et al. “Equivariance with learned canonicalization functions”. In: International Conference on Machine Learning. PMLR. 2023, pp. 15546–15566.arxiv.org/abs/2211.06489
- [34]Zakhar Shumaylov et al. Lie Algebra Canonicalization: Equivariant Neural Operators under arbitrary Lie Groups. 2025. arXiv: 2410.02698 [cs.LG]. URL: https://arxiv.org/abs/2410.02698.
- [35]Jinwoo Kim et al. “Learning Probabilistic Symmetrization for Architecture Agnostic Equivariance”. In: Advances in Neural Information Processing Systems. Vol. 36. 2023, pp. 18582–18612. arXiv: 2306.02866 [cs.LG].arxiv.org/abs/2306.02866
- [36]Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. “ImageNet Classification with Deep Convolutional Neural Networks”. In: Advances in Neural Information Processing Systems. Vol. 25. 2012, pp. 1097–1105.
- [37]Christian Szegedy et al. “Going Deeper with Convolutions”. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015, pp. 1–9.
- [38]Simone Monaco, Sebastiano Barresi, and Daniele Apiletti. “Lorentz-Invariant Augmentation for High-Energy Physics Deep Learning Models”. In: Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2023). Ed. by Rosa Meo and Fabrizio Silvestri. Vol. 2135. Communications in Computer and Information Science. Cham: Springer, 2025, pp. 368–376. DOI: 10.1007/978-3-031-74633-8\_26.DOI
- [39]Kimberly Stachenfeld et al. “Learned coarse models for efficient turbulence simulation”. In: arXiv preprint arXiv:2112.15275 (2021).arxiv.org/abs/2112.15275
- [40]Michael McCabe et al. “Multiple physics pretraining for physical surrogate models”. In: arXiv preprint arXiv:2310.02994 (2023).arxiv.org/abs/2310.02994
- [41]Liu Yang et al. “In-context operator learning with data prompts for differential equation problems”. In: Proceedings of the National Academy of Sciences 120.39 (2023), e2310142120.DOI
- [42]Md Ashiqur Rahman et al. “Pretraining codomain attention neural operators for solving multiphysics pdes”. In: Advances in Neural Information Processing Systems 37 (2024), pp. 104035–104064.arxiv.org/abs/2403.12553
- [43]Junhong Shen, Tanya Marwah, and Ameet Talwalkar. “Ups: Efficiently building foundation models for pde solving via cross-modal adaptation”. In: arXiv preprint arXiv:2403.07187 (2024).arxiv.org/abs/2403.07187
- [44]Maximilian Herde et al. “Poseidon: Efficient foundation models for pdes”. In: Advances in Neural Information Processing Systems 37 (2024), pp. 72525–72624.arxiv.org/abs/2405.19101
- [45]Tianyu Chen et al. “OmniArch: Building Foundation Model For Scientific Computing”. In: arXiv preprint arXiv:2402.16014 (2024).arxiv.org/abs/2402.16014
- [46]Jingmin Sun et al. “Towards a foundation model for partial differential equations: Multioperator learning and extrapolation”. In: Physical Review E 111.3 (2025), p. 035304.DOI
- [47]Makoto Takamoto et al. “Pdebench: An extensive benchmark for scientific machine learning”. In: Advances in Neural Information Processing Systems 35 (2022), pp. 1596–1611.arxiv.org/abs/2210.07182
- [48]Jeroen Audenaert et al. “The multimodal universe: enabling large-scale machine learning with 100TB of astronomical scientific data”. In: arXiv preprint arXiv:2412.02527 (2024).arxiv.org/abs/2412.02527
- [49]Hermann Weyl. The Classical Groups: Their Invariants and Representations. Princeton University Press, 1939.
- [50]Ben Gripaios, Ward Haddadin, and Christopher G. Lester. “Lorentz- and Permutation-Invariants of Particles”. In: Journal of Physics A: Mathematical and Theoretical 54.15 (2021), p. 155201. DOI: 10.1088/1751-8121/abe58c. arXiv: 2003.05487.DOI
- [51]Ward Haddadin. Invariant Polynomials and Machine Learning. 2021. arXiv: 2104.12733.arxiv.org/abs/2104.12733
- [52]Patrick T. Komiske, Eric M. Metodiev, and Jesse Thaler. “Energy Flow Networks: Deep Sets for Particle Jets”. In: Journal of High Energy Physics 2019.1 (2019), p. 121. DOI: 10.1007/JHEP01(2019)121. arXiv: 1810.05165 [hep-ph].DOI
- [53]Huilin Qu and Loukas Gouskos. “ParticleNet: Jet Tagging via Particle Clouds”. In: Physical Review D 101.5 (2020), p. 056019. DOI: 10.1103/PhysRevD.101.056019. arXiv: 1902.08570 [hep-ph].DOI
- [54]Huilin Qu, Congqiao Li, and Sitian Qian. “Particle Transformer for Jet Tagging”. In: Proceedings of the 39th International Conference on Machine Learning. Vol. 162. Proceedings of Machine Learning Research. PMLR, 2022, pp. 18281–18292. arXiv: 2202.03772 [hep-ph].arxiv.org/abs/2202.03772
- [55]Anja Butter et al. “Deep-Learned Top Tagging with a Lorentz Layer”. In: SciPost Physics 5.3 (2018), p. 028. DOI: 10.21468/SciPostPhys.5.3.028. arXiv: 1707.08966 [hep-ph].DOI
- [56]Alexander Bogatskiy et al. “Lorentz Group Equivariant Neural Network for Particle Physics”. In: Proceedings of the 37th International Conference on Machine Learning. Vol. 119. Proceedings of Machine Learning Research. PMLR, 2020, pp. 992–1002. arXiv: 2006.04780 [hep-ph].arxiv.org/abs/2006.04780
- [57]Shiqi Gong et al. “An Efficient Lorentz Equivariant Graph Neural Network for Jet Tagging”. In: Journal of High Energy Physics 2022.7 (2022), p. 030. DOI: 10.1007/JHEP07(2022)030. arXiv: 2201.08187 [hep-ph].DOI
- [58]Shikai Qiu et al. “A Holistic Approach to Predicting Top Quark Kinematic Properties with the Covariant Particle Transformer”. In: Physical Review D 107.11 (2023), p. 114029. DOI: 10.1103/PhysRevD.107.114029. arXiv: 2203.05687 [hep-ph].DOI
- [59]Alexander Bogatskiy et al. “Explainable Equivariant Neural Networks for Particle Physics: PELICAN”. In: Journal of High Energy Physics 2024.3 (2024), p. 113. DOI: 10.1007/JHEP03(2024)113. arXiv: 2307.16506 [hep-ph].DOI
- [60]David Ruhe, Johannes Brandstetter, and Patrick Forré. “Clifford Group Equivariant Neural Networks”. In: Advances in Neural Information Processing Systems. Vol. 36. 2023. arXiv: 2305.11141 [cs.LG].arxiv.org/abs/2305.11141
- [61]Jonas Spinner et al. Lorentz-Equivariant Geometric Algebra Transformers for High-Energy Physics. 2024. arXiv: 2405.18406 [physics.data-an].
- [62]Johann Brehmer et al. A Lorentz-Equivariant Transformer for All of the LHC. 2025. arXiv: 2411.00446 [hep-ph].DOI
- [63]Soledad Villar et al. “Dimensionless machine learning: Imposing exact units equivariance”. In: Journal of Machine Learning Research 24.109 (2023), pp. 1–32.arxiv.org/abs/2204.00887
- [64]Edgar Buckingham. “On Physically Similar Systems; Illustrations of the Use of Dimensional Equations”. In: Physical Review 4.4 (1914), pp. 345–376. DOI: 10.1103/PhysRev.4.345.DOI
- [65]Hannah Lawrence et al. To Augment or Not to Augment? Diagnosing Distributional Symmetry Breaking. 2025. arXiv: 2510.01349 [cs.LG].arxiv.org/abs/2510.01349
- [67]Keaton J Burns et al. “Dedalus: A flexible framework for numerical simulations with spectral methods”. In: Physical Review Research 2.2 (2020), p. 023068.DOI
- [68]Drummond B Fielding et al. “Multiphase gas and the fractal nature of radiative turbulent mixing layers”. In: The Astrophysical Journal Letters 894.2 (2020), p. L24.DOI
- [69]Olaf Ronneberger, Philipp Fischer, and Thomas Brox. “U-net: Convolutional networks for biomedical image segmentation”. In: International Conference on Medical image computing and computer-assisted intervention. Springer, 2015, pp. 234–241.arxiv.org/abs/1505.04597
- [70]Zhuang Liu et al. “A convent for the 2020s”. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022, pp. 11976–11986.
- [71]Jonas Spinner et al. “Lorentz Local Canonicalization: How to Make Any Network Lorentz-Equivariant”. In: Advances in Neural Information Processing Systems. Vol. 38. 2025. arXiv: 2505.20280.arxiv.org/abs/2505.20280
- [72]William Fulton and Joe Harris. Representation Theory: A First Course. Vol. 129. Graduate Texts in Mathematics. New York: Springer-Verlag, 1991. DOI: 10.1007/978-1-4612-0979-9.DOI
- [73]Nadav Dym, Hannah Lawrence, and Jonathan W. Siegel. Equivariant Frames and the Impossibility of Continuous Canonicalization. 2024. arXiv: 2402.16077 [cs.LG]. URL: https://arxiv.org/abs/2402.16077.
- [74]Patrick T. Komiske, Eric M. Metodiev, and Jesse Thaler. Pythia8 Quark and Gluon Jets for Energy Flow. Version v1. 2019. DOI: 10.5281/zenodo.3164691. URL: https://doi.org/10.5281/zenodo.3164691.

