betaFeedback

WorksComputer Science

Enforcing Symmetries on Pre-trained ML Models for Physics

Abstract

Post-hoc group averaging (or test-time augmentation) is a simple way to impose symmetry on a trained predictor without changing its architecture or retraining it. For finite and compact groups, averaging over a normalized Haar measure on the group gives the Reynolds operator: a canonical projection onto invariant or equivariant predictors with known non-increasing risk guarantees under certain assumptions. This paper provides numerical experiments on physics emulation models that demonstrate the effectiveness of the approach in this setting. We then extend this framework to non-compact group symmetries. For non-compact groups, the Haar measure cannot be normalized, so group averaging cannot be defined in the same way. We replace it with three alternatives: the Weyl unitary trick (which requires complexifying the space and extending the predictor), a data-dependent orthogonal projection (which involves a non-trivial optimization on the space of invariant or equivariant functions), and a frame averaging approach (which involves mapping each point to a set of orbit representatives). We illustrate the constructions theoretically and empirically on the unit-rescaling group and the Lorentz group. Our results show that in the non-compact case, the Weyl trick and frame averaging do not provide a consistent improvement over the baseline, except where the test data is out-of-distribution (an orders-of-magnitude improvement in this case) and, for the Weyl trick, in low-data regimes. The orthogonal projection can provide improvements in-distribution and out-of-distribution. We also observe that models trained directly on invariant features matched or beat the projection of an unconstrained one. We conclude that post-hoc symmetrization extends to non-compact groups, but it does not consistently outperform training a symmetric model.

Introduction

A common guiding principle in machine learning for the physical sciences is that predictive models should be consistent with the laws of physics. These laws are often manifestly symmetric, which constrains admissible observables and determines how physical quantities transform. When learning from data generated by a physical system, respecting these symmetries is not only conceptually natural, but can also improve robustness.

One way to incorporate symmetry into learning models is through invariant or equivariant architectures, which enforce symmetries by construction. Although such approaches have been successful in many settings, they are not always used in practice. Implementing equivariant models typically requires nontrivial representation-theoretic or invariant-theoretic machinery, and existing equivariant methods are not always compatible with state-of-the-art architectures. As a result, many modern machine learning models for physics prioritize well-understood, symmetry-agnostic architectures, relying on data to learn the physics rather than explicit symmetry constraints.

An alternative approach is to impose symmetry post hoc by projecting a trained model onto the space of symmetry-preserving functions [1, 2, 3, 4]. In this framework, a pretrained predictor ff is treated as an arbitrary function, and a new predictor f~\tilde{f} is obtained by averaging ff over group orbits. For compact groups, this procedure is canonically implemented by Reynolds averaging with respect to the normalized Haar measure of the group (a.k.a. the invariant volume measure). Theoretical work shows that this projection yields a provably nonnegative, and in some cases strictly positive, improvement in expected risk under invariance assumptions on the data distribution, without requiring retraining or architectural modification [5]. However, this result applies only to compact groups, without results for non-compact ones. Yet, many physical symmetries are non-compact. For instance, the Lorentz group plays a central role in particle physics.

In this work, we first present empirical evidence showing that the group averaging approach works in the compact case. To this end, we take pretrained surrogate models released with The Well [6] and compare each with its group-averaged version. In many cases, this mere act of averaging, even when the group is small, already improves the results significantly. We then adapt the methodology to non-compact groups. We first argue why the approach and its guarantees fail, and we then investigate alternatives.

The remainder of the paper is organized as follows. § 2 reviews related work on invariant and equivariant learning, data augmentation, and post-hoc symmetrization. § 3 introduces Reynolds averaging for compact groups, recalls its risk-improvement guarantee, and evaluates it on pretrained foundation models for physics. § 4 explains the obstruction posed by non-compact groups and develops projection methods for unit rescalings and Lorentz transformations, together with numerical experiments. § 5 discusses the limitations and concludes.

Related work

Symmetry has long served as an inductive bias in machine learning for physical systems, where predictors are constrained by transformation laws; building symmetries into network structure dates back at least to [7]. A large body of work enforces these constraints directly in the architecture: group-equivariant convolutional networks [8], their generalization to arbitrary compact group actions [9, 10], steerable and irreducible-representation-based layers [11, 12, 13], equivariant multilayer perceptrons for arbitrary matrix groups, including the Lorentz group O(1,3)O(1, 3) [14], and further exactly or approximately equivariant designs [15, 16, 17, 18]. A related group-averaging construction appears in variational quantum machine learning, where twirling projects gate generators onto the commutant of a unitary symmetry representation to construct equivariant quantum circuits [19]. A complementary invariant-theoretic viewpoint characterizes equivariant maps by generating scalar invariants and tensorial output transformation laws [20, 21, 22, 23]. This perspective is especially natural in physics, where scalar contractions often determine the allowed form of a model. All of these methods fix the symmetry constraint before training, by modifying the model class or the input representation.

A different route is to impose symmetry after training. Averaging a predictor over the group orbit is the classical symmetrization device, used, for example, to establish universality of invariant models [24] and to define permutation-invariant pooling [25]. For compact groups, averaging with respect to the normalized Haar measure gives the Reynolds operator, which projects an arbitrary predictor onto the invariant or equivariant class. When the data distribution and target are GG-invariant, this projection cannot increase squared risk and produces a strict improvement whenever the predictor has a nonzero non-equivariant component [5]. Related theoretical work quantifies the generalization and sample-complexity benefits of enforcing symmetry [26, 27, 28, 29].

On the empirical side, inexpensive post-hoc group averaging has been shown to improve already-trained surrogate models at test time, without retraining or architectural modification [30]. Several related methods impose symmetry without averaging over the entire group. Frame averaging evaluates a non-equivariant backbone over a small, input-dependent set of canonical frames and averages the resulting predictions [31, 32]. Learned canonicalization instead learns a transformation that maps each input to a canonical representative, applies a non-equivariant backbone, and transforms the output back [33]. Lie-algebra-based canonicalization extends this paradigm to continuous non-compact symmetry groups and can be applied to pre-trained neural operators [34]. Reynolds networks build the projection directly into the network architecture [4]. Probabilistic symmetrization instead learns an input-dependent probability distribution over group transformations. It evaluates the backbone in transformations sampled from this distribution, transforms the predictions back, and averages them. The distribution is designed to shift consistently when the input is transformed, making the exact expected prediction equivariant [35]. Finite sampled averages over transformations are closely related to data augmentation and test-time ensembling [2], and are used for non-compact groups as well: multi-crop and multi-scale evaluation average a trained network over translations and rescalings [36, 37], and Lorentz-boost augmentation has been used to train jet taggers [38]. Unlike compact Reynolds projection, such procedures depend on the chosen sampling distribution and need not define an exact projection.

Our compact-group experiments concern surrogate models for spatiotemporal dynamical systems. Learned surrogates have demonstrated strong performance across many physical domains [39, 40, 41, 42, 43, 44, 45, 46], and these successes have motivated large simulation benchmarks [47, 48], including the Well [6], which provides the datasets and pretrained models used in § 3.3.

Lorentz symmetry is central in particle physics and has motivated a substantial literature on symmetry-aware collider learning. Classical invariant theory implies that scalar polynomial invariants are generated by pairwise Minkowski products [49], this viewpoint has been developed for Lorentz- and permutation-invariant particle features and invariant-polynomial machine learning [50, 51, 20]. Early successful jet taggers represented jets as sets or clouds of particles, including energy-flow and particle-flow networks [52], ParticleNet [53], and transformer-based models such as the Particle Transformer [54]. Other methods incorporate relativistic structure explicitly: the Lorentz Layer introduced Lorentz-invariant combinations into top tagging [55]; the Lorentz Group Network builds equivariant layers from finite-dimensional Lorentz representations and tensor products [56]; LorentzNet uses Minkowski products and four-vector message passing [57]; the Covariant Particle Transformer gives a partially Lorentz-covariant transformer for top-quark reconstruction [58]; and PELICAN constructs permutation-equivariant, Lorentz-invariant or covariant aggregators [59]. More recent Clifford- and geometric-algebra architectures, including CGENN and L-GATr, use multivector representations to build Lorentz-aware networks for high-energy physics [60, 61, 62]. These works impose Lorentz structure by architecture design, whereas our focus is on post-hoc symmetrization of a given black-box predictor.

Since the Lorentz group is non-compact, the compact Reynolds-averaging argument does not apply directly. Similar issues arise for other non-compact physical symmetries, such as dimensional rescalings. In [63] the authors use the Buckingham–Π theorem to replace dimensional inputs by dimensionless invariant features [64] and they explain how to project functions onto the space of invariants using the Weyl unitarian trick. In § 4.6 we show how to apply the Weyl trick to the Lorentz group. Our Weyl unitarian Lorentz average can therefore be viewed as a post-hoc invariant projection for polynomial or holomorphic predictors.

A further complication is that collider tasks often exhibit only partial Lorentz symmetry. The beam axis, detector frame, fiducial cuts, jet definitions, and preprocessing choices can introduce preferred directions or frames. Recent Lorentz-equivariant architectures address this through controlled symmetry breaking, for example, by adding reference multivectors associated with the beam or detector frame [62]. This is in part related to broader work showing that augmentation or equivariance can be harmful when transformed examples are not representative of the relevant data distribution [65]. Our jet-tagging experiments should therefore not be interpreted as constructing a fully Lorentz-invariant predictor. Averaging over a compact subgroup enforces invariance only under that subgroup, while averaging over a bounded boost window depends on the chosen window and is not a group projection. Instead, these experiments test whether averaging over selected physically relevant transformations improves prediction.

Compact groups

Compact group averaging and Reynolds projection

We begin with the setting in which post-hoc symmetrization is canonical. Let GG be a group acting on a vector space XX, and write the action as

ϕ(g)x=g⋅x,g∈G, x∈X.\phi(g)x = g \cdot x,\qquad g \in G,\ x \in X.

where ϕ:G→GL(X)\phi:G \to\mathrm{GL}(X), with GL(X)\mathrm{GL}(X) the group of invertible linear maps X→XX \to X, and ϕ(g)\phi(g) is a matrix that specifies how elements in XX transform under gg. In particular ϕ\phi is a group representation, and therefore it satisfies the transformation rules ϕ(gh)=ϕ(g)ϕ(h)\phi(gh) = \phi(g)\phi(h) and ϕ(g−1)=ϕ(g)−1\phi(g^{-1}) = \phi(g)^{-1}.

We consider another real vector space YY equipped with a representation ψ:G→GL(Y)\psi:G \to\mathrm{GL}(Y). A predictor f:X→Yf:X \to Y is GG-equivariant if

f(ϕ(g)x)=ψ(g)f(x)∀x∈X, ∀g∈G.(1)f\bigl(\phi(g)x\bigr) = \psi(g)f(x) \qquad\forall x \in X,\ \forall g \in G. \tag*{(1)}

In the special case where ψ(g)=I\psi(g)=I, this reduces to invariance: f(ϕ(g)x)=f(x)f\bigl(\phi(g)x\bigr)=f(x). In physical applications, equivariance expresses the requirement that transforming the input physical state and then predicting gives the same result as predicting first and then transforming the output.

Equivariance can be written as an invariance condition on the space of predictors. Define the induced action

(Tgf)(x):=ψ(g−1)f(ϕ(g)x).(T_gf)(x) := \psi(g^{-1})f(\phi(g)x).

Then ff is GG-equivariant if and only if

Tgf=f∀g∈G.T_gf = f \qquad\forall g \in G.

Thus the equivariant predictors are precisely the fixed points of the induced action TgT_g.

Now assume that GG is compact. Then GG admits a normalized Haar probability measure λ\lambda. That is, a measure satisfying λ(G)=1\lambda(G) = 1 and the invariance property

λ(gA)=λ(Ag)=λ(A)\lambda(gA) = \lambda(Ag) = \lambda(A)

for every measurable set A⊆GA \subseteq G and every g∈Gg \in G. This allows us to define the Reynolds operator as a generalization of the concept of averaging over the group orbits

(QGf)(x):=∫Gψ(g−1)f(ϕ(g)x) dλ(g)=∫G(Tgf)(x) dλ(g).(2)(Q_Gf)(x) := \int_G \psi(g^{-1})f(\phi(g)x)\,\mathrm{d}\lambda(g) = \int_G (T_gf)(x)\,\mathrm{d}\lambda(g). \tag*{(2)}

For a finite group, this is simply

(QGf)(x)=1∣G∣∑g∈Gψ(g−1)f(ϕ(g)x).(3)(Q_Gf)(x) = \frac{1}{\lvert G\rvert}\sum_{g\in G}\psi(g^{-1})f(\phi(g)x). \tag*{(3)}

In words, evaluate the trained model ff on transformed versions of the same input, transform the outputs back, and average.

The averaged predictor QGfQ_Gf is exactly GG-equivariant. Indeed, for any h∈Gh \in G,

ThQGf=∫GThTgf dλ(g)=∫GTghf dλ(g)=QGf,T_hQ_Gf = \int_G T_hT_gf\,\mathrm{d}\lambda(g) = \int_G T_{gh}f\,\mathrm{d}\lambda(g) = Q_Gf,

where the final equality follows from right invariance of Haar measure. Hence QGfQ_Gf is fixed by every ThT_h, and therefore satisfies the equivariance condition (1).

The statistical benefit of Reynolds averaging follows from its relation to orthogonal projections. Let μ\mu be a GG-invariant probability distribution on XX. Here L2(X,μ;Y)L^2(X,\mu;Y) carries the inner product

⟨f,h⟩L2(μ)=∫X⟨f(x),h(x)⟩Y dμ(x).\langle f,h\rangle_{L^2(\mu)} = \int_X \langle f(x),h(x)\rangle_Y\,\mathrm{d}\mu(x).

If we further assume that the induced group action TgT_g is unitary on L2(X,μ;Y)L^2(X,\mu;Y) we have that ⟨Tgf,Tgh⟩L2(μ)=⟨f,h⟩L2(μ)\langle T_gf,T_gh\rangle_{L^2(\mu)} = \langle f,h\rangle_{L^2(\mu)} for all f,h∈L2(X,μ;Y)f,h \in L^2(X,\mu;Y) and g∈Gg \in G. We define the equivariant subspace

HμG:={h∈L2(X,μ;Y):h∘ϕ=ψ∘h}.\mathcal{H}_{\mu}^{G} := \{h \in L^2(X,\mu;Y) : h \circ\phi= \psi\circ h\}.

The L2(μ)L^2(\mu)-orthogonal projection onto this subspace is

Pμf:=argmin⁡h∈HμG∥f−h∥L2(μ)2.(4)P_\mu f := \underset{h\in\mathcal{H}_{\mu}^{G}}{\operatorname{argmin}} \lVert f-h\rVert_{L^2(\mu)}^2. \tag*{(4)}

For a compact group, if μ\mu is GG-invariant, the Reynolds operator coincides with this projection [5],

QGf=Pμf.(5)Q_Gf = P_\mu f. \tag*{(5)}

Consequently, under squared loss with a GG-equivariant target f∗f^*, the orthogonal decomposition gives

R(f)−R(QGf)=∥f−QGf∥L2(μ)2≥0.\mathcal{R}(f) - \mathcal{R}(Q_Gf) = \lVert f-Q_Gf\rVert_{L^2(\mu)}^2 \ge0.

Thus under the assumptions above, Reynolds averaging cannot increase the expected risk and decreases it strictly whenever ff has a nonzero non-equivariant component, equivalently whenever f≠QGff \ne Q_Gf in L2(μ)L^2(\mu).

Application to surrogate models

Surrogate models estimate the state variables of a spatiotemporal dynamical system, such as a parameterized family of partial differential equations (PDEs) with fixed parameters or the observations of a physical system. The solution to a system of ss state variables over a spatial domain D\mathbb{D} and for all times t≥0t \ge0 is represented as u:D×[0,∞)→Rs\mathbf{u} : \mathbb{D} \times[0,\infty) \to\mathbb{R}^s. The domain D\mathbb{D} is typically a subset of two- or three-dimensional space, such as [0,Lx]×[0,Ly][0,L_x] \times[0,L_y] or [0,Lx]×[0,Ly]×[0,Lz][0,L_x] \times[0,L_y] \times[0,L_z]. We employ a standard uniform discretization of the system in time and space and denote by NDN_D the number of elements in the discretized domain D\mathbb{D}. A snapshot ut:R→Rs×ND\mathbf{u}_t : \mathbb{R} \to\mathbb{R}^{s \times N_D} represents the values of all ss state variables at all NDN_D points in the discretized domain at a fixed time tt.

The surrogate model M:Rk×s×ND→Rs×NDM : \mathbb{R}^{k \times s \times N_D} \to\mathbb{R}^{s \times N_D} is an autoregressive prediction problem that maps a sequence of kk snapshots Uti=[uti−k+1,…,uti−1,uti]\mathbf{U}_{t_i} = [\mathbf{u}_{t_i-k+1}, \ldots, \mathbf{u}_{t_i-1}, \mathbf{u}_{t_i}] to the snapshot of the following time step such that

uti+1≈u^ti+1=M(Uti).(6)\mathbf{u}_{t_{i+1}} \approx\widehat{\mathbf{u}}_{t_{i+1}} = M(\mathbf{U}_{t_i}). \tag*{(6)}

The goal of the model MM is to minimize some distance between its predictions u^t\widehat{\mathbf{u}}_t and the truth ut\mathbf{u}_t. The initial conditions at t0t_0 are given by kk snapshots Ut0\mathbf{U}_{t_0}, so that the estimated next sequence is updated according to U^t1=[ut0−k+2,…,ut0,u^t1]\widehat{\mathbf{U}}_{t_1} = [\mathbf{u}_{t_0-k+2}, \ldots, \mathbf{u}_{t_0}, \widehat{\mathbf{u}}_{t_1}]. The model MM is applied repeatedly to generate estimates for all desired times.

Our goal is to improve already-trained models that do not inherently respect some particular symmetry of the problem, by group averaging by the use of (3). In [5] it is proven that this averaging procedure reduces the error of the predictions. The assumptions to obtain provable improvements are:

  • The underlying physical problem is GG-invariant. This often holds because governing equations satisfy symmetries.

  • The distribution of the training data is GG-invariant.

Our goal is to predict the dynamics of a system with a known symmetry. We can improve a surrogate model MM as in (6) with initial sequence Vt0≡Ut0\mathbf{V}_{t_0} \equiv\mathbf{U}_{t_0} by group averaging

v^ti+1=1∣G∣∑g∈Gψ(g−1)M(ϕ(g)Vti).(7)\widehat{\mathbf{v}}_{t_{i+1}} = \frac{1}{|G|} \sum_{g \in G} \psi(g^{-1})M(\phi(g)\mathbf{V}_{t_i}). \tag*{(7)}

When the symmetry group is continuous, our approach approximates the continuous group average (2) using a Monte Carlo method with nn random samples from its elements. In the minimal case, for n=1n=1, the Monte Carlo approximation reduces to

v^ti+1(1)=ψ(gi−1)M(ϕ(gi)Vti),(8)\widehat{\mathbf{v}}_{t_{i+1}}^{(1)} = \psi(g_i^{-1})M(\phi(g_i)\mathbf{V}_{t_i}), \tag*{(8)}

where each gig_i is a randomly sampled group element of GG.

We provide numerical examples for surrogate models in the next Section. We argue that this simple procedure provides an efficient, zero-shot method that allows us to inject physical knowledge into a model after it has been trained.

Numerical experiments

We use the simulations and models published in “The Well” [6]. The project provides 15 TB of simulation data across 16 datasets, ranging from biological systems, fluid dynamics, acoustic scattering, as well as magneto-hydrodynamic simulations of extra-galactic fluids and supernova explosions. We chose to analyze models corresponding to datasets defined on a Cartesian 2D grid with symmetric initial and boundary conditions. For an overview of the groups used in this work, see Appendix § 6. The dataset properties and state variables are summarized in Appendix § 7.

The active matter dataset [6] consists of simulations of a continuum theory describing the dynamics of rod-like active particles immersed in a Stokes fluid. The simulations were run at constant particle density and varying values for the parameters dimensionless active dipole strength and strength of particle alignment through steric interactions. The initial conditions of the quantities are plane-wave perturbations about the isotropic state. The gray_scott_reaction_diffusion dataset (from now on abbreviated as gray_scott) contains numerical solutions of the Gray-Scott equations. They are a set of coupled reaction-diffusion equations that describe two chemical species, AA and BB, whose concentrations vary in space and time. Two parameters control the “feed” and “kill” rates in the reaction. The initial conditions are either a random Fourier series or random clusters of Gaussians. The boundary conditions are periodic for both datasets. The underlying physical problems are sampled on a square grid and have the symmetry of the torus T2\mathbb{T}^2. Toroidal symmetry allows all variables to translate periodically in both directions. Additionally, the problem is invariant under rotations and inversions, which is the discrete symmetry of the square D4D_4.

Additionally we analyze two datasets on a rectangular, non-square grid. The boundary conditions are periodic in one direction and Neumann in the other. The rayleigh_benard dataset [67] contains simulations of convection between a hot and cold fluid layer. Initial random Gaussian noise creates density variations which causes fluid motion. The underlying problem has the symmetry of the circle S1S^1 as well as inversions D2D_2. The turbulent_radiative_layer_2D dataset [68] contains simulations of cold dense gas moving relative to hot dilute gas. Once turbulence is induced due to the Kelvin Helmholtz instability, mixing of the two layers occurs. While the equations governing this process are invariant under translations and inversions, the initial conditions are not sampled from a distribution that is invariant under inversions. In all samples of the training set, the gas moves preferred in one direction. Consequently, the surrogate model does not respect this symmetry either.

The predictive power of the trained models vary greatly between the different datasets and models. Group averaging provides no improvement for the shear_flow dataset because the specific initial conditions break the system’s underlying symmetry, even though the theory itself and the boundary conditions are symmetric.

The CNextU-Net [69, 70] was chosen in the analysis of the datasets due to its consistent performance. The predictions u^t\hat{\mathbf{u}}_t and v^t\hat{\mathbf{v}}_t generated by (6) and (7) must be compared to the simulations ut\mathbf{u}_t. To measure how well a prediction fits the simulation, we employ the variance scaled root mean squared error (VRMSE)

LVRMSE(u,v)=[⟨∣u−v∣2⟩⟨∣v−⟨v⟩∣2⟩+ϵ]1/2,(9)\mathcal{L}_{\mathrm{VRMSE}}(\mathbf{u},\mathbf{v}) = \left[\frac{\langle|\mathbf{u}-\mathbf{v}|^2\rangle}{\langle|\mathbf{v}-\langle\mathbf{v}\rangle|^2\rangle+\epsilon}\right]^{1/2}, \tag*{(9)}

where ⟨⋅⟩\langle\cdot\rangle denotes the spatial mean operator, and ϵ=10−7\epsilon=10^{-7} to prevent division by zero in case Var⁡(v)=0\operatorname{Var}(\mathbf{v})=0. Lower VRMSE indicates higher agreement. LVRMSE≳1\mathcal{L}_{\mathrm{VRMSE}}\gtrsim1 signifies poor performance, as it indicates a prediction that is less accurate than a naively predicting the constant mean.

Results

We evaluate the performance improvement of the group averaging method by comparing the errors of the baseline prediction LVRMSE(u^t,ut)\mathcal{L}_{\mathrm{VRMSE}}(\hat{\mathbf{u}}_t,\mathbf{u}_t) and of the equivariant prediction LVRMSE(v^t,ut)\mathcal{L}_{\mathrm{VRMSE}}(\hat{\mathbf{v}}_t,\mathbf{u}_t). Figure 1 provides illustrations. The results are summarized in Table 1 and Figure 2.

Example comparisons of time evolutions of the concentration of two datasets

Figure 1. Example comparisons of time evolutions of the concentration of two datasets. (a) After t=10t = 10 steps we visually start to observe the benefits of group averaging. The equivariant model has a lower loss than the baseline at all times and is closer to the simulation (truth). At later times significant deviations between the approaches become apparent. (b) The equivariant model captures the features of the simulation relatively well for long times, while the baseline model develops several fictitious clusters.

Figure 2. Losses for the concentration predictions. In both cases the translations of T2\mathrm{T}^{2} are performed with a single (n=1n = 1) random translation per time step.

PanelModelStart15102050Rollout
active_matterBaseline0.0630.3860.93810.81
active_matterD4D_4100.0350.2360.5947.012
active_matterT2\mathbb{T}^20.0660.3190.8039.277
active_matterBaseline500.0380.2790.6637.659
active_matterD4D_40.0260.1930.4975.821
active_matterT2\mathbb{T}^20.0380.2020.5075.976
gray_scottBaseline0.0080.1540.7421.833
gray_scottD4D_4100.0070.1460.6801.730
gray_scottT2\mathbb{T}^20.0080.1260.5981.511
gray_scottBaseline500.0040.0730.5300.879
gray_scottD4D_40.0030.0640.4530.764
gray_scottT2\mathbb{T}^20.0040.0510.3570.608
rayleigh_benardBaseline0.0650.3460.6274.246
rayleigh_benardD2D_2500.0540.3110.5893.795
rayleigh_benardS1S^10.0650.2940.5263.664
rayleigh_benardBaseline1000.0350.2430.4362.908
rayleigh_benardD2D_20.0320.2160.3942.586
rayleigh_benardS1S^10.0340.1980.3632.395
turbulent_radiative_layer_2DBaseline0.1420.3500.5797.834
turbulent_radiative_layer_2DSn=11S^1_{n=1}00.1710.3550.5377.813
turbulent_radiative_layer_2DSn=81S^1_{n=8}0.1440.3060.4876.911
turbulent_radiative_layer_2DBaseline500.2010.4990.7449.128
turbulent_radiative_layer_2DSn=11S^1_{n=1}0.2040.4850.7308.939
turbulent_radiative_layer_2DSn=81S^1_{n=8}0.1740.4490.7228.605

Table 1. Average loss values for the datasets. The different starting positions in the trajectory and the used groups are given. The loss values are the averages of the LVRMSE\mathcal{L}_{\mathrm{VRMSE}} over all trajectories and state variables in the respective test sets. The loss is evaluated at the time steps indicated above each column. The rollout loss is the sum of the first 15 errors after the initial sequence. The continuous symmetries S1S^1 and T2\mathbb{T}^2 are approximated with n=1n=1 random group elements, except in (d), as indicated.

We experimented with the number of random Monte Carlo samples, nn, used to approximate the continuous groups. We observe significant improvements even at n=1n=1 using Eq. (8), demonstrating an enhancement over the baseline surrogate models at no additional computational cost. Table 1d shows how the group average improves as nn increases.

The group averaging procedure outperforms the baseline prediction on all time frames, even if we sum over only one random representative of a continuous group per time step. At sufficiently late times, the models lose predictive power and the loss approaches 1. The rate of loss growth and the optimal averaging group are model-dependent.

Non-compact groups

For non-compact groups, the averaging construction encounters a few obstructions. We first describe the issues, then introduce the Weyl unitary trick and frame averaging as the alternatives, and finally study unit rescaling and Lorentz symmetry numerically.

Why the compact argument fails for non-compact groups

The first obstruction concerns the measure. If GG is locally compact, then the group invariant (Haar) measure exists, but it has finite total mass if and only if GG is compact. Thus, for non-compact GG, the formal expression

∫GTgf dg\int_G T_g f\,\mathrm{d}g

is not a normalized average and typically diverges. One may introduce a weight or restrict the integral to a finite window,

Qwf:=∫Gw(g)Tgf dg,(10)Q_w f := \int_G w(g)T_g f\,\mathrm{d}g, \tag*{(10)}

but this no longer gives a canonical Reynolds projection. The weight ww selects a preferred region of the group, which is shifted by the group action. Consequently, windowed averaging may be a useful numerical heuristic, but it does not generally produce an exactly GG-equivariant predictor. A related localized sampling scheme is used for Lorentz data augmentation in [71], where boosts are sampled from a distribution concentrated near the identity. Although this procedure is used during training rather than for post-hoc averaging, it similarly selects a preferred region of the non-compact group.

The second obstruction concerns the data distribution. Group averaging evaluates the predictor along the orbit

{g⋅x:g∈G}.\{g \cdot x : g \in G\}.

For a compact group, this orbit is compact, whereas for a non-compact group it may be unbounded. A trained model is typically reliable only on or near the support of its training distribution. Since the training data is finite, the training distribution is bounded, and non-compact transformations can move an input far outside this region.

The third obstruction is statistical. As discussed in § 3.1, for a compact group with a GG-invariant probability distribution μ\mu, the Reynolds operator Eq. (2) coincides with the L2(μ)L^{2}(\mu)-orthogonal projection Eq. (4) onto the equivariant subspace. For a non-compact group, we may not have a probability distribution that is GG-invariant (due to the non-compactness of orbits discussed above). We may be able to extend it to arbitrary invariant distributions, and still define PμP_{\mu} whenever that subspace is closed, but this projection depends on μ\mu, and there is generally no group average that coincides with it. For the unit-rescaling group, Appendix § 6 of [63] shows explicitly that the Weyl-type Reynolds projection on Laurent monomials is not, in general, the orthogonal projection for a nontrivial real data distribution. Therefore the compact-group risk-improvement identity does not extend automatically to the non-compact setting.

The Weyl unitarian trick

In this section, we explain an analogous group averaging method for non-compact groups: the Weyl unitarian trick [72]. It extends the group averaging construction to any group arising as a real form of a complex reductive group (for example, scalings, Lorentz, symplectic, and many other groups of relevance for physics applications) at the cost of passing through the complex numbers.

We first explain how the projection is defined for a simple example. A Laurent polynomial is a function of the form

f(x)=∑α∈Scαxα(11)f(x) = \sum_{\alpha\in S} c_\alpha x^\alpha \tag*{(11)}

which allows both positive and negative integer powers and S⊂ZS \subset\mathbb{Z} finite. Consider the scaling action x↦gxx \mapsto gx for g∈R>0g \in\mathbb{R}_{>0} and x∈R×:=R∖{0}x \in\mathbb{R}^{\times} := \mathbb{R} \setminus\{0\}. Of the functions of the form (11), only the constant function fˉ(x)=c0\bar{f}(x) = c_0 is invariant. To apply the average to this scaling group, the Weyl unitarian trick first extends ff from R×\mathbb{R}^{\times} to C×:=C∖{0}\mathbb{C}^{\times} := \mathbb{C} \setminus\{0\} using the same Laurent-polynomial formula. The acting group R>0\mathbb{R}_{>0} is likewise complexified to the multiplicative group C×\mathbb{C}^{\times}, which contains the compact unit circle T={z∈C:∣z∣=1}\mathbb{T} = \{z \in\mathbb{C}: |z| = 1\} with normalized measure λ=dθ/2π\lambda= d\theta/2\pi. Averaging ff over this compact subgroup gives

fˉ(x)=∫Tf(zx) dλ(z)=∑α∈Scαxα∫Tzα dλ(z)=c0.\bar{f}(x) = \int_{\mathbb{T}} f(zx)\,d\lambda(z) = \sum_{\alpha\in S} c_\alpha x^\alpha\int_{\mathbb{T}} z^\alpha\,d\lambda(z) = c_0.

Two properties of this example hold in general. The first is the configuration of subgroups. Inside C×\mathbb{C}^{\times} sit two one-dimensional real subgroups: the scaling group R>0\mathbb{R}_{>0} whose symmetry we wished to impose, and the compact circle T\mathbb{T} over which we averaged. Each has real dimension equal to the complex dimension of C×\mathbb{C}^{\times}, and each has C×\mathbb{C}^{\times} as its complexification; such subgroups are the real forms of a complex group. Real forms of one group can be very different, one non-compact and another compact, and this is what the trick exploits: the symmetry is carried by one form and the averaging is carried out over the other. For connected GCG_{\mathbb{C}} the compact form is its maximal compact subgroup and is unique up to conjugacy. For the Lorentz group the two forms are O(1,3)O(1,3) and O(4)O(4), each of real dimension six, which become real forms of a single complex group only after the coordinate change that is described in § 4.6, which identifies O(1,3)O(1,3), with OC(4)O_{\mathbb{C}}(4). The second is rigidity. Holomorphic functions (i.e. complex differentiable) are rigid, whereas smooth ones (i.e. real infinitely differentiable) are not. A classic complex analysis theorem states that holomorphic function on a connected domain are determined by their values on a suitable slice.

Passing to complex inputs buys this rigidity, and rigidity carries an invariance imposed over the compact subgroup alone to the rest of complexified group.

We now state the general construction. Let GCG_{\mathrm{C}} be a complex reductive algebraic group with non-compact real form GRG_{\mathrm{R}} and compact real form K⊂GCK \subset G_{\mathrm{C}}. Assume the input and output actions extend to algebraic representations ϕC:GC→GL(XC)\phi_{\mathrm{C}}: G_{\mathrm{C}} \to GL(X_{\mathrm{C}}) and ψC:GC→GL(YC)\psi_{\mathrm{C}}: G_{\mathrm{C}} \to GL(Y_{\mathrm{C}}); and let FC\mathcal{F}_{\mathrm{C}} be a finite-dimensional, GCG_{\mathrm{C}}-stable space of algebraic maps F:XC→YCF: X_{\mathrm{C}} \to Y_{\mathrm{C}} containing an extension of the real predictor f:X→Yf: X \to Y. The induced action TgT_g of § 3.1 carries over unchanged, with equivariant maps again its fixed points, and the compact KK carries a normalized Haar measure λK\lambda_K, so we may define

(QKF)(u):=∫KψC(k−1)F(ϕC(k)u) dλK(k)=∫K(TkF)(u) dλK(k),(12)(Q_KF)(u) := \int_K \psi_{\mathrm{C}}(k^{-1})F(\phi_{\mathrm{C}}(k)u)\,d\lambda_K(k) = \int_K (T_kF)(u)\,d\lambda_K(k), \tag*{(12)}

which is the Reynolds operator (2) with KK in place of GG.

We can understand this construction in two different perspectives:

Group-average view. Similar to the compact-group average, this trick evaluates the complexified predictor at transformed inputs, transforms the outputs back, and averages against a probability measure. It is the same group averaging that has been done for the compact group, but we take a route that does the average which avoids the issues of non-compactness.

Representation-theoretic view. The same operator has an equivalent description in the language of representations: QKQ_K projects onto the trivial component of the induced representation of KK on FC\mathcal{F}_{\mathrm{C}}, that is, onto the KK-fixed subspace, and rigidity reads

FCK=FCGC.\mathcal{F}_{\mathrm{C}}^K = \mathcal{F}_{\mathrm{C}}^{G_{\mathrm{C}}}.

Retaining the trivial isotypic piece for KK is therefore the same as retaining it for the whole complex group.

Limitations. The construction is narrower than compact-group averaging in two respects. First, it requires a common complex domain on which both the predictor and the Weyl unitarian projection are defined: polynomial, Laurent-polynomial, and entire holomorphic predictors qualify, whereas a network with non-analytic activations such as ReLU may not. Second, it also loses the risk guarantee. For a compact group, we have the results in Eq. (5); for a non-compact group they do not necessarily hold. Here QKQ_K is canonical, closed-form, and blind to the data distribution, whereas PμP_\mu carries the risk interpretation but depends on μ\mu and has no closed form. Which of the two yields lower risk is therefore an empirical question, taken up in Section 4.5.2 and § 4.6.3.

Data-dependent L2(μ)L^2(\mu) projection

The other way to avoid averaging over a non-compact group is to define projection relative to the input distribution. Let μ\mu be a probability distribution on the input space XX, and let YY be an inner-product space. For predictors u,v:X→Yu,v:X\to Y, define

⟨u,v⟩μ:=Ex∼μ[⟨u(x),v(x)⟩Y],∥u∥Hμ2:=Ex∼μ[∥u(x)∥Y2].\langle u,v\rangle_{\mu} := \mathbb{E}_{x\sim\mu}[\langle u(x),v(x)\rangle_Y], \qquad\|u\|_{\mathcal{H}_{\mu}}^2 := \mathbb{E}_{x\sim\mu}[\|u(x)\|_Y^2].

where Hμ:=L2(X,μ;Y)\mathcal{H}_{\mu} := L^2(X,\mu;Y) is the space of predictors with finite L2(μ)L^2(\mu) norm.

Let HμG⊆Hμ\mathcal{H}_{\mu}^G \subseteq\mathcal{H}_{\mu} be the set of GG-equivariant predictors, namely those that transform according to the prescribed group actions on XX and YY. We assume that HμG\mathcal{H}_{\mu}^G is a closed linear subspace, which guarantees that every f∈Hμf\in\mathcal{H}_{\mu} has a unique closest equivariant predictor. This predictor is the L2(μ)L^2(\mu)-orthogonal projection

Pμf:=arg⁡min⁡h∈HμG Ex∼μ[∥f(x)−h(x)∥Y2].(13)P_{\mu}f := \underset{h\in\mathcal{H}_{\mu}^G}{\arg\min}\ \mathbb{E}_{x\sim\mu}\left[\|f(x)-h(x)\|_Y^2\right]. \tag*{(13)}

Its residual is orthogonal to every equivariant predictor:

⟨f−Pμf,h⟩μ=0for every h∈HμG.\langle f-P_{\mu}f,h\rangle_{\mu}=0 \qquad\text{for every } h\in\mathcal{H}_{\mu}^G.

Let f⋆(x)=E[y∣x]f_{\star}(x)=\mathbb{E}[y\mid x] be the regression function for squared loss. If the prediction problem respects the symmetry, then f⋆∈HμGf_{\star}\in\mathcal{H}_{\mu}^G. Orthogonality therefore gives

∥f−f⋆∥Hμ2=∥Pμf−f⋆∥Hμ2+∥f−Pμf∥Hμ2.(14)\|f-f_{\star}\|_{\mathcal{H}_{\mu}}^2=\|P_{\mu}f-f_{\star}\|_{\mathcal{H}_{\mu}}^2+\|f-P_{\mu}f\|_{\mathcal{H}_{\mu}}^2. \tag*{(14)}

Here ∥f−f⋆∥Hμ2\lVert f-f_{\star}\rVert_{\mathcal{H}_{\mu}}^{2} is the excess squared-loss risk of ff. Hence PμfP_{\mu}f cannot increase squared risk under μ\mu.

For compact GG and GG-invariant μ\mu, the group average is self-adjoint on Hμ\mathcal{H}_{\mu} and coincides with PμP_{\mu}, so (14) recovers the compact result of Elesedy and Zaidi [5]. For non-compact GG, Haar measure has infinite mass and μ\mu is not GG-invariant, so an averaging operator would not be self-adjoint in Hμ\mathcal{H}_{\mu} even where one can be defined. Averaging and projection come apart, and closedness of HμG\mathcal{H}_{\mu}^{G} must be assumed rather than inherited from a bounded averaging operator.

Frame averaging

Frame averaging provides a third route to post-hoc symmetrization. Intuitively, it assigns each input a finite set of input-dependent changes of coordinates that carry it to one or more canonical representatives of its group orbit. The original predictor is evaluated at these canonicalized inputs, after which each output is transported back to the coordinates of the original input and the results are averaged. Averaging over all admissible choices removes dependence on any finite ambiguity in the canonicalization, while the consistency condition below ensures that the resulting predictor is equivariant.

Using the input and output actions ϕ\phi and ψ\psi introduced in § 3.1, a finite frame assigns each x∈Xx \in X a nonempty finite subset F(x)⊆G\mathcal{F}(x) \subseteq G satisfying

F(ϕ(h)x)=hF(x)for every h∈G,(15)\mathcal{F}(\phi(h)x)=h\mathcal{F}(x)\qquad\text{for every }h\in G, \tag*{(15)}

where hF(x)={hg:g∈F(x)}h\mathcal{F}(x)=\{hg:g\in\mathcal{F}(x)\} [31]. Given an arbitrary predictor f:X→Yf:X\to Y, define

(QFf)(x)=1∣F(x)∣∑g∈F(x)ψ(g)f(ϕ(g−1)x).(16)(Q_{\mathcal{F}}f)(x)=\frac{1}{\lvert\mathcal{F}(x)\rvert}\sum_{g\in\mathcal{F}(x)}\psi(g)f\left(\phi(g^{-1})x\right). \tag*{(16)}

Indeed, writing g=hug=hu in the frame at ϕ(h)x\phi(h)x gives

(QFf)(ϕ(h)x)=1∣F(x)∣∑u∈F(x)ψ(hu)f(ϕ(u−1)x)=ψ(h)(QFf)(x).(Q_{\mathcal{F}}f)(\phi(h)x)=\frac{1}{\lvert\mathcal{F}(x)\rvert}\sum_{u\in\mathcal{F}(x)}\psi(hu)f\left(\phi(u^{-1})x\right)=\psi(h)(Q_{\mathcal{F}}f)(x).

Thus QFfQ_{\mathcal{F}}f is exactly GG-equivariant; when ψ(g)=I\psi(g)=I, it is invariant.

Because the average is finite, this construction does not require a normalized Haar measure and can be applied to a non-compact group whenever an equivariant finite frame is available. The concrete frames used below are obtained by bringing input-dependent quantities to canonical forms and averaging over any remaining finite ambiguity; their groups and regularity conditions are stated in the corresponding application sections. Although QFQ_{\mathcal{F}} fixes equivariant predictors and is therefore idempotent, it is generally not the Reynolds/Haar projection or the L2(μ)L^{2}(\mu)-orthogonal projection and hence does not automatically inherit their risk-reduction guarantees. A further limitation concerns regularity: unweighted frame averaging can turn a continuous predictor into a discontinuous equivariant one, motivating continuity preserving weighted-frame constructions [73].

Example: Unit-rescaling group

Consider predicting the drag force on a body moving through a fluid from the fluid density ρ\rho, the flow speed VV, and the body diameter ℓ\ell. In SI base units a configuration might read ρ=1.2 kg m−3\rho=1.2\ \mathrm{kg\,m^{-3}}, V=30 m s−1V=30\ \mathrm{m\,s^{-1}}, ℓ=0.1 m\ell=0.1\ \mathrm{m}, with drag force F=2.1 NF=2.1\ \mathrm{N}. Recorded in CGS units, with masses in grams and lengths in centimeters, the same configuration reads

ρ=1.2×10−3 g cm−3,V=3.0×103 cm s−1,ℓ=10 cm,F=2.1×105 dyn.\rho=1.2\times10^{-3}\ \mathrm{g\,cm^{-3}},\qquad V=3.0\times10^{3}\ \mathrm{cm\,s^{-1}},\qquad\ell=10\ \mathrm{cm},\qquad F=2.1\times10^{5}\ \mathrm{dyn}.

Each value is rescaled by the combination of conversion factors dictated by its physical dimensions. The physical state is unchanged; only the numbers representing it are.

A change of units is specified by one positive factor per base unit, here one each for mass, length, and time. These factors multiply under composition and are invertible, so the admissible conversions form the group (R>0)3(\mathbb{R}_{>0})^{3}, acting on numerical data by rescaling each coordinate by the monomial in the conversion factors determined by its dimensions.

We ask that a predictor commute with this action: converting the inputs to CGS units, predicting, and converting the predicted force back to newtons should return the SI prediction. A model fitted to raw numerical values need not have this property, since nothing prevents it from combining coordinates in dimensionally inconsistent ways.

We now state this in general, following [63]. Fix kk base units and consider an input x=(x1,…,xd)∈X:=(R>0)dx=(x_{1},\ldots,x_{d})\in X:=(\mathbb{R}_{>0})^{d}. The units of each coordinate xix_{i} are encoded by a unit vector ui=(ui1,…,uik)∈Zku_i=(u_{i1},\ldots,u_{ik})\in\mathbb{Z}^k, where uiju_{ij} is the exponent of the jj-th base unit in the units of xix_i. Collect these vectors as the columns of the unit matrix

U:=[u1 ⋯ ud]∈Zk×d.U := [u_1\ \cdots\ u_d]\in\mathbb{Z}^{k\times d}.

For w∈Zkw\in\mathbb{Z}^k and g∈(R>0)kg\in(\mathbb{R}_{>0})^k, write gw:=∏j=1kgjwjg^w:=\prod_{j=1}^k g_j^{w_j}. The unit-rescaling group is

Gunits:=(R>0)k.G_{\mathrm{units}} := (\mathbb{R}_{>0})^k.

An element g=(g1,…,gk)g=(g_1,\ldots,g_k) rescales the jj-th base unit by gjg_j and acts on the numerical input coordinates by

(g⋅x)i=g−uixi=(∏j=1kgj−uij)xi,i=1,…,d,(17)(g\cdot x)_i=g^{-u_i}x_i=\left(\prod_{j=1}^k g_j^{-u_{ij}}\right)x_i,\qquad i=1,\ldots,d, \tag*{(17)}

the exponent being negative because a larger unit gives a smaller numerical value. In the example above k=3k=3, the base units are (kg,m,s)(\mathrm{kg},\mathrm{m},\mathrm{s}), and the passage to CGS units is g=(10−3,10−2,1)g=(10^{-3},10^{-2},1); the density has unit vector uρ=(1,−3,0)u_\rho=(1,-3,0), and (17) returns the factor g−uρ=(10−3)−1(10−2)3=10−3g^{-u_\rho}=(10^{-3})^{-1}(10^{-2})^3=10^{-3} used there.

Together with the assigned units vectors u1,…,udu_1,\ldots,u_d, the input space XX is a units-typed space, which we denote by ZX\mathcal{Z}_X. A function f:ZX→ZXf:\mathcal{Z}_X\to\mathcal{Z}_X is unit-equivariant if

f(g⋅x)=g⋅f(x)for all g∈Gunits, x∈X.(18)f(g\cdot x)=g\cdot f(x)\qquad\text{for all }g\in G_{\mathrm{units}},\ x\in X. \tag*{(18)}

The drag force has vF=(1,1,−2)v_F=(1,1,-2), so g−vF=105g^{-v_F}=10^5, the newton-to-dyne factor. An output with v=0v=0 is dimensionless, and (18) reduces to unit invariance.

Weyl projection for the unit-rescaling group

Although GunitsG_{\mathrm{units}} is not itself an algebraic real form, it is the identity component of (R×)k(\mathbb{R}^{\times})^k, the real points of a split algebraic torus. The corresponding complex algebraic group and a compact real form are

GC=(C×)k,Tk:={z∈(C×)k:∣zj∣=1, j=1,…,k}.G_{\mathbb{C}}=(\mathbb{C}^{\times})^k,\qquad \mathbb{T}^k:=\{z\in(\mathbb{C}^{\times})^k:|z_j|=1,\ j=1,\ldots,k\}.

Both GunitsG_{\mathrm{units}} and Tk\mathbb{T}^k are Zariski dense in GCG_{\mathbb{C}}. Consequently, a Laurent polynomial satisfies a given units-equivariant transformation law under GunitsG_{\mathrm{units}} if and only if its complexification satisfies the corresponding transformation law under Tk\mathbb{T}^k.

Let

FC∈C[x1±1,…,xd±1]F_{\mathbb{C}}\in\mathbb{C}[x_1^{\pm1},\ldots,x_d^{\pm1}]

be a Laurent-polynomial predictor on XC:=(C×)dX_{\mathbb{C}}:=(\mathbb{C}^{\times})^d. The complex torus acts by the extension of equation (17), Appendix A of [63] defines the corresponding torus projection for invariant, dimensionless outputs. For a scalar output with units vector vv, the equivariant extension is

(QvFC)(x):=∫TkzvFC(z⋅x) dλT(z),(19)(Q_vF_{\mathbb{C}})(x):=\int_{\mathbb{T}^k}z^vF_{\mathbb{C}}(z\cdot x)\,d\lambda_{\mathbb{T}}(z), \tag*{(19)}

where λT\lambda_{\mathbb{T}} is normalized Haar measure. The factor zvz^v is the inverse output action: since the output transforms as y↦z−vyy\mapsto z^{-v}y, transforming the output back contributes zvz^v.

Proposition 4.1 (Units Weyl unitary projection). For a=(a1,…,ad)∈Zda=(a_1,\ldots,a_d)\in\mathbb{Z}^d, write

xa:=∏i=1dxiai.x^a:=\prod_{i=1}^d x_i^{a_i}.

If

FC(x)=∑a∈Scaxa,S⊂Zd finite,F_{\mathbb{C}}(x)=\sum_{a\in S}c_ax^a,\qquad S\subset\mathbb{Z}^d\ \text{finite},

then

(QvFC)(x)=∑a∈SUa=vcaxa.(Q_vF_{\mathbb{C}})(x)=\sum_{\substack{a\in S\\Ua=v}}c_ax^a.

Thus QvQ_v is the projection onto the Laurent-polynomial predictors satisfying

FC(z⋅x)=z−vFC(x).F_{\mathbb{C}}(z\cdot x)=z^{-v}F_{\mathbb{C}}(x).

In particular, Q0Q_0 retains exactly the dimensionless monomials.

Proof. Since

(z⋅x)a=z−Uaxa,(z \cdot x)^a = z^{-Ua}x^a,

we have

(Qvxa)(x)=xa∫Tkzv−Ua dλT(z).(Q_vx^a)(x) = x^a \int_{\mathbb{T}^k} z^{v-Ua}\,\mathrm{d}\lambda_{\mathbb{T}}(z).

The integrand is a character of Tk\mathbb{T}^k, whose Haar average is 1 when Ua=vUa=v and 0 otherwise. The result follows by linearity.

If FCF_C has real coefficients, then so does QvFCQ_vF_C. Its restriction to XX is therefore a real-valued units-equivariant predictor.

In words, QvQ_v creates no equivariant features; it only deletes the ones that are not. Each monomial xax^a is a joint eigenvector of the scaling action, with eigenvalue z−Uaz^{-Ua}, so QvQ_v is diagonal in the monomial basis: it keeps the monomials with Ua=vUa=v and annihilates the rest. The cost is expressivity: a predictor whose dictionary contains no monomial of weight vv projects to zero.

Numerical experiments with the unit-rescaling group

We consider force regression and Reynolds-number classification tasks. The regression inputs are density ρ\rho, dynamic viscosity μ\mu, speed VV, area AA, and length ℓ\ell, with units ML−3ML^{-3}, ML−1T−1ML^{-1}T^{-1}, LT−1LT^{-1}, L2L^2, LL, respectively. Classification uses the same variables except AA. The regression target is

y⋆=ρV2A(0.7+0.2Re⁡+0.15Re⁡−1),Re⁡:=ρVℓμ.y_\star= \rho V^2 A\left(0.7 + 0.2\operatorname{Re} + 0.15\operatorname{Re}^{-1}\right), \qquad\operatorname{Re} := \frac{\rho V\ell}{\mu}.

Since Re⁡\operatorname{Re} is dimensionless, y⋆y_\star has units of force, MLT−2MLT^{-2}, corresponding to the unit-weight vector vF=(1,1,−2)v_F=(1,1,-2).

For classification, the decision boundary is Re⁡=1\operatorname{Re}=1: the regime Re⁡>1\operatorname{Re}>1 is inertia dominated, whereas Re⁡<1\operatorname{Re}<1 is viscosity dominated. We define the true logit and class probability by

s⋆(x)=2.5(Re⁡−Re⁡−1),p⋆(x)=sigmoid⁡(s⋆(x)),sigmoid⁡(t):=(1+e−t)−1.s_\star(x) = 2.5\left(\operatorname{Re}-\operatorname{Re}^{-1}\right), \qquad p_\star(x)=\operatorname{sigmoid}(s_\star(x)), \qquad\operatorname{sigmoid}(t):=(1+e^{-t})^{-1}.

We compare the following methods:

Baseline: following the experiments from [63] we fit a ridge regression or ℓ2\ell^2-penalized logistic regression on a set of features, of which 10 are units-invariant and 250 are not invariant.

Weyl-projection: the projection of the baseline to the space of invariant/equivariant functions using the Weyl trick in this case gives us the restriction of the baseline model to the invariant features only. That is, zeroing out the coefficients of the non-invariant features.

Empirical L2L^2 projection: the L2L^2 projection of the baseline to the space of invariant/equivariant functions, obtained by minimizing Eq. (4), as discussed in Section 4.3.

Frame average: A change of mass, length, and time units has three scaling factors. Each of the following triples uniquely determines a change of units that makes its three variables equal to one:

S={(ρ,μ,V),(ρ,μ,ℓ),(ρ,V,ℓ),(μ,V,ℓ)}.\mathcal{S}=\{(\rho,\mu,V),(\rho,\mu,\ell),(\rho,V,\ell),(\mu,V,\ell)\}.

For S∈SS\in\mathcal{S}, let USU_S contain the powers of MM, LL, TT in the units of the selected variables, and define

gS(x):=exp⁡(US−T(log⁡x)S),κS(x):=gS(x)⋅x.g_S(x):=\exp\left(U_S^{-\mathsf{T}}(\log x)_S\right), \qquad\kappa_S(x):=g_S(x)\cdot x.

Then (κS(x))S=1(\kappa_S(x))_S=1; the remaining variables are rescaled consistently but are not generally equal to one. Moreover, gS(q⋅x)=gS(x)q−1g_S(q\cdot x)=g_S(x)q^{-1} and κS(q⋅x)=κS(x)\kappa_S(q\cdot x)=\kappa_S(x), so the inverse transformations gS(x)−1g_S(x)^{-1} form an equivariant frame.

We average the frozen baseline over these four canonicalized inputs:

(Qframef)(x):=14∑S∈SgS(x)vf(κS(x)).(20)(Q_{\mathrm{frame}}f)(x):=\frac{1}{4}\sum_{S\in\mathcal{S}}g_S(x)^v f(\kappa_S(x)). \tag*{(20)}

where gv:=gMvMgLvLgTvTg^v:=g_M^{v_M}g_L^{v_L}g_T^{v_T}. We use v=vFv=v_F for regression and v=0v=0 for the classification logit, applying the sigmoid after averaging. The factor gS(x)vg_S(x)^v converts the prediction back to the original units. This is the “Frame average m=4m=4” reported below. It reuses the fitted coefficients and original RMS scales without refitting.

Invariant regression: an invariant regression using only the invariant terms as aforementioned.

We train our model with two different training set,

Fixed units: all the training data is in one fixed unit system.

Transformed units (small shift): the training data are transformed with random units transformations sampled from the distribution

log⁡10gM,log⁡10gL,log⁡10gT∼iidUnif⁡[−1,1].(21)\log_{10} g_M, \log_{10} g_L, \log_{10} g_T \mathrel{\overset{\mathrm{iid}}{\sim}} \operatorname{Unif}[-1,1]. \tag*{(21)}

We further evaluate the performance in the following three scenarios:

Fixed units: the test data is in one fixed unit system as above.

Transformed units (small shift): the test data are transformed with random units transformations sampled from Eq. (21).

Transformed units (larger shift): the test data are transformed with random units transformations, sampled from

log⁡10gM,log⁡10gL,log⁡10gT∼iidUnif⁡[−2,2].(22)\log_{10} g_M, \log_{10} g_L, \log_{10} g_T \mathrel{\overset{\mathrm{iid}}{\sim}} \operatorname{Unif}[-2,2]. \tag*{(22)}

We also verify that the numerical group averaging produces an invariant/equivariant output. We compare the integral over a fixed grid with a Monte Carlo approximation. This, combined with further experiments studying the dependency on the number of samples are described in Appendix Appendix § 8.

The regression risk is dimensionless

R=NMSE⁡:=∑i(y^i−yi)2∑iyi2,NRMSE⁡:=R.R = \operatorname{NMSE} := \frac{\sum_i(\hat{y}_i-y_i)^2}{\sum_i y_i^2}, \qquad\operatorname{NRMSE} := \sqrt{R}.

The classification risk is the sample mean Bernoulli Kullback–Leibler (KL) divergence from the true probability pip_i to the fitted probability p^i\hat{p}_i:

RKL:=1n∑i[pilog⁡pip^i+(1−pi)log⁡1−pi1−p^i].R_{\mathrm{KL}} := \frac{1}{n}\sum_i\left[p_i\log\frac{p_i}{\hat{p}_i}+(1-p_i)\log\frac{1-p_i}{1-\hat{p}_i}\right].

All risks are nonnegative and lower is better. For each test set with transformed units, the risk is first averaged over 16 held-out unit systems within each seed. The tables report mean ±\pm sample standard deviation over ten independently generated and fitted seeds unless stated otherwise.

For classification, the baseline model outputs a logit s^(x)\hat{s}(x). We apply the projection to this logit and then convert the projected logit into a probability:

Regression training and validation targets contain one-percent multiplicative noise; tests use the noiseless target.

TaskTrainingModelFixed unitsTransformed units (small shift)Transformed units (large shift)
Regression (NRMSE)Fixed unitsBaseline0.0164 ± 0.00647(1.42±4.27)×1013(1.42 \pm4.27) \times10^{13}(7.12±21.9)×1026(7.12 \pm21.9) \times10^{26}
Regression (NRMSE)Fixed unitsWeyl-Projected0.0806 ± 0.06410.0806 ± 0.06370.0814 ± 0.0656
Regression (NRMSE)Fixed unitsEmpirical L2(μ)L^2(\mu)0.00936 ± 0.004740.0100 ± 0.004130.0101 ± 0.00379
Regression (NRMSE)Fixed unitsInvariant regression0.00487 ± 0.001250.00532 ± 0.001570.00525 ± 0.00163
Regression (NRMSE)Fixed unitsFrame average m=4m=42.11 ± 1.262.02 ± 1.142.31 ± 1.48
Regression (NRMSE)Transformed units (small shift)Baseline0.0325 ± 0.01520.703 ± 1.96(1.20±3.55)×1013(1.20 \pm3.55) \times10^{13}
Regression (NRMSE)Transformed units (small shift)Weyl-Projected0.0398 ± 0.02060.0390 ± 0.02000.0391 ± 0.0200
Regression (NRMSE)Transformed units (small shift)Empirical L2(μ)L^2(\mu)0.0402 ± 0.01740.0409 ± 0.01750.0430 ± 0.0243
Regression (NRMSE)Transformed units (small shift)Invariant regression0.00487 ± 0.001250.00532 ± 0.001570.00525 ± 0.00163
Regression (NRMSE)Transformed units (small shift)Frame average m=4m=40.0366 ± 0.01800.0362 ± 0.01730.0374 ± 0.0194
Classification (KL)Fixed unitsBaseline0.0156 ± 0.0023310.26 ± 1.3814.72 ± 1.04
Classification (KL)Fixed unitsWeyl-Projected0.137 ± 0.05140.138 ± 0.05160.138 ± 0.0520
Classification (KL)Fixed unitsEmpirical L2(μ)L^2(\mu)(1.55±1.25)×10−3(1.55 \pm1.25) \times10^{-3}(1.56±1.27)×10−3(1.56 \pm1.27) \times10^{-3}(1.55±1.26)×10−3(1.55 \pm1.26) \times10^{-3}
Classification (KL)Fixed unitsInvariant regression(0.971±0.743)×10−3(0.971 \pm0.743) \times10^{-3}(0.978±0.756)×10−3(0.978 \pm0.756) \times10^{-3}(0.964±0.738)×10−3(0.964 \pm0.738) \times10^{-3}
Classification (KL)Fixed unitsFrame average m=4m=40.0185 ± 0.01120.0186 ± 0.01130.0186 ± 0.0113
Classification (KL)Transformed units (small shift)Baseline0.0240 ± 0.01890.0675 ± 0.06276.01 ± 2.10
Classification (KL)Transformed units (small shift)Weyl-Projected0.0258 ± 0.01940.0259 ± 0.01940.0257 ± 0.0193
Classification (KL)Transformed units (small shift)Empirical L2(μ)L^2(\mu)0.0147 ± 0.01420.0147 ± 0.01410.0146 ± 0.0140
Classification (KL)Transformed units (small shift)Invariant regression(0.971±0.743)×10−3(0.971 \pm0.743) \times10^{-3}(0.978±0.756)×10−3(0.978 \pm0.756) \times10^{-3}(0.964±0.738)×10−3(0.964 \pm0.738) \times10^{-3}
Classification (KL)Transformed units (small shift)Frame average m=4m=40.0247 ± 0.01920.0248 ± 0.01930.0247 ± 0.0192

Table 2. Regression uses NRMSE and classification uses expected Bernoulli KL; lower is better. Here, training a new invariant model gives the best result in all cases.

The frame average is nearly insensitive to the test unit system, as expected from its exact equivariance. With fixed-unit training, it prevents catastrophic failures under unit shifts but can substantially reduce accuracy: the regression NRMSE remains near 2.1, compared with 0.0164 for the baseline in fixed units, while the classification KL remains near 0.0186 across all unit systems. With transformed-unit training, its performance improves considerably, with regression NRMSE near 0.037 and classification KL near 0.025 across all test conditions. Thus, frame averaging guarantees the correct transformation law, but its accuracy still depends on how well the frozen predictor performs on the canonicalized inputs.

The Weyl projection satisfies the prescribed units scaling law to numerical precision, so its risk is unchanged across the three unit conditions. It nevertheless increased the in-distribution risk for both tasks. When the baseline models were tested in unit systems with transformed units, the projection reduced the regression NRMSE from 1.56×10131.56 \times10^{13} to 0.0821 and the classification KL from 10.25 to 0.138. These large gains arose because the baseline models’ non-invariant terms transformed incorrectly. Using transformed units during training reduced, but did not eliminate, this sensitivity, especially for transformations out-of-distribution (such as the large shifts).

Conclusion. The Weyl projection can greatly improve predictions when test examples are expressed in unit systems not seen by an already trained baseline model, because it removes dependence on the non-invariant terms. It is not a universal accuracy improvement: it worsened the risk for the fixed units experiment. Its demonstrated benefit is its robustness to unit transformations. Frame averaging showed the same robustness with accuracy tradeoff: it prevented the catastrophic failures under unseen unit transformations, but it did not provide a uniform in-distribution improvement and performed particularly poorly for the fixed-unit regression baseline.

Example: Lorentz group

We now specialize the construction of § 4.2 to scalar-valued predictors that take Lorentz vectors as inputs. Invariance under O(1,d−1)O(1,d-1) means that applying the same Lorentz transformation to every input vector leaves the prediction unchanged. Unlike the unit-rescaling example, the compact real form is not contained in the real Lorentz group in the original coordinates. After Wick-rotating the time coordinate, however, the noncompact form O(1,d−1)O(1,d-1) and the compact form O(d)O(d) become real forms of the same complex orthogonal group. The projection therefore has three steps: pass to Wick coordinates, average over O(d)O(d), and restrict back to the real slice.

Throughout, let d≥2d \ge2 and use the sign convention η=diag⁡(−1,1,…,1)\eta=\operatorname{diag}(-1,1,\ldots,1). For p=(t,x)∈R1,d−1p=(t,x)\in\mathbb{R}^{1,d-1}, with x=(x1,…,xd−1)x=(x_1,\ldots,x_{d-1}), define the metric and the Minkowski invariant

⟨p,p′⟩η:=pTηp′=−tt′+x⋅x′,s(p):=⟨p,p⟩η=−t2+∥x∥2.(23)\langle p,p'\rangle_{\eta}:=p^{\mathsf{T}}\eta p'=-tt'+x\cdot x', \qquad s(p):=\langle p,p\rangle_{\eta}=-t^2+\lVert x\rVert^2. \tag*{(23)}

The full Lorentz group is

O(1,d−1)={A∈GLd(R):ATηA=η}.O(1,d-1)=\{A\in\mathrm{GL}_{d}(\mathbb{R}):A^{\mathsf{T}}\eta A=\eta\}.

and its complexification is

OC(1,d−1):={A∈GLd(C):ATηA=η}.O_{\mathbb{C}}(1,d-1):=\{A\in\mathrm{GL}_{d}(\mathbb{C}):A^{\mathsf{T}}\eta A=\eta\}.

Let

S=diag⁡(i,1,…,1).S=\operatorname{diag}(i,1,\ldots,1).

Since STS=ηS^{\mathsf{T}}S=\eta, conjugation by SS gives an isomorphism

OC(1,d−1)⟶OC(d)(24)O_{\mathbb{C}}(1,d-1)\longrightarrow O_{\mathbb{C}}(d) \tag*{(24)}
A⟼SAS−1.(25)A\longmapsto SAS^{-1}. \tag*{(25)}

In these Wick coordinates, O(d)⊂OC(d)O(d)\subset O_{\mathbb{C}}(d) is a compact real form. We define the Wick embedding

W:R1,d−1⟶Cd(26)W:\mathbb{R}^{1,d-1}\longrightarrow\mathbb{C}^{d} \tag*{(26)}
p=(t,x1,…,xd−1)⟼Sp=(it,x1,…,xd−1).(27)p=(t,x_1,\ldots,x_{d-1})\longmapsto Sp=(it,x_1,\ldots,x_{d-1}). \tag*{(27)}

It intertwines the Lorentz action with the complex orthogonal action:

W(Λp)=(SΛS−1)W(p),Λ∈O(1,d−1).(28)W(\Lambda p)=(S\Lambda S^{-1})W(p),\qquad\Lambda\in O(1,d-1). \tag*{(28)}

On Cd\mathbb{C}^d, let

B(u,v):=uTv,q(u):=B(u,u)=u02+⋯+ud−12.(29)B(u,v) := u^{\mathsf{T}}v,\qquad q(u) := B(u,u)=u_0^2+\cdots+u_{d-1}^2. \tag*{(29)}

For p,p′∈R1,d−1p,p' \in\mathbb{R}^{1,d-1},

B(W(p),W(p′))=⟨p,p′⟩η,q(W(p))=s(p).(30)B(W(p),W(p'))=\langle p,p'\rangle_\eta,\qquad q(W(p))=s(p). \tag*{(30)}

Thus WW converts Lorentz transformations into complex orthogonal transformations and Minkowski contractions into complex orthogonal contractions. We can therefore average in Wick coordinates and then restrict back.

Lorentz invariant functions with a single four-vector input

Let f:R1,d−1⟶Rf:\mathbb{R}^{1,d-1}\longrightarrow\mathbb{R} be a polynomial predictor, and let fCf_{\mathbb{C}} denote its coefficientwise complexification. Define the corresponding predictor in Wick coordinates by

F(u):=fC(S−1u),(31)F(u):=f_{\mathbb{C}}(S^{-1}u), \tag*{(31)}

so that F(W(p))=f(p)F(W(p))=f(p). Let K=O(d)K=O(d) act complex-linearly on Cd\mathbb{C}^d, and define

(QKF)(u):=∫KF(ku) dk,(32)(Q_KF)(u):=\int_K F(ku)\,\mathrm{d}k, \tag*{(32)}

where dk\mathrm{d}k is normalized Haar probability measure. The corresponding predictor on the Lorentz real slice is

(QLf)(p):=(QKF)(W(p)).(33)(Q_Lf)(p):=(Q_KF)(W(p)). \tag*{(33)}

Haar averaging is a linear idempotent projection onto the KK-invariant polynomials. Therefore QKQ_K, and hence QLQ_L, is a linear idempotent projection on the corresponding polynomial function space. Equation (32) is the scalar specialization of the average (12).

Proposition 4.2 (Single-vector Weyl unitary projection). Let f:R1,d−1→Rf:\mathbb{R}^{1,d-1}\to\mathbb{R} be polynomial, and let FF be its Wick-coordinate complexification defined by (31). Then there exists HF∈C[z]H_F\in\mathbb{C}[z] such that

(QKF)(u)=HF(q(u)).(34)(Q_KF)(u)=H_F(q(u)). \tag*{(34)}

Consequently,

(QLf)(p)=HF(s(p)),(35)(Q_Lf)(p)=H_F(s(p)), \tag*{(35)}

and QLfQ_Lf is a real-valued, O(1,d−1)O(1,d-1)-invariant polynomial.

Proof. Since FF is polynomial, QKFQ_KF is polynomial. Haar invariance implies that QKFQ_KF is KK-invariant. The compact real form K=O(d)K=O(d) is Zariski dense in O(d,C)O(d,\mathbb{C}), so QKFQ_KF is O(d,C)O(d,\mathbb{C})-invariant. By the first fundamental theorem for the complex orthogonal group,

C[u0,…,ud−1]O(d,C)=C[q].\mathbb{C}[u_0,\ldots,u_{d-1}]^{O(d,\mathbb{C})}=\mathbb{C}[q].

Hence QKF=HF∘qQ_KF=H_F\circ q for some HF∈C[z]H_F\in\mathbb{C}[z]. Equation (30) then gives

(QLf)(p)=HF(s(p)),(Q_Lf)(p)=H_F(s(p)),

which is O(1,d−1)O(1,d-1)-invariant.

It remains to verify that the restriction is real-valued. Define the antiholomorphic involution

σ(u):=ηu‾.\sigma(u):=\overline{\eta u}.

Its set of fixed points is W(R1,d−1)W(\mathbb{R}^{1,d-1}). Because ff has real coefficients, the extension (31) satisfies

F(σ(u))=F(u)‾.F(\sigma(u))=\overline{F(u)}.

Moreover,

σ(ku)=(ηkη)σ(u),k∈O(d),\sigma(ku)=(\eta k\eta)\sigma(u),\qquad k\in O(d),

and the automorphism k↦ηkηk\mapsto\eta k\eta preserves Haar measure on O(d)O(d). Therefore

(QKF)(σ(u))=(QKF)(u)‾.(Q_KF)(\sigma(u))=\overline{(Q_KF)(u)}.

The averaged function is thus real-valued on the set of fixed points.

The Wick embedding is essential. For example, let d=2d=2 and f(t,x)=t2f(t,x)=t^2. Averaging directly over O(2)O(2) gives

∫O(2)f(k(t,x)) dk=t2+x22,\int_{O(2)} f(k(t,x))\,\mathrm{d}k=\frac{t^2+x^2}{2},

which is Euclidean rather than Lorentz invariant. In Wick coordinates, however,

F(u0,u1)=−u02,F(u_0,u_1)=-u_0^2,

and Weyl averaging gives

(QO(2)F)(u)=−12q(u).(Q_{O(2)}F)(u)=-\frac{1}{2}q(u).

Restricting to u=W(t,x)u=W(t,x) yields

(QLf)(t,x)=−12(−t2+x2),(Q_Lf)(t,x)=-\frac{1}{2}(-t^2+x^2),

which depends only on the Lorentz invariant s(t,x)s(t,x).

Lorentz invariant functions with several vector inputs

We can extend the results above to the case of multiple input vectors. Let

P=(p1,…,pm)∈(R1,d−1)mP=(p_1,\ldots,p_m)\in(\mathbb{R}^{1,d-1})^m

and write

W(P):=(W(p1),…,W(pm))∈(Cd)m.W(P):=(W(p_1),\ldots,W(p_m))\in(\mathbb{C}^d)^m.

The compact group K=O(d)K=O(d) acts diagonally by

k(u1,…,um)=(ku1,…,kum).k(u_1,\ldots,u_m)=(ku_1,\ldots,ku_m).

For a real polynomial predictor

f:(R1,d−1)m⟶R,f:(\mathbb{R}^{1,d-1})^m\longrightarrow\mathbb{R},

let fCf_{\mathbb{C}} denote its coefficientwise complexification and define

F(u1,…,um):=fC(S−1u1,…,S−1um).F(u_1,\ldots,u_m):=f_{\mathbb{C}}(S^{-1}u_1,\ldots,S^{-1}u_m).

Then F(W(P))=f(P)F(W(P))=f(P). Define

(QKF)(u):=∫KF(ku) dk,(QLf)(P):=(QKF)(W(P)).(36)(Q_KF)(\mathbf{u}):=\int_K F(k\mathbf{u})\,\mathrm{d}k,\qquad(Q_Lf)(P):=(Q_KF)(W(P)). \tag*{(36)}

The complex bilinear Gram map is

Γ:(Cd)m⟶Sym⁡m(C),Γ(u)ij:=B(ui,uj)=uiTuj.(37)\Gamma:(\mathbb{C}^d)^m\longrightarrow\operatorname{Sym}_m(\mathbb{C}),\qquad\Gamma(\mathbf{u})_{ij}:=B(u_i,u_j)=u_i^{\mathsf{T}}u_j. \tag*{(37)}

By equation (30),

Γ(W(P))ij=⟨pi,pj⟩η.(38)\Gamma(W(P))_{ij}=\langle p_i,p_j\rangle_\eta. \tag*{(38)}

The image of Γ\Gamma is the Gram variety

Gm,d:={G∈Sym⁡m(C):rank⁡G≤d}.\mathcal{G}_{m,d}:=\{G\in\operatorname{Sym}_m(\mathbb{C}):\operatorname{rank}G\le d\}.

Thus Gm,d=Sym⁡m(C)\mathcal{G}_{m,d}=\operatorname{Sym}_m(\mathbb{C}) when m≤dm\le d, whereas for m>dm>d the Gram entries satisfy the corresponding determinantal relations.

Proposition 4.3 (Multi-vector Weyl unitary projection). Let f:(R1,d−1)m→Rf:(\mathbb{R}^{1,d-1})^m\to\mathbb{R} be polynomial, and let FF be its Wick-coordinate complexification defined above. Then there exists a polynomial

HF∈C[zij:1≤i≤j≤m]H_F\in\mathbb{C}[z_{ij}:1\le i\le j\le m]

such that

(QKF)(u)=HF(Γ(u)).(39)(Q_KF)(\mathbf{u})=H_F(\Gamma(\mathbf{u})). \tag*{(39)}

Consequently,

(QLf)(P)=HF((⟨pi,pj⟩η)i,j=1m),(40)(Q_Lf)(P)=H_F\left(\left(\langle p_i,p_j\rangle_\eta\right)_{i,j=1}^m\right), \tag*{(40)}

so QLfQ_Lf is real-valued and invariant under the diagonal action of O(1,d−1)O(1,d-1). When m>dm>d, the ambient polynomial HFH_F is determined only up to polynomials that vanish on Gm,d\mathcal{G}_{m,d}.

Proof of the polynomial statement. The average QKFQ_KF is polynomial and is KK-invariant by Haar invariance. Since K=O(d)K = O(d) is Zariski dense in O(d,C)O(d,\mathbb{C}), QKFQ_KF is also O(d,C)O(d,\mathbb{C})-invariant. The first fundamental theorem for the complex orthogonal group states that invariant polynomials on mm vectors are generated by the pairwise contractions B(ui,uj)B(u_i,u_j). Hence QKFQ_KF factors polynomially through Γ\Gamma, proving (39). Equation (38) gives the restriction (40). Applying the real-structure argument from the single-vector case diagonally to (Cd)m(\mathbb{C}^d)^m shows that this restriction is real-valued.

In particular, if ff has total degree at most two, then its Weyl unitary projection has the form

(QLf)(P)=c+∑1≤i≤j≤mbij⟨pi,pj⟩η,(41)(Q_Lf)(P) = c + \sum_{1 \le i \le j \le m} b_{ij}\langle p_i,p_j\rangle_\eta, \tag*{(41)}

for suitable real coefficients cc and bijb_{ij}.

Numerical experiments for the Lorentz group

We now test the Lorentz Weyl construction from Section 4. We consider three settings. The first is a synthetic regression problem whose target is exactly Lorentz invariant by construction. This allows us to test the effect of Weyl unitary averaging with respect to different sizes of the training data. The second is jet tagging, a realistic classification task from collider physics in which the inputs are particle four-momenta. Analogous to improving the time-series prediction of Section 3, we also apply the discussed methods to a relativistic dynamic system, the driven covariant double well.

Synthetic experiments We first evaluate the Weyl projection on a controlled regression problem with an exactly Lorentz-invariant conditional mean. Each input consists of m=6m = 6 four-vectors

P=(p1,…,pm)∈(R1,3)m.P = (p_1,\ldots,p_m) \in(\mathbb{R}^{1,3})^m.

Writing η=diag⁡(−1,1,1,1)\eta= \operatorname{diag}(-1,1,1,1) and Gij(P)=piTηpjG_{ij}(P) = p_i^{\mathsf{T}}\eta p_j, we generate

y(P)=∑i,j=1mAijGij(P)+ϵ,ϵ∼N(0,0.12),(42)y(P) = \sum_{i,j=1}^{m} A_{ij}G_{ij}(P) + \epsilon,\qquad\epsilon\sim\mathcal{N}(0,0.1^2), \tag*{(42)}

where A=ATA = A^{\mathsf{T}} is sampled once per seed and held fixed across training-set sizes. Since the Gram matrix is unchanged by a simultaneous Lorentz transformation of all input vectors, it is Lorentz invariant.

We consider two input distributions. In the Euclidean-isotropic setting, pi∼N(0,I4)p_i \sim\mathcal{N}(0,I_4). In the time–space anisotropic setting, the components are independent with

ti∼N(0,1.52),xi∼N(0,0.72I3).t_i \sim\mathcal{N}(0,1.5^2),\qquad\mathbf{x}_i \sim\mathcal{N}(0,0.7^2 I_3).

We compare four different models.

Baseline model is a ridge regression on all coordinate monomials of degree at most two.

Spatially averaged model post-processes the fitted predictor using a Monte Carlo O(3)O(3) average with N=16,32N = 16,32 samples.

Weyl-projected model applies the Weyl O(4)O(4) projection from Section 4.6.2. For degree-two polynomials, this projection eliminates the linear component and maps the quadratic component into the span of the Minkowski products GijG_{ij}, yielding a Lorentz-invariant predictor.

Invariant baseline is trained directly on the 21 distinct Gram entries that are invariant with respect to unit-rescaling group actions.

We use training sizes between 100 and 5000, five random seeds, and 5000 independent test examples per seed. Figure 3 reports the test mean-squared error; points and error bars denote the mean and one standard deviation across seeds.

Test MSE for the baseline polynomial, its exact Weyl projection, the finite spatial $O(3)$ average, and the Gram-feature invariant baseline

Figure 3. Test MSE for the baseline polynomial, its exact Weyl projection, the finite spatial O(3)O(3) average, and the Gram-feature invariant baseline. Both axes are logarithmic.

The invariant baseline is already close to the irreducible noise level E[ϵ2]=0.01\mathbb{E}[\epsilon^2] = 0.01 with 100 training examples. In the Euclidean-isotropic setting, post-training Weyl projection also substantially improves sample efficiency: at Ntrain=300N_{\mathrm{train}} = 300, it reduces the MSE from 1.37±0.491.37 \pm0.49 to 0.183±0.0830.183 \pm0.083.

This improvement is not universal. In the time–space anisotropic setting with Ntrain=100N_{\mathrm{train}} = 100, Weyl projection increases the MSE from 8.56±2.418.56 \pm2.41 to 15.7±4.115.7 \pm4.1, whereas the NMC=32N_{\mathrm{MC}} = 32 spatial average gives 6.48±1.986.48 \pm1.98. Hence, the exact dataset distribution is important, and which specific projection used depends on the distribution.

We also evaluate prediction variance under common Lorentz boosts. The Weyl-projected and Gram-feature models remain invariant to numerical precision, whereas the baseline and spatially averaged predictors do not. The diagnostic and complete results are reported in Appendix § 8.

Conclusion. With limited training data, an unconstrained model may fail to recover the symmetry of the regression function. The Weyl projection removes non-invariant components improves test error for the Euclidean-isotropic distribution, which is invariant under the compact O(4)O(4) group used in the projection. However, exact Lorentz invariance does not by itself guarantee lower prediction error: under time–space anisotropy, the Weyl projection can increase the MSE. Thus, while the projection always enforces the desired symmetry, its statistical benefit depends on the input distribution.

Chaotic relativistic dynamics We now analyze a spatiotemporal chaotic relativistic dynamical system, analogous to § 3. As a toy model for a continuous, Lorentz-equivariant model we choose the driven covariant double well in (1+1)(1 + 1) dimensions.

System. The state of the particle consists of the space-time event X=(t,x)X = (t, x) and the momentum P=(e,p)P = (e, p), with metric η=diag⁡(−1,1)\eta= \operatorname{diag}(-1, 1) and s(A)=⟨A,A⟩η=ATηAs(A) = \langle A, A\rangle_{\eta} = A^{\mathsf{T}}\eta A. All physical variables are functions of the Lorentz-invariant evolution parameter τ\tau. The dynamics follows from the Lagrangian

L=M2s(X˙)+λ4(s(X)+b2)2−κs(X)cos⁡ϑ,(43)\mathcal{L} = \frac{\mathcal{M}}{2}s(\dot{X}) + \frac{\lambda}{4}\left(s(X) + b^2\right)^2 - \kappa s(X)\cos\vartheta, \tag*{(43)}

where dots denote derivatives with respect to τ\tau and ϑ=Ωτ+ϑ0\vartheta= \Omega\tau+ \vartheta_0 is the drive phase. The equations of motion are

X˙=PM,P˙=[λ(s(X)+b2)−2κcos⁡ϑ]X,(44)\dot{X} = \frac{P}{\mathcal{M}}, \qquad\dot{P} = \left[\lambda\left(s(X) + b^2\right) - 2\kappa\cos\vartheta\right]X, \tag*{(44)}

we choose M=b=λ=Ω=1\mathcal{M} = b = \lambda= \Omega= 1 and drive amplitude κ=0.3\kappa= 0.3. The force is a Lorentz scalar times a Lorentz vector, so the flow of (44) is exactly equivariant under the full Lorentz group O(1,1)O(1,1), generated by the boosts, parity, and time reversal

B(ξ)=(cosh⁡ξsinh⁡ξsinh⁡ξcosh⁡ξ),P=diag⁡(1,−1),T=diag⁡(−1,1).(45)B(\xi) = \begin{pmatrix} \cosh\xi& \sinh\xi\\ \sinh\xi& \cosh\xi\end{pmatrix}, \qquad\mathcal{P} = \operatorname{diag}(1,-1), \qquad\mathcal{T} = \operatorname{diag}(-1,1). \tag*{(45)}

Due to the driving force, the total energy H=s(P)/2M−λ(s(X)+b2)2/4+κs(X)cos⁡ϑH = s(P)/2\mathcal{M} - \lambda\left(s(X) + b^2\right)^2/4 + \kappa s(X)\cos\vartheta is not conserved, dH/dτ=−κΩs(X)sin⁡ϑ\mathrm{d}H/\mathrm{d}\tau= -\kappa\Omega s(X)\sin\vartheta, which permits chaotic motion.

Data. We generate N=2000N = 2000 trajectories by numerical integration and split them by trajectory into 1400 training, 300 validation, and 300 test trajectories. Since the external drive breaks time-translation symmetry, we include the drive phase ϑ\vartheta as a state variable, encoded as (cos⁡ϑ,sin⁡ϑ)(\cos\vartheta, \sin\vartheta) to avoid the discontinuity at ϑ=2π\vartheta= 2\pi. The phase is a Lorentz scalar and is advanced analytically, so the state is

u=(t,x,e,p,cos⁡ϑ,sin⁡ϑ)∈R6.(46)\mathbf{u} = (t, x, e, p, \cos\vartheta, \sin\vartheta) \in\mathbb{R}^{6}. \tag*{(46)}

We restrict to trajectories with vanishing boost charge J=tp−xe=0J=tp-xe=0. JJ is conserved by (44), and the sector J=0J=0, in which XX and PP are parallel, is invariant under both the flow and O(1,1)\mathrm{O}(1,1). Trajectories in this sector remain bounded, whereas generic trajectories with J≠0J\ne0 escape to infinity. All trajectories start at the space-time origin, X(0)=0X(0)=0, with P(0)=m0(cosh⁡χ0,sinh⁡χ0)P(0)=m_0(\cosh\chi_0,\sinh\chi_0), so that J=0J=0 holds automatically and P(0)P(0) is timelike. Mass, rapidity, and phase are drawn independently with m0∼Rayleigh⁡(0.5)m_0\sim\operatorname{Rayleigh}(0.5), χ0∼N(0,0.82)\chi_0\sim\mathcal{N}(0,0.8^2) truncated at ∣χ0∣≤4|\chi_0|\le4, and ϑ0∼U(0,2π)\vartheta_0\sim\mathcal{U}(0,2\pi). The first Δτ=30\Delta\tau=30 of each trajectory are discarded as burn-in, and the following Δτ=150\Delta\tau=150 are recorded at intervals Δτs=0.1\Delta\tau_s=0.1, i.e. 1500 states per trajectory. The finite-time Lyapunov exponent of the chaotic trajectories has a median of 0.15, a Lyapunov time of ≈6.5\approx6.5; 7% of the trajectories are regular.

Reduced coordinates. On the sector J=0J=0 both vectors lie on a fixed timelike ray n=(cosh⁡χ0,sinh⁡χ0)n=(\cosh\chi_0,\sinh\chi_0), X=ρnX=\rho n and P=ωnP=\omega n. The rapidity χ0\chi_0 is a constant of motion, and

ρ=sign⁡(t)−s(X),ω=sign⁡(e)−s(P)(47)\rho=\operatorname{sign}(t)\sqrt{-s(X)},\qquad\omega=\operatorname{sign}(e)\sqrt{-s(P)} \tag*{(47)}

are Lorentz invariants, the position and momentum in the rest frame of the ray. In these coordinates (44) reduces to a parametrically driven Duffing double well,

Mρ¨=[λ(b2−ρ2)−2κcos⁡ϑ]ρ,ω=Mρ˙,(48)M\ddot{\rho}=\left[\lambda\left(b^2-\rho^2\right)-2\kappa\cos\vartheta\right]\rho,\qquad\omega=M\dot{\rho}, \tag*{(48)}

with wells at ρ=±b\rho=\pm b separated by a barrier at ρ=0\rho=0. To generate trajectories, we integrate (48) and map the result to (t,x,e,p)(t,x,e,p). The prediction model only ever sees the physical coordinates.

Surrogate model. As surrogate model (6) we use a fully connected network ff that maps a window of k=8k=8 consecutive states (46) to the increment of the two Lorentz vectors. The input is the flattened window of dimension 8×6=488\times6=48, the network has three hidden layers of width 128 with ReLU activations (39 812 parameters), and the output is the residual (Δt,Δx,Δe,Δp)(\Delta t,\Delta x,\Delta e,\Delta p). Inputs and outputs are normalized by one scalar per Lorentz vector (X,P,ΔX,ΔP)(X,P,\Delta X,\Delta P), computed on the training set, so that the normalization commutes with the group action. The network is trained on all 2.1×1062.1\times10^6 windows of the training trajectories with a mean-squared loss on the normalized increments, using AdamW with batch size 1024 for 100 epochs, a linear warm-up over 10310^3 steps to a learning rate of 2×10−32\times10^{-3}, and cosine decay to zero; we keep the checkpoint with the lowest validation loss. Predictions are autoregressive rollouts from the first kk states of each test trajectory. We test this network with three types of averages:

Boosts average (not equivariant): (Qwf)(U)=∑j=18wjB(ξj)−1f(B(ξj) U),ξ∼N(0,0.32),(Q_wf)(\mathrm{U})=\sum_{j=1}^{8}w_jB(\xi_j)^{-1}f(B(\xi_j)\,\mathrm{U}),\quad\xi\sim\mathcal{N}(0,0.3^2),

(Qwf)(U)=∑j=18wjB(ξj)−1f(B(ξj) U),ξ∼N(0,0.32),(49)(Q_wf)(\mathrm{U})=\sum_{j=1}^{8}w_jB(\xi_j)^{-1}f(B(\xi_j)\,\mathrm{U}),\qquad\xi\sim\mathcal{N}(0,0.3^2), \tag*{(49)}

Inversions average (equivariant): (QSf)(U)=14∑g∈Sgf(gU),S={I,P,T,PT},(Q_Sf)(\mathrm{U})=\frac{1}{4}\sum_{g\in S}g f(g\mathrm{U}),\qquad S=\{I,P,T,PT\},

(QSf)(U)=14∑g∈Sgf(gU),S={I,P,T,PT},(50)(Q_Sf)(\mathrm{U})=\frac{1}{4}\sum_{g\in S}g f(g\mathrm{U}),\qquad S=\{I,P,T,PT\}, \tag*{(50)}

and the rest-frame average QF,wfQ_{F,w}f (53) below. QwfQ_wf averages over Lorentz boosts around the frame of the input. Since the boosts are non-compact and carry no normalizable Haar measure, the Gaussian weight is a choice and QwfQ_wf is not exactly equivariant. QSfQ_Sf averages over the finite subgroup of time and space inversions; it is an exact group average. Each group element acts on both Lorentz vectors of every state in the window and leaves the phase unchanged.

Rest-frame average. To compute a frame average as in § 4.4, we apply a Lorentz boost that transforms the particle into its rest frame. The rapidity can be computed from a short window of data: for the input window Uτi=[uτi−k+1,…,uτi]\mathrm{U}_{\tau_i}=[\mathrm{u}_{\tau_i-k+1},\ldots,\mathrm{u}_{\tau_i}],

tanh⁡χ^U=∑ℓ(tℓxℓ+eℓpℓ)∑ℓ(tℓ2+eℓ2),ΛU=B(−χ^U),(51)\tanh\hat{\chi}_{\mathrm{U}}=\frac{\sum_{\ell}(t_\ell x_\ell+e_\ell p_\ell)}{\sum_{\ell}(t_\ell^2+e_\ell^2)},\qquad\Lambda_{\mathrm{U}}=B(-\hat{\chi}_{\mathrm{U}}), \tag*{(51)}

which is exact on the sector J=0J=0, where it transforms with the input, χ^B(ξ)U=χ^U+ξ\hat{\chi}_{B(\xi)\mathrm{U}}=\hat{\chi}_{\mathrm{U}}+\xi. Following [31], we use the frame F(U)={(gΛU)−1:g∈S}\mathcal{F}(\mathrm{U})=\{(g\Lambda_{\mathrm{U}})^{-1}:g\in S\}, which on this sector satisfies F(hU)=hF(U)\mathcal{F}(h\mathrm{U})=h\mathcal{F}(\mathrm{U}) for all h∈O(1,1)h\in\mathrm{O}(1,1). The frame average

(QFf)(U)=1∣F(U)∣∑g∈F(U)gf(g−1U)=ΛU−114∑g∈Sgf(gΛUU)(52)(Q_{\mathcal{F}}f)(\mathrm{U})=\frac{1}{|\mathcal{F}(\mathrm{U})|}\sum_{g\in\mathcal{F}(\mathrm{U})}g f(g^{-1}\mathrm{U})=\Lambda_{\mathrm{U}}^{-1}\frac{1}{4}\sum_{g\in S}g f(g\Lambda_{\mathrm{U}}\mathrm{U}) \tag*{(52)}

is then exactly O(1,1)O(1,1)-equivariant on the sector J=0J=0 for any network ff. Here we extend this procedure by applying, in the rest frame, additional boosts in addition to the inversions.

Extended frame average: (QF,wf)(U)=ΛU−1∑g∈S∑j=14wj4(gB(ζj))−1f(gB(ζj)ΛUU),(Q_{F,wf})(\mathrm{U}) = \Lambda_{\mathrm{U}}^{-1} \sum_{g \in S} \sum_{j=1}^{4} \frac{w_j}{4} (g B(\zeta_j))^{-1} f(g B(\zeta_j) \Lambda_{\mathrm{U}}\mathrm{U}),

ζ∼N(0,0.42),S={I,P,T,PT}(53)\zeta\sim\mathcal{N}(0,0.4^2), \qquad S=\{I,\mathcal{P},\mathcal{T},\mathcal{P}\mathcal{T}\} \tag*{(53)}

which requires 16 network evaluations. Because the nodes are symmetric and gB(ζ)g−1=B(±ζ)g B(\zeta) g^{-1}=B(\pm\zeta) for g∈Sg \in S, the weighted frame is still equivariant, and QF,wfQ_{F,wf} remains exactly equivariant. Every rollout of QF,wfQ_{F,wf} stays on the ray of its initial window.

The Gaussian averages in (49) and (53) are evaluated by Gauss–Hermite quadrature,

Eζ∼N(0,σ2)[F(ζ)]≈∑j=1rwjF(ζj),ζj=2 σxj,wj=γjπ,(54)\mathbb{E}_{\zeta\sim\mathcal{N}(0,\sigma^2)}[F(\zeta)] \approx\sum_{j=1}^{r} w_j F(\zeta_j), \qquad\zeta_j=\sqrt{2}\,\sigma x_j,\quad w_j=\frac{\gamma_j}{\sqrt{\pi}}, \tag*{(54)}

where xjx_j are the roots of the Hermite polynomial HrH_r and γj\gamma_j the corresponding weights. We use r=8r=8 for QwfQ_{wf} and r=4r=4 for QF,wfQ_{F,wf}. The widths 0.3 and 0.4 were selected on the validation set.

Error. The error is measured by the total deviation in the rest frame of the true trajectory. Both the prediction and the truth are transformed by Λtrue=B(−χ^true)\Lambda_{\mathrm{true}}=B(-\hat{\chi}_{\mathrm{true}}), with χ^true\hat{\chi}_{\mathrm{true}} from (51) applied to the whole true trajectory, (ρ~,x~)=ΛtrueX(\tilde{\rho},\tilde{x})=\Lambda_{\mathrm{true}}X and (ω~,p~)=ΛtrueP(\tilde{\omega},\tilde{p})=\Lambda_{\mathrm{true}}P. Then the error (or loss) dd is

d(τ)=Δρ~(τ)2+Δω~(τ)2+d⊥(τ)2,d⊥(τ)=x~(τ)2+p~(τ)2,(55)d(\tau)=\sqrt{\Delta\tilde{\rho}(\tau)^2+\Delta\tilde{\omega}(\tau)^2+d_{\perp}(\tau)^2}, \qquad d_{\perp}(\tau)=\sqrt{\tilde{x}(\tau)^2+\tilde{p}(\tau)^2}, \tag*{(55)}

where Δ\Delta denotes prediction minus truth and d⊥d_{\perp} is evaluated on the prediction. The first two terms measure the error within the dynamics and d⊥d_{\perp} the departure from the ray of the true trajectory. The distance is invariant under a joint Lorentz transformation of prediction and truth. The reported errors are averages over the test set at a given time τ\tau. Trajectories that erroneously escape to infinity are excluded from the test set averages.

Group averaging can here too significantly increase the predictive power of a model without pre-training. The results are shown in Figure 4 and Table 3. We find that for this example a combination of the Monte-Carlo method for average boosts together with a frame-average gives the best results.

The rollout test-set average error and the phase plot of a sample trajectory

Figure 4. (a) The rollout test-set average error dd. (b) The phase plot of a sample trajectory. The averaging procedures on average decrease the rollout loss. We compare 3 different averages to the baseline prediction (the model evaluated only once). On the right, we see with an example how group averaging improves prediction: The averaged trajectory (red) stay closer to the truth (black) for longer.

Model110100Rollout
Baseline0.00180.01260.23210.1497
Boosts0.00140.01000.19110.1201
Inversions0.00100.00650.12980.0786
Extended frame0.00030.00270.06740.0313

Table 3. Average invariant loss values dd for the test-set (N=300N = 300 test trajectories) after n=1,10,100n = 1, 10, 100 autoregressive steps with Δτs=0.1\Delta\tau_s = 0.1 per step. The rollout loss is the sum of the first 15 errors.

Jet tagging We next study quark–gluon jet tagging, a standard classification problem in collider physics. A jet is represented as an unordered collection of constituent four-momenta,

X={pi}i=1nX,pi=(Ei,px,i,py,i,pz,i).(56)X=\{p_i\}_{i=1}^{n_X}, \qquad p_i=(E_i,p_{x,i},p_{y,i},p_{z,i}). \tag*{(56)}

We use the two-class Pythia8 quark–gluon sample distributed with EnergyFlow [74, 52], with quark jets treated as signal and gluon jets as background. The split contains 1.6×1061.6 \times10^6 training jets, 2×1052 \times10^5 validation jets, and 2×1052 \times10^5 test jets, with equal numbers of the two classes in each split. We evaluate two complementary classifiers. The first is ParticleNet-Lite [53], a realistic high-capacity jet tagger. The second is a degree-three polynomial classifier whose logit is a polynomial in the Cartesian components (Ei,px,i,py,i,pz,i)(E_i,p_{x,i},p_{y,i},p_{z,i}) of the constituent momenta, built from the permutation-invariant aggregates N=nN = n, A=∑ipi∈R4A = \sum_i p_i \in\mathbb{R}^4 and B=∑ipipiT∈R4×4B = \sum_i p_i p_i^{\mathsf T} \in\mathbb{R}^{4 \times4}, which supply D=15D = 15 features once the symmetry of BB is used, and fitted by ℓ2\ell_2-penalized logistic regression. It serves two purposes. Because it retains explicit dependence on the azimuthal coordinate, it isolates the effect of post-training SO(2)SO(2) averaging on a predictor whose fitted coefficients are not already azimuthally invariant. And because its logit is polynomial in the four-momenta, it lies in the function class on which the Weyl projection of § 4.6 is defined.

Across the two models, we consider the following post-hoc averaging procedures:

Beam-axis SO(2)SO(2) averaging. We average over rotations about the beam axis. This is a compact-group average aligned with the azimuthal symmetry of the proton–proton collision sample.

SO(3)SO(3) averaging. We average over spatial rotations while holding the laboratory beam axis fixed. Because these transformations do not preserve the saved-jet distribution, this provides a deliberate symmetry-mismatch control.

Bounded boost averaging. We separately average over boosts parallel and transverse to the beam axis, with rapidities drawn from a fixed bounded law, ρ∼N(0,0.32)\rho\sim\mathcal{N}(0,0.3^2) rejection-truncated to ∣ρ∣≤0.3|\rho| \le0.3. Since boosts form a non-compact group, these are windowed test-time averages rather than Reynolds projections.

Weyl-unitarian projection. For degree-three polynomial classifier, we use the formula derived and applied to the polynomial to act as the classifier.

Collider-aligned frame averaging. We also apply the frame average from § 4.4 to ParticleNet-Lite. Let

P=(P0,Px,Py,Pz):=∑ipi,pT:=Px2+Py2,mT:=P02−Pz2.P = (P_0,P_x,P_y,P_z) := \sum_i p_i,\qquad p_T := \sqrt{P_x^2+P_y^2},\qquad m_T := \sqrt{P_0^2-P_z^2}.

We remove the jet azimuth and longitudinal rapidity through

ϕJ:=atan2⁡(Py,Px),yJ:=atanh⁡(PzP0),qi:=Bz(−yJ)Rz(−ϕJ)pi.(57)\phi_J := \operatorname{atan2}(P_y,P_x),\qquad y_J := \operatorname{atanh}\left(\frac{P_z}{P_0}\right),\qquad q_i := B_z(-y_J)R_z(-\phi_J)p_i. \tag*{(57)}

so that ∑iqi=(mT,pT,0,0)\sum_i q_i = (m_T,p_T,0,0). We then average over the residual reflection sets

R1={I},R2={I,ry},R4={I,ry,rz,ryrz},\mathcal{R}_1 = \{I\},\qquad\mathcal{R}_2 = \{I,r_y\},\qquad\mathcal{R}_4 = \{I,r_y,r_z,r_y r_z\},

where ryr_y and rzr_z reverse the corresponding momentum components. For a scalar predictor ff, the resulting KK-view predictor is

(Qcoll,Kf)(X):=1K∑r∈RKf(r(qi)i=1nX).(58)(Q_{\mathrm{coll},K}f)(X) := \frac{1}{K}\sum_{r\in\mathcal{R}_K} f\left(r(q_i)_{i=1}^{n_X}\right). \tag*{(58)}

The construction enforces invariance under rotations about and boosts along the beam axis, together with the selected reflections, and is well defined for pT>0p_T > 0 and ∣Pz/P0∣<1|P_z/P_0| < 1.

For each probability-valued classifier ff, we form

(QG,Mf)(X)=1M∑ℓ=0M−1f(gℓX).(59)(Q_{G,M}f)(X) = \frac{1}{M} \sum_{\ell=0}^{M-1} f(g_{\ell}X). \tag*{(59)}

Each transformation acts on every constituent four-momentum in the jet before all derived kinematic features are recomputed. For SO(2)SO(2), we use the deterministic angles θℓ=2πℓ/M\theta_{\ell}=2\pi\ell/M. For SO(3)SO(3), we use a fixed list of Haar-sampled proper rotations while holding the laboratory beam axis fixed. The ParticleNet-Lite results below use M=128M=128, average the two class probabilities, and use the same transformation list for every jet and every trained model.

To resolve changes hidden by aggregate accuracy, let N−+N_{-+} be the number of test jets whose prediction changes from incorrect before averaging to correct after averaging, and let N+−N_{+-} count changes in the opposite direction. We report

ΔNcorrect:=N−+−N+−,(60)\Delta N_{\mathrm{correct}} := N_{-+} - N_{+-}, \tag*{(60)}

which is exactly the change in the number of correctly classified test jets. We also report the background rejection Rejα=1/ϵB\mathrm{Rej}_{\alpha}=1/\epsilon_{B} at quark efficiencies α=50%\alpha=50\% and 30%30\%, where ϵB\epsilon_{B} is the gluon acceptance at the threshold chosen to obtain the stated quark efficiency.

PredictorAccuracyN−+N_{-+}N+−N_{+-}ΔNcorrect\Delta N_{\mathrm{correct}}Rej50%\mathrm{Rej}_{50\%}Rej30%\mathrm{Rej}_{30\%}
Baseline (ParticleNet-Lite)0.825031±0.0002930.825031 \pm0.000293———33.294±0.22933.294 \pm0.22984.064±0.82084.064 \pm0.820
Collider-aligned frame average (K=4K=4)0.825078±0.0003080.825078 \pm0.0003082026.0±38.72026.0 \pm38.72016.6±48.22016.6 \pm48.2+9.4±36.7+9.4 \pm36.733.408±0.21833.408 \pm0.21884.597±0.50384.597 \pm0.503
SO(2)SO(2) average0.825032±0.0002950.825032 \pm0.0002950.33±0.500.33 \pm0.500.22±0.440.22 \pm0.44+0.11±0.78+0.11 \pm0.7833.294±0.22933.294 \pm0.22984.064±0.82084.064 \pm0.820
SO(3)SO(3) average0.799649±0.0016500.799649 \pm0.0016505804.3±121.55804.3 \pm121.510880.8±408.210880.8 \pm408.2−5076.4±313.6-5076.4 \pm313.628.964±0.25228.964 \pm0.25274.424±1.04074.424 \pm1.040
Longitudinal boost average0.825184±0.0003300.825184 \pm0.000330287.7±16.0287.7 \pm16.0257.1±12.9257.1 \pm12.9+30.6±22.8+30.6 \pm22.833.350±0.20033.350 \pm0.20084.007±0.67984.007 \pm0.679
Transverse boost average0.824858±0.0004120.824858 \pm0.000412951.3±47.5951.3 \pm47.5986.0±46.1986.0 \pm46.1−34.7±30.5-34.7 \pm30.533.259±0.17733.259 \pm0.17783.587±0.77683.587 \pm0.776

Table 4. ParticleNet-Lite performance before and after spatial-rotation, bounded-boost, and input-dependent collider-aligned frame averaging. Entries are the mean ±\pm sample standard deviation over nine independently trained models evaluated on the same 200,000-jet test set. The spatial-rotation and bounded-boost entries use 128 frames and average class probabilities. The collider-aligned entry uses four deterministic canonical views and averages pre-softmax binary log-odds. The boost rapidities are sampled from a rejection-truncated Gaussian ρ∼N(0,0.32)\rho\sim\mathcal{N}(0,0.3^{2}), with ∣ρ∣≤0.3\lvert\rho\rvert\le0.3. Here N−+N_{-+} counts incorrect-to-correct changes and N+−N_{+-} counts correct-to-incorrect changes.

SO(2)SO(2) averaging leaves the ParticleNet predictions unchanged to numerical precision, as a result of the construction of the model that extract numerical values of a jet making it invariant with respect to SO(2)SO(2) changes. Fixed-lab SO(3)SO(3) averaging, however, changes many decisions in both directions. This shows that the exact Lorentz symmetry is not present in the Quark-gluon dataset as the particles are recorded with a fixed beam axis. The collider-aligned frame average is the method which showed consistent improvements across all seeds.

To probe the finite-sample effect of beam-axis averaging and also attempt to apply an exact Weyl unitary trick, we also train the degree-three polynomial classifier. For each size and each of ten seeds, all post-hoc methods are applied to the best checkpoint.

Figure 5 shows that beam-axis SO(2)SO(2) averaging improves AUC throughout the training-size sweep, with the largest gain in the low-data regime and a diminishing advantage as the training set grows. This is consistent with compact-group projection. Averaging removes non-invariant components that become less important as the baseline learner sees more data [5].

Figure 5. ROC AUC and background rejection at quark efficiencies of 30% and 50% for the degree-three polynomial classifier.

The Exact Weyl projection supplies a stronger low-data inductive bias. The saved-jet distribution is not strictly Lorentz invariant, for the beam-axis reason described above, yet restricting the classifier to the fixed 18-dimensional invariant subspace already saturates its achievable performance within a few hundred examples. That subspace is spanned by the products of total degree at most three in NN, the Minkowski traces B^\hat{B}, B^2\hat{B}^{2}, and B^3\hat{B}^{3} of B^μν\hat{B}_{\mu\nu}, and the contractions AμAμA_{\mu}A^{\mu} and AμBμνAνA_{\mu}B^{\mu\nu}A_{\nu}. The projection collapses the 816-coefficient hypothesis class onto these 18 directions, and the surviving coefficients converge almost immediately: the projected discriminant direction at 256 examples lies within one degree of its 1.6×1061.6 \times10^{6} counterpart, while its norm grows by a factor of 23. Because a positive rescaling leaves the ordering of scores unchanged, AUC and background rejection are flat across the entire sweep. The projected model is therefore the strongest predictor in the low-data regime, but it stops improving, but it stops improving: the unprojected baseline overtakes it in AUC and at 50% quark efficiency between 8,192 and 16,384 examples, while at 30% it stays at or above the baseline across the sweep. The ROC curves cross, so the projection remains competitive at the high-purity working points.

Overall, performance follows the alignment between the imposed transformation and the collider setup. Beam-axis SO(2)SO(2) preserves that setup, whereas fixed-laboratory SO(3)SO(3) averaging is detrimental because arbitrary spatial rotations respect neither the beam direction nor the selected-jet distribution. The bounded longitudinal-boost average stays close to the raw model, while transverse averaging is generally mildly detrimental. We therefore read the boost curves only as bounded-window sensitivity checks rather than canonical projections over the noncompact Lorentz group; their uncertainty intervals additionally condition on a single fixed transformation bank.

Discussion and conclusion

This work asks how a known physical symmetry can be imposed on an already trained predictor, and when doing so improves prediction. For compact groups, these objectives are closely connected. As shown in § 3.1, normalized Haar averaging defines the Reynolds projection onto the invariant or equivariant subspace. When the input data distribution is invariant, the induced group action is unitary, and the squared-loss regression function is equivariant, this operator coincides with the L2(μ)L^2(\mu)-orthogonal projection in Eq. (5) and therefore cannot increase population squared risk. The surrogate-model experiments of § 3.3 are consistent with this theory: averaging generally reduced prediction errors when the implemented transformations were genuine symmetries of the complete learning problem.

For non-compact groups, this correspondence breaks down. There is no normalized Haar probability measure, so the compact Reynolds construction cannot be applied directly. More fundamentally, the algebraic projection that enforces a transformation law need not coincide with the L2(μ)L^2(\mu) projection that minimizes error under a specified data distribution. The data-dependent projection of Section 4.3 retains the latter statistical interpretation, but its implementation requires to solve an optimization problem akin to training an equivariant model. The Weyl construction of § 4.2 instead provides a distribution-independent algebraic projection for the polynomial, Laurent-polynomial, and suitably holomorphic function classes covered by the construction. It enforces the desired symmetry exactly, but it does not inherit the compact-group risk guarantee.

Frame averaging, in § 4.4, is a third route. It replaces the integral over the group with a finite average over an input-dependent set of transformations, so it requires neither a normalized measure nor an analytic predictor and applies to an arbitrary black-box model. Like the Weyl construction it is exact, in that the averaged predictor satisfies the prescribed transformation law identically, and like the Weyl construction it is in general neither the Haar projection nor the L2(μ)L^2(\mu) projection, so it carries no risk guarantee either.

The non-compact-group experiments give four main empirical conclusions. First, the Weyl projection did not consistently improve in-distribution accuracy, but it produced substantial gains with limited training data and when test inputs were expressed in unit systems not seen during training. Second, frame averaging gives the same exactness without the function-class restriction: it is units-equivariant by construction and applies to an arbitrary black-box predictor, and it prevented the catastrophic failures the baseline suffers under unseen unit transformations. Its in-distribution cost was severe for the fixed-unit-trained regression. Third, the empirical L2(μ)L^2(\mu) projection improved on the baseline under most tested unit shift and in-distribution task and training-regime combinations, illustrating the benefit of adapting the projection to the data distribution; it was also the least stable of the three, degrading sharply at small training-set sizes (Appendix § 8). Fourth, when retraining was possible, models trained directly using symmetry-adapted features matched or outperformed the post-hoc projections in our controlled unit-rescaling and Lorentz experiments. These comparisons position post-hoc symmetrization as a complement to symmetry-aware training, particularly when retraining or architectural modification is impractical.

Several questions remain open. Theoretically, it would be useful to characterize the distributions and predictor classes for which the Weyl projection reduces risk, and to quantify its discrepancy from the L2(μ)L^2(\mu) projection in terms of the geometry of the data distribution. Computationally, extending the construction beyond polynomial or holomorphically extendable predictors would be necessary for its direct application to modern black-box neural networks.

Acknowledgements

SV and WZ were partially funded by NSF DMS 2031985. SV is also funded by NSF CAREER 2339682, NSF BSF 2430292, and a Sloan Fellowship. We thank Lara Knezevic for drawing our attention to related work on twirling in quantum machine learning.

Data availability statement

The data that support the findings of this study are openly available. The quark–gluon jet dataset is available at https://doi.org/10.5281/zenodo.3164691. The Well dataset used in § 3 is part of the Well collection [6] and is available at https://polymathic-ai.org/the_well/. The synthetic unit-rescaling and Lorentz datasets, together with the code used to generate them and to reproduce all experiments reported here, are available at https://github.com/wenjie271/Post-hoc-group-averaging.

Conflicts of interest statement

The authors have no conflicts of interest.

Ethics statement

Not applicable.

Groups

Dihedral groups

For auto-regressive prediction problems, we are interested in datasets on a square or rectangular Cartesian grid. Datasets defined on a square grid with appropriate initial and boundary conditions have the largest possible symmetry.

The symmetry group of the square is D4D_4 and contains ∣D4∣=8|D_4| = 8 elements. For datasets defined on a rectangle, the biggest symmetry group is D2D_2, containing ∣D2∣=4|D_2| = 4 elements. Inversions correspond to D1≅C2≅Z2D_1 \cong C_2 \cong Z_2 with ∣D1∣=2|D_1| = 2 elements.

Generally, the dihedral groups DnD_n refer to the symmetry groups of the nn-gon, groups of order 2n2n. We can think of these as nn rotational symmetries and nn reflection symmetries.

The group representation of D4D_4 is

D4=⟨r,i∣r4=i2=(ir)2=e⟩,(61)D_4 = \langle r, i \mid r^4 = i^2 = (ir)^2 = e \rangle, \tag*{(61)}

where its members can be thought of as rotations and inversions. One representation is given by the matrices

R=(0−110),I=(−1001),(62)R = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}, \qquad I = \begin{pmatrix} -1 & 0 \\ 0 & 1 \end{pmatrix}, \tag*{(62)}

and the ∣D4∣=8|D_4| = 8 group elements can be constructed by

id⁡,R,R2,R3,I,IR,IR2,IR3.(63)\operatorname{id}, R, R^2, R^3, I, IR, IR^2, IR^3. \tag*{(63)}

D2D_2 can be constructed as above using only even powers of RR, and D1D_1 by using only the inversion.

Circle groups

Translation invariant problems with periodic boundary conditions obey the symmetry of the circle. The circle group is an example of a compact continuous Lie group. It is known by several names, such as T≅S1≅U(1)≅SO(2)T \cong S^1 \cong U(1) \cong SO(2). The irreducible representations are one-dimensional maps to the unit circle. If the problem is translation invariant and periodic in two directions, it is defined on a 2-torus. The torus group T2T^2 is the continuous Abelian group formed from the direct product of two circle groups T2≅S1×S1T^2 \cong S^1 \times S^1.

Since numerical simulation data is always given on a discrete grid of dimension N×MN \times M, we approximate the continuous groups with discrete cyclic groups CN≅Z/ZNC_N \cong\mathbb{Z}/\mathbb{Z}_N, that is S1S^1 with CNC_N and T2T^2 with CN×CMC_N \times C_M.

Experimental details

Compact group averaging

“The Well”1 [6] is published under CC BY 4.0. Each test set contains 0.1×N0.1 \times N of the trajectories. We excluded 9 trajectories from the gray_scott dataset that converge to a steady state in our analysis.

DatasetSize (GB)NDN_DTTNNss
active_matter51.3256×256256 \times2568136011
gray_scott153.8128×128128 \times128100112002
rayleigh_benard342512×128512 \times12820017504
turbulent_radiative_layer_2D6.9384×128384 \times128101904

Table 5. Dataset specifications. (a) Total size, resolution NDN_D, number of time steps per trajectory TT, number of trajectories in the data set NN, number of state variables ss.

DatasetQuantityTypeComponents
active_matterConcentrationScalar1
active_matterVelocityVector2
active_matterOrientationTensor4
active_matterRate-of-strainTensor4
gray_scottConcentration AAScalar1
gray_scottConcentration BBScalar1
rayleigh_benardBuoyancyScalar1
rayleigh_benardPressureScalar1
rayleigh_benardVelocityVector2
turbulent_radiative_layer_2DBuoyancyScalar1
turbulent_radiative_layer_2DPressureScalar1
turbulent_radiative_layer_2DVelocityVector2

Table 5b. (b) The state variables of the datasets.

Jet-tagging dataset

The quark–gluon jet dataset [52] is published under CC BY 4.0.2 The sample was generated with Pythia 8.226 at s=14\sqrt{s} = 14 TeV, with quark jets produced via qg→Z(→ννˉ)qqg \to Z(\to\nu\bar{\nu})q and gluon jets via qqˉ→Z(→ννˉ)gq\bar{q} \to Z(\to\nu\bar{\nu})g; no detector simulation or pileup is included. Final-state particles are clustered with the anti-kTk_{\mathrm{T}} algorithm (R=0.4R = 0.4, FastJet 3.3.0), and jets with pTjet∈[500,550]p_T^{\mathrm{jet}} \in[500, 550] GeV and ∣yjet∣<1.7|y^{\mathrm{jet}}| < 1.7 are retained. We use the default sample, which excludes charm and bottom quark jets at the level of the hard process. The dataset contains 2×1062 \times10^6 jets with equal numbers of the two classes; we use 1.6×1061.6 \times10^6 jets for training, 2×1052 \times10^5 for validation, and 2×1052 \times10^5 for testing, preserving the class balance in each split. Each jet constituent is stored as (pT,y,ϕ,PDG id)(p_T, y, \phi, \mathrm{PDG\ id}), from which we reconstruct the Cartesian four-momenta (Ei,px,i,py,i,pz,i)(E_i, p_{x,i}, p_{y,i}, p_{z,i}) of equation (56).

Model and preprocessing. Our primary classifier is ParticleNet-Lite [53]. Each jet retains at most the 100 constituents with largest original transverse momentum, and the model is trained on the standard ParticleNet input features derived from the constituent four-momenta. The degree-three polynomial classifier of § 4.6.3 is built from coordinate-dependent jet aggregates of the same constituent four-momenta.

GeneratorPythia 8.226, s=14 TeV\sqrt{s}=14\,\mathrm{TeV}
Processesqg→Z(→ννˉ)qqg \to Z(\to\nu\bar{\nu})q (quark), qqˉ→Z(→ννˉ)gq\bar{q} \to Z(\to\nu\bar{\nu})g (gluon)
Clusteringanti-kTk_{\mathrm{T}}, R=0.4R=0.4 (FastJet 3.3.0)
SelectionpTjet∈[500,550] GeVp_{\mathrm{T}}^{\mathrm{jet}} \in[500,550]\,\mathrm{GeV}, ∣yjet∣<1.7|y^{\mathrm{jet}}|<1.7
Constituent features(pT,y,ϕ,PDG id)(p_{\mathrm{T}},y,\phi,\mathrm{PDG\ id})
Split (train/val/test)1.6×106/2×105/2×1051.6 \times10^{6} / 2 \times10^{5} / 2 \times10^{5}
Classesquark (signal) vs. gluon (background), balanced

Table 6. Specifications of the quark–gluon jet dataset [52].

Additional experiments of unit-rescaling group

Numerical realization of the projection. Proposition 4.1 evaluates QvQ_v in closed form by deleting the coefficients of monomials in the wrong units sector. Outside the Laurent-polynomial class that closed form is unavailable and the projection must instead be computed as an average over the compact torus, so we first check that such an average reproduces the algebraic operator.

We compare three evaluations of the same operator. Exact coefficient filtering applies QvQ_v by deleting wrong-sector coefficients. Deterministic torus quadrature averages over a finite roots-of-unity grid. Monte Carlo projection averages random phases z=eiθz=e^{i\theta}, where MM denotes the total number of model evaluations. The comparison uses 20 frozen fitted predictors: five seeds for each combination of force regression or Reynolds classification and fixed-unit or unit-randomized training. Writing ff for a frozen fitted predictor, the error of a numerical approximation Q^\widehat{Q} is ∥Q^f−Qvf∥2/∥Qvf∥2\|\widehat{Q}f-Q_vf\|_2/\|Q_vf\|_2 over the evaluation set.

OperatorMean errorMaximum error
Deterministic7.32×10−167.32 \times10^{-16}1.11×10−151.11 \times10^{-15}
M=32M=320.06810.449
M=128M=1280.03250.203
M=256M=2560.02060.109

Table 7. Relative prediction error against exact coefficient filtering. Monte Carlo rows aggregate combinations of task, training regime, seed, and independent phase sequence.

For every tested fit, deterministic quadrature reproduced coefficient filtering to machine precision. Monte Carlo error decreased as MM increased but remained heterogeneous across predictors: at M=256M=256, the mean was 0.0206 and the maximum was 0.109.

Hence, averaging over the compact torus realizes the algebraic Weyl projection: the deterministic quadrature is exact up to floating-point error. The Monte Carlo realization converges slowly enough that its maximum error over predictors is still above 10−110^{-1} at 256 evaluations, which bounds what a sampled average can be expected to deliver for predictors outside the Laurent class. This experiment concerns the numerical operator, not its predictive benefit.

Robustness under unit changes. Section 4.5.2 reports a single training-set size and a single dictionary size. Here we vary both, to test how the tradeoff between fixed-unit accuracy and robustness under unit changes depends on them. For drag-force regression, we vary the number of labeled training examples N∈{50,100,300,1000,3000}N \in\{50,100,300,1000,3000\} (Table 8) and the number of non-invariant features W∈{0,25,100,250,500}W \in\{0,25,100,250,500\} (Table 9), that is, dictionary features whose units weight UαU_\alpha does not match the output weight vFv_F and which QvFQ_{v_F} therefore deletes. We compare raw ridge, its exact coefficient projection, the frame average, and a direct invariant regression fitted on the invariant features alone. We also test an empirical L2(μ)L^2(\mu) approximation, where μ\mu is an input distribution: a ridge regression on the invariant features is fitted to predictions of the frozen raw model on a separate unlabelled calibration sample. Unlike exact projection, this refit may change the coefficients of the invariant features.

TrainingNNModel, W=500W = 500Fixed unitsTransformed units (small shift)Transformed units (large shift)
Fixed units25Baseline0.00532±0.002810.00532 \pm0.00281(8.17±18.1)×105(8.17 \pm18.1) \times10^{5}(3.24±6.23)×1017(3.24 \pm6.23) \times10^{17}
Weyl-Projected0.00506±0.001570.00506 \pm0.001570.00488±0.001460.00488 \pm0.001460.00519±0.001700.00519 \pm0.00170
Empirical L2(μ)L^2(\mu)0.00452±0.002280.00452 \pm0.002280.00425±0.001760.00425 \pm0.001760.00449±0.001970.00449 \pm0.00197
Invariant regression0.00435±0.001850.00435 \pm0.001850.00422±0.001510.00422 \pm0.001510.00436±0.001660.00436 \pm0.00166
Frame average m=4m = 40.281±0.1820.281 \pm0.1820.266±0.2100.266 \pm0.2100.310±0.2360.310 \pm0.236
250Baseline0.0154±0.007370.0154 \pm0.00737(1.74±3.71)×1011(1.74 \pm3.71) \times10^{11}(2.16±5.64)×1026(2.16 \pm5.64) \times10^{26}
Weyl-Projected0.0663±0.04940.0663 \pm0.04940.0651±0.04850.0651 \pm0.04850.0656±0.04820.0656 \pm0.0482
Empirical L2(μ)L^2(\mu)0.00808±0.003310.00808 \pm0.003310.00853±0.003770.00853 \pm0.003770.00880±0.004050.00880 \pm0.00405
Invariant regression0.00435±0.001850.00435 \pm0.001850.00422±0.001510.00422 \pm0.001510.00436±0.001660.00436 \pm0.00166
Frame average m=4m = 41.58±1.051.58 \pm1.051.70±1.281.70 \pm1.281.91±1.681.91 \pm1.68
Transformed units (small shift)25Baseline0.00436±0.001820.00436 \pm0.001820.00459±0.001590.00459 \pm0.00159(9.76±21.1)×107(9.76 \pm21.1) \times10^{7}
Weyl-Projected0.00437±0.001820.00437 \pm0.001820.00421±0.001520.00421 \pm0.001520.00444±0.001650.00444 \pm0.00165
Empirical L2(μ)L^2(\mu)0.00463±0.001840.00463 \pm0.001840.00474±0.001790.00474 \pm0.001790.00486±0.001870.00486 \pm0.00187
Invariant regression0.00435±0.001850.00435 \pm0.001850.00422±0.001510.00422 \pm0.001510.00436±0.001660.00436 \pm0.00166
Frame average m=4m = 40.00440±0.001870.00440 \pm0.001870.00425±0.001560.00425 \pm0.001560.00446±0.001670.00446 \pm0.00167
250Baseline0.0138±0.01170.0138 \pm0.01170.0380±0.02500.0380 \pm0.0250(2.41±6.00)×1012(2.41 \pm6.00) \times10^{12}
Weyl-Projected0.0154±0.01290.0154 \pm0.01290.0146±0.01190.0146 \pm0.01190.0145±0.01200.0145 \pm0.0120
Empirical L2(μ)L^2(\mu)0.0402±0.05150.0402 \pm0.05150.0534±0.09630.0534 \pm0.09630.0482±0.07960.0482 \pm0.0796
Invariant regression0.00435±0.001850.00435 \pm0.001850.00422±0.001510.00422 \pm0.001510.00436±0.001660.00436 \pm0.00166
Frame average m=4m = 40.0175±0.01150.0175 \pm0.01150.0167±0.01170.0167 \pm0.01170.0168±0.01110.0168 \pm0.0111

Table 8. Test NRMSE at W=500W = 500 for two training-set sizes. Rows give the training regime and the model; columns give the unit system of the test data. Entries are the mean ±\pm sample standard deviation over seeds.

TrainingWWModel, N=3000N = 3000Fixed unitsTransformed units (small shift)Transformed units (large shift)
Fixed units25Baseline0.00532±0.002810.00532 \pm0.00281(8.17±18.1)×105(8.17 \pm18.1) \times10^{5}(3.24±6.23)×1017(3.24 \pm6.23) \times10^{17}
Weyl-Projected0.00506±0.001570.00506 \pm0.001570.00488±0.001460.00488 \pm0.001460.00519±0.001700.00519 \pm0.00170
Empirical L2(μ)L^2(\mu)0.00452±0.002280.00452 \pm0.002280.00425±0.001760.00425 \pm0.001760.00449±0.001970.00449 \pm0.00197
Invariant regression0.00435±0.001850.00435 \pm0.001850.00422±0.001510.00422 \pm0.001510.00436±0.001660.00436 \pm0.00166
Frame average m=4m = 40.281±0.1820.281 \pm0.1820.266±0.2100.266 \pm0.2100.310±0.2360.310 \pm0.236
250Baseline0.0154±0.007370.0154 \pm0.00737(1.74±3.71)×1011(1.74 \pm3.71) \times10^{11}(2.16±5.64)×1026(2.16 \pm5.64) \times10^{26}
Weyl-Projected0.0663±0.04940.0663 \pm0.04940.0651±0.04850.0651 \pm0.04850.0656±0.04820.0656 \pm0.0482
Empirical L2(μ)L^2(\mu)0.00808±0.003310.00808 \pm0.003310.00853±0.003770.00853 \pm0.003770.00880±0.004050.00880 \pm0.00405
Invariant regression0.00435±0.001850.00435 \pm0.001850.00422±0.001510.00422 \pm0.001510.00436±0.001660.00436 \pm0.00166
Frame average m=4m = 41.58±1.051.58 \pm1.051.70±1.281.70 \pm1.281.91±1.681.91 \pm1.68
Transformed units (small shift)25Baseline0.00436±0.001820.00436 \pm0.001820.00459±0.001590.00459 \pm0.00159(9.76±21.1)×107(9.76 \pm21.1) \times10^{7}
Weyl-Projected0.00437±0.001820.00437 \pm0.001820.00421±0.001520.00421 \pm0.001520.00444±0.001650.00444 \pm0.00165
Empirical L2(μ)L^2(\mu)0.00463±0.001840.00463 \pm0.001840.00474±0.001790.00474 \pm0.001790.00486±0.001870.00486 \pm0.00187
Invariant regression0.00435±0.001850.00435 \pm0.001850.00422±0.001510.00422 \pm0.001510.00436±0.001660.00436 \pm0.00166
Frame average m=4m = 40.00440±0.001870.00440 \pm0.001870.00425±0.001560.00425 \pm0.001560.00446±0.001670.00446 \pm0.00167
250Baseline0.0138±0.01170.0138 \pm0.01170.0380±0.02500.0380 \pm0.0250(2.41±6.00)×1012(2.41 \pm6.00) \times10^{12}
Weyl-Projected0.0154±0.01290.0154 \pm0.01290.0146±0.01190.0146 \pm0.01190.0145±0.01200.0145 \pm0.0120
Empirical L2(μ)L^2(\mu)0.0402±0.05150.0402 \pm0.05150.0534±0.09630.0534 \pm0.09630.0482±0.07960.0482 \pm0.0796
Invariant regression0.00435±0.001850.00435 \pm0.001850.00422±0.001510.00422 \pm0.001510.00436±0.001660.00436 \pm0.00166
Frame average m=4m = 40.0175±0.01150.0175 \pm0.01150.0167±0.01170.0167 \pm0.01170.0168±0.01110.0168 \pm0.0111

Table 9. Test NRMSE at N=3000N = 3000 for two non-invariant feature counts. Rows give the training regime and the model; columns give the unit system of the test data. Entries are the mean ±\pm sample standard deviation over seeds.

Weyl unitary trick with Lorentz group on vector valued function

In this section, we consider vector-valued outputs. Let

F:(Cd)m→Cd.F:(\mathbb{C}^{d})^{m}\to\mathbb{C}^{d}.

The Weyl unitarian projection equivariant Reynolds operator is

(QKeqF)(U):=∫Kk−1F(kU) dk.(64)(Q_{K}^{\mathrm{eq}}F)(U):=\int_{K} k^{-1}F(kU)\,dk. \tag*{(64)}
TrainingNNModel, W=500W = 500Fixed unitsTransformed units (small shift)Transformed units (large shift)
Fixed units50Baseline1.03±0.5941.03 \pm0.594(3.83±7.52)×1011(3.83 \pm7.52) \times10^{11}(1.08±2.21)×1028(1.08 \pm2.21) \times10^{28}
Weyl-Projected0.776±0.06160.776 \pm0.06160.770±0.06430.770 \pm0.06430.776±0.06020.776 \pm0.0602
Empirical L2(μ)L^2(\mu)0.662±0.4990.662 \pm0.4990.629±0.4530.629 \pm0.4530.753±0.7410.753 \pm0.741
Invariant regression0.0224±0.02150.0224 \pm0.02150.0222±0.02040.0222 \pm0.02040.0229±0.02110.0229 \pm0.0211
Frame average m=4m = 411.7±9.3811.7 \pm9.3813.5±13.613.5 \pm13.612.8±10.512.8 \pm10.5
3000Baseline0.0167±0.005630.0167 \pm0.00563(5.01±14.0)×1011(5.01 \pm14.0) \times10^{11}(1.74±4.17)×1027(1.74 \pm4.17) \times10^{27}
Weyl-Projected0.291±0.07470.291 \pm0.07470.287±0.07550.287 \pm0.07550.292±0.07510.292 \pm0.0751
Empirical L2(μ)L^2(\mu)0.00768±0.002440.00768 \pm0.002440.00786±0.002940.00786 \pm0.002940.00807±0.002880.00807 \pm0.00288
Invariant regression0.00435±0.001850.00435 \pm0.001850.00422±0.001510.00422 \pm0.001510.00436±0.001660.00436 \pm0.00166
Frame average m=4m = 42.15±2.162.15 \pm2.161.67±1.061.67 \pm1.062.37±2.182.37 \pm2.18
Transformed units (small shift)50Baseline0.367±0.4710.367 \pm0.471104±229104 \pm229(2.21±6.94)×1017(2.21 \pm6.94) \times10^{17}
Weyl-Projected0.314±0.2070.314 \pm0.2070.309±0.2020.309 \pm0.2020.313±0.2100.313 \pm0.210
Empirical L2(μ)L^2(\mu)287±630287 \pm630287±620287 \pm620267±575267 \pm575
Invariant regression0.0224±0.02150.0224 \pm0.02150.0222±0.02040.0222 \pm0.02040.0229±0.02110.0229 \pm0.0211
Frame average m=4m = 40.794±0.7940.794 \pm0.7940.661±0.5800.661 \pm0.5800.726±0.6950.726 \pm0.695
3000Baseline0.0197±0.01180.0197 \pm0.01180.0623±0.07260.0623 \pm0.0726(2.02±5.37)×1012(2.02 \pm5.37) \times10^{12}
Weyl-Projected0.0241±0.01590.0241 \pm0.01590.0232±0.01570.0232 \pm0.01570.0234±0.01680.0234 \pm0.0168
Empirical L2(μ)L^2(\mu)0.0502±0.04770.0502 \pm0.04770.0663±0.09200.0663 \pm0.09200.0600±0.07530.0600 \pm0.0753
Invariant regression0.00435±0.001850.00435 \pm0.001850.00422±0.001510.00422 \pm0.001510.00436±0.001660.00436 \pm0.00166
Frame average m=4m = 40.0237±0.01150.0237 \pm0.01150.0235±0.01280.0235 \pm0.01280.0232±0.01270.0232 \pm0.0127

Table 10.

This is the specialization of (12) with ϕC(k)U=kU\phi_C(k)U = kU and output representation ψC(k)=k\psi_C(k) = k. By the same fixed-point argument used in the compact case,

(QKeqF)(hU)=h(QKeqF)(U),h∈K.(65)(Q^{\mathrm{eq}}_K F)(hU) = h(Q^{\mathrm{eq}}_K F)(U), \qquad h \in K. \tag*{(65)}

Proposition D.1 (Vector-valued Weyl unitarian projection average). Let F:(Cd)m→CdF : (\mathbb{C}^{d})^m \to\mathbb{C}^{d} be polynomial, or more generally entire holomorphic. Then the Weyl unitarian projection average QKeqFQ^{\mathrm{eq}}_K F is a KK-equivariant holomorphic map. Moreover, it has the form

(QKeqF)(U)=∑j=1mαj(Γ(U))uj,(66)(Q^{\mathrm{eq}}_K F)(U) = \sum_{j=1}^{m} \alpha_j(\Gamma(U))u_j, \tag*{(66)}

where the coefficients αj\alpha_j are invariant scalar functions of the Gram data. For polynomial FF, the αj\alpha_j may be chosen polynomial in the entries of Γ(U)\Gamma(U); for entire FF, they may be chosen holomorphic on the relevant Gram strata.

Proof. Equation (65) gives KK-equivariance. Since the averaged map is holomorphic, KK-equivariance extends to equivariance under the complexified group O(d,C)O(d,\mathbb{C}). The first fundamental theorem for orthogonal covariants states that O(d,C)O(d,\mathbb{C})-equivariant vector-valued polynomial maps from mm standard vectors to one standard vector are generated, as a module over the invariant scalar functions, by the input vectors u1,…,umu_1,\ldots,u_m. Therefore the output has the form (66). The holomorphic case follows by applying the same argument to the homogeneous Taylor pieces and summing locally uniformly.

Thus the Weyl unitarian projection equivariant average always lies in the complex span of the input vectors:

(QKeqF)(U)∈span⁡C{u1,…,um}.(Q^{\mathrm{eq}}_K F)(U) \in\operatorname{span}_{\mathbb{C}}\{u_1,\ldots,u_m\}.

This is the vector-valued analogue of the Gram-matrix statement for scalar invariants.

On the locus where the Gram matrix is nondegenerate, the coefficients can be computed directly. Let UU also denote the d×md \times m matrix whose jj-th column is uju_j, and set

G(U):=UTU=Γ(U).G(U) := U^{T}U = \Gamma(U).

Define

bi(U):=∫KF(kU)⋅(kui) dk,i=1,…,m.(67)b_i(U) := \int_K F(kU)\cdot(ku_i)\,dk,\qquad i=1,\ldots,m. \tag*{(67)}

Since kk preserves the complex bilinear form,

bi(U)=(QKeqF)(U)⋅ui.b_i(U) = (Q^{\mathrm{eq}}_K F)(U)\cdot u_i.

If G(U)G(U) is invertible and

(QKeqF)(U)=Uα(U),(Q^{\mathrm{eq}}_K F)(U) = U\alpha(U),

then

b(U)=G(U)α(U),b(U) = G(U)\alpha(U),

and hence

α(U)=G(U)−1b(U),(QKeqF)(U)=UG(U)−1b(U).(68)\alpha(U) = G(U)^{-1}b(U),\qquad(Q^{\mathrm{eq}}_K F)(U) = U G(U)^{-1}b(U). \tag*{(68)}

More generally, a pseudoinverse formula is valid on strata where ker⁡G(U)=ker⁡U\ker G(U) = \ker U, equivalently where the bilinear form is nondegenerate on span⁡C{u1,…,um}\operatorname{span}_{\mathbb{C}}\{u_1,\ldots,u_m\}. On degenerate strata, especially when null directions are present, the representation (66) still holds, but the coefficients are not uniquely determined by the pairings bib_i. In such cases, a Moore–Penrose pseudoinverse may be used as a numerical regularization, but it should not be interpreted as a canonical algebraic formula.

Finally, suppose the original real output is Lorentz-vector-valued,

f:(R1,d−1)m→R1,d−1,f:(\mathbb{R}^{1,d-1})^m\to\mathbb{R}^{1,d-1},

and the complexified output is chosen so that

F(W(p1),…,W(pm))=W(f(p1,…,pm)).F(W(p_1),\ldots,W(p_m))=W(f(p_1,\ldots,p_m)).

If the Weyl unitarian projection average restricts back to the Lorentz real slice, then (66) becomes

f^(p1,…,pm)=∑j=1mαj((η(pi,pj))i,j=1m)pj.(69)\widehat{f}(p_1,\ldots,p_m)=\sum_{j=1}^{m}\alpha_j\left(\left(\eta(p_i,p_j)\right)_{i,j=1}^{m}\right)p_j. \tag*{(69)}

Thus a Lorentz-equivariant vector output is built from the input four-vectors with Lorentz-invariant scalar coefficients. This is the equivariant analogue of (40).

References

  1. [1]E. L. Lehmann and George Casella. Theory of Point Estimation. 2nd ed. Springer Texts in Statistics. New York: Springer, 1998. DOI: 10.1007/b98854.DOI
  2. [2]Shuxiao Chen, Edgar Dobriban, and Jane H. Lee. “A Group-Theoretic Framework for Data Augmentation”. In: Advances in Neural Information Processing Systems. Vol. 33. 2020, pp. 21321–21333.arxiv.org/abs/1907.10905
  3. [3]Clare Lyle et al. “On the Benefits of Invariance in Neural Networks”. In: arXiv preprint arXiv:2005.00178 (2020). DOI: 10.48550/arXiv.2005.00178. arXiv: 2005.00178 [cs.LG].DOI
  4. [4]Akiyoshi Sannai, Makoto Kawano, and Wataru Kumagai. “Invariant and Equivariant Reynolds Networks”. In: Journal of Machine Learning Research 25.42 (2024), pp. 1–36. URL: https://jmlr.org/papers/v25/22-0891.html.
  5. [5]Bryn Elesedy and Sheheryar Zaidi. “Provably Strict Generalisation Benefit for Equivariant Models”. In: Proceedings of the 38th International Conference on Machine Learning. PMLR, 2021, pp. 2959–2969.arxiv.org/abs/2102.10333
  6. [6]Ruben Ohana et al. “The Well: A large-scale collection of diverse physics simulations for machine learning”. In: Advances in Neural Information Processing Systems 37 (2024), pp. 44989–45037.arxiv.org/abs/2412.00568
  7. [7]John Shawe-Taylor. “Building symmetries into feedforward networks”. In: 1989 First IEEE International Conference on Artificial Neural Networks, (Conf. Publ. No. 313). IET. 1989, pp. 158–162.
  8. [8]Taco Cohen and Max Welling. “Group equivariant convolutional networks”. In: International Conference on Machine Learning. PMLR. 2016, pp. 2990–2999.arxiv.org/abs/1602.07576
  9. [9]Risi Kondor and Shubhendu Trivedi. “On the generalization of equivariance and convolution in neural networks to the action of compact groups”. In: International conference on machine learning. PMLR. 2018, pp. 2747–2755.arxiv.org/abs/1802.03690
  10. [10]Taco S Cohen, Mario Geiger, and Maurice Weiler. “A general theory of equivariant cnns on homogeneous spaces”. In: Advances in Neural Information Processing Systems 32 (2019).arxiv.org/abs/1811.02017
  11. [11]Taco S Cohen and Max Welling. “Steerable cnns”. In: arXiv preprint arXiv:1612.08498 (2016).arxiv.org/abs/1612.08498
  12. [12]Risi Kondor. “N-body networks: a covariant hierarchical neural network architecture for learning atomic potentials”. In: arXiv preprint arXiv:1803.01588 (2018).arxiv.org/abs/1803.01588
  13. [13]Mario Geiger and Tess Smidt. “e3nn: Euclidean neural networks”. In: arXiv preprint arXiv:2207.09453 (2022).arxiv.org/abs/2207.09453
  14. [14]Marc Finzi, Max Welling, and Andrew Gordon Wilson. “A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups”. In: Proceedings of the 38th International Conference on Machine Learning. Vol. 139. Proceedings of Machine Learning Research. PMLR, 2021, pp. 3318–3328. arXiv: 2104.09459 [cs.LG].arxiv.org/abs/2104.09459
  15. [15]Daniel Worrall and Max Welling. “Deep scale-spaces: Equivariance over scale”. In: Advances in Neural Information Processing Systems 32 (2019).arxiv.org/abs/1905.11697
  16. [16]Rui Wang, Robin Walters, and Rose Yu. “Incorporating symmetry into deep dynamics models for improved generalization”. In: arXiv preprint arXiv:2002.03061 (2020).arxiv.org/abs/2002.03061
  17. [17]Ningyuan Huang, Ron Levie, and Soledad Villar. “Approximately equivariant graph networks”. In: Advances in Neural Information Processing Systems 36 (2023), pp. 34627–34660.arxiv.org/abs/2308.10436
  18. [18]Wilson G Gregory, Kaze WK Wong, and Soledad Villar. “Robust Emulator for Compressible Navier-Stokes using Equivariant Geometric Convolutions”. In: Machine Learning and the Physical Sciences Workshop. NeurIPS. 2024.
  19. [19]Johannes Jakob Meyer et al. “Exploiting Symmetry in Variational Quantum Machine Learning”. In: PRX Quantum 4 (1 Mar. 2023), p. 010328. DOI: 10.1103/PRXQuantum.4.010328. URL: https://link.aps.org/doi/10.1103/PRXQuantum.4.010328.DOI
  20. [20]Soledad Villar et al. “Scalars are universal: Equivariant machine learning, structured like classical physics”. In: Advances in neural information processing systems 34 (2021), pp. 28848–28863.
  21. [21]Ben Blum-Smith and Soledad Villar. “Machine learning and invariant theory”. In: Notices of the American Mathematical Society 70.08 (2023), pp. 1–1.
  22. [22]Ben Blum-Smith et al. “A Galois theorem for machine learning: Functions on symmetric matrices and point clouds via lightweight invariant features”. In: arXiv preprint arXiv:2405.08097 (2024).
  23. [23]Wilson G Gregory et al. “Learning equivariant tensor functions with applications to sparse vector recovery”. In: arXiv preprint arXiv:2406.01552 (2024).
  24. [24]Dmitry Yarotsky. “Universal approximations of invariant maps by neural networks”. In: Constructive Approximation 55.1 (2022), pp. 407–474.arxiv.org/abs/1804.10306
  25. [25]Ryan Murphy et al. “Relational pooling for graph representations”. In: International Conference on Machine Learning. PMLR. 2019, pp. 4663–4673.arxiv.org/abs/1903.02541
  26. [26]Mircea Petrache and Shubhendu Trivedi. “Approximation-generalization trade-offs under (approximate) group equivariance”. In: Advances in Neural Information Processing Systems 36 (2023), pp. 61936–61959.arxiv.org/abs/2305.17592
  27. [27]Behrooz Tahmasebi and Stefanie Jegelka. “The exact sample complexity gain from invariances for kernel regression”. In: Advances in Neural Information Processing Systems 36 (2023), pp. 55616–55646.arxiv.org/abs/2303.14269
  28. [28]Alberto Bietti, Luca Venturi, and Joan Bruna. On the Sample Complexity of Learning under Invariance and Geometric Stability. 2021. arXiv: 2106.07148 [stat.ML]. URL: https://arxiv.org/abs/2106.07148.
  29. [29]Behrooz Tahmasebi and Melanie Weber. Achieving Approximate Symmetry Is Exponentially Easier than Exact Symmetry. 2026. arXiv: 2512.11855 [cs.LG]. URL: https://arxiv.org/abs/2512.11855.
  30. [30]Valentino F. Foit, David W. Hogg, and Soledad Villar. Group Averaging for Physics Applications: Accuracy Improvements at Zero Training Cost. 2025. arXiv: 2511.09573 [cs.LG]. URL: https://arxiv.org/abs/2511.09573.
  31. [31]Omri Puny et al. Frame Averaging for Invariant and Equivariant Network Design. 2022. arXiv: 2110.03336 [cs.LG]. URL: https://arxiv.org/abs/2110.03336.
  32. [32]Yuchao Lin et al. Equivariance via Minimal Frame Averaging for More Symmetries and Efficiency. 2026. arXiv: 2406.07598 [cs.LG]. URL: https://arxiv.org/abs/2406.07598.
  33. [33]Sékou-Oumar Kaba et al. “Equivariance with learned canonicalization functions”. In: International Conference on Machine Learning. PMLR. 2023, pp. 15546–15566.arxiv.org/abs/2211.06489
  34. [34]Zakhar Shumaylov et al. Lie Algebra Canonicalization: Equivariant Neural Operators under arbitrary Lie Groups. 2025. arXiv: 2410.02698 [cs.LG]. URL: https://arxiv.org/abs/2410.02698.
  35. [35]Jinwoo Kim et al. “Learning Probabilistic Symmetrization for Architecture Agnostic Equivariance”. In: Advances in Neural Information Processing Systems. Vol. 36. 2023, pp. 18582–18612. arXiv: 2306.02866 [cs.LG].arxiv.org/abs/2306.02866
  36. [36]Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. “ImageNet Classification with Deep Convolutional Neural Networks”. In: Advances in Neural Information Processing Systems. Vol. 25. 2012, pp. 1097–1105.
  37. [37]Christian Szegedy et al. “Going Deeper with Convolutions”. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015, pp. 1–9.
  38. [38]Simone Monaco, Sebastiano Barresi, and Daniele Apiletti. “Lorentz-Invariant Augmentation for High-Energy Physics Deep Learning Models”. In: Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2023). Ed. by Rosa Meo and Fabrizio Silvestri. Vol. 2135. Communications in Computer and Information Science. Cham: Springer, 2025, pp. 368–376. DOI: 10.1007/978-3-031-74633-8\_26.DOI
  39. [39]Kimberly Stachenfeld et al. “Learned coarse models for efficient turbulence simulation”. In: arXiv preprint arXiv:2112.15275 (2021).arxiv.org/abs/2112.15275
  40. [40]Michael McCabe et al. “Multiple physics pretraining for physical surrogate models”. In: arXiv preprint arXiv:2310.02994 (2023).arxiv.org/abs/2310.02994
  41. [41]Liu Yang et al. “In-context operator learning with data prompts for differential equation problems”. In: Proceedings of the National Academy of Sciences 120.39 (2023), e2310142120.DOI
  42. [42]Md Ashiqur Rahman et al. “Pretraining codomain attention neural operators for solving multiphysics pdes”. In: Advances in Neural Information Processing Systems 37 (2024), pp. 104035–104064.arxiv.org/abs/2403.12553
  43. [43]Junhong Shen, Tanya Marwah, and Ameet Talwalkar. “Ups: Efficiently building foundation models for pde solving via cross-modal adaptation”. In: arXiv preprint arXiv:2403.07187 (2024).arxiv.org/abs/2403.07187
  44. [44]Maximilian Herde et al. “Poseidon: Efficient foundation models for pdes”. In: Advances in Neural Information Processing Systems 37 (2024), pp. 72525–72624.arxiv.org/abs/2405.19101
  45. [45]Tianyu Chen et al. “OmniArch: Building Foundation Model For Scientific Computing”. In: arXiv preprint arXiv:2402.16014 (2024).arxiv.org/abs/2402.16014
  46. [46]Jingmin Sun et al. “Towards a foundation model for partial differential equations: Multioperator learning and extrapolation”. In: Physical Review E 111.3 (2025), p. 035304.DOI
  47. [47]Makoto Takamoto et al. “Pdebench: An extensive benchmark for scientific machine learning”. In: Advances in Neural Information Processing Systems 35 (2022), pp. 1596–1611.arxiv.org/abs/2210.07182
  48. [48]Jeroen Audenaert et al. “The multimodal universe: enabling large-scale machine learning with 100TB of astronomical scientific data”. In: arXiv preprint arXiv:2412.02527 (2024).arxiv.org/abs/2412.02527
  49. [49]Hermann Weyl. The Classical Groups: Their Invariants and Representations. Princeton University Press, 1939.
  50. [50]Ben Gripaios, Ward Haddadin, and Christopher G. Lester. “Lorentz- and Permutation-Invariants of Particles”. In: Journal of Physics A: Mathematical and Theoretical 54.15 (2021), p. 155201. DOI: 10.1088/1751-8121/abe58c. arXiv: 2003.05487.DOI
  51. [51]Ward Haddadin. Invariant Polynomials and Machine Learning. 2021. arXiv: 2104.12733.arxiv.org/abs/2104.12733
  52. [52]Patrick T. Komiske, Eric M. Metodiev, and Jesse Thaler. “Energy Flow Networks: Deep Sets for Particle Jets”. In: Journal of High Energy Physics 2019.1 (2019), p. 121. DOI: 10.1007/JHEP01(2019)121. arXiv: 1810.05165 [hep-ph].DOI
  53. [53]Huilin Qu and Loukas Gouskos. “ParticleNet: Jet Tagging via Particle Clouds”. In: Physical Review D 101.5 (2020), p. 056019. DOI: 10.1103/PhysRevD.101.056019. arXiv: 1902.08570 [hep-ph].DOI
  54. [54]Huilin Qu, Congqiao Li, and Sitian Qian. “Particle Transformer for Jet Tagging”. In: Proceedings of the 39th International Conference on Machine Learning. Vol. 162. Proceedings of Machine Learning Research. PMLR, 2022, pp. 18281–18292. arXiv: 2202.03772 [hep-ph].arxiv.org/abs/2202.03772
  55. [55]Anja Butter et al. “Deep-Learned Top Tagging with a Lorentz Layer”. In: SciPost Physics 5.3 (2018), p. 028. DOI: 10.21468/SciPostPhys.5.3.028. arXiv: 1707.08966 [hep-ph].DOI
  56. [56]Alexander Bogatskiy et al. “Lorentz Group Equivariant Neural Network for Particle Physics”. In: Proceedings of the 37th International Conference on Machine Learning. Vol. 119. Proceedings of Machine Learning Research. PMLR, 2020, pp. 992–1002. arXiv: 2006.04780 [hep-ph].arxiv.org/abs/2006.04780
  57. [57]Shiqi Gong et al. “An Efficient Lorentz Equivariant Graph Neural Network for Jet Tagging”. In: Journal of High Energy Physics 2022.7 (2022), p. 030. DOI: 10.1007/JHEP07(2022)030. arXiv: 2201.08187 [hep-ph].DOI
  58. [58]Shikai Qiu et al. “A Holistic Approach to Predicting Top Quark Kinematic Properties with the Covariant Particle Transformer”. In: Physical Review D 107.11 (2023), p. 114029. DOI: 10.1103/PhysRevD.107.114029. arXiv: 2203.05687 [hep-ph].DOI
  59. [59]Alexander Bogatskiy et al. “Explainable Equivariant Neural Networks for Particle Physics: PELICAN”. In: Journal of High Energy Physics 2024.3 (2024), p. 113. DOI: 10.1007/JHEP03(2024)113. arXiv: 2307.16506 [hep-ph].DOI
  60. [60]David Ruhe, Johannes Brandstetter, and Patrick Forré. “Clifford Group Equivariant Neural Networks”. In: Advances in Neural Information Processing Systems. Vol. 36. 2023. arXiv: 2305.11141 [cs.LG].arxiv.org/abs/2305.11141
  61. [61]Jonas Spinner et al. Lorentz-Equivariant Geometric Algebra Transformers for High-Energy Physics. 2024. arXiv: 2405.18406 [physics.data-an].
  62. [62]Johann Brehmer et al. A Lorentz-Equivariant Transformer for All of the LHC. 2025. arXiv: 2411.00446 [hep-ph].DOI
  63. [63]Soledad Villar et al. “Dimensionless machine learning: Imposing exact units equivariance”. In: Journal of Machine Learning Research 24.109 (2023), pp. 1–32.arxiv.org/abs/2204.00887
  64. [64]Edgar Buckingham. “On Physically Similar Systems; Illustrations of the Use of Dimensional Equations”. In: Physical Review 4.4 (1914), pp. 345–376. DOI: 10.1103/PhysRev.4.345.DOI
  65. [65]Hannah Lawrence et al. To Augment or Not to Augment? Diagnosing Distributional Symmetry Breaking. 2025. arXiv: 2510.01349 [cs.LG].arxiv.org/abs/2510.01349
  66. [67]Keaton J Burns et al. “Dedalus: A flexible framework for numerical simulations with spectral methods”. In: Physical Review Research 2.2 (2020), p. 023068.DOI
  67. [68]Drummond B Fielding et al. “Multiphase gas and the fractal nature of radiative turbulent mixing layers”. In: The Astrophysical Journal Letters 894.2 (2020), p. L24.DOI
  68. [69]Olaf Ronneberger, Philipp Fischer, and Thomas Brox. “U-net: Convolutional networks for biomedical image segmentation”. In: International Conference on Medical image computing and computer-assisted intervention. Springer, 2015, pp. 234–241.arxiv.org/abs/1505.04597
  69. [70]Zhuang Liu et al. “A convent for the 2020s”. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022, pp. 11976–11986.
  70. [71]Jonas Spinner et al. “Lorentz Local Canonicalization: How to Make Any Network Lorentz-Equivariant”. In: Advances in Neural Information Processing Systems. Vol. 38. 2025. arXiv: 2505.20280.arxiv.org/abs/2505.20280
  71. [72]William Fulton and Joe Harris. Representation Theory: A First Course. Vol. 129. Graduate Texts in Mathematics. New York: Springer-Verlag, 1991. DOI: 10.1007/978-1-4612-0979-9.DOI
  72. [73]Nadav Dym, Hannah Lawrence, and Jonathan W. Siegel. Equivariant Frames and the Impossibility of Continuous Canonicalization. 2024. arXiv: 2402.16077 [cs.LG]. URL: https://arxiv.org/abs/2402.16077.
  73. [74]Patrick T. Komiske, Eric M. Metodiev, and Jesse Thaler. Pythia8 Quark and Gluon Jets for Energy Flow. Version v1. 2019. DOI: 10.5281/zenodo.3164691. URL: https://doi.org/10.5281/zenodo.3164691.

Paper details

vlnc:W_yrr7ghbk
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents