betaFeedback

WorksComputer Science

Data-driven Discovery of Lie Symmetries and First Integrals with Latent ODEs

Abstract

We present LatentLieODE, a novel neural ODE method to discover linear Lie symmetries and first integrals of the learned dynamics from timeseries observations such as video inputs. It starts with an autoencoder, which learns a low dimensional representation of the observed system and then maps trajectories to decorrelated latent coordinates z∈Rdz \in \mathbb{R}^d. A bank of linear infinitesimal generators Ak∈Rd×d\mathbf{A}_k \in \mathbb{R}^{d \times d} is fitted jointly against a differentiable Lie-bracket residual Jf(z)Az−Af(z)\mathbf{J}_f(z)\mathbf{A}z - \mathbf{A} f(z), while an MLP optimises the latent ODE f(z)=dz/dtf(z) = \mathrm{d}z/\mathrm{d}t to reconstruct the observation-space timeseries. After suitable generators and ff are found, we fit a third network HϕH_ϕ to generate the first integral of the system. Across six benchmark systems (harmonic oscillator, simple and twin pendulums, Lotka-Volterra, pixel pendulum, sphere geodesics) we find each system's relevant symmetries, and the discovered invariants match ground-truth conserved quantities at ∣ρ∣≥0.978|ρ| \geq 0.978. On the sphere, the pipeline recovers a non-abelian so(3)\mathfrak{so}(3), certified by the negative-definite Killing form of its recovered structure constants. This work presents significant advances in data-driven discovery of symmetries and provides a new pathway for neural surrogates of dynamical systems.

Figure 1. An autoencoder maps observations (states or video) to a decorrelated latent chart where a latent ODE z˙=f(z)\dot{z}=f(z) carries the dynamics and orbits are circular. A linear generator A\mathbf{A} is then fitted by minimising the normalised Lie-bracket residual Ez∼data∥Jf(z)Az−Af(z)∥2/∥f(z)∥2\mathbb{E}_{z\sim\mathrm{data}}\|\mathbf{J}_f(z)\mathbf{A}z-\mathbf{A}f(z)\|^2/\|f(z)\|^2 on encoded data.

Introduction

Symmetry is one of the fundamental frameworks of physics. Continuous symmetries are linked with conservation laws via Noether’s theorem [10], they constrain dynamics, and they reduce the dimensionality of systems. A growing body of work aims to build known symmetries into networks as equivariance constraints [4, 2, 14] or to learn dynamics that respect an assumed analytic structure such as a Hamiltonian [7, 5, 13]. However, the inverse problem is also of great interest: Given only noisy observations of a system, can one determine which symmetries (if any) it has? Existing approaches search for symmetries directly in observation space [8, 6] or learn group actions with adversarial objectives [15, 16], typically on state-space observations with the symmetry group’s dimension fixed in advance.

We take a different route, motivated by the work of Yang et al. [16]. A symmetry that is nonlinear and incomprehensible in observation space is often linear in the right coordinate system. Hence, we create a neural pipeline that (1) automatically finds a reduced dimensional representation of a given dynamical system; (2) learns a latent ODE [3, 11] from trajectories, mapping observations into a low-dimensional decorrelated coordinate chart; (3) fits a deliberately over-complete bank of linear generators against a differentiable Lie-bracket residual; and (4) reads off the algebra’s dimension from the generator-norm spectrum, its structure constants from pairwise brackets, and its conservation laws from jointly-fitted first-integral networks.

Main contributions:

  1. A fully differentiable symmetry-discovery objective, paired with matrix-norm rank detection, for which the algebra dimension is an output.

  2. Discovery and optimisation of a first-integral based on the squared cosine between ∇H\nabla H and the flow, invariant to reparameterisation of HH, with post-hoc calibration to physical units.

  3. Validation on five classical system trajectories and a video observation, successfully recovering the true symmetries of each system.

Related work

Noether networks [1] meta-learn conserved quantities from raw observations by tailoring a prediction model at test time to respect a learned conservation law, using Noether’s theorem as an inductive bias for forecasting. The conserved quantity is discovered with the goal of improving prediction accuracy.

LaLiGAN [16], which extends work of Yang et al. [15], learns an autoencoder for a latent space, in which a symmetry becomes a linear group action, representing it by Lie algebra basis matrices. Training is carried out adversarially: A discriminator tests whether group-transformed latents remain indistinguishable from the data distribution. It thereby discovers distribution-level symmetries with the group’s dimension fixed in advance, treating dynamics and downstream structure (e.g. latent equation discovery) as separate, subsequent steps.

Hamiltonian and Lagrangian Neural Networks ([7, 5]) parameterise the scalar generating function of an assumed variational structure, which obtains the dynamics and exact energy conservation by construction. Autoencoder variants extend both approaches to pixel observations.

Key distinctions Hamiltonian and Lagrangian Neural Networks solve the inverse of our problem, with physical structure being imposed, rather than discovered, to learn better dynamics. LaLiGAN is the closest work. It also learns an autoencoder for a latent space where a nonlinear symmetry acts linearly and parameterises the symmetry by Lie algebra basis matrices. LaLiGAN’s symmetry condition is enforced distributionally, through a discriminator that judges whether transformed pairs remain indistinguishable from real ones, and is therefore only defined at an adversarial optimum. We impose the infinitesimal Lie bracket condition [vA,f]=0[v_A, f] = 0 directly on the learned vector field and evaluate it pointwise by automatic differentiation, giving a scalar residual at every state. First, the bracket can be thresholded, compared on- versus off-manifold, and reported as a verification statistic. Secondly, in our method the algebra’s dimension is an output of the spectral gap rather than a hyperparameter. Finally, we recover first integrals alongside the generators and calibrate them to physical units, a connection between discovered symmetry and conservation laws hinted at as future work by Yang et al. [16].

OursLaLiGANNoether NetsHNN/LNN
Discovers Lie generators✓✓––
Continuous-time dynamics learned jointly✓––✓
Invariants discovered, not assumed✓–✓–
Num. of symmetries detected, not specified✓–––
Expressivity guarantee for the action–✓––
Exact conservation by construction–––✓

Table 1. Capability comparison with the closest families of methods.

Method

Latent dynamics model We follow the non-variational latent ODE architecture from Sampson and Melchior [12], a modification of the implementation from Rubanova et al. [11], which encodes time series with a recurrent neural network. We do not employ the explicit path-minimizing penalty used by Sampson and Melchior [12], which is designed to perform parameter inference. It would compress the dynamics, while we seek to reshape them. For video/pixel data, the encoder and decoder are small convolutional stacks. Training minimises reconstruction error plus a latent-consistency term; for video, the per-pixel squared error is weighted by each pixel’s temporal variance (normalised to a mean of one), which removes the otherwise-dominant trivial optimum of predicting a static background.

Coordinate construction The latent chart (our new coordinate system) is recomputed from batch statistics at every training step via:

φ(z)=(z−c)TT,T=D12C~−12D−12,(1)\varphi(z) = (z-c)\mathbf{T}^{\mathsf{T}}, \qquad\mathbf{T} = \mathbf{D}^{\frac{1}{2}}\widetilde{\mathbf{C}}^{-\frac{1}{2}}\mathbf{D}^{-\frac{1}{2}}, \tag*{(1)}

where cc re-centres the batch of latents at the origin, D\mathbf{D} is the diagonal matrix of per-dimension batch variances, and C~\widetilde{\mathbf{C}} is the batch correlation matrix, whose inverse square root is computed by a Newton–Schulz iteration. T\mathbf{T} is therefore a variance-preserving decorrelation. The cross-correlations between latent dimensions are removed, while each dimension’s scale is kept, so that symmetries of the dynamics become linearly representable. A small ε=1×10−8\varepsilon= 1 \times10^{-8} is added to protect against singularities.

Setting and assumed form We seek one-parameter groups of transformations gsg_s of the latent state that commute with the flow Φt\Phi_t of the learned dynamics z˙=f(z)\dot{z} = f(z), i.e. gs∘Φt=Φt∘gsg_s \circ\Phi_t = \Phi_t \circ g_s, so that every solution maps to another. With vv representing the vector field that generates gsg_s, this condition is equivalent to the vanishing of the Lie bracket [v,f]=0[v, f] = 0. We restrict the search to linear generators v(z)=Azv(z) = Az in a learned coordinate chart, so gs=esAg_s = e^{sA}. The coordinate chart is trained to carry the nonlinearity of the systems. The decorrelation and the action-angle priors are chosen so that phase rotations, dilations and rigid rotations of (quasi-)periodic systems become linear, and symmetries with no linear representation at the chosen latent width are outside the search space by construction. No Hamiltonian or Lagrangian structure is assumed and ff is an unconstrained smooth vector field and HϕH_{\phi} is a first integral of it.

Rotational dynamics Apart from decorrelated dimensions, we must explicitly encourage circular trajectories in latent space. This is achieved through a penalty term:

Lcircle=Ei[Var⁡t(ri(t))]Ei,t[ri(t)2]+ε,ri(t)=∥φ(zi(t))∥,(2)\mathcal{L}_{\mathrm{circle}} = \frac{\mathbb{E}_{i}[\operatorname{Var}_{t}(r_i(t))]}{\mathbb{E}_{i,t}[r_i(t)^2] + \varepsilon}, \qquad r_i(t) = \lVert\varphi(z_i(t)) \rVert, \tag*{(2)}

which penalises the temporal variance of the radius ∥φ(zi(t))∥\lVert\varphi(z_i(t))\rVert along each trajectory. We normalise by the squared radius to avoid trivially shrinking rr to 0. This term vanishes exactly when every orbit is an origin-centred circle of constant radius, so periodic motion acquires an action-like radial coordinate and rotations become linearly representable. To further encourage action-angle representations, we include a speed penalty:

Lspeed=Ei[Var⁡t(si(t))]Ei,t[si(t)2]+ε,si(t)=∥Tf(z~i(t))∥,(3)\mathcal{L}_{\mathrm{speed}} = \frac{\mathbb{E}_{i}[\operatorname{Var}_{t}(s_i(t))]}{\mathbb{E}_{i,t}[s_i(t)^2] + \varepsilon}, \qquad s_i(t) = \lVert\mathbf{T}f(\widetilde{z}_i(t)) \rVert, \tag*{(3)}

applying the same scale-free regularisation to the pushforward field magnitude ∥Tf(zi(t))∥\lVert\mathbf{T}f(z_i(t))\rVert1, demanding each orbit be traversed at uniform speed on a constant-radius orbit, i.e. at uniform angular velocity. This penalty encourages the true action-angle representation and thus allows for the discovery of linear Lie point symmetries.

Symmetry discovery We construct an over-complete bank of KK linear infinitesimal generators {Ak}\{A_k\}, where the generators are fitted by minimising the Lie bracket loss:

Lb=Ez∼data1K∑k=1K∥Jf(z)Akz−Akf(z)∥2∥f(z)∥2+ε(4)\mathcal{L}_{b} = \mathbb{E}_{z\sim\mathrm{data}} \frac{1}{K}\sum_{k=1}^{K}\frac{\left\lVert\mathbf{J}_{f}(z)\mathbf{A}_{k}z-\mathbf{A}_{k}f(z)\right\rVert^{2}}{\left\lVert f(z)\right\rVert^{2}+\varepsilon} \tag*{(4)}

evaluated by forward-mode differentiation using a Jacobian-vector product (jax.jvp) to avoid a full Jacobian computation. We sample zz from encoded real trajectories with a small jitter. Generator redundancy is suppressed by decorrelating the actions Akz\mathbf{A}_{k}z in the data metric. The number of surviving generators after a multiplicative-gap test on the norm spectrum (see Appendix C) is the detected algebra dimension, and closure is then checked by projecting pairwise brackets back onto the surviving span.

Independent generators Training several candidate generators is prone to failures: Without decorrelation pressure they all collapse onto the single easiest symmetry, while a naive decorrelation pressure prevents convergence altogether. We adapt the stop-gradient orthogonalisation of Markov and Hvatov [9], where each pair penalty anchors the earlier generator of the pair, so generator 1 settles freely, generator 2 arranges itself to differ from a fixed generator 1, and so on. We measure overlap as the squared correlation of the generator actions (Aiz\mathbf{A}_{i}z) on the data manifold:

L⊥=∑i<j(Ez⟨sg⁡(Aiz),Ajz⟩)2sg⁡[Ez∥Aiz∥2]sg⁡[Ez∥Ajz∥2]+ε(5)\mathcal{L}_{\perp} = \sum_{i<j}\frac{\left(\mathbb{E}_{z}\langle\operatorname{sg}(\mathbf{A}_{i}z),\mathbf{A}_{j}z\rangle\right)^{2}}{\operatorname{sg}\left[\mathbb{E}_{z}\left\lVert\mathbf{A}_{i}z\right\rVert^{2}\right]\operatorname{sg}\left[\mathbb{E}_{z}\left\lVert\mathbf{A}_{j}z\right\rVert^{2}\right]+\varepsilon} \tag*{(5)}

With latent dimension above the true effective dimension, overlap can be cancelled by matrix components acting on dead dimensions, letting duplicate generators hide. The normalisation and the samples zz are also stop-gradiented, so the pressure only steers generator directions and cannot be satisfied by inflating norms or bending the coordinates.

Joint objective All of the symmetry terms above would be trivially minimised by Ak=0\mathbf{A}_{k}=0. We introduce a norm budget to prevent that while still allowing redundant generators to go to 0:

Lbudget=(∑k=1KEz∥Akz∥2−K)2+γ1K∑k=1K(Ez∥Akz∥2−1)2(6)\mathcal{L}_{\mathrm{budget}} = \left(\sum_{k=1}^{K}\mathbb{E}_{z}\left\lVert\mathbf{A}_{k}z\right\rVert^{2}-K\right)^{2}+\gamma\frac{1}{K}\sum_{k=1}^{K}\left(\mathbb{E}_{z}\left\lVert\mathbf{A}_{k}z\right\rVert^{2}-1\right)^{2} \tag*{(6)}

where KK is the bank size. The first term fixes only the total on-manifold norm; the second (weight γ=0.05\gamma=0.05) is a gentle per-generator attraction toward unit norm, preventing a winner-take-all allocation when several genuine symmetries coexist. Norms are measured through the data as Ez∥Akz∥2\mathbb{E}_{z}\left\lVert\mathbf{A}_{k}z\right\rVert^{2}, so a generator acting only off-manifold counts as collapsed. The full symmetry objective

LLie=λ1Lb+λ2L⊥+λ3Lcircle+λ4Lspeed+λ5Lbudget+λ6LHA(7)\mathcal{L}_{\mathrm{Lie}}=\lambda_{1}\mathcal{L}_{b}+\lambda_{2}\mathcal{L}_{\perp}+\lambda_{3}\mathcal{L}_{\mathrm{circle}}+\lambda_{4}\mathcal{L}_{\mathrm{speed}}+\lambda_{5}\mathcal{L}_{\mathrm{budget}}+\lambda_{6}\mathcal{L}_{\mathrm{HA}} \tag*{(7)}

is added to the reconstruction and latent-consistency losses of the dynamics model and optimised jointly in the main training phase; LHA\mathcal{L}_{\mathrm{HA}} is the Noether-compatibility pressure defined in Equation (8) below. The symmetry weights sit well below the fit weights and ramp in over training: at equal weighting, a small gain in bracket residual can buy a large reconstruction degradation, after which every “symmetry” found is a symmetry of model error. Per-system weights λi\lambda_i are given in Appendix D.

Discovering conservation laws Independent first-integral networks HϕH_\phi are learned alongside the generators. A quantity is conserved exactly when its gradient is perpendicular to the motion, and a symmetry is Noether-compatible (i.e. corresponds to a conservation law) when it preserves that quantity. This gives two perpendicularity constraints of the same form, with cos⁡2(a,b)=⟨a,b⟩2/(∥a∥2∥b∥2+ε)\cos^2(a,b)=\langle a,b\rangle^2/(\lVert a\rVert^2\lVert b\rVert^2+\varepsilon):

LH=Ezcos⁡2(∇Hϕ(z),f(z)),LHA=Ezcos⁡2(sg⁡[∇Hϕ(z)],Az).(8)\mathcal{L}_H = \mathbb{E}_z\cos^2(\nabla H_\phi(z), f(z)), \qquad\mathcal{L}_{HA} = \mathbb{E}_z\cos^2(\operatorname{sg}[\nabla H_\phi(z)], Az). \tag*{(8)}

When a system carries several independent invariants, the networks (multiple heads) additionally receive a decorrelation pressure LH⊥=∑a<bEzcos⁡2(sg⁡[∇Ha(z)],∇Hb(z))\mathcal{L}_{H\perp}=\sum_{a<b}\mathbb{E}_z\cos^2(\operatorname{sg}[\nabla H_a(z)],\nabla H_b(z)), i.e. gradients of distinct heads are pushed away from each other on the data manifold, with the stop-gradient anchoring the earlier head as in L⊥\mathcal{L}_\perp, so the two factor energies of the twin pendulums are recovered by separate networks rather than collapsing onto one. The left term trains HH to be constant along the flow2. Because the cosine is scale-free, LH\mathcal{L}_H is also minimised by the degenerate solution ∇H≡0\nabla H\equiv0, we add a unit-gradient penalty Ez(∥∇H(z)∥−1)2\mathbb{E}_z(\lVert\nabla H(z)\rVert-1)^2 to disallow this solution. The right term steers the generators, the stop-gradient sg⁡[⋅]\operatorname{sg}[\cdot] freezes HH, so the penalty pushes AA toward transformations that slide states along level sets of HH rather than across them. Both terms are active during the main phase, so HϕH_\phi co-trains with the dynamics and the bank. Because a fraction of initialisations stall in a cos⁡2\cos^2 plateau, HϕH_\phi is over-complete (several independent networks); after the main phase the networks are re-fitted against the frozen ff, {Ak}\{A_k\}, and coordinate chart by minimising the conservation terms alone, and the best network is kept. We then calibrate the discovered invariant to physical units by a monotone fit on validation data (see Figure 3a).

Figure 3. The discovered invariant, calibrated to energy on validation windows (a), tracks the true energy of held-out trajectories over 10×10\times the training span (b), and yields a falsifiable conservation verdict (c): fresh pendulum data stays within the ±3×\pm3\times noise-floor band (1.6×1.6\times the floor), while a damped control from the same initial conditions is correctly rejected at 13×13\times the noise floor.

Results

Circular dynamics Figure 2 shows the latent trajectories of four of our test systems. The colourbar can be considered to represent the unit-less energy of each trajectory. We see that the autoencoder is able to create almost entirely circular trajectories even for highly nonlinear systems (twin pendulums) or pure pixel space observations (pixel pendulum).

Figure 2. Latent phase portrait of the harmonic oscillator, Lotka-Volterra, twin pendulum, and pixel-pendulum systems. Note that if the reduced latent dimension is >2> 2, a PCA projection is used to convert the nn-dimensional trajectory onto a 2D map (as in the twin pendulum).

Symmetry discovery Across all six systems, our LATENTLIEODE method recovers the expected symmetry (see Table 2). The discovered invariant for each system also matches the ground-truth conserved quantity (correlation coefficient ∣ρ∣\lvert\rho\rvert in Table 2) almost perfectly. Wherever dimension discovery is enabled, we see successful reduction of the latent dimensions to the lowest true dimensionality of each respective system3, and we find the true

Systemobs. dimrankexpecteddiscovered symmetries∣ρ∣|\rho|calib. R2R^2
Harmonic oscillator222rotation + dilation0.9990.999
Pendulum (σ\sigma=5%)211phase rotation0.9980.997
Lotka–Volterra211phase rotation0.9910.983
Twin pendulums422two phase rotations0.9780.963
Sphere geodesics633so(3)\mathfrak{so}(3) rotations0.9970.996
Pixel pendulum2×10242 \times1024411phase rotation0.9950.996

Table 2. Symmetry discovery across the test suite. Rank counts the survivors of the generator-norm spectrum (i.e. the number of discovered symmetries). Expected is the true number of symmetries. ∣ρ∣|\rho| is the absolute Spearman rank correlation between the discovered invariant network HϕH_\phi and the ground-truth invariant, both evaluated on held-out validation windows. Calib. R2R^2 is the variance of the ground-truth invariant explained by g(Hϕ)g(H_\phi), where gg is a monotone map fitted on validation windows (Figure 3a).

number of symmetries for each system (rank) despite never specifying the desired amount a priori.

Transferring the discovered invariant The discovered HH, its coordinate chart, and its calibration together define a standalone function on raw observations, so conservation can be tested on data the model has never seen. Figure 3a shows the monotone calibration map fitted on validation windows. Since the cos⁡2\cos^2 objective is invariant under reparameterisation of HH, this monotone fit converts the invariant to physical units. On unseen pendulum trajectories, the calibrated invariant tracks the true energy (∣ρ∣=0.998|\rho| = 0.998 from Table 2) and remains conserved for over ten times the training data period (Figure 3b). Importantly, we can test this learned invariant on systems that do not conserve energy, e.g. a damped pendulum (Figure 3c). In this case, we see a consistent drop in HH along the observed trajectories, indicating a clear violation of energy conservation for the system.

Pixels to physics For the pixel pendulum, the model receives only 32 × 32 pixel frames with no state supervision. The initial dimensionality reduction prunes the latent space from 8 dimensions to 2, where we then run the main symmetry discovery step. From Table 2 we can see LᴇɴᴛLɪᴇOᴅᴇ finds one symmetry of this system, with the discovered invariant tracking the true pendulum energy at ∣ρ∣=0.995\lvert\rho\rvert= 0.995 (calibrated R2=0.996R^2 = 0.996; energy error 1.7%, and 1.6% drift on fresh trajectories). Figure 4 shows the resulting model rolled out and decoded into observation space from a single context window to twice the training span. We see strong agreement between the true and reconstructed dynamics far into the extrapolated regime.

Figure 4. Pixel-pendulum reconstruction. Ground-truth frames (top) vs. frames decoded from the LatentLieODE rollout (middle) and residuals (truth minus decoded, bottom), continued past the end of the training data. All temporal structure lives in the 2D latent ODE with the decoder rendering each latent state independently.

Non-abelian symmetry groups For all preceding systems the discovered algebras are abelian, so closure is trivial. As a non-abelian test, we simulate free-particle geodesics on S2S^2, specifically, state observations (x,v)∈R6(x,v) \in\mathbb{R}^6, with particle speeds drawn from ∣v∣∼Uniform⁡(0.5,1.5)\lvert v\rvert\sim\operatorname{Uniform}(0.5,1.5) in units of a sphere radius 1. The true algebra of this system is so(3)\mathfrak{so}(3). We recover exactly three symmetries from three surviving generators. This system is visualised in Figure 5, where we show trajectories of increasing particle speeds from left to right.

Figure 5. Ground truth (gray), decoded latent-ODE rollout (blue), and pure group transport esAz0e^{sA}z_0 (orange, dashed; no integration, s=ts=t) for four held-out orbits. The transport generator is the per-orbit projection of the flow onto the discovered so(3)\mathfrak{so}(3) span.

Projecting the pairwise commutators of the three surviving generators back onto their span recovers the bracket table

[A1,A2]=1.82A3,[A2,A3]=1.72A1,[A3,A1]=1.65A2.[\mathbf{A}_1,\mathbf{A}_2]=1.82\mathbf{A}_3,\qquad[\mathbf{A}_2,\mathbf{A}_3]=1.72\mathbf{A}_1,\qquad[\mathbf{A}_3,\mathbf{A}_1]=1.65\mathbf{A}_2.

The Killing form assembled from these structure constants, Kij=∑m,ncinmcjmnK_{ij}=\sum_{m,n}c^{m}_{in}c^{n}_{jm}, has eigenvalues (−6.25,−5.98,−5.66)(-6.25,-5.98,-5.66), hence is negative definite. Definiteness is invariant under change of basis, and among three-dimensional real Lie algebras a negative-definite Killing form identifies so(3)\mathfrak{so}(3) uniquely. As a consistency check, an orthonormal so(3)\mathfrak{so}(3) basis scaled by λ\lambda gives K=−2λ2IK=-2\lambda^2 I, and the observed eigenvalues imply λ≈1.7\lambda\approx1.7, closely matching the recovered structure-constant magnitudes of 1.82, 1.72 and 1.65 (see Appendix § 7 for more details). From the run’s stored per-pair projection residuals and commutator norms, the off-span fractions ∣P⊥[Ai,Aj]∣F/∣[Ai,Aj]∣F\lvert P_{\perp}[\mathbf{A}_i,\mathbf{A}_j]\rvert_F/\lvert[\mathbf{A}_i,\mathbf{A}_j]\rvert_F are:

(A1,A2):1.8%,(A1,A3):1.6%,(A2,A3):2.2%(\mathbf{A}_1,\mathbf{A}_2):1.8\%,\qquad(\mathbf{A}_1,\mathbf{A}_3):1.6\%,\qquad(\mathbf{A}_2,\mathbf{A}_3):2.2\%

i.e., ≥97.8%\geq97.8\% of every commutator lies inside the surviving span.

Integration-free dynamics Figure 5 shows trajectories from the integration of ff as well as the symmetry-evolved dynamics for the particles on a sphere. In both cases, initial states z0z_0 are encoded from held-out trajectory windows, but instead of ordinarily integrating the ODE (blue curves), the dashed orange curves apply the discovered generator via the exponential map esAz0e^{sA}z_0. These dynamics are purely generated by increasing the values of ss. The close agreement between all three curves is further evidence that we have indeed discovered valid symmetries for this physical system.

Discussion

We showed that a latent ODE equipped with a decorrelated coordinate chart, action-angle-encouraging penalties, and a differentiable bracket residual discovers linear Lie symmetries, their algebra dimension, structure constants, and calibrated first integrals from state timeseries and from video frames. We find strong agreement between the rollout of the latent trajectories whether using direct numerical integration of the ODE function ff or via sweeping the action parameter ss of the exponential map to create the temporal dynamics. We have thus demonstrated significant advances in data-driven discovery of symmetries and provide a new pathway for neural surrogates of dynamical systems.

Limitations. The method returns linear generators in a learned chart, so a symmetry is found only if some chart at the chosen latent width linearises it. The circle and speed priors presume (quasi-)periodic orbits, so chaotic regimes are out of scope, and results are single training runs checked by scripted acceptance gates rather than seed ensembles.

Cost and scaling

The symmetry machinery scales with the latent width dd, not the observation dimension DD. The bracket residual costs one Jacobian-vector product per sample and generator, an extra forward pass of ff with no Jacobian materialised, and the chart is a d×dd \times d batch statistic. Figure 6 shows this and we see step time is mostly flat across D=2→6D = 2 \to6, with the pixel-pendulum at D=2048D = 2048 costing roughly 4 times more than the other systems.

Figure 6. Training step time against encoder input dimension, single CPU.

Sphere geodesic Killing form

This section derives the consistency check quoted in the main text, the eigenvalues of the Killing form and the magnitudes of the recovered structure constants.

Throughout, indices i,j,k,m,ni, j, k, m, n run over the three surviving generators A1,A2,A3\mathbf{A}_1, \mathbf{A}_2, \mathbf{A}_3. The structure constants cijkc_{ij}^{k} are the projection coefficients of the pairwise commutators onto the surviving span,

[Ai,Aj]=∑kcijkAk,(9)[\mathbf{A}_i, \mathbf{A}_j] = \sum_k c_{ij}^{k} \mathbf{A}_k, \tag*{(9)}

and the Killing form is the symmetric 3×33 \times3 matrix built from them:

Kij=∑m,ncinmcjmn=tr⁡(ad⁡iad⁡j),(ad⁡i)jk=cijk.(10)K_{ij} = \sum_{m,n} c_{in}^{m}c_{jm}^{n} = \operatorname{tr}(\operatorname{ad}_i\operatorname{ad}_j), \qquad(\operatorname{ad}_i)_{j}^{k} = c_{ij}^{k}. \tag*{(10)}

KK depends only on the structure constants.

Baseline: the orthonormal so(3)\mathfrak{so}(3) basis. For an orthonormal basis X1,X2,X3X_1, X_2, X_3 of so(3)\mathfrak{so}(3) the structure constants are given by,

[Xi,Xj]=∑kεijkXk,ε123=ε231=ε312=+1,ε213=ε132=ε321=−1,(11)[X_i, X_j] = \sum_k \varepsilon_{ijk} X_k, \qquad\varepsilon_{123} = \varepsilon_{231} = \varepsilon_{312} = +1, \qquad\varepsilon_{213} = \varepsilon_{132} = \varepsilon_{321} = -1, \tag*{(11)}

(all other components zero). Substituting cijk=εijkc_{ij}^{k} = \varepsilon_{ijk} into Equation (10) and using the contraction identity ∑m,nεinmεjmn=−2δij\sum_{m,n} \varepsilon_{inm}\varepsilon_{jmn} = -2\delta_{ij} (where δij\delta_{ij} is the Kronecker delta) gives

K=−2I.(12)K = -2I. \tag*{(12)}

A uniformly scaled basis. The basis returned by rank detection is arbitrary in scale. Suppose that every generator carries the same unknown scale λ>0\lambda> 0,

Ai=λXi.(13)\mathbf{A}_i = \lambda\mathbf{X}_i. \tag*{(13)}

The commutator is bilinear, so

[Ai,Aj]=λ2[Xi,Xj]=λ2∑kεijkXk=λ∑kεijkAk,(14)[\mathbf{A}_i, \mathbf{A}_j] = \lambda^2 [\mathbf{X}_i, \mathbf{X}_j] = \lambda^2 \sum_k \varepsilon_{ijk} \mathbf{X}_k = \lambda\sum_k \varepsilon_{ijk} \mathbf{A}_k, \tag*{(14)}

i.e. the structure constants of the scaled basis are cijk=λεijkc_{ij}^{k} = \lambda\varepsilon_{ijk}. Since Equation (10) is quadratic in the structure constants,

K=−2λ2I.(15)K = -2\lambda^2 I. \tag*{(15)}

Each eigenvalue κ\kappa of the measured Killing form therefore yields an estimate of the basis scale:

λ=−κ/2.(16)\lambda= \sqrt{-\kappa/2}. \tag*{(16)}

The three measured eigenvalues, computed from the recovered structure constants of the sphere-geodesic run, are κ=(−6.25,−5.98,−5.66)\kappa= (-6.25, -5.98, -5.66), giving

λ=(6.25/2,5.98/2,5.66/2)=(1.77,1.73,1.68).(17)\lambda= \left(\sqrt{6.25/2}, \sqrt{5.98/2}, \sqrt{5.66/2}\right) = (1.77, 1.73, 1.68). \tag*{(17)}

Generator norm ranking

To determine the number of symmetries of a certain system, post training we rank each infinitesimal generator Ak\mathbf{A}_k by its norm. We show an example of this in Figure 7 for the sphere geodesic test, where we can see we have 3 surviving generators, corresponding to 3 independent symmetries discovered. We show the generator plots for all the test cases in Figure 8.

Figure 7. Generator norm spectrum for the sphere-geodesic system of K=8K = 8 deliberately over-complete generators, three survive and five collapse, leaving a 263×263\times gap. The detected algebra dimension matches the expected rank of 3 for the so(3)\mathfrak{so}(3) group.

Generator norm spectra for the six systems

Figure 8. Generator norm spectra for the six systems of Table 2, in the same form as Figure 7

Experimental details

Latent ODE architecture The encoder is a reverse-time GRU over [t,x][t, x] context windows, followed by a small GELU MLP head producing the latent initial condition z0z_0. The vector field is f(z)=σs⋅MLP⁡(z)f(z) = \sigma_s \cdot\operatorname{MLP}(z) with softplus activations and tanh output using smooth activations only, so the Jacobian in the bracket residual is differentiable everywhere. We note σs\sigma_s is a learned scale parameter. The decoder is an MLP applied pointwise to each latent state. For pixel observations the encoder and decoder are small convolutional stacks (channels 16/32/64) over 32×3232 \times32 frames. Latent rollouts use diffrax (Tsit5, adaptive PID controller, rtol = atol = 10−510^{-5}). We train with AdamW (weight decay 10−410^{-4}, gradient clip 1.0) with a cosine warmup schedule, with a separate Adam optimiser for the generator bank at learning rate ηgen\eta_{\mathrm{gen}}. Training runs a dimension-discovery (reduction) warmup, then joint co-training in which the symmetry weights follow a quadratic ramp reaching full strength at 75% of the run. Reconstruction is trained over an increasing horizon of the training data, and the Lie bracket samples are drawn from encoded trajectories plus Gaussian jitter at 5% of the latent spread. The Latent ODE architecture is identical across all datasets.

Harmonic oscillator Closed-form trajectories of q˙=p\dot{q} = p, p˙=−q\dot{p} = -q with radii drawn uniformly from [0.4,1.4][0.4, 1.4] and uniform phases; 1024 training and 128 validation trajectories of 50 samples over t∈[0,8]t \in[0, 8], observation noise σ=0.01\sigma= 0.01. Dimension discovery prunes the latent space 6→26 \to2.

Simple pendulum θ¨=−(g/L)sin⁡θ\ddot{\theta} = -(g/L)\sin\theta with g/L=1g/L = 1, integrated numerically; amplitudes drawn uniformly from [0.5,2.4][0.5, 2.4] radians with 60 samples over t∈[0,12]t \in[0, 12], observation noise σ=0.05\sigma= 0.05. Dimension discovery prunes 6→26 \to2.

Twin pendulums Two uncoupled pendulums with g/L=1g/L = 1 and 2 observed jointly as a 4-dimensional state, amplitudes drawn as above, σ=0.02\sigma= 0.02. The latent width is fixed at 4.

Lotka–Volterra x˙=x−xy\dot{x}=x-xy, y˙=xy−y\dot{y}=xy-y (all rates unity); initial conditions sampled inside the box [0.25,3.2]2[0.25,3.2]^2 filtered to invariant values in [2.15,3.0][2.15,3.0]; 70 samples over t∈[0,14]t\in[0,14], σ=0.01\sigma=0.01. Dimension discovery prunes 20→220\to2.

Pixel pendulum The pendulum simulator rendered as 32×3232\times32 frames (rod and bob, amplitudes uniform in [0.5,2.0][0.5,2.0], pixel noise σ=0.05\sigma=0.05), with a temporal-difference channel appended so the state is per-frame observable. Training is staged. Stage 1 (perception only, all symmetry weights zero): convolutional codec with an autoencoder warm-start of 5000 steps, variance-weighted reconstruction, latent width 8, 45,000 steps at learning rate 3×10−33\times10^{-3}. Stage 2: the full symmetry pipeline on the frozen stage-1 encoder’s decorrelated latent trajectories (observation dimension 8); dimension discovery prunes 8→28\to2, and a metric-anneal curriculum (held for the first 10% of training, annealed to the plain metric by 40%) locks the single-cover chart; 80,000 steps.

Sphere geodesic Closed-form circle geodesics of a free particle on the unit sphere, observed as the state (x,v)∈R6(x,v)\in\mathbb{R}^{6}; initial positions uniform on S2S^{2}, tangent directions uniform, speeds ∣v∣∼Uniform⁡(0.5,1.5)|v|\sim\operatorname{Uniform}(0.5,1.5); 3000 training trajectories of 125 samples over t∈[0,25]t\in[0,25], σ=0.01\sigma=0.01. The latent width is fixed at 6 with the sparsity weight zero: the linear so(3)\mathfrak{so}(3) realisation lives in R6\mathbb{R}^{6} while the data manifold is only 4-dimensional, so the embedding slack is required. After co-training, a field-only polish phase freezes everything except the vector field and fine-tunes full-horizon reconstruction for 25,000 steps at learning rate 3×10−43\times10^{-4}, sharpening the frequency profile without touching the chart or the generators.

HarmonicPendulumLotka–VolterraTwin pend.Pixel (stage 2)Sphere
trajectories (train/val)1024/1281024/1281024/1281024/1281024/323000/128
samples × span50 × 860 × 1270 × 1460 × 12—125 × 25
noise σ\sigma0.010.050.010.02—0.01
obs. dim222486
latent dim (found)6 → 26 → 220 → 248 → 26
generators KK345648
HH heads444444
context length403235321240
batch size161616161632
lr / lrgen_{\mathrm{gen}}10−2/3⋅10−210^{-2}/3\cdot10^{-2}10−2/3⋅10−210^{-2}/3\cdot10^{-2}10−2/3⋅10−210^{-2}/3\cdot10^{-2}10−2/3⋅10−210^{-2}/3\cdot10^{-2}5⋅10−3/10−25\cdot10^{-3}/10^{-2}5⋅10−3/10−25\cdot10^{-3}/10^{-2}
co-train / polish steps25k/5k25k/5k25k/5k40k/5k80k/6k100k/10k
λbracket\lambda_{\mathrm{bracket}}0.10.20.10.10.10.3
λ⊥\lambda_{\perp}0.10.20.10.10.10.1
λcircle\lambda_{\mathrm{circle}}1.02.01.01.01.01.0
λspeed\lambda_{\mathrm{speed}}1.02.01.01.01.01.0
λbudget\lambda_{\mathrm{budget}}0.050.10.10.10.10.1
λHA\lambda_{HA}0.10.20.10.10.10.1
λH\lambda_H (+ grad. norm)1.01.00.20.20.20.2
λsparsity\lambda_{\mathrm{sparsity}}0.050.150.05–0.05–
λH⊥\lambda_{H\perp}–––0.5––

Table 3. Per-system hyperparameters. Loss weights are the λi\lambda_i of the joint objective; reconstruction and latent-consistency weights are 1 throughout. Pixel stage 1 trains perception only (all symmetry weights zero) and is omitted; its settings appear in the text.

References

  1. [1]Ferran Alet, Dylan Doblar, Allan Zhou, Josh Tenenbaum, Kenji Kawaguchi, and Chelsea Finn. Noether networks: meta-learning useful conserved quantities. Advances in Neural Information Processing Systems, 34:16384–16397, 2021.arxiv.org/abs/2112.03321
  2. [2]Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine, 34(4):18–42, 2017.DOI
  3. [3]Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.arxiv.org/abs/1806.07366
  4. [4]Taco Cohen and Max Welling. Group equivariant convolutional networks. In International conference on machine learning, pages 2990–2999. PMLR, 2016.arxiv.org/abs/1602.07576
  5. [5]Miles Cranmer, Sam Greydanus, Stephan Hoyer, Peter Battaglia, David Spergel, and Shirley Ho. Lagrangian neural networks. In ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equations, 2020.arxiv.org/abs/2003.04630
  6. [6]Krish Desai, Benjamin Nachman, and Jesse Thaler. Symmetry discovery with deep learning. Physical Review D, 105(9):096031, 2022.
  7. [7]Samuel Greydanus, Misko Dzamba, and Jason Yosinski. Hamiltonian neural networks. Advances in neural information processing systems, 32, 2019.arxiv.org/abs/1906.01563
  8. [8]Ziming Liu and Max Tegmark. Machine learning conservation laws from trajectories. Physical Review Letters, 126(18):180604, 2021.DOI
  9. [9]Ilya Markov and Alexander Hvatov. Data-driven discovery of lie algebra generators via fréchet-invariant learning. In Conference of Mathematics of AI, 2026. URL https://openreview.net/forum?id=UNJ2Z8pvxH.
  10. [10]E. Noether. Invariante variationsprobleme. Nachrichten von der Gesellschaft der Wissenschaften zu Göttingen, Mathematisch-Physikalische Klasse, 1918:235–257, 1918. URL http://eudml.org/doc/59024.
  11. [11]Yulia Rubanova, Ricky TQ Chen, and David K Duvenaud. Latent ordinary differential equations for irregularly-sampled time series. Advances in neural information processing systems, 32, 2019.
  12. [12]Matt L Sampson and Peter Melchior. Path-minimizing latent ODEs for improved extrapolation and inference. Machine Learning: Science and Technology, 6(2):025047, June 2025.arxiv.org/abs/2410.08923
  13. [13]Peter Toth, Danilo J Rezende, Andrew Jaegle, Sébastien Racanière, Aleksandar Botev, and Irina Higgins. Hamiltonian generative networks. In International Conference on Learning Representations, 2020.arxiv.org/abs/1909.13789
  14. [14]Soledad Villar, David W Hogg, Kate Storey-Fisher, Weichi Yao, and Ben Blum-Smith. Scalars are universal: Equivariant machine learning, structured like classical physics. Advances in neural information processing systems, 34:28848–28863, 2021.
  15. [15]Jianke Yang, Robin Walters, Nima Dehmamy, and Rose Yu. Generative adversarial symmetry discovery. In International conference on machine learning, pages 39488–39508. PMLR, 2023.arxiv.org/abs/2302.00236
  16. [16]Jianke Yang, Nima Dehmamy, Robin Walters, and Rose Yu. Latent space symmetry discovery. In Proceedings of the 41st International Conference on Machine Learning, 2024.arxiv.org/abs/2310.00105

Paper details

vlnc:W_n7jp2guu
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 9

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents