WorksComputer Science
Symmetry Breaking at the Edge of Stability via Central Flow Analysis
Abstract
Continuous symmetries in neural networks lead to conserved quantities that are invariant under gradient flow, such as layer balancedness in linear and ReLU networks. Finite-step gradient descent need not preserve these laws, particularly at the edge of stability (EOS), where the iterates oscillate around a smoother trajectory, which can be described by the central flow dynamics (Cohen et al., 2025). We study how gradient flow conservation laws evolve under the central flow description of this regime. We show that the curvature correction induces a finite-dimensional drift determined by the oscillation covariance along critical parameter directions. We thus take the first steps to analyse which conservation laws remain and are broken at the EOS according to the central flow dynamics by analysing a scalar two-layer model, two-layer matrix factorisation, and a two-layer ReLU network, where the known conservation laws are shown to be complete (Marcotte et al., 2023).
Introduction
Continuous symmetries often leave traces in optimisation dynamics. Under gradient flow, architectural symmetries of neural networks give rise to conserved quantities. For example, Arora et al. [1] build on linear networks preserving the matrix-valued balancedness quantity , while positively homogeneous networks admit corresponding neuron-wise balancedness quantities Du et al. [5]. Marcotte et al. (2023) formalised this phenomenon intrinsically to the gradient flow. Writing the model through a reparametrisation , they characterise gradient flow conservation laws through orthogonality to the family of admissible data-generated gradient vector fields , making conservation laws solely architecture-dependent and showing that the maximum number of independent laws can be computed from the dimension of the model and the Lie algebra generated by , namely . For the shallow linear and ReLU architectures considered in this work, this machinery recovers the familiar balancedness quantities and certifies their completeness.
Finite-step gradient descent does not inherit this conservation guarantee. Symmetry-induced conservation laws are known to acquire finite-step corrections Kunin et al. (2021), and gradient descent on neural networks, with learning rate , commonly enters the EOS regime, where the largest Hessian eigenvalue (or a set of eigenvalues) approaches the stability threshold , and the iterates oscillate Cohen et al. [2]; ); Damian et al. [4]. Cohen et al. [3] propose the central flow as a continuous-time description of the time-averaged trajectory in this oscillatory regime. This raises the question we study: Which gradient flow conservation laws survive the central flow dynamics, and which are broken by edge-of-stability oscillations?
We take the first steps to answer this question for three increasingly structured two-layer examples: a scalar factorisation, matrix factorisation, and a ReLU network. In each case, the gradient flow conservation laws can be represented as (where of two matrices and denotes their Frobenius inner product)
a matrix-valued invariant , defined in § 2 against an architecture-dependent space of admissible test matrices . More generally, conservation law survival at a fixed is determined by the span of the symmetry-breaking directions reachable as the admissible oscillation covariance varies, whose dimension is bounded in (17); survival along a trajectory is characterised in (14).
Our viewpoint is complementary to recent analyses of symmetry breaking in the discrete dynamics. Kunin et al. (2021) study finite-step breaking through modified-equation analysis. Recent work at the edge of stability studies balancedness and implicit bias directly in discrete dynamics, including scalar or layer-aggregated conservation quantities Medeiros (2026); Hofmann [7], while Ghosh et al. [6] analyse balancing in deep matrix factorisation beyond the edge of stability. Here the object of study is instead the central flow dynamics: we identify which directions in that finite-dimensional conservation-law space are broken by the EOS curvature field. We then compare gradient flow, discrete gradient descent, and the central flow to illustrate how the predicted breaking emerges along central flow trajectories.
Background on symmetries and gradient flow conservation laws
We briefly recall the framework we use following Marcotte et al. (2023, 2024). Suppose the model admits a reparametrisation , so that, for a loss of the form ,
A function is therefore conserved by gradient flow whenever , since . The maximal number of locally independent laws, under the assumption that the loss spans the output space and that is locally constant, is .
The conservation laws used below arise from continuous invariances of the loss (2). Let be a one parameter transformation on an open subset of the model parameter space satisfying , . Its infinitesimal generator satisfies , and hence lies in , leaving the loss invariant. If, in addition, for some scalar potential , then is a gradient flow conservation law. Importantly, differentiating an invariance only implies that its generator lies in ; it does not require every element of this kernel to arise from the particular symmetry family considered below.
Two-layer matrix factorisation. Let , , , . For any , the transformation leaves unchanged, with generator . This vector field is a Euclidean gradient field precisely when . For such , define , . Direct differentiation gives . Thus the admissible test space for matrix factorisation gradient flow laws is . If has full rank , a complete set of locally independent laws has dimension , Marcotte et al. (2023).
Two-layer ReLU. Consider , with . Positive homogeneity gives the neuron-wise invariance . Equivalently, within the matrix family above the admissible continuous generators are, generically, . The corresponding independent conservation laws are , , or, for , . Hence , and . These laws form a complete set of independent gradient flow conservation laws for the two-layer ReLU architecture Marcotte et al. (2023).
The scalar model is the special case . Thus all three examples admit the common representation in Equation (1) but the admissible test space is architecture-dependent: , for the scalar, matrix-factorisation, and ReLU examples, respectively. § 3 shows that this distinction determines which component of the central flow perturbation can break the corresponding laws.
Symmetry breaking under the central flow
We now ask how the gradient flow conservation laws of § 2 evolve under the central flow. Cohen et al. [3] model the time-averaged trajectory of gradient descent at the edge of stability by
Here models the covariance of the fast oscillations. At the edge of stability its range is contained in the critical eigenspace
while the solution to the semidefinite complementarity problem (SDCP) selects the realised , pinning the active eigenvalues at . Away from the edge of stability, the selected covariance is , and (3) reduces to gradient flow. Writing an oscillatory displacement as , with and , the curvature contraction has the equivalent form
Thus records where the finite-step dynamics oscillate, while measures the curvature encountered by those oscillations.
The central flow conservation law drift. Let be a gradient flow conservation law. Since , contracting (3) with eliminates the gradient flow field:
The directional derivative in (6) is taken with held fixed. It is therefore not the total time derivative of , which would contain an additional term involving . For the laws of § 2, , so
Collapse to the factor dimension. Write . The Hessian decomposes as
For the squared losses considered below, for the MF and scalar cases as well as for the ReLU case as considered in Appendix D we get
In our two-layer examples, is invariant along the symmetry generator, so . For ReLU, this statement and the explicit Hessian calculation below are local to a region in which the activation pattern is fixed. Because is bilinear in the two factors, (9) reduces to contractions. Writing
differentiation along yields a linear functional of . For all three examples this functional is in the form of the raw symmetry-breaking matrix
where , denote the two orderings of the cross contraction. Their explicit forms, and the corresponding ReLU substitutions, are given in the examples in Apps. B–D. Importantly, every term in (11) is linear in . The raw matrix need not belong to the architecture’s admissible test space . We therefore define the canonical symmetry-breaking matrix by , where is the orthogonal projection onto . Equation (7) then becomes
Criterion for survival of the conservation laws is given in Appendix § 5. The three examples differ in their admissible test space and in the low-dimensional contractions entering the raw breaking matrix . Table 1 summarises the resulting conservation laws and observable symmetry-breaking matrices; full derivations and empirical results are given in Apps. B–D.
| model | gradient flow laws | ||
| scalar | |||
| matrix factorisation | |||
| two-layer ReLU |
Table 1. The three instances of the symmetry-breaking construction. Here . The raw matrix is projected onto the architecture-dependent admissible test space before determining survival.
Conclusions and outlook
In this work we show how the central flow dynamics can break conservation laws that hold on gradient flows in scalar networks, matrix factorisation and two-layer ReLU networks. We identify a symmetry-breaking drift matrix, laying the ground for future analysis of more complex architectures and EOS conservation laws. The limitations of this work, that future work will address, include actual counting and completeness arguments, to identify the number of independent conservation laws that survive under central flow dynamics. Moreover, a formal distinction of uniform (for all admissible ) and trajectory-dependent laws should be elaborated, to answer the question of whether new trajectory-dependent laws emerge at the EOS under the central flow.
Survival of gradient flow conservation laws
For the realised central flow covariance , define
Equation (12) immediately gives
Hence the central flow breaks conservation laws geometrically: the realised matrices span the broken directions, and their orthogonal complement gives the surviving constant- conservation laws along this trajectory. It is also useful to characterise all breaking directions that can be generated by covariance perturbations supported on the current critical eigenspace. Since is linear in , we define
This is a linear extension over the span of supported covariance matrices; the actual central flow selects a through its SDCP solution. Since
the potential breaking space obeys
In particular, when , all covariance perturbations supported on the critical eigenspace produce at most one breaking direction. For a single realised , independently of , the instantaneous drift is always the single functional ; larger matters because it allows a larger set of breaking directions to be reached as the covariance changes.
Scalar factorisation
Consider and . Writing
we have
Hence is, up to functions of , the unique gradient-flow conservation law in this case.
Central-flow drift. The Hessian and covariance are
Holding fixed,
Contracting with , the cross-covariance cancels:
Therefore
Thus the scalar symmetry-breaking quantity is
One critical direction. Suppose the top Hessian eigenvalue is simple and the active critical eigenspace is one-dimensional. Then , where is the unit top eigenvector, is the variance of the oscillations along , and
and
Let
The largest eigenvalue is
Direct differentiation gives
and therefore
Consequently,
For and ,
with equality if and only if . This is an instantaneous statement during the active oscillatory phase; it does not imply that reaches zero before reaches zero.
Stable terminal values. On the solution manifold ,
Any stable solution-manifold endpoint satisfies , and hence
Thus the EOS dynamics can reduce the balancedness gap and then freeze at a generically nonzero value once the oscillatory phase terminates.
Empirical analysis. Figure 1 isolates the symmetry-breaking mechanism. Gradient flow remains on the level set , while both gradient descent (GD) and the central flow (CF) leave that level set once EOS effects become active. When reaching the EOS dynamics, the independently measured agrees with the analytic prediction. Since in these runs and , the curvature correction moves the two factors toward balance, consistent with the sign relation .

Figure 1. Scalar factorisation: central-flow breaking of the gradient-flow balancedness law. Results are shown over three random initialisations. Left: relative change of the gradient-flow invariant . Gradient flow preserves up to numerical integration error, whereas finite-step GD and the hybrid central flow produce an change. The GD parameter midpoint, , removes the dominant period-two oscillation before evaluating . Centre-left: pointwise test of the central-flow drift at the first EOS contact. For each seed, the directly measured agrees with the closed-form prediction of Eq. (18), with the dashed line denoting equality. Centre-right: normalised sharpness along hybrid central flow. Each trajectory follows ordinary gradient flow before EOS and, once the constraint becomes active, remains pinned at . Right: long-time value of at against the stability bound . The three runs lie on the identity line, showing empirically that the bound is saturated in these instances.
Two-layer matrix factorisation
Let
with squared loss
where denotes the Frobenius norm. The gradient-flow invariant is
with and .
Hessian contraction. For an oscillatory perturbation ,
Since is bilinear, the Hessian quadratic form splits as
Along the symmetry generator,
so is unchanged and therefore
Only the pullback term contributes to symmetry breaking. Define
Expanding the pullback quadratic form,
Hence
Differentiation along the symmetry. Holding fixed gives
Therefore
For the cross term,
so
Define
Then
and consequently
Combining the layer and cross terms gives
Thus
Projection onto the conservation-law space. Because ,
Hence the canonical symmetry-breaking matrix is
and the matrix-valued invariant obeys
The cross contribution is the expected commutator
It vanishes automatically at , but is generally nonzero for . Since a commutator need be neither symmetric nor skew-symmetric, itself need not lie in ; only is observable by the conservation laws.
At a realised time ,
while a constant- law survives along the whole trajectory precisely when
Finally, the sufficient conditions
give . The ordinary factor-balance condition alone does not imply the corresponding covariance balance or vanishing of the cross term.
Empirical analysis. Figure 2 shows that no nonzero constant test matrix remains orthogonal to the breaking direction for the entire trajectory although no additional critical Hessian directions appear (the second eigenvalue stays uniformly below the EOS threshold).

Figure 2. Matrix factorisation: the broken conservation-law direction over the EOS trajectory. We consider rank- matrix factorisation, for which the gradient-flow invariant contains independent scalar conservation laws. Results are shown over three random seeds. Top left: relative Frobenius displacement . Gradient flow preserves up to numerical error, whereas GD and hybrid central flow produce an change; the adjacent-iterate GD parameter midpoint removes the dominant period-two oscillation before evaluating . Top right: pointwise validation of , with . The relative discrepancy between the directly differentiated matrix invariant and the symmetry-breaking prediction is approximately – for all seeds. Bottom left: normalised singular values of the matrices , represented in Frobenius-orthonormal coordinates of and stacked over the active EOS trajectory. All three singular values are nonzero well above the numerical error floor, showing that the realised breaking direction spans all of over the observed trajectory. Bottom right: rank-one EOS diagnostic. The second Hessian eigenvalue remains below both and the displayed safety margin throughout the active trajectory, so the rank-one central-flow approximation remains well separated from the onset of a second critical direction.
Two-layer ReLU
Consider
with
The gradient-flow conservation laws are
or equivalently
Fixed activation pattern. For the explicit Hessian calculation, assume that the activation pattern is constant on the support of the oscillatory perturbations. For each example define
and
For , let
Within this fixed-gating region,
and the Hessian quadratic form is
For , the symmetry generator satisfies
Since commutes with each gate matrix ,
Thus the residual term is invariant and only the pullback term contributes. Define
together with
Expanding the pullback term gives
where
The Gram terms satisfy
so
For the cross term, define
Differentiating and using cyclicity gives
Hence
This is exactly the matrix-factorisation form under
Projection and survival. For ReLU the admissible test space is , so
In particular,
An individual conservation law therefore has zero instantaneous drift exactly when
For a general diagonal test matrix ,
Thus individual ’s may all drift while a nontrivial linear combination has zero instantaneous drift.
Across activation boundaries. The matrix formula above assumes fixed gates. An exact identity for the averaged dynamics does not require this assumption. Suppose
and let the linear generator satisfy the symmetry identity
wherever is differentiable. Since ,
Using the invariance identity at together with gives
Identity (25) is not specific to ReLU: it uses only that is linear, that , and that is invariant along at every point where it is differentiable, so it holds equally for the scalar and matrix factorisation examples. We state it here because, for ReLU, the loss is not smooth: when is smooth near , the central flow (3) is the second-order truncation of these averaged dynamics and (23) is the corresponding explicit drift, which requires the oscillation to stay within one activation region. Identity (25) allows the oscillation to cross activation boundaries between and , only limiting the sampled points to not lie exactly on a boundary. Nevertheless, there is a dependence on the full distribution of , whereas the central flow only looks at .
Empirical analysis. Figure 3 verifies the architecture-dependent projection underlying the ReLU result. The ReLU conservation laws only probe the diagonal component of the raw symmetry-breaking matrix. The directly measured agrees with . The agreement includes both positive and negative drifts, showing that the calculation predicts not only whether a law is broken but also its signed instantaneous motion. The explicit ReLU Hessian calculation is only valid while the oscillation remains inside a single activation region.

Figure 3. Two-layer ReLU: local symmetry breaking inside a fixed activation region. We consider a two-layer ReLU network with hidden units and evaluate the central-flow prediction at an active rank-one EOS point for three independent seeds. The admissible test space is , so only the diagonal projection of the raw breaking matrix is observable. Left: pointwise validation of the per-neuron drift , . Each marker corresponds to one neuron from one seed; the measured drift lies on the identity line against the prediction. Centre: verification of the fixed-gating assumption. For every seed we report , , where . All ratios exceed one, so neither side of the rank-one oscillation crosses a ReLU activation boundary. Right: instantaneous surviving combinations. Writing , we construct an orthonormal basis of , which has dimension whenever . The maximum measured absolute drift among these two combinations is for every seed.
Figure 4 shows what happens beyond the fixed-gate regime. When a pre-activation crosses zero, the Hessian changes discontinuously, and along the hybrid central-flow trajectories the rank-one oscillation violates the fixed-gate condition after fewer than 10 optimisation steps. Beyond that point (23) is no longer certified: the orange curves are obtained by recomputing the local Hessian in each new activation region. Gradient flow and centred GD are shown for reference. Characterising the central flow conservation laws across activation boundaries is left as a follow-up to this work.

Figure 4. Two-layer ReLU beyond fixed gates: piecewise continuation of the central flow. For each of three random seeds (rows), we track the three per-neuron gradient-flow laws (columns). Gradient flow (solid blue) preserves every , while finite-step GD breaks them. To remove the dominant period-two oscillation, the dashed green curve evaluates at the adjacent-iterate parameter midpoint . The orange curve is the hybrid central flow, continued piecewise by recomputing the local ReLU Hessian after activation-pattern changes. The vertical dotted line marks the first sampled point at which the rank-one oscillation fails the fixed-gate condition for some sample-neuron pair. Consequently, the explicit fixed-gate symmetry-breaking formula (23) is certified only to the left of this line; to the right, the orange curve is a heuristic piecewise continuation shown for illustration, with gradient flow and centred GD for reference.
Experimental settings
The git repository can be found at: https://github.com/naimaeb/CF_conserved_quantities.git and the experimental settings are given by Table 2.
| scalar | matrix fact. | ReLU (local) | ReLU (trajectory) | |
| dimensions | , | , , | , , | |
| data | rank-2 target | , | as local | |
| initialisation | , | , | , | as local |
| n seeds | 3 | 3 | 3 | 3 |
| 0.90, 0.95, 0.96 | 0.87, 0.12, 0.19 | 2.22, 0.69, 2.99 | 0.56, 1.62, 1.19 | |
| horizon (steps) | 200 | 50 | — | 80 |
| CF step | 0.02 | 0.1 | — | 0.1 |
| 1 | 3 | 3 | 3 |
Table 2. Configuration of the four experiments. ReLU targets are with , and the ReLU loss is a dataset mean. Note that when considering the dataset mean in the ReLU case, we need to scale the loss, the Hessian quadratic form, , , , and by .
References
- [1]Sanjeev Arora, Nadav Cohen, Noah Golowich, and Wei Hu. A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks. In International Conference on Learning Representations, 2019. URL https://mlanthology.org/iclr/2019/arora2019iclr-convergence/.arxiv.org/abs/1810.02281
- [2]Jeremy Cohen, Simran Kaur, Yuanzhi Li, J Zico Kolter, and Ameet Talwalkar. Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability. In International Conference on Learning Representations, 2021. URL https://mlanthology.org/iclr/2021/cohen2021iclr-gradient/.arxiv.org/abs/2103.00065
- [3]Jeremy Cohen, Alex Damian, Ameet Talwalkar, J Zico Kolter, and Jason D. Lee. Understanding optimization in deep learning with central flows. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=SI2rI3ZPs.arxiv.org/abs/2410.24206
- [4]Alex Damian, Eshaan Nichani, and Jason D. Lee. Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability. In International Conference on Learning Representations, 2023. URL https://mlanthology.org/iclr/2023/damian2023iclr-selfstabilization/.arxiv.org/abs/2209.15594
- [5]Simon S Du, Wei Hu, and Jason Lee. Algorithmic Regularization in Learning Deep Homogeneous Models: Layers Are Automatically Balanced. In Neural Information Processing Systems, pages 384–395, 2018. URL https://mlanthology.org/neurips/2018/du2018neurips-algorithmic/.arxiv.org/abs/1806.00900
- [6]Avrajit Ghosh, Soo Min Kwon, Rongrong Wang, Saiprasad Ravishankar, and Qing Qu. Learning Dynamics of Deep Matrix Factorization Beyond the Edge of Stability. In International Conference on Learning Representations, 2025. URL https://mlanthology.org/iclr/2025/ghosh2025iclr-learning/.arxiv.org/abs/2502.20531
- [7]Thomas Hofmann. The map behind the flow: Finite-step gradient descent as a dynamical system, 2026. URL https://arxiv.org/abs/2607.04993.