betaFeedback

WorksPhysics and Astronomy

Invariance can prevent overfitting and shortcut learning in Quantum Neural Networks

Abstract

Many image classification tasks have labels that are unchanged by rotations and reflections. Equivariant quantum neural networks incorporate such symmetries by design, but the specific benefit of symmetry is difficult to isolate. Equivariant quantum circuits are usually compared with architectures that also differ in gates, structure or size. Here, we investigate the role of invariance in a controlled setting, classifying images whose labels are invariant under the dihedral group D4D_4. We compare four eight-qubit models with identical encoding, circuit structure and number of trainable parameters, and equivalent dynamical Lie algebras, and benchmark invariant implementations against non-invariant ones in terms of generalization error, robustness to shortcut learning and resilience to a realistic noise model. On four binary tasks, including MNIST and two scientific data sets of galaxies and red blood cells, the invariant models trained on a few tens of images show better generalization than non-invariant ones. Invariance also makes the models immune to orientation shortcuts: when trained with a class-dependent corner watermark, the non-invariant models learn the watermark and often drop to chance level when it is moved. We show that the gain in accuracy of invariant models persists also under a realistic noise model.

Introduction

Variational quantum algorithms, in which a parametrized quantum circuit is trained by a classical optimizer, are a leading approach to machine learning on near-term quantum hardware [1, 2, 3]. Used as classifiers, these quantum neural networks (QNNs) encode classical data into quantum states, process them with a trainable unitary and read out the prediction from the expectation value of an observable [3, 4, 5, 6]. Two constraints limit their use. The size and noise of current devices limit the number of qubits, parameters and layers, and expressive circuits suffer from barren plateaus, in which the gradients vanish exponentially with the number of qubits [7, 8, 9, 10, 11]. As a result, QNNs are small and typically trained on small data sets, and whether they generalize from few examples is a central question for their use [12, 13, 14, 15, 16].

Symmetry offers a natural way to tailor a model to its task. In classical deep learning, architectures that respect the symmetries of the data yield models that are both expressive and data-efficient: convolutional networks are equivariant under translations, and group-equivariant networks extend this to rotations and reflections [17, 18]. Geometric quantum machine learning transfers this principle to QNNs [19, 20, 21, 22, 23, 24, 25, 26, 27]: if the circuit is equivariant with respect to a representation of the symmetry group and the measured observable is equivariant, the model output is invariant by construction. Two benefits are expected. First, restricting the model to invariant functions should improve generalization from few examples [28]. Second, restricting the generators of the circuit reduces its dynamical Lie algebra, which should mitigate barren plateaus [8, 20, 24, 29]. Both have been proven for permutation-equivariant QNNs [25], but for a small finite group, such as the symmetries of an image, the constraint is weaker, and whether these benefits appear in practice remains an open question. Here we address this question on image classification tasks, including real-world data sets, and find that invariance prevents overfitting and shortcut learning.

In image classification, the relevant geometric symmetries are the transformations under which the image-label association is preserved. For square images, reflections and rotations by multiples of 90° are described by the dihedral group D4D_4. Equivariant and approximately equivariant QNNs for this group and its subgroups have been proposed and shown to outperform non-equivariant models on image tasks [29, 30, 31, 32]. In these works, however, the equivariant model is usually compared with generic circuits that differ in their gates, entangling pattern or number of parameters. Ref. [31] partly addresses this gap with a hybrid model that shares the variational circuit of the equivariant one but uses a different encoding, so that the symmetry of the circuit no longer corresponds to a transformation of the images. A systematic comparison on several data sets, in which the circuit structure is fixed and the symmetry is varied in a controlled way, is still missing, and the effect of the symmetry itself remains unclear. Such a comparison must take into account two facts. First, circuits that differ in their gates also differ in properties that depend on their dynamical Lie algebra (DLA). These include the unitaries they can reach, the number of parameters beyond which they are overparametrized [33] and the size of their gradients [8, 9]: a difference in generalization may come from any of them. Second, an equivariant model combines two distinct ingredients: an equivariant circuit and an equivariant readout. Only together do they make the model invariant. An invariant model can also be obtained without an equivariant circuit, by averaging a generic model over the group (twirling) [21, 27]. In this work we ask which of these ingredients produces the benefit.

We isolate the role of the symmetry with four models that share the same circuit structure (Figure 1). Images are amplitude-encoded so that the group D4D_4 acts on the qubits by bit flips and by a swap of two registers. On this encoding we use an equivariant circuit [30] and a non-equivariant one that is the same circuit in a rotated basis of one register. The two circuits therefore have the same number and arrangement of trainable angles, and conjugate DLAs of the same dimension. The equivariant circuit measured with an equivariant observable gives an invariant model (Equiv); measured with a non-equivariant observable, it gives a non-invariant one (Equiv-NoSym); the non-equivariant circuit gives a non-invariant model even with the equivariant observable (NonEquiv), and becomes invariant when averaged over the group (NonEquiv-Twirled). The four models thus form a two-by-two design, in which the equivariance of the circuit and the invariance of the model are varied independently.

Figure 1. The study design. a Symmetry-aware amplitude encoding, shown for a 4×44 \times4 image: pixel (i,j)(i,j) is the amplitude of the basis state ∣i⟩r⊗∣j⟩c|i\rangle_r \otimes|j\rangle_c, and the reflections txt_x, tyt_y and the rotation rr, which generate D4D_4, act on the qubits as XX layers on one register and a swap of the two registers (Eq. (22)). b The four models. Two circuits, equivariant (UU) or not (VV), are combined with an equivariant observable OO, with a non-equivariant observable OnsO_{\mathrm{ns}}, or with the average over D4D_4; the four models have the same circuit structure and trainable angles, and equivalent dynamical Lie algebras. The tags mark the models of the two comparisons of § 2.4: comparison 1 (how the invariance is obtained) and comparison 2 (the same circuit with and without an equivariant readout).

We study the regime in which the symmetry is a property of the data: the label is invariant under D4D_4 and the images appear in every pose. We use four such tasks, which differ in origin, image statistics and size: MNIST digits 4 and 5 in random pose [34]; a synthetic task, a thin cross against a filled square; and two scientific data sets, face-on against edge-on disk galaxies from Galaxy10 [35] and sickle-shaped against other deformed red blood cells [36]. We also test whether the models learn shortcuts, spurious cues that predict the label only in the training data [37], and whether the results survive the noise of current hardware.

Our main findings are the following.

  1. Invariance can prevent overfitting, on every task. With a few tens of training images, the non-invariant models fit the particular images and poses they have seen. The invariant models are also more accurate on new images (§ 2.4 A).

  1. Circuit equivariance is not enough. The equivariant circuit measured with a non-equivariant observable (Equiv-NoSym) overfits as much as the non-equivariant circuit. What matters is whether the model as a whole is invariant, not whether its circuit is equivariant (§ 2.4 B).

  1. Overfitting is the cost of learning the symmetry from the data. On upright digits the same non-equivariant circuit overfits less than half as much, because it no longer has to learn the symmetry from the images (§ 2.4 A).

  1. Built-in invariance and invariance imposed by classical post-processing (group twirling) both prevent overfitting, but differ in accuracy. Depending on the task, either can be the more accurate (§ 2.4 A).

  1. Invariant models are immune to orientation-dependent shortcuts. Trained with a watermark that reveals the class, the non-invariant models learn the watermark and drop close to chance level without it on the three tasks with real images. The invariant models learn the content of the images (§ 2.4 C).

  1. The advantage survives a realistic noise model. It persists under the noise of a typical superconducting quantum processor, whether the models are trained on the noisy device or only run on it (§ 2.4 D).

The paper is structured as follows. Sec. II recalls equivariance, twirling, DLA and the function class of a model, and shows why an invariant model can learn from fewer images. § 1 introduces the encoding, the circuits and the four models under consideration, and compares their DLAs and function classes. § 2 describes the tasks, the shortcuts, the training and the noise model. § 2.4 reports the results and Sec. VI discusses them. In § 2.6, we draw our conclusions. Technical details, the computation of the DLAs and function classes, and additional results are in the Appendices.

Background

Variational quantum classifiers

We consider an nn-qubit Hilbert space H=(C2)⊗n\mathcal{H} = (\mathbb{C}^{2})^{\otimes n}, and denote by B(H)\mathcal{B}(\mathcal{H}) the space of bounded linear operators on H\mathcal{H} and by U(H)\mathcal{U}(\mathcal{H}) the subset of unitary operators. A variational quantum model is specified by a family of parametrized unitaries

U(θ)∈U(H),θ=(θ1,…,θp)∈Rp,(1)U(\boldsymbol{\theta}) \in\mathcal{U}(\mathcal{H}), \qquad\boldsymbol{\theta} = (\theta_1,\ldots,\theta_p) \in\mathbb{R}^{p}, \tag*{(1)}

where pp is the number of trainable parameters. Given a classical input x∈Xx \in\mathcal{X}, an encoding map associates it with a quantum state ρ(x)∈B(H)\rho(x) \in\mathcal{B}(\mathcal{H}). The state is then processed by the variational circuit and measured through an observable O∈B(H)O \in\mathcal{B}(\mathcal{H}), which we call the readout, yielding the model output

fθ(x)=Tr⁡[OU(θ)ρ(x)U(θ)†].(2)f_{\boldsymbol{\theta}}(x) = \operatorname{Tr}\left[O U(\boldsymbol{\theta})\rho(x)U(\boldsymbol{\theta})^\dagger\right]. \tag*{(2)}

The encoding, the parametrized circuit, and the readout observable jointly determine the class of functions that the model can represent.

We now specialize this framework to supervised classification. A model is trained on a set of labeled examples

D={(xi,yi)}i=1N,(3)\mathcal{D} = \{(x_i,y_i)\}_{i=1}^{N}, \tag*{(3)}

where xi∈Xx_i \in\mathcal{X} is the ii-th input data point of the training set, X\mathcal{X} is the data space, and yiy_i is its associated label. We focus on binary classification tasks, so that yi∈{0,1}y_i \in\{0,1\}, and the model predicts class 1 if fθ(x)>0f_\theta(x)>0 and 0 otherwise. The examples in D\mathcal{D} are used to optimize the parameters θ\theta so that the resulting model can predict the labels of previously unseen inputs.

Training consists of adjusting θ\theta with a classical optimizer to minimize the empirical loss

L(θ)=1N∑i=1Nℓ(fθ(xi),yi),(4)\mathcal{L}(\theta)=\frac{1}{N}\sum_{i=1}^{N}\ell(f_\theta(x_i),y_i), \tag*{(4)}

where ℓ\ell penalizes discrepancies between the model output and the corresponding labels.

More generally, noisy dynamics can be incorporated by replacing the unitary evolution with a parametrized quantum channel Φθ:B(H)→B(H)\Phi_\theta:\mathcal{B}(\mathcal{H})\to\mathcal{B}(\mathcal{H}), in which case

fθ(x)=Tr⁡[OΦθ(ρ(x))].(5)f_\theta(x)=\operatorname{Tr}[O\Phi_\theta(\rho(x))]. \tag*{(5)}

This more general formulation will also allow us to distinguish between the symmetry properties of the quantum evolution and the invariance of the resulting classifier, as discussed below.

Equivariance, invariance and twirling

Symmetries of a classification task are transformations of the input that preserve its target label. For images, these may include rotations and reflections when orientation is irrelevant to the class. We describe a chosen family of such transformations by a finite group GG, with representation RXR_X on the input space X\mathcal{X}. Thus, each g∈Gg\in G acts as

x↦RX(g)x,(6)x\mapsto R_X(g)x, \tag*{(6)}

and the group operation is represented by composition,

RX(g1g2)=RX(g1)RX(g2).(7)R_X(g_1g_2)=R_X(g_1)R_X(g_2). \tag*{(7)}

For example, rotations by multiples of 90∘90^\circ and reflections of a square form the dihedral group D4D_4. An invariant classifier assigns the same output to all inputs related by these transformations.

The action of the group of transformations on quantum states is described by ρ↦RH(g)ρRH(g)†\rho\mapsto R_{\mathcal{H}}(g)\rho R_{\mathcal{H}}(g)^\dagger, where RH:G→U(H)R_{\mathcal{H}}:G\to\mathcal{U}(\mathcal{H}) is a unitary representation of GG on the Hilbert space H\mathcal{H}. This representation depends on the particular encoding of the classical data into quantum states x↦ρ(x)x\mapsto\rho(x), with a compatibility condition between the representation RXR_X on the data and the representation RHR_{\mathcal{H}} on the quantum states. More precisely, an encoding is said to be (G,RH,RX)(G,R_{\mathcal{H}},R_X)-equivariant if

ρ(RX(g)x)=RH(g)ρ(x)RH(g)†.(8)\rho(R_X(g)x)=R_{\mathcal{H}}(g)\rho(x)R_{\mathcal{H}}(g)^\dagger. \tag*{(8)}

Informally, equivariance means that the encoding of the transformed image is the same as the transformed quantum state of the original image. Equivariance of a quantum channel can be defined analogously: transforming the input before processing gives the same state as transforming the output afterwards. Formally, a quantum channel Φ:B(H)→B(H)\Phi: \mathcal{B}(\mathcal{H}) \to\mathcal{B}(\mathcal{H}) is said to be (G,RH)(G, R_{\mathcal{H}})-equivariant if

Φ(RH(g)ρRH(g)†)=RH(g)Φ(ρ)RH(g)†(9)\Phi\left(R_{\mathcal{H}}(g)\rho R_{\mathcal{H}}(g)^\dagger\right) = R_{\mathcal{H}}(g)\Phi(\rho)R_{\mathcal{H}}(g)^\dagger \tag*{(9)}

for every g∈Gg \in G and for every ρ∈B(H)\rho\in\mathcal{B}(\mathcal{H}) [20, 21, 27]. A linear operator O∈B(H)O \in\mathcal{B}(\mathcal{H}) is (G,RH)(G, R_{\mathcal{H}})-equivariant if

[O,RH(g)]=0(10)[O, R_{\mathcal{H}}(g)] = 0 \tag*{(10)}

for every g∈Gg \in G. In particular, a parametrized quantum circuit U(θ)=∏ke−iθkHk/2U(\theta) = \prod_k e^{-i\theta_k H_k/2} is (G,RH)(G, R_{\mathcal{H}})-equivariant for every θ\theta if every generator HkH_k is (G,RH)(G, R_{\mathcal{H}})-equivariant. A quantum learning model obtained from an equivariant circuit and an equivariant observable is invariant under the action of the transformation group [20, 21].

Proposition 1. If Φ\Phi is a (G,RH)(G, R_{\mathcal{H}})-equivariant quantum channel and OO is a (G,RH)(G, R_{\mathcal{H}})-equivariant observable, then the model f(ρ):=Tr⁡[OΦ(ρ)]f(\rho) := \operatorname{Tr}[O\Phi(\rho)] is (G,RH)(G, R_{\mathcal{H}})-invariant:

f(RH(g)ρRH(g)†)=f(ρ),(11)f\left(R_{\mathcal{H}}(g)\rho R_{\mathcal{H}}(g)^\dagger\right) = f(\rho), \tag*{(11)}

for all g∈Gg \in G and ρ∈B(H)\rho\in\mathcal{B}(\mathcal{H}).

Proof. This follows immediately using (9), cyclicity of the trace and the fact that (10) implies RH(g)†ORH(g)=OR_{\mathcal{H}}(g)^\dagger O R_{\mathcal{H}}(g) = O. □\square

In this construction both conditions matter: an equivariant circuit measured with a non-equivariant observable is in general not invariant. A necessary and sufficient condition for invariance is more easily stated in the Heisenberg picture. Let Φ†\Phi^\dagger be the adjoint of Φ\Phi, defined by Tr⁡[AΦ(ρ)]=Tr⁡[Φ†(A)ρ]\operatorname{Tr}[A\Phi(\rho)] = \operatorname{Tr}[\Phi^\dagger(A)\rho]. The output of a model is fθ(x)=Tr⁡[Aθρ(x)]f_\theta(x) = \operatorname{Tr}[A_\theta\rho(x)], with the effective observable

Aθ=Φθ†(O),(12)A_\theta= \Phi_\theta^\dagger(O), \tag*{(12)}

which is U(θ)†OU(θ)U(\theta)^\dagger O U(\theta) for a unitary circuit. Since Tr⁡[ARH(g)ρRH(g)†]=Tr⁡[RH(g)†ARH(g)ρ]\operatorname{Tr}[A R_{\mathcal{H}}(g)\rho R_{\mathcal{H}}(g)^\dagger] = \operatorname{Tr}[R_{\mathcal{H}}(g)^\dagger A R_{\mathcal{H}}(g)\rho], the model is invariant on all input states if and only if

RH(g)†AθRH(g)=Aθfor every g∈G.(13)R_{\mathcal{H}}(g)^\dagger A_\theta R_{\mathcal{H}}(g) = A_\theta\qquad\text{for every } g \in G. \tag*{(13)}

Proposition 1 is the special case in which this holds because Φθ\Phi_\theta and OO are both equivariant. When the encoded states do not span all the states, as for the real images of § 1 A, the condition is required only of the part of AθA_\theta that they detect (§ 1 D).

An invariant model can also be obtained from a non-equivariant circuit by averaging its output over the group, an operation called twirling [21, 27]. For an equivariant encoding,

ftw(x):=1∣G∣∑g∈Gfθ(RX(g)x)=Tr⁡[PG(Aθ)ρ(x)],(14)f_{\mathrm{tw}}(x) := \frac{1}{|G|}\sum_{g\in G} f_\theta(R_X(g)x) = \operatorname{Tr}[P_G(A_\theta)\rho(x)], \tag*{(14)}

where

PG(A):=1∣G∣∑g∈GRH(g)†ARH(g)(15)P_G(A) := \frac{1}{|G|}\sum_{g\in G} R_{\mathcal{H}}(g)^\dagger A R_{\mathcal{H}}(g) \tag*{(15)}

is the projection onto the operators that commute with the representation. The twirled model therefore satisfies Eq. (13) for every channel and every observable; if OO is equivariant, it coincides with the model obtained by twirling the channel instead [27]. On a physical device, Eq. (14) is estimated by running the circuit on the transformed inputs: all ∣G∣|G| of them, fewer if the model already has part of the symmetry (§ 1 C), or a random sample of them. In classical simulation, PG(Aθ)P_G(A_\theta) can instead be computed once per parameter update and reused for all the inputs of a batch.

Dynamical Lie algebra and function class

A quantum model is built from three ingredients: the gates of the circuit, the readout observable and the encoding. We characterize it at three nested levels, each of which adds one ingredient: the dynamical Lie algebra depends only on the gates, the span of the effective observables also on the readout, and the feature span also on the encoding.

Dynamical Lie algebra.

The dynamical Lie algebra (DLA) of a set of gates e−iθkHk/2e^{-i\theta_k H_k/2} is the real Lie algebra generated by their generators [8, 33],

g=span⁡R{iHk, [iHk,iHl], [iHk,[iHl,iHm]],…}.(16)\mathfrak{g} = \operatorname{span}_{\mathbb{R}}\{iH_k,\ [iH_k,iH_l],\ [iH_k,[iH_l,iH_m]],\ldots\}. \tag*{(16)}

Every circuit built from these gates lies in the Lie group ege^{\mathfrak{g}}, for every choice of the angles and every depth, and deep enough circuits with independent angles reach all of it. The DLA thus describes the unitaries available to the circuit. It also determines bounds on the number of parameters needed to overparametrize a circuit [33] and, for deep circuits and under assumptions on the input states and the observable, the variance of the gradients, which is exponentially small when gg is exponentially large [8, 9]; neither quantity is fixed by the dimension of g\mathfrak{g} alone. If every generator commutes with RH(g)R_{\mathcal{H}}(g), so does every element of g\mathfrak{g}: an equivariant circuit has a DLA contained in the commutant of the representation, which is the sense in which symmetry restricts the DLA [20]. The DLA, however, sees neither the observable nor the encoding. A fixed change of basis WW maps a circuit and its DLA to conjugate ones, which reach the same unitaries up to WW, but with the same encoding and readout it can change completely the functions that the model computes.

Effective observables.

The effective observables Aθ=U(θ)†OU(θ)A_\theta= U(\theta)^\dagger O U(\theta) of all the circuits built from the gates span the space

MO=span⁡{U†OU:U∈eg},(17)\mathcal{M}_O = \operatorname{span}\{U^\dagger O U : U \in e^{\mathfrak{g}}\}, \tag*{(17)}

which is the smallest space that contains OO and is closed under commutators with g\mathfrak{g}, that is, the span of OO and of its nested commutators [iHk1,[iHk2,…[iHkr,O]]…][iH_{k_1},[iH_{k_2},\ldots[iH_{k_r},O]]\ldots] [8]. Its dimension depends on the gates and on the observable. If the circuit and OO are both equivariant, every element of MO\mathcal{M}_O is equivariant. c. Function class and feature span. The function class of a model is the family F={x↦fθ(x)}\mathcal{F}=\{x\mapsto f_\theta(x)\} of the functions that it computes as its angles vary, a nonlinear family with at most as many local degrees of freedom as trainable angles. We characterize it by its linear span, the feature span, and call features the elements of a basis of this span. The feature span is the image of the span of the effective observables, which is contained in MO\mathcal{M}_{O}, under the linear map A↦Tr⁡[Aρ(⋅)]A\mapsto\operatorname{Tr}[A\rho(\cdot)], so its dimension is at most dim⁡MO\dim\mathcal{M}_{O}, and smaller when different observables coincide on all the encoded states. This dimension counts the directions that the outputs can explore, not the number of independently adjustable coefficients, and a larger span does not by itself imply better or worse generalization. If every element of MO\mathcal{M}_{O} is equivariant and the encoding is equivariant, every function in F\mathcal{F} is invariant (Eq. (13)). Twirling replaces every effective observable with its projection PG(Aθ)P_{G}(A_{\theta}): the resulting feature span is invariant, but in general different from that of an equivariant circuit.

The three levels answer different questions. The DLA describes the unitaries available to the circuit; the effective observables take into account the readout measurement; the feature span describes the functions available to the model, and its invariance—whether the output can depend on the pose of the input. Two models with the same DLA can differ in the last two (§ 1 D).

Invariance forces a model to assign the same output to all symmetry-related images, thus preventing the model from exploiting pose-dependent patterns in the finite training sample. When pose carries no information about the target label, this constraint supplies an appropriate inductive bias. However, it can still memorize the training images and their transformed copies without generalizing to new examples. For example, two handwritten “4”s generally belong to different orbits: rotating or reflecting one does not produce the other. An invariant model may correctly classify every transformed version of a training “4”, yet misclassify a new “4” with different handwriting.

For an invariant model, the training loss on NN images equals the loss on all their N∣G∣N|G| transformed copies: the model is trained as if on every pose of its images, although these copies are not independent samples.

Models

Symmetry-aware encoding

Each grayscale image is resized to d×dd\times d pixels, where d=2n/2d=2^{n/2} and nn is a positive even integer. Its pixel intensities are rescaled to real ℓ2\ell_{2}-normalized coefficients xijx_{ij} and amplitude-encoded on nn qubits [3, 38],

∣ψ(x)⟩=∑i,j=0d−1xij∣i⟩r⊗∣j⟩c.(18)|\psi(x)\rangle=\sum_{i,j=0}^{d-1}x_{ij}|i\rangle_{r}\otimes|j\rangle_{c}. \tag*{(18)}

The row and column indices are stored in binary, with the most significant bit first, in two registers of n/2n/2 qubits each. The row register occupies wires 0,…,n/2−10,\ldots,n/2-1, and the column register occupies wires n/2,…,n−1n/2,\ldots,n-1. The case n=8n=8, corresponding to 16×1616\times16 pixels, is illustrated in Figure 1a.

The symmetries of the square pixel grid form the dihedral group

D4={e,r,r2,r3,t,tr,tr2,tr3},(19)D_{4}=\{e,r,r^{2},r^{3},t,tr,tr^{2},tr^{3}\}, \tag*{(19)}

where ee is the identity, rr is the anticlockwise rotation by 90∘90^\circ, and tt is the reflection about the vertical axis. Thus, the elements rkr^{k} describe rotations by multiples of 90∘90^\circ, while the elements trktr^{k} describe reflections about the horizontal, vertical, and two diagonal axes. The encoding induces the unitary representation

RH(tx)(∣i⟩r⊗∣j⟩c)=∣d−1−i⟩r⊗∣j⟩c,R_{\mathcal{H}}(t_{x})(|i\rangle_{r}\otimes|j\rangle_{c})=|d-1-i\rangle_{r}\otimes|j\rangle_{c},
RH(ty)(∣i⟩r⊗∣j⟩c)=∣i⟩r⊗∣d−1−j⟩c.(20)R_{\mathcal{H}}(t_{y})(|i\rangle_{r}\otimes|j\rangle_{c})=|i\rangle_{r}\otimes|d-1-j\rangle_{c}. \tag*{(20)}

Here ty=tt_{y}=t is the reflection about the vertical axis and tx=tr2t_{x}=tr^{2} the reflection about the horizontal axis. Since d=2n/2d=2^{n/2}, replacing an index ii by d−1−id-1-i complements all its binary digits. These reflections are therefore implemented by applying Pauli XX gates to every qubit of the corresponding register.

With rows counted from top to bottom and columns from left to right, the anticlockwise rotation by 90∘90^\circ maps pixel positions (i,j)↦(d−1−j,i)(i,j)\mapsto(d-1-j,i). Its representation then satisfies

RH(r)(∣i⟩r⊗∣j⟩c)=∣d−1−j⟩r⊗∣i⟩c.(21)R_{\mathcal{H}}(r)(|i\rangle_{r}\otimes|j\rangle_{c})=|d-1-j\rangle_{r}\otimes|i\rangle_{c}. \tag*{(21)}

which can be obtained by first swapping the two registers and then applying RH(tx)R_{\mathcal{H}}(t_{x}). The register swap itself represents the reflection tr3tr^{3} about the main diagonal. Consequently,

RH(tx)=X⊗(n/2)⊗I⊗(n/2),RH(ty)=I⊗(n/2)⊗X⊗(n/2),RH(r)=RH(tx)S,(22)\begin{aligned} R_{\mathcal{H}}(t_{x})&=X^{\otimes(n/2)}\otimes I^{\otimes(n/2)},\\ R_{\mathcal{H}}(t_{y})&=I^{\otimes(n/2)}\otimes X^{\otimes(n/2)},\\ R_{\mathcal{H}}(r)&=R_{\mathcal{H}}(t_{x})S, \tag*{(22)} \end{aligned}
S:=RH(tr3)=∏a=0n/2−1SWAP⁡a,a+n/2.S:=R_{\mathcal{H}}(tr^{3})=\prod_{a=0}^{n/2-1}\operatorname{SWAP}_{a,a+n/2}.

The representations of the remaining group elements follow by composition. By construction, the encoding is D4D_{4}-equivariant:

∣ψ(RX(g)x)⟩=RH(g)∣ψ(x)⟩for every g∈D4.(23)|\psi(R_{X}(g)x)\rangle=R_{\mathcal{H}}(g)|\psi(x)\rangle\qquad\text{for every }g\in D_{4}. \tag*{(23)}

Equivariant circuits and controlled symmetry breaking

To isolate the role of circuit equivariance, we construct two families with the same parameter count, parameter sharing and gate connectivity. By parameter sharing we mean that a single trainable angle is used by several gates, which are then rotated together during training; two circuits have the same parameter sharing when their gates use the angles in the same pattern. In both circuits, the rotations of a row qubit and of its column partner share one angle, as required by the symmetry under the swap of the two registers, and in every layer the four blocks on the bit pairs (0, 1), (2, 3), (1, 2) and (3, 0) share three angles, while the block on (0, 2) has three of its own (Eq. (25)): the 25 gates of a layer depend on only six trainable angles. In the code (eqnn/circuits.py), each gate is stored with the index of the entry of θ\theta that it uses, and the gradient with respect to a shared angle, the sum of the contributions of all the gates that use it, is computed by automatic differentiation. We first build a D4D_4-equivariant circuit U(θ)U(\theta), then define a comparison circuit V(θ)V(\theta) by conjugating U(θ)U(\theta) with a fixed unitary acting on the column register. Throughout this construction, the encoding and the representation of the image symmetries are kept fixed.

We label the row and column qubits by rjr_j and cjc_j, respectively, with j=0,…,n/2−1j=0,\ldots,n/2-1; these correspond to wires jj and j+n/2j+n/2 in Figure 2. For two bit positions j,kj,k, define the four-qubit unitary

Figure 2. One layer of the two circuits described in § 1 B (wires 0–3: row register; wires 4–7: column register). The first box is the feature encoding, which prepares the state ∣ψ(x)⟩|\psi(x)\rangle of Eq. (18); RP(η)=e−iηP/2R_P(\eta)=e^{-i\eta P/2} is the rotation generated by the Pauli string PP, and gates with the same color share the angle η\eta, a component of αl\alpha_l or βl\beta_l (Eq. (25)). The non-equivariant circuit (right) has RYR_Y rotations on the column register and RYYZZR_{Y Y Z Z} rotations in place of RXR_X and RYYYYR_{Y Y Y Y}, with the same gate count, wiring and depth.

Wjk(η)=e−iη32YjYkYj+n/2Yk+n/2e−iη22(Xk+Xk+n/2)e−iη12(Xj+Xj+n/2),(24)W_{jk}(\eta)=e^{-i\frac{\eta_3}{2}Y_jY_kY_{j+n/2}Y_{k+n/2}}e^{-i\frac{\eta_2}{2}(X_k+X_{k+n/2})}e^{-i\frac{\eta_1}{2}(X_j+X_{j+n/2})}, \tag*{(24)}

where η=(η1,η2,η3)\eta=(\eta_1,\eta_2,\eta_3). Each rotation acts in the same way on a row qubit and on its column partner, and the four-qubit rotation acts on both registers with the same Pauli operators. This makes the block symmetric under the reflections of either register and under their swap, and hence D4D_4-equivariant for every choice of the angles.

The circuit follows the arrangement in Figure 2, adapted from Chang et al. [30]. In layer ll, the four pairs (0, 1), (2, 3), (1, 2) and (3, 0) share a parameter triple αl\alpha_l, while the pair (0, 2) uses an independent triple βl\beta_l. With the rightmost factor acting first, the layer is

Ul=W02(βl)W30(αl)W12(αl)W23(αl)W01(αl),(25)U_l=W_{02}(\beta_l)W_{30}(\alpha_l)W_{12}(\alpha_l)W_{23}(\alpha_l)W_{01}(\alpha_l), \tag*{(25)}

and the full circuit is

U(θ)=UL⋯U1,θ=(α1,β1,…,αL,βL).(26)U(\theta)=U_L\cdots U_1,\qquad\theta=(\alpha_1,\beta_1,\ldots,\alpha_L,\beta_L). \tag*{(26)}

Thus, an LL-layer circuit has 6L6L trainable parameters and is D4D_4-equivariant for every choice of these parameters.

To construct the comparison circuit, let

C=exp⁡(−iπ3X+Y+Z3)(27)C=\exp\left(-i\frac{\pi}{3}\frac{X+Y+Z}{\sqrt{3}}\right) \tag*{(27)}

be the single-qubit Clifford unitary that cycles the Pauli axes,

CXC†=Y,CYC†=Z,CZC†=X,(28)CXC^\dagger=Y,\qquad CYC^\dagger=Z,\qquad CZC^\dagger=X, \tag*{(28)}

and set Cc=I⊗(n/2)⊗C⊗(n/2)C_c=\mathbb{I}^{\otimes(n/2)}\otimes C^{\otimes(n/2)}. We define

V(θ):=CcU(θ)Cc†.(29)V(\theta):=C_cU(\theta)C_c^\dagger. \tag*{(29)}

Conjugation acts on each block as

CcWjk(η)Cc†=e−iη32YjYkZj+n/2Zk+n/2e−iη22(Xk+Yk+n/2)e−iη12(Xj+Yj+n/2).(30)C_cW_{jk}(\eta)C_c^\dagger=e^{-i\frac{\eta_3}{2}Y_jY_kZ_{j+n/2}Z_{k+n/2}}e^{-i\frac{\eta_2}{2}(X_k+Y_{k+n/2})}e^{-i\frac{\eta_1}{2}(X_j+Y_{j+n/2})}. \tag*{(30)}

Implementing these conjugated gates directly preserves the gate count, connectivity, depth and parameter sharing. The transformed generators no longer all commute with the column reflection or with the swap of the two registers. Consequently, V(θ)V(\theta) is generally not D4D_4-equivariant with respect to the fixed representation (22). It nevertheless retains equivariance under the row reflection txt_x, since this reflection commutes with both U(θ)U(\theta) and CcC_c.

Finally, unitary conjugation preserves commutators, so the dynamical Lie algebras of the two circuit families satisfy

gV=CcgUCc†,dim⁡gV=dim⁡gU.(31)\mathfrak{g}_V = C_c \mathfrak{g}_U C_c^\dagger,\qquad\dim\mathfrak{g}_V = \dim\mathfrak{g}_U . \tag*{(31)}

Four models

The output of a model is an expectation value, with no trainable classical post-processing: the predicted class is 1 if f(x)>0f(x)>0 and 0 otherwise. We measure either

O=12(X0+X4)O = \frac{1}{2}(X_0+X_4)

or

Ons=X0+Z0+Z41+2,(32)O_{\mathrm{ns}} = \frac{X_0+Z_0+Z_4}{1+\sqrt{2}}, \tag*{(32)}

both normalized so that f(x)∈[−1,1]f(x)\in[-1,1]. OO is equivariant, while OnsO_{\mathrm{ns}} breaks every symmetry of D4D_4 (Z0Z_0 and Z4Z_4 change sign under the reflections, and the swap maps X0+Z0X_0+Z_0 to X4+Z4X_4+Z_4). The four models of Figure 1b, Equiv (feqf_{\mathrm{eq}}), NonEquiv (fneqf_{\mathrm{neq}}), NonEquiv-Twirled (ftwf_{\mathrm{tw}}) and Equiv-NoSym (fnsf_{\mathrm{ns}}), are

feq(x)=⟨ψ(x)∣U†OU∣ψ(x)⟩,fneq(x)=⟨ψ(x)∣V†OV∣ψ(x)⟩,ftw(x)=18∑g∈D4fneq(RX(g)x),fns(x)=⟨ψ(x)∣U†OnsU∣ψ(x)⟩.(33)\begin{aligned} f_{\mathrm{eq}}(x) &= \langle\psi(x)|U^\dagger O U|\psi(x)\rangle,\\ f_{\mathrm{neq}}(x) &= \langle\psi(x)|V^\dagger O V|\psi(x)\rangle,\\ f_{\mathrm{tw}}(x) &= \frac{1}{8}\sum_{g\in D_4}f_{\mathrm{neq}}(R_X(g)x),\\ f_{\mathrm{ns}}(x) &= \langle\psi(x)|U^\dagger O_{\mathrm{ns}} U|\psi(x)\rangle. \tag*{(33)} \end{aligned}

Equiv is invariant by Proposition 1 and NonEquiv-Twirled by construction: the eight transformed copies of the image, obtained classically by permuting the pixels, go through the same circuit with the same angles (Figure 3). NonEquiv and Equiv-NoSym are not invariant under D4D_4, and they differ in how the invariance is broken. NonEquiv retains invariance under the row reflection txt_x, since both its circuit VV and its readout observable OO commute with RH(tx)R_H(t_x). It is therefore invariant under the subgroup {e,tx}\{e,t_x\}: for any g∈D4g\in D_4, applying txt_x after gg leaves the model output unchanged, the inputs RX(g)xR_X(g)x and RX(txg)xR_X(t_xg)x giving identical outputs. Since tx2=et_x^2=e, this groups the eight transformations into four disjoint pairs {g,txg}\{g,t_xg\}, called the right cosets of {e,tx}\{e,t_x\} in D4D_4. Consequently, the eight terms defining ftwf_{\mathrm{tw}} form four pairs of equal terms in the noiseless model, and only one evaluation per pair is needed. The noise model of § 2 D breaks this residual symmetry and removes the reduction.

Figure 3. NonEquiv-Twirled: the image is transformed by the eight elements of D4D_4, the copies go through the same NonEquiv circuit with the same angles, and the output is the mean of the eight expectation values. The model is exactly invariant, at eight times the cost of NonEquiv.

Equiv-NoSym breaks invariance through its readout: although its circuit is equivariant, the observable OnsO_{\mathrm{ns}} breaks every nontrivial symmetry of D4D_4. Thus, this model has no residual symmetry enforced by its architecture. The relation between Equiv-NoSym and Equiv can be characterized explicitly. Writing Ons=(O+Q)/(1+2)O_{\mathrm{ns}}=(O+Q)/(1+\sqrt{2}), with Q=12(X0−X4)+Z0+Z4Q=\frac{1}{2}(X_0-X_4)+Z_0+Z_4 and PG(Q)=0\mathcal{P}_G(Q)=0,

fns(x)=feq(x)+hθ(x)1+2,(34)f_{\mathrm{ns}}(x)=\frac{f_{\mathrm{eq}}(x)+h_\theta(x)}{1+\sqrt{2}}, \tag*{(34)}

where hθ(x)=⟨ψ(x)∣U†QU∣ψ(x)⟩h_\theta(x)=\langle\psi(x)|U^\dagger Q U|\psi(x)\rangle averages to zero over the group because UU commutes with every RH(g)R_H(g). Equiv-NoSym thus adds a pose-dependent term to a rescaled Equiv; the same angles set both terms, which cannot in general be tuned independently. Twirling Equiv-NoSym removes this term and returns Equiv, up to the constant factor 1/(1+2)1/(1+\sqrt{2}).

Same algebra, different function classes

The two parametrized circuits U(θ)U(\theta) and V(θ)V(\theta) have conjugate DLAs, and Equiv-NoSym and NonEquiv-Twirled inherit the circuits of Equiv and NonEquiv. All four models therefore share the DLA dimension, dim⁡g=569\dim\mathfrak{g}=569, which we computed with the Lie closure of PennyLane (qml.lie_closure) [39] (Appendix A). The four models are thus equivalent in what the DLA alone determines, the reachable unitaries (up to the fixed change of basis CcC_{c}) and the overparametrization threshold: no circuit is larger or more expressive than another. Their gradients depend also on the observable and on the inputs, so we measured them in the regime of our training (Figure 9 in Appendix A): none of them vanishes, and they differ by at most a factor of a few, Equiv-NoSym having the smallest. All four models reach comparable training accuracies (Figure 6a–Figure 6d), along similar learning curves (Figure 10).

Figure 6. The four models in the main settings (L=4L = 4; N=40N = 40, or 20 on the synthetic task; 15 seeds). a–d Training (solid) and test accuracy (hatched), mean and standard error, with the single seeds (dots) and the generalization gap of each model above its bars. e–h Per-seed differences in test accuracy between pairs of models. The invariant models (Equiv, NonEquiv-Twirled) have small gaps on every task, the non-invariant ones (NonEquiv, Equiv-NoSym) large ones. Paired comparisons in Table IV; values in the repository.

Figure 9. Gradients of the training loss at the initialization used in training (angles uniform in [−0.1,0.1][-0.1, 0.1]), on the training images of the main settings. a–d Variance over 100 draws of the angles of each partial derivative, averaged over the angles, as a function of the number of layers. e–h Histograms of the moduli of all the partial derivatives over 200 draws, with L=4L = 4; dotted lines: medians.

Figure 10. Learning curves in the main settings: training accuracy (solid) and accuracy on 250 test images (dashed) every seven epochs; mean and standard error over 15 seeds. The test accuracy of the non-invariant models stops growing after a few epochs while their training accuracy keeps rising.

What differs is whether the symmetry of the circuit coincides with the action of D4D_{4} on the images and with the symmetry of the readout, and this fixes the function class (Figure 4). Equiv combines 237 features, all invariant. NonEquiv combines 1304, almost none of which is invariant. NonEquiv-Twirled projects them onto 666 invariant features, nearly disjoint from those of Equiv. Equiv-NoSym combines the 237 features of Equiv and 535 non-invariant ones. The invariance of a model, which the DLA does not see, decides whether its features can depend on the pose of the image.

Figure 4. Equivalent DLAs, different function classes: number of independent quadratic features xTMxx^{T}Mx of the image combined by each model, split into D4D_{4}-invariant (left) and non-invariant (right) ones; hatched: features shared with Equiv. All four models have a DLA of dimension 569 (Appendix A).

Setup

Tasks

We use four binary tasks (Figure 5): MNIST digits 4 against 5 [34]; a synthetic task, a thin cross against a filled square, of random size and position on a weak noisy background; face-on against edge-on disk galaxies from Galaxy10 SDSS [35, 40]; and elongated (sickle) against other deformed red blood cells from erythrocytesIDB [36, 41]. In all of them the label is invariant under D4D_{4}, and every image, in training and in test, appears in a random pose: MNIST, Galaxy10 and the blood cells are transformed by an element of D4D_{4} drawn at random, and the synthetic shapes are themselves symmetric and placed at random. On the galaxies and the blood cells the orientation is already random by nature, and the random pose only makes the invariance exact. As a control we also train and test on upright MNIST digits. The training sets are balanced and drawn, for each seed, from a pool disjoint from the test set. Appendix B describes the data sets and how the synthetic task and the pairs of classes were chosen.

Figure 5. The four tasks (rows). Left: training images without cues. Right: one image of each class with each of the cues used in the experiments on shortcuts (§ 2 B). The corner watermarks and the brightness gradients of the two classes are related by a rotation of 90∘90^\circ; the center/corner mark is invariant under D4D_4.

Shortcuts

To test what the models learn when the training set offers an easier rule than the true one [37], we add to every training image a class-dependent cue (Figure 5, right): a corner watermark, a small patch in one corner for class 0 and in the corner rotated by 90° for class 1; a brightness gradient along an oblique direction rotated by 90° for class 1; and, as a control, a center/corner mark, a bright patch at the center for class 0 and in the four corners for class 1. The watermarks and the gradients of the two classes are related by an element of D4D_{4}, and the content of the images is in a random pose, so an invariant model cannot tell them apart; the control cue is itself invariant, and every model can read it. Each model is tested on images with the cue of their class, as in training; with the cued image moved by a random element of D4D_{4}, which in general carries the cue to a position not seen in training, or onto the cue of the other class; and without the cue, which measures whether the model has learned the true rule.

Training and evaluation

The circuits are simulated exactly, with expectation values computed without shot noise, and trained with Adam on a logistic loss of the output; all the details are in Appendix C. We report the training accuracy, the test accuracy and their difference, the generalization gap [42]. A seed fixes the training images, the poses, the initial angles and the mini-batches, and every model is trained with the same seeds, so all the comparisons are paired; differences are tested with the Wilcoxon signed-rank test. The results in the main text are for the main settings, L=4L=4 layers (24 angles) and N=40N=40 training images (N=20N=20 on the synthetic task), with 15 seeds; this is the depth at which the non-equivariant circuit can fit its training images and the equivariant one trains reliably (Appendix C). The dependence on NN and LL is in Appendix C, and every other setting in the repository of the paper.

Hardware noise

We also simulate the circuits under a noise model with the typical parameters of current superconducting processors: after every gate, thermal relaxation and depolarizing noise on each of its qubits, relaxation of the idle qubits, and asymmetric readout errors. All the error channels act on single qubits, including those of the multi-qubit gates, with no correlated errors, and their parameters differ from qubit to qubit (Appendix D). A global scale ss multiplies all the error rates: s=0s = 0 is the noiseless circuit and s=1s = 1 a typical device, with the median parameters of current superconducting processors (Table V). We consider two scenarios in the main settings: models trained and tested under noise (ss up to 4, five seeds), and models trained without noise and only run on the noisy device (15 seeds). Realistic noise breaks the symmetry of Equiv, because amplitude damping does not commute with the reflections and the qubits exchanged by the swap have different parameters, but only slightly: on a typical device its output changes by less than 2% under the group (Appendix D). NonEquiv-Twirled, whose symmetry is classical, stays exactly invariant.

ParameterMedianDispersion
Error, 1-qubit rotation4×10−44 \times10^{-4}0.5
Duration, 1-qubit rotation100 ns–
Error, 4-qubit rotation (per qubit)9×10−39 \times10^{-3}0.4
Duration, 4-qubit rotation480 ns–
T1T_1200 μ\mus0.3
T2T_2 (≤2T1\le2T_1)150 μ\mus0.3
Readout p10=P(1∣0)p_{10} = P(1|0)0.010.5
Readout p01=P(0∣1)p_{01} = P(0|1)0.020.5

Table V. Parameters of the noise model at s=1s = 1: medians of a typical superconducting processor and log-normal dispersion from qubit to qubit (standard deviation of the logarithm). A single-qubit rotation is two X\sqrt{X} pulses; a rotation on four qubits is charged, on each qubit, three two-qubit gates of error 3×10−33 \times10^{-3} and the six layers (80 ns each) of its compilation.

Results

Figure 6 summarizes the results without cues and without noise. The four tasks differ in the origin and statistics of the images and in how hard they are for the circuits, yet the picture is the same on all of them: the two invariant models do not overfit, and the two non-invariant ones do.

Comparison 1: invariance improves generalization

The four models fit their training images comparably, but generalize very differently (Figure 6a–Figure 6d). On every task the generalization gap of NonEquiv is two to five times that of the invariant models, and the difference holds in most seeds (Table IV). The invariant models are also more accurate on new images: Equiv beats NonEquiv in at least 12 of the 15 seeds on every task (Figure 6e–Figure 6h, Table IV). The learning curves show how the gap forms: after a few epochs the test accuracy of NonEquiv stops growing while its training accuracy keeps rising, as the circuit fits the particular images and poses of the training set; for the invariant models the two grow together (Figure 10). The advantage is largest with few images and shrinks as NN grows, because more images show more poses (Appendix C).

TaskPairTest accuracy meankkppReduction of the gap meankkpp
MNISTEquiv vs NonEquiv+0.11(1)157×10−47 \times10^{-4}+0.10(2)142×10−32 \times10^{-3}
Twirled vs NonEquiv+0.10(1)157×10−47 \times10^{-4}+0.09(2)133×10−43 \times10^{-4}
Equiv vs Twirled+0.01(1)90.24+0.01(1)80.60
Equiv vs NoSym+0.12(1)157×10−47 \times10^{-4}+0.11(2)141×10−31 \times10^{-3}
NoSym vs NonEquiv−0.01(1)60.44−0.01(2)60.68
SyntheticEquiv vs NonEquiv+0.14(1)148×10−48 \times10^{-4}+0.12(2)157×10−47 \times10^{-4}
Twirled vs NonEquiv+0.07(2)148×10−38 \times10^{-3}+0.08(2)132×10−32 \times10^{-3}
Equiv vs Twirled+0.07(3)137×10−37 \times10^{-3}+0.04(1)120.03
Equiv vs NoSym+0.13(2)148×10−48 \times10^{-4}+0.11(1)157×10−47 \times10^{-4}
NoSym vs NonEquiv+0.01(2)100.52+0.01(2)90.72
Galaxy10Equiv vs NonEquiv+0.03(1)120.02+0.07(2)136×10−46 \times10^{-4}
Twirled vs NonEquiv+0.03(1)100.09+0.06(2)113×10−33 \times10^{-3}
Equiv vs Twirled+0.00(2)80.93+0.01(2)90.76
Equiv vs NoSym+0.09(1)157×10−47 \times10^{-4}+0.07(2)122×10−32 \times10^{-3}
NoSym vs NonEquiv−0.07(1)06×10−46 \times10^{-4}+0.00(2)80.82
Blood cellsEquiv vs NonEquiv+0.04(3)120.12+0.11(2)146×10−46 \times10^{-4}
Twirled vs NonEquiv+0.09(1)148×10−48 \times10^{-4}+0.09(2)141×10−31 \times10^{-3}
Equiv vs Twirled−0.05(2)20.03+0.01(2)90.49
Equiv vs NoSym+0.07(3)120.03+0.10(2)139×10−49 \times10^{-4}
NoSym vs NonEquiv−0.03(2)60.16+0.00(2)80.80

Table IV. Paired comparisons in the main settings (L=4L = 4, 15 seeds). For the pair “A vs B”: difference in test accuracy (A minus B) and reduction of the gap (gap of B minus gap of A), mean with the standard error on the last digit; kk: seeds in which the difference is positive; pp: two-sided Wilcoxon signed-rank test.

a. The role of the pose. The control on upright digits identifies the cause. On upright images the same NonEquiv circuit overfits less than half as much as in random pose, and it is even more accurate than Equiv, because a fixed orientation is informative and a non-invariant model can use it. The advantage is fragile, though: on rotated digits the same trained circuit drops to 0.64 (not shown; values in the repository). The overfitting of NonEquiv in random pose is thus the cost of learning the symmetry from the data, which the invariant models do not pay.

b. Built-in or averaged invariance. Both invariant models avoid overfitting, but which one is more accurate depends on the task. On MNIST and Galaxy10 they are equivalent. On the synthetic task Equiv is clearly better, and NonEquiv-Twirled approaches it only with more images; on the blood cells NonEquiv-Twirled is better, because Equiv does not fit its training images as well. The function classes explain both cases. That of NonEquiv-Twirled is almost three times as large and nearly disjoint from that of Equiv (§ 1 D). A small class generalizes from fewer images when it contains a good rule, as on the synthetic task; when it does not, as on the blood cells, a larger class fits the images better without overfitting more.

Comparison 2: the equivariant circuit needs an equivariant readout

Comparison 1 does not separate the circuit from the invariance of the model: Equiv differs from NonEquiv in both. Equiv-NoSym has the circuit of Equiv, and therefore the same DLA, but a non-equivariant readout. On all four tasks it overfits as much as NonEquiv (Figure 6a–Figure 6d), and Equiv is more accurate than Equiv-NoSym in at least 12 of the 15 seeds (Figure 6e–Figure 6h, Table IV). The two non-invariant models have the same test accuracy, except on Galaxy10, where Equiv-NoSym is less accurate, in part because two of its runs fail to train.

The four models form a 2×22 \times2 design, the circuit (equivariant or not) crossed with the model (invariant or not), and the effect of each ingredient can be estimated per seed. Making the model invariant reduces the gap and raises the test accuracy on every task, by 0.06–0.11 (Table I). Making the circuit equivariant has no consistent effect, as the pairs that differ only in the circuit (Equiv and NonEquiv-Twirled, Equiv-NoSym and NonEquiv) show in Table IV: it changes the test accuracy by a few hundredths, in either direction depending on the task, and changes the gap significantly only on the synthetic task, where Equiv overfits less than NonEquiv-Twirled (p=0.03p = 0.03). Where it helps, on the synthetic task, it does so through the invariant model it makes possible (Equiv beats NonEquiv-Twirled). The equivariance of the circuit thus matters only through the invariant model. Conversely, averaging Equiv-NoSym over the group turns it back into Equiv (§ 1 C): the equivariant circuit recovers the advantage as soon as the model is invariant.

Tasktest acc.ppgappp
MNIST+0.11(1)6×10−56 \times10^{-5}+0.10(1)6×10−56 \times10^{-5}
Synthetic+0.10(2)2×10−42 \times10^{-4}+0.09(1)6×10−56 \times10^{-5}
Galaxy10+0.06(1)1×10−41 \times10^{-4}+0.07(1)6×10−56 \times10^{-5}
Blood cells+0.08(2)3×10−33 \times10^{-3}+0.10(2)6×10−56 \times10^{-5}

Table 1. Effect of the invariance of the model in the main settings (15 seeds): per seed, the mean of the invariant models (Equiv, NonEquiv-Twirled) minus the mean of the non-invariant ones (Equiv-NoSym, NonEquiv). For the test accuracy a positive effect is an increase, for the gap a reduction; mean with the standard error on the last digit, and Wilcoxon pp value.

Shortcuts

Figure 7 shows what the models learn when every training image carries a cue that agrees with its label. The corner watermark, which the group moves, separates the models on every task. The non-invariant models learn it: they are almost perfect on test images that carry the watermark of their class, and drop to chance level when the watermark is moved by the group. On the three tasks with real images they are also close to chance level without the watermark: they have learned the watermark rather than the digit, the galaxy or the cell. On the synthetic task the shapes are easier to learn and the shortcut is only partial. The invariant models cannot tell the two watermarks apart, so moving the watermark leaves their accuracy unchanged, and they learn the content of the images: without the watermark they remain well above the non-invariant models on MNIST, on the synthetic task and on the blood cells. On Galaxy10, however, removing the watermark lowers the accuracy also of the invariant models. They cannot see its position, but they are affected by its weight in the encoded state, a fifth of the squared norm of every image: removing it shifts their output by almost the same amount for both classes, and on Galaxy10, where the outputs of the two classes are close, many test images cross the threshold. Moving the threshold by the shift measured on the training images restores the accuracy obtained with the watermark (analysis and values in the repository, experiments/watermark_shift.py).

Figure 7. Shortcuts: test accuracy of the four models trained with a cue in every training image, in the main settings (five seeds; mean and standard error). Each group of bars is one model, tested on the images with the cue of the true class (solid), on the cued images moved by a random element of D4D_4 (dotted) and on the images without the cue (hatched). The non-invariant models learn the watermark and fail when it is moved; the invariant ones cannot see its position and learn the content of the images. The center/corner mark is invariant under the group, and all the models learn it.

The brightness gradient separates the two non-invariant models. Equiv-NoSym relies on it almost entirely, while NonEquiv uses it only in part. The readout explains the difference: at the start of training, when the circuit is close to the identity, Z0Z_0 and Z4Z_4 measure the difference between the squared intensities of the two halves of the image, which is exactly what a brightness gradient changes. The invariant models are unaffected by the gradient.

The control cue, invariant under the group, is learned by all four models, which fall to chance level without it. The invariant models therefore do not ignore the watermark because they cannot see small patches: they ignore the cues that the group moves, and only those.

Hardware noise model

Figure 8 shows the four models under the noise model of § 2 D.

Figure 8. Hardware noise: test accuracy (a–d) and generalization gap (e–h) of the four models in the main settings as functions of the noise scale ss (s=0s = 0: noiseless; s=1s = 1: a typical superconducting processor, dotted line). Solid lines: models trained and tested under noise; dashed lines: models trained without noise and run under noise. Mean and standard error over the five seeds shared by the two scenarios.

a. Noise in training and test. Up to twice the noise of a typical device, the invariant models keep their noiseless accuracy and do not overfit, on every task. The non-invariant models keep overfitting, except NonEquiv on MNIST, whose generalization gap decreases as the noise increases. The decrease comes from the training accuracy, which drops, and not from the test accuracy, which remains approximately constant: the noise prevents NonEquiv from fitting its training images, but does not make it more accurate on unseen ones. The advantage of Equiv over NonEquiv persists at every noise level, and on the synthetic task and on Galaxy10 it grows with the noise. At the strongest noise (s=4s = 4), the blood cells are the only task on which every model, invariant or not, drops to chance level, so that no advantage is left.

b. Noise in the test only. When the models are trained without noise and only tested under noise, the advantage is intact: the noise does not change what the models have learned, and the overfitted ones remain overfitted. Equiv-NoSym is the most fragile model, because relaxation shifts ⟨Z⟩\langle Z\rangle directly, but not ⟨X⟩\langle X\rangle.

Beyond a typical device every model trained without noise degrades towards a constant prediction: relaxation and the asymmetric readout errors shift the output by a constant that eventually exceeds its variation from image to image. A model trained under noise learns to compensate the shift.

Discussion

Why invariance prevents overfitting.

The four models share the gates, the angles and, up to a change of basis, the DLA; none of them has vanishing gradients (Figure 9), and they fit their training images comparably (Figure 6a–Figure 6d). The usual explanations of a difference in generalization, a more expressive circuit [12, 14] or one that is harder to train [7, 8, 10], therefore do not apply. Nor does the size of the function class: NonEquiv-Twirled and Equiv-NoSym combine a similar number of features, yet the first overfits as little as Equiv and the second as much as NonEquiv, whose class is almost twice as large. What decides the overfitting is whether the model is invariant. An invariant model trained on NN images is effectively trained on their whole orbits (Sec. II C), so the pose of an image carries no information for it. A non-invariant model must infer from NN images that the pose does not matter, and its features, most of which change with the pose, let it memorize the poses it has seen instead.

A partial invariance is not enough: NonEquiv is invariant under the row reflection, so, applied to the subgroup {e,tx}\{e,t_x\}, the same argument gives that it effectively sees two of the eight poses of every image, yet it overfits as much as Equiv-NoSym, which has no residual symmetry. The upright control confirms the mechanism: once the pose stops varying, the same non-invariant circuit overfits far less. The result does not depend on the data set. It holds on a benchmark, on a synthetic task and on two scientific data sets collected for other purposes, in which the random orientation is a property of the objects themselves.

What the equivariant circuit adds.

An equivariant circuit alone brings no benefit to generalization: with a non-equivariant readout it overfits like a generic circuit. Its role is to give an invariant model at no extra cost, whereas twirling multiplies the number of circuit evaluations by ∣G∣|G|, and to fix which invariant model: the function classes of Equiv and NonEquiv-Twirled are nearly disjoint and of very different size. Which class is better is a property of the task, as for any choice of model, and both avoid overfitting. One cost of the equivariant circuit is that it sometimes fails to train at large depth, as does Equiv-NoSym, which shares it; we have not identified the cause (Appendix C).

c. When the symmetry does not help. The advantage requires the symmetry to be a property of the data. On upright digits a fixed orientation is informative, and an invariant model discards it: the symmetry restricts what a model can learn, and the restriction helps only when what it excludes carries no information about the label. The advantage also requires a task that the equivariant circuit can represent. In a first version of the synthetic task, with small shapes on a strong background, no circuit learned anything (Appendix B3).

d. Shortcuts. Shortcut learning [37, 43] is an extreme form of the same mechanism: the model relies on a property of the training images that does not determine the label in general. A cue that the transformations of D4D_4 move is invisible to an invariant model, which learns the content of the images regardless of how easy the cue is; a cue that the group does not move is learned by every model. The invariance thus protects against exactly the cues that the group moves. Among the non-invariant models, which cue is preferred depends on the readout.

e. Hardware noise. Two facts explain why the advantage survives realistic noise. The invariant models do not rely on fine details of the training images, which the noise blurs; and since all the channels act on single qubits, the noise breaks the symmetry of Equiv only through non-unital relaxation and the differences between qubits, by 1–2% of the output. The readout matters here too: an observable that relaxation shifts directly, such as ZZ, makes a model fragile. Beyond a typical device, the bias of the noisy output makes it necessary to train on the device, or at least to recalibrate the decision threshold, for every model.

Conclusion

We have investigated how incorporating task symmetries affects the generalization and robustness of quantum neural networks for image classification. Our study compares four models with matched circuit layouts, parameter counts and parameter-sharing patterns, using variational circuits with equivalent dynamical Lie algebras. This provides a controlled setting for assessing the benefits of symmetry constraints across four tasks involving handwritten digits, synthetic shapes, galaxy morphology and red blood cells.

In the random-pose setting, the invariant models achieve smaller generalization gaps and higher average test accuracy on all four tasks, despite being trained on only a few tens of images. The controls on image orientation support the interpretation that these constraints improve generalization by making the predictions insensitive to pose-dependent patterns in the training sample. The shortcut experiments demonstrate a related benefit: the invariant models resist artificial cues that distinguish the classes through their orientation or through positions related by the task symmetries. This protection is specific to the imposed symmetry, since cues that are themselves invariant can still be learned as shortcuts. The observed advantage also persists in simulations with noise parameters representative of superconducting processors, including thermal relaxation, depolarizing noise and readout errors.

The relative performance of models respecting the same symmetry nevertheless depends on the task, highlighting the importance of which invariant functions a chosen architecture can represent. Architectural constraints and explicit group averaging also differ in the number of circuit evaluations needed to obtain a prediction. These results support using known task symmetries to guide quantum model design when labeled data are limited, while accounting for representational flexibility and evaluation cost. They also illustrate the value of controlled benchmarks that match the parametrization, gate layout and algebraic structure of the circuits being compared. Extending this analysis to larger symmetry groups, alternative encodings and deeper circuits would clarify how these benefits interact with trainability and computational resources beyond the regimes studied here.

Data and code availability

MNIST [34], Galaxy10 SDSS [44] and erythrocytesIDB [41] are publicly available, and the synthetic task is generated by the code. The code used to generate the data, train and evaluate the models, compute the dynamical Lie algebras and produce every figure and table, the results of every run, and supplementary tables with every setting not reported here (all numbers of layers and of training images, the shortcuts at every NN, the noise scenarios at every noise scale and the pilot runs) are available at https://github.com/giuseppedambrousio/eqnn. The code is released under the GNU General Public License v3.0 or later, and the results, figures and tables under the Creative Commons Attribution-ShareAlike 4.0 International license.

Acknowledgments

We acknowledge support from INFN through projects “QUANTUM” and “NPQCD”, and from the Italian funding within the “Budget MUR - Dipartimenti di Eccellenza 2023-2027” - Quantum Sensing and Modelling for One-Health (QuaSiModO). We acknowledge support from Projects PN-RIC Q-SUD (Q-SUD B99H26000380007) and PN-RIC QUANTAS (SYNERGIA B99H26000410007). We acknowledge computational resources provided by the University of Bari and the INFN cluster ReCaS [45]. GG acknowledges support from the Italian National Group of Mathematical Physics (GNFMI-IndAM) and from the project “Patto territoriale sistema universitario pugliese” (PATTI).

AI tools usage. The authors used AI tools (ChatGPT and Claude) during the preparation of this manuscript to assist with code writing, proofreading and language editing of the manuscript. All resulting text was reviewed and edited by the authors, who take full responsibility for the content of the work.

SDSS data acknowledgments. This work uses images from the Sloan Digital Sky Survey. Funding for the SDSS and SDSS-II has been provided by the Alfred P. Sloan Foundation, the Participating Institutions, the National Science Foundation, the U.S. Department of Energy, the National Aeronautics and Space Administration, the Japanese Monbukagakusho, the Max Planck Society, and the Higher Education Funding Council for England. The SDSS Web Site is http://www.sdss.org/. The SDSS is managed by the Astrophysical Research Consortium for the Participating Institutions. The Participating Institutions are the American Museum of Natural History, Astrophysical Institute Potsdam, University of Basel, University of Cambridge, Case Western Reserve University, University of Chicago, Drexel University, Fermilab, the Institute for Advanced Study, the Japan Participation Group, Johns Hopkins University, the Joint Institute for Nuclear Astrophysics, the Kavli Institute for Particle Astrophysics and Cosmology, the Korean Scientist Group, the Chinese Academy of Sciences (LAMOST), Los Alamos National Laboratory, the Max-Planck-Institute for Astronomy (MPIA), the Max-Planck-Institute for Astrophysics (MPA), New Mexico State University, Ohio State University, University of Pittsburgh, University of Portsmouth, Princeton University, the United States Naval Observatory, and the University of Washington.

Dynamical Lie algebra, function classes and gradients

a. Dynamical Lie algebra. The DLA is generated by the gates of one layer. The two rotations of a pair share one angle and act simultaneously, so the pair is a single gate with the summed generator. For the equivariant circuit the generators are Xa+Xa′X_a + X_{a'} (a=0,…,3a = 0,\ldots,3, a′=a+n/2a' = a + n/2) and YaYbYa′Yb′Y_aY_bY_{a'}Y_{b'} for the five blocks $(a,b) \in

TaskSourceClass 0Class 1Test images
MNISThandwritten digits [34], random posedigit 4digit 51000
Syntheticgeneratedfilled squarethin cross1000
Galaxy10SDSS images [46], Galaxy Zoo labels [40, 47], random poseface-on diskedge-on disk630
Blood cellserythrocytesIDB, sickle cell disease [36, 41], random poseelongated cellother deformed cell212

caption: The four tasks. “Random pose”: every image, in training and in test, is transformed by an element of D4D_4 drawn uniformly at random, independently of the label. The test sets are balanced.

:::

ModelDLAobs.feat.inv.sh.
Equiv569417237237237
Nonequiv-Twirled569127266666614
Nonequiv56914801304314
Equiv-NoSym5691480772237237

Table III. Dimensions of the DLA, of the space of observables MO\mathcal{M}_O (obs.; for Nonequiv-Twirled, the twirled observables), of the function class (feat.), of its invariant part (inv.) and of the features shared with Equiv (sh.).

{(0,1),(2,3),(1,2),(3,0),(0,2)}\{(0,1),(2,3),(1,2),(3,0),(0,2)\}, nine in all; for the nonequivariant circuit they are Xa+Ya′X_a+Y_{a'} and YaYbZa′Zb′Y_aY_bZ_{a'}Z_{b'}. Stacking layers adds no generator, so the DLA does not depend on LL. As is customary, we compute the DLA of the gate set, with an independent angle for every gate. We computed it with the Lie closure of PennyLane (qml.lie_closure), which takes the nested commutators of the generators, expressed as sums of Pauli strings, until no new linearly independent element appears. For both circuits it gives dim⁡g=569\dim\mathfrak{g}=569.

b. Function classes. Since the images are real, every model output is f(x)=xTRe⁡(U†OU)xf(x)=x^{T}\operatorname{Re}(U^\dagger O U)x, so the size of the function class of a model is the dimension of the span of the matrices Re⁡(U†OU)\operatorname{Re}(U^\dagger O U) over its circuits. We estimated it as the numerical rank of these matrices for many random circuits, more than the expected dimension, computed in the Heisenberg picture (and group-averaged for Nonequiv-Twirled); the singular values above and below the rank differ by at least thirteen orders of magnitude. With an independent angle per gate the ranks are 237 for Equiv, 666 for Nonequiv-Twirled, 772 for Equiv-NoSym and 1304 for Nonequiv. With the angles shared as in the trained circuits they are smaller by at most four, less than 0.5% of each class; they are reached with four layers, and two layers already reach almost all of them.

c. Gradients. To check that none of the models is harder to train than the others, we computed the gradient of the training loss (C1) in the regime of our training: angles drawn uniformly in [−0.1,0.1][-0.1,0.1], as at initialization, on the training images of the main settings (experiments/gradient_variance.py and gradient_histograms.py). Figure 9 shows the variance of the partial derivatives as a function of the depth, and the distribution of their moduli in the main settings. No model has vanishing gradients. The variance grows with the depth for all four models, and the moduli of the partial derivatives are spread over two to three orders of magnitude around medians of over 0.1. Nonequiv and Nonequiv-Twirled have almost the same gradients, and Equiv comparable or larger ones; Equiv-NoSym has the smallest, by up to a factor of six on Galaxy10. Since Adam takes steps that do not depend on the scale of the gradient, these differences do not hinder training, and all four models reach comparable training accuracies. They do not follow the overfitting either: Equiv-NoSym overfits as much as Nonequiv with smaller gradients, and Nonequiv-Twirled is little as Equiv with the gradients of Nonequiv.

Tasks

Data sets

Table Table II summarizes the four tasks. All the images are converted to grayscale, brought to 16×1616 \times16 pixels and ℓ2\ell_2-normalized. The color images of Galaxy10 and erythrocytesIDB are first converted to their luminance, 0.299R+0.587G+0.114B0.299R+0.587G+0.114B. The images are then downsampled from their original size (28×2828 \times28 pixels for MNIST, 69×6969 \times69 for Galaxy10 and 80×8080 \times80 for the blood cells) by bilinear interpolation with antialiasing, so that every pixel of the 16×1616 \times16 image is a weighted average of the original pixels around it, and fine details are smoothed rather than aliased. The synthetic images are generated directly at 16×1616 \times16 pixels.

a. MNIST. The training pool is the first 50 000 images of the training split, and the test set a fixed balanced set of 1000 images of the test split. Every image is transformed once by a random element of D4D_4; the control uses the upright images.

b. Synthetic task. Each image contains a filled square of side 8–14 pixels (class 0) or a plus-shaped cross, one pixel thick, with arms of total length 9–15 pixels (class 1). The size, the position (uniform over the placements inside the frame) and the intensity (uniform in [0.5,1][0.5,1]) of the shape are random, and the background pixels are ∣ϵ∣|\epsilon| with ϵ∼N(0,0.052)\epsilon\sim\mathcal{N}(0,0.05^2). Both shapes are symmetric under D4D_4 about their center and the set of positions is invariant, so the distribution of each class is invariant.

c. Galaxy10. Galaxy10 SDSS [44] contains 21 785 color images of galaxies from the Sloan Digital Sky Survey [46], in ten morphological classes from the votes of Galaxy Zoo [40, 47]. We distinguish face-on disk galaxies without spiral arms (class 0 of the data set) from edge-on disk galaxies with a rounded bulge (class 4): a round disk from an elongated one, at any orientation. We use the luminance of the images. The data set has no official split, so we split it once at random into 80% for training and 20% for testing; the test set contains all the 315 edge-on galaxies of the test split and as many face-on ones.

d. Red blood cells. erythrocytesIDB [36, 41] contains 629 images of single red blood cells from peripheral blood smears of patients with sickle cell disease, classified by a specialist as circular, elongated (the sickle cells) or other (deformed without being sickle-shaped). We distinguish the elongated cells from the other ones. Since the amplitude encoding normalizes the image, the amplitude must lie on the cell rather than on the background: the cells are dark on a bright background, so we use the inverted luminance minus the background level (the median of the border pixels), clipped at zero. The data set is small, and we split every class once at random into two halves, for training and for testing.

Cues

The cues are added to every training image, after the random pose, in the frame of the image. The corner watermark is a 2×22 \times2 patch near a corner, at a reference position for class 0 and at the position rotated by 90∘90^\circ for class 1; its amplitude is set so that it carries 20% of the squared norm of every image, so that neither its intensity nor its weight in the encoded state depends on the class. The brightness gradient multiplies the image by a ramp that changes by a factor of ten along an oblique direction, rotated by 90∘90^\circ for class 1. The center/corner mark is a patch of twice the maximal intensity of the image, at the center of the frame for class 0 and in its four corners for class 1. Since the content of the images is in a random pose, the orbit of a cued image does not reveal which of the two watermarks or gradients it carries.

Choice of the tasks

The parameters of the synthetic task and the pairs of classes of Galaxy10 and erythrocytesIDB were chosen in short pilot runs with Equiv and NonEquiv only, at two depths and with three seeds. The pilot results are in the repository.

a. Synthetic task. The first version had small shapes on a strong background noise, and no circuit learned it. At the start of training the circuits are close to the identity, and the observable (X0+X4)/2(X_0 + X_4)/2 measures the correlation between each pixel and the pixel half an image away in the same row or column, since qubits 0 and 4 hold the most significant bits of the indices. A small shape gives no such correlation, so the training starts without any signal about it. We therefore made the shapes large and the noise weak, and compared a dozen variants that differ in the range of the sizes, the thickness of the cross and the strength of the noise. Equiv overfit less than NonEquiv in all of them, and was more accurate in all but one (hollow squares), where the two circuits were equally accurate. Smaller shapes increased the overfitting of both circuits, and a thicker cross or more noise reduced the difference between them. We chose the variant with the thinnest cross, on which Equiv is most accurate; the variants closest to it give similar results.

b. Galaxy10. We considered five pairs of the classes of Galaxy10 [44] whose distinction rests on the shape of the galaxy and not on its orientation (Figure 11): face-on disk without spiral arms (class 0) against edge-on disk with a rounded bulge (class 4); smooth, completely round galaxies (class 1) against classes 0 and 4; smooth, in-between round galaxies (class 2) against class 4; and class 1 against class 2.

Figure 11. The four classes of Galaxy10 considered in the pilot runs; the task of the paper is class 0 against class 4. Left: the first five images of each class in the data set (69 × 69 pixels, color). Right: the same images as seen by the circuits, converted to luminance and downsampled to 16 × 16 pixels.

c. Red blood cells. Of the three possible pairs, circular against elongated cells is so easy that both circuits learn it without overfitting, so there is little overfitting for the invariance to prevent; circular against other cells is separated poorly by both circuits. We chose the remaining pair, elongated against other cells.

Training and evaluation

Simulation and loss. The circuits are simulated exactly, in double precision, with PennyLane (default.qubit) and backpropagation in PyTorch [39, 48]. The state ∣ψ(x)⟩|\psi(x)\rangle is prepared exactly. We minimize the logistic loss

L=1N∑i=1Nlog⁡(1+e−βy~if(xi)),y~i=2yi−1,(35)\mathcal{L} = \frac{1}{N}\sum_{i=1}^{N} \log\left(1+e^{-\beta\tilde{y}_i f(x_i)}\right), \qquad\tilde{y}_i = 2y_i-1, \tag*{(35)}

with β=20\beta= 20. Since ff is bounded in [−1,1][-1,1], with β=1\beta= 1 the margin y~if(xi)\tilde{y}_i f(x_i), positive for a correctly classified image, would stay in the region where log⁡(1+e−z)≈log⁡2−z/2\log(1+e^{-z}) \approx\log2-z/2 is almost linear, and every image would contribute almost equally to the gradient, whatever its margin. With β=20\beta= 20 the loss saturates for images classified correctly with a margin larger than a few times 1/β1/\beta, and grows linearly for misclassified ones, as a smooth hinge loss; a fixed scale of this kind is commonly applied to bounded outputs in classical classifiers [49]. The scale enters the training only, not the prediction.

Optimization. The angles are initialized uniformly in [−0.1,0.1][-0.1,0.1] and trained with Adam [50], learning rate 0.05 and decay rates 0.5 and 0.999 for the first and second moment estimates, for 70 epochs of ten mini-batches of N/10N/10 images (four images for N=40N = 40, two for N=20N = 20). A seed fixes the training images, the random poses, the initial angles and the order of the mini-batches. The learning curves (Figure 10) evaluate the test accuracy on 250 test images every seven epochs.

Grid and statistics. We ran L∈{1,2,4,8}L \in\{1, 2, 4, 8\} layers (L∈{1,2,4}L \in\{1, 2, 4\} on the synthetic task) and N∈{10,20,40,80}N \in\{10, 20, 40, 80\} training images with five seeds each, and added ten seeds in the main settings, with L∈{2,4}L \in\{2, 4\}. The paper reports L=4L = 4; the results with L=2L = 2 are similar and are in the repository. Error bars are standard errors over the seeds. Comparisons use the seeds shared by all the compared models and the two-sided Wilcoxon signed-rank test; we also report the number of seeds in which a difference is positive. The experiments on shortcuts use five seeds per setting.

Choice of the main settings. The main settings were chosen after running the whole grid, by three criteria. First, enough layers for the non-equivariant circuit to fit its training images: with one layer it fits them poorly, and the comparison would partly measure underfitting rather than overfitting (Figure 12i–l). Second, a depth at which the equivariant circuit trains reliably, which excludes eight layers (see below). Third, a number of images at which the training accuracies of the four models are close, so that the gap compares generalization rather than fitting: N=40N = 40 on MNIST, and N=20N = 20 on the synthetic task, which is easier to fit. On Galaxy10 and on the blood cells, whose pairs of classes were chosen at N=40N = 40, we kept that value.

Figure 12. Dependence on the number of training images and on the depth. a–d Test accuracy and e–h generalization gap as functions of the number of training images, with L=4L = 4; the yellow dotted line in e is NonEquiv trained and tested on upright digits. i–l Generalization gap as a function of the number of layers, with the NN of the main setting. Mean and standard error over five seeds (15 in the main settings).

Failures to train. We call a failure a run whose training accuracy stays at or below 0.6. Failures occur mostly with eight layers, where EQUiV fails in about a third of the runs on MNIST, Galaxy10 and the blood cells, and EQUIV-NOSYM, which shares its circuit, in half of the runs on Galaxy10 and the blood cells; NonEQUIV and NonEQUIV-TWIRLED rarely fail, except NonEQUIV-TWIRLED with eight layers on Galaxy10 (a quarter of the runs). In the main settings there are four failures in total (one of EQUIV on the blood cells, two of EQUIV-NOSYM on Galaxy10 and one of NonEQUIV-TWIRLED on the synthetic task). The counts are in the repository.

Number of images and depth

Figure 12 shows how the results depend on the number of training images and on the depth. The gap of the non-invariant models is largest with few images and shrinks as NN grows, but stays above that of the invariant models, and their test accuracy approaches that of the invariant models only slowly: on MNIST, NonEquiv trained on 80 images is still less accurate than Equiv trained on 40. On the synthetic task Equiv-NoSym, whose function class contains that of Equiv, eventually finds functions close to the invariant ones as NN grows. With one layer the non-equivariant circuit fits its training images poorly; on MNIST, on the synthetic task and on the blood cells its gap is then smaller, and grows with LL, as the depth gives it the capacity to memorize. On Galaxy10 it overfits already with one layer. The gap of the invariant models does not grow with the depth. Comparison 1 over the whole grid, with the values of every setting, is in the repository.

Learning curves

Figure 10 shows how the gap forms during training in the main settings. The test accuracy of the non-invariant models stops growing after a few epochs while their training accuracy keeps rising, whereas for the invariant models the two grow together.

Values of the main settings

Table IV gives the paired comparisons of Figure 6e–Figure 6h; the training accuracy, test accuracy and generalization gap of Figure 6a–Figure 6d are in the repository.

Noise model

Channels

The noise model consists of single-qubit channels with the typical parameters of superconducting processors (Table V, Figure 13). The gates are those of Figure 2, not compiled into native gates, and scheduled as soon as possible: a gate starts when all its qubits are free, and the qubits exchanged by the swap start and end their gates at the same times. After every gate, each of its qubits undergoes the depolarizing channel

Figure 13. The noise model on one block of the circuit. After every gate, each of its qubits undergoes depolarizing noise and thermal relaxation for the duration of the gate (N\mathcal{N}); qubits that wait relax for the waiting time (R\mathcal{R}); the measured qubits have asymmetric readout errors. All the channels act on single qubits, with parameters that differ from qubit to qubit. The four-qubit rotation RPPPP(η3)R_{PPPP}(\eta_3) acts on qubits 0, 1, 4 and 5; it is drawn as two boxes joined by a thick vertical line, which crosses qubits 2 and 3 without acting on them.

Dλ(ρ)=(1−λ)ρ+λI2Tr⁡ρ(36)\mathcal{D}_{\lambda}(\rho) = (1-\lambda)\rho+ \lambda\frac{\mathbb{I}}{2}\operatorname{Tr}\rho \tag*{(36)}

and then thermal relaxation for the duration tt of the gate [51]: amplitude damping with γ=1−e−t/T1\gamma= 1 - e^{-t/T_1}, followed by pure dephasing with γϕ=1−e−2t/Tϕ\gamma_{\phi} = 1 - e^{-2t/T_{\phi}}, where 1/Tϕ=1/T2−1/(2T1)1/T_{\phi} = 1/T_2 - 1/(2T_1), so that the coherences decay as e−t/T2e^{-t/T_2} (the Kraus operators are given below). As in the device noise models of Qiskit Aer [52], the strength λ\lambda is chosen so that the average infidelity of the noise on the qubit equals the error ϵ\epsilon of the gate. A single-qubit channel with entanglement fidelity FeF_e has average infidelity 1−(2Fe+1)/3=λ/21 - (2F_e + 1)/3 = \lambda/2 for Dλ\mathcal{D}_{\lambda}; since to first order the infidelities of the two channels add, λ=2(ϵ−rrel)\lambda= 2(\epsilon- r_{\rm rel}), with rrelr_{\rm rel} the average infidelity of the relaxation (and λ=0\lambda= 0 if rrel≥ϵr_{\rm rel} \ge\epsilon). A qubit that waits, for the other qubits of a gate or for the readout, relaxes for the waiting time. The measured qubits have asymmetric readout errors, which turn a measured ⟨P⟩\langle P\rangle into (1−p10−p01)⟨P⟩+p01−p10(1-p_{10}-p_{01})\langle P\rangle+p_{01}-p_{10}. The parameters of each qubit are drawn once from log-normal distributions around the typical values, and the global scale ss multiplies every error rate and relaxation rate. We use log-normal distributions because these parameters are positive and vary from qubit to qubit by multiplicative factors rather than by fixed amounts; the median and the standard deviation of the logarithm (Table V) fix the typical value and the relative spread. The state preparation is ideal, and for NonEquiv-Twirled the transformations of the input remain classical and noiseless.

b. Noise of the four-qubit rotations. A rotation RP(η)=e−iηP/2R_{P}(\eta) = e^{-i\eta P/2} on four qubits, with PP a Pauli string such as YjYkYj+n/2Yk+n/2Y_{j}Y_{k}Y_{j+n/2}Y_{k+n/2}, is applied as an ideal unitary, followed on each of its four qubits by the same single-qubit channels as a single-qubit gate, with the parameters of its usual compilation into two-qubit gates: an error of 9×10−39 \times10^{-3} per qubit (three two-qubit gates of error 3×10−33 \times10^{-3}) and a duration of 480 ns (six layers of two-qubit gates). The model therefore contains no correlated multi-qubit error: the noise of a four-qubit gate is the product of four independent single-qubit channels, each with the parameters of its own qubit.

c. Heisenberg picture. The noisy circuit is a sequence, in time order, of gates and single-qubit channels, Φ=ET∘⋯∘E1\Phi= \mathcal{E}_{T} \circ\cdots\circ\mathcal{E}_{1}, and the output of a model is

f(x)=Tr⁡ ⁣[O Φ(ρ(x))]=Tr⁡ ⁣[Φ†(O) ρ(x)],Φ†=E1†∘⋯∘ET†(37)f(x) = \operatorname{Tr}\!\left[O\,\Phi(\rho(x))\right] = \operatorname{Tr}\!\left[\Phi^{\dagger}(O)\,\rho(x)\right], \qquad \Phi^{\dagger} = \mathcal{E}_{1}^{\dagger} \circ\cdots\circ\mathcal{E}_{T}^{\dagger} \tag*{(37)}

where E†\mathcal{E}^{\dagger}, defined by Tr⁡[A E(ρ)]=Tr⁡[E†(A) ρ]\operatorname{Tr}[A\,\mathcal{E}(\rho)] = \operatorname{Tr}[\mathcal{E}^{\dagger}(A)\,\rho], is the adjoint of E\mathcal{E}. We start from the measured observable, in which the readout errors are already included (each measured Pauli operator PP is replaced by (1−p10−p01)P+(p01−p10)I(1-p_{10}-p_{01})P + (p_{01}-p_{10})I), and apply the adjoints of the operations in reverse time order. The adjoints are:

  • for a gate UU, E†(A)=U†AU\mathcal{E}^{\dagger}(A) = U^{\dagger}AU;

  • for a channel with Kraus operators {Kj}\{K_j\}, E†(A)=∑jKj†AKj\mathcal{E}^{\dagger}(A) = \sum_j K_j^{\dagger} A K_j; amplitude damping has K0=diag⁡(1,1−γ)K_0 = \operatorname{diag}(1,\sqrt{1-\gamma}) and K1=γ∣0⟩⟨1∣K_1 = \sqrt{\gamma}\lvert0\rangle\langle1\rvert, pure dephasing K0=diag⁡(1,1−γϕ)K_0 = \operatorname{diag}(1,\sqrt{1-\gamma_{\phi}}) and $K_1 = \sqrt{\gamma_{\phi}}\lvert 1\rangle\langle 1\rvert;

  • the depolarizing channel on qubit ww is self-adjoint, Et(A)=(1−λ)A+λ12⊗Tr⁡wA\mathcal{E}^{t}(A)=(1-\lambda)A+\lambda\frac{\mathbb{1}}{2}\otimes\operatorname{Tr}_{w}A.

d. Symmetry breaking. A depolarizing channel of the same strength on every qubit would keep Equiv exactly invariant: it commutes with every single-qubit unitary, hence with the reflections, and the swap maps the noise on a qubit to the same noise on its partner. Realistic noise breaks the symmetry in two ways. Amplitude damping is not unital and does not commute with XX, which breaks the reflections; and the qubits exchanged by the swap have different parameters, which breaks the transposition. Both effects are small (Table VI).

Qubitssstransp.rot. 90°refl.rot. 180°
identical0.250.0%0.2%0.2%0.3%
10.0%0.6%0.6%0.8%
40.0%0.4%0.4%0.6%
dispersion0.250.4%0.5%0.2%0.3%
11.5%1.7%0.6%0.8%
42.3%2.4%0.3%0.4%

Table VI. Breaking of the symmetry of Equiv by the noise: mean of ∣f(x)−f(RX(g)x)∣|f(x) - f(R_X(g)x)| over 200 MNIST test images in random pose, relative to the mean of ∣f(x)∣|f(x)|, for angles drawn uniformly in [−0.5,0.5][-0.5, 0.5] (ten draws), L=4L = 4, with identical qubits or with the dispersion of Table V. With identical qubits the transposition remains an exact symmetry.

References

References

  1. [1]J. Preskill, Quantum computing in the NISQ era and beyond, Quantum 2, 79 (2018).DOI
  2. [2]M. Cerezo, A. Arrasmith, R. Babbush, S. C. Benjamin, S. Endo, K. Fujii, J. R. McClean, K. Mitarai, X. Yuan, L. Cincio, and P. J. Coles, Variational quantum algorithms, Nature Reviews Physics 3, 625 (2021).DOI
  3. [3]M. Schuld and F. Petruccione, Machine Learning with Quantum Computers (Springer, Cham, 2021).DOI
  4. [4]V. Havlíček, A. D. Córcoles, K. Temme, A. W. Harrow, A. Kandala, J. M. Chow, and J. M. Gambetta, Supervised learning with quantum-enhanced feature spaces, Nature 567, 209 (2019).DOI
  5. [5]M. Schuld, R. Sweke, and J. J. Meyer, Effect of data encoding on the expressive power of variational quantum-machine-learning models, Physical Review A 103, 032430 (2021).DOI
  6. [6]R. LaRose and B. Coyle, Robust data encodings for quantum classifiers, Physical Review A 102, 032420 (2020).DOI
  7. [7]J. R. McClean, S. Boixo, V. N. Smelyanskiy, R. Babbush, and H. Neven, Barren plateaus in quantum neural network training landscapes, Nature Communications 9, 4812 (2018).DOI
  8. [8]M. Ragone, B. N. Bakalov, F. Sauvage, A. F. Kemper, C. Ortiz Marrero, M. Larocca, and M. Cerezo, A Lie algebraic theory of barren plateaus for deep parameterized quantum circuits, Nature Communications 15, 7172 (2024).DOI
  9. [9]E. Fontana, D. Herman, S. Chakrabarti, N. Kumar, R. Yalovetzky, J. Heredge, S. H. Sureshbabu, and M. Pistoia, Characterizing barren plateaus in quantum ansätze with the adjoint representation, Nature Communications 15, 7171 (2024).DOI
  10. [10]M. Larocca, S. Thanaslip, S. Wang, K. Sharma, J. Biamonte, P. J. Coles, L. Cincio, J. R. McClean, Z. Holmes, and M. Cerezo, Barren plateaus in variational quantum computing, Nature Reviews Physics 7, 174 (2025).DOI
  11. [11]S. Wang, E. Fontana, M. Cerezo, K. Sharma, A. Sone, L. Cincio, and P. J. Coles, Noise-induced barren plateaus in variational quantum algorithms, Nature Communications 12, 6961 (2021).DOI
  12. [12]M. C. Caro, H.-Y. Huang, M. Cerezo, K. Sharma, A. Sornborger, L. Cincio, and P. J. Coles, Generalization in quantum machine learning from few training data, Nature Communications 13, 4919 (2022).DOI
  13. [13]L. Banchi, J. Pereira, and S. Pirandola, Generalization in quantum machine learning: A quantum information standpoint, PRX Quantum 2, 040321 (2021).DOI
  14. [14]A. Abbas, D. Sutter, C. Zoufal, A. Lucchi, A. Figalli, and S. Woerner, The power of quantum neural networks, Nature Computational Science 1, 403 (2021).DOI
  15. [15]E. Gil-Fuster, J. Eisert, and C. Bravo-Prieto, Understanding quantum machine learning also requires rethinking generalization, Nature Communications 15, 2277 (2024).
  16. [16]J. Bowles, S. Ahmed, and M. Schuld, Better than classical? The subtle art of benchmarking quantum machine learning models, arXiv preprint arXiv:2403.07059 (2024).arxiv.org/abs/2403.07059
  17. [17]T. Cohen and M. Welling, Group equivariant convolutional networks, in Proceedings of the 33rd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 48 (2016) pp. 2990–2999.arxiv.org/abs/1602.07576
  18. [18]M. M. Bronstein, J. Bruna, T. Cohen, and P. Veličković, Geometric deep learning: Grids, groups, graphs, geodesics, and gauges, arXiv preprint arXiv:2104.13478 (2021).arxiv.org/abs/2104.13478
  19. [19]I. Cong, S. Choi, and M. D. Lukin, Quantum convolutional neural networks, Nature Physics 15, 1273 (2019).DOI
  20. [20]M. Larocca, F. Sauvage, F. M. Sbahi, G. Verdon, P. J. Coles, and M. Cerezo, Group-invariant quantum machine learning, PRX Quantum 3, 030341 (2022).DOI
  21. [21]J. J. Meyer, M. Mularski, E. Gil-Fuster, A. A. Mele, F. Arzani, A. Wilms, and J. Eisert, Exploiting symmetry in variational quantum machine learning, PRX Quantum 4, 010328 (2023).DOI
  22. [22]A. Skolik, M. Cattelan, S. Yarkoni, T. Bäck, and V. Dunjko, Equivariant quantum circuits for learning on weighted graphs, npj Quantum Information 9, 47 (2023).arxiv.org/abs/2205.06109
  23. [23]H. Zheng, Z. Li, J. Liu, S. Strelchuk, and R. Kondor, Speeding up learning quantum states through group equivariant convolutional quantum ansätze, PRX Quantum 4, 020327 (2023).DOI
  24. [24]F. Sauvage, M. Larocca, P. J. Coles, and M. Cerezo, Building spatial symmetries into parameterized quantum circuits for faster training, Quantum Science and Technology 9, 015029 (2024).
  25. [25]L. Schatzki, M. Larocca, Q. T. Nguyen, F. Sauvage, and M. Cerezo, Theoretical guarantees for permutation-equivariant quantum neural networks, npj Quantum Information 10, 12 (2024).DOI
  26. [26]M. Ragone, P. Braccia, Q. T. Nguyen, L. Schatzki, P. J. Coles, F. Sauvage, M. Larocca, and M. Cerezo, Representation theory for geometric quantum machine learning, arXiv preprint arXiv:2210.07980 (2022).arxiv.org/abs/2210.07980
  27. [27]Q. T. Nguyen, L. Schatzki, P. Braccia, M. Ragone, P. J. Coles, F. Sauvage, M. Larocca, and M. Cerezo, Theory for equivariant quantum neural networks, PRX Quantum 5, 020328 (2024).DOI
  28. [28]J. M. Kübler, S. Buchholz, and B. Schölkopf, The inductive bias of quantum kernels, in Advances in Neural Information Processing Systems, Vol. 34 (2021) arXiv:2106.03747.arxiv.org/abs/2106.03747
  29. [29]M. T. West, J. Heredge, M. Sevior, and M. Usman, Provably trainable rotationally equivariant quantum machine learning, PRX Quantum 5, 030320 (2024).DOI
  30. [30]S. Y. Chang, M. Grossi, B. Le Saux, and S. Vallercosa, Approximately equivariant quantum neural network for p4m group symmetries in images, in 2023 IEEE International Conference on Quantum Computing and Engineering (QCE) (IEEE, 2023) pp. 229–235.DOI
  31. [31]M. T. West, M. Sevior, and M. Usman, Reflection equivariant quantum neural networks for enhanced image classification, Machine Learning: Science and Technology 4, 035027 (2023).DOI
  32. [32]P. S. S. Sein, M. Cañizo, and R. Orús, Image classification with rotation-invariant variational quantum circuits, Phys. Rev. Res. 7, 013082 (2025).
  33. [33]M. Larocca, N. Ju, D. García-Martín, P. J. Coles, and M. Cerezo, Theory of overparametrization in quantum neural networks, Nature Computational Science 3, 542 (2023).DOI
  34. [34]Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE 86, 2278 (1998).
  35. [35]H. W. Leung and J. Bovy, Deep learning of multi-element abundances from high-resolution spectroscopic data, Monthly Notices of the Royal Astronomical Society 483, 3255 (2019).DOI
  36. [36]M. González-Hidalgo, F. A. Guerrero-Peña, S. Herold-García, A. Jaume-i-Capó, and P. D. Marrero-Fernández, Red blood cell cluster separation from digital images for use in sickle cell disease, IEEE Journal of Biomedical and Health Informatics 19, 1514 (2015).
  37. [37]B. Geirhos, J.-H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, and F. A. Wichmann, Shortcut learning in deep neural networks, Nature Machine Intelligence 2, 665 (2020).DOI
  38. [38]X.-W. Yao, H. Wang, Z. Liao, M.-C. Chen, J. Pan, J. Li, K. Zhang, X. Lin, Z. Wang, Z. Luo, W. Zheng, J. Li, M. Zhao, X. Peng, and D. Suter, Quantum image processing and its application to edge detection: Theory and experiment, Physical Review X 7, 031041 (2017).
  39. [39]V. Bergholm, J. Isaac, M. Schuld, C. Gogolin, S. Ahmed, V. Ajith, et al., PennyLane: Automatic differentiation of hybrid quantum-classical computations, arXiv preprint arXiv:1811.04968 (2018).arxiv.org/abs/1811.04968
  40. [40]C. J. Lintott, K. Schawinski, A. Slosar, K. Land, S. Bamford, D. Thomas, M. J. Raddick, R. C. Nichol, A. Szalay, D. Andreescu, P. Murray, and J. Vandenberg, Galaxy Zoo: morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey, Monthly Notices of the Royal Astronomical Society 389, 1179 (2008).
  41. [41]P. D. Marrero Fernández, G. Coello Said, W. E. Delgado Font, S. Herold García, K. Fernández García, A. Montoya Padrón, M. González-Hidalgo, and A. Jaumei-Capó, erythrocytesIDB (version 2, october 2017), Zenodo (2017).
  42. [42]D. Pomarico, A. Monaco, N. Amoroso, L. Bellantuono, A. Lacalamita, M. La Rocca, T. Maggipinto, E. Pantaleo, S. Tangaro, S. Stramaglia, and R. Bellotti, Emerging generalization advantage of quantum-inspired machine learning in the diagnosis of hepatocellular carcinoma, Discover Applied Sciences 7, 205 (2025).DOI
  43. [43]S. Lapuschkin, S. Wä​ldchen, A. Binder, G. Montavon, W. Samek, and K.-R. Müller, Unmasking Clever Hans predictors and assessing what machines really learn, Nature Communications 10, 1096 (2019).
  44. [44]H. W. Leung and J. Bovy, Galaxy10 SDSS dataset, Part of the astroNN package, https://astronn.readthedocs.io/en/latest/galaxy10sdss.html.
  45. [45]ReCaS-Bari, ReCaS-Bari data centre, https://www.recas-bari.it.
  46. [46]D. G. York et al., The Sloan Digital Sky Survey: Technical summary, The Astronomical Journal 120, 1579 (2000).DOI
  47. [47]C. Lintott, K. Schawinski, S. Bamford, A. Slosar, K. Land, D. Thomas, et al., Galaxy Zoo 1: data release of morphological classifications for nearly 900 000 galaxies, Monthly Notices of the Royal Astronomical Society 410, 166 (2011).DOI
  48. [48]A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, et al., PyTorch: An imperative style, high-performance deep learning library, in Advances in Neural Information Processing Systems, Vol. 32 (2019).arxiv.org/abs/1912.01703
  49. [49]F. Wang, X. Xiang, J. Cheng, and A. L. Yuille, NormFace: L2 hypersphere embedding for face verification, in Proceedings of the 25th ACM International Conference on Multimedia (2017) pp. 1041–1049.DOI
  50. [50]D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, in 3rd International Conference on Learning Representations (ICLR) (2015) arXiv:1412.6980.arxiv.org/abs/1412.6980
  51. [51]M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information: 10th Anniversary Edition (Cambridge University Press, 2010).
  52. [52]A. Javadi-Abhari, M. Treinish, K. Ursulich, C. J. Wood, J. Lishman, J. Gacon, S. Martiel, P. D. Nation, L. S. Bishop, A. W. Cross, B. R. Johnson, and J. M. Gambetta, Quantum computing with Qiskit (2024), arXiv:2405.08810.arxiv.org/abs/2405.08810

Paper details

vlnc:W_wfwu9h7c
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 1

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

ChatGPT, Claude

View statement
Looking for related papers…
Contents