betaFeedback

WorksComputer Science

The Sign Code: The Hidden Binary Nature of Deep Networks

Abstract

Every neuron in a trained network outputs a number that can be separated into its sign (positive or negative) and its magnitude (absolute size). We call the set of all neuron signs the sign code. This raises a key question: does classification depend more on the sign code or on the magnitudes? For overparameterized vision classifiers with sign-preserving activations, the answer is overwhelmingly the sign code. A kk-nearest neighbor classifier operating only on this binary sign code achieves accuracy comparable to the full model across different architectures, activation functions, and datasets. Moreover, replacing all magnitudes in every layer with random noise while keeping the original signs barely changes residual network performance. In contrast, flipping just 20% of the signs reduces accuracy to chance. Models generalize well when same-class inputs share similar sign codes. When this overlap is disrupted by a regularizer, training accuracy remains high, but test performance collapses. A margin decomposition helps explain these results: the sign-only contribution to each logit grows as d\sqrt{d} in network width, dominating the magnitude residual. The natural unit of analysis for these networks may not be through their continuous weight space but instead through their discrete sign codes.

Introduction

ReLU networks admit a clean piecewise-linear decomposition: each input activates a subset of neurons, and within the resulting region the network acts as an affine map. This combinatorial structure has been a productive lens on deep learning, underpinning analyses of linear-region counts [36, 29, 37, 39, 18, 12], spline and max-affine views [2], and gate-and path-based studies of expressivity and generalization [Neyshabur et al., 2015a,b, Fiat et al., 2018, Lakshminarayanan and Singh, 2020, Gonon et al., 2023, Lebeurrier et al., 2026]. But modern networks rarely use ReLU, while SiLU, GELU, and tanh have no piecewise-linear structure to exploit so that the tools built around ReLU do not transfer. We ask whether the combinatorial object underlying these ReLU analyses, which side of each neuron’s hyperplane the input falls on, survives the move to smooth activations, and whether it remains the dominant carrier of classification signal.

For any sign-preserving activation φ\varphi (ReLU, SiLU, GELU, tanh), each neuron output decomposes exactly as hi(x)=Si(x)⋅Mi(x)h_i(x)=S_i(x)\cdot M_i(x), where Si∈{−1,+1}S_i\in\{-1,+1\} is a binary sign and Mi=∣hi∣M_i=|h_i| is a non-negative magnitude (Figure 1a,b). The sign identifies which combinatorial region of representation space the input occupies; the magnitude specifies where within that region. We call the collection {Si(x)}\{S_i(x)\} the sign code of the network, and ask whether it alone carries the essential information for classification. Across architectures, activations, and datasets, our experiments reveal a sharp asymmetry: the sign code is sufficient to recover full-model accuracy and causally dominant under intervention, while magnitudes are largely redundant (Figure 1c).

Figure 1. a) Every neuron in a feedforward network computes a pre-activation zj(ℓ)z_{j}^{(\ell)}, which the activation function maps to hj(ℓ)=φ(zj(ℓ))h_{j}^{(\ell)} = \varphi(z_{j}^{(\ell)}). b) For any sign-preserving activation, this output decomposes as hj=Sj⊙Mjh_j = S_j \odot M_j: a binary sign Sj∈{−1,+1}S_j \in\{-1,+1\} encoding which side of the neuron’s hyperplane the input falls on, and a non-negative magnitude Mj=∣hj∣M_j = |h_j| encoding how far. We test the causal role of each component with two channel-isolating interventions: randomly flipping signs while preserving magnitudes, or replacing magnitudes while enforcing the sign code. c) Flipping just 20% of sign bits collapses accuracy to chance (red); replacing every magnitude in every layer preserves far more classification performance (blue), and is nearly lossless for residual networks. The forward computation is dominated by binary routing decisions rather than raw activation strengths.

Our contributions are:

  1. The sign code is informationally sufficient for classification. A kk-nearest-neighbour classifier on the last-layer binary sign code, with no access to magnitudes, weights, or logits, matches full model accuracy in the overparameterized regime, across architectures, activations, and datasets. We give a formal sufficient certificate for this transition (§ 3 and § 6).

  1. Networks are robust to magnitude perturbations but not to sign code perturbations. We compare two interventions that isolate the sign and magnitude channels: destroying signs while preserving magnitudes is catastrophic, whereas randomizing magnitudes while preserving the clean sign trajectory is much less harmful. We check this asymmetry across four activations and two architecture families (§ 4, Figure 3).

Figure 3. Left (2×2): Accuracy under channel destruction. Red: sign flip (sign code routing destroyed, magnitudes intact). Blue: magnitude replacement with random ∣N(0,1)∣|\mathcal{N}(0,1)| draws (magnitudes destroyed, sign code routing intact). Solid: FFNN; dashed: ResNet; mean ±1\pm1 s.d. over 5 draws. Right: Intervention formulas and summary of what each intervention removes. Routing destruction collapses accuracy to chance by α=0.2\alpha= 0.2 across all conditions; projected replacement of every magnitude preserves far more classification performance, and is nearly lossless for residual networks.

  1. Class-coherent sign geometry is tied to generalization. Models generalize when same-class inputs share similar sign codes. A regularizer that actively repels within-class sign codes maintains high training accuracy while sharply degrading test accuracy, suggesting that class-coherent sign structure is a necessary ingredient for generalization in these models (§ 5).

Taken together, these results suggest that the natural unit of analysis for these networks may not be their continuous weight space but their discrete sign code.

Related Work

A large body of work studies ReLU networks through their input-space partitions, counting linear regions and bounding expressive gains from depth [36, 29, 37, 39, 18, 12] or interpreting deep networks as input-dependent affine maps indexed by activation patterns [2]. Binary activation patterns have been examined as hash representations [19], learned via specialised objectives [35], and used for verification [Gong et al., 2023]; on the theoretical side, path-norm, path-kernel, and gate-decoupling analyses elevate gates to a primary object of study [Neyshabur et al., 2015a,b, Lakshminarayanan and Singh, 2020, Fiat et al., 2018, Gonon et al., 2023, Lebluer et al., 2026]. The gate-decoupling line is closest to our work: Fiat et al. [ [10] separates gating from linearity and Lakshminarayanan and Singh [ [26] builds kernels from the active subnetwork. Both treat gates as a useful analytical device for ReLU; we make three stronger claims: informational sufficiency, causal dominance under intervention, and a link between sign geometry and generalization (§ 3 to § 5). We establish them across all sign-preserving activations, including smooth ones with no piecewise-linear structure. Neural collapse [34, 17] describes convergence of continuous features toward class means; our results hold outside this regime (see Section B.2 for a thorough comparison). A comprehensive discussion of related work is given in Appendix § 9.

Background: The Sign–Magnitude Decomposition

Setup

We consider an LL-layer feedforward network f:X→RKf:\mathcal{X}\to\mathbb{R}^{K} (FFNN) with hidden representations h(0)(x)=x\mathbf{h}^{(0)}(x)=x and, for ℓ=1,…,L\ell=1,\ldots,L,

z(ℓ)=W(ℓ)h(ℓ−1)+b(ℓ),h(ℓ)=φ(z(ℓ)),(1)\begin{aligned} \mathbf{z}^{(\ell)} &= W^{(\ell)}\mathbf{h}^{(\ell-1)}+\mathbf{b}^{(\ell)}, \qquad\mathbf{h}^{(\ell)}=\varphi\left(\mathbf{z}^{(\ell)}\right), \tag*{(1)} \end{aligned}

where W(ℓ)∈Rdℓ×dℓ−1W^{(\ell)}\in\mathbb{R}^{d_{\ell}\times d_{\ell-1}}, b(ℓ)∈Rdℓ\mathbf{b}^{(\ell)}\in\mathbb{R}^{d_{\ell}}, and φ\varphi is a coordinatewise activation function. The output layer produces logits f(x)=W(L+1)h(L)+b(L+1)f(x)=W^{(L+1)}\mathbf{h}^{(L)}+\mathbf{b}^{(L+1)}.

Each neuron jj in layer ℓ\ell computes a pre-activation zj(ℓ)=wjTh(ℓ−1)+bjz_j^{(\ell)}=\mathbf{w}_j^{\mathsf{T}}\mathbf{h}^{(\ell-1)}+b_j. The zero set {zj(ℓ)=0}\{z_j^{(\ell)}=0\} is a hyperplane in the representation space of the preceding layer. A layer of dℓd_{\ell} neurons therefore partitions its input space by dℓd_{\ell} hyperplanes, assigning each input a binary code, one bit per neuron, that records which side of every hyperplane it falls on. The question is whether the activation function preserves this partition.

Sign-preserving activations

Definition 2.1 (Sign-preserving activation). An activation φ:R→R\varphi:\mathbb{R}\to\mathbb{R} is sign-preserving if sgn⁡(φ(z))=sgn⁡(z)\operatorname{sgn}(\varphi(z))=\operatorname{sgn}(z) for every z≠0z\ne0, where sgn⁡(t)=+1\operatorname{sgn}(t)=+1 for t>0t>0 and −1-1 otherwise.

ReLU, SiLU, GELU, and tanh are all sign-preserving which will be the central point of study. A ReLU neuron zeroes out the negative half-space entirely, producing the sharpest possible partition; smooth activations such as SiLU and GELU attenuate rather than annihilate the negative side, but the sign of the output still mirrors the sign of the pre-activation. We discuss non sign-preserving activations such as softplus in Appendix C.

The decomposition

Every sign-preserving activation allows us to split each neuron’s output into two factors: a binary routing decision and a continuous magnitude.

Definition 2.2 (Sign–magnitude decomposition). Let φ\varphi be sign-preserving and let hj(ℓ)(x)=φ(zj(ℓ)(x))h_j^{(\ell)}(x)=\varphi(z_j^{(\ell)}(x)) be the output of neuron jj at layer ℓ\ell. The sign and magnitude of neuron jj are

Sj(ℓ)(x)=sgn⁡(hj(ℓ)(x))∈{−1,+1},Mj(ℓ)(x)=∣hj(ℓ)(x)∣∈R≥0,(2)S_j^{(\ell)}(x)=\operatorname{sgn}\left(h_j^{(\ell)}(x)\right)\in\{-1,+1\}, \qquad M_j^{(\ell)}(x)=\left|h_j^{(\ell)}(x)\right|\in\mathbb{R}_{\ge0}, \tag*{(2)}

so that the neuron’s output is

hj(ℓ)(x)=Sj(ℓ)(x)⋅Mj(ℓ)(x).(3)h_j^{(\ell)}(x)=S_j^{(\ell)}(x)\cdot M_j^{(\ell)}(x). \tag*{(3)}

In vector form, the full hidden representation decomposes as h(ℓ)=S(ℓ)⊙M(ℓ)\mathbf{h}^{(\ell)}=\mathbf{S}^{(\ell)}\odot\mathbf{M}^{(\ell)}. The sign code of the network is the concatenation of all per-layer sign vectors:

S(x)=(S(1)(x),…,S(L)(x))∈{−1,+1}D,D=∑ℓ=1Ldℓ.(4)\mathbf{S}(x)=\left(\mathbf{S}^{(1)}(x),\ldots,\mathbf{S}^{(L)}(x)\right)\in\{-1,+1\}^{D}, \qquad D=\sum_{\ell=1}^{L}d_{\ell}. \tag*{(4)}

Because φ\varphi is sign-preserving, Sj(ℓ)=sgn⁡(zj(ℓ))S_j^{(\ell)}=\operatorname{sgn}(z_j^{(\ell)}): the sign code is determined entirely by the pre-activation signs and is therefore identical across all sign-preserving activations sharing the same weights. Intuitively, S(x)\mathbf{S}(x) records which combinatorial region of representation space the input occupies, which side of every learned hyperplane it falls on, while M(x)\mathbf{M}(x) records where within that region. The sign code recovers the classical ReLU gate pattern g(ℓ)=1[z(ℓ)>0]\mathbf{g}^{(\ell)}=\mathbf{1}[z^{(\ell)}>0] under the relabelling +1↔+1\leftrightarrow active, −1↔-1\leftrightarrow inactive. The sign–magnitude decomposition for smooth activations (SiLU, GELU, tanh) captures the same combinatorial structure without requiring exact piecewise-affine behaviour. We discuss the natural generalization of this decomposition to convolutional layers as well as residual blocks in Section B.1.

Experiment: Sign Codes Are Informationally Sufficient

The sign–magnitude decomposition (Definition 2.2) splits every hidden representation into a discrete sign code and a continuous magnitude. We begin with a natural question:

Question 1: Does the sign code alone, without any magnitude information, already contain enough structure to predict class labels?

We test this by building the simplest possible classifier that operates only on the binary sign code of a single hidden layer, and comparing its accuracy to that of the full network.

Experimental setup: Sign-kkNN

For each trained model we extract the sign code S(L)(x)∈{−1,+1}dL\mathbf{S}^{(L)}(x) \in\{-1,+1\}^{d_L} of the last hidden layer only: sgn⁡(h(L)(x))\operatorname{sgn}(\mathbf{h}^{(L)}(x)) for FFNNs, the output code of the last BasicBlock for ResNets, and the output code of the final transformer block for ViTs. This is a strict subset of the full-network sign code (4). Full extraction details are given in Appendix § 12. We equip the sign code space with the normalised Hamming distance

dH(S(L)(x),S(L)(x′))=1dL∑j=1dL[Sj(L)(x)≠Sj(L)(x′)],(5)d_H\left(\mathbf{S}^{(L)}(x),\mathbf{S}^{(L)}(x')\right)=\frac{1}{d_L}\sum_{j=1}^{d_L}\left[\mathbf{S}^{(L)}_j(x)\ne\mathbf{S}^{(L)}_j(x')\right], \tag*{(5)}

and classify each test point by majority vote among its kk-nearest training-set neighbours in this distance. The resulting classifier, sign-kkNNk_k, uses only the binary sign code of a single layer. It has no access to activation magnitudes, to the sign codes of any earlier layer, or to the network’s weights or logits. To quantify how well the sign code captures the network’s classification, we define the sign gap:

Δsgn=∣err⁡SignKNN−err⁡model∣,(6)\Delta_{\mathrm{sgn}}=\left|\operatorname{err}_{\mathrm{SignKNN}}-\operatorname{err}_{\mathrm{model}}\right|, \tag*{(6)}

the absolute difference between the sign-kkNN test error and the full model’s test error. When Δsgn≈0\Delta_{\mathrm{sgn}}\approx0, the binary sign code is sufficient to match the network’s test error; when Δsgn\Delta_{\mathrm{sgn}} is large, information outside the last-layer sign code remains relevant to the model’s predictions.

Results: The sign gap across activation functions and architectures

Figure 2a displays Δsgn\Delta_{\mathrm{sgn}} as a function of network depth and width across all three architectures (FFNN, ResNet, ViT; SiLU activation, CIFAR-10). See Appendix § 13 for a description of the exact architectures and the general training setup. For shallow or narrow networks the sign gap is substantial: the sign code loses information that the full network retains. As depth and width increase, Δsgn\Delta_{\mathrm{sgn}} shrinks monotonically and becomes negligible (<0.03< 0.03) once both dimensions are sufficiently large. Sign-code sufficiency is not an unconditional property of the architecture; it is an emergent property of overparameterized networks. There is a clear phase boundary in the depth–width plane beyond which the last-layer sign code matches the full network’s test performance. This is the regime targeted by the certificate in § 6. Figure 2b plots sign-kkNN against full model test accuracy for overparameterized networks (depth 6, width 1024 FFNN, standard ResNet-18 and depth 6, 256 embedding dimension, 8 heads ViT). We train 540 models spanning three architectures, four sign-preserving activations, three datasets (MNIST, CIFAR-10, CIFAR-100), 5 dataset sizes and 3 seeds. In the overparameterized regime, points cluster tightly around the diagonal with no systematic dependence on architecture, activation, or dataset. The experiment is conservative: it uses only the last layer’s sign code, a small fraction of the full routing record, yet this already matches model performance across all four activations, including smooth ones (SiLU, GELU, tanh) with no piecewise-affine structure (thus generalizes the observations in [19]). This is not a consequence of neural collapse: our models are far from the collapsed regime (NC1 >1> 1, NCM accuracy well below model accuracy, see Section B.2).

Three-architecture sign-gap heatmaps and sign-kNN accuracy plots

Figure 2. a) The sign gap Δsgn\Delta_{\mathrm{sgn}} (Eq. (6)) as a function of network depth and width for three architectures (FFNN, ResNet, ViT) with SiLU on CIFAR-10. The gap shrinks monotonically with increasing depth and width, with a clear phase boundary (yellow lines) beyond which Δsgn<0.03\Delta_{\mathrm{sgn}} < 0.03. b) Sign-kNN accuracy versus full model test accuracy across standard architectures (rows: FFNN, ResNet-18, ViT), activations (columns: ReLU, SiLU, GELU, Tanh), and datasets (colours: MNIST, CIFAR-10, CIFAR-100). The dashed line is y=xy=x. Training and model details for all figures are in Appendix § 13.

Observation 1: Last-layer sign codes become informationally sufficient in the overparameterized regime. The last-layer sign code empirically suffices to match the model’s test error; the sign gap Δsgn\Delta_{\mathrm{sgn}} vanishes with depth and width.

Experiment: Testing robustness to sign code and magnitude perturbations

The last section showed that sign codes are informative: a nearest-neighbour classifier on the binary sign code of a single hidden layer tracks model accuracy. But informative does not mean causal. Both sign and magnitude are deterministic functions of the same weights, so either channel, or both, could carry the classification signal:

Question 2: How does the network respond to distinct perturbations: one that perturbs only the discrete sign code, and one that perturbs only the continuous magnitudes?

If the sign code is merely a by-product and the real computation flows through magnitudes, then corrupting signs should be tolerable and corrupting magnitudes catastrophic. If the sign code is the dominant computational channel, the pattern should be the reverse.

Experimental setup: Two channel-isolating interventions

Recall that at every hidden layer ℓ\ell, the representation decomposes as h(ℓ)=S(ℓ)⊙M(ℓ)h^{(\ell)} = S^{(\ell)} \odot M^{(\ell)} (Definition 2.2). Both interventions select neurons independently with probability α∈[0,1]\alpha\in[0, 1] at every hidden layer, but they are not meant to be symmetric natural forward passes. They are channel-isolating tests: one destroys routing while preserving magnitudes, and the other destroys magnitude information while enforcing the clean routing trajectory. The interventions differ in which channel is corrupted (Figure 3, right) and in the projection needed to keep that corruption isolated.

Intervention 1: destroy routing, preserve magnitudes. Each selected neuron has its sign negated while its magnitude is preserved. The corrupted representation propagates freely through all downstream layers, potentially altering their sign codes and magnitudes.

Intervention 2: destroy magnitudes, preserve routing. We first cache the clean sign code S(ℓ)(x)S^{(\ell)}(x) at every layer via an unmodified forward pass. Each selected neuron then has its magnitude replaced by ∣η∣\lvert\eta\rvert, η∼N(0,1)\eta\sim\mathcal{N}(0,1), a class-agnostic random draw carrying zero information about the input. The replacement is projected: at every downstream layer, the cached clean sign code is enforced and only magnitudes are recomputed. This is a causal-mediation test: it asks what remains of the computation when the sign trajectory is held fixed, not whether the network could recover that trajectory after magnitude corruption. At α=1\alpha= 1, every magnitude in every layer is a random number, yet the sign code is perfectly preserved.

Results of the robustness tests

We sweep α∈[0,1]\alpha\in[0,1] for four activations (ReLU, SiLU, GELU, tanh) and two architectures (FFNN on SVHN, ResNet-18 on CIFAR-10), averaging over five independent mask draws. Flipping just 20% of sign bits (α=0.2\alpha=0.2) collapses accuracy to near chance (∼10%\sim10\%) across all eight conditions; the ResNet’s skip connections do not compensate (Figure 3). At α=1\alpha=1, the ResNet maintains 93–95% accuracy across all four activations: every magnitude in every layer is an independent random number, yet the network classifies as well as with correct magnitudes. The FFNN retains near-full accuracy for tanh and degrades substantially for the rest, but remains far above the corresponding sign-flip failures.

The two interventions are deliberately asymmetric. Intervention 2 asks: does the network need any magnitude information beyond what the sign code encodes? Answering this requires isolating the magnitude channel, if corrupted magnitudes cascaded through downstream layers, they would induce secondary sign flips, corrupting both channels simultaneously. Projected replacement caches the clean sign code and enforces it at every layer, holding routing fixed while destroying magnitude content. Intervention 1 needs no analogous projection: negating a sign preserves ∣−m∣=∣m∣|-m|=|m|, so the magnitude channel is untouched by construction. The FFNN degradation under Intervention 2 is itself predicted by activation symmetry: tanh is odd, so its output is entirely determined by the sign code (near-zero drop for magnitudes and symmetry upon full sign-code swaps); ReLU, GELU, and SiLU each possess a sign-blind even component [φ(z)+φ(−z)]/2[\varphi(z)+\varphi(-z)]/2 invisible to the sign code, with values 0.50, 0.34, 0.23 at z=1z=1 which matches the degradation ranking (ReLU ∼36\sim36 pp; GELU ∼34\sim34 pp; SiLU ∼32\sim32 pp; tanh ∼2\sim2 pp). The ResNet’s immunity follows from the skip connection preserving sign structure under projected replacement.

Observation 2: Networks are robust to magnitude perturbations but not to sign code perturbations. Projected replacement of every magnitude preserves far more classification performance than sign corruption; flipping ∼20%\sim20\% of signs destroys the forward pass.

Experiment: Testing class-coherent sign geometry

Sections 3 and 4 showed that overparameterized networks can classify through sign code, and that corrupting the sign trajectory is far more destructive than corrupting magnitudes. But computation through signs does not by itself explain generalization. A network could classify through signs and still assign each example an idiosyncratic, non-transferable sign code, hence:

Question 3: What sign code structure supports generalization?

We focus here on evidence for necessity and leave sufficiency to future work.

Experimental setup: Varying class neighbourhood purity through a regularizer LrepL_{\mathrm{rep}}

Measuring sign code class structure. A sign code neighbourhood of a datapoint xix_i is label-pure if the nearest training points in Hamming distance share the class of xix_i. We measure this property with the leave-one-out (LOO) purity πk(xi)\pi_k(x_i) at kk neighbours:

πk(xi)=1k∑j∈Nki(xi)1[yj=yi],(7)\pi_k(x_i) = \frac{1}{k} \sum_{j \in\mathcal{N}^{i}_{k}(x_i)} 1[y_j = y_i], \tag*{(7)}

where Nki\mathcal{N}^{i}_{k} denotes the kk nearest training points in Hamming distance excluding xix_i itself. When πˉk=Ei[πk(xi)]\bar{\pi}_k = \mathbb{E}_i[\pi_k(x_i)] is high, same-class inputs occupy tight, overlapping regions in sign space, exactly the structure that sign-kNN exploits. When πˉk\bar{\pi}_k is near 1/C1/C (chance), the sign code carries no local class geometry. To test the role of class-pure neighbourhoods in generalization, we design a regularizer that actively destroys them.

Causal test: within-class sign repulsion. For each neuron we form a continuous proxy of its binary routing decision, concatenate across layers, and ℓ2\ell_2-normalise. The repulsion loss LrepL_{\mathrm{rep}} penalises routing overlap between same-class pairs:

Lrep=1∣P∣∑(i,j)∈P(S(xi)⊤S(xj))2,P={(i,j):i≠j, yi=yj}.(8)L_{\mathrm{rep}} = \frac{1}{|\mathcal{P}|} \sum_{(i,j)\in\mathcal{P}} \left(\mathbf{S}(x_i)^{\top}\mathbf{S}(x_j)\right)^2,\qquad\mathcal{P} = \{(i,j): i \ne j,\, y_i = y_j\}. \tag*{(8)}

For ReLU, SiLU, and GELU we use a sigmoid gate proxy Sj=σ(zj/τ)∈(0,1)S_j = \sigma(z_j/\tau) \in(0,1), which targets the active set that dominates the forward pass; for tanh, where positive and negative neurons contribute equally, we instead use a tanh sign proxy Sj=tanh⁡(zj/τ)∈(−1,1)S_j = \tanh(z_j/\tau) \in(-1,1). The choice of proxy, and why gate repulsion is strictly stronger than sign repulsion for non-odd activations, is discussed in Appendix H.6.

Results

Figure 4a plots train LOO purity against test accuracy for a range of models. Label-pure sign-code neighbourhoods predict good generalization; label-mixed ones the opposite. The LrepL_{\mathrm{rep}}-trained models land in the low-purity, low-accuracy corner on the same trend line, confirming the relationship is not an artifact of architecture or dataset. Panel b) shows SiLU dynamics: train accuracy reaches 97% while test accuracy monotonically drops to ∼\sim40% as the regularizer destroys class-aligned routing. Panel c) summarises across activations: for ReLU, SiLU, and GELU, repulsion sharply degrades test accuracy and purity while training accuracy stays above 97%; tanh is partially robust due to its magnitude symmetry (δ≈0\delta\approx0, Appendix H). The network still memorises every training example–representational capacity is intact—but the routing patterns are no longer shared across same-class inputs, and generalization collapses. While LrepL_{\mathrm{rep}} may affect correlated properties, this combination of high training and degraded test accuracy provides causal evidence that within-class sign-code coherence plays a direct role in generalization for the models studied.

Figure 4. a) Train LOO purity (k=5k=5) versus test accuracy across models (grey: FFNN on SVHN, widths 4–1024, depths 1–12, SiLU) and LrepL_{\mathrm{rep}}-trained models (coloured). High neighbourhood purity tracks generalization (Spearman ρ=0.834\rho= 0.834); LrepL_{\mathrm{rep}} models fall on the same trend line. b) Training dynamics under LrepL_{\mathrm{rep}} (SiLU, width 1024, depth 8, SVHN). Train accuracy reaches 97%; test accuracy drops to ∼\sim40%. Dashed: Test accuracy of the normally trained baseline. c) Summary across activations (depth 8, width 1024, SVHN). Repulsion sharply degrades test accuracy and LOO purity for ReLU, SiLU, and GELU while training accuracy remains ≥\geq97%. Tanh is partially robust (δ≈0\delta\approx0; Appendix H).

Observation 3: Class-coherent sign geometry is necessary for generalization. Networks generalize when same-class inputs share similar sign codes; a regularizer that repels within-class sign codes preserves training accuracy but collapses test accuracy.

Theory: Overparameterization Makes Neural Networks Binary

Sections 3 and 4 showed that sign codes become informationally sufficient in the overparameterized regime and are causally dominant under channel-isolating interventions. We now provide an exact decomposition of the logit into a sign-only term and a magnitude residual, together with a certificate that predicts when the sign-only classifier agrees with the full network.

Decomposing the logit

The class-cc logit decomposes as fc(x)=∑j=1dwcjSj(x)Mj(x)+bcf_c(x) = \sum_{j=1}^{d} w_{cj} S_j(x) M_j(x) + b_c for a last hidden layer of width dd. Replacing each magnitude MjM_j by its conditional mean μj(s)=E[Mj∣Sj=s]\mu_j(s) = \mathbb{E}[M_j \mid S_j=s], the unique MMSE predictor of MjM_j from the single bit SjS_j (Appendix § 14, Proposition F.2), splits this into two exact pieces:

fc(x)=∑jwcjSj(x)μj(Sj(x))+bc⏟f~c(x): sign-only logit+∑jwcjSj(x)[Mj(x)−μj(Sj(x))]⏟Rc(x): magnitude residual(9)f_c(x) = \underbrace{\sum_j w_{cj} S_j(x) \mu_j(S_j(x)) + b_c}_{\widetilde{f}_c(x):\ \text{sign-only logit}} + \underbrace{\sum_j w_{cj} S_j(x)\left[M_j(x)-\mu_j(S_j(x))\right]}_{R_c(x):\ \text{magnitude residual}} \tag*{(9)}

The sign-only logit f~c\widetilde{f}_c depends on xx only through the binary sign code. The residual RcR_c captures everything the sign code misses: the per-neuron deviations of each magnitude from its sign-conditional average. The pairwise margin inherits the same split:

Ma,c(x)=M~a,c(x)+Ra,c(x)(10)M_{a,c}(x) = \widetilde{M}_{a,c}(x) + R_{a,c}(x) \tag*{(10)}

and the sign-only prediction disagrees with the full network only when the residual overwhelms the sign-only margin: ∣Ra,c∣≥∣M~a,c∣\lvert R_{a,c}\rvert\ge\lvert\widetilde{M}_{a,c}\rvert.

A certificate for sign domination

The decomposition raises a quantitative question: when is the sign-only margin large enough that the residual cannot flip the prediction? For each input xx with full-model predicted class y^=f^(x)\hat{y} = \hat{f}(x) and each competitor c≠y^c \ne\hat{y}, define the per-neuron vote bias mc(x)=1d[∑j=1d(wy^,j−wc,j)Sj(x)μj(Sj(x))+(by^−bc)]m_c(x) = \frac{1}{d}\left[\sum_{j=1}^{d}(w_{\hat{y},j} - w_{c,j})S_j(x)\mu_j(S_j(x)) + (b_{\hat{y}} - b_c)\right], which measures how decisively, on average, each neuron's sign-weighted contribution favours y^\hat{y} over cc, and the conditional residual moment ry^(x),c2(x)=E[Ry^(x),c2(X)∣S(X)=S(x), f^(X)=y^(x)]r^2_{\hat{y}(x),c}(x) = \mathbb{E}[R^2_{\hat{y}(x),c}(X) \mid S(X)=S(x),\ \hat{f}(X)=\hat{y}(x)], which captures how much the residual fluctuates among inputs that share the same sign code and full-model prediction. Conditioning on both ensures that the class pair and sign-only margin are fixed under the conditional measure. This is a population quantity; in the experiments we estimate the same signal using the residual diagnostics described in Appendix § 15. The gate-domination score is:

Z1(x)=min⁡c≠y^∣M~y^,c(x)∣rc(x)(11)Z_1(x) = \min_{c \ne\hat{y}} \frac{\left\lvert\widetilde{M}_{\hat{y},c}(x)\right\rvert}{r_c(x)} \tag*{(11)}

Theorem 6.1 (Gate domination). Let ϵ1=Pr⁡x(f^(x)≠f~(x))\epsilon_1 = \Pr_x(\hat{f}(x) \ne\widetilde{f}(x)) be the disagreement rate between the full network and its sign-only version. Assume that ties in both classifiers occur with probability zero under the data distribution, and that all residual second moments are finite.

(i) Disagreement bound: ϵ1≤(C−1)Ex[Z1−2(x)]\epsilon_1 \le(C-1)\mathbb{E}_x[Z_1^{-2}(x)].

(ii) Width scaling: If rc2(x)≤d σˉc2(x)r_c^2(x) \le d\,\bar{\sigma}_c^2(x) for all competitors cc, then Z1(x)≥(min⁡c≠y^∣mc(x)∣σˉc(x))dZ_1(x) \ge\left(\min_{c\ne\hat{y}}\frac{|m_c(x)|}{\bar{\sigma}_c(x)}\right)\sqrt{d}.

Proof. The full proof is in Appendix F.7.

Interpretation and empirical validation

Figure 5 displays Z1Z_1, ε1\varepsilon_1, and the sign-gap Δsgn\Delta_{\mathrm{sgn}} across 360 trained FFNNs (12 depths ×\times 10 widths ×\times 3 seeds, SiLU, CIFAR-10). The theorem gives a sufficient explanation for why overparameterized networks can become effectively binary. Each neuron casts a sign vote for or against each class; depth composes these votes into a decisive bias (mc>0m_c > 0), and width averages out the magnitude noise through d\sqrt{d}-scaling. The sign-only margin grows as d⋅mcd \cdot m_c while the residual grows at most as d⋅σˉc\sqrt{d}\cdot\bar{\sigma}_c, so Z1Z_1 improves with added neurons until the sign-only classifier agrees with the full network. Neither mechanism suffices alone: at depth 1, votes are unbiased (mc≈0m_c \approx0) and Z1<0.23Z_1 < 0.23 even at width 4096 (panel b), bottom row); narrow-but-deep networks have biased votes but too few neurons for the averaging to suppress magnitude noise. The yellow line in panels (a)–(c) marks where both conditions are jointly met: above it, Δsgn<0.03\Delta_{\mathrm{sgn}} < 0.03 and ε1<0.1\varepsilon_1 < 0.1. Panel d) is the key result: the plotted configurations collapse onto a single Z1Z_1-vs-ε1\varepsilon_1 curve, regardless of whether a model achieves its Z1Z_1 through width, depth, or both. We verify all theorem conditions and discuss their scope in Appendix G; the analysis is most informative for FFNNs with asymmetric activations, while for tanh and residual architectures the causal interventions (§ 4) provide the stronger evidence.

Discussion

In overparameterized networks, classification depends far more on a discrete sign code with class-coherent sign geometry than on precise activation magnitudes. Preserving sign trajectories while randomizing magnitudes retains high accuracy, whereas small sign perturbations collapse performance catastrophically. This suggests that existing generalization theory may be missing an important granularity: the binary sign code quotients weight space into equivalence classes, and generalization bounds formulated directly over this discrete object would be invariant to magnitude redundancies that can make norm-based bounds vacuous. Our analysis also offers a hypothesis on why gated activations (SiLU, GELU, and GLU variants) are dominantly used in modern transformers: their large magnitude asymmetry (Section H) amplifies the sign-only margin, making the binary routing channel maximally decisive, whereas symmetric activations like tanh place no signal in this channel.

Limitations. While we empirically treat multiple architectures, our theoretical analysis centres on feedforward networks. The gate-domination score Z1Z_1 can remain modest for residual networks and odd activations even when the sign trajectory is causally dominant under intervention (Section H), and we restrict attention to sign-preserving activations. Our empirical evidence comes from small-scale vision benchmarks (CIFAR, SVHN, MNIST) and classification tasks; whether the sign code mechanism extends to large-scale settings, language modelling, or regression is left to future work.

Acknowledgments and Disclosure of Funding

This work was supported by the Laboratory for AI Security Research (LASR). The views expressed in this paper are those of the authors and do not necessarily reflect the position of LASR or His Majesty’s Government.

Related work

A central motivation for this paper is the generalization puzzle in overparameterized deep learning. Modern networks can interpolate random labels while still generalizing on natural data, forcing a re-examination of classical capacity arguments [41]. This led to a large literature on norm-, margin-, PAC-Bayes-, and compression-based explanations of generalization [8, 32, 3, 1, 23], and to studies of the interpolation regime through double descent and overparameterization [4, 30, 33]. A complementary line analyzes the implicit bias of optimization, showing convergence toward max-margin or otherwise structured solutions in linear, homogeneous, and wide-network settings [40, 28, 5]. Another influential debate ties generalization to sharpness or flatness of minima [24, 6]. Our work is motivated by the same puzzle, but shifts the explanatory target from continuous weights to the discrete routing pattern induced by the network.

A second line studies neural networks through their activation patterns and the partitions they induce. Early work on rectifiers emphasized sparse activation and the practical advantages of non-saturating units [14]. For ReLU and related piecewise-linear networks, several papers quantified the number of response or linear regions and the expressive gains from depth [36, 29, 37, 39]. Later work showed that the number of linear regions realized in practice can be far below worst-case counts [18], and that linear regions are not equally important functionally [12]. Closely related spline and max-affine views interpret deep networks as input-dependent affine maps indexed by activation patterns [2]. Our sign code perspective is closest to this tradition, but differs in emphasis: the goal here is not primarily to count regions or characterize expressivity, but to argue that the binary code identifying a region can itself become the dominant carrier of class information.

Some aspects of binary activation patterns have been studied. It was shown in He et al. [ [19] that the sign code induced by a trained classifier can behave as a hash representation. Park et al. introduces a specialized unsupervised objective that learns binary activation codes in order to increase encoder expressivity and improve downstream classification and retrieval [2021]. Related work has shown that class-dependent partial sign codes can be extracted from trained ReLU networks and used to verify that the network assigns the same label to inputs consistent with those partial sign codes [13].

This paper is especially close to the path- and gate-based literature. Path-SGD and path-norm analyses emphasized geometries and complexity measures that respect ReLU rescaling symmetries better than raw Euclidean weight space [31 b]. Lakshminarayanan and Singh introduced neural path features and the neural path kernel, explicitly encoding the active subnetwork selected by the gates [26]. Fiat et al. made the separation between gating and linear computation explicit by decoupling the two roles in the neuron [10]. More recent work extended path-based analyses to modern DAG architectures and principled rescaling procedures [15, 27]. Among existing lines, this is the closest antecedent to our thesis: we likewise elevate gates from a bookkeeping device to a primary object of analysis, but we go further by arguing for three complementary claims—informational sufficiency, causal dominance under intervention, and theoretical dominance of the sign code once magnitude residuals concentrate.

Our empirical results also connect to work on representation geometry in trained classifiers. Visualization and representation-similarity studies show that networks that generalize organize their internal representations differently from networks that memorize [22, 25]. Neural collapse sharpened this picture by showing that late-layer features can converge toward class means and nearest-class-mean decision rules during terminal training phases [34, 17]. Our sign-kkNN and codebook-collapse analyses are complementary to this literature: they replace continuous Euclidean geometry by discrete Hamming geometry on sign codes, and ask whether class structure is already recoverable before or even without full collapse of the continuous feature cloud.

Finally, because the present paper studies sign-preserving activations and multiple backbone families, it also interfaces with the activation-function and architecture literature. The experiments span residual networks and vision transformers [20, 7], and include GELU and SiLU/Swish-style smooth activations in addition to ReLU and tanh [21, 38, 9]. The redundancy and erasure-robustness story we observe is also adjacent to pruning, compression, and sparse-subnetwork results [16, 11]: heavily overparameterized networks often contain substantial slack. The distinction is that pruning and compression typically operate in parameter space, whereas our thesis is about input-dependent routing structure. The claim is not merely that many weights are removable, but that overparameterization can induce a simpler, more redundant binary partition whose class geometry increasingly governs prediction.

Additional material for the main paper

Extension of sign extraction

Convolutional blocks. For a convolutional output h(ℓ)∈RC×H×W\mathbf{h}^{(\ell)} \in\mathbb{R}^{C \times H \times W}, Definition 2.2 applies entrywise, yielding a sign code tensor S(ℓ)∈{−1,+1}C×H×W\mathbf{S}^{(\ell)} \in\{-1,+1\}^{C \times H \times W} that records both which channels and where in space each feature responds positively. We compress the spatial dimensions by majority vote on a P×PP \times P grid (P=2P=2 throughout), giving C⋅P2C \cdot P^2 bits per layer.

Residual blocks. A residual block adds a learned branch to a skip connection:

h(ℓ+1)=φ(h(ℓ)+F(ℓ)(h(ℓ))),(12)\mathbf{h}^{(\ell+1)} = \varphi\left(\mathbf{h}^{(\ell)} + \mathcal{F}^{(\ell)}\left(\mathbf{h}^{(\ell)}\right)\right), \tag*{(12)}

where F(ℓ)\mathcal{F}^{(\ell)} is a sub-network (typically two convolutional or fully-connected layers with their own activations). Two families of sign codes arise. The activations inside the branch F(ℓ)\mathcal{F}^{(\ell)} produce internal sign codes, computed exactly as in Definition 2.2. For a standard two-layer branch with intermediate activation, the internal code is Sint(ℓ)=sgn⁡(W1(ℓ)h(ℓ)+b1(ℓ))\mathbf{S}_{\mathrm{int}}^{(\ell)} = \operatorname{sgn}\left(\mathbf{W}_{1}^{(\ell)}\mathbf{h}^{(\ell)} + \mathbf{b}_{1}^{(\ell)}\right). The block-output code Sout(ℓ)=sgn⁡(h(ℓ+1))\mathbf{S}_{\mathrm{out}}^{(\ell)} = \operatorname{sgn}\left(\mathbf{h}^{(\ell+1)}\right) records the sign of the sum of skip and branch, and need not equal S(ℓ)\mathbf{S}^{(\ell)}; it is precisely the sign flips introduced by the branch that allow each block to refine the combinatorial partition of the preceding layer. The full sign code of an LL-block residual network concatenates both internal and output codes across all blocks:

S(x)=(Sint(1),Sout(1),…,Sint(L),Sout(L))∈{−1,+1}D.(13)\mathbf{S}(x) = \left(\mathbf{S}_{\mathrm{int}}^{(1)}, \mathbf{S}_{\mathrm{out}}^{(1)}, \ldots, \mathbf{S}_{\mathrm{int}}^{(L)}, \mathbf{S}_{\mathrm{out}}^{(L)}\right) \in\{-1,+1\}^{D}. \tag*{(13)}

Omitting either the internal or the output codes would lose information about the routing decisions the network makes.

Relationship to Neural Collapse

Neural collapse [34, 17] describes a terminal-phase phenomenon in which (NC1) within-class feature variance vanishes relative to between-class variance, (NC2) class means converge to an equiangular tight frame, (NC3) the last-layer classifier aligns with the class means, and (NC4) a nearest-class-mean (NCM) classifier matches network accuracy. A natural question is whether sign code sufficiency is simply a consequence of neural collapse: if features have already collapsed onto class means, any reasonable representation—including the sign code—would trivially separate classes.

We measure all four NC metrics across the full FFNN width–depth grid (SiLU, SVHN, 80 training epochs) in Figure 6. The models are far from the collapsed regime. NC1 (SW/SBS_W/S_B) remains above 1 throughout most of the grid, indicating that within-class scatter still dominates between-class scatter; values below 1 would indicate meaningful collapse. NC3 (cosine alignment between the classifier weights and the class-mean directions) stays below 0.4 for most configurations, far from the ideal value of 1. NC4 (NCM test accuracy) peaks around 60–70%, substantially below the full model accuracy of ∼\sim80% achieved by the same networks. NC2 (ETF deviation) is near zero at large widths but this alone does not indicate collapse—it reflects only the angular structure of the class means, not the tightness of the feature clouds around them.

Neural collapse diagnostics across the width–depth grid

Figure 6. Neural collapse diagnostics across the width–depth grid (FFNN, SiLU, SVHN, 80 epochs). Top row: NC1 (SW/SBS_W/S_B, log scale), NC2 (ETF deviation), NC3 (weight–mean cosine alignment). Bottom row: NC4 (nearest-class-mean test accuracy), sign purity, and NC summary curves across widths for selected depths. NC1 remains above 1 and NC3 below 0.4 across most configurations, confirming that the models have not undergone neural collapse. NCM test accuracy (NC4) peaks at ∼70%\sim70\%, well below full model accuracy. Despite this, sign purity is high wherever the network is sufficiently wide and deep—class-discriminative sign code structure emerges independently of, and prior to, neural collapse.

Despite this absence of collapse, the sign code is already informationally sufficient (§ 3) and causally dominant under intervention (§ 4) in precisely the same models. Sign purity—the fraction of training points whose sign code is shared only with same-class examples—is high across much of the grid even where NC1 remains large and NC4 is poor (compare bottom-left and bottom-centre panels of Figure 6). Class structure is thus recoverable in discrete Hamming space well before the continuous features converge toward their class means. This makes the sign code mechanism strictly more general than neural collapse: it does not require prolonged terminal-phase training, and it operates through combinatorial routing geometry rather than through concentration of continuous features.

Non-Sign-Preserving Activations

The sign-preserving property (Definition 2.1) covers the activations dominant in modern practice: ReLU, SiLU/Swish, GELU, and tanh are all sign-preserving and account for the vast majority of architectures in current use. The decomposition hj=Sj⋅Mjh_j = S_j \cdot M_j with Sj=sgn⁡(zj)S_j = \operatorname{sgn}(z_j) applies directly to all of them. We briefly discuss what happens outside this class.

Shifted-threshold activations (e.g. softplus). Softplus, φ(z)=log⁡(1+ez)\varphi(z) = \log(1 + e^z), is strictly positive for all zz and therefore not sign-preserving: sgn⁡(φ(z))=+1\operatorname{sgn}(\varphi(z)) = +1 always. However, softplus is monotone, so it preserves the ordering of pre-activations. A natural generalization is to replace the zero threshold with the activation’s symmetry axis. For softplus, φ(0)=log⁡2\varphi(0) = \log2, and the function is approximately linear for z≫0z \gg0 and approximately flat for z≪0z \ll0. Defining Sj=sgn⁡(zj)S_j = \operatorname{sgn}(z_j) directly—bypassing the activation output—recovers a meaningful binary partition: it still records which side of the neuron’s learned hyperplane the input falls on. The magnitude can then be defined relative to this threshold, e.g. Mj=∣φ(zj)−φ(0)∣M_j = |\varphi(z_j) - \varphi(0)|. We have not tested this variant experimentally, but the monotonicity of softplus ensures that the partition induced by sgn⁡(zj)\operatorname{sgn}(z_j) is identical to the one studied throughout the paper—only the magnitude profile changes. Symmetric activations (e.g. z2z^2, cos⁡(z)\cos(z)). Squaring or other even functions destroy sign information entirely: (−z)2=z2(-z)^2 = z^2. The sign–magnitude decomposition as stated does not apply. Yet a binary routing question remains meaningful—one could ask whether ∣zj∣|z_j| exceeds some data-dependent threshold τj\tau_j (e.g. the median pre-activation magnitude), partitioning inputs into high-activation and low-activation regimes. Whether such a partition carries class-discriminative structure comparable to the sign code is an open empirical question. We note that even activations like z2z^2 are rarely used in isolation in modern networks; they typically appear inside gated units (e.g. z1⋅z2z_1 \cdot z_2 in some GLU variants) where the gating component restores a sign-preserving partition on at least one branch.

In summary, the sign-preserving assumption is not restrictive in practice: it covers all standard activations, and for the main exception (softplus), the pre-activation sign provides an equivalent partition. Extending the framework to genuinely non-monotone or even activations is possible in principle but would require a different choice of binary code, and we leave this to future work.

Sign Code Extraction

This appendix documents the precise extraction procedure for the sign code S(x)S(x) used throughout the paper. We describe the procedure separately for each architecture family and specify exactly which tensors are recorded, which are discarded, and how the full-network sign code is assembled. All sign codes use the convention sgn⁡(t)=+1\operatorname{sgn}(t) = +1 if t>0t > 0 and −1-1 otherwise (in particular, sgn⁡(0)=−1\operatorname{sgn}(0) = -1).

Feedforward networks (FFNN)

Architecture. Our FFNN models are of the form

x→flattenh(0)→W(1), φh(1)→W(2), φ⋯→W(L), φh(L)→W(L+1)logits,x \xrightarrow{\text{flatten}} \mathbf{h}^{(0)} \xrightarrow{W^{(1)},\,\varphi} \mathbf{h}^{(1)} \xrightarrow{W^{(2)},\,\varphi} \cdots\xrightarrow{W^{(L)},\,\varphi} \mathbf{h}^{(L)} \xrightarrow{W^{(L+1)}} \text{logits},

where every hidden layer applies the same sign-preserving activation φ\varphi (one of ReLU, SiLU, GELU, or tanh). The model exposes an nn.ModuleList of nn.Linear layers (hidden_layers) and a final nn.Linear output layer (output_layer).

Extraction. We register a forward hook on each nn.Linear in hidden_layers. The hook records the pre-activation z(ℓ)=W(ℓ)h(ℓ−1)+b(ℓ)\mathbf{z}^{(\ell)} = W^{(\ell)}\mathbf{h}^{(\ell-1)} + \mathbf{b}^{(\ell)}, i.e. the output of the linear layer before the activation is applied. After the forward pass completes, we apply φ\varphi to each captured pre-activation to obtain the post-activation h(ℓ)=φ(z(ℓ))\mathbf{h}^{(\ell)} = \varphi(\mathbf{z}^{(\ell)}), and compute:

S(ℓ)(x)=sgn⁡(h(ℓ)(x))∈{−1,+1}dℓ,(14)S^{(\ell)}(x) = \operatorname{sgn}(\mathbf{h}^{(\ell)}(x)) \in\{-1,+1\}^{d_\ell}, \tag*{(14)}
M(ℓ)(x)=∣h(ℓ)(x)∣∈R≥0dℓ.(15)M^{(\ell)}(x) = \left|\mathbf{h}^{(\ell)}(x)\right| \in\mathbb{R}_{\geq0}^{d_\ell}. \tag*{(15)}

Because φ\varphi is sign-preserving, S(ℓ)=sgn⁡(z(ℓ))S^{(\ell)} = \operatorname{sgn}(\mathbf{z}^{(\ell)}), so the sign code is equivalently determined by the pre-activation signs.

Full sign code. The full-network sign code is the concatenation across all LL hidden layers:

S(x)=(S(1)(x),…,S(L)(x))∈{−1,+1}D,D=∑ℓ=1Ldℓ.S(x) = \left(S^{(1)}(x), \ldots, S^{(L)}(x)\right) \in\{-1,+1\}^{D}, \qquad D = \sum_{\ell=1}^{L} d_\ell.

The output layer W(L+1)W^{(L+1)} is not activated and is therefore excluded from the sign code.

What is not extracted. The output logits are recorded for evaluation but do not contribute to the sign code. No sign code is computed for the input h(0)=x\mathbf{h}^{(0)} = x (there is no activation to decompose).

Residual networks (ResNet)

Architecture. We use torchvision BasicBlock-based ResNets (ResNet-18, ResNet-34). The architecture comprises a convolutional stem followed by four layer groups, each containing multiple residual blocks, followed by global average pooling and a linear classifier:

x→conv1, bn1, relu, maxpoolhstem→layer1⋯→layer4hfinal→avgpool, fclogits.x \xrightarrow{\text{conv1, bn1, relu, maxpool}} \mathbf{h}_{\mathrm{stem}} \xrightarrow{\text{layer1}} \cdots\xrightarrow{\text{layer4}} \mathbf{h}_{\mathrm{final}} \xrightarrow{\text{avgpool, fc}} \text{logits}.

Each BasicBlock has the following internal structure:

h(ℓ)→conv1,bn1zint→relua→conv2,bn2b→+ skipzout→reluh(ℓ+1)\mathbf{h}^{(\ell)} \xrightarrow{\mathrm{conv1,bn1}} \mathbf{z}_{\mathrm{int}} \xrightarrow{\mathrm{relu}} \mathbf{a} \xrightarrow{\mathrm{conv2,bn2}} \mathbf{b} \xrightarrow{+\ \mathrm{skip}} \mathbf{z}_{\mathrm{out}} \xrightarrow{\mathrm{relu}} \mathbf{h}^{(\ell+1)}

where the skip connection is the identity when dimensions match, or a learned 1×11 \times1 convolution with batch normalisation (downsample) when a spatial or channel-dimension change occurs. Note that torchvision BasicBlock uses a single nn.ReLU module that is called twice during the forward pass: once after bn1 (internal) and once after the residual addition (output).

Extraction. We register five hooks per block and one hook on the stem:

  1. Stem activation. A forward hook on model.relu captures hstem=ReLU⁡(bn1⁡(conv1⁡(x)))\mathbf{h}_{\mathrm{stem}} = \operatorname{ReLU}(\operatorname{bn1}(\operatorname{conv1}(x))), the stem output after the first ReLU (but before max-pooling). We record Sstem=sgn⁡(hstem)\mathbf{S}_{\mathrm{stem}} = \operatorname{sgn}(\mathbf{h}_{\mathrm{stem}}) and Mstem=∣hstem∣\mathbf{M}_{\mathrm{stem}} = \lvert\mathbf{h}_{\mathrm{stem}}\rvert.

  1. Block input. A forward pre-hook on the block module captures the input tensor h(ℓ)\mathbf{h}^{(\ell)} entering the block. This serves as the skip connection when no downsampling is needed.

  1. Internal branch activation. A forward hook on block.relu captures every ReLU output inside the block. In a BasicBlock the shared ReLU is called exactly twice; the first output is the internal branch activation a=ReLU⁡(bn1⁡(conv1⁡(h(ℓ))))\mathbf{a} = \operatorname{ReLU}(\operatorname{bn1}(\operatorname{conv1}(\mathbf{h}^{(\ell)}))). We record Sint(ℓ)=sgn⁡(a)\mathbf{S}_{\mathrm{int}}^{(\ell)} = \operatorname{sgn}(\mathbf{a}) and Mint(ℓ)=∣a∣\mathbf{M}_{\mathrm{int}}^{(\ell)} = \lvert\mathbf{a}\rvert.

  1. Branch output (pre-addition). A forward hook on block.bn2 captures b=bn2⁡(conv2⁡(a))\mathbf{b} = \operatorname{bn2}(\operatorname{conv2}(\mathbf{a})), the output of the residual branch before addition with the skip.

  1. Skip connection. If the block has a downsample module, a forward hook captures its output. Otherwise, the skip is the block input h(ℓ)\mathbf{h}^{(\ell)} recorded in step 2.

  1. Residual sum and block output. We compute the residual sum zout=b+skip\mathbf{z}_{\mathrm{out}} = \mathbf{b} + \mathrm{skip} from the tensors captured in steps 4 and 5. This is the pre-activation of the block’s output ReLU. The block-output activation h(ℓ+1)=ReLU⁡(zout)\mathbf{h}^{(\ell+1)} = \operatorname{ReLU}(\mathbf{z}_{\mathrm{out}}) is captured via a forward hook on the block module itself. We record Sout(ℓ)=sgn⁡(h(ℓ+1))\mathbf{S}_{\mathrm{out}}^{(\ell)} = \operatorname{sgn}(\mathbf{h}^{(\ell+1)}) and Mout(ℓ)=∣h(ℓ+1)∣\mathbf{M}_{\mathrm{out}}^{(\ell)} = \lvert\mathbf{h}^{(\ell+1)}\rvert. Because ReLU is sign-preserving, Sout(ℓ)=sgn⁡(zout)\mathbf{S}_{\mathrm{out}}^{(\ell)} = \operatorname{sgn}(\mathbf{z}_{\mathrm{out}}); we additionally store sgn⁡(zout)\operatorname{sgn}(\mathbf{z}_{\mathrm{out}}) and ∣zout∣\lvert\mathbf{z}_{\mathrm{out}}\rvert for diagnostic use.

Full sign code. The full-network sign code concatenates the stem code and both the internal and output codes from every block:

S(x)=(Sstem,Sint(1),Sout(1),…,Sint(B),Sout(B)),\mathbf{S}(x) = \left(\mathbf{S}_{\mathrm{stem}}, \mathbf{S}_{\mathrm{int}}^{(1)}, \mathbf{S}_{\mathrm{out}}^{(1)}, \ldots, \mathbf{S}_{\mathrm{int}}^{(B)}, \mathbf{S}_{\mathrm{out}}^{(B)}\right),

where BB is the total number of BasicBlocks across all four layer groups (e.g. B=8B = 8 for ResNet-18). For convolutional layers the sign code tensors have shape C×H×WC \times H \times W; we compress each to C⋅P2C \cdot P^2 bits using majority-vote spatial pooling on a P×PP \times P grid (P=2P = 2).

What is not extracted. The global average-pooling layer and the final linear classifier (fc) are not activated by a sign-preserving nonlinearity and do not contribute to the sign code. The second ReLU output per block (captured by the shared block.relu hook) is functionally identical to the block-output activation captured by the block-level hook; we use the latter and discard the former. The stem’s max-pooling layer is applied after the stem ReLU and does not introduce a new sign decision, so no separate code is extracted for it.

Vision Transformer (ViT)

Architecture. Our ViT follows the standard pre-norm design:

x→patch embedt→+ cls,posh(0)→block1⋯→blockBh(B)→norm,headlogits,x \xrightarrow{\mathrm{patch\ embed}} \mathrm{t} \xrightarrow{+\ \mathrm{cls,pos}} \mathbf{h}^{(0)} \xrightarrow{\mathrm{block}_1} \cdots\xrightarrow{\mathrm{block}_B} \mathbf{h}^{(B)} \xrightarrow{\mathrm{norm,head}} \mathrm{logits},

where each transformer block has the structure

hattn(ℓ)=h(ℓ−1)+Attn⁡(LN⁡1(h(ℓ−1))),\mathbf{h}_{\mathrm{attn}}^{(\ell)} = \mathbf{h}^{(\ell-1)} + \operatorname{Attn}\left(\operatorname{LN}_1\left(\mathbf{h}^{(\ell-1)}\right)\right),
h(ℓ)=hattn(ℓ)+MLP⁡(LN⁡2(hattn(ℓ))).\mathbf{h}^{(\ell)} = \mathbf{h}_{\mathrm{attn}}^{(\ell)} + \operatorname{MLP}\left(\operatorname{LN}_2\left(\mathbf{h}_{\mathrm{attn}}^{(\ell)}\right)\right).

The MLP sub-block is Linear →\to GELU →\to Dropout →\to Linear →\to Dropout.

Extraction. Unlike the FFNN and ResNet extractors, the ViT extractor performs a manual forward pass through the model’s modules rather than relying on hooks. This gives explicit access to every intermediate tensor. For each transformer block ℓ=1,…,B\ell= 1, \ldots, B, we record three sign–magnitude pairs:

  1. Post-attention residual stream. hattn(ℓ)=h(ℓ−1)+Attn⁡(LN⁡1(h(ℓ−1)))\mathbf{h}_{\mathrm{attn}}^{(\ell)} = \mathbf{h}^{(\ell-1)} + \operatorname{Attn}(\operatorname{LN}_{1}(\mathbf{h}^{(\ell-1)})). We record Sattn(ℓ)=sgn⁡(hattn(ℓ))\mathbf{S}_{\mathrm{attn}}^{(\ell)} = \operatorname{sgn}(\mathbf{h}_{\mathrm{attn}}^{(\ell)}) and Mattn(ℓ)=∣hattn(ℓ)∣\mathbf{M}_{\mathrm{attn}}^{(\ell)} = \lvert\mathbf{h}_{\mathrm{attn}}^{(\ell)}\rvert. This is the sign of the residual stream after the attention sub-block has added its contribution.

  1. MLP hidden activation. a(ℓ)=GELU⁡(W1(ℓ)LN⁡2(hattn(ℓ))+b1(ℓ))\mathbf{a}^{(\ell)} = \operatorname{GELU}(W_{1}^{(\ell)}\operatorname{LN}_{2}(\mathbf{h}_{\mathrm{attn}}^{(\ell)})+\mathbf{b}_{1}^{(\ell)}). We record Smlp(ℓ)=sgn⁡(a(ℓ))\mathbf{S}_{\mathrm{mlp}}^{(\ell)} = \operatorname{sgn}(\mathbf{a}^{(\ell)}) and Mmlp(ℓ)=∣a(ℓ)∣\mathbf{M}_{\mathrm{mlp}}^{(\ell)} = \lvert\mathbf{a}^{(\ell)}\rvert. This is the internal sign code of the MLP, analogous to the internal branch code in a ResNet block. GELU is sign-preserving, so Smlp(ℓ)\mathbf{S}_{\mathrm{mlp}}^{(\ell)} is determined by the pre-activation signs.

  1. Block output. h(ℓ)=hattn(ℓ)+MLP⁡(LN⁡2(hattn(ℓ)))\mathbf{h}^{(\ell)} = \mathbf{h}_{\mathrm{attn}}^{(\ell)} + \operatorname{MLP}(\operatorname{LN}_{2}(\mathbf{h}_{\mathrm{attn}}^{(\ell)})). We record Sout(ℓ)=sgn⁡(h(ℓ))\mathbf{S}_{\mathrm{out}}^{(\ell)} = \operatorname{sgn}(\mathbf{h}^{(\ell)}) and Mout(ℓ)=∣h(ℓ)∣\mathbf{M}_{\mathrm{out}}^{(\ell)} = \lvert\mathbf{h}^{(\ell)}\rvert. As with ResNets, the output sign code need not equal the input sign code: sign flips introduced by the MLP branch refine the partition.

Additionally, we record the patch-embedding tokens t=PatchEmbed⁡(x)\mathbf{t} = \operatorname{PatchEmbed}(x) and their sign and magnitude. These serve as the “stem” of the ViT and are available for diagnostic use, though they do not pass through a sign-preserving activation and are therefore not included in the default sign code.

Full sign code. The full-network sign code concatenates all three code families across all BB transformer blocks:

S(x)=(Sattn(1),Smlp(1),Sout(1),…,Sattn(B),Smlp(B),Sout(B)).\mathbf{S}(x) = \left(\mathbf{S}_{\mathrm{attn}}^{(1)},\mathbf{S}_{\mathrm{mlp}}^{(1)},\mathbf{S}_{\mathrm{out}}^{(1)},\ldots,\mathbf{S}_{\mathrm{attn}}^{(B)},\mathbf{S}_{\mathrm{mlp}}^{(B)},\mathbf{S}_{\mathrm{out}}^{(B)}\right).

Each sign code tensor has shape (batch,T,dmodel)(\mathrm{batch}, T, d_{\mathrm{model}}) for the residual-stream codes and (batch,T,dmlp)(\mathrm{batch}, T, d_{\mathrm{mlp}}) for the MLP hidden code, where TT is the sequence length (number of patches plus one [CLS] token). For the experiments in the main paper we flatten over the token dimension.

What is not extracted. The self-attention weights A∈ΔTA \in\Delta^{T} are continuous routing variables and do not admit a binary sign decomposition; they are not included in the sign code. The final layer norm and the classification head do not involve a sign-preserving activation and are excluded.

Spatial compression of convolutional sign codes

Convolutional sign code tensors S(ℓ)∈{−1,+1}C×H×W\mathbf{S}^{(\ell)} \in\{-1,+1\}^{C \times H \times W} can be very high-dimensional (e.g. 256×8×8=16,384256 \times8 \times8 = 16{,}384 bits for a single layer of ResNet-18 on 32×3232 \times32 inputs). To obtain a fixed-length code that is comparable across spatial resolutions, we partition each channel’s H×WH \times W feature map into a P×PP \times P grid of (approximately) equal patches and replace each patch by its majority sign:

Sˉc,p,q(ℓ)=sgn⁡(∑(h,w)∈patch⁡(p,q)Sc,h,w(ℓ)).\bar{\mathbf{S}}_{c,p,q}^{(\ell)} = \operatorname{sgn}\left(\sum_{(h,w)\in\operatorname{patch}(p,q)} S_{c,h,w}^{(\ell)}\right).

This yields C⋅P2C \cdot P^{2} bits per layer. We use P=2P = 2 throughout, which balances spatial resolution against code dimensionality. The majority-vote rule means that Sˉc,p,q(ℓ)=+1\bar{S}_{c,p,q}^{(\ell)} = +1 if more than half of the spatial locations in the patch have a positive activation, preserving the dominant sign in each spatial neighbourhood.

Random projection for high-dimensional codes

Even after spatial compression, the full-network sign code can have D∼104D \sim10^{4}–10510^{5} dimensions. Computing exact Hamming distances between all pairs of nn points in {−1,+1}D\{-1,+1\}^{D} costs O(n2D)O(n^{2}D), which becomes prohibitive for large nn and DD. We reduce the dimensionality by projecting the sign code with a random binary matrix R∈{−1,+1}D×dR \in\{-1,+1\}^{D \times d} (entries i.i.d. Rademacher, d≪Dd \ll D) and re-binarising:

S~(x)=sgn⁡(S(x)R)∈{−1,+1}d.\widetilde{\mathbf{S}}(x) = \operatorname{sgn}(\mathbf{S}(x)R) \in\{-1,+1\}^{d}.

By standard results on random hyperplane hashing, the normalised Hamming distance between two projected codes satisfies E[dH(S~,S~′)]=1πarccos⁡(cos⁡(θ))\mathbb{E}[d_{H}(\widetilde{\mathbf{S}},\widetilde{\mathbf{S}}')] = \frac{1}{\pi}\arccos(\cos(\theta)) where θ\theta is the angle between the original sign vectors treated as real-valued vectors, providing an unbiased approximation to the original Hamming distance. In all experiments labelled “projected” in the main paper, we use d=2,048d = 2{,}048.

Summary of extracted quantities

Table 1 summarises which tensors are recorded for each architecture and how they contribute to the full sign code.

ArchitectureTensorCode typeFull S§Section 3
FFNNh(ℓ)=φ(z(ℓ))h^{(\ell)} = \varphi(z^{(\ell)})Layer sign✓last only
Output logits—××
ResNetStem ReLU outputStem✓×
a=ReLU⁡(bn1⁡(conv1⁡(⋅)))a = \operatorname{ReLU}(\operatorname{bn1}(\operatorname{conv1}(\cdot)))Int.✓×
h(ℓ+1)=ReLU⁡(b+skip⁡)h^{(\ell+1)} = \operatorname{ReLU}(b + \operatorname{skip})Out.✓last only
Avgpool / fc—××
ViTPatch tokens (pre-transformer)Stem××
hattn(ℓ)h_{\mathrm{attn}}^{(\ell)}Attn.✓×
a(ℓ)=GELU⁡(⋅)a^{(\ell)} = \operatorname{GELU}(\cdot)Int.✓×
h(ℓ)h^{(\ell)} (block output)Out.✓last only
Final norm / head—××

Table 1. Summary of sign code extraction across architectures. Int. = internal branch/MLP hidden code; Out. = block-output code; Attn. = post-attention residual stream code. “Full S” indicates inclusion in the full-network sign code; indicates the code used for the last-layer sign-space kk-NN experiment in Section 3.

Which codes are used where. The informational sufficiency experiment (§ 3) uses only the last hidden layer’s sign code: the final hidden layer for FFNNs, the output code of the last BasicBlock for ResNets, and the output code of the last transformer block for ViTs. This is a deliberately conservative choice: a single layer’s binary code—a small fraction of the full-network sign code—already accounts for the network’s generalization performance. The full-network sign code (all layers concatenated) is used in later sections where tighter certificates or mechanistic decompositions require the complete routing record.

Training Details

Training Details & Hyperparameters

Scope. This appendix summarizes the training protocol used for the main correlation sweeps, the intervention experiments, and the repulsion-regularized runs.

Baseline sweeps. The main width–depth heatmaps use CIFAR-10 with the full 50,000-example training set. The cross-dataset scatter in Figure 2b uses three architecture families (FFNN, ResNet-18, ViT), four activations (ReLU, SiLU, GELU, tanh), three datasets (MNIST, CIFAR-10, CIFAR-100), five training-set sizes (1k, 5k, 10k, 25k, 50k), and three random seeds, for a total of 540 trained models.

Optimization.

  • Loss: cross-entropy throughout; label smoothing is zero by default, with ViT on CIFAR-10 using 0.1.

  • Optimizers: AdamW for FFNN and ViT; SGD with Nesterov momentum for ResNet-18.

  • Base learning rates: FFNN uses base learning rate 3×10−43 \times10^{-4}; ResNet-18 uses 10−110^{-1}; ViT uses 1.5×10−41.5 \times10^{-4} on CIFAR-10.

  • Scaling rule: for FFNN sweeps, the effective learning rate is

lreff=3×10−4⋅clip⁡((1024w)1/2(6d)1/2,0.1,1.0).\mathrm{lr}_{\mathrm{eff}} = 3 \times10^{-4} \cdot\operatorname{clip}\left(\left(\frac{1024}{w}\right)^{1/2}\left(\frac{6}{d}\right)^{1/2}, 0.1, 1.0\right).

with an additional factor 0.2 for tanh. For ViT sweeps, the effective learning rate is

lreff=1.5×10−4⋅clip⁡((1024w)1/2(6d)1/2,0.125,1.0).\mathrm{lr}_{\mathrm{eff}} = 1.5 \times10^{-4} \cdot\operatorname{clip}\left(\left(\frac{1024}{w}\right)^{1/2}\left(\frac{6}{d}\right)^{1/2}, 0.125, 1.0\right).
  • Schedules: FFNN uses a constant learning rate in the baseline sweeps. ResNet-18 and ViT use cosine decay; ResNet-18 uses 5 warmup epochs and ViT uses 10 warmup epochs.

  • Batch sizes: 256 for FFNN, 64 for ResNet-18, and 128 for ViT.

  • Weight decay: 10−410^{-4} for FFNN, 5×10−45 \times10^{-4} for ResNet-18, and 5×10−25 \times10^{-2} for ViT.

  • Mixed precision: enabled for all large sweeps unless explicitly disabled.

Stopping rule. The main heatmap sweeps use a maximum of 120 epochs together with a practical early-stop rule based on training accuracy: 0.95 for FFNN, 0.995 for ResNet-18, and 0.98 for ViT. These thresholds were used as a compute-saving heuristic once the model had clearly fit the training set; they were not used in the repulsion runs, where late-training dynamics are part of the result.

Data and augmentation.

  • Splits: standard train/test splits; no held-out validation split.

  • Augmentation: ResNet-18 and ViT use RandomCrop(32, padding = 4) and RandomHorizontalFlip on CIFAR; horizontal flips are disabled for SVHN. FFNN uses no data augmentation.

  • Normalization: standard per-dataset mean/std normalization.

Initialization and normalization. FFNN layers use Kaiming initialization for ReLU/SiLU/GELU and Xavier initialization for tanh. ResNet-18 uses the standard convolutional and batch-normalized residual-block parameterization. ViT uses the standard pre-norm transformer parameterization with layer norm.

Seeds and counts. The fixed-size cross-dataset sweep uses three seeds per configuration. The main width–depth heatmaps use one seed per grid point. The CIFAR-10 heatmaps in Figure 2a comprise 132 FFNN models, 108 ResNet-18 models, and 113 ViT models (353 total).

Intervention experiments. The sign-flip and magnitude-replacement experiments use corruption rates

0,0.01,0.02,0.05,0.1,0.15,0.2,0.25,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0,0, 0.01, 0.02, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0,

with five independent corruption masks per setting, applied to all hidden layers.

Repulsion runs. The repulsion experiments are trained on SVHN with longer schedules (maximum 2500 epochs), AdamW, the same FFNN learning-rate rule, and no early stopping. All other baseline hyperparameters are kept fixed unless explicitly overridden by the regularizer.

Compute budget. The full project required approximately 400 H100 GPU-hours on a 4×H100 cluster, including the main sweeps, intervention experiments, repulsion reruns, and auxiliary failed/restarted jobs. This number should be understood as a rough end-to-end estimate rather than an exact accounting.

Architecture Specifications

FFNN. Inputs are flattened to vectors. All hidden layers share the same width ww, and the depth dd equals the number of hidden layers. The activation is ReLU, SiLU, GELU, or tanh, followed by a linear classifier. There is no final normalization layer before the output head.

ResNet-18. We use a CIFAR-style residual network rather than the ImageNet torchvision stem: the input stem is a 3×33 \times3 convolution with stride 1 and no max-pooling. Residual blocks use batch normalization, and the residual branch is scaled by β/B\beta/\sqrt{B} with β=1\beta= 1 and BB the total number of residual blocks. Channel widths follow (1,2,4,8)×(1, 2, 4, 8) \times (base width), scaled uniformly by the chosen width multiplier.

Depth allocation for ResNet-18. When the total residual depth is varied, blocks are distributed across the four stages by round-robin allocation. Concretely, depth d=5d = 5 gives stage layout (2,1,1,1)(2, 1, 1, 1), depth d=6d = 6 gives (2,2,1,1)(2, 2, 1, 1), and depth d=10d = 10 gives (3,3,2,2)(3, 3, 2, 2).

ViT. The ViT models use patch size 4, a learned class token, learned positional embeddings, pre-norm transformer blocks, multi-head self-attention with 8 heads, an MLP ratio of 4, and a linear classifier on the [CLS] token. Width controls the embedding dimension and depth controls the number of transformer blocks.

Sweep grids.

  • FFNN (Figure 2a): widths {4,8,16,32,64,128,256,512,1024,2048,4096}\{4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096\} and depths {1,…,12}\{1, \ldots, 12\}.

  • ResNet-18 (Figure 2a): widths {1,2,4,8,16,32,64,128,256}\{1, 2, 4, 8, 16, 32, 64, 128, 256\} and depths {1,…,12}\{1, \ldots, 12\}.

  • ViT (Figure 2a): widths {8,16,32,64,128,256,512,1024,2048,4096}\{8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096\} and depths {1,…,12}\{1, \ldots, 12\}, with the seven largest models (4096, 6–12) omitted.

  • Purity/repulsion grid (Figure 4): FFNN widths {4,16,64,256,512,1024}\{4, 16, 64, 256, 512, 1024\} and depths {1,2,4,6,8,10}\{1, 2, 4, 6, 8, 10\}; ResNet-18 widths {1,4,16,32,64,128}\{1, 4, 16, 32, 64, 128\} and depths {1,3,5,7,8,9,10}\{1, 3, 5, 7, 8, 9, 10\}.

Reference parameter counts. For CIFAR-sized inputs, representative models have:

  • FFNN w=1024w=1024, d=8d=8: 10,504,202 parameters.

  • ResNet-18 w=16w=16, d=8d=8: 701,466 parameters.

  • ViT w=256w=256, d=6d=6: 4,771,082 parameters.

Repulsion Regularizer

Definition. For a batch with labels yy, we form a continuous proxy for the sign or gate pattern by concatenating selected pre-activations across layers, applying a temperature-scaled proxy, g=σ(z/τ)g = \sigma(z/\tau) for ReLU/SiLU/GELU and g=tanh⁡(z/τ)g = \tanh(z/\tau) for tanh, and then ℓ2\ell_2-normalizing. The repulsion loss penalizes within-class alignment:

Lrep=1∣P∣∑(i,j)∈P(giTgj)2,P={(i,j):i≠j,yi=yj}.\mathcal{L}_{\mathrm{rep}} = \frac{1}{|\mathcal{P}|} \sum_{(i,j)\in\mathcal{P}} (g_i^\mathsf{T} g_j)^2,\qquad\mathcal{P} = \{(i,j): i \ne j, y_i = y_j\}.

The total loss is

L=LCE+λ(t)Lrep.\mathcal{L} = \mathcal{L}_{\mathrm{CE}} + \lambda(t)\mathcal{L}_{\mathrm{rep}}.

Schedule and layers. The regularization weight is linearly ramped from zero:

λ(t)=λ0⋅min⁡(1,max⁡(0,t−t0T)).\lambda(t) = \lambda_0 \cdot\min\left(1,\max\left(0,\frac{t-t_0}{T}\right)\right).

For the Figure 4 runs, t0=5t_0 = 5 epochs and T=200T = 200. The proxy is computed from the pre-activations of all FFNN hidden layers.

Training protocol. The repulsion runs keep the baseline optimizer and data pipeline, but use a long schedule (2500 epochs) with no early stopping so that the late-stage collapse in test accuracy is visible. In the main SVHN experiments, the regularizer uses τ=0.005\tau= 0.005 and λ0=500\lambda_0 = 500.

Sensitivity. The effect is not confined to a single regularization strength: in additional SVHN reruns, both λ0=250\lambda_0 = 250 and λ0=500\lambda_0 = 500 preserved near-perfect training accuracy while producing the same qualitative collapse in test accuracy and purity.

Algorithm 1 Repulsion-regularized training

  1. for each minibatch (x,y)(x, y) do

  2. compute logits and hidden pre-activations {z(ℓ)}\{z^{(\ell)}\}

  3. form proxy features g←concat⁡ℓϕ(z(ℓ)/τ)g \leftarrow\operatorname{concat}_{\ell}\phi(z^{(\ell)}/\tau), where ϕ=σ\phi= \sigma or tanh

  4. normalize each gi←gi/(∥gi∥2+ε)g_i \leftarrow g_i/(\lVert g_i\rVert_2 + \varepsilon)

  5. form same-class pairs P={(i,j):i≠j,yi=yj}\mathcal{P} = \{(i,j): i \ne j, y_i = y_j\}

  6. compute Lrep=∣P∣−1∑(i,j)∈P(giTgj)2\mathcal{L}_{\mathrm{rep}} = |\mathcal{P}|^{-1}\sum_{(i,j)\in\mathcal{P}}(g_i^{\mathsf{T}}g_j)^2

  7. compute L=LCE+λ(t)Lrep\mathcal{L} = \mathcal{L}_{\mathrm{CE}} + \lambda(t)\mathcal{L}_{\mathrm{rep}}

  8. backpropagate L\mathcal{L} and update parameters

  9. end for

Regularizer-specific hyperparameters.

  • Weight: λ0=500\lambda_0 = 500 in the main runs.

  • Temperature: τ=0.005\tau= 0.005.

  • Normalization constant: ε=10−8\varepsilon= 10^{-8}.

  • Proxy: sigmoid for ReLU/SiLU/GELU; tanh for tanh.

  • Layers: all hidden layers.

Sign-kNN Protocol & Sensitivity

Protocol. Sign-kkNN is computed from the last-layer sign code. Codes are stored as {−1,+1}\{-1, +1\} vectors and compared by dot product, which is equivalent to Hamming distance up to an affine transformation. The main-paper sweeps use a fixed k=15k = 15 with majority-vote prediction and deterministic tie-breaking by class count. Training examples form the reference bank and test examples are used as queries.

Random projection. When projected sign-kkNN is reported, the code is first mapped by a random {±1}\{\pm1\} projection to dimension d=512d = 512 and averaged over five independent projections. The main sign-kkNN results in the paper use the full sign code rather than the projected code.

Sensitivity to kk. To check that the main conclusions are not sensitive to the choice of kk, we reevaluated three overparameterized SVHN SiLU FFNN checkpoints using a 20k-example training bank and 5k test queries. The table reports accuracy (not error rate):

Modelmodel acc.k=1k=1k=3k=3k=5k=5k=10k=10k=20k=20
w=256w=256, d=8d=80.8170.7990.8070.8080.8090.809
w=1024w=1024, d=8d=80.8260.7770.7700.7800.7800.778
w=1024w=1024, d=10d=100.8300.8230.8260.8250.8260.825

Table 1 (PDF p. 22).

Across the regime where sign-kkNN is already close to the network accuracy, performance varies only weakly with kk; changing kk from 1 to 20 alters accuracy by at most a few points. The w=1024w=1024, d=8d=8 row sits near the transition region, where the sign gap is still visible but already small enough to preserve the same qualitative conclusion.

The Canonical Gauge, the Logit Decomposition, and the Gate Domination Bound

This appendix provides a self-contained derivation of the logit decomposition (Eq. (9)), explains why the conditional mean magnitude μj(s)\mu_j(s) is the unique optimal choice of reference magnitude—the canonical gauge—and gives the full proof of the gate domination bound (Theorem 6.1).

Setup and notation

Recall from Definition 2.2 that every neuron jj in the last hidden layer produces an output

hj(x)=Sj(x)⋅Mj(x),Sj(x)=sgn⁡(hj(x))∈{−1,+1},Mj(x)=∣hj(x)∣≥0.h_j(x)=S_j(x)\cdot M_j(x),\qquad S_j(x)=\operatorname{sgn}(h_j(x))\in\{-1,+1\},\qquad M_j(x)=|h_j(x)|\ge0.

Sign convention. We adopt sgn⁡(0)=−1\operatorname{sgn}(0)=-1 throughout. For ReLU, this assigns the entire inactive region {x:zj(x)≤0}\{x:z_j(x)\le0\} to the negative sign (Sj=−1S_j=-1), so Sj∈{−1,+1}S_j\in\{-1,+1\} is well defined for every input. For smooth sign-preserving activations (SiLU, GELU, tanh), only the boundary {zj=0}\{z_j=0\} is affected by this convention; under an absolutely continuous pre-activation distribution, this set has probability zero.

The class-cc logit is

fc(x)=∑j=1dwcjSj(x)Mj(x)+bc.(16)f_c(x)=\sum_{j=1}^{d}w_{cj}S_j(x)M_j(x)+b_c. \tag*{(16)}

We write S(x)=(S1(x),…,Sd(x))∈{−1,+1}d\mathbf{S}(x)=(S_1(x),\ldots,S_d(x))\in\{-1,+1\}^{d} for the full last-layer sign code, where dd is the last hidden layer width.

Our goal is to split (16) into a part that depends only on S(x)\mathbf{S}(x) and a remainder. Any such split requires choosing, for each neuron jj, a reference magnitude νj(s)\nu_j(s) that stands in for MjM_j when we know only that Sj=sS_j=s. Different choices of νj\nu_j yield different decompositions; the question is which choice is best.

Tie-breaking convention. Throughout this appendix, both the full classifier f^(x)=arg⁡max⁡cfc(x)\hat{f}(x)=\arg\max_c f_c(x) and the sign-only classifier f~^(x)=arg⁡max⁡cf~c(x)\hat{\tilde{f}}(x)=\arg\max_c\tilde{f}_c(x) use a fixed deterministic tie-breaking rule (e.g. lowest index). We assume that ties fa(x)=fc(x)f_a(x)=f_c(x) and f~a(x)=f~c(x)\tilde{f}_a(x)=\tilde{f}_c(x) occur with probability zero under the data distribution D\mathcal{D}. This is a generic nondegeneracy condition; in particular, the sign-only logits f~c\tilde{f}_c depend only on the discrete sign code S∈{−1,+1}d\mathbf{S}\in\{-1,+1\}^{d} and therefore take finitely many values, so ties in the sign-only classifier can occur with positive probability unless the weights and biases place distinct logit values on every occupied sign code. In practice this nondegeneracy holds for all trained models we tested (see Section G.4); when it fails, the bound remains valid up to an additive correction equal to the tie probability.

The general decomposition for an arbitrary gauge

Fix any reference-magnitude function νj:{−1,+1}→R≥0\nu_j:\{-1,+1\}\to\mathbb{R}_{\ge0} for each neuron jj. Substituting Mj=νj(Sj)+[Mj−νj(Sj)]M_j=\nu_j(S_j)+[M_j-\nu_j(S_j)] into (16):

fc(x)=∑jwcjSjνj(Sj)+bc+∑jwcjSj[Mj−νj(Sj)]≡f~c(ν)(x)⏟sign-only logit+Rc(ν)(x)⏟residual.(17)\begin{aligned} f_c(x)&=\sum_j w_{cj}S_j\nu_j(S_j)+b_c+\sum_j w_{cj}S_j\left[M_j-\nu_j(S_j)\right]\\ &\equiv\underbrace{\tilde{f}^{(\nu)}_c(x)}_{\text{sign-only logit}}+\underbrace{R^{(\nu)}_c(x)}_{\text{residual}}. \tag*{(17)} \end{aligned}

The sign-only logit f~c(ν)\tilde{f}^{(\nu)}_c depends on xx only through S(x)\mathbf{S}(x); the residual Rc(ν)R^{(\nu)}_c absorbs everything else. Equation (17) is an exact identity—it holds for every xx and every choice of ν\nu, with no approximation.

Remark F.1 (Codomain of νj\nu_j). We define νj:{−1,+1}→R≥0\nu_j:\{-1,+1\}\to\mathbb{R}_{\ge0} (not R>0\mathbb{R}_{>0}) because for ReLU, μj(−1)=E[∣hj∣∣Sj=−1]=0\mu_j(-1)=\mathbb{E}[|h_j|\mid S_j=-1]=0, and the gauge must be able to take this value.

Why the conditional mean is the canonical gauge

For each neuron jj and each sign value s∈{−1,+1}s\in\{-1,+1\}, define the conditional mean magnitude

μj(s)=E[Mj∣Sj=s],(18)\mu_j(s)=\mathbb{E}\left[M_j\mid S_j=s\right], \tag*{(18)}

which is well defined whenever Pr⁡(Sj=s)>0\Pr(S_j=s)>0.

Proposition F.2 (MMSE optimality of the conditional mean). For each neuron jj, assuming E[Mj2]<∞\mathbb{E}[M_j^2]<\infty, the conditional mean μj(s)=E[Mj∣Sj=s]\mu_j(s)=\mathbb{E}[M_j\mid S_j=s] minimises the mean squared prediction error

E[(Mj−νj(Sj))2](19)\mathbb{E}\left[(M_j-\nu_j(S_j))^2\right] \tag*{(19)}

over all functions νj:{−1,+1}→R≥0\nu_j:\{-1,+1\}\to\mathbb{R}_{\ge0}. The minimiser is unique for every ss with $\Pr(S_j=s)>0.

Proof. For each fixed s∈{−1,+1}s \in\{-1,+1\} with Pr⁡(Sj=s)>0\Pr(S_j=s)>0, the bias–variance decomposition gives

E[(Mj−νj(s))2∣Sj=s]=Var⁡(Mj∣Sj=s)+(μj(s)−νj(s))2.\mathbb{E}\left[(M_j-\nu_j(s))^2 \mid S_j=s\right] = \operatorname{Var}(M_j \mid S_j=s) + \left(\mu_j(s)-\nu_j(s)\right)^2.

The first term is independent of νj(s)\nu_j(s). The second is non-negative and vanishes if and only if νj(s)=μj(s)\nu_j(s)=\mu_j(s). Averaging over SjS_j completes the proof.

The conditional mean μj(s)\mu_j(s) is the unique minimum mean-squared-error (MMSE) predictor of MjM_j from the single sign bit Sj=sS_j=s. It extracts exactly the information about MjM_j that is available in SjS_j, no more and no less.

Effect of gauge choice on the residual

Lemma F.3 (Gauge change is a sign code-dependent shift). For any two gauges ν\nu and μ\mu,

Rc(ν)(x)=Rc(μ)(x)+Δc(S(x)),Δc(S):=∑jwcjSj[μj(Sj)−νj(Sj)].(20)R_c^{(\nu)}(x) = R_c^{(\mu)}(x) + \Delta_c(S(x)), \qquad\Delta_c(S) := \sum_j w_{cj}S_j\left[\mu_j(S_j)-\nu_j(S_j)\right]. \tag*{(20)}

Proof. From Rc(ν)=∑jwcjSj(Mj−νj(Sj))R_c^{(\nu)}=\sum_j w_{cj}S_j(M_j-\nu_j(S_j)):

Rc(ν)=∑jwcjSj[(Mj−μj(Sj))+(μj(Sj)−νj(Sj))]=Rc(μ)+Δc(S).R_c^{(\nu)} = \sum_j w_{cj}S_j\left[(M_j-\mu_j(S_j))+(\mu_j(S_j)-\nu_j(S_j))\right] = R_c^{(\mu)}+\Delta_c(S).

:::

Corollary F.4 (Conditional variance is gauge-invariant). Since Δc(S)\Delta_c(S) is a constant conditional on SS:

Var⁡[Rc(ν)∣S]=Var⁡[Rc(μ)∣S]\operatorname{Var}[R_c^{(\nu)} \mid S]=\operatorname{Var}[R_c^{(\mu)} \mid S] for every gauge ν\nu.

Activation-specific forms of the canonical gauge

ReLU. μj(−1)=0\mu_j(-1)=0, μj(+1)=E[zj∣zj>0]\mu_j(+1)=\mathbb{E}[z_j\mid z_j>0]. The sign code recovers the classical gate vector gj=1[zj>0]g_j=\mathbf{1}[z_j>0] under Sj=2gj−1S_j=2g_j-1.

SiLU and GELU. These attenuate but do not zero out the negative side: 0<μj(−1)≪μj(+1)0<\mu_j(-1)\ll\mu_j(+1), with magnitude asymmetry δj∈[0.4,0.95]\delta_j\in[0.4,0.95] across tested configurations (Appendix § 16).

Tanh. Because tanh is odd and trained networks produce approximately symmetric pre-activations, μj(+1)≈μj(−1)\mu_j(+1)\approx\mu_j(-1) (i.e. δj≈0\delta_j\approx0). The sign-only logit simplifies to f~c(x)=∑jwcjμˉjSj(x)+bc\tilde f_c(x)=\sum_j w_{cj}\bar{\mu}_jS_j(x)+b_c where μˉj=12(μj(+1)+μj(−1))\bar{\mu}_j=\frac{1}{2}(\mu_j(+1)+\mu_j(-1)).

The pairwise margin decomposition

From (17) with the canonical gauge ν=μ\nu=\mu, the pairwise margin between classes aa and cc decomposes as:

Ma,c(x)=fa(x)−fc(x)=f~a(x)−f~c(x)⏟M~a,c(x)+Ra(x)−Rc(x)⏟Ra,c(x).(21)M_{a,c}(x)=f_a(x)-f_c(x)=\underbrace{\tilde f_a(x)-\tilde f_c(x)}_{\widetilde M_{a,c}(x)}+\underbrace{R_a(x)-R_c(x)}_{R_{a,c}(x)}. \tag*{(21)}

where the sign-only margin M~a,c(x)=∑j(wa,j−wc,j)Sj(x)μj(Sj(x))+(ba−bc)\widetilde M_{a,c}(x)=\sum_j(w_{a,j}-w_{c,j})S_j(x)\mu_j(S_j(x))+(b_a-b_c) depends on xx only through the sign code S(x)S(x), and the pairwise residual Ra,c(x)=∑j(wa,j−wc,j)Sj(x)[Mj(x)−μj(Sj(x))]R_{a,c}(x)=\sum_j(w_{a,j}-w_{c,j})S_j(x)[M_j(x)-\mu_j(S_j(x))]. The full network agrees with the sign-only classifier whenever ∣Ra,c(x)∣<∣M~a,c(x)∣|R_{a,c}(x)|<|\widetilde M_{a,c}(x)| for every competitor class cc.

Full proof of the gate domination bound

We restate the theorem for convenience and give the complete proof.

Notation and definitions

For a test input xx with full-model predicted class y^=f^(x)\hat{y}=\hat{f}(x) and each competitor c≠y^c\ne\hat{y}, recall from the main text:

  • The per-neuron vote bias mc(x)=1d[∑j(w^y,j−wc,j)Sj(x)μj(Sj(x))+(by^−bc)]m_c(x)=\frac{1}{d}\left[\sum_j(\hat{w}_{y,j}-w_{c,j})S_j(x)\mu_j(S_j(x))+(b_{\hat{y}}-b_c)\right], so that M~y^,c(x)=d⋅mc(x)\widetilde{M}_{\hat{y},c}(x)=d\cdot m_c(x).

  • The conditional residual moment rc2(x)=E[Ry^(x),c2(X)∣S(X)=S(x),f^(X)=y^(x)]r_c^2(x)=\mathbb{E}[R_{\hat{y}(x),c}^2(X)\mid S(X)=S(x),\hat{f}(X)=\hat{y}(x)].

  • The pair-specific and overall gate-domination scores:

Z1,c(x)=∣M~y^(x),c(x)∣rc(x),Z1(x)=min⁡c≠y^(x)Z1,c(x).(22)Z_{1,c}(x)=\frac{\left|\widetilde{M}_{\hat{y}(x),c}(x)\right|}{r_c(x)},\qquad Z_1(x)=\min_{c\ne\hat{y}(x)}Z_{1,c}(x). \tag*{(22)}

Why we condition on both S and f^\hat{f}

The sign-only margin M~y^,c\widetilde{M}_{\hat{y},c} is a deterministic function of S, but the class pair (y^,c)(\hat{y},c) is not: two inputs sharing the same sign code can have different full-model predictions (since the full model also depends on magnitudes). Conditioning on (S,f^)(S,\hat{f}) jointly fixes both the threshold ∣M~y^,c∣\left|\widetilde{M}_{\hat{y},c}\right| and the class pair, making the Markov step valid.

The bound

Theorem F.5 (Gate domination—restated). Let ϵ1=Pr⁡x(f^(x)≠f~^(x))\epsilon_1=\Pr_x(\hat{f}(x)\ne\hat{\tilde{f}}(x)). Assume that ties in both classifiers occur with probability zero under DD, and that all conditional residual second moments rc2(x)r_c^2(x) are finite (guaranteed on bounded-input datasets with finite network parameters).

(i) Disagreement bound.

ϵ1≤(C−1)Ex[Z1−2(x)].(23)\epsilon_1\le(C-1)\mathbb{E}_x[Z_1^{-2}(x)]. \tag*{(23)}

(ii) Width scaling. If rc2(x)≤dσˉc2(x)r_c^2(x)\le d\bar{\sigma}_c^2(x) for all competitors c≠y^c\ne\hat{y}, then

Z1(x)≥(min⁡c≠y^∣mc(x)∣σˉc(x))d.(24)Z_1(x)\ge\left(\min_{c\ne\hat{y}}\frac{|m_c(x)|}{\bar{\sigma}_c(x)}\right)\sqrt{d}. \tag*{(24)}

Proof. The proof has two parts.

Part (i): Disagreement bound.

We proceed in three steps.

Step 1: Disagreement requires a margin violation. Suppose f^(x)≠f~^(x)\hat{f}(x)\ne\hat{\tilde{f}}(x). Let y^=f^(x)\hat{y}=\hat{f}(x) and c∗=f~^(x)≠y^c^*=\hat{\tilde{f}}(x)\ne\hat{y}. Since y^=arg⁡max⁡afa(x)\hat{y}=\arg\max_a f_a(x), we have fy^(x)−fc∗(x)>0f_{\hat{y}}(x)-f_{c^*}(x)>0 (strict inequality by the zero-tie assumption). Since c∗=arg⁡max⁡af~a(x)c^*=\arg\max_a\tilde{f}_a(x) and c∗≠y^c^*\ne\hat{y}, we have M~y^,c∗(x)≤0\widetilde{M}_{\hat{y},c^*}(x)\le0. From fy^−fc∗=M~y^,c∗+Ry^,c∗f_{\hat{y}}-f_{c^*}=\widetilde{M}_{\hat{y},c^*}+R_{\hat{y},c^*} with fy^−fc∗>0f_{\hat{y}}-f_{c^*}>0 and M~y^,c∗≤0\widetilde{M}_{\hat{y},c^*}\le0:

∣Ry^,c∗(x)∣≥∣M~y^,c∗(x)∣.(25)\left|R_{\hat{y},c^*}(x)\right|\ge\left|\widetilde{M}_{\hat{y},c^*}(x)\right|. \tag*{(25)}

Disagreement therefore implies that at least one class c∗c^* satisfies (25).

Step 2: Conditional Markov’s inequality. Define the σ\sigma-algebra G=σ(S(X),f^(X))\mathcal{G}=\sigma(S(X),\hat{f}(X)). For each fixed class c∈{1,…,C}c\in\{1,\ldots,C\}, define the event

Bc={c≠f^(X), ∣Rf^(X),c(X)∣≥∣M~f^(X),c(X)∣}.B_c=\left\{c\ne\hat{f}(X),\ \left|R_{\hat{f}(X),c}(X)\right|\ge\left|\widetilde{M}_{\hat{f}(X),c}(X)\right|\right\}.

Step 1 shows that {f^(X)≠f~^(X)}⊆⋃c=1CBc\{\hat{f}(X)\ne\hat{\tilde{f}}(X)\}\subseteq\bigcup_{c=1}^{C}B_c, and hence

1[f^(X)≠f~^(X)]≤∑c=1C1Bc.(26)\mathbf{1}[\hat{f}(X)\ne\hat{\tilde{f}}(X)]\le\sum_{c=1}^{C}\mathbf{1}_{B_c}. \tag*{(26)}

We now bound E[1Bc∣G]\mathbb{E}[\mathbf{1}_{B_c}\mid\mathcal{G}] for each cc. Conditional on G\mathcal{G}, the full-model prediction y^=f^(X)\hat{y}=\hat{f}(X) is fixed. If c=y^c=\hat{y}, then 1Bc=0\mathbf{1}_{B_c}=0. If c≠y^c\ne\hat{y}, the class pair (y^,c)(\hat{y},c) is fixed, the sign-only margin ∣M~y^,c∣|\widetilde{M}_{\hat{y},c}| is a deterministic constant (being a function of SS and the fixed y^\hat{y}), and Markov’s inequality on Ry^,c2R_{\hat{y},c}^{2} gives:

E[1Bc∣G]=1{c≠y^}Pr⁡(Ry^,c2≥M~y^,c2∣G)≤1{c≠y^}E[Ry^,c2∣G]M~y^,c2=1{c≠y^}Z1,c−2(X).(27)\mathbb{E}[\mathbf{1}_{B_c}\mid\mathcal{G}]=\mathbf{1}_{\{c\ne\hat{y}\}}\Pr(R_{\hat{y},c}^{2}\ge\widetilde{M}_{\hat{y},c}^{2}\mid\mathcal{G})\le\mathbf{1}_{\{c\ne\hat{y}\}}\frac{\mathbb{E}[R_{\hat{y},c}^{2}\mid\mathcal{G}]}{\widetilde{M}_{\hat{y},c}^{2}}=\mathbf{1}_{\{c\ne\hat{y}\}}Z_{1,c}^{-2}(X). \tag*{(27)}

This step requires only finite conditional second moments.

Step 3: Integrate and combine. Taking expectations of (26) and applying tower property with (27):

ε1=E[1{f^(X)≠f^~(X)}]≤∑c=1CE[1Bc]=∑c=1CE[E[1Bc∣G]]≤E[∑c≠f^(X)Z1,c−2(X)]≤(C−1)E[Z1−2(X)],(28)\begin{aligned} \varepsilon_1=\mathbb{E}[\mathbf{1}\{\hat{f}(X)\ne\widetilde{\hat{f}}(X)\}]\le\sum_{c=1}^{C}\mathbb{E}[\mathbf{1}_{B_c}]=\sum_{c=1}^{C}\mathbb{E}[\mathbb{E}[\mathbf{1}_{B_c}\mid\mathcal{G}]] \\ &\le\mathbb{E}\left[\sum_{c\ne\hat{f}(X)}Z_{1,c}^{-2}(X)\right]\le(C-1)\mathbb{E}[Z_1^{-2}(X)], \tag*{(28)} \end{aligned}

where the last inequality uses Z1−2=max⁡c≠y^Z1,c−2≥Z1,c−2Z_1^{-2}=\max_{c\ne\hat{y}}Z_{1,c}^{-2}\ge Z_{1,c}^{-2} for each cc, and the sum over c≠f^(X)c\ne\hat{f}(X) has exactly C−1C-1 terms.

Part (ii): Width scaling.

By definition, ∣M~y^,c(x)∣=d∣mc(x)∣|\widetilde{M}_{\hat{y},c}(x)|=d|m_c(x)|. The assumption rc2(x)≤dσˉc2(x)r_c^2(x)\le d\bar{\sigma}_c^2(x) gives rc(x)≤σˉc(x)dr_c(x)\le\bar{\sigma}_c(x)\sqrt{d}. Therefore:

Z1,c(x)=∣M~y^,c(x)∣rc(x)≥d∣mc(x)∣σˉc(x)d=∣mc(x)∣σˉc(x)d.Z_{1,c}(x)=\frac{|\widetilde{M}_{\hat{y},c}(x)|}{r_c(x)}\ge\frac{d|m_c(x)|}{\bar{\sigma}_c(x)\sqrt{d}}=\frac{|m_c(x)|}{\bar{\sigma}_c(x)}\sqrt{d}.

Minimising over c≠y^c\ne\hat{y} gives Z1(x)≥(min⁡c∣mc∣/σˉc)dZ_1(x)\ge(\min_c|m_c|/\bar{\sigma}_c)\sqrt{d}.

Exponential rate under sub-Gaussian tails

The main theorem uses only Markov’s inequality, yielding a polynomial bound in Z1Z_1. Under stronger tail assumptions, the rate improves to exponential. This is not required for the main theorem; we include it to contextualize the empirical contour in Figure 5 uses the exponential form.

Figure 5. (FFNN, SiLU, CIFAR-10; widths 4–4096, depths 1–12, mean over 3 seeds). The yellow line marks the empirical gate-domination region where both the sign gap and sign-only disagreement are small; above it, Δsgn<0.03\Delta_{\mathrm{sgn}} < 0.03 and ε1<0.1\varepsilon_1 < 0.1. (a) Sign code kNN gap Δsgn\Delta_{\mathrm{sgn}}: the sign code matches model test error (gap <0.03<0.03) throughout this region. (b) Median Z1Z_1: increases with both width (d\sqrt{d}-averaging) and depth (stronger vote bias mcm_c). (c) Disagreement rate ε1\varepsilon_1: mirrors Z1Z_1 with the same diagonal phase boundary. (d) Z1Z_1 predicts ε1\varepsilon_1: the disagreement rate decreases monotonically with Z1Z_1 across the plotted configurations, independent of how each model achieves its Z1Z_1 value.

Assumption F.6 (Conditional sub-Gaussianity). Fix G=σ(S(X),f^(X))\mathcal{G}=\sigma(S(X),\hat{f}(X)) and a competitor c≠f^(X)c\ne\hat{f}(X). Conditional on G\mathcal{G}, the pairwise residual Rf^(X),cR_{\hat{f}(X),c} is sub-Gaussian with variance proxy σc2\sigma_c^2:

Pr⁡(∣Rf^(X),c∣≥t∣G)≤2exp⁡(−t22σc2),∀ t≥0.\Pr(|R_{\hat{f}(X),c}|\ge t\mid\mathcal{G})\le2\exp\left(-\frac{t^2}{2\sigma_c^2}\right),\qquad\forall\ t\ge0.

The proxy σc2\sigma_c^2 may differ from the conditional second moment rc2r_c^2; in general σc2≥Var⁡(Ry^,c∣G)\sigma_c^2\ge\operatorname{Var}(R_{\hat{y},c}\mid\mathcal{G}).

Under Assumption F.6, the proof of Part (i) carries through with the sub-Gaussian tail replacing Markov at Step 2, yielding:

ε1≤2(C−1)Ex[exp⁡(−12(Z1(exp))2(x))],(29)\varepsilon_1\le2(C-1)\mathbb{E}_x\left[\exp\left(-\frac{1}{2}\left(Z_1^{(\mathrm{exp})}\right)^2(x)\right)\right], \tag*{(29)}

where Z1,c(exp)(x)=∣M~y^,c(x)∣/σc(x)Z_{1,c}^{(\mathrm{exp})}(x)=|\widetilde{M}_{\hat{y},c}(x)|/\sigma_c(x) and Z1(exp)(x)=min⁡c≠y^Z1,c(exp)(x)Z_1^{(\mathrm{exp})}(x)=\min_{c\ne\hat{y}}Z_{1,c}^{(\mathrm{exp})}(x). This becomes non-vacuous at Z1(exp)>2log⁡(2(C−1))≈2.4Z_1^{(\mathrm{exp})}>\sqrt{2\log(2(C-1))}\approx2.4 for C=10C=10. The diagnostics in Section G.5 indicate that sub-Gaussian concentration holds approximately at moderate widths.

The expanded form of Z1Z_1 for ReLU networks

For ReLU networks, inactive neurons contribute zero magnitude (μj(−1)=0\mu_j(-1)=0), so only the active set J={j:Sj(x)=+1}J=\{j:S_j(x)=+1\} with dact=∣J∣d_{\mathrm{act}}=|J| contributes. The pair-specific score admits the approximation

Z1,c(x)≈dact(x) ρ^y^,c(x)σ^r(x)1+ρ^m,(30)Z_{1,c}(x)\approx\frac{\sqrt{d_{\mathrm{act}}(x)}\,\hat{\rho}_{\hat{y},c}(x)}{\hat{\sigma}_r(x)\sqrt{1+\hat{\rho}_m}}, \tag*{(30)}

where ρ^y^,c\hat{\rho}_{\hat{y},c} is the normalised gate-only margin and σ^r\hat{\sigma}_r the empirical per-neuron relative-magnitude standard deviation. Width provides dact\sqrt{d_{\mathrm{act}}} averaging; depth provides larger ρmin⁡\rho_{\min}. For general sign-preserving activations, the sum runs over all dd neurons and μj(Sj)\mu_j(S_j) replaces μj(+1)\mu_j(+1).

Empirical Diagnostics Supporting the Theorem Conditions

The disagreement bound in Theorem 6.1 requires only finite conditional second moments and nondegeneracy (no ties). The width-scaling statement (Part ii) additionally requires the residual-scaling condition rc2≤dσˉc2r_c^2 \le d\bar{\sigma}_c^2. The interpretation that Z1Z_1 grows with width further relies on positive vote bias (mc>0m_c > 0). We provide empirical diagnostics supporting each of these conditions across the full SVHN width–depth grid (FFNN, SiLU, 12 depths × 10 widths, 120 models). All diagnostics are computed from two passes: canonical-gauge statistics μj(s)\mu_j^{(s)}, σj(s)2\sigma_j^{(s)2} from the training split, and residual diagnostics from the test split. Figure 7 summarises the results.

Figure 7. Empirical diagnostics for the gate-domination theorem conditions (FFNN, SiLU, SVHN; widths 4–1024, depths 1–12). Left: Positive vote bias. Each cell shows the fraction of test inputs for which mc(x)>0m_c(x) > 0 for all competitor classes. In the overparameterized regime (depth ≥4\ge4, width ≥64\ge64), this fraction exceeds 0.80; the sign-only and full classifiers agree on exactly these inputs. Centre: Residual scaling diagnostic. Each cell shows the median of the per-input ratio R~y^,c2(x)/Dc(x)\widetilde{R}_{\hat{y},c}^2(x)/D_c(x), where Dc=∑j(w~y^,j−wc,j)2σj2(Sj)D_c = \sum_j(\widetilde{w}_{\hat{y},j}-w_{c,j})^2\sigma_j^2(S_j) is the diagonal (uncorrelated) variance proxy. Under approximate conditional uncorrelatedness and a CLT approximation over dd neurons, this ratio is approximately χ2(1)\chi^2(1) with median ≈0.45\approx0.45. The observed values are consistent with this prediction in the transitional regime (depth ≥3\ge3, width ≥32\ge32), supporting the residual-scaling condition. Right: Finite second moments. All cells show 1.0: every gauge value, residual, and logit is finite across the entire grid, as guaranteed by bounded SVHN inputs and finite network parameters.

Positive vote bias (mc>0m_c > 0)

For each test input xx, we compute the sign-only margin M~y^,c(x)\widetilde{M}_{\hat{y},c}(x) for every competitor c≠y^c \ne\hat{y} and check whether all margins are positive, i.e. whether mc(x)>0m_c(x) > 0 for every competitor. Figure 7 (left) displays the fraction of test inputs satisfying this condition across the width–depth grid.

Depth is the bottleneck. At depth 1, the all-positive fraction is below 0.35 across all widths: the sign-only classifier disagrees with the full model on most inputs. This is consistent with the theorem’s prediction: at depth 1, no compositional feature extraction has occurred, so per-neuron vote contributions are approximately unbiased (mc≈0m_c \approx0) and no amount of width can compensate.

The transition is sharp. Between depth 2 and depth 4, the fraction jumps from ∼0.50\sim0.50 to ∼0.85\sim0.85 at width 128. By depth 6 and width ≥64\ge64, the fraction exceeds 0.90 across all tested configurations.

Vote bias tracks agreement exactly. In our diagnostic data, the fraction of inputs where the sign-only classifier agrees with the full model equals the fraction where all vote biases are positive, in every row of the grid. Under the no-ties assumption, this is expected: mc(x)>0m_c(x) > 0 for all c≠y^c \ne\hat{y} if and only if f^(x)=y^\hat{f}(x) = \hat{y}. This confirms that the mechanism identified by the theorem—disagreement occurs when the residual overwhelms a negative sign-only margin—is the actual mechanism at work.

Residual scaling (rc2≤dσˉc2r_c^2 \le d\bar{\sigma}_c^2)

The theorem’s condition requires that the conditional residual moment rc2(x)=E[Ry^,c2∣S,f^]r_c^2(x) = \mathbb{E}[R_{\hat{y},c}^2 \mid S,\hat{f}] grows at most linearly in width. We do not estimate rc2(x)r_c^2(x) exactly (this would require grouping test inputs by their full sign code, which is infeasible in high dimensions). Instead, we use the diagonal proxy

Dc(x)=∑j=1d(w^y,j−wc,j)2σSj2(x),(31)D_c(x) = \sum_{j=1}^{d}(\hat{w}_{y,j} - w_{c,j})^2 \sigma_{S_j}^2(x), \tag*{(31)}

which equals rc2(x)r_c^2(x) when the per-neuron residual contributions are exactly uncorrelated conditional on (S,f^)(S,\hat{f}). We examine the per-input ratio ρc(x)=Ry^,c2(x)/Dc(x)\rho_c(x) = R_{\hat{y},c}^2(x) / D_c(x).

Calibration benchmark. If the per-neuron residual contributions were exactly uncorrelated conditional on (S,f^)(S,\hat{f}), then by the central limit theorem (over dd neurons) the standardised residual Ry^,c/DcR_{\hat{y},c}/\sqrt{D_c} would be approximately standard normal, so ρc∼χ2(1)\rho_c \sim\chi^2(1) with median ≈0.455\approx0.455 and mean =1= 1. This benchmark requires both approximate uncorrelatedness and the CLT approximation.

Empirical results. Figure 7 (centre) shows the median of ρc\rho_c across the grid. In the transitional regime (depth ≥3\ge3, width 32–256), the median is 0.4–0.9, consistent with the χ2(1)\chi^2(1) prediction. This supports the interpretation that the diagonal proxy DcD_c correctly captures the typical residual scale, and is therefore consistent with the theorem’s residual-scaling condition rc2=O(d)r_c^2 = O(d). The mean of ρc\rho_c ranges from 1.5 to 3.0, reflecting a heavy right tail: a small fraction of inputs have Ry^,c2≫DcR_{\hat{y},c}^2 \gg D_c. If the condition holds with an effective constant C0≈2C_0 \approx2–3 rather than C0=1C_0 = 1, the width-scaling bound becomes Z1≥(∣mc∣/(C01/2σˉc))dZ_1 \ge(|m_c|/(C_0^{1/2}\bar{\sigma}_c))\sqrt{d}: the pre-factor changes by at most 3≈1.7\sqrt{3} \approx1.7, but the d\sqrt{d} rate is unaffected.

Cross-correlation diagnostic. We directly measure pairwise correlations between per-neuron residual contributions by computing the off-diagonal entries of the sample correlation matrix of the standardised per-unit residuals. The mean absolute off-diagonal correlation ρ^m\hat{\rho}_m satisfies ρ^m≤0.10\hat{\rho}_m \le0.10 for width ≤256\le256 across all depths, supporting the approximate uncorrelatedness interpretation. At width ≥512\ge512, ρ^m\hat{\rho}_m can increase (up to ∼0.6\sim0.6) because most neurons become effectively always-on or always-off, reducing the number of variable units. This degradation occurs precisely where Z1Z_1 is already large and the bound is conservative.

At depth 1, the median ratio exceeds 2.0 at width ≥128\ge128, indicating stronger correlations. However, depth 1 already fails on vote bias (mcm_c not positive), so the scaling interpretation is inapplicable regardless.

Finite second moments

Figure 7 (right) confirms that every cell in the grid has value 1.0: all gauge values μj(s)\mu_j(s), σj2(s)\sigma_j^2(s), all residuals Ry^,c(x)R_{\hat{y},c}(x), all logits, and all magnitudes are finite. This is guaranteed by the bounded support of SVHN images propagated through networks with finite parameters. No special verification is needed beyond confirming the absence of numerical overflow, which we do programmatically.

Nondegeneracy (ties)

The theorem assumes that ties in both classifiers occur with probability zero. We check this with tolerance 10−810^{-8}: the fraction of test inputs where the top two logits differ by less than 10−810^{-8} is exactly 0.0 for both the full model and the sign-only classifier in every cell of the grid.

Summary

The main theorem (Theorem 6.1) requires only finite conditional second moments and nondegeneracy, both of which hold trivially across the entire grid. The diagnostics supporting the width-scaling interpretation and the positive-vote-bias corollary are as follows:

  • Positive vote bias: the fraction of inputs with mc>0m_c > 0 for all competitors exceeds 0.85 for depth ≥4\ge4 and width ≥64\ge64, and exceeds 0.95 for depth ≥6\ge6. The sign-only and full classifiers agree on precisely these inputs.

  • Residual scaling: the per-input diagnostic ratio R2/DcR^2/D_c has median 0.4–0.9 (consistent with the χ2(1)\chi^2(1) benchmark under approximate uncorrelatedness and a CLT approximation) and mean 1.5–3.0 in the transitional regime. Cross-correlations satisfy ρ^m≤0.10\hat{\rho}_m \le0.10 for width ≤256\le256. These diagnostics are consistent with the theorem’s condition rc2=O(d)r_c^2 = O(d), though they do not constitute a direct estimate of the conditional moment rc2(x)r_c^2(x).

  • Finite second moments: trivially satisfied on bounded-input datasets with finite network parameters.

The sub-Gaussian condition required for the exponential remark (Section F.8) holds in the transitional regime (width ≤128\le128, tail ratios ≤1.3\le1.3 at t=2t=2) and degrades at larger widths where the main Markov bound is already sufficient.

Activation Asymmetry and the Limits of Gate Domination

The gate-domination bound (Theorem F.5) is activation-universal: it holds for any sign-preserving activation under the finite-residual-moment and zero-tie conditions stated in the theorem. What does depend on the activation is the size of the gate-domination score Z1Z_1—and therefore how useful the bound is in practice.

This appendix makes that dependence precise. We introduce the magnitude asymmetry ratio δj\delta_j (§H.1), show how it enters the sign-only margin (§H.2), and use it to explain why Z1Z_1 is large for ReLU/SiLU/GELU but remains modest for tanh (§H.3), with quantitative validation across 240 trained models (§H.4). We then discuss the distinct mechanism by which residual architectures produce weak Z1Z_1 (§H.5). Finally, we explain why our repulsion regularizer LrepL_{\mathrm{rep}} uses a sigmoid gate proxy for ReLU-family activations but a tanh sign proxy for tanh networks (§§H.6–H.7).

The magnitude asymmetry ratio

Let μj(s)=E[Mj∣Sj=s]\mu_j(s) = \mathbb{E}[M_j \mid S_j = s] be the sign-conditional mean magnitude from the canonical gauge (Appendix § 14, Eq. (18)). For any neuron jj with μj(+1)+μj(−1)>0\mu_j(+1) + \mu_j(-1) > 0, define

μˉj:=12(μj(+1)+μj(−1)),δj:=μj(+1)−μj(−1)μj(+1)+μj(−1)∈[−1,1].(32)\bar{\mu}_j := \frac{1}{2}\left(\mu_j(+1) + \mu_j(-1)\right), \qquad \delta_j := \frac{\mu_j(+1) - \mu_j(-1)}{\mu_j(+1) + \mu_j(-1)} \in[-1,1]. \tag*{(32)}

For dead neurons with μj(+1)=μj(−1)=0\mu_j(+1) = \mu_j(-1) = 0 we set δj=0\delta_j = 0 by convention. The identity

μj(Sj)=μˉj(1+δjSj)(33)\mu_j(S_j) = \bar{\mu}_j\left(1 + \delta_j S_j\right) \tag*{(33)}

is immediate from the definitions.

The quantity δj\delta_j measures how strongly a neuron’s expected magnitude depends on its sign. Three regimes arise:

  1. ReLU (δj=1\delta_j = 1 exactly). Inactive neurons have μj(−1)=0\mu_j(-1) = 0; the sign bit completely determines whether the neuron contributes.

  1. SiLU / GELU (δj\delta_j large and positive empirically, typically 0.4–0.95). The negative side is attenuated but not zeroed. Whether μj(−1)≪μj(+1)\mu_j(-1) \ll\mu_j(+1) holds in a given network depends on the pre-activation distribution, not only on the activation (see §F.5).

  1. Tanh (δj≈0\delta_j \approx0 empirically). Because tanh is odd, ∣tanh⁡(z)∣=∣tanh⁡(−z)∣|\tanh(z)| = |\tanh(-z)| for all zz. Under the approximately symmetric pre-activation distributions produced by batch-normalised networks, this yields μj(+1)≈μj(−1)\mu_j(+1) \approx\mu_j(-1) and hence δj≈0\delta_j \approx0. This is an empirical statement about trained networks, not a consequence of oddness alone (see the discussion in §F.5).

How asymmetry enters the sign-only margin

Under the canonical gauge, the sign-only logit is f~c(x)=∑jwcjSj(x)μj(Sj(x))+bc\tilde{f}_c(x) = \sum_j w_{cj} S_j(x) \mu_j(S_j(x)) + b_c. Substituting (33) and using Sj2=1S_j^2 = 1:

f~c(x)=∑jwcjμˉjSj(1+δjSj)+bc=∑jwcjμˉjSj⏟sign-linear term+∑jwcjμˉjδj⏟constant asymmetry offset+bc.(34)\begin{aligned} \tilde{f}_c(x) &= \sum_j w_{cj}\bar{\mu}_j S_j(1+\delta_jS_j) + b_c \\ &= \underbrace{\sum_j w_{cj}\bar{\mu}_j S_j}_{\text{sign-linear term}} + \underbrace{\sum_j w_{cj}\bar{\mu}_j\delta_j}_{\text{constant asymmetry offset}} + b_c. \tag*{(34)} \end{aligned}

The pairwise sign-only margin therefore decomposes as

M~a,c(x)=∑j(wa,j−wc,j)μˉjSj(x)⏟M~a,clin(x)+∑j(wa,j−wc,j)μˉjδj⏟Δa,c(δ)+(ba−bc).(35)\widetilde{M}_{a,c}(x) = \underbrace{\sum_j (w_{a,j}-w_{c,j})\bar{\mu}_j S_j(x)}_{\widetilde{M}^{\mathrm{lin}}_{a,c}(x)} + \underbrace{\sum_j (w_{a,j}-w_{c,j})\bar{\mu}_j\delta_j}_{\Delta^{(\delta)}_{a,c}} + (b_a-b_c). \tag*{(35)}

(35) is the key observation. Asymmetry affects gate domination entirely through the numerator of the pair-specific score Z1,c(x)=∣M~y,c(x)∣/σ^y,c(S(x))Z_{1,c}(x)=\lvert\widetilde{M}_{y,c}(x)\rvert/\widehat{\sigma}_{y,c}(S(x)) from Eq. (22); the structure of the disagreement bound itself (Theorem F.5) is unchanged.

Interpretation. The sign-linear term M~a,clin\widetilde{M}^{\mathrm{lin}}_{a,c} depends on the input through S(x)S(x). The asymmetry offset Δa,c(δ)\Delta^{(\delta)}_{a,c} is a constant that depends only on the learned weights and the activation-specific asymmetry profile. When this offset is aligned with the output-weight differences (i.e. has the same sign as M~a,clin\widetilde{M}^{\mathrm{lin}}_{a,c} for correctly classified inputs), it enlarges the sign-only margin and increases Z1Z_1. When δj≈0\delta_j \approx0 for all jj, the offset vanishes and the sign-only margin is governed entirely by the sign-linear term. There is no general monotone lower bound on ∣M~a,c∣\lvert\widetilde{M}_{a,c}\rvert in terms of the mean asymmetry δˉ=dL−1∑jδj\bar{\delta}=d_L^{-1}\sum_j\delta_j alone, because the effect depends on alignment with the class-weight structure.

Why Z1 is strong for SiLU/ReLU but weak for tanh

The gate-domination bound ε1≤2(C−1)Ex[exp⁡(−Z12/2)]\varepsilon_1 \leq2(C-1)\mathbb{E}_x[\exp(-Z_1^2/2)] is activation-universal. Asymmetry does not alter the bound; it alters how large Z1Z_1 typically is.

For ReLU (δj=1\delta_j=1), inactive neurons contribute zero to the sign-only logit and the margin is dominated by the active set. The asymmetry offset Δa,c(δ)\Delta^{(\delta)}_{a,c} is maximal. For SiLU and GELU, δj\delta_j remains strongly positive empirically: positive-sign neurons contribute substantially more than negative-sign neurons, so the offset in (35) amplifies the sign-only margin. This makes large Z1Z_1 values typical in wide, deep feedforward networks with these activations.

For tanh, trained networks typically satisfy μj(+1)≈μj(−1)\mu_j(+1)\approx\mu_j(-1), so δj≈0\delta_j\approx0 and the offset is negligible. The sign-only margin then reduces approximately to the sign-linear term:

M~a,c(x)≈∑j(wa,j−wc,j)μˉjSj(x)+(ba−bc).\widetilde{M}_{a,c}(x) \approx\sum_j (w_{a,j}-w_{c,j})\bar{\mu}_j S_j(x) + (b_a-b_c).

Because tanh is odd, every neuron contributes with comparable magnitude regardless of sign. The margin is therefore the difference of two nearly balanced sums of mixed-sign terms, which tends to cancel and yields a typical scale much smaller than in the asymmetric case. Consequently, Z1Z_1 remains modest even when the sign code is causally dominant under intervention and highly informative.

Empirical validation: tanh versus SiLU

We now verify the predictions of (35) quantitatively. We train 240 FFNN models on CIFAR-10 with each activation (12 depths ×\times 10 widths ×\times 2 seeds) and compute all decomposition quantities from a single forward pass over the test set.

Notation. To compare across models we report the median gate-only margin

γ~med=median⁡xmin⁡c≠y^(x)M~y^,c(x),(36)\widetilde{\gamma}_{\mathrm{med}}=\operatorname{median}_{x}\min_{c\ne\hat{y}(x)}\widetilde{M}_{\hat{y},c}(x), \tag*{(36)}

i.e. the median over test points of the minimum pairwise sign-only margin from (35). We abbreviate this as γ~\tilde{\gamma} in Table 2. The residual noise σ^r\hat{\sigma}_r is the empirical residual-scale estimator associated with the diagonal proxy in Section G.2; δˉ\bar{\delta} is the layer-mean asymmetry δˉL=∑jδj\bar{\delta}_L=\sum_j\delta_j; and Z1Z_1 is the median of the overall gate-domination score from Eq. (22).

Table 2 shows representative configurations; Figure 8 displays the full heatmaps across the width–depth grid, and Figure 9 shows that δˉ\bar{\delta} is a strong predictor of the gate-only margin across models.

Figure 8. Activation asymmetry explains the Z1Z_1 gap between SiLU and tanh. All panels show FFNN on CIFAR-10, widths 4–4096, depths 1–12, averaged over 2 seeds. Top row: the magnitude asymmetry δˉ\bar{\delta} (Eq. (32)) is uniformly large for SiLU and uniformly near zero for tanh. Middle row: gate-only margins γ~med\widetilde{\gamma}_{\mathrm{med}} (Eq. (36)) are 2–3× larger for SiLU, because the asymmetry offset in (35) amplifies the contribution of correctly-routed neurons. Bottom row: residual noise σ^r\hat{\sigma}_r is comparable or larger for SiLU, confirming that the Z1Z_1 gap is entirely due to the margin, not the noise.

Figure 9. Asymmetry predicts gate-only margin. Each point is one trained FFNN on CIFAR-10. SiLU models (orange) span δˉ∈[0.3,0.95]\bar{\delta} \in[0.3, 0.95] with margins up to 3.2; tanh models (blue) cluster at δˉ≈0\bar{\delta} \approx0 with margins below 1.6.

Key observations.

  1. Tanh has δˉ≈0\bar{\delta}\approx0 everywhere. Across all 120 (width, depth) configurations, the median ∣μj(+1)−μj(−1)∣|\mu_j(+1)-\mu_j(-1)| is 0.0014, and 95 of 120 cells have gap ≤0.005\le0.005 (Figure 8, top row). This confirms that the magnitude is effectively independent of the sign in trained tanh networks, as predicted by approximate pre-activation symmetry.

  1. SiLU has large δˉ\bar{\delta}. At small widths, δˉ>0.9\bar{\delta}>0.9 (a near-ReLU regime where the negative side is strongly suppressed). At large widths and depths, δˉ\bar{\delta} decreases to ∼0.4\sim0.4–0.70.7 as pre-activations concentrate closer to zero, but remains far above zero (Figure 8, top row).

  1. σ^r\hat{\sigma}_r is smaller for tanh than for SiLU. SiLU has larger σ^r\hat{\sigma}_r in 118 of 120 cells (Figure 8, bottom row). The residual noise is not the bottleneck for tanh—the signal (margin) is.

  1. Gate-only margins are 2–3× larger for SiLU. At (4096, 12), SiLU achieves γ~med=2.89\tilde{\gamma}_{\mathrm{med}}=2.89 versus tanh’s 0.95 (Figure 8, middle row; Figure 9). This gap, combined with the σ^r\hat{\sigma}_r ratio, explains the order-of-magnitude difference in Z1Z_1.

  1. Despite small Z1Z_1, tanh sign codes are causally necessary. Section 4 showed that tanh networks on SVHN retain 78% accuracy (from 80% baseline) under total magnitude replacement, and collapse to chance under 10% sign corruption. The sign code carries the classification signal; the Z1Z_1 diagnostic cannot certify this because the conditional-mean gauge extracts no asymmetry from an approximately odd activation.

Why residual architectures can have weak Z1Z_1

Residual architectures present a distinct mechanism for weak Z1Z_1.

In a residual block the output is h(ℓ+1)=φ(h(ℓ)+F(ℓ)(h(ℓ)))h^{(\ell+1)}=\varphi(h^{(\ell)}+F^{(\ell)}(h^{(\ell)})), and the sign–magnitude decomposition at the block output mixes the skip connection h(ℓ)h^{(\ell)} (which carries its own sign–magnitude structure) with the branch output F(ℓ)F^{(\ell)}. The last-layer magnitude ∣hj(L)∣|h_j^{(L)}| therefore depends on the entire history of skip connections, creating dependencies that the single-layer conditional mean μj(Sj)\mu_j(S_j) cannot capture. This inflates the effective residual scale σ^y,c(S)\hat{\sigma}_{y,c}(S) in the denominator of Z1,cZ_{1,c}, rather than shrinking the numerator as in the tanh case.

The two mechanisms are thus complementary:

  1. Odd activations (tanh): weak Z1Z_1 primarily because the sign-only margin ∣M~fa,c∣\lvert\widetilde{M}_{\mathrm{fa},c}\rvert is small (the asymmetry offset vanishes).

  1. Residual architectures: weak Z1Z_1 primarily because the residual scale σ^y,c\hat{\sigma}_{y,c} is large (skip-path magnitude dependencies inflate the denominator).

In both cases, the causal intervention results (§ 4) provide the more robust, architecture- and activation-universal evidence that signs carry the decisive classification signal. For ResNet-18 with

SiLU, replacing every magnitude with a random draw preserves 93–95% accuracy across all four tested activations; flipping 10% of sign bits collapses accuracy to chance.

A multi-layer decomposition that conditions on the sign codes of all preceding blocks—rather than using a single last-layer conditional mean—would yield tighter residual estimates and potentially larger Z1Z_1 values. We leave such multi-layer certificates to future work, noting that the single-layer Z1Z_1 is best understood as a diagnostic for feedforward architectures, while the causal intervention framework provides the architecture-universal evidence.

Theoretical quantities across the depth-width grid for a ResNet on Cifar-10 with SiLU activation

Figure 10. Theoretical quantities across the depth-width grid for a ResNet on Cifar-10 with SiLU activation.

Choice of repulsion regularizer: gate proxy vs. sign proxy

The repulsion loss Lrep\mathcal{L}_{\mathrm{rep}} (Eq. (8)) is designed to destroy same-class sharing of the dominant computational pathway. Which pathway is dominant depends on the activation’s asymmetry profile, so the correct continuous proxy for the regularizer depends on the activation.

ReLU, SiLU, and GELU: sigmoid gate proxy

For these activations, δj\delta_j is large and positive: neurons with Sj=+1S_j = +1 contribute far more strongly than neurons with Sj=−1S_j = -1. The dominant computational pathway is therefore the active set {j:gj=1}\{j : g_j = 1\}, where gj=1[zj>0]g_j = \mathbb{1}[z_j > 0]. We regularise the continuous gate proxy

g(x)=σ(z(x)/τ)∈(0,1)D,(37)g(x) = \sigma\left(z(x)/\tau\right) \in(0,1)^D, \tag*{(37)}

which for small temperature τ\tau approximates the binary gate. Penalising (g(xi)⊤g(xj))2(g(x_i)^\top g(x_j))^2 for same-class pairs directly drives their active-set overlap toward zero.

Why gate repulsion is stronger than sign repulsion for non-odd activations. Two inputs with DD neurons, each with roughly D/2D/2 active, can have orthogonal sign codes in {−1,+1}D\{-1,+1\}^{D} while still sharing approximately D/4D/4 co-active neurons. Sign code orthogonality treats agreements on the active side and agreements on the inactive side symmetrically, but only the active side carries substantial forward-pass signal when δj\delta_j is large. Gate repulsion, by contrast, can drive the co-active count all the way to zero, destroying the shared computational channel entirely.

Empirical confirmation. On SVHN (FFNN, width 1024, depth 8, SiLU), sigmoid-gate repulsion achieves the intended effect: training accuracy exceeds 97%, test accuracy collapses to approximately 40%, and leave-one-out purity at k=5k=5 drops to 0.34. Matched sign-based repulsion at the same nominal strength either destabilises training (SiLU/GELU/ReLU, due to steeper gradients from the tanh proxy) or leaves enough active-set overlap for partial generalization to survive.

Tanh networks: sign proxy

Tanh is the important exception. Because δj≈0\delta_j \approx0 in trained tanh networks, the forward pass does not privilege an active set: positive-sign and negative-sign neurons contribute with comparable magnitude. A sigmoid gate proxy would impose an artificial positive-vs-nonpositive split that is not aligned with the tanh network’s geometry.

For tanh models we therefore use the sign proxy

q(x)=tanh⁡(z(x)/τ)∈(−1,1)D,(38)q(x) = \tanh(z(x)/\tau) \in(-1,1)^{D}, \tag*{(38)}

which for small τ\tau approximates the binary sign code Sj=sgn⁡(zj)∈{−1,+1}S_j = \operatorname{sgn}(z_j) \in\{-1,+1\}. This is the correct matched intervention: when the activation treats both sides approximately symmetrically, the relevant binary structure is the sign code itself, not a privileged active set.

Consistency with the rest of the paper. The distinction between gate proxy and sign proxy arises only in the training regularizer. All theoretical diagnostics (the gate-domination score Z1Z_1, the error chain, the sub-Gaussian bound), all empirical measurements (sign-kNN, LOO purity), and all causal interventions (sign flip, magnitude replacement) are reported throughout the paper in the common binary sign-code framework Sj∈{−1,+1}S_j \in\{-1,+1\}. The two frameworks are related by the bijection Sj=2gj−1S_j = 2g_j - 1; the gate representation gj∈{0,1}g_j \in\{0,1\} appears only inside the regularizer where it provides the stronger intervention for non-odd activations.

Summary

  1. The gate-domination theorem (Theorem F.5) is activation-universal; asymmetry does not change the bound itself.

  2. Asymmetry enters through the sign-only margin via the decomposition (35): large positive δj\delta_j amplifies the margin; δj≈0\delta_j \approx0 eliminates the asymmetry offset and leaves only the sign-linear term.

  3. For ReLU, SiLU, and GELU (large δj\delta_j), gate domination is readily achieved in wide, deep feedforward networks.

  4. For tanh (δj≈0\delta_j \approx0) and for residual architectures (large residual scale), Z1Z_1 can remain modest even when the sign code is causally dominant under intervention and informationally sufficient (Table 2; Figure 8 and Figure 9).

  5. The training regularizer is matched to the activation’s geometry: sigmoid gate proxy (37) for ReLU-family activations (where the active set is the dominant channel), tanh sign proxy (38) for tanh (where no such asymmetric split exists).

WidthDepth\multicolumn{4}{c}{SiLU}\multicolumn{4}{c}{Tanh}
6460.948.391.46∼4\sim40.0070.840.86∼0.6\sim0.6
64120.936.531.44∼6\sim60.0030.891.00∼0.5\sim0.5
25660.863.681.40∼5\sim50.0000.871.26∼0.6\sim0.6
256120.711.822.53∼9\sim90.0010.751.17∼0.5\sim0.5
102460.551.192.08∼7\sim70.0010.650.98∼0.5\sim0.5
1024120.410.533.01∼10\sim100.0010.610.85∼0.5\sim0.5
409660.370.492.71∼9\sim90.0000.600.92∼0.5\sim0.5
4096120.660.292.89∼10\sim100.0000.590.95∼0.4\sim0.4

Table 2. Magnitude asymmetry and gate-domination diagnostics for SiLU versus tanh on CIFAR-10 (FFNN, selected configurations, averaged over 2 seeds). Columns under SiLU and Tanh are δˉ\bar{\delta}, σ^r\hat{\sigma}_r, γ~\tilde{\gamma}, and Z1Z_1.

Table 3 collects the evidence across activations and architectures.

ConfigurationSign causal (§ 4)Sign-kNN tracks accuracy (§ 3.1)Z1Z_{1} certifies
FFNN + SiLU/GELU/ReLU✓✓✓
FFNN + Tanh✓✓× (small δ\delta)
ResNet-18 + SiLU✓✓× (skip connections)
ViT + GELU✓✓not tested

Table 3. Summary of gate-domination evidence across activations and architectures.

References

  1. [1]Sanjeev Arora, Rong Ge, Behnam Neyshabur, and Yi Zhang. Stronger generalization bounds for deep nets via a compression approach. In International conference on machine learning, pages 254–263. PMLR, 2018.arxiv.org/abs/1802.05296
  2. [2]Randall Balestiero and Richard Baraniuk. A spline theory of deep networks. In International Conference on Machine Learning, 2018. URL https://api.semanticscholar.org/CorpusID: 51882762.
  3. [3]Peter L Bartlett, Dylan J Foster, and Matus J Telgarsky. Spectrally-normalized margin bounds for neural networks. Advances in neural information processing systems, 30, 2017.arxiv.org/abs/1706.08498
  4. [4]Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal. Reconciling modern machine-learning practice and the classical bias–variance trade-off. Proceedings of the National Academy of Sciences, 116(32):15849–15854, 2019.DOI
  5. [5]Lénaïc Chizat and Francis R. Bach. Implicit bias of gradient descent for wide two-layer neural networks trained with the logistic loss. In Annual Conference Computational Learning Theory, 2020. URL https://api.semanticscholar.org/CorpusID:211076120.
  6. [6]Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. Sharp minima can generalize for deep nets. In International Conference on Machine Learning, 2017. URL https://api.semanticscholar.org/CorpusID:7636159.
  7. [7]Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. ArXiv, abs/2010.11929, 2020. URL https://api.semanticscholar.org/CorpusID:225039882.arxiv.org/abs/2010.11929
  8. [8]Gintare Karolina Dziugaite and Daniel M Roy. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data. arXiv preprint arXiv:1703.11008, 2017.
  9. [9]Stefan Elfwing, Eiji Uchibe, and Kenji Doya. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural Networks, 107:3–11, 2018. ISSN 0893-6080. doi: https://doi.org/10.1016/j.neunet.2017.12.012. URL https://www.sciencedirect.com/science/article/pii/S0893608017302976. Special issue on deep reinforcement learning.
  10. [10]Jonathan Fiat, Eran Malach, and Shai Shalev-Shwartz. Decoupling gating from linearity. ArXiv, abs/1906.05032, 2018. URL https://api.semanticscholar.org/CorpusID:127096892.arxiv.org/abs/1906.05032
  11. [11]Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv: Learning, 2018. URL https://api.semanticscholar.org/CorpusID:53388625.arxiv.org/abs/1803.03635
  12. [12]Matteo Gamba, Adrian Chmielewski-Anders, Josephine Sullivan, Hossein Azizpour, and Marten Bjorkman. Are all linear regions created equal? In International Conference on Artificial Intelligence and Statistics, 2022. URL https://api.semanticscholar.org/CorpusID:247084088.
  13. [13]Chuqin Geng, Nham Le, Xiaojie Xu, Zhaoyue Wang, Arie Gurfinkel, and Xujie Si. Towards reliable neural specifications. In International Conference on Machine Learning, pages 11196–11212. PMLR, 2023.arxiv.org/abs/2210.16114
  14. [14]Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep sparse rectifier neural networks. In International Conference on Artificial Intelligence and Statistics, 2011. URL https://api.semanticscholar.org/CorpusID:2239473.
  15. [15]Antoine Gonon, Nicolas Brisebarre, Elisa Riccietti, and Rémi Gribonval. A path-norm toolkit for modern networks: consequences, promises and challenges. ArXiv, abs/2310.01225, 2023. URL https://api.semanticscholar.org/CorpusID:263605441.arxiv.org/abs/2310.01225
  16. [16]Song Han, Huizi Mao, and William J. Dally. Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding. arXiv: Computer Vision and Pattern Recognition, 2015. URL https://api.semanticscholar.org/CorpusID:2134321.arxiv.org/abs/1510.00149
  17. [17]X. Y. Han, Vardan Papyan, and David L. Donoho. Neural collapse under mse loss: Proximity to and dynamics on the central path. ArXiv, abs/2106.02073, 2021. URL https://api.semanticscholar.org/CorpusID:235352583.arxiv.org/abs/2106.02073
  18. [18]Boris Hanin and David Rolnick. Complexity of linear regions in deep networks. In International Conference on Machine Learning, 2019. URL https://api.semanticscholar.org/CorpusID:59291990.arxiv.org/abs/1901.09021
  19. [19]Fengxiang He, Shiye Lei, Jianmin Ji, and Dacheng Tao. Neural networks behave as hash encoders: An empirical study. arXiv preprint arXiv:2101.05490, 2021.arxiv.org/abs/2101.05490
  20. [20]Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2015. URL https://api.semanticscholar.org/CorpusID:206594692.arxiv.org/abs/1512.03385
  21. [21]Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv: Learning, 2016. URL https://api.semanticscholar.org/CorpusID:125617073.
  22. [22]W. Ronny Huang, Zeyad Ali Sami Emam, Micah Goldblum, Liam H. Fowl, Justin K. Terry, Furong Huang, and Tom Goldstein. Understanding generalization through visualizations. In ICMLBN@NeurIPS, 2019. URL https://api.semanticscholar.org/CorpusID:182952973.arxiv.org/abs/1906.03291
  23. [23]Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantastic generalization measures and where to find them. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=S1gIjJBFvH.
  24. [24]Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large-batch training for deep learning: Generalization gap and sharp minima. ArXiv, abs/1609.04836, 2016. URL https://api.semanticscholar.org/CorpusID:5834589.arxiv.org/abs/1609.04836
  25. [25]Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey E. Hinton. Similarity of neural network representations revisited. ArXiv, abs/1905.00414, 2019. URL https://api.semanticscholar.org/CorpusID:141460329.arxiv.org/abs/1905.00414
  26. [26]Chandrashekar Lakshminarayanan and Amit Singh. Neural path features and neural path kernel : Understanding the role of gates in deep learning. ArXiv, abs/2006.10529, 2020. URL https://api.semanticscholar.org/CorpusID:219792921.arxiv.org/abs/2006.10529
  27. [27]Arthur Lebeurrier, Titouan Vayer, and Rémi Gribonval. Path-conditioned training: a principled way to rescale relu neural networks. ArXiv, abs/2602.19799, 2026. URL https://api.semanticscholar.org/CorpusID:285973448.
  28. [28]Kaifeng Lyu and Jian Li. Gradient descent maximizes the margin of homogeneous neural networks. ArXiv, abs/1906.05890, 2019. URL https://api.semanticscholar.org/CorpusID:189898036.arxiv.org/abs/1906.05890
  29. [29]Guido Montúfar, Razvan Pascanu, Kyunghyun Cho, and Yoshua Bengio. On the number of linear regions of deep neural networks. In Neural Information Processing Systems, 2014. URL https://api.semanticscholar.org/CorpusID:5941770.arxiv.org/abs/1402.1869
  30. [30]Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Ilya Sutskever. Deep double descent: Where bigger models and more data hurt. Journal of Statistical Mechanics: Theory and Experiment, 2021(12):124003, 2021.arxiv.org/abs/1912.02292
  31. [31]Behnam Neyshabur, Ruslan Salakhutdinov, and Nathan Srebro. Path-sgd: Path-normalized optimization in deep neural networks. ArXiv, abs/1506.02617, 2015a. URL https://api.semanticscholar.org/CorpusID:2101905.arxiv.org/abs/1506.02617
  32. [32]Behnam Neyshabur, Ryota Tomioka, and Nathan Srebro. Norm-based capacity control in neural networks. In Conference on learning theory, pages 1376–1401. PMLR, 2015b.DOI
  33. [33]Behnam Neyshabur, Zhiyuan Li, Srinadh Bhojanapalli, Yann LeCun, and Nathan Srebro. Towards understanding the role of over-parametrization in generalization of neural networks. ArXiv, abs/1805.12076, 2018. URL https://api.semanticscholar.org/CorpusID:44130076.arxiv.org/abs/1805.12076
  34. [34]Vardan Papyan, Xuemei Han, and David L. Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences of the United States of America, 117:24652 – 24663, 2020. URL https://api.semanticscholar.org/CorpusID:221172897.DOI
  35. [35]Yookoon Park, Sangho Lee, Gunhee Kim, and David Blei. Unsupervised representation learning via neural activation coding. In International Conference on Machine Learning, pages 8391–8400. PMLR, 2021.arxiv.org/abs/2112.04014
  36. [36]Razvan Pascanu, Guido Montúfar, and Yoshua Bengio. On the number of response regions of deep feed forward networks with piece-wise linear activations. arXiv: Learning, 2013. URL https://api.semanticscholar.org/CorpusID:9254343.arxiv.org/abs/1312.6098
  37. [37]Maithra Raghu, Ben Poole, Jon M. Kleinberg, Surya Ganguli, and Jascha Narain Sohl-Dickstein. On the expressive power of deep neural networks. In International Conference on Machine Learning, 2016. URL https://api.semanticscholar.org/CorpusID:2838204.arxiv.org/abs/1606.05336
  38. [38]Prajit Ramachandran, Barret Zoph, and Quoc V. Le. Searching for activation functions. ArXiv, abs/1710.05941, 2018. URL https://api.semanticscholar.org/CorpusID:10919244.arxiv.org/abs/1710.05941
  39. [39]Thiago Serra, Christian Tjandraatmadja, and Srikumar Ramalingam. Bounding and counting linear regions of deep neural networks. ArXiv, abs/1711.02114, 2017. URL https://api.semanticscholar.org/CorpusID:34019680.arxiv.org/abs/1711.02114
  40. [40]Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson, Suriya Gunasekar, and Nathan Srebro. The implicit bias of gradient descent on separable data. J. Mach. Learn. Res., 19(1):2822–2878, January 2018. ISSN 1532-4435.arxiv.org/abs/1710.10345
  41. [41]Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. In International Conference on Learning Representations, 2017.arxiv.org/abs/1611.03530

Paper details

vlnc:W_fmqzy2wh
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents