WorksMathematics
Spectral outliers of self-coupled covariance matrices with exact zero weights
Abstract
We study the right spectral outliers of , where the columns of are drawn from a Gaussian mixture with a fixed-dimensional signal subspace and each diagonal weight is a function of the class label and of the low-dimensional Gaussian coordinates of the same sample. The effective outlier theory of Ben Arous, Gheissari, Huang and Jagannath for such self-coupled matrices assumes that the weights do not accumulate at zero, which excludes hard gates such as the derivative of a rectified linear unit. Assuming , we prove that exact zero weights do not change the positive right-outlier equation. The proof deletes inactive columns before any linearization, keeps their random normalization, and applies the invertible-weight theory to the exact conditional mark law of the active samples; we show, by conditioning on the marks, that this theory extends to non-Gaussian fixed-dimensional marks. The limiting spectrum acquires the atom at zero, while positive outliers are the roots of a determinant. We obtain multiplicity-preserving eigenvalue counts, absence of spurious right outliers, a projected-eigenvector normalization, an invariant-subspace formula at repeated roots, and the exact finite-dimensional kernel.
Introduction
Random covariance spectra separate macroscopic bulk behaviour from finitely many signal-bearing outliers. In classical spiked models this separation is controlled by deterministic transforms of the bulk law and produces the Baik–Ben Arous–Péché transition [8, 2, 4]. In self-coupled models, the diagonal weight of a sample depends on the same low-dimensional projection that enters the sample itself. Such matrices describe spectral estimators, information matrices, and blocks of neural-network curvature [3, 7].
An effective spectral theory for Gaussian-mixture self-coupling was developed by Ben Arous, Gheissari, Huang and Jagannath [3]. Its outlier proof linearizes the weighted covariance matrix using the inverse of the diagonal weights, and its assumptions forbid weights accumulating at zero. Consequently a hard gate, for example the derivative of a rectified linear unit (ReLU) [10], which vanishes on a set of positive probability, is excluded. We show that this singularity is algebraic rather than spectral: inactive columns can be removed exactly, without changing the effective equation for positive outliers.
Exact zero weights are already covered in a related centred setting. For isotropic standard Gaussian covariates, Kovačević, Zhang and Mondelli [7] treat spectral estimators for multi-index models with a bounded preprocessing function subject only to , so that the weights may vanish with positive probability; they locate the top eigenvalues and determine the eigenvector overlaps with the signal subspace. The present note treats the noncentred Gaussian-mixture model of [3], and gives multiplicity-preserving counts on arbitrary compact intervals to the right of the bulk. Recent local-law and block-Wishart results [5, 9] provide broader context.
The argument has two parts. First, an exact finite-dimensional identity (Section 3) writes as a random scalar multiple of an invertible-weight matrix built from the active columns only; these columns are i.i.d. from a truncated, generally non-Gaussian, mark law. Second, we show (Proposition 4.1) that the invertible-weight theory of [3] holds for arbitrary fixed-dimensional marks that are independent of the orthogonal Gaussian block: conditionally on the marks, every estimate used in the source proof concerns either the Gaussian block with a fixed weight matrix, or laws of large numbers for mark functionals.
Scope. All statements are asymptotic in the proportional regime with fixed summary dimension and fixed number of classes . They require nonnegative weights and a fixed gap between zero and the nonzero weights. Signed weights, positive weights accumulating continuously at zero, growing summary dimension, and uniformity along a data-dependent training trajectory are not covered. The numerical computations in § 12 are illustrations and are not used in any proof. Whether the conclusions persist when the gap tends to zero with , and at what quantitative rate the empirical outliers approach the effective roots, are open questions.
Model and main theorem
Fix with , , , probabilities summing to one, and a positive-semidefinite matrix . For each large , let
satisfy . Choose with orthonormal columns and
No inverse of or is used. If is rank deficient, unused columns of may be chosen arbitrarily outside . Let complete to an orthogonal matrix.
For class , let . Independently over sample
and define
Let , with , and let
where is measurable and
for some . Put
Let be the physical Stieltjes branch
with for and . Let be its probability measure and
For
define, outside the support,
and .
Theorem 2.1 (Zero-atom outlier theorem). If , then almost surely and all assertions reduce to , , and . Assume .
(1) The empirical spectral distribution of converges weakly in probability to , and
(2) For every compact ,
Hence and are real analytic to the right of the bulk.
(3) Let and assume . Then, with probability tending to one,
with algebraic and analytic multiplicity.
(4) For each root *,
The sum of right-root multiplicities is at most .
(5) Every isolated root of multiplicity attracts exactly empirical eigenvalues. Conversely, for every , all empirical eigenvalues above lie within of a bounded effective root.
(6) If is simple, is its empirical eigenvalue, is a unit eigenvector, and , then
If is a unit vector spanning , then after sign choice,
(7) If has multiplicity , choose an isolating interval and let be the corresponding empirical spectral projection. Let
Then , rank with probability tending to one, and
(8) For every finite , almost surely,
Consequently,
The remainder of the note proves Theorem 2.1. § 3–§ 4 contain the reduction and the extension of the invertible-weight theory; § 5–§ 10 derive the individual assertions.
Exact active-column reduction
Define
After a column permutation,
where is and
Therefore
This is an exact finite-dimensional identity. The inactive columns are absent before any determinant identity, Schur complement, or resolvent is introduced.
Lemma 3.1 (Conditional active law). Conditioned on activity, the low-dimensional mark has law
and remains an independent standard Gaussian. Conditioned on , the unordered active columns are distributed as independent draws from this active law.
Proof. The activity event is measurable with respect to and independent of . Thus
Independence across sample indices is preserved under coordinatewise conditioning. Given the number of successes, exchangeability makes their positions uniform, and the unordered successful marks are independent with law (27). □
Remark 3.2. The active is generally truncated or reweighted. It must not be replaced by an unconditioned Gaussian vector: replacing (27) by the unconditioned law would alter the low-rank term. Thus the active columns are not, in general, an ordinary Gaussian mixture.
Because are independent Bernoulli variables of mean ,
almost surely, for example by Hoeffding’s inequality and Borel–Cantelli.
Invertible covariance with fixed-dimensional marks
The invertible-weight theory of [3] treats a weighted covariance matrix whose weights do not accumulate at zero and whose low-dimensional coordinates are Gaussian. After the reduction of § 3 the weights are bounded below, but the low-dimensional mark of an active column has the truncated law (27). This section shows that Gaussianity of the mark is not used.
Statement
Fix , and . Let be a probability law on with
and let denote its first marginal. For each let with , let have orthonormal columns, and let complete it to an orthogonal matrix. Let be i.i.d. with law , and let be i.i.d. and independent of the marks. Define
Let be the Stieltjes transform of the probability measure determined by
with on , let , and set
Expectations are over .
Almost-sure statements refer to one probability space carrying the variables for all , on which the family of all marks (over all ) is independent of the family of all Gaussian vectors (over all ). The marks may be coupled across arbitrarily; the Gaussian vectors are coupled across in any way under which the almost-sure statements of [3] for a Gaussian block independent of a deterministic weight matrix hold (the source does not fix a coupling). Every later use of the proposition (§ 8 and § 10) is through probabilities at fixed , which depend only on the law at that and not on these couplings.
Proposition 4.1 (Invertible covariance with fixed-dimensional marks). In this setting the following hold.
(i) The empirical spectral distribution of converges weakly to almost surely.
(ii) For every compact , . Hence and are real analytic on .
(iii) Let with . Almost surely, for all large , the number of eigenvalues of in , counted with multiplicity, equals the number of zeros of in , counted with order.
(iv) For every there is such that, with , there are random matrix functions , holomorphic on a neighbourhood of for all large , with
and, almost surely, and .
(v) *If , , is a unit eigenvector and , then almost surely and .
Two laws of large numbers
The source proof invokes the strong law of large numbers for triangular arrays. We record the version used here; it requires no coupling between different values of .
Lemma 4.2. Let be a real random variable with . For each let be i.i.d. copies of , with . Then almost surely.
Proof. Put and . In the expansion of over index tuples , a tuple containing an index exactly once contributes zero. The remaining tuples take at most three distinct values, so there are at most of them for an absolute constant , and each contributes at most by Hölder’s inequality. Markov’s inequality gives , which is summable in because eventually. The Borel–Cantelli lemma completes the proof.
Lemma 4.3. Let be i.i.d. with law supported in , and let . Almost surely, weakly and the Hausdorff distance between and tends to zero.
Proof. Lemma 4.2 applied to the countably many bounded variables , , gives almost sure convergence of all moments; on this implies weak convergence by the Weierstrass approximation theorem. Almost surely every lies in , so . Conversely, fix and cover the compact set by finitely many open intervals of length centred at support points. Each has , and the probability that some contains no sample is at most , which is summable in . By Borel–Cantelli, eventually every point of is within of a sample. Take , .
The finite-d transform for a convergent aspect ratio
The source’s finite- transform solves its equation (2.11), which carries the ratio ; this ratio is not constant even when . Here . The following lemma gives the convergence on in this generality.
Lemma 4.4. Let be probability measures on with weakly, and let with . For and a probability measure on , , consider
(a) (35) has at most one solution, and every solution satisfies and .
(b) If solves (35) with and solves it with , then for every . If moreover the are holomorphic on , the convergence is uniform on compact subsets of .
Proof. For and , , so the integrand is defined. Since and , the imaginary part of (35) reads
This gives the integral bound in (a), and also , hence . If are solutions, subtracting the two equations gives
By the Cauchy–Schwarz inequality and the integral bound, the modulus of the integral term is strictly less than , so the bracket is nonzero and .
For (b), fix and put . If , then on and (35) gives . Hence , and by (36), . Consequently for . The sequence is bounded by (a); let be a limit point along a subsequence. Then , so on , and
along the subsequence. Since is bounded and continuous on and weakly,
With , solves (35) for and has positive imaginary part, so by (a). Every subsequence has a further subsequence converging to , so . Under holomorphy, the bound makes locally bounded on , and Vitali’s theorem upgrades pointwise to locally uniform convergence.
The next lemma controls the right edge of the finite- measures; it is used only in the paragraph Avoiding the modification for (ii)–(v) below.
Lemma 4.5. Let be finitely supported probability measures on , where , with weakly, and let . Let and be the probability measures whose Stieltjes transforms solve (35) with and with , respectively. Then .
Proof. For a measure and write . Let and . By the Silverstein–Choi characterization used in the paragraph Denominators below, for and ; since is continuous on and as , in fact on , so . Letting in (35) gives . Now let . By [6 Lemmas 2.4–2.6], applied with , , , where is the unique critical point of in . This function tends to at both ends of that interval, because has an atom at , so is its minimum there. Since ,
On the function is bounded and continuous, because . Weak convergence and give . Hence , and completes the proof.
Proof of Proposition 4.1
All references in this proof are to the numbered statements of [3], version 3, Sections 1.4 and 2.
Normalization. The source works with , where . Write
Let have rows , let have columns , and let . This is the decomposition (2.34)–(2.36) of the source, with the Gaussian vector there replaced by .
The conditioning. Let be the -field generated by the marks for all and . For each , and are -measurable, and, by the independence of the two families, conditionally on the sequence has its unconditional joint law; in particular each has i.i.d. entries. In the source, Assumption 1 (weights are functions of the label and of the Gaussian summary coordinates) enters the proof of Theorem 2.1 only through this structure and through the computation of a mean: it is used in Section 2.6 to conclude that is independent of , while is a function of the same summary coordinates as , and in Lemma 2.18 to write (Step 3(d) below). The remark at the end of Section 2.6 that has full rank by Gaussianity is not used in the proof of Theorem 2.1. We now go through the steps of the source proof and record, for each, what it uses.
Step 1: the weights alone. Because almost surely, is invertible and the empirical measure of is -regular in the sense of Definition 2.2 for every , deterministically: its support lies in and it gives mass to . (The endpoint is excluded: if has an atom at , the mass of can exceed .) Thus Lemma 2.3 holds with probability one. Assumption 2 holds because . Assumption 3 holds almost surely by Lemma 4.3. Lemmas 2.4–2.6 are deterministic statements about -regular measures. Since may be taken arbitrarily small, the ball appearing in Theorem 2.12 may be taken to contain any prescribed compact set.
Step 2: the Gaussian block with fixed weights. The following results of the source involve only and : Proposition 2.7 (bulk law of ), Proposition 2.8 (no eigenvalues of in eventually, a modification of [1]), Theorem 2.9 and Lemmas 2.10–2.11 (deterministic equivalent for the Green function of the linearization ), Theorem 2.12, and Lemma 2.13 (convergence of the finite- transform ). Their hypotheses on are those verified in Step 1 (Proposition 2.7 needs only the weak convergence in Assumption 3; Proposition 2.8 and Theorem 2.12 also use its Hausdorff part, Theorem 2.9 uses invertibility and -regularity, and Lemmas 2.10–2.11 use -regularity and the event of Proposition 2.8), and their only hypothesis linking and is independence. Fix a realization of the marks in the -measurable probability-one event of Step 1 and Lemma 4.2. Along it the sequence is deterministic and satisfies these hypotheses, and is Gaussian and independent of it, so these results apply and their conclusions hold with conditional probability one; integrating over the marks (Fubini’s theorem), they hold almost surely. Theorem 2.12 allows the test vectors to be random provided they are independent of . We apply it with the columns of , where ; these are -measurable, hence independent of conditionally on . Their norms are bounded almost surely, because almost surely by Lemma 4.2 and (30).
Hypotheses of Theorem 2.9. Theorem 2.9 of the source is stated for a fixed , an invertible diagonal whose empirical spectral measure is -regular, and an independent Gaussian matrix with i.i.d. entries. Its conclusion is a bound on for each pair of deterministic unit vectors, with probability , uniformly in . Its proof follows [6], Section 5 and Lemma 6.2. The standing assumptions there [6], Section 2.1 are: the population matrix (here ) is deterministic, and positive definite in the diagonal case; the noise matrix has independent centred entries of variance , where is the dimension of the block carrying , and every moment of times an entry is bounded; the ratio of the dimension of the population matrix to lies in ; the population matrix has norm at most , and its empirical eigenvalue distribution gives mass at most to . The high-probability bounds there are uniform over all data satisfying these conditions with the same [6, Definition 3.4]. Here the population dimension is and ; the letter keeps its meaning as the upper bound of the weights. Fix a realization of the marks as in Step 2 and put . For all large , is deterministic, diagonal and positive definite (the source’s extension to weights of either sign is not needed), , has no eigenvalue in , , the matrix is Gaussian and independent of the marks, and the test vectors used below are -measurable.
Two points are not covered literally. First, the normalization. The source states Theorem 2.9 for a matrix with standard Gaussian entries, and Propositions 2.7–2.8 for with standard Gaussian , but applies all three, in its Sections 2.9–2.10, to the matrix of its Section 2.6. We read them in that normalization, in which the limiting measure is the one solving (2.11). The variance is the one under which (2.11), with ratio , is the correct equation; has variance , as in Section 2.6 of the source. Write and . Then
and Theorem 2.9 applies to and , which is -regular for large because . The resulting deterministic equivalent for the original Green function has diagonal blocks and , where solves (2.11) for the empirical law of the . These laws converge weakly to and are supported in for large , so Lemma 4.4, with in place of , gives and locally uniformly on , the same limit as in item (4) below. The extension to compact subsets of is as in item (4). Since , the same rescaling applies to Propositions 2.7–2.8. This discrepancy, of relative order , is already present in the source, and it changes no limit below.
Second, the source’s Theorem 2.9 does without the lower bound on and the stability condition assumed in [6 Section 3.4 and Definition 5.4]; the source replaces them by bounds in terms of which it takes from [1 (3.20)–(3.21)]. This modification is internal to the source’s proof and is not re-checked here. It is used only at spectral parameters with (Lemma 2.11) and bounded above and below, which lie in the spectral domain of [6 Section 3.1] after decreasing .
Avoiding the modification for (ii)–(v). The outlier assertions do not need the unverified part of the proof of Theorem 2.9. Work in the scale of , let be the limiting measure of , put , and fix with . For (ii)–(v), Steps 2–5 need Lemma 2.11 and Theorem 2.12 only for with and small (the lower half-plane follows from ), so they use Theorem 2.9 only at points with and . Let be the measure whose Stieltjes transform is ; it is the measure of [6 Definition 2.3] for and ratio . By Lemma 4.5, applied to the empirical law of the (support in for large , and converging weakly to ), at these points for all large . These points therefore lie in the domain of [6 Section 3.3], for suitable constants. Theorem 3.16(i) of [6], the anisotropic local law outside the spectrum, assumes only [6 Assumption 2.1 and (2.9)], which were checked above with and . It assumes no regularity of the spectrum, no lower bound on , and no stability condition. For fixed , and deterministic unit vectors , it gives
with probability at least , for all large , uniformly in these ; here and are written for , and the bound follows from . For large this is smaller than the bound of Theorem 2.9. A union bound over a grid of mesh makes it simultaneous in , because and are Lipschitz in there with constants polynomial in on the event , whose complement has probability at most for large. Lemmas 2.10–2.11 and Theorem 2.12, restricted to a complex neighbourhood of , then hold as stated; their proofs still use Proposition 2.8. The modified Theorem 2.9 therefore enters only the alternative argument for (i) in item (2) of the next paragraph.
Aspect ratio. The source states Theorem 2.1 with ; here . The ratio enters the proof in the following places. (1) Proposition 2.8 is first stated for ; in its proof the function is formed with the finite- ratio, and the uniform convergence , on persists because $\phi_d\to c. (2) Proposition 2.7 is quoted by the source from Silverstein and Bai. Independently of how that theorem treats the ratio, its conclusion here also follows from results already listed: applying Theorem 2.9 to the coordinate vectors of the upper block, a union bound and the Borel–Cantelli lemma show that, for each fixed , the normalized trace of minus (in the normalization above) tends to zero almost surely; converges to the limiting transform, as shown above, and convergence of Stieltjes transforms on a countable dense subset of to the Stieltjes transform of a probability measure gives almost sure weak convergence. (3) Theorem 2.9 and Lemmas 2.4–2.6 and 2.10–2.11 are finite- statements written with the finite- ratio of (2.11), which varies with already in the source. (4) Lemma 2.13: by Lemma 4.4(b), applied to the empirical weight laws (which converge weakly by Lemma 4.3) and to , the finite- transforms converge to the limiting one locally uniformly on ; Lemma 4.4(a) also identifies the source’s as the unique solution of (2.11) in . The extension to compact subsets of is then as in the source, through Proposition 2.8 and Lemma 2.10. (5) Lemmas 2.16–2.18 contain the factor ; here it is , see Step 3.
Step 3: the reduced matrix. Theorem 2.14 states that converges to its deterministic limit uniformly on . Its proof uses four ingredients.
(a) Lemma 2.15 compares with a cut-off version; it uses only Step 1, Lemma 2.13 and Lemmas 2.5–2.6.
(b) Lemma 2.16 combines Lemma 2.15 with almost sure boundedness of
which holds by Lemma 4.2 and (4.1).
(c) Lemma 2.17 applies the law of large numbers, at fixed , to i.i.d. summands , where is the deterministic limit and is the smooth cut-off; the cut-off bounds the scalar factor, so all moments are finite by (4.1) and Lemma 4.2 applies. Equicontinuity in uses the same bound times , which converges almost surely. A countable dense set of and the Arzelà–Ascoli theorem give local uniformity, as in the source.
(d) Lemmas 2.18–2.19 identify the mean. Because the summands are i.i.d. with law determined by ,
which converges to ; since does not depend on , Lemma 2.19 is not needed. The source uses Gaussianity of only to rewrite this expectation in terms of and . For the removal of the cut-off as , note that for positive weights is bounded below by some under and in a compact subset of : for nonreal , ; for real , by the Silverstein–Choi characterization used in the paragraph Denominators below (which involves only the limiting equation); continuity and compactness give . For the cut-off is identically one, so the truncated and untruncated limits coincide.
Therefore uniformly on almost surely, where
Step 4: linearization and Schur complement. Since is invertible, Section 2.10 of the source applies verbatim: rotating by , the linearization of becomes
with
By Step 2, is invertible on for all large . Put . Since is the upper-left block of the inverse linearization, the Schur complement formula gives
and the eigenvalues of in are the zeros of , with multiplicity equal to the order of the zero. By Step 2 and the bounded norms of the columns of , uniformly, and by Step 3, uniformly on , almost surely. Since these functions are holomorphic, their derivatives converge uniformly on slightly smaller sets by the Cauchy estimates. Hurwitz’s theorem gives the root counts, exactly as in the source.
Step 5: eigenvectors. Sections 2.11–2.12 of the source use only the linearization identity (2.51)–(2.52), the invertibility of , and the convergence of and from Step 4. They give, for an eigenvalue of with unit eigenvector , that every limit point of satisfies and . In fact the computation there gives, for , the exact identities and : with real symmetric for real , , and the second term evaluated at is the squared norm of the component of orthogonal to . Together with the uniform convergence of and from Step 4 these identities give the limits in (v) directly.
Rescaling. Let be the limiting transform of , so . The equation for becomes (32), and (4.10) gives $\widetilde{F}(\lambda c x)=\lambda cF_c(x)$, hence $\widetilde{F}'(\lambda c x)=F'_c(x)$. Define $B_d(x)=(\lambda c_d)^{-1}\Delta_d(\lambda c_d x)$. Then (4.11) becomes {xref}(4.5)`, and, since , together with the derivative, uniformly on a thin complex neighbourhood of contained in the rescaled set . This proves (iv); (iii) and (v) follow from Steps 4–5, and (i) from Section 2.9 of the source, which uses only Proposition 2.7 and Cauchy interlacing for a perturbation of rank at most .
Denominators. For (ii), the Silverstein–Choi characterization of the complement of the support [11 Theorem 4.1], applied to the bulk measure of , which solves (2.11) with weight law , shows that for real outside the support; this characterization is also the one used in the proof of Proposition 2.8. Equivalently for and . Continuity of on and compactness of give a positive infimum, and real analyticity follows by differentiating under the expectation. □
Remark 4.6 (Where the inverse weight matrix enters). In the source proof the diagonal inverse appears only in the lower-right block of the linearization, in the Schur complement producing the characteristic determinant, in the relation (2.52) between the eigenvector and the linearization vector, and in the differentiation of the Schur complement giving the norm identity. After (26), each occurrence is an inverse of and is bounded by . The bulk theorem does not require the inverse and already permits a zero atom. No Moore–Penrose substitution is made. The rotation-invariance argument of Section 2.13 of the source, which uses the Gaussian law of , is not used; its root bound is replaced by the monotonicity argument of Section 7.
Bulk law, exact scaling, and the zero atom
Under the active law (27), set
This mark law satisfies the hypotheses of Proposition 4.1: , and . Let solve
Define
Since
substitution in (42) yields (8). Inactive terms vanish because the numerator is zero; no inverse of zero is used.
If is the active bulk measure, then
This exact pushforward supplies the right-edge statement without invoking continuity of support under weak convergence.
For the effective matrix,
Using and (43),
which expands exactly to (11). Hence
Roots and analytic multiplicities correspond bijectively under .
Lemma 5.1 (Zero mass). The limiting measure satisfies (12).
Proof. The rank theorem proved in § 6 gives the lower bound
Alternatively, the same lower bound follows from (45) and the standard active covariance rank deficit. Let . For ,
If , then . Dominated convergence in (8) gives
Thus when . If , a positive would imply , which is impossible. Therefore . □
Exact rank and kernel
Lemma 6.1 (General position of active columns). Conditioned on activity, each has an absolutely continuous law on . Independent active columns are in general linear position almost surely.
Proof. Under (27), has a density on its active set, and remains a full standard Gaussian. For each class , the affine orthogonal map
has nonsingular linear part. Its image law is absolutely continuous on . Inductively, conditioned on the first active columns, their span is a proper linear subspace and hence has Lebesgue measure zero. The next active column avoids it almost surely. □
Therefore
Since is invertible,
almost surely. Equation (22) follows from (29). The exact zeros are the finite-dimensional realization of the atom (12); they are not positive outliers.
Analyticity and root multiplicity
On a compact set to the right of the active bulk, Proposition 4.1(ii) supplies
Equations (43) and (47) imply (13). Thus is real analytic on .
For real ,
Differentiating (11),
Hence
Every ordered eigenvalue of the Hermitian matrix is strictly increasing, so each can cross zero at most once and the sum of root multiplicities is at most .
Let be a root and , . In the orthogonal decomposition ,
where is invertible and
The Schur complement on equals , and therefore
This proves (15).
Random active size and empirical outlier counts
Lemma 8.1 (Random-index transfer). Let be a failure probability for a model with active columns. Suppose
for every deterministic sequence . If in probability, then
Proof. Choose such that
If
did not tend to zero, a subsequence and a choice would contradict the deterministic-sequence hypothesis. Splitting the expectation on the typical and atypical events gives (63). □
For each deterministic , Proposition 4.1 gives the exact active eigenvalue count on compact intervals separated from the active bulk; almost sure eventual statements imply that the corresponding failure probabilities tend to zero. Because
one has
The active endpoints converge to and . Uniform determinant convergence in a small endpoint neighbourhood permits these converging endpoints. Equation (48) identifies the active root count with the target root count. The random-index lemma removes conditioning and proves (14).
No spurious right outliers and existence
The operator norm is tight because
Writing , the Gaussian norm satisfies , while
Thus .
Also remains bounded as , while diverges, so effective roots lie in a bounded interval. Fix and a large deterministic . The right roots in are finite. Remove small disjoint neighbourhoods of these roots. The complement is a finite union of compact root-free intervals, and the count theorem gives no empirical eigenvalues there with probability tending to one. This proves the no-spurious direction. Applying the count theorem to an isolating interval, and then to nested isolating intervals, proves existence and convergence at each effective root.
Simple roots and spectral projections
For the active model, Proposition 4.1(v) gives, near an isolated root ,
Multiplication by changes eigenvalues but not eigenvectors. From (47),
Local uniformity and yield (16)–(17). At a simple root, (15) makes the kernel one-dimensional, and (17) fixes the norm, proving (18).
For a repeated root, choose a positively oriented contour enclosing and no other effective root or bulk point, of radius smaller than the strip width in Proposition 4.1(iv). The active compressed resolvent identity (34), the uniform convergence , and the scaling (47) give
The full-space spectral projection is
Thus
The block expansion near gives
The residue is , proving (20). If is any orthonormal basis of and , then
Only the invariant subspace is canonical; individual eigenvectors may rotate.
ReLU gates
Proof. A nonconstant affine functional of a standard Gaussian is a nondegenerate one-dimensional Gaussian, so its positive half-space has probability strictly between zero and one. The weights belong to almost surely. □
For the pure gate ,
is a half-space-gated empirical second moment. For a one-unit ReLU model
at a fixed parameter for which no sample lies at the kink,
The – Jacobian-Gram or Gauss–Newton block is therefore
For squared loss and fixed deterministic , Gaussian data avoid the kink almost surely, the activation pattern is locally constant, and the classical Hessian block equals this matrix. This does not establish a trajectory-uniform theorem, cross-neuron blocks, or a generalized Hessian at a kink. Hard ReLU gates thus create a zero atom, not additional positive outliers.
Examples and numerical illustration
Independent Bernoulli thinning
12.1. Independent Bernoulli thinning. If independently of the columns, then
The continuous bulk edges are
and the zero mass is . Equation (8) becomes
Centred pure gate
12.2. Centred pure gate. Let with and . Conditional on , the only Gaussian preimages are and , with equal probability, and the gate takes complementary values. Therefore
so is independent of . The model is exactly independent Bernoulli thinning with .
Noncentred scalar gate
Let ,
and define
Then
For , set
The scalar effective equation gives the standard spiked-covariance threshold
and, above threshold,
Fixed-epsilon comparison
For comparison only, define
Then
and
This perturbation estimate does not, by itself, prove support-edge or root-multiplicity stability. The exact active reduction avoids that missing limit interchange.
Numerical illustration
Figure 1 shows the noncentred scalar gate. The active conditional moment predicts an outlier at 4.6653, separated from the bulk edge 2. Direct simulations approach the prediction while the associated eigenvectors retain large overlap with the summary direction. Representative values from the same computations are listed below.

Figure 1. Largest eigenvalue for the gate with , , and summary mean . The effective determinant predicts and the bulk edge is 2.
| Quantity | Value |
| Bernoulli model rank | 254 observed, 254 predicted |
| Bernoulli zero mass | 0.20625 observed, 0.2 limiting |
| Bernoulli top eigenvalue | 1.1641 observed, 1.1963 edge |
| Stieltjes residual | |
| Centred gate top eigenvalue | 1.9728, edge 2 |
| Noncentred predicted outlier | 4.6653 |
| Noncentred simulations | 4.48–4.87 |
| Determinant residual | |
| Regularization identity error | |
| Two-construction operator difference |
Table 2.
A two-class check with non-constant weights
The following check, added in the present revision, is numerical evidence only. Take , , , , , and , so that and . Let
where equals 0.5, 1, 1.5 on , , , and equals 0.6, 1.2 on , . This is a gate of the form of § 11 with and . Here , so . Because the weights are piecewise constant, every expectation in (8) and (11) is a finite sum of truncated Gaussian moments. Inverting the map defined by (8) gives . Bracketing the increasing eigenvalues of gives two simple roots, and ; by (18) the predicted values of are 0.8411 and 0.7348. Table 1 compares these with direct simulations of , . In all 80 eigenvector comparisons the
| runs | ||||||
| 500 | 20 | |||||
| 1000 | 10 | |||||
| 2000 | 6 | |||||
| 4000 | 4 | |||||
| prediction | 7.8129 | 4.7294 | edge 3.0976 | 0.8411 | 0.7348 |
Table 1. Two-class model: mean standard deviation over independent runs of the three largest eigenvalues of and of for the top two eigenvectors.
cosine of the angle between and the predicted kernel direction was at least 0.98. For , in all 30 runs, as (21) requires. The same code applied to the scalar gate of Figure 1 gives mean top eigenvalues 4.61 0.13, 4.68 0.10, 4.67 0.06 and 4.62 0.05 for , 1000, 2000, 4000 (20, 10, 6, 4 runs), against 4.6653. The mean of lies between 0.859 and 0.862 for each , against 0.8609 from (18), which here coincides with the classical spiked-covariance overlap.
AI assistance disclosure
OpenAI GPT-5.6 Sol, accessed through ChatGPT and Codex, was used to develop the reduction, the numerical checks and the first draft. Anthropic Claude Opus 5.5, used through Claude Code, was used to merge and edit the note and to write the conditioning argument of § 4, including Lemma 4.4. Anthropic Claude Opus 5.5 (Claude Code) also produced Lemma 4.5, the paragraph of § 4 that replaces the unverified part of Theorem 2.9 of [3] by [6 Theorem 3.16] for the outlier assertions, and the two-class numerical check of § 12. The author reviewed and takes responsibility for every statement and proof. AI systems are not authors of this work, and the work has not been peer reviewed.
References
DEPARTMENT OF PHYSICS, ÉCOLE NORMALE SUPÉRIEURE, UNIVERSITÉ PSL, PARIS, FRANCE
Email address: samü[email protected]
References
- [1]Z. D. Bai and Jack W. Silverstein, No eigenvalues outside the support of the limiting spectral distribution of large-dimensional sample covariance matrices, Annals of Probability 26 (1998), no. 1, 316–345.
- [2]Jinho Baik, Gérard Ben Arous, and Sandrine Péché, Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices, Annals of Probability 33 (2005), no. 5, 1643–1697.DOI
- [3]Gérard Ben Arous, Reza Gheissari, Jiaoyang Huang, and Aikosh Jagannath, Local geometry of high-dimensional mixture models: Effective spectral theory and dynamical transitions, 2025, arXiv:2502.15655, version 3 (22 January 2026).arxiv.org/abs/2502.15655
- [4]Florent Benaych-Georges and Raj Rao Nadakuditi, The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices, Advances in Mathematics 227 (2011), no. 1, 494–521.
- [5]Zhou Fan, Renyuan Ma, Elliot Paquette, and Zhichao Wang, Anisotropic local law for non-separable sample covariance matrices, 2026, arXiv:2602.17960.arxiv.org/abs/2602.17960
- [6]Antti Knowles and Jun Yin, Anisotropic local laws for random matrices, Probability Theory and Related Fields 169 (2017), no. 1-2, 257–352, arXiv:1410.3516; statement numbers as in the arXiv version.arxiv.org/abs/1410.3516
- [7]Filip Kovačević, Yihan Zhang, and Marco Mondelli, Spectral estimators for multi-index models: Precise asymptotics and optimal weak recovery, Proceedings of Thirty Eighth Conference on Learning Theory, Proceedings of Machine Learning Research, vol. 291, PMLR, 2025, https://proceedings.mlr.press/v291/kovacevic25a.html, pp. 3354–3404.arxiv.org/abs/2502.01583
- [8]V. A. Marčenko and L. A. Pastur, Distribution of eigenvalues for some sets of random matrices, Mathematics of the USSR-Sbornik 1 (1967), no. 4, 457–483.DOI
- [9]Andrea Montanari and Basil Saeed, Variational formulas for the spectrum of block Wishart matrices, 2026, arXiv:2606.27774.arxiv.org/abs/2606.27774
- [10]Vinod Nair and Geoffrey E. Hinton, Rectified linear units improve restricted Boltzmann machines, Proceedings of the 27th International Conference on Machine Learning (Haifa, Israel), 2010.
- [11]Jack W. Silverstein and Sang-Il Choi, Analysis of the limiting spectral distribution of large dimensional random matrices, Journal of Multivariate Analysis 54 (1995), no. 2, 295–309.