betaFeedback

WorksComputer Science

Can Metadata Fix the Gauge? Calibration Turns Sparse Multi-Source Learning from Sparse PCA into Sparse Mean Recovery

Abstract

Source labels tell us where a sample came from, but not how that source oriented its coordinates. In a sparse Gaussian multi-source model with source-specific sign gauges, that distinction changes the learning problem. Without calibration, pooling across sources erases the mean signal, and the source-labelled experiment reduces to sparse PCA. With task-independent relative-gauge metadata, synchronization aligns sources and the same observations become a sparse mean problem. We formalize the value of metadata through AlignedMass, which quantifies how much first-order signal a metadata-induced alignment restores. Source labels create only limited AlignedMass, whereas valid calibration can create AlignedMass proportional to the full sample size. This yields a statistical separation in micro-source regimes and, under standard sparse-PCA hardness assumptions, a computational separation as well. Synthetic experiments check the predicted scaling; controlled real-feature and physical sensor studies examine sign cancellation, separately acquired calibration, and reuse across tasks.

Introduction

Multi-source learning is written as if every source uses the same coordinate convention. In practice this convention can fail. A sensor polarity may be reversed, a scanner pipeline may reflect a latent factor, or a client representation may flip the sign of a hidden component. Within each source the signal may be coherent, but across sources naive pooling can cancel it. This paper asks when metadata can fix such a source-specific gauge, and why this is different from merely knowing source labels.

A concrete example is a sensor whose mounting reverses the sign of a measured axis. Source identity tells us which recordings share a mounting, but not whether two mountings use the same or opposite axis convention. A separately recorded reference motion can provide that comparison before any downstream task is observed. In our physical sensor study (Section E.5), these same/opposite records are synchronized once and reused across four classification tasks. Here synchronization means estimating consistent source orientations up to one global sign; linearization means averaging after this alignment to restore a mean signal. We study this question in a stylized sparse Gaussian model. Each source observes the same unknown sparse direction, but through an unknown source-specific sign gauge. Source labels tell the learner which samples belong together, yet they do not reveal how one source is oriented relative to another. Calibration metadata is different: relative polarity logs, orientation edges, or noisy absolute orientation records can carry information about the source gauges. Such metadata can be synchronized into a common convention, using ideas related to synchronization over Z2\mathbb{Z}_2 and other groups [5, 17]. The main message is that these two kinds of side information induce different statistical experiments. With only source labels, the learner can aggregate within each source, but the unknown gauges remove first-order signal across sources. The resulting source-labelled experiment is exactly sparse PCA, plus residual Gaussian noise independent of the task direction. With valid relative-gauge metadata, synchronization selects a gauge section and restores first-order structure. Conditional on source amplitudes, the same downstream observations become a sparse mean problem. Thus labels identify groups, whereas calibration identifies orientation.

Source identifiers, group labels, and institution tags are useful for organizing data and modeling heterogeneity [3, 6, 13, 14, 16, 24], but they do not by themselves specify how source-specific coordinate conventions should be aligned. Calibration metadata plays a narrower role: it provides information about nuisance source conventions rather than the downstream task. In our model, this distinction changes the statistical experiment: the available signal changes from second-order to first-order, leading to different statistical and computational thresholds.

To quantify the value of metadata, we introduce aligned mass. Given metadata-induced sample weights qq and the hidden gauge vector gg, aligned mass is nn times their squared cosine similarity. In the constant-amplitude companion model, θ2Mq(g)\theta^2 M_q(g) is exactly the squared first-order signal strength. In the source-Gaussian model it still controls the nominal first-order signal, while an aligned-source-count condition controls concentration of random source amplitudes. This viewpoint makes the comparison sharp: source labels can create only limited aligned mass because they know which samples share a source but not which way that source points; valid relative-gauge metadata can create aligned mass proportional to the full sample size.

The resulting separation is most transparent in the micro-source regime, where each source is individually weak. Calibrated recovery succeeds at the sparse-mean scale, while source-labelled uncalibrated recovery is statistically insufficient at the corresponding sparse-PCA scale. Under standard sparse-PCA hardness assumptions [8, 11], there is also a conditional computational separation: calibrated recovery is polynomial-time after synchronization, whereas source-labelled recovery remains as hard as sparse PCA through the usual hard window.

A useful feature of calibration metadata is that it is task-independent. It fixes a source convention rather than revealing the downstream sparse direction. One relative-gauge graph can therefore be collected and synchronized once, then reused across many independent downstream tasks sharing the same source gauges. This differs from feature- or label-derived alignment, which may be useful but uses task-coupled information rather than external calibration. Our contributions are:

  1. We identify a sparse multi-source setting in which source-labelled uncalibrated data is statistically equivalent to sparse PCA, while calibrated data becomes sparse mean recovery.

  2. We introduce aligned mass as a quantitative measure of the first-order value of metadata-induced alignment, and show that source labels cannot reliably create extensive aligned mass whereas valid relative-gauge metadata can.

  3. We derive an unconditional statistical separation in micro-source regimes and a conditional computational separation under standard sparse-PCA hardness. Experiments check phase transitions, uninformative or mismatched metadata controls, calibration reuse, and controlled real-feature behavior.

Spectral synchronization, weighted averaging, and top-kk thresholding are standard components. Our contribution is the experiment-level converse without calibration, together with the quantitative link from noisy calibration to restored first-order signal, rather than a new synchronization algorithm.

Model, metadata, and alignment

A sparse multi-source model

Let Vd,k={v∈{0,±1/k}d:∥v∥0=k}\mathcal{V}_{d,k} = \{v \in\{0,\pm1/\sqrt{k}\}^{d} : \lVert v\rVert_0 = k\}, Ksp=klog⁡(ed/k)K_{\mathrm{sp}} = k\log(ed/k). There are LL sources, each contributing ss samples, so the total sample size is n=Lsn = Ls. The unknown sparse direction is V∈Vd,kV \in\mathcal{V}_{d,k}. Source ℓ\ell has an unknown sign gauge σℓ∼i.i.d.Unif⁡{±1}\sigma_\ell\overset{\mathrm{i.i.d.}}{\sim} \operatorname{Unif}\{\pm1\} and a nonnegative amplitude Rℓ∼i.i.d.∣N(0,1)∣R_\ell\overset{\mathrm{i.i.d.}}{\sim} |N(0,1)|, independent of σℓ\sigma_\ell. Writing aℓ=σℓRℓ∼N(0,1)a_\ell= \sigma_\ell R_\ell\sim N(0,1), the observation model is

Wℓr=θRℓσℓV+εℓr,εℓr∼N(0,Id),ℓ=1,…,L, r=1,…,s.W_{\ell r} = \theta R_\ell\sigma_\ell V + \varepsilon_{\ell r}, \qquad\varepsilon_{\ell r} \sim N(0,I_d), \qquad\ell= 1,\ldots,L,\ r = 1,\ldots,s.

Here θ>0\theta> 0 is the signal magnitude, ∥v∥0\|v\|_0 counts nonzero coordinates, and Ksp=klog⁡(ed/k)K_{\mathrm{sp}} = k\log(ed/k) is the sparse-complexity term, with ee Euler’s number and log⁡\log the natural logarithm. The feature noises are mutually independent and independent of VV, σ\sigma, RR. The Rademacher gauge prior means that each source sign is independently and uniformly sampled from {−1,+1}\{-1,+1\}. The downstream theoretical task is to recover VV up to a global sign, or its support supp⁡(V)={j:Vj≠0}\operatorname{supp}(V) = \{j : V_j \ne0\}. We write ≳\gtrsim, ≲\lesssim, and ≍\asymp for inequalities or equivalence up to universal positive constants. The same statistic arises from a simple two-view classification model; the derivation appears in Appendix § 8. The model separates two roles that are easy to conflate:

  • σℓ\sigma_\ell is a coordinate convention. This is the part calibration metadata may reveal.

  • RℓR_\ell is a source amplitude. This is nuisance randomness and is not something valid metadata should depend on.

Probabilities involving σ\sigma are with respect to this Rademacher prior. We also use the constant-amplitude companion Wℓr=θσℓV+εℓrW_{\ell r} = \theta\sigma_\ell V + \varepsilon_{\ell r}, where aligned mass is exact.

What counts as valid calibration metadata?

Source labels identify which samples belong to the same source. Calibration metadata instead provides information about source signs. A relative-polarity log or orientation edge is a noisy record of whether two sources use the same or opposite sign convention; a noisy absolute orientation record compares one source with a known reference.

Let EE contain only exogenous source-side structure, such as source identities, source sizes, calibration-graph topology, and a pre-specified calibration design. In the theoretical model, EE is fixed or sampled independently of σ\sigma, RR, VV and task randomness; it cannot encode downstream labels or features. Let DtaskD_{\mathrm{task}} collect downstream observations, their feature noise, and any downstream labels. In the two-view example, these labels are YℓrY_{\ell r}; the main recovery model requires no additional labels.

Definition 1 (Valid calibration metadata). A metadata channel M\mathcal{M} is valid calibration metadata if

P(M∣V,R,Dtask,σ,E)=P(M∣σ,E).\mathbb{P}(\mathcal{M}\mid V,R,D_{\mathrm{task}},\sigma,E)=\mathbb{P}(\mathcal{M}\mid\sigma,E).

Thus, conditional on source signs and the exogenous source design, calibration is independent of the downstream direction, amplitudes, observations, labels, and feature noise.

Validity concerns task independence, not accuracy. Random metadata can be valid but uninformative; missing or independently corrupted calibration records can also remain valid. Conversely, feature- or label-derived alignment can be useful while violating task independence.

With externally collected valid calibration, weights are estimated without using any downstream observations, so scarce micro-source samples need not be split to estimate alignment. Sample splitting is relevant to task-derived comparison procedures: it can separate weight estimation from the noise in a final statistic, but does not make that alignment task-independent or reusable calibration.

aligned mass: how much first-order signal metadata restores

Let q∈Rnq \in\mathbb{R}^n be nonzero sample weights constructed from calibration metadata and source structure. For example, estimated source signs σ^ℓ\widehat{\sigma}_\ell give qℓr=σ^ℓq_{\ell r}=\widehat{\sigma}_\ell; spectral synchronization can instead supply real-valued source weights. We also use such metadata-derived weights a section. At the sample level, the hidden sign vector is gℓr=σℓg_{\ell r}=\sigma_\ell.

The constant-amplitude companion model makes the role of these weights explicit:

Xq=∑ℓ,rqℓrWℓr∥q∥2=θqTg∥q∥2V+Z,Z∼N(0,Id).X_q=\frac{\sum_{\ell,r}q_{\ell r}W_{\ell r}}{\|q\|_2}=\theta\frac{q^\mathsf{T}g}{\|q\|_2}V+Z,\qquad Z\sim N(0,I_d).

This motivates

Mq(g)=(qTg)2∥q∥22,M_q(g)=\frac{(q^\mathsf{T}g)^2}{\|q\|_2^2},

because θ2Mq(g)\theta^2M_q(g) is exactly the squared restored mean-shift strength. Since ∥g∥22=n\|g\|_2^2=n,

0≤Mq(g)≤n,Mq(g)n=(qTg)2∥q∥22∥g∥22=cos⁡2(q,g).0\le M_q(g)\le n,\qquad\frac{M_q(g)}{n}=\frac{(q^\mathsf{T}g)^2}{\|q\|_2^2\|g\|_2^2}=\cos^2(q,g).

Thus aligned mass is an effective first-order sample size, and Mq(g)/nM_q(g)/n is normalized alignment. No assumption ∥q∥2=∥g∥2\lVert q\rVert_2 = \lVert g\rVert_2 is needed. Exact aligned mass is a theoretical and evaluation quantity, not an algorithm input: the algorithm uses qq, not the unknown gg. Planted signs are available for evaluation in simulations; recorded mounting signs in the physical study are used only for alignment evaluation and the oracle control.

For equal source size ss, we can also quantify the metadata channel’s posterior expected alignment. Define the posterior source-sign correlation matrix

Γ(M,E)=E[σσT∣M,E]∈RL×L,\Gamma(\mathcal{M}, E) = \mathbb{E}[\boldsymbol{\sigma}\boldsymbol{\sigma}^{\mathsf{T}} \mid\mathcal{M}, E] \in\mathbb{R}^{L \times L},

and its alignment capacity

C(M,E)=sup⁡q=q(M,E)q≠0E[Mq(g)∣M,E].\mathcal{C}(\mathcal{M}, E) = \sup_{\substack{q=q(\mathcal{M},E)\\q\ne0}} \mathbb{E}[M_q(g) \mid\mathcal{M}, E].

The supremum is over deterministic metadata-measurable weights; independent algorithmic randomness may be conditioned on.

Proposition 2 (Posterior alignment capacity). For equal source size ss,

C(M,E)=s λmax⁡(Γ(M,E)),\mathcal{C}(\mathcal{M}, E) = s\,\lambda_{\max}(\Gamma(\mathcal{M}, E)),

where λmax⁡\lambda_{\max} denotes the largest eigenvalue. An optimal source-level weighting is a top eigenvector of Γ(M,E)\Gamma(\mathcal{M}, E).

Under the independent Rademacher prior, source labels give Γ=IL\Gamma= I_L, whereas perfect relative calibration determines σσT\boldsymbol{\sigma}\boldsymbol{\sigma}^{\mathsf{T}}, even though the global sign remains unknown. Consequently,

LABELS:C=sCALIBRATION:C=n.\mathsf{LABELS}: \mathcal{C}=s \qquad\mathsf{CALIBRATION}: \mathcal{C}=n.

This is a posterior expectation statement; the synchronization result below controls realized alignment with high probability.

In the source-Gaussian model, amplitudes vary across sources. Write

Qℓ(q)=∑r=1sqℓr,Q_\ell(q) = \sum_{r=1}^{s} q_{\ell r},

and define the aligned source count

Nsrc(q,g)=(∑ℓ=1LQℓ(q)σℓ)2∑ℓ=1LQℓ(q)2,N_{\mathrm{src}}(q,g) = \frac{\left(\sum_{\ell=1}^{L} Q_\ell(q)\sigma_\ell\right)^2} {\sum_{\ell=1}^{L} Q_\ell(q)^2},

with value zero when the denominator vanishes. Aligned mass controls the nominal first-order signal, while NsrcN_{\mathrm{src}} controls concentration of the weighted random amplitudes around their mean. High aligned mass concentrated on too few sources need not produce a stable realized signal.

Main results

We first show that source labels alone expose only second-order information. We then show how calibration metadata restores first-order structure.

Source labels alone reduce to sparse PCA

If source labels are available but gauges are not aligned, the sufficient source statistic is Xℓ=s−1/2∑rWℓr=θs aℓV+ZℓX_\ell= s^{-1/2}\sum_r W_{\ell r} = \theta\sqrt{s}\,a_\ell V + Z_\ell, with Zℓ∼N(0,Id)Z_\ell\sim N(0,I_d). Since aℓ∼N(0,1)a_\ell\sim N(0,1), Xℓ∼N(0,Id+λVVT)X_\ell\sim N(0,I_d+\lambda VV^{\mathsf{T}}), where λ=θ2s\lambda=\theta^2s.

Proposition 3 (Source-labelled data is sparse PCA). In the source-Gaussian model with equal source size ss under the Rademacher gauge prior, the source-labelled uncalibrated observations are statistically equivalent to

Xℓ∼N(0,Id+λVVT),ℓ=1,…,L,λ=θ2s,X_\ell\sim N(0,I_d+\lambda VV^{\mathsf{T}}), \qquad\ell=1,\ldots,L,\qquad\lambda=\theta^2s,

together with residual Gaussian noise independent of VV. Hence any source-labelled uncalibrated estimator can be simulated from LL sparse PCA samples plus VV-independent randomness.

Under the Rademacher-gauge experiment, this is an equivalence of statistical experiments, not a statement about a particular algorithm or a particular linear statistic. Consequently it already covers arbitrary feature-dependent procedures applied to the source-labelled uncalibrated samples, including attempts to infer source signs or align sources from the downstream features themselves. Such procedures can only use the sparse-PCA information in the source aggregates XℓX_{\ell} together with VV-independent residual noise. In this sense, feature-derived self-alignment from the uncalibrated task data is not external calibration metadata; if it succeeds in the corresponding hard regime, it is exploiting or solving the sparse-PCA instance exposed by Proposition 3.

Thus marginalizing the gauges removes first-order signal and leaves only the second-order spike.

Calibration turns the same observations into a sparse mean problem

Now suppose metadata supplies a section qq. Form the weighted linear statistic

Xq=∑ℓ,rqℓrWℓr∥q∥2.X_q = \frac{\sum_{\ell,r} q_{\ell r}W_{\ell r}}{\lVert q\rVert_2}.

Conditional on qq, σ\sigma, RR, VV, we obtain

Xq=θAqV+Z,Z∼N(0,Id),Aq=∑ℓ,rqℓrRℓσℓ∥q∥2.X_q = \theta A_q V + Z,\qquad Z \sim N(0,I_d),\qquad A_q = \frac{\sum_{\ell,r}q_{\ell r}R_{\ell}\sigma_{\ell}}{\lVert q\rVert_2}.

So after gauge selection, the problem is sparse mean recovery.

In the constant-amplitude model, Aq2=Mq(g)A_q^2 = M_q(g) exactly. In the source-Gaussian model, aligned mass still governs the nominal signal and aligned source count guarantees concentration.

Theorem 4 (Calibration linearizes sparse recovery). Let q≠0q \ne0 be constructed exclusively from valid calibration metadata and EE, with independent algorithmic randomness, and let 0<εrec,δ<1/20 < \varepsilon_{\mathrm{rec}},\delta< 1/2. Conditional on qq, σ\sigma, EE, VV, suppose

θ2κ12Mq(g)≳εrec−2(Ksp+log⁡(2/δ)),κ1=2π,\theta^2\kappa_1^2 M_q(g) \gtrsim\varepsilon_{\mathrm{rec}}^{-2}\left(K_{\mathrm{sp}}+\log(2/\delta)\right),\qquad\kappa_1=\sqrt{\frac{2}{\pi}},

and

Nsrc(q,g)≳log⁡(2/δ).N_{\mathrm{src}}(q,g) \gtrsim\log(2/\delta).

Then top-kk thresholding of XqX_q, with observed coordinate signs, returns V^\widehat{V} satisfying

∣⟨V^,V⟩∣≥1−εrec\left|\langle\widehat{V},V\rangle\right| \ge1-\varepsilon_{\mathrm{rec}}

with probability at least 1−δ1-\delta over the source amplitudes and downstream feature noise, up to universal constants.

The proof combines a standard sparse-mean thresholding bound with a weighted half-normal concentration lemma; both are given in Appendix § 9.

Why source labels cannot create enough aligned mass

The next lemma formalizes a simple but decisive limitation: if a weighting rule only knows source labels and not source gauges, then its correlation with the hidden gauge vector behaves like a Rademacher sum.

Lemma 5 (Source labels do not choose a gauge). Assume equal source size ss and the Rademacher gauge prior. Let qq be any possibly randomized section measurable with respect to source labels and independent randomness, but independent of the unknown gauges σ\sigma. Then, for every 0<δ<10 < \delta< 1,

Pσ(Mq(g)≤2slog⁡2δ | q)≥1−δ.\mathbb{P}_{\sigma}\left(M_q(g)\le2s\log\frac{2}{\delta}\,\middle|\,q\right)\ge1-\delta.

Thus source labels alone cannot reliably create aligned mass Mq(g)=Θ(n)M_q(g)=\Theta(n) unless s=Θ(n)s=\Theta(n).

Thus labels group samples but do not choose their orientation.

Relative-gauge metadata creates extensive aligned mass

A natural calibration channel is a noisy relative-sign graph over sources. For each 1≤ℓ<m≤L1 \le\ell< m \le L, let Aℓm∼Ber⁡(p)A_{\ell m} \sim\operatorname{Ber}(p) independently, where Ber⁡(p)\operatorname{Ber}(p) is the Bernoulli distribution with success probability pp, with Amℓ=AℓmA_{m\ell}=A_{\ell m} and Aℓℓ=0A_{\ell\ell}=0. On observed edges Aℓm=1A_{\ell m}=1, we observe

Cℓm=σℓσmνℓm,P(νℓm=1)=1−ϵ,α=1−2ϵ>0.C_{\ell m} = \sigma_\ell\sigma_m \nu_{\ell m}, \qquad\mathbb{P}(\nu_{\ell m}=1)=1-\epsilon, \qquad\alpha=1-2\epsilon>0.

Set Cmℓ=CℓmC_{m\ell}=C_{\ell m}, and define the signed adjacency matrix

Bℓm=AℓmCℓm(ℓ≠m),Bℓℓ=0.B_{\ell m}=A_{\ell m}C_{\ell m}\quad(\ell\ne m), \qquad B_{\ell\ell}=0.

Then

E[B∣σ]=pα(σσT−I).\mathbb{E}[B\mid\sigma]=p\alpha(\sigma\sigma^\mathsf{T}-I).

Let u^\hat{u} be a unit-norm top eigenvector of BB. The edge noises νℓm\nu_{\ell m} are independent across unordered source pairs and independent of the graph indicators, source variables, and downstream feature noise.

Lemma 6 (Spectral synchronization). There exist universal constants C,c>0C,c>0 such that if

pL≥Clog⁡(L/δ),α2pL≥C,pL\ge C\log(L/\delta), \qquad\alpha^2pL\ge C,

then with probability at least 1−δ1-\delta,

β2:=∣⟨u^,σL⟩∣2≥1−Cα2pL.\beta^2:=\left|\left\langle\hat{u},\frac{\sigma}{\sqrt{L}}\right\rangle\right|^2\ge1-\frac{C}{\alpha^2pL}.

Assign each sample in source ℓ\ell the weight qℓ=u^ℓq_\ell=\hat{u}_\ell. Then

Mq(g)=β2n,Nsrc(q,g)=β2L.M_q(g)=\beta^2n, \qquad N_{\mathrm{src}}(q,g)=\beta^2L.

So synchronization lifts the effective sample size from O(s)O(s), which is the best source-label sections can do, to Θ(n)\Theta(n) whenever the synchronized section has constant overlap with the hidden gauges.

Corollary 7 (Calibrated recovery scale). For fixed recovery tolerance 0<ϵrec<1/20<\epsilon_{\mathrm{rec}}<1/2 and failure probability 0<δ<1/20<\delta<1/2, if synchronization yields β2=Ω(1)\beta^2=\Omega(1) and β2L≳log⁡(1/δ)\beta^2L\gtrsim\log(1/\delta), then calibrated top-kk recovery succeeds once

θ2β2n≳klog⁡(ed/k).\theta^2\beta^2n\gtrsim k\log(ed/k).

up to universal constants. Under the two-view parametrization θ2=2μ2\theta^2=2\mu^2, this is equivalently

μ2β2n≳klog⁡(ed/k).\mu^2\beta^2n\gtrsim k\log(ed/k).

Metadata budget. The calibrated and uncalibrated experiments use the same downstream feature sample size n=Lsn=Ls, but calibration also uses a task-independent metadata resource. In the relative-sign graph model the expected number of calibration edges is

m=E∣{(ℓ,m):Aℓm=1, ℓ<m}∣=p(L2).m=\mathbb{E}\left|\{(\ell,m):A_{\ell m}=1,\ \ell<m\}\right|=p\binom{L}{2}.

Thus the elementary spectral guarantee above uses

m=O(Lmax⁡{log⁡(L/δ),α−2})m=O\left(L\max\{\log(L/\delta),\alpha^{-2}\}\right)

calibration edges to reach the overlap level certified by Lemma 6, up to universal constants, when pp is chosen at the stated threshold. To certify β2≥1−ϵ\beta^2\ge1-\epsilon, the same lemma gives the sufficient additional degree condition

α2pL≳ϵ−1,\alpha^2pL\gtrsim\epsilon^{-1},

together with the stated logarithmic degree requirement. This cost is separate from downstream feature samples and can be paid once and reused across tasks sharing the same source gauges. The separation therefore compares two metadata channels: source labels have alignment capacity ss, whereas relative-gauge metadata, at the stated calibration-edge budget, can produce realized aligned mass Θ(n)\Theta(n).

A separation in the micro-source regime

The calibration result above follows the sparse-mean scale. By contrast, Proposition 3 shows that source labels alone give a sparse PCA problem with

N=L=ns,λ=θ2s.N = L = \frac{n}{s}, \qquad\lambda= \theta^2s.

Its information scale is therefore

Nλ2=θ4ns≲μ4ns.N\lambda^2 = \theta^4ns \lesssim\mu^4ns.

The following lower bound turns that heuristic into a support-recovery statement.

Theorem 8 (High-F1 lower bound for source labels). Assume 1≤k≤d/21 \le k \le d/2 and Ksp≥C0K_{\mathrm{sp}} \ge C_0 for a sufficiently large universal constant C0C_0. In the source-Gaussian model, let VV be uniform on the sparse support packing constructed in Appendix § 9. Define

Isrc=L2[λ−log⁡(1+λ)],λ=θ2s.I_{\mathrm{src}} = \frac{L}{2}\left[\lambda-\log(1+\lambda)\right], \qquad\lambda= \theta^2s.

There exist universal constants c,τ>0c,\tau>0 such that if Isrc≤cKspI_{\mathrm{src}} \le cK_{\mathrm{sp}}, then every estimator using source labels alone satisfies

P(SupportF1⁡(V^,V)≥1−τ)≤12.\mathbb{P}\left(\operatorname{SupportF1}(\widehat{V},V) \ge1-\tau\right) \le\frac{1}{2}.

In the weak-source regime λ≤1\lambda\le1, this condition becomes μ4ns≲Ksp\mu^4ns \lesssim K_{\mathrm{sp}}.

Remark 9 (Bayes prior and fixed gauges). The Rademacher prior on σ\sigma is used to obtain a hard average-case gauge ensemble. The lower bound also implies a worst-case fixed-gauge statement: if the success probability averaged over σ\sigma is at most $1/2,∗thenthereexistsafixedgaugevector∗, *then there exists a fixed gauge vector*\sigma^* for which the same estimator has success probability at most 1/21/2. Equivalently, the Bayes lower bound under the Rademacher gauge prior implies a minimax lower bound over fixed unknown gauges. We do not claim that every fixed gauge is equally hard; for example, if all sources happen to be already aligned, first-order pooling is possible. The lower bound concerns the absence of calibration in the random-gauge average-case and fixed-gauge minimax senses.

Together, Theorem 4 and Theorem 8 give an unconditional statistical separation. In the micro-source regime sμ2≪1s\mu^2 \ll1, there is a window where

μ2n≳Kspbutμ4ns≲Ksp.\mu^2n \gtrsim K_{\mathrm{sp}} \qquad\text{but}\qquad\mu^4ns \lesssim K_{\mathrm{sp}}.

In this window, calibrated recovery succeeds while source-labelled recovery is statistically insufficient for high-F1 support recovery.

A conditional computational separation

The statistical gap above does not rely on computational assumptions. A computational separation appears when the source-labelled experiment is interpreted through standard sparse-PCA hardness.

Conjecture 10 (Sparse PCA constant-advantage hardness, informal). Consider the testing problem

N(0,Id)⊗NversusN(0,Id+λVV⊤)⊗N,V∈Vd,k.\mathcal{N}(0,I_d)^{\otimes N} \qquad\text{versus}\qquad\mathcal{N}(0,I_d+\lambda VV^\top)^{\otimes N}, \qquad V\in\mathcal{V}_{d,k}.

In the standard sparse-PCA hard regime, no polynomial-time algorithm distinguishes the two hypotheses with constant advantage throughout Ksp≪Nλ2≪k2K_{\mathrm{sp}}\ll N\lambda^2\ll k^2, up to logarithmic and standard polynomial-scaling qualifications. High-overlap recovery implies detection by sample splitting and a held-out variance test.

Corollary 11 (Metadata–computation separation). Assume Conjecture 10. If sμ2≪1s\mu^2\ll1, Ksp≪k2K_{\mathrm{sp}}\ll k^2, and polynomial-time synchronization of valid calibration metadata yields β2=Ω(1)\beta^2=\Omega(1), then at

ncal≲Kspμ2n_{\mathrm{cal}}\lesssim\frac{K_{\mathrm{sp}}}{\mu^2}

calibration enables polynomial-time high-overlap recovery, while source labels alone are statistically insufficient. Moreover, for larger uncalibrated sample size nmixn_{\mathrm{mix}}, source-labelled recovery remains computationally hard throughout the sparse-PCA window

Kspsμ4≪nmix≪k2sμ4,\frac{K_{\mathrm{sp}}}{s\mu^4}\ll n_{\mathrm{mix}}\ll\frac{k^2}{s\mu^4},

up to constants and logarithmic factors.

Thus the calibrated and uncalibrated observations fall into different algorithmic regimes: sparse mean recovery after synchronization versus sparse PCA without calibration.

One calibration, many downstream tasks

Calibration metadata concerns source gauges, not the downstream sparse direction. It can therefore be reused across tasks sharing the same source gauges and calibration graph.

Remark 12 (Task-independence and reuse). Let V1,…,VTV_1,\ldots,V_T be independent sparse directions, sampled independently of (σ,E)(\sigma,E). If the metadata satisfies

P(M∣V1,…,VT,σ,E)=P(M∣σ,E),P(\mathcal{M}\mid V_1,\ldots,V_T,\sigma,E)=P(\mathcal{M}\mid\sigma,E),

then I(M;V1,…,VT∣σ,E)=0I(\mathcal{M};V_1,\ldots,V_T\mid\sigma,E)=0. Nevertheless, if the metadata induces a section qq with Mq(g)=Ω(n)M_q(g)=\Omega(n) and sufficient aligned source count, then the same qq linearizes every task:

Xq(t)=θAq(t)Vt+Z(t).X_q^{(t)}=\theta A_q^{(t)}V_t+Z^{(t)}.

Thus calibration is reusable: it fixes a shared source convention without revealing any task-specific support.

The resulting algorithm, GSL (Gauge–Synchronize–Linearize), is immediate from the theory. It synchronizes the valid relative-gauge graph to obtain source weights u^ℓ\widehat{u}_{\ell}. For sample ii, let ei∈{1,…,L}e_i\in\{1,\ldots,L\} denote its source label, and set qi=u^eiq_i=\widehat{u}_{e_i} on the downstream estimation samples. No downstream samples are used to estimate these calibration weights. The algorithm then forms

Xq=∑iqiWi∥q∥2,X_q=\frac{\sum_i q_iW_i}{\lVert q\rVert_2},

and returns the top-kk coordinates of ∣Xq∣\lvert X_q\rvert. In the two-view model, where Ai,BiA_i,B_i are the two views and Yi∈{±1}Y_i\in\{\pm1\} is the label, Wi=(Ai+YiBi)/2W_i=(A_i+Y_iB_i)/\sqrt{2}, and the recovered direction is used in the bilinear classifier h(A,B)=sign⁡((V^TA)(V^TB))h(A,B)=\operatorname{sign}((\widehat{V}^{\mathsf T}A)(\widehat{V}^{\mathsf T}B)). Pseudocode is deferred to Appendix § 7.

The exact separation assumes sign gauges, equal-magnitude sparse signals, Gaussian source amplitudes, isotropic Gaussian noise, and task-independent calibration. Appendix § 10 gives a non-isotropic Gaussian extension and clarifies the limits of the sparse-PCA hardness comparison.

Experiments

Experiments check finite-sample predictions rather than prove the experiment equivalence. Synthetic studies test the predicted recovery scales; controlled real-feature studies test sign cancellation and restoration. Separately acquired physical calibration and reuse are reported in Appendix § 11.§ 5. Unless stated otherwise, θ2=2μ2\theta^2=2\mu^2. Synthetic recovery is measured by Corr⁡=∣⟨V^,V⟩∣\operatorname{Corr}=\lvert\langle\widehat{V},V\rangle\rvert and support F1, with corresponding success threshold 0.9. The plotted strength is ρcal=θ2n/Ksp\rho_{\mathrm{cal}}=\theta^2n/K_{\mathrm{sp}}. Pseudocode is in Appendix § 7; experimental records and standard-error units are in Appendix § 11.

Synthetic mechanism tests

Figure 1(a) tests the sparse-mean scaling predicted by Theorem 4: oracle calibration and spectral synchronization nearly coincide across problem sizes. Figure 1(b) compares calibration quality: informative records shift the transition, random and shuffled controls remain near chance, and partial calibration gives an intermediate transition. These controls need not violate task independence. Figure 1(c) fixes the calibrated first-order scale and varies sμ2s\mu^2. Calibrated recovery remains strong for weak sources, whereas source-label PCA improves as the second-order quantity μ4ns/Ksp\mu^4n s/K_{\mathrm{sp}} increases.

Additional mechanism checks

Aligned source count. Theory separates signal from amplitude concentration. We fix Mq(g)=4096M_q(g)=4096 and vary NsrcN_{\mathrm{src}}. Table 1 confirms that aligned mass alone is not enough when concentrated on too few sources: as NsrcN_{\mathrm{src}} grows, Aq/MqA_q/\sqrt{M_q} stabilizes near the half-normal scale and recovery improves. This matches the role of NsrcN_{\mathrm{src}} in Theorem 4: it controls whether source amplitudes concentrate around their mean rather than being dominated by a few lucky or unlucky sources.

NsrcN_{\mathrm{src}}Aq/MqA_q/\sqrt{M_q}CorrSucc90
10.825±0.0680.825 \pm0.0680.632±0.0380.632 \pm0.0380.460±0.0500.460 \pm0.050
40.828±0.0310.828 \pm0.0310.832±0.0240.832 \pm0.0240.630±0.0490.630 \pm0.049
160.783±0.0150.783 \pm0.0150.880±0.0110.880 \pm0.0110.600±0.0490.600 \pm0.049
320.800±0.0110.800 \pm0.0110.917±0.0080.917 \pm0.0080.800±0.0400.800 \pm0.040
1280.790±0.0050.790 \pm0.0050.916±0.0060.916 \pm0.0060.770±0.0420.770 \pm0.042

Table 1. At fixed aligned mass, increasing aligned source count stabilizes amplitude and improves recovery. Entries are mean ±\pm SE over 100 seeds.

Reusable calibration. We test Remark 12 by synchronizing one relative-gauge graph before sampling any downstream task and reusing it for T=32T = 32 independent sparse tasks. Table 2 shows that the reused section matches oracle calibration; the downstream directions are distinct, so the gain is not due to task similarity. This illustrates the main distinction between calibration and supervision: the graph fixes a shared source convention once, and that convention can then be applied to new tasks without revealing their supports.

MethodCorrF1
Oracle calibration0.928±0.0020.928 \pm0.0020.928±0.0020.928 \pm0.002
Same sync reused0.928±0.0020.928 \pm0.0020.928±0.0020.928 \pm0.002
No calibration0.018±0.0010.018 \pm0.0010.019±0.0010.019 \pm0.001
Random metadata reused0.020±0.0020.020 \pm0.0020.021±0.0020.021 \pm0.002
Shuffled metadata reused0.022±0.0010.022 \pm0.0010.023±0.0010.023 \pm0.001
Source diagonal PCA0.040±0.0020.040 \pm0.0020.049±0.0020.049 \pm0.002

Table 2. One calibration graph reused across 32 independent sparse tasks. Entries are mean ±\pm task-level SE conditional on the graph.

Controlled real features. Finally, we use real features and labels while synthetically imposing source gauges and calibration logs. Table 3 shows valid calibration restores clean no-gauge performance on digits and breast-cancer data, while random or shuffled metadata does not. These are controlled mechanism checks: gauges and calibration logs are imposed synthetically, while the feature distributions and labels are real. Source-label task alignment uses downstream labels and is therefore a supervised control.

MethodDigits Acc.Digits Mq/nM_q/nBreast Acc.Breast Mq/nM_q/n
Clean no gauge0.968±0.0010.968 \pm0.0011.000±0.0001.000 \pm0.0000.933±0.0030.933 \pm0.0031.000±0.0001.000 \pm0.000
Oracle calibration0.968±0.0010.968 \pm0.0011.000±0.0001.000 \pm0.0000.933±0.0030.933 \pm0.0031.000±0.0001.000 \pm0.000
Relative sync calibration0.968±0.0010.968 \pm0.0011.000±0.0001.000 \pm0.0000.932±0.0030.932 \pm0.0030.997±0.0030.997 \pm0.003
Noisy abs. calibration, 10% flips0.874±0.0060.874 \pm0.0060.650±0.0180.650 \pm0.0180.851±0.0110.851 \pm0.0110.667±0.0310.667 \pm0.031
No calibration0.500±0.0040.500 \pm0.0040.003±0.0000.003 \pm0.0000.501±0.0050.501 \pm0.005(1.180±0.234)×10−5(1.180 \pm0.234) \times10^{-5}
Random metadata0.525±0.0050.525 \pm0.0050.011±0.0020.011 \pm0.0020.556±0.0080.556 \pm0.0080.022±0.0050.022 \pm0.005
Shuffled metadata0.533±0.0070.533 \pm0.0070.016±0.0030.016 \pm0.0030.557±0.0070.557 \pm0.0070.024±0.0050.024 \pm0.005
Source-label task alignment0.818±0.0080.818 \pm0.0080.704±0.0180.704 \pm0.0180.933±0.0030.933 \pm0.0031.000±0.0001.000 \pm0.000

Table 3. Controlled real-feature experiments. Features and labels are real; source gauges and calibration metadata are imposed. Entries are mean ±\pm SE over 30 seeds.

Related work

Our work connects sparse PCA, synchronization, orbit recovery, and metadata-aware robust learning. Sparse PCA is a standard model for high-dimensional statistical and computational gaps [2, 8, 11, 12]; here it arises instead as the exact experiment induced by source-labelled uncalibrated observations. Our calibration step is closely related to synchronization over groups such as Z2\mathbb{Z}_2, SO(d)SO(d), and U(1)U(1) [5, 17], with additional connections to community-detection and graph-based recovery phenomena [1]; here synchronization is used to turn task-independent gauge metadata into restored first-order signal. The model is also related to orbit recovery and multi-reference alignment, where latent group actions suppress first-order information [7], and to related high-order-moment phenomena in spiked models [15]. More broadly, the paper complements work on invariant learning, robustness under group or domain shift, and metadata-aware pipelines [3, 14, 16, 19, 20, 25]. Related phenomena arise in attribute recognition, where label co-occurrence bias and heterogeneous feature distributions induce spurious correlations that downstream methods must remove or balance [21, 22, 23, 24]. Our focus is narrower: metadata that identifies nuisance coordinate conventions rather than providing downstream supervision.

Conclusion

This paper isolates a basic distinction in multi-source learning. Source labels tell us which samples belong together, but they do not tell us how different sources orient the same latent signal. In our stylized sparse model, that distinction changes the problem completely. Without calibration, the experiment is sparse PCA; with valid relative-gauge metadata, synchronization restores a first-order sparse mean signal. The aligned-mass viewpoint makes this change measurable and leads to clear statistical and computational separations in weak-source regimes. More broadly, the paper highlights a simple principle: metadata can matter not because it labels the task, but because it fixes the coordinate system in which the task becomes visible. This suggests treating calibration as a distinct resource from supervision or grouping information. Extending this viewpoint to richer transformation groups and approximately valid metadata is a natural next step.

Acknowledgments and Disclosure of Funding

We declare that this work received no external funding and that the authors have no competing interests to disclose.

Deferred algorithm

GSL pseudocode

Algorithm 1 GSL: Gauge–Synchronize–Linearize

Require: Linearization samples (Wi,ei)(W_i,e_i), where ei∈{1,…,L}e_i \in\{1,\ldots,L\} is the source label of sample ii; valid calibration metadata M\mathcal{M}; sparsity kk

  1. Construct a signed source calibration graph from M\mathcal{M}

  2. Run Z2\mathbb{Z}_2 synchronization to obtain source weights u^ℓ\widehat{u}_{\ell}

  3. Set qi=u^eiq_i=\widehat{u}_{e_i} for each downstream sample ii

  4. Compute Xq=∑iqiWi/∥q∥2X_q=\sum_i q_iW_i/\lVert q\rVert_2

  5. Top-kk threshold ∣Xq∣\lvert X_q\rvert to obtain V^\widehat{V}

  6. return V^\widehat{V}

In the two-view setting of Appendix § 8, the linearization samples are obtained from Wi=(Ai+YiBi)/2W_i=(A_i+Y_iB_i)/\sqrt{2}, where AiA_i and BiB_i are the two views and Yi∈{±1}Y_i\in\{\pm1\} is the label. After recovering V^\widehat{V}, prediction uses h(A,B)=sign⁡((V^TA)(V^TB))h(A,B)=\operatorname{sign}((\widehat{V}^{\mathsf{T}}A)(\widehat{V}^{\mathsf{T}}B)).

Two-view origin

The source-Gaussian statistic arises from a simple two-view classification model. Let aℓ∼i.i.d.N(0,1)a_{\ell}\mathrel{\overset{\mathrm{i.i.d.}}{\sim}}\mathcal{N}(0,1), and let

ξℓr,ζℓr∼i.i.d.N(0,Id),Yℓr∼Unif⁡{±1},\xi_{\ell r},\zeta_{\ell r}\mathrel{\overset{\mathrm{i.i.d.}}{\sim}}\mathcal{N}(0,I_d),\qquad Y_{\ell r}\sim\operatorname{Unif}\{\pm1\},

with all variables independent of aℓa_{\ell} and VV. Define

Aℓr=μaℓV+ξℓr,Bℓr=μYℓraℓV+ζℓr.A_{\ell r}=\mu a_{\ell}V+\xi_{\ell r},\qquad B_{\ell r}=\mu Y_{\ell r}a_{\ell}V+\zeta_{\ell r}.

Then

Wℓr=Aℓr+YℓrBℓr2=θaℓV+εℓr,θ=2 μ,W_{\ell r}=\frac{A_{\ell r}+Y_{\ell r}B_{\ell r}}{\sqrt{2}}=\theta a_{\ell}V+\varepsilon_{\ell r},\qquad\theta=\sqrt{2}\,\mu,

where

εℓr=ξℓr+Yℓrζℓr2∼N(0,Id).\varepsilon_{\ell r}=\frac{\xi_{\ell r}+Y_{\ell r}\zeta_{\ell r}}{\sqrt{2}}\sim\mathcal{N}(0,I_d).

The orthogonal combination

Aℓr−YℓrBℓr2\frac{A_{\ell r}-Y_{\ell r}B_{\ell r}}{\sqrt{2}}

is pure Gaussian noise. Recovering VV in the main text therefore corresponds to recovering the discriminative sparse direction used by the bilinear classifier.

Proofs

Proof of Proposition 2

Fix (M,E)(\mathcal{M},E). For a sample-level weighting qq, let

Qℓ=∑r=1sqℓr.Q_{\ell}=\sum_{r=1}^{s}q_{\ell r}.

Then

qTg=∑ℓ=1LQℓσℓ=QTσ.q^{\mathsf{T}}g=\sum_{\ell=1}^{L}Q_{\ell}\sigma_{\ell}=Q^{\mathsf{T}}\sigma.

By Cauchy’s inequality within each source,

Qℓ2≤s∑r=1sqℓr2.Q_{\ell}^{2}\le s\sum_{r=1}^{s}q_{\ell r}^{2}.

and hence

∥q∥22≥1s∥Q∥22.\lVert q\rVert_{2}^{2} \ge\frac{1}{s}\lVert Q\rVert_{2}^{2}.

If Q=0Q=0, the expected aligned mass is zero, so the desired upper bound is immediate. We therefore assume Q≠0Q\ne0 in the following Rayleigh quotient calculation. Therefore, conditional on (M,E)(\mathcal{M},E),

E[Mq(g)∣M,E]=QTΓ(M,E)Q∥q∥22≤sQTΓ(M,E)Q∥Q∥22≤sλmax⁡(Γ(M,E)).\mathbb{E}[M_q(g)\mid\mathcal{M},E]=\frac{Q^{\mathsf{T}}\Gamma(\mathcal{M},E)Q}{\lVert q\rVert_{2}^{2}}\le s\frac{Q^{\mathsf{T}}\Gamma(\mathcal{M},E)Q}{\lVert Q\rVert_{2}^{2}}\le s\lambda_{\max}(\Gamma(\mathcal{M},E)).

The upper bound is attained by taking QQ to be a top eigenvector of Γ(M,E)\Gamma(\mathcal{M},E) and spreading the weight uniformly within each source, for instance qℓr=Qℓ/sq_{\ell r}=Q_\ell/s. This gives

E[Mq(g)∣M,E]=sλmax⁡(Γ(M,E)).\mathbb{E}[M_q(g)\mid\mathcal{M},E]=s\lambda_{\max}(\Gamma(\mathcal{M},E)).

This proves the claim.

Bayes-to-minimax consequence for fixed gauges

Let S(V^,V)S(\widehat{V},V) denote the success event, for example

S(V^,V)={SupportF1⁡(V^,V)≥1−τ}.S(\widehat{V},V)=\{\operatorname{SupportF1}(\widehat{V},V)\ge1-\tau\}.

Suppose that for an arbitrary estimator V^\widehat{V},

EσPr⁡(S(V^,V)∣σ)≤12,\mathbb{E}_{\sigma}\Pr\left(S(\widehat{V},V)\mid\sigma\right)\le\frac{1}{2},

where the outer expectation is over the uniform Rademacher prior on σ\sigma. Then there exists at least one fixed gauge vector σ⋆\sigma^{\star} such that

Pr⁡(S(V^,V)∣σ=σ⋆)≤12.\Pr\left(S(\widehat{V},V)\mid\sigma=\sigma^{\star}\right)\le\frac{1}{2}.

Equivalently,

Pr⁡(S(V^,V)∁∣σ=σ⋆)≥12.\Pr\left(S(\widehat{V},V)^{\complement}\mid\sigma=\sigma^{\star}\right)\ge\frac{1}{2}.

Since this holds for every estimator, the Bayes lower bound under the Rademacher gauge prior implies the corresponding minimax lower bound over fixed unknown gauges.

Unequal source sizes

For completeness we record the source-label aligned-mass bound for unequal source sizes. Let source ℓ\ell contain sℓs_\ell samples and let smax⁡=max⁡ℓsℓs_{\max}=\max_\ell s_\ell. If qq is measurable with respect to source labels and independent of the unknown gauges, then conditional on qq,

qTg=∑ℓ=1LQℓσℓ,Qℓ=∑r=1sℓqℓr.q^{\mathsf{T}}g=\sum_{\ell=1}^{L}Q_\ell\sigma_\ell,\qquad Q_\ell=\sum_{r=1}^{s_\ell}q_{\ell r}.

Hoeffding’s inequality gives

Pσ(∣∑ℓ=1LQℓσℓ∣≥t∣q)≤2exp⁡(−t22∑ℓQℓ2).\mathbb{P}_{\sigma}\left(\left|\sum_{\ell=1}^{L}Q_\ell\sigma_\ell\right|\ge t\mathrel{\big|}q\right)\le2\exp\left(-\frac{t^{2}}{2\sum_{\ell}Q_\ell^{2}}\right).

Moreover,

Qℓ2≤sℓ∑r=1sℓqℓr2≤smax⁡∑r=1sℓqℓr2,Q_\ell^{2}\le s_\ell\sum_{r=1}^{s_\ell}q_{\ell r}^{2}\le s_{\max}\sum_{r=1}^{s_\ell}q_{\ell r}^{2},

so

∑ℓQℓ2≤smax⁡∥q∥22.\sum_{\ell}Q_\ell^{2}\le s_{\max}\lVert q\rVert_{2}^{2}.

Taking

t=∥q∥22smax⁡log⁡(2/δ)t=\lVert q\rVert_{2}\sqrt{2s_{\max}\log(2/\delta)}

yields

Pσ(Mq(g)≤2smax⁡log⁡2δ∣q)≥1−δ.\mathbb{P}_{\sigma}\left(M_q(g)\le2s_{\max}\log\frac{2}{\delta}\mathrel{\big|}q\right)\ge1-\delta.

Thus source-label-measurable first-order sections cannot produce extensive aligned mass unless one source already contains a constant fraction of the samples.

Proof of Proposition 3

Fix a source ℓ\ell. Since Wℓr=θaℓV+εℓrW_{\ell r} = \theta a_{\ell} V + \varepsilon_{\ell r},

Xℓ=1s∑r=1sWℓr=θsaℓV+Zℓ,Zℓ∼N(0,Id).X_{\ell} = \frac{1}{\sqrt{s}}\sum_{r=1}^{s} W_{\ell r} = \theta\sqrt{s}a_{\ell}V + Z_{\ell}, \qquad Z_{\ell} \sim N(0,I_d).

Because aℓ∼N(0,1)a_{\ell} \sim N(0,1),

Xℓ∼N(0,Id+θ2sVVT).X_{\ell} \sim N(0,I_d+\theta^2sVV^{\mathsf T}).

Now decompose the ss samples within each source into the all-ones direction and its orthogonal complement. The all-ones direction is exactly XℓX_{\ell}. Every orthogonal contrast cancels the common source signal and is pure Gaussian noise independent of VV. This proves equivalence to sparse PCA samples plus VV-independent Gaussian residuals.

A standard sparse-mean recovery lemma

Lemma 13 (Top-kk sparse mean recovery). Let

X=γV+Z,Z∼N(0,Id),V∈Vd,k.X = \gamma V + Z,\qquad Z\sim N(0,I_d),\qquad V\in\mathcal{V}_{d,k}.

Let V^∈arg maxv∈Vd,k∣⟨X,v⟩∣\widehat{V}\in\mathop{\rm arg\,max}_{v\in\mathcal{V}_{d,k}}|\langle X,v\rangle|, equivalently top-kk threshold ∣Xj∣|X_j| and use observed signs. There is a universal constant C>0C>0 such that, for 0<ε,δ<10<\varepsilon,\delta<1, if

γ2≥Cε−2(klog⁡2edk+log⁡2δ),\gamma^2\ge C\varepsilon^{-2}\left(k\log\frac{2ed}{k}+\log\frac{2}{\delta}\right),

then

P(∣⟨V^,V⟩∣≥1−ε)≥1−δ.\mathbb{P}\left(|\langle\widehat{V},V\rangle|\ge1-\varepsilon\right)\ge1-\delta.

Proof. Let

η=sup⁡v∈Vd,k∣⟨Z,v⟩∣.\eta=\sup_{v\in\mathcal{V}_{d,k}}|\langle Z,v\rangle|.

Since

∣Vd,k∣≤(dk)2k≤(2edk)k,|\mathcal{V}_{d,k}|\le{d\choose k}2^k\le\left(\frac{2ed}{k}\right)^k,

a union bound over Gaussian tails gives, with probability at least 1−δ1-\delta,

η≤Cklog⁡2edk+log⁡2δ.\eta\le C\sqrt{k\log\frac{2ed}{k}+\log\frac{2}{\delta}}.

On this event, optimality of V^\widehat{V} implies

∣⟨X,V^⟩∣≥∣⟨X,V⟩∣≥∣γ∣−η,|\langle X,\widehat{V}\rangle|\ge|\langle X,V\rangle|\ge|\gamma|-\eta,

while also

∣⟨X,V^⟩∣≤∣γ∣∣⟨V,V^⟩∣+η.|\langle X,\widehat{V}\rangle|\le|\gamma||\langle V,\widehat{V}\rangle|+\eta.

Hence

∣⟨V,V^⟩∣≥1−2η∣γ∣.|\langle V,\widehat{V}\rangle|\ge1-\frac{2\eta}{|\gamma|}.

The stated condition ensures 2η/∣γ∣≤ε2\eta/|\gamma|\le\varepsilon.

Weighted half-normal concentration

Lemma 14 (Weighted half-normal concentration). Let G1,…,GL∼i.i.d.N(0,1)G_1,\ldots,G_L \mathrel{\overset{\mathrm{i.i.d.}}{\sim}} N(0,1), set Rℓ=∣Gℓ∣R_\ell=|G_\ell|, and let wℓw_\ell be fixed real weights. Put

S=∑ℓwℓ,B2=∑ℓwℓ2,κ1=ERℓ=2/π.S=\sum_{\ell}w_\ell,\qquad B^2=\sum_{\ell}w_\ell^2,\qquad\kappa_1=\mathbb{E}R_\ell=\sqrt{2/\pi}.

There exist universal constants c,C>0c,C>0 such that

P(∣∑ℓwℓ(Rℓ−κ1)∣≥t)≤2exp⁡(−t2B2)\mathbb{P}\left(\left|\sum_{\ell}w_\ell(R_\ell-\kappa_1)\right|\ge t\right)\le2\exp\left(-\frac{t^2}{B^2}\right)

for all t≥0t\ge0. Consequently, if S2/B2≥Clog⁡(2/δ)S^2/B^2\ge C\log(2/\delta), then with probability at least 1−δ1-\delta,

∣∑ℓwℓRℓ∣≥κ12∣S∣.\left|\sum_{\ell}w_\ell R_\ell\right|\ge\frac{\kappa_1}{2}|S|.

Proof. The case B=0B=0 is trivial. Otherwise, the function

F(G1,…,GL)=∑ℓwℓ∣Gℓ∣F(G_1,\ldots,G_L)=\sum_{\ell}w_\ell|G_\ell|

is BB-Lipschitz in Euclidean norm, and EF=κ1S\mathbb{E}F=\kappa_1S. Gaussian concentration for Lipschitz functions [9] therefore gives

P(∣F−κ1S∣≥t)≤2exp⁡(−t22B2).\mathbb{P}\left(|F-\kappa_1S|\ge t\right)\le2\exp\left(-\frac{t^2}{2B^2}\right).

Taking t=κ1∣S∣/2t=\kappa_1|S|/2 and using ∣F∣≥κ1∣S∣−∣F−κ1S∣|F|\ge\kappa_1|S|-|F-\kappa_1S| proves the second claim whenever S2/B2≥Clog⁡(2/δ)S^2/B^2\ge C\log(2/\delta).

Proof of Theorem 4

Conditional on q,σ,R,Vq,\sigma,R,V,

Xq=θ∑ℓ,rqℓrRℓσℓ∥q∥2V+∑ℓ,rqℓrϵℓr∥q∥2.X_q=\theta\frac{\sum_{\ell,r}q_{\ell r}R_\ell\sigma_\ell}{\lVert q\rVert_2}V+\frac{\sum_{\ell,r}q_{\ell r}\epsilon_{\ell r}}{\lVert q\rVert_2}.

Because valid calibration makes qq independent of the downstream feature noise conditional on the source signs and design,

∑ℓ,rqℓrϵℓr∥q∥2∼N(0,Id).\frac{\sum_{\ell,r}q_{\ell r}\epsilon_{\ell r}}{\lVert q\rVert_2}\sim N(0,I_d).

Thus XqX_q has the sparse-mean form.

In the constant-amplitude model,

Aq2=(q⊤g)2∥q∥22=Mq(g).A_q^2=\frac{(q^\top g)^2}{\lVert q\rVert_2^2}=M_q(g).

For source-Gaussian amplitudes, write Qℓ=∑rqℓrQ_\ell=\sum_r q_{\ell r} and wℓ=Qℓσℓw_\ell=Q_\ell\sigma_\ell. Then

Mq(g)=S2∥q∥22,Nsrc(q,g)=S2B2,M_q(g)=\frac{S^2}{\lVert q\rVert_2^2},\qquad N_{\mathrm{src}}(q,g)=\frac{S^2}{B^2},

where S=∑ℓwℓS=\sum_\ell w_\ell and B2=∑ℓwℓ2B^2=\sum_\ell w_\ell^2. By Lemma 14,

∣∑ℓwℓRℓ∣≥κ12∣S∣\left|\sum_\ell w_\ell R_\ell\right|\ge\frac{\kappa_1}{2}|S|

with probability at least 1−δ1-\delta whenever Nsrc(q,g)≳log⁡(2/δ)N_{\mathrm{src}}(q,g)\gtrsim\log(2/\delta). Therefore

∣Aq∣≥κ12Mq(g).|A_q|\ge\frac{\kappa_1}{2}\sqrt{M_q(g)}.

The factor 1/41/4 in θ2Aq2\theta^2A_q^2 is absorbed into ≳\gtrsim. Using failure probability δ/2\delta/2 for the half-normal event and δ/2\delta/2 for Lemma 13, and adjusting constants, completes the proof.

Proof of Lemma 5

Conditional on qq, define Qℓ=∑r=1sqℓrQ_{\ell} = \sum_{r=1}^{s} q_{\ell r}. Then

qTg=∑ℓ=1LQℓσℓ.q^{\mathsf{T}} g = \sum_{\ell=1}^{L} Q_{\ell}\sigma_{\ell}.

This is a Rademacher sum. Hoeffding’s inequality gives

Pσ(∣∑ℓ=1LQℓσℓ∣≥t ∣ q)≤2exp⁡(−t22∑ℓQℓ2).\mathbb{P}_{\sigma}\left(\left.\left|\sum_{\ell=1}^{L} Q_{\ell}\sigma_{\ell}\right| \ge t \,\right|\, q\right) \le2\exp\left(-\frac{t^{2}}{2\sum_{\ell}Q_{\ell}^{2}}\right).

By Cauchy,

∑ℓQℓ2≤s∑ℓ,rqℓr2=s∥q∥22.\sum_{\ell}Q_{\ell}^{2} \le s\sum_{\ell,r}q_{\ell r}^{2} = s\lVert q\rVert_{2}^{2}.

Taking t=∥q∥22slog⁡(2/δ)t = \lVert q\rVert_{2}\sqrt{2s\log(2/\delta)} yields the claim.

Proof of Lemma 6

Conditional on σ\sigma, put

H=B−E[B∣σ].H = B - \mathbb{E}[B\mid\sigma].

The upper-triangular entries of HH are independent and centered. Moreover,

∣Hℓm∣≤2,max⁡ℓ∑mE[Hℓm2∣σ]≤pL.|H_{\ell m}| \le2,\qquad\max_{\ell}\sum_{m}\mathbb{E}[H_{\ell m}^{2}\mid\sigma] \le pL.

We use the sharp bounded-entry spectral-norm estimate for symmetric matrices with independent centered upper-triangular entries [4]. In the present notation, its bounded-entry consequence, with symmetrization if the entry distributions are not symmetric, gives

P(∥H∥op>C(pL+log⁡(2L/δ)) ∣ σ)≤δ.\mathbb{P}\left(\left.\lVert H\rVert_{\mathrm{op}} > C\left(\sqrt{pL}+\sqrt{\log(2L/\delta)}\right)\,\right|\,\sigma\right)\le\delta.

Here ∥⋅∥op\lVert\cdot\rVert_{\mathrm{op}} is the operator norm. Under pL≥Clog⁡(L/δ)pL\ge C\log(L/\delta), this yields

∥H∥op≤CpL\lVert H\rVert_{\mathrm{op}}\le C\sqrt{pL}

with probability at least 1−δ1-\delta. This sharp bound, rather than a direct application of standard matrix Bernstein, is what removes the multiplicative logarithmic factor.

The population matrix

E[B∣σ]=pα(σσT−I)\mathbb{E}[B\mid\sigma]=p\alpha(\sigma\sigma^{\mathsf{T}}-I)

has leading eigenvector σ/L\sigma/\sqrt{L}, leading eigenvalue pα(L−1)p\alpha(L-1), and all remaining eigenvalues equal to −pα-p\alpha. Its eigengap is therefore exactly αpL\alpha pL. If α2pL\alpha^{2}pL exceeds a sufficiently large universal constant, then ∥H∥op≤αpL/4\lVert H\rVert_{\mathrm{op}}\le\alpha pL/4. The Davis–Kahan eigenvector perturbation bound [18] consequently gives

1−β2≤C∥H∥op2(αpL)2≤Cα2pL.1-\beta^{2}\le C\frac{\lVert H\rVert_{\mathrm{op}}^{2}}{(\alpha pL)^{2}}\le\frac{C}{\alpha^{2}pL}.

This proves the lemma without changing its stated degree conditions.

Proof of Theorem 8

Support packing. Write [d]={1,…,d}[d]=\{1,\ldots,d\}, and set ρ=1/64\rho=1/64. A greedy construction on all kk-subsets of [d][d], deleting after each selection every support obtained by replacing at most ⌊ρk⌋\lfloor\rho k\rfloor coordinates, gives supports S1,…,SMS_{1},\ldots,S_{M} satisfying

∣Si∣=k,∣Si△Sj∣>2ρk(i≠j),|S_i|=k,\qquad|S_i\mathbin{\triangle}S_j|>2\rho k\quad(i\ne j),

and

M≥(dk)∑j=0⌊ρk⌋(kj)(d−kj).M \ge\frac{\binom{d}{k}}{\sum_{j=0}^{\lfloor\rho k\rfloor} \binom{k}{j}\binom{d-k}{j}}.

Here △\triangle denotes symmetric difference. For completeness, let h(x)=−xlog⁡x−(1−x)log⁡(1−x)h(x)=-x\log x-(1-x)\log(1-x). The denominator is at most

exp⁡(kh(ρ)+(d−k)h(ρkd−k))≤exp⁡(ρklog⁡(d/k)+2ρklog⁡(e/ρ)).\exp\left(kh(\rho)+(d-k)h\left(\frac{\rho k}{d-k}\right)\right)\le\exp\left(\rho k\log(d/k)+2\rho k\log(e/\rho)\right).

Using (dk)≥(d/k)k\binom{d}{k}\ge(d/k)^k and d/k≥2d/k\ge2, we obtain

log⁡M≥(1−ρ)klog⁡(d/k)−2ρklog⁡(e/ρ)≥c1Ksp\log M\ge(1-\rho)k\log(d/k)-2\rho k\log(e/\rho)\ge c_1K_{\mathrm{sp}}

for a universal c1>0c_1>0, for example c1=1/4c_1=1/4. Choose C0C_0 sufficiently large that log⁡M≥4log⁡2\log M\ge4\log2. Let JJ be uniform on {1,…,M}\{1,\ldots,M\} and set

VJ=k−1/21SJ∈Vd,k.V_J=k^{-1/2}\mathbf{1}_{S_J}\in\mathcal{V}_{d,k}.

From high F1 to index recovery. For any estimated support S^\widehat{S}, define

F1⁡(S^,SJ)=2∣S^∩SJ∣∣S^∣+k.\operatorname{F1}(\widehat{S},S_J)=\frac{2|\widehat{S}\cap S_J|}{|\widehat{S}|+k}.

If this is at least 1−τ1-\tau, then, writing D=∣S^△SJ∣D=|\widehat{S}\triangle S_J|,

D≤τ(∣S^∣+k)≤τ(2k+D),D\le\tau(|\widehat{S}|+k)\le\tau(2k+D),

so

D≤2τk1−τ.D\le\frac{2\tau k}{1-\tau}.

Choose the universal constant τ>0\tau>0 so that 2τ/(1−τ)<ρ2\tau/(1-\tau)<\rho. Then D<ρkD<\rho k, whereas distinct packing supports have distance greater than 2ρk2\rho k. Thus nearest-support decoding in symmetric-difference distance uniquely recovers JJ. This argument does not require ∣S^∣=k|\widehat{S}|=k.

Information bound and Fano. By Proposition 3, the observations reduce to LL independent samples with covariance Id+λVJVJTI_d+\lambda V_JV_J^{\mathsf{T}}, together with independent residual noise whose distribution does not depend on JJ. Let PjP_j denote the aggregate-sample distribution under J=jJ=j, and let P0=N(0,Id)⊗LP_0=\mathcal{N}(0,I_d)^{\otimes L}. The Gaussian KL formula gives

KL⁡(Pj∥P0)=L2[tr⁡(λVjVjT)−log⁡det⁡(Id+λVjVjT)]=L2[λ−log⁡(1+λ)].\operatorname{KL}(P_j\mathbin{\|}P_0)=\frac{L}{2}\left[\operatorname{tr}(\lambda V_jV_j^{\mathsf{T}})-\log\det(I_d+\lambda V_jV_j^{\mathsf{T}})\right]=\frac{L}{2}[\lambda-\log(1+\lambda)].

Consequently,

I(J;observations)=I(J;X1:L)≤1M∑j=1MKL⁡(Pj∥P0)=Isrc.I(J;\text{observations})=I(J;X_{1:L})\le\frac{1}{M}\sum_{j=1}^{M}\operatorname{KL}(P_j\mathbin{\|}P_0)=I_{\mathrm{src}}.

Fano’s inequality [10] implies, for every index decoder J^\widehat{J},

P(J^=J)≤Isrc+log⁡2log⁡M.\mathbb{P}(\widehat{J}=J)\le\frac{I_{\mathrm{src}}+\log2}{\log M}.

If Isrc≤cKspI_{\mathrm{src}}\le cK_{\mathrm{sp}} with c≤c1/4c\le c_1/4, the right-hand side is at most 1/21/2, since log⁡M≥4log⁡2\log M\ge4\log2. The decoding implication above proves the high-F1 lower bound.

Finally, for 0≤λ≤10\le\lambda\le1,

λ24≤λ−log⁡(1+λ)=∫0λt1+t dt≤λ22.\frac{\lambda^2}{4}\le\lambda-\log(1+\lambda)=\int_0^\lambda\frac{t}{1+t}\,dt\le\frac{\lambda^2}{2}.

Hence

Isrc≍Lλ2=θ4ns≍μ4ns,I_{\mathrm{src}}\asymp L\lambda^2=\theta^4n_s\asymp\mu^4n_s,

which gives the stated weak-source condition.

Proof of Remark 12

The mutual-information statement follows from conditional independence:

P(M,V1:T∣σ,E)=P(M∣σ,E)P(V1:T∣σ,E).P(\mathcal{M}, V_{1:T} \mid\sigma, E) = P(\mathcal{M} \mid\sigma, E)P(V_{1:T} \mid\sigma, E).

Therefore I(M;V1:T∣σ,E)=0I(\mathcal{M}; V_{1:T} \mid\sigma, E) = 0. For each task tt, conditional on the same metadata-induced section qq, Theorem 4 applies with V=VtV = V_t, giving

Xq(t)=θAq(t)Vt+Z(t).X_q^{(t)} = \theta A_q^{(t)} V_t + Z^{(t)}.

Proof of Corollary 11

The calibrated claim follows from Theorem 4 with Mq(g)=β2nM_q(g) = \beta^2 n. Source-label statistical insufficiency follows from Theorem 8. For larger uncalibrated sample size nmixn_{\mathrm{mix}}, the compressed source experiment has

N=nmixs,λ=θ2s,Nλ2=θ4nmixs.N = \frac{n_{\mathrm{mix}}}{s}, \qquad\lambda= \theta^2 s, \qquad N\lambda^2 = \theta^4 n_{\mathrm{mix}}s.

The stated window is exactly the sparse-PCA hardness window under Conjecture 10, up to constants and logarithmic factors. High-overlap recovery would imply detection by splitting independent sources into two groups, estimating V^\widehat{V} from the source aggregates in one group, and testing the variance along V^\widehat{V} using the independent aggregates in the other group. The split is across sources, not across repeated observations sharing the same source amplitude.

Noise extension and scope

The first-order versus second-order distinction has a limited extension beyond isotropic noise. Suppose independently across samples

εℓr∼N(0,Σℓ),\varepsilon_{\ell r} \sim N(0, \Sigma_\ell),

where the source covariances Σℓ\Sigma_\ell are fixed and do not depend on VV. Retain the source-amplitude model and valid calibration assumptions from the main text. Conditional on the calibration weights and source variables,

Xq=θAqV+Zq,Zq∼N(0,Σq),Σq=∑ℓ,rqℓr2Σℓ∥q∥22.X_q = \theta A_q V + Z_q, \qquad Z_q \sim N(0, \Sigma_q), \qquad\Sigma_q = \frac{\sum_{\ell,r} q_{\ell r}^{2}\Sigma_\ell}{\lVert q\rVert_2^2}.

Thus calibration still restores a first-order mean signal, although its noise covariance is no longer the identity.

A sufficient support-recovery bound. Let

vq=max⁡j(Σq)jj.v_q = \max_j(\Sigma_q)_{jj}.

For 0<δ<1/20 < \delta< 1/2, a sufficient condition for exact support recovery by top-kk thresholding is

Nsrc(q,g)≳log⁡(4/δ),θ2Mq(g)≳kvqlog⁡(4d/δ).N_{\mathrm{src}}(q,g) \gtrsim\log(4/\delta), \qquad\theta^2 M_q(g) \gtrsim k v_q \log(4d/\delta).

To see this, coordinatewise Gaussian tails and a union bound give

P(max⁡j∣(Zq)j∣>2vqlog⁡(4d/δ) | q,σ,R,V,E)≤δ/2.\mathbb{P}\left(\max_j |(Z_q)_j| > \sqrt{2v_q\log(4d/\delta)} \,\middle|\, q,\sigma,R,V,E\right) \leq\delta/2.

No independence between feature coordinates is needed for this step. Lemma 14, with failure probability δ/2\delta/2, also gives

∣Aq∣≥κ12Mq(g)|A_q| \geq\frac{\kappa_1}{2}\sqrt{M_q(g)}

under the stated aligned-source-count condition. Since every nonzero entry of VV has magnitude 1/k1/\sqrt{k}, a sufficiently large universal constant in the signal condition makes every signal coordinate more than twice the displayed noise bound. The top-kk support is therefore correct with probability at least 1−δ1-\delta.

If

∥Σℓ−Id∥op≤ηfor every source ℓ,\lVert\Sigma_\ell-I_d\rVert_{\mathrm{op}} \leq\eta\qquad\text{for every source } \ell,

then Σq\Sigma_q is a convex combination of the source covariances, so

∥Σq−Id∥op≤η,vq≤1+η.\lVert\Sigma_q-I_d\rVert_{\mathrm{op}} \leq\eta, \qquad v_q \leq1+\eta.

Here ∥⋅∥op\lVert\cdot\rVert_{\mathrm{op}} denotes the operator norm.

The uncalibrated experiment. Without calibration, the source aggregate has distribution

Xℓ∼N(0,Σℓ+θ2sVV⊤).X_{\ell} \sim\mathcal{N}(0, \Sigma_{\ell} + \theta^{2}sV V^{\top}).

The remaining s−1s-1 orthogonal within-source contrasts are independent background observations with distribution N(0,Σℓ)\mathcal{N}(0, \Sigma_{\ell}), and are independent of XℓX_{\ell}. Their distributions do not depend on VV, but they can help estimate unknown source covariances. The uncalibrated experiment is therefore a generalized, possibly heteroscedastic spiked-covariance experiment with background-only observations, not standard identity-background sparse PCA. Its dependence on VV remains second-order, but sharp thresholds and computational hardness for arbitrary unknown Σℓ\Sigma_{\ell} are not established here.

Calibration validity concerns the information channel producing qq; noise covariance concerns the downstream observations after qq is fixed. These are separate assumptions. This extension does not establish robustness to task-dependent metadata, approximate sparsity, or richer source transformations.

Additional experimental details

Simulation records use deterministic base seed

base_seed = 12345

and per-cell seeds generated by

stable_seed(base_seed, experiment, configuration, seed, method)

Unless stated otherwise, repeated-run cells use B=30B = 30 independent randomized runs. The aligned-source-count experiment uses B=100B = 100. The controlled real-feature experiments use seeds 12345, . . . , 12374.

Role of the experiments. The methods are calibration-information controls rather than claims of general algorithmic superiority: oracle signs represent perfect calibration, relative synchronization represents noisy calibration, source labels alone represent the uncalibrated experiment, and random or shuffled records represent uninformative or mismatched metadata. Synthetic experiments test finite-sample scaling predictions. The scikit-learn studies test cancellation and restoration on real features with imposed source signs. The physical study tests separately acquired calibration and task reuse. These experiments do not prove the statistical equivalence in Proposition 3.

Meaning and unit of SE. SE means standard error, not squared error. If mbm_b is the aggregated metric from randomized run bb, then

SE=sd⁡(m1,…,mB)B\mathrm{SE} = \frac{\operatorname{sd}(m_1,\ldots,m_B)}{\sqrt{B}}

where sd is the sample standard deviation. For digits, each mbm_b first averages test accuracy across the five digit-pair tasks; SE is then computed across the 30 run-level averages. For breast cancer, it is computed across 30 randomized-run test accuracies.

The reuse experiment instead fixes one calibration graph and reports mean and SE across 32 independently sampled downstream tasks, conditional on that shared graph. These are not 30-seed estimates of uncertainty across calibration graphs. For the physical study, SE is the sample standard deviation of source-cluster bootstrap replicate estimates, as described in Appendix § 11.§ 5; it is not divided by the square root of the number of bootstrap replicates. These uncertainty units are distinct.

Table 4 and Table 5 provide the numerical records underlying Figure 1(a) and Figure 1(c), respectively, including additional phase-curve configurations.

Mechanism tests

Figure 1. Mechanism tests. Calibration follows sparse-mean scaling; uninformative or mismatched metadata fails to restore alignment. The calibrated/source-label gap is largest for weak sources. Gray dashed lines mark unit strength.

(d,k)(d,k)ρcal\rho_{\mathrm{cal}}nnOracle CorrOracle Succ90Sync CorrSync Succ90
(1000,20)(1000,20)0.2502420.095 ±\pm 0.0100.0000.098 ±\pm 0.0130.000
0.5004840.192 ±\pm 0.0120.0000.220 ±\pm 0.0180.000
1.0009660.377 ±\pm 0.0210.0000.378 ±\pm 0.0160.000
2.00019300.680 ±\pm 0.0160.0330.668 ±\pm 0.0140.000
4.00038580.913 ±\pm 0.0080.8330.900 ±\pm 0.0080.700
8.00077160.997 ±\pm 0.0021.0000.995 ±\pm 0.0031.000
(3000,50)(3000,50)0.2506260.090 ±\pm 0.0060.0000.095 ±\pm 0.0090.000
0.50012520.195 ±\pm 0.0100.0000.179 ±\pm 0.0100.000
1.00025020.369 ±\pm 0.0110.0000.386 ±\pm 0.0100.000
2.00050020.661 ±\pm 0.0090.0000.657 ±\pm 0.0100.000
4.000100040.908 ±\pm 0.0070.6670.908 ±\pm 0.0050.700
8.000200060.995 ±\pm 0.0021.0000.994 ±\pm 0.0021.000
(8000,80)(8000,80)0.25011020.069 ±\pm 0.0050.0000.078 ±\pm 0.0060.000
0.50022020.167 ±\pm 0.0080.0000.165 ±\pm 0.0080.000
1.00044040.362 ±\pm 0.0080.0000.372 ±\pm 0.0080.000
2.00088060.649 ±\pm 0.0080.0000.648 ±\pm 0.0080.000
4.000176100.913 ±\pm 0.0050.8330.915 ±\pm 0.0040.800
8.000352200.996 ±\pm 0.0011.0000.997 ±\pm 0.0011.000

Table 4. Source-Gaussian calibrated phase transition. Corr entries are mean ±\pm SE, and Succ90 is reported in adjacent columns. Each entry uses 30 seeds.

sssμ2s\mu^2LLμ4ns/Ksp\mu^4ns/K_{\mathrm{sp}}OracleSyncDiag+PCADiag F1Sync Succ90
10.020196490.0800.955 ±\pm 0.0040.962 ±\pm 0.0070.031 ±\pm 0.0090.038 ±\pm 0.0091.000
20.04098240.1600.960 ±\pm 0.0060.960 ±\pm 0.0070.022 ±\pm 0.0060.033 ±\pm 0.0070.967
40.08049120.3200.962 ±\pm 0.0060.963 ±\pm 0.0060.032 ±\pm 0.0070.050 ±\pm 0.0091.000
80.16024560.6400.958 ±\pm 0.0080.950 ±\pm 0.0050.045 ±\pm 0.0100.065 ±\pm 0.0111.000
160.32012281.2800.950 ±\pm 0.0070.953 ±\pm 0.0070.084 ±\pm 0.0120.110 ±\pm 0.0120.967
320.6406142.5600.957 ±\pm 0.0080.952 ±\pm 0.0070.194 ±\pm 0.0240.165 ±\pm 0.0130.967
641.2803075.1200.963 ±\pm 0.0070.943 ±\pm 0.0060.454 ±\pm 0.0240.283 ±\pm 0.0160.967
1282.56015410.2730.940 ±\pm 0.0070.947 ±\pm 0.0080.610 ±\pm 0.0180.422 ±\pm 0.0180.933
2565.1207720.5470.950 ±\pm 0.0070.943 ±\pm 0.0110.774 ±\pm 0.0170.635 ±\pm 0.0250.900
51210.2403840.5600.925 ±\pm 0.0190.928 ±\pm 0.0150.856 ±\pm 0.0190.767 ±\pm 0.0290.767
102420.4801981.1190.915 ±\pm 0.0160.917 ±\pm 0.0160.899 ±\pm 0.0180.843 ±\pm 0.0270.733

Table 5. Source-Gaussian micro-source separation. Entries are mean ±\pm SE, except Sync Succ90, which reports the success rate. Each entry uses 30 seeds.

Compute environment

All experiments were run on the following hardware:

  • GPU: NVIDIA V100 (32GB)

  • CPU: Intel Xeon Gold 6248 (2.5GHz, 20 cores)

  • RAM: 256GB DDR4

  • Storage: NVMe SSD

The retained real-data log used Python 3.6.9, NumPy 1.19.5, and scikit-learn 0.24.2.

Source-Gaussian calibrated phase transition

Table 4 reports the full numerical records underlying Figure 1(a). Its purpose is twofold: first, to make the phase-transition plot fully auditable at the level of individual experimental cells; and second, to show that the calibrated transition is stable across several ambient dimensions and sparsity levels. The main pattern is the same as in the figure: once the nominal calibrated strength is large enough, spectral synchronization closely tracks oracle calibration, supporting the sparse-mean scaling predicted by the Corollary 7.

Micro-source separation

Table 5 gives the full numerical records for Figure 1(c), where the calibrated first-order scale is held fixed while the per-source strength sμ2s\mu^2 varies. Here Diag+PCA is sparse PCA on Xℓ=s−1/2∑rWℓrX_\ell= s^{-1/2}\sum_r W_{\ell r}. The table makes the separation especially transparent: in the weak-source regime, calibrated methods remain strong, whereas source-label baselines improve only when the second-order sparse-PCA quantity μ4ns\mu^4ns becomes sufficiently large. The explicit column μ4ns/Ksp\mu^4ns/K_{\mathrm{sp}} is included to show how the empirical transition aligns with the information scale highlighted in the Theorem 8.

Controlled real-feature setup

Digits use the scikit-learn handwritten-digits dataset with 1797 images, 64-dimensional pixel features, and five binary digit-pair tasks: 0 vs 6, 1 vs 7, 2 vs 5, 3 vs 8, and 4 vs 9. Breast cancer predicts malignant versus benign using the scikit-learn breast-cancer dataset with 569 examples and 30 real-valued features. In both cases, examples are randomly partitioned into sources, source gauges and calibration logs are imposed synthetically, and labels and non-gauge feature processing are otherwise unchanged; calibration graphs use the same relative-sign model as in § 3.

Separately acquired physical calibration

The physical study tests whether source calibration can be acquired separately and reused across tasks. It is not a test of the exact Gaussian model or evidence for arbitrary multi-source transformations.

Calibration acquisition. We use 24 fixed device–mounting sources with physically induced axis-sign conventions, rather than signs imposed on stored features. Each source contributes 10 separately recorded reference-motion trials, giving 240 calibration trials in total. The time-normalized calibration traces are aggregated into one reference trace per source. Pairwise correlation signs produce a complete graph with

(242)=276\binom{24}{2} = 276

same/opposite records. No downstream labels, recordings, or task-derived features enter this graph. It is synchronized once, and the resulting source alignment is reused unchanged across four positive-versus-negative directed-motion classification tasks. Recorded mounting signs are used only for alignment evaluation and the oracle control, not for relative graph construction, synchronization, or ordinary downstream classifier training.

Downstream split. The downstream dataset contains

24×4×2×20=384024 \times4 \times2 \times20 = 3840

complete trials. Within every source–task–class combination, the 20 trials are split 12/4/4 into training/validation/test sets before window extraction, giving 2304/768/768 complete trials. Windows from the same trial never cross splits. No downstream observations are used to estimate the calibration weights.

Results and uncertainty. Table 6 reports macro balanced test accuracy across the four tasks. Relative calibration gives per-task balanced accuracies 0.928, 0.905, 0.892, and 0.883. Reported SE is the sample standard deviation of source-cluster bootstrap replicate estimates, resampling the 24 device–mounting sources rather than individual trials or windows.

MethodBalanced test accuracyMq/nM_q/n
Oracle mounting polarity0.921 ± 0.0041.000 ± 0.000
Relative calibration + synchronization0.902 ± 0.0060.916 ± 0.015
No calibration0.558 ± 0.0110.044 ± 0.010
Shuffled metadata0.553 ± 0.0120.040 ± 0.009

Table 6. Physical calibration reused across four tasks. Entries are estimates ± source-cluster bootstrap SE. Mounting signs are unavailable to the relative-calibration procedure.

The 276 graph records are derived from 240 calibration trials; they are not 276 independent physical acquisitions. Edges sharing reference traces can have dependent errors. Consequently, this experiment supports the acquisition-and-reuse mechanism, but does not verify the independent-edge graph law or exact satisfaction of the theoretical metadata-validity condition.

Limitations

The main analysis isolates a minimal source ambiguity: one sign bit can erase a first moment and leave a covariance spike. Its sharp separation uses exactly sparse equal-magnitude signals, independent Gaussian source amplitudes, isotropic Gaussian feature noise, and task-independent calibration. The non-isotropic Gaussian extension in Appendix § 10 gives observation formulas and a sufficient support-recovery bound, not sharp thresholds or computational hardness for arbitrary unknown source covariances. Rotations, permutations, nonlinear transformations, approximate sparsity, and task-dependent drift require additional arguments; imperfect alignment under richer transformations need not preserve coordinate sparsity. A practical heuristic for unequal marginal noise scales is to estimate coordinatewise variances from within-source centered residuals, combine them using normalized squared calibration weights, and standardize the calibrated aggregate before selection. This is variance normalization, not whitening or a general covariance-adaptive guarantee. It ignores off-diagonal geometry and can be inefficient for highly correlated features. Estimates can be unstable with few repeats; when s=1s = 1, they require additional structure, pooling, or separate noise-only repeats. Full whitening is not a drop-in solution because covariance estimation is difficult in the micro-source regime and whitening can destroy coordinate sparsity. Task-independent missing or corrupted calibration can reduce alignment while remaining valid. Dependence on downstream features, labels, or amplitudes is a different issue: approximate validity is not analyzed. The synthetic studies test the formal model, the controlled real-feature studies impose source signs, and the physical study supplies one concrete acquisition-and-reuse setting. It does not establish applicability to general multi-source transformations or exact satisfaction of the theoretical channel assumptions. Finally, the computational separation remains conditional on sparse-PCA hardness.

Broader impact

This work studies the statistical value of calibration metadata in a stylized multi-source setting. The positive motivation is straightforward. In scientific sensing, distributed monitoring, medical measurement pipelines, and federated systems, source-specific conventions can hide a signal that is present in every source. Understanding when lightweight metadata can recover that signal may reduce data waste, improve robustness across institutions, and lower the need for expensive task-specific relabeling. At the same time, any method that uses source metadata deserves care. Metadata can encode operational details, institution identity, or deployment structure. In real systems, this may create privacy, governance, or fairness concerns if calibration channels reveal sensitive attributes or if metadata quality differs systematically across groups. A second risk is conceptual: invalid metadata can quietly act like supervision. The paper therefore explicitly separates reusable calibration from task-specific alignment.

The present work is theoretical and does not introduce a new surveillance or decision pipeline. Still, if similar methods are deployed in practice, we recommend: (i) auditing what metadata is collected and why, (ii) documenting whether the metadata is task-independent, (iii) checking whether calibration quality differs across subpopulations, and (iv) reporting failure modes when metadata is missing, corrupted, or strategically manipulated. Finally, the compute footprint of our experiments is modest by current machine learning standards. The method itself consists of spectral synchronization, weighted averaging, and thresholding, rather than large-scale model training.

NeurIPS Paper Checklist

  1. Claims

    Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

    Answer: [Yes]

    Justification: In the abstract and introduction, we state the same claims that we prove and test in the paper: source-labelled uncalibrated data reduces to sparse PCA, calibration restores a sparse-mean signal, and this leads to statistical and conditional computational separations in the micro-source regime. We also state the scope of the paper through the sparse Gaussian hard-core model introduced in § 2.

  1. Limitations

    Question: Does the paper discuss the limitations of the work performed by the authors?

    Answer: [Yes]

    Justification: We discuss limitations explicitly in Appendix § 12. In particular, we note the stylized Z2\mathbb{Z}_2 gauge model, the validity assumptions on metadata, the simplifying assumptions used in several results, and the controlled nature of the experiments.

  1. Theory assumptions and proofs

    Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

    Answer: [Yes]

    Justification: We state the model assumptions in § 2, and each theorem and lemma gives the assumptions needed for the result. Full proofs are provided in Appendix § 9, and we clearly mark the computational separation as conditional on Conjecture 10.

  1. Experimental result reproducibility

    Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

    Answer: [Yes]

    Justification: We describe the experimental pipeline in § 2, report the main settings and outcomes in § 4, and give detailed records in Appendix § 11. We also provide complete experimental reproduction code in the supplemental material.

  1. Open access to data and code

    Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

    Answer: [Yes]

    Justification: We provide anonymized reproduction code and instructions in the supplemental material. The synthetic experiments are generated from the stated model, and the real-feature experiments use public scikit-learn datasets described in Appendix § 11.

  1. Experimental setting/details

    Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer) necessary to understand the results?

    Answer: [Yes]

    Justification: We specify the algorithm in Algorithm 1 and describe the experimental settings, datasets, seeds, metrics, and task constructions in § 4 and Appendix § 11. Our method does not rely on iterative optimization, so there are no optimizer choices to report.

  1. Experiment statistical significance

    Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

    Answer: [Yes]

    Justification: Throughout the experimental section and appendix, we report means together with standard errors or success rates, and we state the number of random seeds used in each experiment. Appendix § 11 also makes clear that the reported error bars are standard errors.

  1. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We report the hardware and software environment in Appendix § 11, including GPU, CPU, RAM, storage, and library versions.

  1. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?

Answer: [Yes]

Justification: To the best of our knowledge, yes. This work is primarily theoretical, uses synthetic data and public datasets, does not involve human subjects, and preserves anonymity in the submission.

  1. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We discuss broader impacts in Appendix § 13. There we describe possible benefits in multi-source sensing and measurement settings, as well as risks related to privacy, governance, fairness, and misuse of metadata.

  1. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pre-trained language models, image generators, or scraped datasets)?

Answer: [N/A]

Justification: Our release does not include high-risk generative models, scraped datasets, or similar assets that would require special safeguards of that kind. The released asset is reproduction code for the theory-driven experiments in the paper.

  1. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We cite the public datasets and software packages we use, including the scikit-learn datasets and package versions in Appendix § 11.

  1. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We provide complete experimental reproduction code together with instructions in the supplemental material. We do not introduce a new dataset or pretrained model beyond this code release.

  1. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [N/A]

Justification: This paper does not involve crowdsourcing or research with human subjects.

  1. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [N/A]

Justification: This paper does not involve crowdsourcing or research with human subjects.

Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigor, or originality of the research, declaration is not required.

Answer: [N/A]

Justification: LLMs are not part of the core methods, experiments, or theoretical development in this research.

References

  1. [1]Emmanuel Abbe, Afonso S. Bandeira, and Georgina Hall. Exact recovery in the stochastic block model. IEEE Transactions on Information Theory, 62(1):471–487, 2016. doi: 10.1109/TIT.2015.2490670.DOI
  2. [2]Arash A. Amini and Martin J. Wainwright. High-dimensional analysis of semidefinite relaxations for sparse principal components. The Annals of Statistics, 37(5B):2877–2921, 2009. doi: 10.1214/08-AOS664.DOI
  3. [3]Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:1907.02893, 2019. URL https://arxiv.org/abs/1907.02893.
  4. [4]Afonso S. Bandeira and Ramon van Handel. Sharp nonasymptotic bounds on the norm of random matrices with independent entries. The Annals of Probability, 44(4):2479–2506, 2016. doi: 10.1214/15-AOP1025.DOI
  5. [5]Afonso S. Bandeira, Amit Singer, and Daniel A. Spielman. A cheeger inequality for the graph connection laplacian. SIAM Journal on Matrix Analysis and Applications, 34(4):1611–1630, 2013. doi: 10.1137/120875338.DOI
  6. [6]Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine Learning, 79(1–2):151–175, 2010. doi: 10.1007/s10994-009-5152-4.
  7. [7]Tamir Bendory, Nicolas Boumal, Chao Ma, Zhizhen Zhao, and Amit Singer. Bispectrum inversion with application to multireference alignment. IEEE Transactions on Signal Processing, 66(4):1037–1050, 2018. doi: 10.1109/TSP.2017.2775591.DOI
  8. [8]Quentin Berthet and Philippe Rigollet. Complexity theoretic lower bounds for sparse principal component detection. In Proceedings of the 26th Annual Conference on Learning Theory, volume 30 of Proceedings of Machine Learning Research, pages 1046–1066, 2013.
  9. [9]Stéphane Boucheron, Gábor Lugosi, and Pascal Massart. Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford University Press, 2013. ISBN 978-0-19-953525-5. doi: 10.1093/acprof:oso/9780199535255.001.0001.DOI
  10. [10]Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. Wiley-Interscience, 2 edition, 2006. ISBN 978-0-471-24195-9. doi: 10.1002/047174882X.DOI
  11. [11]Yash Deshpande and Andrea Montanari. Sparse PCA via covariance thresholding. In Advances in Neural Information Processing Systems 27, pages 334–342, 2014. URL https://proceedings.neurips.cc/paper/2014/hash/07a45842fcab1f6116c50549a437c254-Abstract.html.arxiv.org/abs/1311.5179
  12. [12]Iain M. Johnstone and Arthur Yu Lu. On consistency and sparsity for principal components analysis in high dimensions. Journal of the American Statistical Association, 104(486):682–693, 2009. doi: 10.1198/jasa.2009.0121.DOI
  13. [13]Peter Kairouz, H. Brendan McMahan, et al. Advances and open problems in federated learning. Foundations and Trends in Machine Learning, 14(1–2):1–210, 2021. doi: 10.1561/2200000083.DOI
  14. [14]Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness, Wei Guo, Berton Earnshaw, Imran Haque, Sara M. Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. WILDS: A benchmark of in-the-wild distribution shifts. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 5637–5664, 2021.arxiv.org/abs/2012.07421
  15. [15]Amelia Perry, Alexander S. Wein, and Afonso S. Bandeira. Statistical limits of spiked tensor models. Annales de l’Institut Henri Poincaré, Probabilités et Statistiques, 56(1):230–264, 2020. doi: 10.1214/19-AIHP960.DOI
  16. [16]Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=ryxGuJrFvS.
  17. [17]Amit Singer. Angular synchronization by eigenvectors and semidefinite programming. Applied and Computational Harmonic Analysis, 30(1):20–36, 2011. doi: 10.1016/j.jacha.2010.02.001.
  18. [18]Yi Yu, Tengyao Wang, and Richard J. Samworth. A useful variant of the Davis–Kahan theorem for statisticians. Biometrika, 102(2):315–323, 2015. doi: 10.1093/biomet/asv008.
  19. [19]Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4396–4415, 2023. doi: 10.1109/TPAMI.2022.3195549.DOI
  20. [20]Yibo Zhou. Rethinking reconstruction autoencoder-based out-of-distribution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7379–7387, 2022. doi: 10.1109/CVPR52688.2022.00723. URL https://doi.org/10.1109/CVPR52688.2022.00723.
  21. [21]Yibo Zhou, Hai-Miao Hu, Jinzuo Yu, Zhenbo Xu, Weiqing Lu, and Yuran Cao. A solution to co-occurrence bias: Attributes disentanglement via mutual information minimization for pedestrian attribute recognition. In Edith Elkind, editor, Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23, pages 1831–1839. International Joint Conferences on Artificial Intelligence Organization, 8 2023. doi: 10.24963/ijcai.2023/203. URL https://doi.org/10.24963/ijcai.2023/203. Main Track.
  22. [22]Yibo Zhou, Hai-Miao Hu, Yirong Xiang, Xiaokang Zhang, and Haotian Wu. Pedestrian attribute recognition as label-balanced multi-label learning. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 61964–61978. PMLR, 21–27 Jul 2024. URL https://proceedings.mlr.press/v235/zhou24j.html.arxiv.org/abs/2405.04858
  23. [23]Yibo Zhou, Hai-Miao Hu, Jinzuo Yu, Haotian Wu, Shiliang Pu, and Hanzi Wang. A solution to co-occurrence bias in pedestrian attribute recognition: Theory, algorithms, and improvements. International Journal of Computer Vision, 133(7):4712–4726, 2025. doi: 10.1007/s11263-025-02405-7. URL https://doi.org/10.1007/s11263-025-02405-7.
  24. [24]Yibo Zhou, Bo Li, Hai-Miao Hu, Xiaokang Zhang, Dongping Zhang, and Hanzi Wang. Heterogeneous feature re-sampling for balanced pedestrian attribute recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(4):2706–2722, 2025. doi: 10.1109/TPAMI.2025.3526930. URL https://doi.org/10.1109/TPAMI.2025.3526930.
  25. [25]Yibo Zhou, Bo Li, Hai-Miao Hu, Hanzi Wang, Xiaokang Zhang, and Ruifan Zhang. Is spurious correlation removal always learnable?, 2026. URL https://arxiv.org/abs/2606.12930.

Paper details

vlnc:W_w8sfxf97
Version
v1 · latest
Published
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents