WorksComputer Science
Can Metadata Fix the Gauge? Calibration Turns Sparse Multi-Source Learning from Sparse PCA into Sparse Mean Recovery
Abstract
Source labels tell us where a sample came from, but not how that source oriented its coordinates. In a sparse Gaussian multi-source model with source-specific sign gauges, that distinction changes the learning problem. Without calibration, pooling across sources erases the mean signal, and the source-labelled experiment reduces to sparse PCA. With task-independent relative-gauge metadata, synchronization aligns sources and the same observations become a sparse mean problem. We formalize the value of metadata through AlignedMass, which quantifies how much first-order signal a metadata-induced alignment restores. Source labels create only limited AlignedMass, whereas valid calibration can create AlignedMass proportional to the full sample size. This yields a statistical separation in micro-source regimes and, under standard sparse-PCA hardness assumptions, a computational separation as well. Synthetic experiments check the predicted scaling; controlled real-feature and physical sensor studies examine sign cancellation, separately acquired calibration, and reuse across tasks.
Introduction
Multi-source learning is written as if every source uses the same coordinate convention. In practice this convention can fail. A sensor polarity may be reversed, a scanner pipeline may reflect a latent factor, or a client representation may flip the sign of a hidden component. Within each source the signal may be coherent, but across sources naive pooling can cancel it. This paper asks when metadata can fix such a source-specific gauge, and why this is different from merely knowing source labels.
A concrete example is a sensor whose mounting reverses the sign of a measured axis. Source identity tells us which recordings share a mounting, but not whether two mountings use the same or opposite axis convention. A separately recorded reference motion can provide that comparison before any downstream task is observed. In our physical sensor study (Section E.5), these same/opposite records are synchronized once and reused across four classification tasks. Here synchronization means estimating consistent source orientations up to one global sign; linearization means averaging after this alignment to restore a mean signal. We study this question in a stylized sparse Gaussian model. Each source observes the same unknown sparse direction, but through an unknown source-specific sign gauge. Source labels tell the learner which samples belong together, yet they do not reveal how one source is oriented relative to another. Calibration metadata is different: relative polarity logs, orientation edges, or noisy absolute orientation records can carry information about the source gauges. Such metadata can be synchronized into a common convention, using ideas related to synchronization over and other groups [5, 17]. The main message is that these two kinds of side information induce different statistical experiments. With only source labels, the learner can aggregate within each source, but the unknown gauges remove first-order signal across sources. The resulting source-labelled experiment is exactly sparse PCA, plus residual Gaussian noise independent of the task direction. With valid relative-gauge metadata, synchronization selects a gauge section and restores first-order structure. Conditional on source amplitudes, the same downstream observations become a sparse mean problem. Thus labels identify groups, whereas calibration identifies orientation.
Source identifiers, group labels, and institution tags are useful for organizing data and modeling heterogeneity [3, 6, 13, 14, 16, 24], but they do not by themselves specify how source-specific coordinate conventions should be aligned. Calibration metadata plays a narrower role: it provides information about nuisance source conventions rather than the downstream task. In our model, this distinction changes the statistical experiment: the available signal changes from second-order to first-order, leading to different statistical and computational thresholds.
To quantify the value of metadata, we introduce aligned mass. Given metadata-induced sample weights and the hidden gauge vector , aligned mass is times their squared cosine similarity. In the constant-amplitude companion model, is exactly the squared first-order signal strength. In the source-Gaussian model it still controls the nominal first-order signal, while an aligned-source-count condition controls concentration of random source amplitudes. This viewpoint makes the comparison sharp: source labels can create only limited aligned mass because they know which samples share a source but not which way that source points; valid relative-gauge metadata can create aligned mass proportional to the full sample size.
The resulting separation is most transparent in the micro-source regime, where each source is individually weak. Calibrated recovery succeeds at the sparse-mean scale, while source-labelled uncalibrated recovery is statistically insufficient at the corresponding sparse-PCA scale. Under standard sparse-PCA hardness assumptions [8, 11], there is also a conditional computational separation: calibrated recovery is polynomial-time after synchronization, whereas source-labelled recovery remains as hard as sparse PCA through the usual hard window.
A useful feature of calibration metadata is that it is task-independent. It fixes a source convention rather than revealing the downstream sparse direction. One relative-gauge graph can therefore be collected and synchronized once, then reused across many independent downstream tasks sharing the same source gauges. This differs from feature- or label-derived alignment, which may be useful but uses task-coupled information rather than external calibration. Our contributions are:
We identify a sparse multi-source setting in which source-labelled uncalibrated data is statistically equivalent to sparse PCA, while calibrated data becomes sparse mean recovery.
We introduce aligned mass as a quantitative measure of the first-order value of metadata-induced alignment, and show that source labels cannot reliably create extensive aligned mass whereas valid relative-gauge metadata can.
We derive an unconditional statistical separation in micro-source regimes and a conditional computational separation under standard sparse-PCA hardness. Experiments check phase transitions, uninformative or mismatched metadata controls, calibration reuse, and controlled real-feature behavior.
Spectral synchronization, weighted averaging, and top- thresholding are standard components. Our contribution is the experiment-level converse without calibration, together with the quantitative link from noisy calibration to restored first-order signal, rather than a new synchronization algorithm.
Model, metadata, and alignment
A sparse multi-source model
Let , . There are sources, each contributing samples, so the total sample size is . The unknown sparse direction is . Source has an unknown sign gauge and a nonnegative amplitude , independent of . Writing , the observation model is
Here is the signal magnitude, counts nonzero coordinates, and is the sparse-complexity term, with Euler’s number and the natural logarithm. The feature noises are mutually independent and independent of , , . The Rademacher gauge prior means that each source sign is independently and uniformly sampled from . The downstream theoretical task is to recover up to a global sign, or its support . We write , , and for inequalities or equivalence up to universal positive constants. The same statistic arises from a simple two-view classification model; the derivation appears in Appendix § 8. The model separates two roles that are easy to conflate:
is a coordinate convention. This is the part calibration metadata may reveal.
is a source amplitude. This is nuisance randomness and is not something valid metadata should depend on.
Probabilities involving are with respect to this Rademacher prior. We also use the constant-amplitude companion , where aligned mass is exact.
What counts as valid calibration metadata?
Source labels identify which samples belong to the same source. Calibration metadata instead provides information about source signs. A relative-polarity log or orientation edge is a noisy record of whether two sources use the same or opposite sign convention; a noisy absolute orientation record compares one source with a known reference.
Let contain only exogenous source-side structure, such as source identities, source sizes, calibration-graph topology, and a pre-specified calibration design. In the theoretical model, is fixed or sampled independently of , , and task randomness; it cannot encode downstream labels or features. Let collect downstream observations, their feature noise, and any downstream labels. In the two-view example, these labels are ; the main recovery model requires no additional labels.
Definition 1 (Valid calibration metadata). A metadata channel is valid calibration metadata if
Thus, conditional on source signs and the exogenous source design, calibration is independent of the downstream direction, amplitudes, observations, labels, and feature noise.
Validity concerns task independence, not accuracy. Random metadata can be valid but uninformative; missing or independently corrupted calibration records can also remain valid. Conversely, feature- or label-derived alignment can be useful while violating task independence.
With externally collected valid calibration, weights are estimated without using any downstream observations, so scarce micro-source samples need not be split to estimate alignment. Sample splitting is relevant to task-derived comparison procedures: it can separate weight estimation from the noise in a final statistic, but does not make that alignment task-independent or reusable calibration.
aligned mass: how much first-order signal metadata restores
Let be nonzero sample weights constructed from calibration metadata and source structure. For example, estimated source signs give ; spectral synchronization can instead supply real-valued source weights. We also use such metadata-derived weights a section. At the sample level, the hidden sign vector is .
The constant-amplitude companion model makes the role of these weights explicit:
This motivates
because is exactly the squared restored mean-shift strength. Since ,
Thus aligned mass is an effective first-order sample size, and is normalized alignment. No assumption is needed. Exact aligned mass is a theoretical and evaluation quantity, not an algorithm input: the algorithm uses , not the unknown . Planted signs are available for evaluation in simulations; recorded mounting signs in the physical study are used only for alignment evaluation and the oracle control.
For equal source size , we can also quantify the metadata channel’s posterior expected alignment. Define the posterior source-sign correlation matrix
and its alignment capacity
The supremum is over deterministic metadata-measurable weights; independent algorithmic randomness may be conditioned on.
Proposition 2 (Posterior alignment capacity). For equal source size ,
where denotes the largest eigenvalue. An optimal source-level weighting is a top eigenvector of .
Under the independent Rademacher prior, source labels give , whereas perfect relative calibration determines , even though the global sign remains unknown. Consequently,
This is a posterior expectation statement; the synchronization result below controls realized alignment with high probability.
In the source-Gaussian model, amplitudes vary across sources. Write
and define the aligned source count
with value zero when the denominator vanishes. Aligned mass controls the nominal first-order signal, while controls concentration of the weighted random amplitudes around their mean. High aligned mass concentrated on too few sources need not produce a stable realized signal.
Main results
We first show that source labels alone expose only second-order information. We then show how calibration metadata restores first-order structure.
Source labels alone reduce to sparse PCA
If source labels are available but gauges are not aligned, the sufficient source statistic is , with . Since , , where .
Proposition 3 (Source-labelled data is sparse PCA). In the source-Gaussian model with equal source size under the Rademacher gauge prior, the source-labelled uncalibrated observations are statistically equivalent to
together with residual Gaussian noise independent of . Hence any source-labelled uncalibrated estimator can be simulated from sparse PCA samples plus -independent randomness.
Under the Rademacher-gauge experiment, this is an equivalence of statistical experiments, not a statement about a particular algorithm or a particular linear statistic. Consequently it already covers arbitrary feature-dependent procedures applied to the source-labelled uncalibrated samples, including attempts to infer source signs or align sources from the downstream features themselves. Such procedures can only use the sparse-PCA information in the source aggregates together with -independent residual noise. In this sense, feature-derived self-alignment from the uncalibrated task data is not external calibration metadata; if it succeeds in the corresponding hard regime, it is exploiting or solving the sparse-PCA instance exposed by Proposition 3.
Thus marginalizing the gauges removes first-order signal and leaves only the second-order spike.
Calibration turns the same observations into a sparse mean problem
Now suppose metadata supplies a section . Form the weighted linear statistic
Conditional on , , , , we obtain
So after gauge selection, the problem is sparse mean recovery.
In the constant-amplitude model, exactly. In the source-Gaussian model, aligned mass still governs the nominal signal and aligned source count guarantees concentration.
Theorem 4 (Calibration linearizes sparse recovery). Let be constructed exclusively from valid calibration metadata and , with independent algorithmic randomness, and let . Conditional on , , , , suppose
and
Then top- thresholding of , with observed coordinate signs, returns satisfying
with probability at least over the source amplitudes and downstream feature noise, up to universal constants.
The proof combines a standard sparse-mean thresholding bound with a weighted half-normal concentration lemma; both are given in Appendix § 9.
Why source labels cannot create enough aligned mass
The next lemma formalizes a simple but decisive limitation: if a weighting rule only knows source labels and not source gauges, then its correlation with the hidden gauge vector behaves like a Rademacher sum.
Lemma 5 (Source labels do not choose a gauge). Assume equal source size and the Rademacher gauge prior. Let be any possibly randomized section measurable with respect to source labels and independent randomness, but independent of the unknown gauges . Then, for every ,
Thus source labels alone cannot reliably create aligned mass unless .
Thus labels group samples but do not choose their orientation.
Relative-gauge metadata creates extensive aligned mass
A natural calibration channel is a noisy relative-sign graph over sources. For each , let independently, where is the Bernoulli distribution with success probability , with and . On observed edges , we observe
Set , and define the signed adjacency matrix
Then
Let be a unit-norm top eigenvector of . The edge noises are independent across unordered source pairs and independent of the graph indicators, source variables, and downstream feature noise.
Lemma 6 (Spectral synchronization). There exist universal constants such that if
then with probability at least ,
Assign each sample in source the weight . Then
So synchronization lifts the effective sample size from , which is the best source-label sections can do, to whenever the synchronized section has constant overlap with the hidden gauges.
Corollary 7 (Calibrated recovery scale). For fixed recovery tolerance and failure probability , if synchronization yields and , then calibrated top- recovery succeeds once
up to universal constants. Under the two-view parametrization , this is equivalently
Metadata budget. The calibrated and uncalibrated experiments use the same downstream feature sample size , but calibration also uses a task-independent metadata resource. In the relative-sign graph model the expected number of calibration edges is
Thus the elementary spectral guarantee above uses
calibration edges to reach the overlap level certified by Lemma 6, up to universal constants, when is chosen at the stated threshold. To certify , the same lemma gives the sufficient additional degree condition
together with the stated logarithmic degree requirement. This cost is separate from downstream feature samples and can be paid once and reused across tasks sharing the same source gauges. The separation therefore compares two metadata channels: source labels have alignment capacity , whereas relative-gauge metadata, at the stated calibration-edge budget, can produce realized aligned mass .
A separation in the micro-source regime
The calibration result above follows the sparse-mean scale. By contrast, Proposition 3 shows that source labels alone give a sparse PCA problem with
Its information scale is therefore
The following lower bound turns that heuristic into a support-recovery statement.
Theorem 8 (High-F1 lower bound for source labels). Assume and for a sufficiently large universal constant . In the source-Gaussian model, let be uniform on the sparse support packing constructed in Appendix § 9. Define
There exist universal constants such that if , then every estimator using source labels alone satisfies
In the weak-source regime , this condition becomes .
Remark 9 (Bayes prior and fixed gauges). The Rademacher prior on is used to obtain a hard average-case gauge ensemble. The lower bound also implies a worst-case fixed-gauge statement: if the success probability averaged over is at most $1/2\sigma^* for which the same estimator has success probability at most . Equivalently, the Bayes lower bound under the Rademacher gauge prior implies a minimax lower bound over fixed unknown gauges. We do not claim that every fixed gauge is equally hard; for example, if all sources happen to be already aligned, first-order pooling is possible. The lower bound concerns the absence of calibration in the random-gauge average-case and fixed-gauge minimax senses.
Together, Theorem 4 and Theorem 8 give an unconditional statistical separation. In the micro-source regime , there is a window where
In this window, calibrated recovery succeeds while source-labelled recovery is statistically insufficient for high-F1 support recovery.
A conditional computational separation
The statistical gap above does not rely on computational assumptions. A computational separation appears when the source-labelled experiment is interpreted through standard sparse-PCA hardness.
Conjecture 10 (Sparse PCA constant-advantage hardness, informal). Consider the testing problem
In the standard sparse-PCA hard regime, no polynomial-time algorithm distinguishes the two hypotheses with constant advantage throughout , up to logarithmic and standard polynomial-scaling qualifications. High-overlap recovery implies detection by sample splitting and a held-out variance test.
Corollary 11 (Metadata–computation separation). Assume Conjecture 10. If , , and polynomial-time synchronization of valid calibration metadata yields , then at
calibration enables polynomial-time high-overlap recovery, while source labels alone are statistically insufficient. Moreover, for larger uncalibrated sample size , source-labelled recovery remains computationally hard throughout the sparse-PCA window
up to constants and logarithmic factors.
Thus the calibrated and uncalibrated observations fall into different algorithmic regimes: sparse mean recovery after synchronization versus sparse PCA without calibration.
One calibration, many downstream tasks
Calibration metadata concerns source gauges, not the downstream sparse direction. It can therefore be reused across tasks sharing the same source gauges and calibration graph.
Remark 12 (Task-independence and reuse). Let be independent sparse directions, sampled independently of . If the metadata satisfies
then . Nevertheless, if the metadata induces a section with and sufficient aligned source count, then the same linearizes every task:
Thus calibration is reusable: it fixes a shared source convention without revealing any task-specific support.
The resulting algorithm, GSL (Gauge–Synchronize–Linearize), is immediate from the theory. It synchronizes the valid relative-gauge graph to obtain source weights . For sample , let denote its source label, and set on the downstream estimation samples. No downstream samples are used to estimate these calibration weights. The algorithm then forms
and returns the top- coordinates of . In the two-view model, where are the two views and is the label, , and the recovered direction is used in the bilinear classifier . Pseudocode is deferred to Appendix § 7.
The exact separation assumes sign gauges, equal-magnitude sparse signals, Gaussian source amplitudes, isotropic Gaussian noise, and task-independent calibration. Appendix § 10 gives a non-isotropic Gaussian extension and clarifies the limits of the sparse-PCA hardness comparison.
Experiments
Experiments check finite-sample predictions rather than prove the experiment equivalence. Synthetic studies test the predicted recovery scales; controlled real-feature studies test sign cancellation and restoration. Separately acquired physical calibration and reuse are reported in Appendix § 11.§ 5. Unless stated otherwise, . Synthetic recovery is measured by and support F1, with corresponding success threshold 0.9. The plotted strength is . Pseudocode is in Appendix § 7; experimental records and standard-error units are in Appendix § 11.
Synthetic mechanism tests
Figure 1(a) tests the sparse-mean scaling predicted by Theorem 4: oracle calibration and spectral synchronization nearly coincide across problem sizes. Figure 1(b) compares calibration quality: informative records shift the transition, random and shuffled controls remain near chance, and partial calibration gives an intermediate transition. These controls need not violate task independence. Figure 1(c) fixes the calibrated first-order scale and varies . Calibrated recovery remains strong for weak sources, whereas source-label PCA improves as the second-order quantity increases.
Additional mechanism checks
Aligned source count. Theory separates signal from amplitude concentration. We fix and vary . Table 1 confirms that aligned mass alone is not enough when concentrated on too few sources: as grows, stabilizes near the half-normal scale and recovery improves. This matches the role of in Theorem 4: it controls whether source amplitudes concentrate around their mean rather than being dominated by a few lucky or unlucky sources.
| Corr | Succ90 | ||
| 1 | |||
| 4 | |||
| 16 | |||
| 32 | |||
| 128 |
Table 1. At fixed aligned mass, increasing aligned source count stabilizes amplitude and improves recovery. Entries are mean SE over 100 seeds.
Reusable calibration. We test Remark 12 by synchronizing one relative-gauge graph before sampling any downstream task and reusing it for independent sparse tasks. Table 2 shows that the reused section matches oracle calibration; the downstream directions are distinct, so the gain is not due to task similarity. This illustrates the main distinction between calibration and supervision: the graph fixes a shared source convention once, and that convention can then be applied to new tasks without revealing their supports.
| Method | Corr | F1 |
| Oracle calibration | ||
| Same sync reused | ||
| No calibration | ||
| Random metadata reused | ||
| Shuffled metadata reused | ||
| Source diagonal PCA |
Table 2. One calibration graph reused across 32 independent sparse tasks. Entries are mean task-level SE conditional on the graph.
Controlled real features. Finally, we use real features and labels while synthetically imposing source gauges and calibration logs. Table 3 shows valid calibration restores clean no-gauge performance on digits and breast-cancer data, while random or shuffled metadata does not. These are controlled mechanism checks: gauges and calibration logs are imposed synthetically, while the feature distributions and labels are real. Source-label task alignment uses downstream labels and is therefore a supervised control.
| Method | Digits Acc. | Digits | Breast Acc. | Breast |
| Clean no gauge | ||||
| Oracle calibration | ||||
| Relative sync calibration | ||||
| Noisy abs. calibration, 10% flips | ||||
| No calibration | ||||
| Random metadata | ||||
| Shuffled metadata | ||||
| Source-label task alignment |
Table 3. Controlled real-feature experiments. Features and labels are real; source gauges and calibration metadata are imposed. Entries are mean SE over 30 seeds.
Related work
Our work connects sparse PCA, synchronization, orbit recovery, and metadata-aware robust learning. Sparse PCA is a standard model for high-dimensional statistical and computational gaps [2, 8, 11, 12]; here it arises instead as the exact experiment induced by source-labelled uncalibrated observations. Our calibration step is closely related to synchronization over groups such as , , and [5, 17], with additional connections to community-detection and graph-based recovery phenomena [1]; here synchronization is used to turn task-independent gauge metadata into restored first-order signal. The model is also related to orbit recovery and multi-reference alignment, where latent group actions suppress first-order information [7], and to related high-order-moment phenomena in spiked models [15]. More broadly, the paper complements work on invariant learning, robustness under group or domain shift, and metadata-aware pipelines [3, 14, 16, 19, 20, 25]. Related phenomena arise in attribute recognition, where label co-occurrence bias and heterogeneous feature distributions induce spurious correlations that downstream methods must remove or balance [21, 22, 23, 24]. Our focus is narrower: metadata that identifies nuisance coordinate conventions rather than providing downstream supervision.
Conclusion
This paper isolates a basic distinction in multi-source learning. Source labels tell us which samples belong together, but they do not tell us how different sources orient the same latent signal. In our stylized sparse model, that distinction changes the problem completely. Without calibration, the experiment is sparse PCA; with valid relative-gauge metadata, synchronization restores a first-order sparse mean signal. The aligned-mass viewpoint makes this change measurable and leads to clear statistical and computational separations in weak-source regimes. More broadly, the paper highlights a simple principle: metadata can matter not because it labels the task, but because it fixes the coordinate system in which the task becomes visible. This suggests treating calibration as a distinct resource from supervision or grouping information. Extending this viewpoint to richer transformation groups and approximately valid metadata is a natural next step.
Acknowledgments and Disclosure of Funding
We declare that this work received no external funding and that the authors have no competing interests to disclose.
Deferred algorithm
GSL pseudocode
Algorithm 1 GSL: Gauge–Synchronize–Linearize
Require: Linearization samples , where is the source label of sample ; valid calibration metadata ; sparsity
Construct a signed source calibration graph from
Run synchronization to obtain source weights
Set for each downstream sample
Compute
Top- threshold to obtain
return
In the two-view setting of Appendix § 8, the linearization samples are obtained from , where and are the two views and is the label. After recovering , prediction uses .
Two-view origin
The source-Gaussian statistic arises from a simple two-view classification model. Let , and let
with all variables independent of and . Define
Then
where
The orthogonal combination
is pure Gaussian noise. Recovering in the main text therefore corresponds to recovering the discriminative sparse direction used by the bilinear classifier.
Proofs
Proof of Proposition 2
Fix . For a sample-level weighting , let
Then
By Cauchy’s inequality within each source,
and hence
If , the expected aligned mass is zero, so the desired upper bound is immediate. We therefore assume in the following Rayleigh quotient calculation. Therefore, conditional on ,
The upper bound is attained by taking to be a top eigenvector of and spreading the weight uniformly within each source, for instance . This gives
This proves the claim.
Bayes-to-minimax consequence for fixed gauges
Let denote the success event, for example
Suppose that for an arbitrary estimator ,
where the outer expectation is over the uniform Rademacher prior on . Then there exists at least one fixed gauge vector such that
Equivalently,
Since this holds for every estimator, the Bayes lower bound under the Rademacher gauge prior implies the corresponding minimax lower bound over fixed unknown gauges.
Unequal source sizes
For completeness we record the source-label aligned-mass bound for unequal source sizes. Let source contain samples and let . If is measurable with respect to source labels and independent of the unknown gauges, then conditional on ,
Hoeffding’s inequality gives
Moreover,
so
Taking
yields
Thus source-label-measurable first-order sections cannot produce extensive aligned mass unless one source already contains a constant fraction of the samples.
Proof of Proposition 3
Fix a source . Since ,
Because ,
Now decompose the samples within each source into the all-ones direction and its orthogonal complement. The all-ones direction is exactly . Every orthogonal contrast cancels the common source signal and is pure Gaussian noise independent of . This proves equivalence to sparse PCA samples plus -independent Gaussian residuals.
A standard sparse-mean recovery lemma
Lemma 13 (Top- sparse mean recovery). Let
Let , equivalently top- threshold and use observed signs. There is a universal constant such that, for , if
then
Proof. Let
Since
a union bound over Gaussian tails gives, with probability at least ,
On this event, optimality of implies
while also
Hence
The stated condition ensures .
Weighted half-normal concentration
Lemma 14 (Weighted half-normal concentration). Let , set , and let be fixed real weights. Put
There exist universal constants such that
for all . Consequently, if , then with probability at least ,
Proof. The case is trivial. Otherwise, the function
is -Lipschitz in Euclidean norm, and . Gaussian concentration for Lipschitz functions [9] therefore gives
Taking and using proves the second claim whenever .
Proof of Theorem 4
Conditional on ,
Because valid calibration makes independent of the downstream feature noise conditional on the source signs and design,
Thus has the sparse-mean form.
In the constant-amplitude model,
For source-Gaussian amplitudes, write and . Then
where and . By Lemma 14,
with probability at least whenever . Therefore
The factor in is absorbed into . Using failure probability for the half-normal event and for Lemma 13, and adjusting constants, completes the proof.
Proof of Lemma 5
Conditional on , define . Then
This is a Rademacher sum. Hoeffding’s inequality gives
By Cauchy,
Taking yields the claim.
Proof of Lemma 6
Conditional on , put
The upper-triangular entries of are independent and centered. Moreover,
We use the sharp bounded-entry spectral-norm estimate for symmetric matrices with independent centered upper-triangular entries [4]. In the present notation, its bounded-entry consequence, with symmetrization if the entry distributions are not symmetric, gives
Here is the operator norm. Under , this yields
with probability at least . This sharp bound, rather than a direct application of standard matrix Bernstein, is what removes the multiplicative logarithmic factor.
The population matrix
has leading eigenvector , leading eigenvalue , and all remaining eigenvalues equal to . Its eigengap is therefore exactly . If exceeds a sufficiently large universal constant, then . The Davis–Kahan eigenvector perturbation bound [18] consequently gives
This proves the lemma without changing its stated degree conditions.
Proof of Theorem 8
Support packing. Write , and set . A greedy construction on all -subsets of , deleting after each selection every support obtained by replacing at most coordinates, gives supports satisfying
and
Here denotes symmetric difference. For completeness, let . The denominator is at most
Using and , we obtain
for a universal , for example . Choose sufficiently large that . Let be uniform on and set
From high F1 to index recovery. For any estimated support , define
If this is at least , then, writing ,
so
Choose the universal constant so that . Then , whereas distinct packing supports have distance greater than . Thus nearest-support decoding in symmetric-difference distance uniquely recovers . This argument does not require .
Information bound and Fano. By Proposition 3, the observations reduce to independent samples with covariance , together with independent residual noise whose distribution does not depend on . Let denote the aggregate-sample distribution under , and let . The Gaussian KL formula gives
Consequently,
Fano’s inequality [10] implies, for every index decoder ,
If with , the right-hand side is at most , since . The decoding implication above proves the high-F1 lower bound.
Finally, for ,
Hence
which gives the stated weak-source condition.
Proof of Remark 12
The mutual-information statement follows from conditional independence:
Therefore . For each task , conditional on the same metadata-induced section , Theorem 4 applies with , giving
Proof of Corollary 11
The calibrated claim follows from Theorem 4 with . Source-label statistical insufficiency follows from Theorem 8. For larger uncalibrated sample size , the compressed source experiment has
The stated window is exactly the sparse-PCA hardness window under Conjecture 10, up to constants and logarithmic factors. High-overlap recovery would imply detection by splitting independent sources into two groups, estimating from the source aggregates in one group, and testing the variance along using the independent aggregates in the other group. The split is across sources, not across repeated observations sharing the same source amplitude.
Noise extension and scope
The first-order versus second-order distinction has a limited extension beyond isotropic noise. Suppose independently across samples
where the source covariances are fixed and do not depend on . Retain the source-amplitude model and valid calibration assumptions from the main text. Conditional on the calibration weights and source variables,
Thus calibration still restores a first-order mean signal, although its noise covariance is no longer the identity.
A sufficient support-recovery bound. Let
For , a sufficient condition for exact support recovery by top- thresholding is
To see this, coordinatewise Gaussian tails and a union bound give
No independence between feature coordinates is needed for this step. Lemma 14, with failure probability , also gives
under the stated aligned-source-count condition. Since every nonzero entry of has magnitude , a sufficiently large universal constant in the signal condition makes every signal coordinate more than twice the displayed noise bound. The top- support is therefore correct with probability at least .
If
then is a convex combination of the source covariances, so
Here denotes the operator norm.
The uncalibrated experiment. Without calibration, the source aggregate has distribution
The remaining orthogonal within-source contrasts are independent background observations with distribution , and are independent of . Their distributions do not depend on , but they can help estimate unknown source covariances. The uncalibrated experiment is therefore a generalized, possibly heteroscedastic spiked-covariance experiment with background-only observations, not standard identity-background sparse PCA. Its dependence on remains second-order, but sharp thresholds and computational hardness for arbitrary unknown are not established here.
Calibration validity concerns the information channel producing ; noise covariance concerns the downstream observations after is fixed. These are separate assumptions. This extension does not establish robustness to task-dependent metadata, approximate sparsity, or richer source transformations.
Additional experimental details
Simulation records use deterministic base seed
base_seed = 12345
and per-cell seeds generated by
stable_seed(base_seed, experiment, configuration, seed, method)
Unless stated otherwise, repeated-run cells use independent randomized runs. The aligned-source-count experiment uses . The controlled real-feature experiments use seeds 12345, . . . , 12374.
Role of the experiments. The methods are calibration-information controls rather than claims of general algorithmic superiority: oracle signs represent perfect calibration, relative synchronization represents noisy calibration, source labels alone represent the uncalibrated experiment, and random or shuffled records represent uninformative or mismatched metadata. Synthetic experiments test finite-sample scaling predictions. The scikit-learn studies test cancellation and restoration on real features with imposed source signs. The physical study tests separately acquired calibration and task reuse. These experiments do not prove the statistical equivalence in Proposition 3.
Meaning and unit of SE. SE means standard error, not squared error. If is the aggregated metric from randomized run , then
where sd is the sample standard deviation. For digits, each first averages test accuracy across the five digit-pair tasks; SE is then computed across the 30 run-level averages. For breast cancer, it is computed across 30 randomized-run test accuracies.
The reuse experiment instead fixes one calibration graph and reports mean and SE across 32 independently sampled downstream tasks, conditional on that shared graph. These are not 30-seed estimates of uncertainty across calibration graphs. For the physical study, SE is the sample standard deviation of source-cluster bootstrap replicate estimates, as described in Appendix § 11.§ 5; it is not divided by the square root of the number of bootstrap replicates. These uncertainty units are distinct.
Table 4 and Table 5 provide the numerical records underlying Figure 1(a) and Figure 1(c), respectively, including additional phase-curve configurations.

Figure 1. Mechanism tests. Calibration follows sparse-mean scaling; uninformative or mismatched metadata fails to restore alignment. The calibrated/source-label gap is largest for weak sources. Gray dashed lines mark unit strength.
| Oracle Corr | Oracle Succ90 | Sync Corr | Sync Succ90 | |||
| 0.250 | 242 | 0.095 0.010 | 0.000 | 0.098 0.013 | 0.000 | |
| 0.500 | 484 | 0.192 0.012 | 0.000 | 0.220 0.018 | 0.000 | |
| 1.000 | 966 | 0.377 0.021 | 0.000 | 0.378 0.016 | 0.000 | |
| 2.000 | 1930 | 0.680 0.016 | 0.033 | 0.668 0.014 | 0.000 | |
| 4.000 | 3858 | 0.913 0.008 | 0.833 | 0.900 0.008 | 0.700 | |
| 8.000 | 7716 | 0.997 0.002 | 1.000 | 0.995 0.003 | 1.000 | |
| 0.250 | 626 | 0.090 0.006 | 0.000 | 0.095 0.009 | 0.000 | |
| 0.500 | 1252 | 0.195 0.010 | 0.000 | 0.179 0.010 | 0.000 | |
| 1.000 | 2502 | 0.369 0.011 | 0.000 | 0.386 0.010 | 0.000 | |
| 2.000 | 5002 | 0.661 0.009 | 0.000 | 0.657 0.010 | 0.000 | |
| 4.000 | 10004 | 0.908 0.007 | 0.667 | 0.908 0.005 | 0.700 | |
| 8.000 | 20006 | 0.995 0.002 | 1.000 | 0.994 0.002 | 1.000 | |
| 0.250 | 1102 | 0.069 0.005 | 0.000 | 0.078 0.006 | 0.000 | |
| 0.500 | 2202 | 0.167 0.008 | 0.000 | 0.165 0.008 | 0.000 | |
| 1.000 | 4404 | 0.362 0.008 | 0.000 | 0.372 0.008 | 0.000 | |
| 2.000 | 8806 | 0.649 0.008 | 0.000 | 0.648 0.008 | 0.000 | |
| 4.000 | 17610 | 0.913 0.005 | 0.833 | 0.915 0.004 | 0.800 | |
| 8.000 | 35220 | 0.996 0.001 | 1.000 | 0.997 0.001 | 1.000 |
Table 4. Source-Gaussian calibrated phase transition. Corr entries are mean SE, and Succ90 is reported in adjacent columns. Each entry uses 30 seeds.
| Oracle | Sync | Diag+PCA | Diag F1 | Sync Succ90 | ||||
| 1 | 0.020 | 19649 | 0.080 | 0.955 0.004 | 0.962 0.007 | 0.031 0.009 | 0.038 0.009 | 1.000 |
| 2 | 0.040 | 9824 | 0.160 | 0.960 0.006 | 0.960 0.007 | 0.022 0.006 | 0.033 0.007 | 0.967 |
| 4 | 0.080 | 4912 | 0.320 | 0.962 0.006 | 0.963 0.006 | 0.032 0.007 | 0.050 0.009 | 1.000 |
| 8 | 0.160 | 2456 | 0.640 | 0.958 0.008 | 0.950 0.005 | 0.045 0.010 | 0.065 0.011 | 1.000 |
| 16 | 0.320 | 1228 | 1.280 | 0.950 0.007 | 0.953 0.007 | 0.084 0.012 | 0.110 0.012 | 0.967 |
| 32 | 0.640 | 614 | 2.560 | 0.957 0.008 | 0.952 0.007 | 0.194 0.024 | 0.165 0.013 | 0.967 |
| 64 | 1.280 | 307 | 5.120 | 0.963 0.007 | 0.943 0.006 | 0.454 0.024 | 0.283 0.016 | 0.967 |
| 128 | 2.560 | 154 | 10.273 | 0.940 0.007 | 0.947 0.008 | 0.610 0.018 | 0.422 0.018 | 0.933 |
| 256 | 5.120 | 77 | 20.547 | 0.950 0.007 | 0.943 0.011 | 0.774 0.017 | 0.635 0.025 | 0.900 |
| 512 | 10.240 | 38 | 40.560 | 0.925 0.019 | 0.928 0.015 | 0.856 0.019 | 0.767 0.029 | 0.767 |
| 1024 | 20.480 | 19 | 81.119 | 0.915 0.016 | 0.917 0.016 | 0.899 0.018 | 0.843 0.027 | 0.733 |
Table 5. Source-Gaussian micro-source separation. Entries are mean SE, except Sync Succ90, which reports the success rate. Each entry uses 30 seeds.
Compute environment
All experiments were run on the following hardware:
GPU: NVIDIA V100 (32GB)
CPU: Intel Xeon Gold 6248 (2.5GHz, 20 cores)
RAM: 256GB DDR4
Storage: NVMe SSD
The retained real-data log used Python 3.6.9, NumPy 1.19.5, and scikit-learn 0.24.2.
Source-Gaussian calibrated phase transition
Table 4 reports the full numerical records underlying Figure 1(a). Its purpose is twofold: first, to make the phase-transition plot fully auditable at the level of individual experimental cells; and second, to show that the calibrated transition is stable across several ambient dimensions and sparsity levels. The main pattern is the same as in the figure: once the nominal calibrated strength is large enough, spectral synchronization closely tracks oracle calibration, supporting the sparse-mean scaling predicted by the Corollary 7.
Micro-source separation
Table 5 gives the full numerical records for Figure 1(c), where the calibrated first-order scale is held fixed while the per-source strength varies. Here Diag+PCA is sparse PCA on . The table makes the separation especially transparent: in the weak-source regime, calibrated methods remain strong, whereas source-label baselines improve only when the second-order sparse-PCA quantity becomes sufficiently large. The explicit column is included to show how the empirical transition aligns with the information scale highlighted in the Theorem 8.
Controlled real-feature setup
Digits use the scikit-learn handwritten-digits dataset with 1797 images, 64-dimensional pixel features, and five binary digit-pair tasks: 0 vs 6, 1 vs 7, 2 vs 5, 3 vs 8, and 4 vs 9. Breast cancer predicts malignant versus benign using the scikit-learn breast-cancer dataset with 569 examples and 30 real-valued features. In both cases, examples are randomly partitioned into sources, source gauges and calibration logs are imposed synthetically, and labels and non-gauge feature processing are otherwise unchanged; calibration graphs use the same relative-sign model as in § 3.
Separately acquired physical calibration
The physical study tests whether source calibration can be acquired separately and reused across tasks. It is not a test of the exact Gaussian model or evidence for arbitrary multi-source transformations.
Calibration acquisition. We use 24 fixed device–mounting sources with physically induced axis-sign conventions, rather than signs imposed on stored features. Each source contributes 10 separately recorded reference-motion trials, giving 240 calibration trials in total. The time-normalized calibration traces are aggregated into one reference trace per source. Pairwise correlation signs produce a complete graph with
same/opposite records. No downstream labels, recordings, or task-derived features enter this graph. It is synchronized once, and the resulting source alignment is reused unchanged across four positive-versus-negative directed-motion classification tasks. Recorded mounting signs are used only for alignment evaluation and the oracle control, not for relative graph construction, synchronization, or ordinary downstream classifier training.
Downstream split. The downstream dataset contains
complete trials. Within every source–task–class combination, the 20 trials are split 12/4/4 into training/validation/test sets before window extraction, giving 2304/768/768 complete trials. Windows from the same trial never cross splits. No downstream observations are used to estimate the calibration weights.
Results and uncertainty. Table 6 reports macro balanced test accuracy across the four tasks. Relative calibration gives per-task balanced accuracies 0.928, 0.905, 0.892, and 0.883. Reported SE is the sample standard deviation of source-cluster bootstrap replicate estimates, resampling the 24 device–mounting sources rather than individual trials or windows.
| Method | Balanced test accuracy | |
| Oracle mounting polarity | 0.921 ± 0.004 | 1.000 ± 0.000 |
| Relative calibration + synchronization | 0.902 ± 0.006 | 0.916 ± 0.015 |
| No calibration | 0.558 ± 0.011 | 0.044 ± 0.010 |
| Shuffled metadata | 0.553 ± 0.012 | 0.040 ± 0.009 |
Table 6. Physical calibration reused across four tasks. Entries are estimates ± source-cluster bootstrap SE. Mounting signs are unavailable to the relative-calibration procedure.
The 276 graph records are derived from 240 calibration trials; they are not 276 independent physical acquisitions. Edges sharing reference traces can have dependent errors. Consequently, this experiment supports the acquisition-and-reuse mechanism, but does not verify the independent-edge graph law or exact satisfaction of the theoretical metadata-validity condition.
Limitations
The main analysis isolates a minimal source ambiguity: one sign bit can erase a first moment and leave a covariance spike. Its sharp separation uses exactly sparse equal-magnitude signals, independent Gaussian source amplitudes, isotropic Gaussian feature noise, and task-independent calibration. The non-isotropic Gaussian extension in Appendix § 10 gives observation formulas and a sufficient support-recovery bound, not sharp thresholds or computational hardness for arbitrary unknown source covariances. Rotations, permutations, nonlinear transformations, approximate sparsity, and task-dependent drift require additional arguments; imperfect alignment under richer transformations need not preserve coordinate sparsity. A practical heuristic for unequal marginal noise scales is to estimate coordinatewise variances from within-source centered residuals, combine them using normalized squared calibration weights, and standardize the calibrated aggregate before selection. This is variance normalization, not whitening or a general covariance-adaptive guarantee. It ignores off-diagonal geometry and can be inefficient for highly correlated features. Estimates can be unstable with few repeats; when , they require additional structure, pooling, or separate noise-only repeats. Full whitening is not a drop-in solution because covariance estimation is difficult in the micro-source regime and whitening can destroy coordinate sparsity. Task-independent missing or corrupted calibration can reduce alignment while remaining valid. Dependence on downstream features, labels, or amplitudes is a different issue: approximate validity is not analyzed. The synthetic studies test the formal model, the controlled real-feature studies impose source signs, and the physical study supplies one concrete acquisition-and-reuse setting. It does not establish applicability to general multi-source transformations or exact satisfaction of the theoretical channel assumptions. Finally, the computational separation remains conditional on sparse-PCA hardness.
Broader impact
This work studies the statistical value of calibration metadata in a stylized multi-source setting. The positive motivation is straightforward. In scientific sensing, distributed monitoring, medical measurement pipelines, and federated systems, source-specific conventions can hide a signal that is present in every source. Understanding when lightweight metadata can recover that signal may reduce data waste, improve robustness across institutions, and lower the need for expensive task-specific relabeling. At the same time, any method that uses source metadata deserves care. Metadata can encode operational details, institution identity, or deployment structure. In real systems, this may create privacy, governance, or fairness concerns if calibration channels reveal sensitive attributes or if metadata quality differs systematically across groups. A second risk is conceptual: invalid metadata can quietly act like supervision. The paper therefore explicitly separates reusable calibration from task-specific alignment.
The present work is theoretical and does not introduce a new surveillance or decision pipeline. Still, if similar methods are deployed in practice, we recommend: (i) auditing what metadata is collected and why, (ii) documenting whether the metadata is task-independent, (iii) checking whether calibration quality differs across subpopulations, and (iv) reporting failure modes when metadata is missing, corrupted, or strategically manipulated. Finally, the compute footprint of our experiments is modest by current machine learning standards. The method itself consists of spectral synchronization, weighted averaging, and thresholding, rather than large-scale model training.
NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Answer: [Yes]
Justification: In the abstract and introduction, we state the same claims that we prove and test in the paper: source-labelled uncalibrated data reduces to sparse PCA, calibration restores a sparse-mean signal, and this leads to statistical and conditional computational separations in the micro-source regime. We also state the scope of the paper through the sparse Gaussian hard-core model introduced in § 2.
Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discuss limitations explicitly in Appendix § 12. In particular, we note the stylized gauge model, the validity assumptions on metadata, the simplifying assumptions used in several results, and the controlled nature of the experiments.
Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: We state the model assumptions in § 2, and each theorem and lemma gives the assumptions needed for the result. Full proofs are provided in Appendix § 9, and we clearly mark the computational separation as conditional on Conjecture 10.
Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We describe the experimental pipeline in § 2, report the main settings and outcomes in § 4, and give detailed records in Appendix § 11. We also provide complete experimental reproduction code in the supplemental material.
Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide anonymized reproduction code and instructions in the supplemental material. The synthetic experiments are generated from the stated model, and the real-feature experiments use public scikit-learn datasets described in Appendix § 11.
Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer) necessary to understand the results?
Answer: [Yes]
Justification: We specify the algorithm in Algorithm 1 and describe the experimental settings, datasets, seeds, metrics, and task constructions in § 4 and Appendix § 11. Our method does not rely on iterative optimization, so there are no optimizer choices to report.
Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Throughout the experimental section and appendix, we report means together with standard errors or success rates, and we state the number of random seeds used in each experiment. Appendix § 11 also makes clear that the reported error bars are standard errors.
Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: We report the hardware and software environment in Appendix § 11, including GPU, CPU, RAM, storage, and library versions.
Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?
Answer: [Yes]
Justification: To the best of our knowledge, yes. This work is primarily theoretical, uses synthetic data and public datasets, does not involve human subjects, and preserves anonymity in the submission.
Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss broader impacts in Appendix § 13. There we describe possible benefits in multi-source sensing and measurement settings, as well as risks related to privacy, governance, fairness, and misuse of metadata.
Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pre-trained language models, image generators, or scraped datasets)?
Answer: [N/A]
Justification: Our release does not include high-risk generative models, scraped datasets, or similar assets that would require special safeguards of that kind. The released asset is reproduction code for the theory-driven experiments in the paper.
Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cite the public datasets and software packages we use, including the scikit-learn datasets and package versions in Appendix § 11.
New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We provide complete experimental reproduction code together with instructions in the supplemental material. We do not introduce a new dataset or pretrained model beyond this code release.
Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [N/A]
Justification: This paper does not involve crowdsourcing or research with human subjects.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [N/A]
Justification: This paper does not involve crowdsourcing or research with human subjects.
Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigor, or originality of the research, declaration is not required.
Answer: [N/A]
Justification: LLMs are not part of the core methods, experiments, or theoretical development in this research.
References
- [1]Emmanuel Abbe, Afonso S. Bandeira, and Georgina Hall. Exact recovery in the stochastic block model. IEEE Transactions on Information Theory, 62(1):471–487, 2016. doi: 10.1109/TIT.2015.2490670.DOI
- [2]Arash A. Amini and Martin J. Wainwright. High-dimensional analysis of semidefinite relaxations for sparse principal components. The Annals of Statistics, 37(5B):2877–2921, 2009. doi: 10.1214/08-AOS664.DOI
- [3]Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:1907.02893, 2019. URL https://arxiv.org/abs/1907.02893.
- [4]Afonso S. Bandeira and Ramon van Handel. Sharp nonasymptotic bounds on the norm of random matrices with independent entries. The Annals of Probability, 44(4):2479–2506, 2016. doi: 10.1214/15-AOP1025.DOI
- [5]Afonso S. Bandeira, Amit Singer, and Daniel A. Spielman. A cheeger inequality for the graph connection laplacian. SIAM Journal on Matrix Analysis and Applications, 34(4):1611–1630, 2013. doi: 10.1137/120875338.DOI
- [6]Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine Learning, 79(1–2):151–175, 2010. doi: 10.1007/s10994-009-5152-4.
- [7]Tamir Bendory, Nicolas Boumal, Chao Ma, Zhizhen Zhao, and Amit Singer. Bispectrum inversion with application to multireference alignment. IEEE Transactions on Signal Processing, 66(4):1037–1050, 2018. doi: 10.1109/TSP.2017.2775591.DOI
- [8]Quentin Berthet and Philippe Rigollet. Complexity theoretic lower bounds for sparse principal component detection. In Proceedings of the 26th Annual Conference on Learning Theory, volume 30 of Proceedings of Machine Learning Research, pages 1046–1066, 2013.
- [9]Stéphane Boucheron, Gábor Lugosi, and Pascal Massart. Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford University Press, 2013. ISBN 978-0-19-953525-5. doi: 10.1093/acprof:oso/9780199535255.001.0001.DOI
- [10]Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. Wiley-Interscience, 2 edition, 2006. ISBN 978-0-471-24195-9. doi: 10.1002/047174882X.DOI
- [11]Yash Deshpande and Andrea Montanari. Sparse PCA via covariance thresholding. In Advances in Neural Information Processing Systems 27, pages 334–342, 2014. URL https://proceedings.neurips.cc/paper/2014/hash/07a45842fcab1f6116c50549a437c254-Abstract.html.arxiv.org/abs/1311.5179
- [12]Iain M. Johnstone and Arthur Yu Lu. On consistency and sparsity for principal components analysis in high dimensions. Journal of the American Statistical Association, 104(486):682–693, 2009. doi: 10.1198/jasa.2009.0121.DOI
- [13]Peter Kairouz, H. Brendan McMahan, et al. Advances and open problems in federated learning. Foundations and Trends in Machine Learning, 14(1–2):1–210, 2021. doi: 10.1561/2200000083.DOI
- [14]Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness, Wei Guo, Berton Earnshaw, Imran Haque, Sara M. Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. WILDS: A benchmark of in-the-wild distribution shifts. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 5637–5664, 2021.arxiv.org/abs/2012.07421
- [15]Amelia Perry, Alexander S. Wein, and Afonso S. Bandeira. Statistical limits of spiked tensor models. Annales de l’Institut Henri Poincaré, Probabilités et Statistiques, 56(1):230–264, 2020. doi: 10.1214/19-AIHP960.DOI
- [16]Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=ryxGuJrFvS.
- [17]Amit Singer. Angular synchronization by eigenvectors and semidefinite programming. Applied and Computational Harmonic Analysis, 30(1):20–36, 2011. doi: 10.1016/j.jacha.2010.02.001.
- [18]Yi Yu, Tengyao Wang, and Richard J. Samworth. A useful variant of the Davis–Kahan theorem for statisticians. Biometrika, 102(2):315–323, 2015. doi: 10.1093/biomet/asv008.
- [19]Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4396–4415, 2023. doi: 10.1109/TPAMI.2022.3195549.DOI
- [20]Yibo Zhou. Rethinking reconstruction autoencoder-based out-of-distribution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7379–7387, 2022. doi: 10.1109/CVPR52688.2022.00723. URL https://doi.org/10.1109/CVPR52688.2022.00723.
- [21]Yibo Zhou, Hai-Miao Hu, Jinzuo Yu, Zhenbo Xu, Weiqing Lu, and Yuran Cao. A solution to co-occurrence bias: Attributes disentanglement via mutual information minimization for pedestrian attribute recognition. In Edith Elkind, editor, Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23, pages 1831–1839. International Joint Conferences on Artificial Intelligence Organization, 8 2023. doi: 10.24963/ijcai.2023/203. URL https://doi.org/10.24963/ijcai.2023/203. Main Track.
- [22]Yibo Zhou, Hai-Miao Hu, Yirong Xiang, Xiaokang Zhang, and Haotian Wu. Pedestrian attribute recognition as label-balanced multi-label learning. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 61964–61978. PMLR, 21–27 Jul 2024. URL https://proceedings.mlr.press/v235/zhou24j.html.arxiv.org/abs/2405.04858
- [23]Yibo Zhou, Hai-Miao Hu, Jinzuo Yu, Haotian Wu, Shiliang Pu, and Hanzi Wang. A solution to co-occurrence bias in pedestrian attribute recognition: Theory, algorithms, and improvements. International Journal of Computer Vision, 133(7):4712–4726, 2025. doi: 10.1007/s11263-025-02405-7. URL https://doi.org/10.1007/s11263-025-02405-7.
- [24]Yibo Zhou, Bo Li, Hai-Miao Hu, Xiaokang Zhang, Dongping Zhang, and Hanzi Wang. Heterogeneous feature re-sampling for balanced pedestrian attribute recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(4):2706–2722, 2025. doi: 10.1109/TPAMI.2025.3526930. URL https://doi.org/10.1109/TPAMI.2025.3526930.
- [25]Yibo Zhou, Bo Li, Hai-Miao Hu, Hanzi Wang, Xiaokang Zhang, and Ruifan Zhang. Is spurious correlation removal always learnable?, 2026. URL https://arxiv.org/abs/2606.12930.