Introduction
A classifier can predict observed labels accurately while the data weakly distinguish a specified task effect from admissible nuisance variation. This ambiguity matters for classification when statistically similar alternatives prescribe different labels. Autonomous bird monitoring provides a practical motivation: species classifiers operate under variable illumination and occlusion [1 ] . We ask when it creates a learning obstacle, and how to audit the remaining task information on predefined and learned populations. Group-robust learning and error-based reweighting emphasize subpopulation performance [2 , 3 ] ; our objective is to establish which information claims the data support and assess their relevance through controlled nuisance interventions (Figure 1 ).
Figure 1. Background sensitivity despite high control accuracy. (A,C) A matched Waterbirds image pair preserves the extracted bird foreground and label while changing background type. For the selected waterbird/water group, mean accuracy across five training seeds falls from 93.2% with same-type backgrounds to 40.6% with opposite-type backgrounds, a drop of 52.6 percentage points. (B) Schematic score geometry: projection removes the nuisance-aligned component of the task score, leaving a small residual.
Efficient-score projection removes task variation that nuisance scores can reproduce [4 , 5 ] . The Fisher-glass framework of Mansouri and Arai [6 ] connects this geometry to inverse-information tails and a population certificate. We develop its classification consequence: decision separation turns limited distinguishability into excess-risk bounds. A worked regression family shows that constant raw information at a reference parameter can conceal an effective sample size set by the mass near the decision boundary. Statistical corrections then determine which population information statements survive estimation error, regularization, and adaptive selection.
On Waterbirds, a frozen feature-based rule selects a group with 92.0% source accuracy and a 52.6-point controlled drop between same-type and opposite-type background replacements. The information, accuracy, and negative log-likelihood (NLL) selectors identify the same eligible group, whose controlled accuracy drop quantifies background sensitivity. Within that group, nuisance-derived coordinates improve species-held-out prediction of intervention failures beyond a specified confidence baseline. A separate search discovers an accurate population with low predictive information; its sensitivity to interventions remains untested.
Our contributions are:
A classification consequence. We derive local excess-risk bounds with explicit decision separation and radius, and exhibit a classification family with matching minimax rates and an interpretable loss of effective sample size.
Population information audits. We account for estimation error, ridge bias, membership normalization, and support correction, and evaluate frozen learned slices independently. The audit distinguishes supported information floors from uncertain or adaptively selected estimates.
Behavioral evaluation. Controlled interventions and confidence-baseline comparisons test the relevance of nuisance-derived measurements. Training and acquisition evaluate whether information gains correspond to predictive gains, which need not occur together.
Task–Nuisance Geometry and Information Collapse
Let e e e index an observed group or latent subpopulation, and let Π \Pi Π be the declared distribution over these environments. The audit fixes the task contrast, admissible nuisance variations, and common task-coordinate normalization. Information describes the task distinction remaining after nuisance adjustment.
Task Information After Nuisance Profiling
Consider a local model P θ , μ , e P_{\theta,\mu,e} P θ , μ , e with task parameter θ \theta θ and nuisance parameter μ \mu μ . A likelihood score measures the local change in log likelihood as a parameter varies. At P e = P 0 , 0 , e P_e = P_{0,0,e} P e = P 0 , 0 , e , assume differentiability in quadratic mean, with centered, square-integrable task-score vector s T , e s_{T,e} s T , e and nuisance-score vector s N , e s_{N,e} s N , e . Let T N , e \mathcal{T}_{N,e} T N , e be the closed span of admissible nuisance scores in L 2 ( P e ) L^2(P_e) L 2 ( P e ) , with inner product ⟨ f , g ⟩ e = E P e [ f g ] \langle f,g\rangle_e=\mathbb{E}_{P_e}[fg] ⟨ f , g ⟩ e = E P e [ f g ] , and let Π T N , e \Pi_{TN,e} Π T N , e denote orthogonal projection onto that span. The efficient task score and its information are
s T , e e f f = s T , e − Π T N , e s T , e , s^{\mathrm{eff}}_{T,e}=s_{T,e}-\Pi_{TN,e}s_{T,e}, s T , e eff = s T , e − Π T N , e s T , e ,
F e : = F T ∣ N , e e f f = E P e [ s T , e e f f ( s T , e e f f ) ⊤ ] = J T T , e − J T N , e J N N , e − 1 J N T , e . (1) F_e:=F^{\mathrm{eff}}_{T|N,e}=\mathbb{E}_{P_e}\left[s^{\mathrm{eff}}_{T,e}(s^{\mathrm{eff}}_{T,e})^\top\right]=J_{TT,e}-J_{TN,e}J_{NN,e}^{-1}J_{NT,e}. \tag*{(1)} F e := F T ∣ N , e eff = E P e [ s T , e eff ( s T , e eff ) ⊤ ] = J T T , e − J T N , e J N N , e − 1 J N T , e . (1)
The projection acts componentwise. The Schur expression applies when finite nuisance scores span T N , e \mathcal{T}_{N,e} T N , e and J N N , e ≻ 0 J_{NN,e}\succ0 J N N , e ≻ 0 , with J A B , e = E P e [ s A , e s B , e ⊤ ] J_{AB,e}=\mathbb{E}_{P_e}[s_{A,e}s_{B,e}^{\top}] J A B , e = E P e [ s A , e s B , e ⊤ ] [4 , 5 , 7 ] . Write λ e = λ min ( F e ) \lambda_e=\lambda_{\min}(F_e) λ e = λ m i n ( F e ) for the weakest task information. A zero eigenvalue identifies a task direction contained in the nuisance span.
For scalar task and nuisance scores with positive raw information, their Fisher correlation ρ e = J T N , e / J T T , e J N N , e \rho_e=J_{TN,e}/\sqrt{J_{TT,e}J_{NN,e}} ρ e = J T N , e / J T T , e J N N , e gives
F e = J T T , e ( 1 − ρ e 2 ) , F e J T T , e = 1 − ρ e 2 . (2) F_e=J_{TT,e}(1-\rho_e^2),\qquad\frac{F_e}{J_{TT,e}}=1-\rho_e^2. \tag*{(2)} F e = J T T , e ( 1 − ρ e 2 ) , J T T , e F e = 1 − ρ e 2 . (2)
Absolute information measures remaining curvature in the declared coordinates. The fraction measures the proportion retained, lies in [ 0 , 1 ] [0,1] [ 0 , 1 ] , and is invariant to nonzero scalar rescaling of either score. It compares alignment across slices with different raw information; the prospective selection statistic uses a ridge-regularized feature–Schur analogue with fixed coordinate scales. The Fisher-Glass certificate summarizes absolute information across environments.
Task–Nuisance Alignment and Loss–Information Separation
Let b e , r e b_e,r_e b e , r e be centered, orthogonal, unit-norm scores, with nuisance span span { b e } \operatorname{span}\{b_e\} span { b e } . For δ e ≥ 0 \delta_e\ge0 δ e ≥ 0 , define the task score a e = b e + δ e r e a_e=b_e+\delta_er_e a e = b e + δ e r e . Then
a e = b e + δ e r e , J T T , e = 1 + δ e 2 , F e = δ e 2 , F e J T T , e = δ e 2 1 + δ e 2 . (3) a_e=b_e+\delta_er_e,\qquad J_{TT,e}=1+\delta_e^2,\qquad F_e=\delta_e^2,\qquad\frac{F_e}{J_{TT,e}}=\frac{\delta_e^2}{1+\delta_e^2}. \tag*{(3)} a e = b e + δ e r e , J T T , e = 1 + δ e 2 , F e = δ e 2 , J T T , e F e = 1 + δ e 2 δ e 2 . (3)
Projection removes b e b_e b e , leaving δ e r e \delta_er_e δ e r e ; raw information stays at least one as efficient information vanishes. § 5.2 illustrates low observed loss alongside the corresponding feature-moment geometry. A classification cost additionally requires different optimal decisions, which the next section makes explicit. The inherited tail mechanism is illustrated in Appendix Figure A.1.
Classification Difficulty and Population Information
We study the statistical consequences of nuisance projection at two scales. Within an environment, local classification bounds and a worked family connect weak task information to learning difficulty under explicit distinguishability and decision-separation conditions. Across environments, geometry and probability mass near Fisher-null sets govern inverse-information tails, motivating a certificate of population information coverage.
Efficient Information and Local Classification Difficulty
Parameter ambiguity need not imply classification difficulty: probabilities 0.90 and 0.95 prescribe the same label. A lower bound requires statistically similar alternatives whose optimal decisions disagree on enough probability mass. Define classification risk R P ( g ) = P { g ( X ) ≠ Y } R_P(g)=P\{g(X)\ne Y\} R P ( g ) = P { g ( X ) = Y } , Bayes risk R P ⋆ = inf g R P ( g ) R_P^\star=\inf_g R_P(g) R P ⋆ = inf g R P ( g ) , and excess risk ε P ( g ) = R P ( g ) − R P ⋆ \varepsilon_P(g)=R_P(g)-R_P^\star ε P ( g ) = R P ( g ) − R P ⋆ . For a fixed local family, let R e ( n ) \mathfrak{R}_e(n) R e ( n ) denote minimax expected excess classification risk based on n n n independent observations from environment e e e . Write KL ( P ∥ Q ) \operatorname{KL}(P\Vert Q) KL ( P ∥ Q ) for the Kullback–Leibler divergence.
Theorem 1 (Local classification difficulty). Fix λ e > 0 \lambda_e>0 λ e > 0 and a unit weakest efficient-information direction u e u_e u e . For every 0 < t ≤ t 0 , e 0<t\le t_{0,e} 0 < t ≤ t 0 , e , suppose the local family contains P e , + t , P e , − t P_{e,+t},P_{e,-t} P e , + t , P e , − t with task coordinates ± t u e \pm tu_e ± t u e and nuisance coordinates allowed to vary. Assume, for constants K e , c e , t 0 , e > 0 K_e,c_e,t_{0,e}>0 K e , c e , t 0 , e > 0 and q > 0 q>0 q > 0 ,
KL ( P e , + t ∥ P e , − t ) ≤ K e t 2 λ e , inf g { ε P e , + t ( g ) + ε P e , − t ( g ) } ≥ c e t q . (4) \operatorname{KL}(P_{e,+t}\Vert P_{e,-t})\le K_et^2\lambda_e,\qquad\inf_g\{\varepsilon_{P_{e,+t}}(g)+\varepsilon_{P_{e,-t}}(g)\}\ge c_et^q. \tag*{(4)} KL ( P e , + t ∥ P e , − t ) ≤ K e t 2 λ e , g inf { ε P e , + t ( g ) + ε P e , − t ( g )} ≥ c e t q . (4) Then, for a universal constant c 0 > 0 c_0>0 c 0 > 0 ,
R e ( n ) ≥ c 0 c e min { t 0 , e q , ( n K e λ e ) − q / 2 } . (5) \mathfrak{R}_e(n)\ge c_0c_e\min\left\{t_{0,e}^q,(nK_e\lambda_e)^{-q/2}\right\}. \tag*{(5)} R e ( n ) ≥ c 0 c e min { t 0 , e q , ( n K e λ e ) − q /2 } . (5)
The KL bound limits distinguishability; decision separation supplies the classification cost. Both must hold over radius t 0 , e t_{0,e} t 0 , e . With quadratic separation, the interior rate is c e / ( n K e λ e ) c_e/(nK_e\lambda_e) c e / ( n K e λ e ) , capped by the local decision scale. Appendix § 11 .§ 3 proves the testing reduction; the next example calculates both conditions and a matching upper bound.
A Worked Classification Mechanism
Let X ∼ Unif [ − 1 , 1 ] X \sim\operatorname{Unif}[-1,1] X ∼ Unif [ − 1 , 1 ] , 0 < δ ≤ 1 / 2 0 < \delta\le1/2 0 < δ ≤ 1/2 , ∣ θ ∣ , ∣ μ ∣ ≤ δ / 8 |\theta|,|\mu| \le\delta/8 ∣ θ ∣ , ∣ μ ∣ ≤ δ /8 , and
η θ , μ ( x ) : = P θ , μ ( Y = 1 ∣ X = x ) = 1 2 + x 4 + θ + μ 1 { ∣ x ∣ > δ } . (6) \eta_{\theta,\mu}(x) := P_{\theta,\mu}(Y=1 \mid X=x) = \frac{1}{2}+\frac{x}{4}+\theta+\mu1\{|x|>\delta\}. \tag*{(6)} η θ , μ ( x ) := P θ , μ ( Y = 1 ∣ X = x ) = 2 1 + 4 x + θ + μ 1 { ∣ x ∣ > δ } . (6)
Here δ \delta δ is boundary-region half-width and probability mass. The nuisance reproduces a task shift outside that region; the Bayes rule is g θ ∗ ( x ) = 1 { x ≥ − 4 θ } g_\theta^*(x)=1\{x\ge-4\theta\} g θ ∗ ( x ) = 1 { x ≥ − 4 θ } throughout the family. At ( θ , μ ) = ( 0 , 0 ) (\theta,\mu)=(0,0) ( θ , μ ) = ( 0 , 0 ) ,
J T T = 4 log 3 , F δ = 4 log 2 + δ 2 − δ ≍ δ . (7) J_{TT}=4\log3,\qquad F_\delta=4\log\frac{2+\delta}{2-\delta}\asymp\delta. \tag*{(7)} J T T = 4 log 3 , F δ = 4 log 2 − δ 2 + δ ≍ δ . (7)
The alternatives ( t , − t ) (t,-t) ( t , − t ) and ( − t , t ) (-t,t) ( − t , t ) agree outside ∣ X ∣ ≤ δ |X|\le\delta ∣ X ∣ ≤ δ , have decision separation 4 t 2 4t^2 4 t 2 , and satisfy KL ( P + t ∥ P − t ) ≤ 5 F δ t 2 \operatorname{KL}(P_{+t}\Vert P_{-t})\le5F_\delta t^2 KL ( P + t ∥ P − t ) ≤ 5 F δ t 2 for 0 < t ≤ δ / 8 0<t\le\delta/8 0 < t ≤ δ /8 . Theorem 1 and a matching estimator give minimax risk over this parameter rectangle, uniformly in n ≥ 1 n\ge1 n ≥ 1 and δ \delta δ ,
R δ ( n ) ≍ min { δ 2 , ( n δ ) − 1 } . (8) \mathcal{R}_\delta(n)\asymp\min\{\delta^2,(n\delta)^{-1}\}. \tag*{(8)} R δ ( n ) ≍ min { δ 2 , ( n δ ) − 1 } . (8)
Approximately n δ n\delta n δ observations distinguish the task effect from nuisance variation where the decision boundary can move. The δ 2 \delta^2 δ 2 cap retains the shrinking local decision scale. This statistical nuisance preserves Bayes labels at fixed θ \theta θ ; it does not model the physical background-swap operation. Appendix § 11 .§ 4 gives the calculation and a finite-sample upper bound.
Information Tails Across Populations
Population aggregation describes how often weak task information occurs; classification still retains the radius and separation factors above. The inherited geometric law is:
Theorem 2 (Geometric tail law [6 ] ). Under the uniform regularity and localization assumptions in Appendix § 11 .§ 2 , let the Fisher-null set have codimension c c c , information vanish to order 2 m 2m 2 m , and nearby density have order ν > − c \nu>-c ν > − c . Then, for some C > 0 C>0 C > 0 ,
Pr ( λ ( z ) − 1 > x ) ∼ C x − β , β = c + ν 2 m . (9) \Pr(\lambda(z)^{-1}>x)\sim Cx^{-\beta},\qquad\beta=\frac{c+\nu}{2m}. \tag*{(9)} Pr ( λ ( z ) − 1 > x ) ∼ C x − β , β = 2 m c + ν . (9)
Quadratic collapse with positive density gives β = c / 2 \beta=c/2 β = c /2 ; the inverse mean is finite exactly when β > 1 \beta>1 β > 1 . Appendix § 11 .§ 2 contains the calculation and Figure A.1.
The Fisher-Glass Certificate
Upper-tail conditional value-at-risk, CVaR α \operatorname{CVaR}_\alpha CVaR α , averages the largest 1 − α 1-\alpha 1 − α fraction of a quantity, with fractional weighting at atoms [8 , 9 ] . We apply the Fisher-glass certificate [6 ] to classification environments e ∼ Π e\sim\Pi e ∼ Π , using the weakest task information λ e \lambda_e λ e and stabilization ε \varepsilon ε :
L e ( ε ) = ( λ e + ε ) − 1 , I α F G = [ CVaR α , e ∼ Π ( L e ( ε ) ) ] − 1 , 0 < α < 1. (10) L_e^{(\varepsilon)}=(\lambda_e+\varepsilon)^{-1},\qquad I_{\alpha}^{\mathrm{FG}}=\left[\operatorname{CVaR}_{\alpha,e\sim\Pi}\left(L_e^{(\varepsilon)}\right)\right]^{-1},\qquad0<\alpha<1. \tag*{(10)} L e ( ε ) = ( λ e + ε ) − 1 , I α FG = [ CVaR α , e ∼ Π ( L e ( ε ) ) ] − 1 , 0 < α < 1. (10)
A larger certificate means a smaller inverse-information tail, expressed in information units. Stabilization ε > 0 \varepsilon>0 ε > 0 caps inverse information at 1 / ε 1/\varepsilon 1/ ε ; it does not remove the underlying null geometry. Under Theorem 2’s power-law regime, the limit ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0 is positive for β > 1 \beta>1 β > 1 and vanishes for β ≤ 1 \beta\le1 β ≤ 1 .
Proposition 1 (Information coverage). For ρ > ε \rho>\varepsilon ρ > ε ,
I α F G ≥ ρ ⟹ Pr e ∼ Π ( λ e ≥ ρ − ε ) ≥ α . (11) I_{\alpha}^{\mathrm{FG}}\ge\rho\quad\Longrightarrow\quad\Pr_{e\sim\Pi}(\lambda_e\ge\rho-\varepsilon)\ge\alpha. \tag*{(11)} I α FG ≥ ρ ⟹ e ∼ Π Pr ( λ e ≥ ρ − ε ) ≥ α . (11)
The certificate establishes an absolute information floor on an α \alpha α -fraction of the declared environment distribution; classification consequences retain the local radius and decision separation. Appendix § 11 .§ 5 gives the proof. With four equally weighted groups and α = 0.95 \alpha=0.95 α = 0.95 , it reduces to the minimum stabilized information, so that real-data audit does not isolate a benefit from tail aggregation.
Computing and Auditing Fisher-Glass
The audit establishes which population information statements are supported after accounting for how the matrix and population were constructed. It fixes the task contrast, nuisance hypothesis, matrix, normalization, and audit distribution before independent evaluation. Ridge can raise an information estimate, adaptive selection can exaggerate extremes, and weak support can destabilize weighted populations. The corrections below address these distinct sources of distortion; fitted nuisance probes and temperatures are frozen before final evaluation.
Operational Scores and Nuisance Coordinates
For a fitted binary predictive probability p w ( x ) p_w(x) p w ( x ) , including any fixed temperature calibration, introduce task and nuisance directions z ( x ) z(x) z ( x ) and b ( x ) b(x) b ( x ) through logit p θ , μ ( x ) = logit p w ( x ) + θ z ( x ) + μ b ( x ) \operatorname{logit}p_{\theta,\mu}(x)=\operatorname{logit}p_w(x)+\theta z(x)+\mu b(x) logit p θ , μ ( x ) = logit p w ( x ) + θ z ( x ) + μ b ( x ) . At θ = μ = 0 \theta=\mu=0 θ = μ = 0 , the score is ( Y i − p i ) v i (Y_i-p_i)v_i ( Y i − p i ) v i , with v i = ( z i , b i ) T v_i=(z_i,b_i)^{\mathsf T} v i = ( z i , b i ) T and p i = p w ( x i ) p_i=p_w(x_i) p i = p w ( x i ) . Feature moments use M i = v i v i T M_i=v_i v_i^{\mathsf T} M i = v i v i T ; predictive Fisher uses M i = p i ( 1 − p i ) v i v i T M_i=p_i(1-p_i)v_i v_i^{\mathsf T} M i = p i ( 1 − p i ) v i v i T . The prospective Waterbirds audit uses the uncentered feature–Schur fraction of task-logit and background-probe coordinates. Hidden-slice discovery uses temperature-calibrated predictive Fisher, mixing 90% of the gated matrix with 10% of the current split’s same-class mean matrix before projection, during both search and held-out evaluation. This corresponds to shrinkage η = 0.10 \eta=0.10 η = 0.10 ; feature moments supply a comparison. Appendix § 12 .§ 1 specifies these constructions, the observed-label score alternative, operational subspaces, and computational cost.
For the scalar, unregularized, unshrunk predictive target with positive nuisance information,
F p r e d = min a E [ p ( X ) ( 1 − p ( X ) ) { z ( X ) − a b ( X ) } 2 ] . (12) F_{\mathrm{pred}}=\min_a \mathbb{E}\left[p(X)(1-p(X))\{z(X)-ab(X)\}^2\right]. \tag*{(12)} F pred = a min E [ p ( X ) ( 1 − p ( X )) { z ( X ) − ab ( X ) } 2 ] . (12)
Small information can therefore reflect task–nuisance alignment or confident probabilities with small p ( 1 − p ) p(1-p) p ( 1 − p ) . Neither mechanism alone establishes intervention sensitivity. Nuisance coordinates come from metadata, fitted annotation probes, or a fixed semantic direction; without annotations, that direction is the hypothesis being audited. Appendices § 13 and § 14 .§ 5 report stability and specification checks.
Ridge Estimation and Lower Information Bounds
Let M i M_i M i be the per-example task–nuisance matrix under the chosen construction, with task and nuisance blocks indexed by T T T and N N N . For a predefined slice g g g of size n g n_g n g , form J ^ g = n g − 1 ∑ i ∈ g M i \widehat{J}_g=n_g^{-1}\sum_{i\in g}M_i J g = n g − 1 ∑ i ∈ g M i and compute the ridge Schur estimate
F ^ g r i d = J ^ T T , g − J ^ T N , g ( J ^ N N , g + λ r i d I ) − 1 J ^ N T , g . (13) \widehat{F}^{\mathrm{rid}}_g=\widehat{J}_{TT,g}-\widehat{J}_{TN,g}\left(\widehat{J}_{NN,g}+\lambda_{\mathrm{rid}}I\right)^{-1}\widehat{J}_{NT,g}. \tag*{(13)} F g rid = J T T , g − J T N , g ( J N N , g + λ rid I ) − 1 J N T , g . (13)
Real-data computations use λ r i d = 10 − 3 \lambda_{\mathrm{rid}}=10^{-3} λ rid = 1 0 − 3 . Ridge stabilizes residualization but raises information relative to the unregularized target. Let a g a_g a g bound estimation error and d g d_g d g bound the ridge-to-unregularized gap on a simultaneous event over the audited slices. For declared weights Π G \Pi_G Π G , use
λ g = [ λ min ( F ^ g r i d ) − a g − d g ] + , \lambda_g=\left[\lambda_{\min}\left(\widehat{F}^{\mathrm{rid}}_g\right)-a_g-d_g\right]_+, λ g = [ λ m i n ( F g rid ) − a g − d g ] + ,
I α , G = [ CVaR α , g ∼ Π G ( ( λ g + ε ) − 1 ) ] − 1 . (14) I_{\alpha,G}=\left[\operatorname{CVaR}_{\alpha,g\sim\Pi_G}\left((\lambda_g+\varepsilon)^{-1}\right)\right]^{-1}. \tag*{(14)} I α , G = [ CVaR α , g ∼ Π G ( ( λ g + ε ) − 1 ) ] − 1 . (14)
Here [ x ] + = max { x , 0 } [x]_+=\max\{x,0\} [ x ] + = max { x , 0 } . On that event, this lower-bounds the unregularized population certificate. A large lower bound supports a positive information floor; a small lower bound alone does not establish small true information. Certification also requires a positive lower confidence bound for λ min ( J N N , g ) \lambda_{\min}(J_{NN,g}) λ m i n ( J N N , g ) . The real-data audits use Bonferroni-adjusted Gaussian/Wald moment simulation at nominal 95% confidence, conditional on fitted models, slices, and normalization. These implemented intervals differ from the concentration guarantees under Appendix § 12 ’s assumptions.
Hidden-Slice Discovery and Held-Out Certification
A gate h ϕ ( x , y ) ∈ [ 0 , 1 ] h_{\phi}(x,y)\in[0,1] h ϕ ( x , y ) ∈ [ 0 , 1 ] assigns membership weights. Discovery maximizes inverse projected information subject to effective-sample-size, maximum-weight, and class-mass constraints. We normalize memberships within class, cap excessive weights, and mix with uniform weights as needed to satisfy the effective-sample-size constraint. Five validation folds assess transfer and stability; a final gate is fitted on all discovery data, frozen, and evaluated on an untouched split. The support correction is recomputed on evaluation data under the frozen gate. Its unshrunk target is the weighted ratio J ϕ = E [ h ϕ M ] / E [ h ϕ ] J_{\phi} = \mathbb{E}[h_{\phi}M]/\mathbb{E}[h_{\phi}] J ϕ = E [ h ϕ M ] / E [ h ϕ ] , with positive membership mass. Uncertainty must account for the numerator, membership normalization, and adjustments used to enforce the weight constraints; the Waterbirds target retains the stated shrinkage. Appendix § 13 specifies the constraints, objective, and held-out target.
Algorithm 1 Fisher-Glass audit
Specify: predictor, matrix, coordinates, normalization, calibration, and audit weights.
Fix predefined slices, or fit and freeze a constrained gate on discovery data.
Audit independently: check support and conditioning; compute ridge estimates, unregularized estimates, and lower confidence bounds.
Report: raw and projected information, surviving fraction, slice support, tail certificate, and uncertainty.
Evaluating Information Audits
The experiments test distinct claims: sensitivity of a frozen predefined group, incremental failure prediction beyond confidence measures, the inherited geometric tail law, and held-out discovery of a declared information target. The predefined-group and learned-slice studies use different models and matrices.
Prospective Diagnosis Under a Nuisance Intervention
On Waterbirds [2 ] , the four groups cross bird class with background type and are written class/background . We fully fine-tune ImageNet-initialized MobileNetV3-small models [10 ] with empirical risk minimization (ERM). Validation data fit the temperature and cross-fitted background probe. Eligible slices have accuracy at least the overall validation accuracy and negative log-likelihood (NLL) at most the median across slices. Among them, we select the smallest surviving fraction, λ min ( F ^ q r i d ) / J ^ T , g \lambda_{\min}(\widehat{F}^{\mathrm{rid}}_q)/\widehat{J}_{T,g} λ m i n ( F q rid ) / J T , g . After pilot experiments, this rule is fixed for five additional training seeds, 22–26. Each checkpoint, temperature, probe, selected slice, and audit constant is frozen before intervention scoring.
The rule selects waterbird/water in all five seeds : its validation surviving fraction is 24.8%, compared with 59.0% for landbird/land, the other eligible slice. Held-out interventions recompose birds onto same-type or opposite-type backgrounds using the same extracted foreground layer, matte, and task labels. Same-type recomposition controls for the image-construction procedure. Selected-slice accuracy is 92.0% on original images, 93.2% under the control, and 40.6% under opposite-type replacement: 52.6 ± 1.4 52.6 \pm1.4 52.6 ± 1.4 percentage-point controlled drop , with a nominal 95% normal-approximation interval across seeds.
Minimum validation accuracy and maximum validation NLL select the same eligible group in all five seeds (Figure 2 ). Among eligible slices, minimum raw and absolute projected information select landbird/land, with a 26.7-point drop. Centering switches selection in all five seeds, identifying slice means as a ranking contributor; predictive Fisher retains waterbird/water in all five. Appendix D.2 reports matched selectors, matrix and aggregation comparisons, and individual flip prediction.
Figure 2. A 52.6-point controlled accuracy drop in the selected group. (A) The uncentered feature–Schur fraction selects waterbird/water from two eligible groups; matched accuracy and NLL selectors agree. (B) Accuracy falls from 93.2% under same-type background recomposition to 40.6% under opposite-type recomposition, with bird foregrounds and labels preserved. Points show means and bars nominal 95% normal-approximation intervals across five training seeds.
Predicting failures beyond confidence. Within the selected group, a retained auxiliary analysis predicts opposite-type failures among originally correct birds using five-fold cross-fitting by species. The baseline includes source NLL, calibrated true-label confidence, absolute raw logit, and standardized task logit z z z . Adding the background component β g b \beta_g b β g b and residual magnitude ∣ z − β g b ∣ |z-\beta_g b| ∣ z − β g b ∣ improves average precision by 0.0257 (nominal 95% paired interval [0.0211, 0.0302]), area under the receiver operating characteristic curve (AUROC) by 0.0150, and mean log likelihood by 0.0128 nats per donor row. All gains are positive in all five seeds; intervals use 1.96 standard errors of paired seed differences (Table D.7). This establishes incremental value of the added coordinates. It does not isolate the residual’s contribution from the background score or test the scalar population certificate as an individual predictor.
Synthetic Classification and Tail Checks
Draw independent s , r ∼ N ( 0 , 1 ) s, r \sim\mathcal{N}(0,1) s , r ∼ N ( 0 , 1 ) , set z = s + δ e r z=s+\delta_e r z = s + δ e r and Y = 1 { z > 0 } Y=\mathbb{1}\{z>0\} Y = 1 { z > 0 } , and evaluate the nuisance-only logit 4 s 4s 4 s . Environments use δ e = 0.35 U e \delta_e=0.35U_e δ e = 0.35 U e , U e ∼ Uniform ( 0 , 1 ) U_e\sim\operatorname{Uniform}(0,1) U e ∼ Uniform ( 0 , 1 ) , with γ = 1 \gamma=1 γ = 1 in the main experiment. Five-seed mean accuracy is 94.54% and CVaR 0.95 \operatorname{CVaR}_{0.95} CVaR 0.95 of environment-mean NLL is 0.262. The information audit uses exact population feature moments of ( z , s ) (z,s) ( z , s ) , giving J T T , e = 1 + δ e 2 J_{TT,e}=1+\delta_e^2 J T T , e = 1 + δ e 2 and λ e = δ e 2 \lambda_e=\delta_e^2 λ e = δ e 2 . Its certificate is I 0.95 F G = ( 1.22 ± 0.53 ) × 10 − 8 I^{\mathrm{FG}}_{0.95}=(1.22\pm0.53)\times10^{-8} I 0.95 FG = ( 1.22 ± 0.53 ) × 1 0 − 8 , with ε = 10 − 12 \varepsilon=10^{-12} ε = 1 0 − 12 .
The inverse-information exponent is 0.498 ± 0.003 0.498\pm0.003 0.498 ± 0.003 , matching β = γ / 2 = 0.5 \beta=\gamma/2=0.5 β = γ /2 = 0.5 in the geometric tail law of Theorem 2 (Figure A.1). Log-survival regression uses the largest 5% of inverse-information observations. Additional experiments check β = K / 2 \beta=K/2 β = K /2 for K K K independent information branches (Appendix § 11 .§ 2 ) and the information floor obtained by allocating observations to an informative source (Appendix § 15 .§ 1 ). Uncertainties are nominal 95% normal-approximation intervals across five seeds. These checks exhibit favorable prediction alongside weak projected information and the predicted tail behavior (Appendix § 14 .§ 3 ).
Discovering Low-Information Subpopulations
In a separate experiment, we fit a linear task head on frozen ImageNet MobileNetV3-small representations. Gate discovery uses task labels and a nuisance score obtained by projecting CLIP image embeddings onto a fixed water-minus-land text direction [11 ] . Background and group annotations remain sealed through test evaluation. We search each task class separately with a class-specific logistic gate and select the class–gate pair with highest discovery utility. Gates target 10% within-class mass under § 4.3 ’s support constraints. The validation split supplies discovery data: five folds assess transfer, after which the gate is refitted on the full split, frozen, and evaluated once on untouched test data.
The discovered slice has 98.8% membership-weighted accuracy and lower estimated projected predictive Fisher information than high-loss and random selectors targeting the same mass in every seed (Figure 3 ). Its calibrated predictive information is 0.005 ± 0.001, and its uncalibrated NLL is 0.031 ± 0.017, with 95% Student-t t t intervals across five paired seeds. Accuracy and NLL use support-corrected membership weights without information pooling (Appendix § 13 .§ 1 ). The Fisher and high-loss gates’ top 10% raw-membership sets within their selected class have mean Jaccard overlap 0.001, indicating largely distinct selections.
Figure 3. Discovery finds accurate slices with low projected predictive information. (A) CLIP-guided search finds low projected predictive Fisher information and 98.8% membership-weighted accuracy; the random-direction control also attains low information. (B) Both searches yield low uncalibrated NLL (log scale). Selectors target 10% within-class mass. Points show means and bars 95% Student-t t t intervals across five paired seeds, with gates frozen before testing.
The random-direction control attains similar predictive information, with the paired CLIP-minus-random interval spanning zero. Feature-moment comparisons with high-loss selection also have paired intervals containing zero. Predictive Fisher includes the confidence factor p ( 1 − p ) p(1-p) p ( 1 − p ) , whereas displayed NLL is uncalibrated; these results do not separate confidence from nuisance alignment.
They establish discovery of a low-information population, not its unusual intervention sensitivity or superiority to confidence-matched selection. Appendix § 14 .§ 4 gives paired intervals, support diagnostics, and composition.
Information-Guided Interventions
We evaluate whether information changes improve the intended predictive outcome. An informative observation source can guarantee an information floor under the Fisher-reserve design [6 ] (Appendix § 15 .§ 1 ); training and acquisition require separate behavioral evaluation. For predictor parameters w w w , empirical task-risk objective R ^ b a s e \widehat{R}_{\mathrm{base}} R base (ERM or group distributionally robust optimization, Group DRO), and penalty λ F G \lambda_{\mathrm{FG}} λ FG , training uses equally weighted groups:
min w R ^ b a s e ( w ) + λ F G CVaR α , g [ { λ min ( F ^ g r i d ( w ) ) + ε } − 1 ] . (15) \min_{w}\ \widehat{R}_{\mathrm{base}}(w)+\lambda_{\mathrm{FG}}\operatorname{CVaR}_{\alpha,g}\left[\left\{\lambda_{\min}\left(\widehat{F}^{\mathrm{rid}}_{g}(w)\right)+\varepsilon\right\}^{-1}\right]. \tag*{(15)} w min R base ( w ) + λ FG CVaR α , g [ { λ m i n ( F g rid ( w ) ) + ε } − 1 ] . (15)
Coordinates follow § 4 . Among the frozen-feature models whose validation worst-group accuracy is within two percentage points of the best, we select the coefficient with the largest validation certificate. This selection uses validation group labels.
Fisher-Glass approximately doubles ERM’s certificate; with Group DRO it raises both the certificate and mean worst-group accuracy (Table 1 ). Stronger penalties can reduce accuracy (Appendix § 15 .§ 3 ). These source-performance gains do not establish a repair of background sensitivity.
Table 1. Frozen-feature Waterbirds training. Means with nominal 95% normal-approximation intervals (±1.96 standard errors) across five seeds; uncentered feature–Schur ridge certificates use α = 0.95 \alpha= 0.95 α = 0.95 . Validation selection follows § 6 .
In prospective fine-tuning, the validation certificate increases from 0.289 to 0.330, while controlled swap drops remain 52.9 points for FG and 52.6 for ERM. A separate Fisher-based frozen-slice repair under validation-accuracy constraints selects the unchanged model in all five seeds. On another cohort, post hoc selection raises rediscovered-slice information but reduces worst-group accuracy by 12.6 points; selected coefficients match validation-NLL minimization in every seed (Appendix E.3). Thus information improvement and the intended behavioral improvement are separate outcomes.
For candidate a a a with modified feature/input x a c f x^{\mathrm{cf}}_a x a cf and label y a y_a y a , tail-transverse influence TTI ( a ) = G α ( a ) ℓ w ( x a c f , y a ) \operatorname{TTI}(a) = G_\alpha(a)\ell_w(x^{\mathrm{cf}}_a, y_a) TTI ( a ) = G α ( a ) ℓ w ( x a cf , y a ) weights current predictive loss by the first-order inverse-information CVaR decrease G α ( a ) G_\alpha(a) G α ( a ) along a a a ’s allocation path. This differentiates the Schur complement along its weakest task direction with tail-sensitive weights (Appendix E.4). Waterbirds ranks candidates by TTI ~ α ( a ) = G ~ α ( a ) ℓ w ( x a c f , y a ) \widetilde{\operatorname{TTI}}_\alpha(a) = \widetilde{G}_\alpha(a)\ell_w(x^{\mathrm{cf}}_a, y_a) TTI α ( a ) = G α ( a ) ℓ w ( x a cf , y a ) , using the tail-weighted squared-residual surrogate G ~ α \widetilde{G}_\alpha G α defined in Appendix E.5.
Waterbirds candidates translate features by the within-class difference of background-group means, scaled by γ c f \gamma_{\mathrm{cf}} γ cf . Training data fit means and the nuisance probe; validation supplies candidates for retraining, and evaluation is held out. Budgets are fractions of original training size. At γ c f = 1 \gamma_{\mathrm{cf}} = 1 γ cf = 1 and a 1% budget, TTI ~ α \widetilde{\operatorname{TTI}}_\alpha TTI α yields a mean worst-group accuracy gain of 4.52 points versus 3.27 for high-loss selection; high loss produces the larger mean certificate gain. At this strength, TTI ~ α \widetilde{\operatorname{TTI}}_\alpha TTI α has the larger mean worst-group accuracy gain at budgets of 1%, 2%, and 5%; selection by G ~ α \widetilde{G}_\alpha G α alone yields smaller improvements (Appendix E.5).
Related Work
Efficient information and classification. Efficient-score theory defines task information after projection away from nuisance tangent spaces [4 , 5 , 12 ] . Fisher-glass theory connects nuisance-adjusted information to inverse-information tails in metrology [6 ] . Our analysis develops classification bounds and statistical guarantees for population audits. The classification argument uses local testing reductions [13 , 14 ] under an explicit Bayes-decision separation condition. The operational analysis addresses Schur estimation, ridge effects, and independent evaluation of learned population gates.
Tail risk, robust learning, and gradient geometry. CVaR concentrates objectives on adverse outcomes [8 , 9 ] ; Group DRO applies robust optimization to predictive loss across groups [2 ] . Invariant Risk Minimization seeks shared optimal classifiers [15 ] , while Fishr regularizes domain-level gradient variances [16 ] . We use inverse projected-information tail risk as a training penalty alongside empirical risk or Group DRO, and evaluate information and predictive performance together.
Hidden stratification and slice discovery. Hidden-stratification research motivates finer population evaluation [17 ] . GEORGE clusters representations to infer subclasses [18 ] , JTT upweights initially misclassified examples [3 ] , and AGRO learns adversarial group assignments using prediction loss [19 ] . Our search targets low predictive projected information given task labels and a fixed nuisance direction. Support constraints and an independently evaluated frozen gate make accurately classified, low-loss populations eligible audit targets.
Information-based design and acquisition. E-optimal design emphasizes the smallest information eigenvalue [20 ] ; BAIT uses Fisher information for neural active learning [21 ] . TTI measures local information gain along a weak nuisance-adjusted task direction, weights environments by tail sensitivity, and multiplies by current task loss. Acquisition experiments assess realized information and accuracy changes after retraining.
Limitations
Specification and interpretation. Audits quantify information for a declared task contrast, nuisance space, representation, and population; admissible nuisance changes must preserve the task. Unregularized nuisance projection of predictive Fisher gives efficient information for the declared predictive model, while feature–Schur quantities describe residual feature geometry. Their relation to population efficient information depends on the score model. Coordinate scaling and matrix construction affect the audit; standardization does not establish representation invariance. The incremental contribution of semantic nuisance projection remains unresolved (Section 5). A general procedure for admitting nuisance specifications through relevance, task preservation, stability, and conditioning checks remains to be evaluated (Appendix B.1). Theory and uncertainty. The local classification bound requires distinguishability and quantitative Bayes-decision separation throughout its radius. The worked family verifies these conditions and a matching rate, but its statistical nuisance is not a model of physical background swaps. Coverage concerns the declared environment distribution and frozen population rules. Small mass and weak nuisance conditioning amplify uncertainty; ridge stabilization can inflate information, so positive ridge values alone do not establish identifiability (§ 4.2 ; Appendix § 12 .§ 2 ). Implemented Gaussian/Wald coverage is nominal and conditional on fitted models, evaluated families, and normalization; it omits uncertainty from evaluation-fitted normalization. Repeated use of certification data requires additional control (§ 4 ; Appendix § 12 ).
Empirical scope and interventions. Evidence centers on Waterbirds, supplementary CivilComments diagnostics, and synthetic constructions. The prospective rule uses fresh seeds of the development benchmark; transfer requires further evaluation. The auxiliary failure-prediction gain combines background and residual coordinates, without a background-only ablation. No retained experiment tests whether the learned low-information slice is more sensitive than a confidence-matched population. Information gains and coefficient-selection policies require separate behavioral validation (§ 6 ). Reserve guarantees assume an informative anchor; acquisition uses group-informed feature shifts whose transfer to realizable task-preserving inputs remains open (Appendix § 15 ).
Conclusion
Task–nuisance ambiguity has a classification cost when statistically similar alternatives require different decisions. Our local bound makes that condition explicit, and a worked family shows how nuisance confounding reduces the effective sample size near a decision boundary despite constant raw information at the reference parameter. A matching upper bound identifies the learning rate and the cap imposed by the local decision scale. Population audits then account for estimation error, ridge bias, adaptive membership, and support, establishing which information statements are supported by data.
The retained Waterbirds experiments establish a 52.6-point controlled accuracy drop in a frozen predefined group and incremental species-held-out failure-prediction value from nuisance-derived coordinates beyond a specified confidence baseline. Independent evaluation validates a discovered low predictive-information population. Training and acquisition show that information gains and predictive gains can diverge.
The classification result, population audit, and behavioral evaluations have complementary roles. They connect ambiguity to learning difficulty, quantify a declared information target with uncertainty, and test its relationship to observed behavior. This supports evaluating information objectives together with predictive performance and controlled nuisance sensitivity when assessing a classifier or a proposed improvement.
Acknowledgments and Disclosure of Funding
This work received no funding. The author declares no competing interests.
Proofs of the Core Results
Throughout, task coordinates use the common normalization of § 2 , and environments follow the declared distribution Π \Pi Π .
Projection, Normalization, and Task–Nuisance Tangency
Use the centered, square-integrable scores and blocks J A B , e J_{AB,e} J A B , e of § 2.1 . Suppose the d N d_N d N coordinates of the finite nuisance-score vector span T N , e \mathcal{T}_{N,e} T N , e and J N N , e ≻ 0 J_{NN,e} \succ0 J N N , e ≻ 0 . With Γ e = J T N , e J N N , e − 1 \Gamma_e = J_{TN,e}J_{NN,e}^{-1} Γ e = J T N , e J N N , e − 1 ,
E P e [ ( s T , e − Γ e s N , e ) s N , e ⊤ ] = J T N , e − Γ e J N N , e = 0 , \mathbb{E}_{P_e}[(s_{T,e}-\Gamma_e s_{N,e})s_{N,e}^{\top}] = J_{TN,e}-\Gamma_e J_{NN,e}=0, E P e [( s T , e − Γ e s N , e ) s N , e ⊤ ] = J T N , e − Γ e J N N , e = 0 ,
F e = E P e [ s T , e e f f ( s T , e e f f ) ⊤ ] = J T T , e − J T N , e J N N , e − 1 J N T , e . (16) F_e = \mathbb{E}_{P_e}[s_{T,e}^{\mathrm{eff}}(s_{T,e}^{\mathrm{eff}})^{\top}] = J_{TT,e}-J_{TN,e}J_{NN,e}^{-1}J_{NT,e}. \tag*{(16)} F e = E P e [ s T , e eff ( s T , e eff ) ⊤ ] = J T T , e − J T N , e J N N , e − 1 J N T , e . (16)
Because Γ e s N , e \Gamma_e s_{N,e} Γ e s N , e lies componentwise in the nuisance span, orthogonality identifies the residual as the efficient score in Equation (1) [4 , 5 ] . Equivalently,
u ⊤ F e u = min v ∈ R d N E P e [ ( u ⊤ s T , e − v ⊤ s N , e ) 2 ] , v ⋆ = J N N , e − 1 J N T , e u . (17) u^{\top}F_eu=\min_{v\in\mathbb{R}^{d_N}}\mathbb{E}_{P_e}[(u^{\top}s_{T,e}-v^{\top}s_{N,e})^2],\qquad v^\star=J_{NN,e}^{-1}J_{NT,e}u. \tag*{(17)} u ⊤ F e u = v ∈ R d N min E P e [( u ⊤ s T , e − v ⊤ s N , e ) 2 ] , v ⋆ = J N N , e − 1 J N T , e u . (17)
Hence F e ⪰ 0 F_e \succeq0 F e ⪰ 0 , with u ⊤ F e u = 0 u^{\top}F_eu=0 u ⊤ F e u = 0 exactly when u ⊤ s T , e u^{\top}s_{T,e} u ⊤ s T , e belongs to the nuisance span. The projection definition extends directly to the closed span of an infinite-dimensional nuisance tangent space.
For separate invertible transformations s ~ T = U s T \widetilde{s}_T=Us_T s T = U s T and s ~ N = V s N \widetilde{s}_N=Vs_N s N = V s N , the nuisance span is unchanged and
s ~ T e f f = U s T e f f , F ~ = U F U ⊤ ; F J T T = 1 − ρ 2 in the scalar case . (18) \widetilde{s}_T^{\mathrm{eff}}=Us_T^{\mathrm{eff}},\qquad\widetilde{F}=UFU^{\top};\qquad\frac{F}{J_{TT}}=1-\rho^2\quad\text{in the scalar case}. \tag*{(18)} s T eff = U s T eff , F = U F U ⊤ ; J T T F = 1 − ρ 2 in the scalar case . (18)
Here ρ = J T N / J T T J N N \rho=J_{TN}/\sqrt{J_{TT}J_{NN}} ρ = J T N / J T T J N N requires positive raw task and nuisance information. Exact information is invariant to nuisance-basis changes; task congruence preserves rank but generally changes eigenvalues. The scalar surviving fraction is invariant to nonzero task rescaling and invertible nuisance-basis changes. Nuisance ridge preserves the latter invariance only if λ r i d I \lambda_{\mathrm{rid}}I λ rid I becomes λ r i d V V ⊤ \lambda_{\mathrm{rid}}VV^{\top} λ rid V V ⊤ under s N ↦ V s N s_N\mapsto Vs_N s N ↦ V s N (Appendix § 12 ).
For the tangency construction in § 2.2 , let b e , r e ∈ L 0 2 ( P e ) b_e,r_e\in L_0^2(P_e) b e , r e ∈ L 0 2 ( P e ) be orthogonal unit vectors, T N , e = span { b e } \mathcal{T}_{N,e}=\operatorname{span}\{b_e\} T N , e = span { b e } , and a e = b e + δ e r e a_e=b_e+\delta_e r_e a e = b e + δ e r e with δ e ≥ 0 \delta_e\ge0 δ e ≥ 0 . The projection coefficient is ⟨ a e , b e ⟩ e / ∥ b e ∥ e 2 = 1 \langle a_e,b_e\rangle_e/\lVert b_e\rVert_e^2=1 ⟨ a e , b e ⟩ e / ∥ b e ∥ e 2 = 1 , so
a e e f f = δ e r e , F e = ∥ δ e r e ∥ e 2 = δ e 2 , J T T , e = 1 + δ e 2 , F e J T T , e = δ e 2 1 + δ e 2 . (19) a_e^{\mathrm{eff}}=\delta_e r_e,\qquad F_e=\lVert\delta_e r_e\rVert_e^2=\delta_e^2,\qquad J_{TT,e}=1+\delta_e^2,\qquad\frac{F_e}{J_{TT,e}}=\frac{\delta_e^2}{1+\delta_e^2}. \tag*{(19)} a e eff = δ e r e , F e = ∥ δ e r e ∥ e 2 = δ e 2 , J T T , e = 1 + δ e 2 , J T T , e F e = 1 + δ e 2 δ e 2 . (19)
This establishes Equation (3) ; coexistence with low prediction risk additionally requires a successful nuisance-based predictor.
The full assumptions of Theorem 2 are as follows. The zero set Z = { λ = 0 } \mathcal{Z}=\{\lambda=0\} Z = { λ = 0 } is a nonempty compact C 2 C^2 C 2 embedded manifold without boundary, of codimension c ∈ { 1 , … , d } c\in\{1,\ldots,d\} c ∈ { 1 , … , d } . Write U r ( Z ) = { z : dist ( z , Z ) < r } U_r(\mathcal{Z})=\{z:\operatorname{dist}(z,\mathcal{Z})<r\} U r ( Z ) = { z : dist ( z , Z ) < r } for its tube, N s Z N_s\mathcal{Z} N s Z for the normal space at s s s , S ( N s Z ) S(N_s\mathcal{Z}) S ( N s Z ) for its unit sphere, and vol Z \operatorname{vol}_{\mathcal{Z}} vol Z for induced volume. The nonnegative λ \lambda λ is continuous on U r ⋆ ( Z ) ‾ \overline{U_{r_\star}(\mathcal{Z})} U r ⋆ ( Z ) and bounded away from zero outside it. In coordinates z = s + r ω z=s+r\omega z = s + r ω , assume uniformly over ω ∈ S ( N s Z ) \omega\in S(N_s\mathcal{Z}) ω ∈ S ( N s Z ) and s ∈ Z s\in\mathcal{Z} s ∈ Z
λ ( s + r ω ) = a ( s , ω ) r 2 m ( 1 + o ( 1 ) ) , p ( s + r ω ) = b ( s , ω ) r ν ( 1 + o ( 1 ) ) , (20) \lambda(s+r\omega)=a(s,\omega)r^{2m}(1+o(1)),\qquad p(s+r\omega)=b(s,\omega)r^\nu(1+o(1)), \tag*{(20)} λ ( s + r ω ) = a ( s , ω ) r 2 m ( 1 + o ( 1 )) , p ( s + r ω ) = b ( s , ω ) r ν ( 1 + o ( 1 )) , (20)
where p p p is the Lebesgue density, m > 0 m>0 m > 0 , ν > − c \nu>-c ν > − c , and continuous a , b a,b a , b have positive uniform lower and upper bounds. Put
β = c + ν 2 m , C Z = 1 c + ν ∫ Z ∫ S ( N s Z ) b ( s , ω ) a ( s , ω ) − β d σ s ( ω ) d vol Z ( s ) . (21) \beta=\frac{c+\nu}{2m},\qquad C_{\mathcal{Z}}=\frac{1}{c+\nu}\int_{\mathcal{Z}}\int_{S(N_s\mathcal{Z})}b(s,\omega)a(s,\omega)^{-\beta}\,d\sigma_s(\omega)\,d\operatorname{vol}_{\mathcal{Z}}(s). \tag*{(21)} β = 2 m c + ν , C Z = c + ν 1 ∫ Z ∫ S ( N s Z ) b ( s , ω ) a ( s , ω ) − β d σ s ( ω ) d vol Z ( s ) . (21)
The sphere measure σ s \sigma_s σ s is unnormalized; for c = 1 c=1 c = 1 , each of the two normal directions has mass one.
Proof of Theorem 2. For any 0 < r 0 < r ⋆ 0<r_0<r_\star 0 < r 0 < r ⋆ , continuity and absence of zeros on the compact annulus r 0 ≤ dist ( z , Z ) ≤ r ⋆ r_0\le\operatorname{dist}(z,\mathcal{Z})\le r_\star r 0 ≤ dist ( z , Z ) ≤ r ⋆ give a positive minimum of λ \lambda λ . Together with the bound outside the outer tube, this localizes { λ < t } \{\lambda<t\} { λ < t } to U r 0 ( Z ) U_{r_0}(\mathcal{Z}) U r 0 ( Z ) for all sufficiently small t t t . The tubular volume element is d z = j ( s , r ω ) r c − 1 d r d σ s d vol Z dz = j(s,r\omega)r^{c-1}\,dr\,d\sigma_s\,d\operatorname{vol}_Z d z = j ( s , r ω ) r c − 1 d r d σ s d vol Z , where the coordinate Jacobian satisfies j = 1 + O ( r ) j=1+O(r) j = 1 + O ( r ) uniformly. Thus, for any 0 < ζ < 1 0<\zeta<1 0 < ζ < 1 , uniformity permits r 0 r_0 r 0 small enough that
\begin{aligned}
(1-\zeta)ar^{2m} &\le\lambda(s+r\omega)\le(1+\zeta)ar^{2m},\\
(1-\zeta)br^\nu&\le p(s+r\omega)j(s,r\omega)\le(1+\zeta)br^\nu
\end{aligned}
\qquad(0<r<r_0). \tag*{(22)}\end{aligned}
For small t t t , all radial thresholds lie inside this tube. The event contains r < ( t / ( ( 1 + ζ ) a ) ) 1 / ( 2 m ) r<(t/((1+\zeta)a))^{1/(2m)} r < ( t / (( 1 + ζ ) a ) ) 1/ ( 2 m ) and is contained in r < ( t / ( ( 1 − ζ ) a ) ) 1 / ( 2 m ) r<(t/((1-\zeta)a))^{1/(2m)} r < ( t / (( 1 − ζ ) a ) ) 1/ ( 2 m ) . Integrating uses ∫ 0 ( t / a ) 1 / ( 2 m ) b r c + ν − 1 d r = b a − β t β / ( c + ν ) \int_0^{(t/a)^{1/(2m)}}br^{c+\nu-1}\,dr=ba^{-\beta}t^{\beta/(c+\nu)} ∫ 0 ( t / a ) 1/ ( 2 m ) b r c + ν − 1 d r = b a − β t β / ( c + ν ) and gives
( 1 − ζ ) ( 1 + ζ ) − β C Z ≤ Pr ( λ < t ) t β ≤ ( 1 + ζ ) ( 1 − ζ ) − β C Z . (23) (1-\zeta)(1+\zeta)^{-\beta}C_Z\le\frac{\Pr(\lambda<t)}{t^\beta}\le(1+\zeta)(1-\zeta)^{-\beta}C_Z. \tag*{(23)} ( 1 − ζ ) ( 1 + ζ ) − β C Z ≤ t β Pr ( λ < t ) ≤ ( 1 + ζ ) ( 1 − ζ ) − β C Z . (23)
Let t ↓ 0 t\downarrow0 t ↓ 0 and then ζ ↓ 0 \zeta\downarrow0 ζ ↓ 0 . Compactness and the coefficient bounds ensure 0 < C Z < ∞ 0<C_Z<\infty 0 < C Z < ∞ , so Pr ( λ < t ) ∼ C Z t β \Pr(\lambda<t)\sim C_Zt^\beta Pr ( λ < t ) ∼ C Z t β . Since c ≥ 1 c\ge1 c ≥ 1 , Z Z Z has zero Lebesgue and environment measure; substituting t = x − 1 t=x^{-1} t = x − 1 proves the inverse tail in the next display.
The tail-integral identity then yields, for every p 0 > 0 p_0>0 p 0 > 0 ,
Pr ( λ − 1 > x ) ∼ C Z x − β , \Pr(\lambda^{-1}>x)\sim C_Zx^{-\beta}, Pr ( λ − 1 > x ) ∼ C Z x − β ,
E [ λ − p 0 ] = p 0 ∫ 0 ∞ x p 0 − 1 Pr ( λ − 1 > x ) d x < ∞ ⟺ p 0 < β . (24) \mathbb{E}[\lambda^{-p_0}]=p_0\int_0^\infty x^{p_0-1}\Pr(\lambda^{-1}>x)\,dx<\infty\quad\Longleftrightarrow\quad p_0<\beta. \tag*{(24)} E [ λ − p 0 ] = p 0 ∫ 0 ∞ x p 0 − 1 Pr ( λ − 1 > x ) d x < ∞ ⟺ p 0 < β . (24)
The bounded-x x x part is finite; the tail has the convergence criterion of ∫ ∞ x p 0 − β − 1 d x \int^\infty x^{p_0-\beta-1}\,dx ∫ ∞ x p 0 − β − 1 d x . In particular, the inverse mean is finite exactly for β > 1 \beta>1 β > 1 , with logarithmic divergence at β = 1 \beta=1 β = 1 .
Scalar branches. For λ = δ 2 \lambda=\delta^2 λ = δ 2 , Pr ( δ < t ) ∼ C δ t γ \Pr(\delta<t)\sim C_\delta t^\gamma Pr ( δ < t ) ∼ C δ t γ gives exponent γ / 2 \gamma/2 γ /2 . More generally, let λ K = ∑ k = 1 K w k δ k 2 \lambda_K=\sum_{k=1}^K w_k\delta_k^2 λ K = ∑ k = 1 K w k δ k 2 , w k > 0 w_k>0 w k > 0 , where δ ∈ [ 0 , ∞ ) K \delta\in[0,\infty)^K δ ∈ [ 0 , ∞ ) K has joint density f f f continuous and positive at the origin from within the orthant. Changing variables gives the scalar portfolio law [6 ] :
Pr ( λ K < t ) = t K / 2 ∏ k w k ∫ B K + f ( t / w 1 u 1 , … , t / w K u K ) d u ∼ f ( 0 ) v K 2 K ∏ k w k t K / 2 . (25) \begin{aligned}
\Pr(\lambda_K<t)
&=\frac{t^{K/2}}{\sqrt{\prod_k w_k}}\int_{B_K^+}f\left(\sqrt{t/w_1}u_1,\ldots,\sqrt{t/w_K}u_K\right)\,du\\
&\sim\frac{f(0)v_K}{2^K\sqrt{\prod_k w_k}}\,t^{K/2}. \tag*{(25)}
\end{aligned} Pr ( λ K < t ) = ∏ k w k t K /2 ∫ B K + f ( t / w 1 u 1 , … , t / w K u K ) d u ∼ 2 K ∏ k w k f ( 0 ) v K t K /2 . (25)
Here B K + B_K^+ B K + is the positive-orthant unit ball and v K v_K v K the full ball’s volume. Independence gives f ( 0 ) = ∏ k f k ( 0 + ) f(0)=\prod_k f_k(0+) f ( 0 ) = ∏ k f k ( 0 + ) for branch marginal densities f k f_k f k ; the inverse mean is finite for K ≥ 3 K\ge3 K ≥ 3 . This calculation concerns additive scalar information or a fixed task direction. A minimum-eigenvalue conclusion instead requires applying Theorem 2 to that eigenvalue.
Local Classification: Testing Reduction and Tail Aggregation
Let E P ( g ) = R P ( g ) − R P ⋆ \mathcal{E}_P(g)=R_P(g)-R_P^\star E P ( g ) = R P ( g ) − R P ⋆ . Fix a local family P e l o c \mathcal{P}_e^{\mathrm{loc}} P e loc independently of sample size, and define
R e ( n ) = inf g ^ n sup P ∈ P e l o c E S n ∼ P ⊗ n [ E P ( g ^ n ( S n ) ) ] . (26) \mathfrak{R}_e(n)=\inf_{\widehat{g}_n}\sup_{P\in\mathcal{P}_e^{\mathrm{loc}}}\mathbb{E}_{S_n\sim P^{\otimes n}}[\mathcal{E}_P(\widehat{g}_n(S_n))]. \tag*{(26)} R e ( n ) = g n inf P ∈ P e loc sup E S n ∼ P ⊗ n [ E P ( g n ( S n ))] . (26)
where learning rules are measurable and n n n counts independent observations from environment e e e ; independent learner randomization is included in the expectation. Assume λ e > 0 \lambda_e>0 λ e > 0 and a unit weakest efficient-information direction u e u_e u e . For every 0 < t ≤ t 0 , e 0<t\le t_{0,e} 0 < t ≤ t 0 , e , the fixed family contains P e , ± t P_{e,\pm t} P e , ± t with task coordinates ± t u e \pm tu_e ± t u e and potentially varying nuisance coordinates. For fixed K e , c e , t 0 , e > 0 K_e,c_e,t_{0,e}>0 K e , c e , t 0 , e > 0 and q > 0 q>0 q > 0 , assume both bounds in Equation (4) hold throughout that radius.
Proof of Theorem 1. Fix an admissible t t t and abbreviate P ± = P e , ± t P_\pm=P_{e,\pm t} P ± = P e , ± t . For any learning rule, the decision-separation condition E P + ( g ) + E P − ( g ) ≥ c e t q \mathcal{E}_{P_+}(g)+\mathcal{E}_{P_-}(g)\ge c_et^q E P + ( g ) + E P − ( g ) ≥ c e t q implies that choosing the alternative with smaller excess risk makes any testing error cost at least c e t q / 2 c_et^q/2 c e t q /2 under the true alternative. Explicitly,
ψ t ( S n ) = + ⟺ E P + ( g ^ n ( S n ) ) ≤ E P − ( g ^ n ( S n ) ) , \psi_t(S_n)=+\quad\Longleftrightarrow\quad\mathcal{E}_{P_+}(\widehat{g}_n(S_n))\le\mathcal{E}_{P_-}(\widehat{g}_n(S_n)), ψ t ( S n ) = + ⟺ E P + ( g n ( S n )) ≤ E P − ( g n ( S n )) ,
max σ ∈ { + , − } E P σ ⊗ n [ E P σ ( g ^ n ) ] ≥ c e t q 4 { P + ⊗ n ( ψ t = − ) + P − ⊗ n ( ψ t = + ) } . (27) \max_{\sigma\in\{+,-\}}\mathbb{E}_{P_\sigma^{\otimes n}}[\mathcal{E}_{P_\sigma}(\widehat{g}_n)]\ge\frac{c_et^q}{4}\left\{P_+^{\otimes n}(\psi_t=-)+P_-^{\otimes n}(\psi_t=+)\right\}. \tag*{(27)} σ ∈ { + , − } max E P σ ⊗ n [ E P σ ( g n )] ≥ 4 c e t q { P + ⊗ n ( ψ t = − ) + P − ⊗ n ( ψ t = + ) } . (27)
Figure A.1. Near-alignment produces a heavy inverse-information tail. (A) Projection leaves a residual score of norm δ \delta δ , giving information λ = δ 2 \lambda= \delta^2 λ = δ 2 . (B) Raw information 1 + δ 2 1+\delta^2 1 + δ 2 remains near one as λ \lambda λ approaches zero. (C) Empirical survival probabilities (blue) follow the x − 1 / 2 x^{-1/2} x − 1/2 reference (dashed). Panels B–C use 300,000 environments with δ ∼ Uniform ( 0 , 0.35 ) \delta\sim\operatorname{Uniform}(0, 0.35) δ ∼ Uniform ( 0 , 0.35 ) and L = ( δ 2 + 10 − 12 ) − 1 L = (\delta^2 + 10^{-12})^{-1} L = ( δ 2 + 1 0 − 12 ) − 1 .
The standard exponential two-point testing inequality [14 Lemma 2.6] , together with product additivity of KL and the assumed KL bound, gives
P + ⊗ n ( ψ t = − ) + P − ⊗ n ( ψ t = + ) ≥ 1 2 exp [ − n KL ( P + ∥ P − ) ] ≥ 1 2 e − n K e t 2 λ e , P_{+}^{\otimes n}(\psi_t=-) + P_{-}^{\otimes n}(\psi_t=+) \ge\frac{1}{2}\exp[-n\operatorname{KL}(P_{+}\mathbin\Vert P_{-})] \ge\frac{1}{2}e^{-nK_e t^2\lambda_e}, P + ⊗ n ( ψ t = − ) + P − ⊗ n ( ψ t = + ) ≥ 2 1 exp [ − n KL ( P + ∥ P − )] ≥ 2 1 e − n K e t 2 λ e ,
R e ( n ) ≥ c e 8 t q e − n K e t 2 λ e . (28) \mathcal{R}_e(n) \ge\frac{c_e}{8}t^q e^{-nK_e t^2\lambda_e}. \tag*{(28)} R e ( n ) ≥ 8 c e t q e − n K e t 2 λ e . (28)
The second line takes the supremum over the fixed family containing both alternatives and then the infimum over learning rules. Choose
t n = min { t 0 , e , ( n K e λ e ) − 1 / 2 } , n K e λ e t n 2 ≤ 1 , t_n = \min\{t_{0,e},(nK_e\lambda_e)^{-1/2}\}, \qquad nK_e\lambda_e t_n^2 \le1, t n = min { t 0 , e , ( n K e λ e ) − 1/2 } , n K e λ e t n 2 ≤ 1 ,
R e ( n ) ≥ c 0 c e min { t 0 , e q , ( n K e λ e ) − q / 2 } , c 0 = e − 1 8 . (29) \mathcal{R}_e(n) \ge c_0c_e\min\{t_{0,e}^q,(nK_e\lambda_e)^{-q/2}\}, \qquad c_0 = \frac{e^{-1}}{8}. \tag*{(29)} R e ( n ) ≥ c 0 c e min { t 0 , e q , ( n K e λ e ) − q /2 } , c 0 = 8 e − 1 . (29)
Independent learner randomization has the same law under both alternatives and leaves their KL divergence unchanged, so the argument also covers randomized rules.
Quadratic decision separation. Suppose the binary alternatives share a feature marginal Q e Q_e Q e , with regression functions η ± , t ( x ) = P e , ± t ( Y = 1 ∣ X = x ) \eta_{\pm,t}(x)=P_{e,\pm t}(Y=1\mid X=x) η ± , t ( x ) = P e , ± t ( Y = 1 ∣ X = x ) and Bayes rules g ± , t ∗ ( x ) = 1 { η ± , t ( x ) ≥ 1 / 2 } g^*_{\pm,t}(x)=1\{\eta_{\pm,t}(x)\ge1/2\} g ± , t ∗ ( x ) = 1 { η ± , t ( x ) ≥ 1/2 } . Let D e , t = { g + , t ∗ ≠ g − , t ∗ } D_{e,t}=\{g^*_{+,t}\ne g^*_{-,t}\} D e , t = { g + , t ∗ = g − , t ∗ } be their disagreement set. Applying the binary excess-risk identity [22 Eq. (1.1)] gives
inf g { E e , + t ( g ) + E e , − t ( g ) } = ∫ D e , t min { ∣ 2 η + , t ( x ) − 1 ∣ , ∣ 2 η − , t ( x ) − 1 ∣ } d Q e ( x ) . (30) \inf_g\{\mathcal{E}_{e,+t}(g)+\mathcal{E}_{e,-t}(g)\}
=
\int_{D_{e,t}}\min\{|2\eta_{+,t}(x)-1|,|2\eta_{-,t}(x)-1|\}\,dQ_e(x). \tag*{(30)} g inf { E e , + t ( g ) + E e , − t ( g )} = ∫ D e , t min { ∣2 η + , t ( x ) − 1∣ , ∣2 η − , t ( x ) − 1∣ } d Q e ( x ) . (30)
Pointwise, a classifier must disagree with at least one Bayes rule on D e , t D_{e,t} D e , t ; choosing the smaller margin attains the integral for unrestricted measurable binary classifiers, while it remains a lower bound for a restricted class. If, for every 0 < t ≤ t 0 , e 0<t\le t_{0,e} 0 < t ≤ t 0 , e , a measurable A e , t ⊆ D e , t A_{e,t}\subseteq D_{e,t} A e , t ⊆ D e , t has Q e ( A e , t ) ≥ c m a s s , e t Q_e(A_{e,t})\ge c_{\mathrm{mass},e}t Q e ( A e , t ) ≥ c mass , e t and both margins at least c m a r g i n , e t c_{\mathrm{margin},e}t c margin , e t , with positive constants independent of t t t , the integral is at least c m a s s , e c m a r g i n , e t 2 c_{\mathrm{mass},e}c_{\mathrm{margin},e}t^2 c mass , e c margin , e t 2 . Thus q = 2 q=2 q = 2 and c e = c m a s s , e c m a r g i n , e c_e=c_{\mathrm{mass},e}c_{\mathrm{margin},e} c e = c mass , e c margin , e satisfy decision separation; the KL condition is still required separately.
Environmental aggregation. If these assumptions hold for Π \Pi Π -almost every environment and all quantities are measurable, monotonicity and positive homogeneity of CVaR give
B e ( n ) = c e min { t 0 , e q , ( n K e λ e ) − q / 2 } , \mathcal{B}_e(n)=c_e\min\{t_{0,e}^q,(nK_e\lambda_e)^{-q/2}\}, B e ( n ) = c e min { t 0 , e q , ( n K e λ e ) − q /2 } ,
CVaR α , e ∼ Π ( R e ( n ) ) ≥ c 0 CVaR α , e ∼ Π ( B e ( n ) ) . (31) \operatorname{CVaR}_{\alpha,e\sim\Pi}(\mathcal{R}_e(n))\ge c_0\operatorname{CVaR}_{\alpha,e\sim\Pi}(\mathcal{B}_e(n)). \tag*{(31)} CVaR α , e ∼ Π ( R e ( n )) ≥ c 0 CVaR α , e ∼ Π ( B e ( n )) . (31)
Separation c e c_e c e , distinguishability K e K_e K e , and radius t 0 , e t_{0,e} t 0 , e remain inside the aggregation. For q = 2 q=2 q = 2 and n K e λ e ≥ t 0 , e 2 nK_e\lambda_e \ge t_{0,e}^{2} n K e λ e ≥ t 0 , e 2 , B e ( n ) = c e / ( n K e λ e ) \mathcal{B}_e(n)=c_e/(nK_e\lambda_e) B e ( n ) = c e / ( n K e λ e ) ; outside that interior regime the radius truncates the bound.
A Boundary-Region Classification Family
Fix a known 0 < δ ≤ 1 / 2 0<\delta\le1/2 0 < δ ≤ 1/2 and let P δ \mathcal{P}_{\delta} P δ be the family in Equation (6) , with X ∼ Unif [ − 1 , 1 ] X\sim\operatorname{Unif}[-1,1] X ∼ Unif [ − 1 , 1 ] and parameter rectangle ∣ θ ∣ , ∣ μ ∣ ≤ δ / 8 |\theta|,|\mu|\le\delta/8 ∣ θ ∣ , ∣ μ ∣ ≤ δ /8 . Define
R δ ( n ) = inf g ^ n sup ∣ θ ∣ , ∣ μ ∣ ≤ δ / 8 E θ , μ [ R θ , μ ( g ^ n ) − R θ , μ ⋆ ] , \mathcal{R}_{\delta}(n)=\inf_{\widehat{g}_n}\sup_{|\theta|,|\mu|\le\delta/8}\mathbb{E}_{\theta,\mu}[R_{\theta,\mu}(\widehat{g}_n)-R_{\theta,\mu}^{\star}], R δ ( n ) = g n inf ∣ θ ∣ , ∣ μ ∣ ≤ δ /8 sup E θ , μ [ R θ , μ ( g n ) − R θ , μ ⋆ ] ,
where the infimum ranges over measurable, possibly randomized learning rules based on n n n independent observations.
Proposition A.1 (Nuisance confounding near a classification boundary). At the reference parameter ( 0 , 0 ) (0,0) ( 0 , 0 ) , the raw and efficient task information satisfy
J T T = 4 log 3 , F δ = 4 log 2 + δ 2 − δ , 4 δ ≤ F δ ≤ 64 15 δ . J_{TT}=4\log3,\qquad F_{\delta}=4\log\frac{2+\delta}{2-\delta},\qquad4\delta\le F_{\delta}\le\frac{64}{15}\delta. J T T = 4 log 3 , F δ = 4 log 2 − δ 2 + δ , 4 δ ≤ F δ ≤ 15 64 δ . For every n ≥ 1 n\ge1 n ≥ 1 ,
1 128 e min { δ 2 , ( n δ ) − 1 } ≤ R δ ( n ) ≤ min { δ 2 32 , 1 − ( 1 − δ ) n + 1 2 ( n + 1 ) δ } . (32) \frac{1}{128e}\min\{\delta^{2},(n\delta)^{-1}\}\le\mathcal{R}_{\delta}(n)\le\min\left\{\frac{\delta^{2}}{32},\frac{1-(1-\delta)^{n+1}}{2(n+1)\delta}\right\}. \tag*{(32)} 128 e 1 min { δ 2 , ( n δ ) − 1 } ≤ R δ ( n ) ≤ min { 32 δ 2 , 2 ( n + 1 ) δ 1 − ( 1 − δ ) n + 1 } . (32) Consequently, the rate in Equation (8) holds with constants independent of n n n and δ \delta δ .
Proof. Write b δ ( x ) = 1 { ∣ x ∣ > δ } b_{\delta}(x)=1\{|x|>\delta\} b δ ( x ) = 1 { ∣ x ∣ > δ } and η 0 ( x ) = 1 / 2 + x / 4 \eta_0(x)=1/2+x/4 η 0 ( x ) = 1/2 + x /4 . Throughout the rectangle, 1 / 8 ≤ η θ , μ ≤ 7 / 8 1/8\le\eta_{\theta,\mu}\le7/8 1/8 ≤ η θ , μ ≤ 7/8 , so the binary likelihood is regular in ( θ , μ ) (\theta,\mu) ( θ , μ ) for each fixed δ > 0 \delta>0 δ > 0 . Inside ∣ x ∣ ≤ δ |x|\le\delta ∣ x ∣ ≤ δ , the regression crosses 1 / 2 1/2 1/2 at − 4 θ ∈ [ − δ / 2 , δ / 2 ] -4\theta\in[-\delta/2,\delta/2] − 4 θ ∈ [ − δ /2 , δ /2 ] . Outside, ∣ θ + μ ∣ ≤ δ / 4 < ∣ x ∣ / 4 |\theta+\mu|\le\delta/4<|x|/4 ∣ θ + μ ∣ ≤ δ /4 < ∣ x ∣/4 , so its sign relative to 1 / 2 1/2 1/2 is the sign of x x x . Thus
g θ , μ ⋆ ( x ) = 1 { x ≥ − 4 θ } , E θ , μ ( g a ) = 2 ( a − θ ) 2 for g a ( x ) = 1 { x ≥ − 4 a } , ∣ a ∣ ≤ δ / 8. (33) g_{\theta,\mu}^{\star}(x)=1\{x\ge-4\theta\},\qquad\mathcal{E}_{\theta,\mu}(g_a)=2(a-\theta)^2\quad\text{for }g_a(x)=1\{x\ge-4a\},\ |a|\le\delta/8. \tag*{(33)} g θ , μ ⋆ ( x ) = 1 { x ≥ − 4 θ } , E θ , μ ( g a ) = 2 ( a − θ ) 2 for g a ( x ) = 1 { x ≥ − 4 a } , ∣ a ∣ ≤ δ /8. (33)
The second identity integrates the margin over the interval between the two thresholds, which lies entirely inside [ − δ , δ ] [-\delta,\delta] [ − δ , δ ] .
Information. Let v 0 ( x ) = η 0 ( x ) ( 1 − η 0 ( x ) ) = ( 4 − x 2 ) / 16 v_0(x)=\eta_0(x)(1-\eta_0(x))=(4-x^2)/16 v 0 ( x ) = η 0 ( x ) ( 1 − η 0 ( x )) = ( 4 − x 2 ) /16 . The reference scores are
s T = Y − η 0 ( X ) v 0 ( X ) , s N = b δ ( X ) s T . s_T=\frac{Y-\eta_0(X)}{v_0(X)},\qquad s_N=b_{\delta}(X)s_T. s T = v 0 ( X ) Y − η 0 ( X ) , s N = b δ ( X ) s T .
Since b δ 2 = b δ b_{\delta}^{2}=b_{\delta} b δ 2 = b δ , J T N = J N N > 0 J_{TN}=J_{NN}>0 J T N = J N N > 0 , so the nuisance-projection coefficient is one and s T e f f = 1 { ∣ X ∣ ≤ δ } s T s_T^{\mathrm{eff}}=1\{|X|\le\delta\}s_T s T eff = 1 { ∣ X ∣ ≤ δ } s T . Hence
J T T = ∫ − 1 1 8 4 − x 2 d x = 4 log 3 , F δ = ∫ − δ δ 8 4 − x 2 d x = 4 log 2 + δ 2 − δ . (34) J_{TT}=\int_{-1}^{1}\frac{8}{4-x^2}\,dx=4\log3,\qquad F_{\delta}=\int_{-\delta}^{\delta}\frac{8}{4-x^2}\,dx=4\log\frac{2+\delta}{2-\delta}. \tag*{(34)} J T T = ∫ − 1 1 4 − x 2 8 d x = 4 log 3 , F δ = ∫ − δ δ 4 − x 2 8 d x = 4 log 2 − δ 2 + δ . (34)
On [ − δ , δ ] [-\delta,\delta] [ − δ , δ ] , 2 ≤ 8 / ( 4 − x 2 ) ≤ 32 / 15 2\le8/(4-x^2)\le32/15 2 ≤ 8/ ( 4 − x 2 ) ≤ 32/15 , giving the asserted bounds on F δ F_{\delta} F δ .
Decision separation and distinguishability. For 0 < t ≤ δ / 8 0<t\le\delta/8 0 < t ≤ δ /8 , take P + t = P t , − t P_{+t}=P_{t,-t} P + t = P t , − t and P − t = P − t , t P_{-t}=P_{-t,t} P − t = P − t , t . Their regression functions are η 0 ± t 1 { ∣ x ∣ ≤ δ } \eta_0\pm t1\{|x|\le\delta\} η 0 ± t 1 { ∣ x ∣ ≤ δ } and their Bayes rules disagree on ( − 4 t , 4 t ) (-4t,4t) ( − 4 t , 4 t ) . The binary excess-risk identity therefore gives the exact separation
inf g { E + t ( g ) + E − t ( g ) } = 1 2 ∫ − 4 t 4 t ( 2 t − ∣ x ∣ 2 ) d x = 4 t 2 . (35) \inf_g\{\mathcal{E}_{+t}(g)+\mathcal{E}_{-t}(g)\}=\frac{1}{2}\int_{-4t}^{4t}\left(2t-\frac{|x|}{2}\right)\,dx=4t^2. \tag*{(35)} g inf { E + t ( g ) + E − t ( g )} = 2 1 ∫ − 4 t 4 t ( 2 t − 2 ∣ x ∣ ) d x = 4 t 2 . (35)
For Bernoulli parameters p , q p,q p , q , KL ( Ber ( p ) ∥ Ber ( q ) ) ≤ ( p − q ) 2 / [ q ( 1 − q ) ] \operatorname{KL}(\operatorname{Ber}(p)\mathbin\Vert\operatorname{Ber}(q))\le(p-q)^2/[q(1-q)] KL ( Ber ( p ) ∥ Ber ( q )) ≤ ( p − q ) 2 / [ q ( 1 − q )] . Here the alternatives agree outside the central region; inside, p − q = 2 t p-q=2t p − q = 2 t and q ∈ [ 5 / 16 , 11 / 16 ] q\in[5/16,11/16] q ∈ [ 5/16 , 11/16 ] . Since P ( ∣ X ∣ ≤ δ ) = δ P(|X|\le\delta)=\delta P ( ∣ X ∣ ≤ δ ) = δ and F δ ≥ 4 δ F_{\delta}\ge4\delta F δ ≥ 4 δ ,
KL ( P + t ∥ P − t ) ≤ 1024 55 δ t 2 ≤ 256 55 F δ t 2 < 5 F δ t 2 . (36) \operatorname{KL}(P_{+t}\mathbin\Vert P_{-t})\le\frac{1024}{55}\delta t^2\le\frac{256}{55}F_{\delta}t^2<5F_{\delta}t^2. \tag*{(36)} KL ( P + t ∥ P − t ) ≤ 55 1024 δ t 2 ≤ 55 256 F δ t 2 < 5 F δ t 2 . (36)
Theorem 1 thus applies with q = 2 q=2 q = 2 , c e = 4 c_e=4 c e = 4 , K e = 5 K_e=5 K e = 5 , t 0 , e = δ / 8 t_{0,e}=\delta/8 t 0 , e = δ /8 , and λ e = F δ \lambda_e=F_{\delta} λ e = F δ . Its proof gives
R δ ( n ) ≥ 1 2 e min { δ 2 64 , 1 5 n F δ } ≥ 1 128 e min { δ 2 , ( n δ ) − 1 } , \mathcal{R}_{\delta}(n)\ge\frac{1}{2e}\min\left\{\frac{\delta^2}{64},\frac{1}{5nF_{\delta}}\right\}\ge\frac{1}{128e}\min\{\delta^2,(n\delta)^{-1}\}, R δ ( n ) ≥ 2 e 1 min { 64 δ 2 , 5 n F δ 1 } ≥ 128 e 1 min { δ 2 , ( n δ ) − 1 } ,
using F δ ≤ 64 δ / 15 F_{\delta}\le64\delta/15 F δ ≤ 64 δ /15 in the last step.
Matching upper bound, including empty central samples. Let I i = 1 { ∣ X i ∣ ≤ δ } I_i = 1\{\lvert X_i\rvert\le\delta\} I i = 1 {∣ X i ∣ ≤ δ } , N = ∑ i I i ∼ Bin ( n , δ ) N = \sum_i I_i \sim\operatorname{Bin}(n,\delta) N = ∑ i I i ∼ Bin ( n , δ ) , and define
θ ^ = clip [ − δ / 8 , δ / 8 ] ( ∑ i = 1 n I i ( Y i − 1 / 2 − X i / 4 ) N + 1 ) . (37) \widehat{\theta} = \operatorname{clip}_{[-\delta/8,\delta/8]}\left(\frac{\sum_{i=1}^{n} I_i(Y_i - 1/2 - X_i/4)}{N+1}\right). \tag*{(37)} θ = clip [ − δ /8 , δ /8 ] ( N + 1 ∑ i = 1 n I i ( Y i − 1/2 − X i /4 ) ) . (37)
This estimator is zero when N = 0 N = 0 N = 0 . Conditional on the feature observations, each selected residual has mean θ \theta θ , variance at most 1 / 4 1/4 1/4 , and independent noise. Before clipping, its mean-squared error is at most ( N / 4 + θ 2 ) / ( N + 1 ) 2 ≤ 1 / [ 4 ( N + 1 ) ] (N/4 + \theta^2)/(N+1)^2 \le1/[4(N+1)] ( N /4 + θ 2 ) / ( N + 1 ) 2 ≤ 1/ [ 4 ( N + 1 )] ; clipping cannot increase squared error because the interval contains θ \theta θ . The identity
E 1 N + 1 = 1 − ( 1 − δ ) n + 1 ( n + 1 ) δ \mathbb{E}\frac{1}{N+1} = \frac{1-(1-\delta)^{n+1}}{(n+1)\delta} E N + 1 1 = ( n + 1 ) δ 1 − ( 1 − δ ) n + 1
and Equation (A.18) give the second upper bound in Equation (A.17). The fixed rule g 0 g_0 g 0 gives the first, since 2 θ 2 ≤ δ 2 / 32 2\theta^2 \le\delta^2/32 2 θ 2 ≤ δ 2 /32 . Taking the better of these two known rules proves the stated upper bound. □
Only the central observations distinguish the least-favorable task alternatives: their expected number is n δ n\delta n δ . The inverse-information rate is active once n δ 3 n\delta^3 n δ 3 is sufficiently large; otherwise the local decision scale δ 2 \delta^2 δ 2 caps the risk. The nuisance parameter here changes conditional probabilities outside the boundary region while preserving the Bayes rule at fixed θ \theta θ . This is a statistical learning mechanism, not a mathematical model of the image-background intervention.
For 0 < α < 1 0 < \alpha< 1 0 < α < 1 and nonnegative Z Z Z , the CVaR minimization formula holds for integrable Z Z Z [9 Theorem 10] ; the threshold-weight argument below gives its dual:
CVaR α ( Z ) = inf τ ∈ R { τ + E [ ( Z − τ ) + ] 1 − α } = sup ξ ∈ W α E [ ξ Z ] , \operatorname{CVaR}_{\alpha}(Z) = \inf_{\tau\in\mathbb{R}}\left\{\tau+ \frac{\mathbb{E}[(Z-\tau)_+]}{1-\alpha}\right\} = \sup_{\xi\in\mathcal{W}_{\alpha}} \mathbb{E}[\xi Z], CVaR α ( Z ) = τ ∈ R inf { τ + 1 − α E [( Z − τ ) + ] } = ξ ∈ W α sup E [ ξ Z ] ,
W α = { ξ : 0 ≤ ξ ≤ ( 1 − α ) − 1 , E ξ = 1 } . (38) \mathcal{W}_{\alpha} = \{\xi: 0 \le\xi\le(1-\alpha)^{-1},\ \mathbb{E}\xi= 1\}. \tag*{(38)} W α = { ξ : 0 ≤ ξ ≤ ( 1 − α ) − 1 , E ξ = 1 } . (38)
Both values are + ∞ +\infty + ∞ if E Z = + ∞ \mathbb{E}Z = +\infty E Z = + ∞ . For integrable Z Z Z , a maximizing weight equals ( 1 − α ) − 1 (1-\alpha)^{-1} ( 1 − α ) − 1 above
v α = inf { v : Pr ( Z ≤ v ) ≥ α } , v_{\alpha} = \inf\{v : \Pr(Z \le v) \ge\alpha\}, v α = inf { v : Pr ( Z ≤ v ) ≥ α } ,
zero below it, and fills the remaining mass at the threshold. Such fractional weighting is possible because Pr ( Z > v α ) ≤ 1 − α ≤ Pr ( Z ≥ v α ) \Pr(Z > v_{\alpha}) \le1-\alpha\le\Pr(Z \ge v_{\alpha}) Pr ( Z > v α ) ≤ 1 − α ≤ Pr ( Z ≥ v α ) . The dual gives monotonicity, positive homogeneity, and v α ≤ CVaR α ( Z ) v_{\alpha} \le\operatorname{CVaR}_{\alpha}(Z) v α ≤ CVaR α ( Z ) . For the finite audit distribution Π G \Pi_G Π G , let π g = Π G ( { g } ) ≥ 0 \pi_g = \Pi_G(\{g\}) \ge0 π g = Π G ({ g }) ≥ 0 , ∑ g π g = 1 \sum_g \pi_g = 1 ∑ g π g = 1 . Then
CVaR α , π ( Z g ) = max ∑ g ω g = 1 0 ≤ ω g ≤ π g / ( 1 − α ) ∑ g ω g Z g . (39) \operatorname{CVaR}_{\alpha,\pi}(Z_g) = \max_{\substack{\sum_g \omega_g=1\\0\le\omega_g\le\pi_g/(1-\alpha)}} \sum_g \omega_g Z_g. \tag*{(39)} CVaR α , π ( Z g ) = ∑ g ω g = 1 0 ≤ ω g ≤ π g / ( 1 − α ) max g ∑ ω g Z g . (39)
These normalized tail weights, including threshold atoms, also govern Appendix § 15 .§ 4 .
Proof of Proposition 1. Let ε > 0 \varepsilon> 0 ε > 0 and λ e ∈ [ 0 , ∞ ) \lambda_e \in[0,\infty) λ e ∈ [ 0 , ∞ ) be finite almost surely. Write L e ( ε ) = ( λ e + ε ) − 1 L_e^{(\varepsilon)} = (\lambda_e+\varepsilon)^{-1} L e ( ε ) = ( λ e + ε ) − 1 and I α F G ( ε ) I_{\alpha}^{\mathrm{FG}}(\varepsilon) I α FG ( ε ) for the certificate in Equation (10) . If I α F G ( ε ) ≥ ρ > ε I_{\alpha}^{\mathrm{FG}}(\varepsilon) \ge\rho> \varepsilon I α FG ( ε ) ≥ ρ > ε , then
v α ( L ( ε ) ) ≤ CVaR α ( L ( ε ) ) = [ I α F G ( ε ) ] − 1 ≤ ρ − 1 , v_{\alpha}(L^{(\varepsilon)}) \le\operatorname{CVaR}_{\alpha}(L^{(\varepsilon)}) = [I_{\alpha}^{\mathrm{FG}}(\varepsilon)]^{-1} \le\rho^{-1}, v α ( L ( ε ) ) ≤ CVaR α ( L ( ε ) ) = [ I α FG ( ε ) ] − 1 ≤ ρ − 1 ,
Pr e ∼ Π ( λ e ≥ ρ − ε ) = Pr e ∼ Π ( L e ( ε ) ≤ ρ − 1 ) ≥ α . (40) \Pr_{e\sim\Pi}(\lambda_e \ge\rho-\varepsilon) = \Pr_{e\sim\Pi}(L_e^{(\varepsilon)} \le\rho^{-1}) \ge\alpha. \tag*{(40)} e ∼ Π Pr ( λ e ≥ ρ − ε ) = e ∼ Π Pr ( L e ( ε ) ≤ ρ − 1 ) ≥ α . (40)
The last step is the quantile definition and includes equality at the information threshold. □
Since 0 < L ( ε ) ≤ ε − 1 0 < L^{(\varepsilon)} \le\varepsilon^{-1} 0 < L ( ε ) ≤ ε − 1 , always I α F G ( ε ) ≥ ε I_{\alpha}^{\mathrm{FG}}(\varepsilon) \ge\varepsilon I α FG ( ε ) ≥ ε ; only ρ > ε \rho> \varepsilon ρ > ε certifies a strictly positive floor. With four equally weighted groups and α = 0.95 \alpha= 0.95 α = 0.95 , one group already contains the entire upper-tail mass, so
CVaR 0.95 ( L ( ε ) ) = max g L g ( ε ) , I 0.95 F G ( ε ) = min g λ g + ε . (41) \operatorname{CVaR}_{0.95}(L^{(\varepsilon)}) = \max_g L_g^{(\varepsilon)}, \qquad I_{0.95}^{\mathrm{FG}}(\varepsilon) = \min_g \lambda_g + \varepsilon. \tag*{(41)} CVaR 0.95 ( L ( ε ) ) = g max L g ( ε ) , I 0.95 FG ( ε ) = g min λ g + ε . (41)
Exact limit and positivity. Set L ( 0 ) = λ − 1 L^{(0)}=\lambda^{-1} L ( 0 ) = λ − 1 , with 1 / 0 = + ∞ 1/0=+\infty 1/0 = + ∞ . As ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0 , L ( ε ) ↑ L ( 0 ) L^{(\varepsilon)}\uparrow L^{(0)} L ( ε ) ↑ L ( 0 ) , and monotone convergence applies to E [ ξ L ( ε ) ] \mathbb{E}[\xi L^{(\varepsilon)}] E [ ξ L ( ε ) ] for every feasible ξ \xi ξ . The supremum over ξ \xi ξ commutes with the increasing supremum along any sequence ε k ↓ 0 \varepsilon_k\downarrow0 ε k ↓ 0 , giving
CVaR α ( L ( ε ) ) ↑ CVaR α ( λ − 1 ) , \operatorname{CVaR}_{\alpha}(L^{(\varepsilon)})\uparrow\operatorname{CVaR}_{\alpha}(\lambda^{-1}), CVaR α ( L ( ε ) ) ↑ CVaR α ( λ − 1 ) ,
I α F G ( ε ) ↓ I α F G ( 0 ) : = [ CVaR α ( λ − 1 ) ] − 1 , 1 / ( + ∞ ) = 0. (42) I_{\alpha}^{\mathrm{FG}}(\varepsilon)\downarrow I_{\alpha}^{\mathrm{FG}}(0):=\left[\operatorname{CVaR}_{\alpha}(\lambda^{-1})\right]^{-1},\qquad1/(+\infty)=0. \tag*{(42)} I α FG ( ε ) ↓ I α FG ( 0 ) := [ CVaR α ( λ − 1 ) ] − 1 , 1/ ( + ∞ ) = 0. (42)
For nonnegative Z Z Z , the feasible weight ξ = 1 \xi=1 ξ = 1 and variational choice τ = 0 \tau=0 τ = 0 imply
E Z ≤ CVaR α ( Z ) ≤ E Z 1 − α , I α F G ( 0 ) > 0 ⟺ E [ λ − 1 ] < ∞ . (43) \mathbb{E}Z\leq\operatorname{CVaR}_{\alpha}(Z)\leq\frac{\mathbb{E}Z}{1-\alpha},\qquad I_{\alpha}^{\mathrm{FG}}(0)>0\Longleftrightarrow\mathbb{E}[\lambda^{-1}]<\infty. \tag*{(43)} E Z ≤ CVaR α ( Z ) ≤ 1 − α E Z , I α FG ( 0 ) > 0 ⟺ E [ λ − 1 ] < ∞. (43)
Under the geometric tail assumptions, this is equivalent to β = ( c + ν ) / ( 2 m ) > 1 \beta=(c+\nu)/(2m)>1 β = ( c + ν ) / ( 2 m ) > 1 ; the exact certificate vanishes for β ≤ 1 \beta\leq1 β ≤ 1 . Information coverage is a population statement under Π \Pi Π , separate from the confidence level of an estimated lower certificate.
Estimation and Certification
The bounds condition on a predictor, coordinates, calibration, and slices fixed independently of evaluation data. Evaluation-fitted normalization requires additional error control. Implementation details and numerical records appear in the artifact’s docs/extended_appendix_details/ (see the artifact availability statement).
Operational Models and Matrix Targets
For evaluation distribution P g P_g P g and calibrated K K K -class logits ℓ w \ell_w ℓ w , maps Φ T ( x ) ∈ R K × d T \Phi_T(x)\in\mathbb{R}^{K\times d_T} Φ T ( x ) ∈ R K × d T and Φ N ( x ) ∈ R K × d N \Phi_N(x)\in\mathbb{R}^{K\times d_N} Φ N ( x ) ∈ R K × d N convert task and nuisance parameters into logit changes:
ℓ θ , μ ( x ) = ℓ w ( x ) + Φ T ( x ) θ + Φ N ( x ) μ , D = ( Φ T Φ N ) , d = d T + d N . (44) \ell_{\theta,\mu}(x)=\ell_w(x)+\Phi_T(x)\theta+\Phi_N(x)\mu,\qquad D=(\Phi_T\ \Phi_N),\qquad d=d_T+d_N. \tag*{(44)} ℓ θ , μ ( x ) = ℓ w ( x ) + Φ T ( x ) θ + Φ N ( x ) μ , D = ( Φ T Φ N ) , d = d T + d N . (44)
With p = softmax ( ℓ w ) p=\operatorname{softmax}(\ell_w) p = softmax ( ℓ w ) , H = diag ( p ) − p p ⊤ H=\operatorname{diag}(p)-pp^{\top} H = diag ( p ) − p p ⊤ , one-hot label vector e Y e_Y e Y , and normalized feature coordinates v = ( v T ⊤ v N ⊤ ) ⊤ v=(v_T^{\top}\ v_N^{\top})^{\top} v = ( v T ⊤ v N ⊤ ) ⊤ , distinguish
M p r e d = D ⊤ H D , M o b s = s s ⊤ , s = D ⊤ ( e Y − p ) , M^{\mathrm{pred}}=D^{\top}HD,\qquad M^{\mathrm{obs}}=ss^{\top},\qquad s=D^{\top}(e_Y-p), M pred = D ⊤ H D , M obs = s s ⊤ , s = D ⊤ ( e Y − p ) ,
M f e a t = v v ⊤ , J g = E P g [ M ] . (45) M^{\mathrm{feat}}=vv^{\top},\qquad J_g=\mathbb{E}_{P_g}[M]. \tag*{(45)} M feat = v v ⊤ , J g = E P g [ M ] . (45)
For a binary scalar logit ℓ θ , μ = ℓ w + θ z + μ b \ell_{\theta,\mu}=\ell_w+\theta z+\mu b ℓ θ , μ = ℓ w + θ z + μ b , this gives M p r e d = p ( 1 − p ) v v ⊤ M^{\mathrm{pred}}=p(1-p)vv^{\top} M pred = p ( 1 − p ) v v ⊤ , with p = σ ( ℓ w ) p=\sigma(\ell_w) p = σ ( ℓ w ) and v = ( z , b ) ⊤ v=(z,b)^{\top} v = ( z , b ) ⊤ . Predictive scores have conditional mean zero under the fitted label distribution even when features do not; their Schur complement is efficient information for this operational model. Observed-label moments equal predictive Fisher under conditional correct specification, P g ( Y = ⋅ ∣ X ) = p ( X ) P_g(Y=\cdot\mid X)=p(X) P g ( Y = ⋅ ∣ X ) = p ( X ) ; otherwise their centered covariance subtracts E [ s ] E [ s ] ⊤ \mathbb{E}[s]\mathbb{E}[s]^{\top} E [ s ] E [ s ] ⊤ . Feature geometry satisfies J g = Σ g + μ g μ g ⊤ J_g=\Sigma_g+\mu_g\mu_g^{\top} J g = Σ g + μ g μ g ⊤ , with μ g = E [ v ] \mu_g=\mathbb{E}[v] μ g = E [ v ] and Σ g = Cov ( v ) \Sigma_g=\operatorname{Cov}(v) Σ g = Cov ( v ) . The prospective centered comparison uses within-slice covariance Σ g \Sigma_g Σ g with divisor n g n_g n g and unchanged global scales, removing the mean contribution without rescaling.
Prospective selection uses uncentered feature–Schur fractions; Appendix § 14 .§ 2 compares centered and predictive targets. Within-group normalization of p ( 1 − p ) p(1-p) p ( 1 − p ) weights changes the effective nuisance ridge and differs from the predictive moment. Hidden discovery leaves this factor unnormalized and shrinks toward the same-class pooled matrix, with η = 0.10 \eta=0.10 η = 0.10 . Removing ridge leaves the declared matrix, including shrinkage, unchanged. Normalization conventions and nuisance-specification checks appear in Appendices § 14 and § 14 .§ 5 .
Operational subspaces and cost. A task coordinate specifies the predictive contrast to be audited; nuisance coordinates specify variations expected to preserve it. Candidate constructions include an answer-logit contrast with controlled prompt-template variation for a transformer, a region-level class contrast with validated appearance changes for segmentation, and a conditional denoising-score contrast for a diffusion model. Logits, probes, intermediate representations, and low-rank parameter perturbations provide finite-dimensional coordinates. For total operational dimension d = d T + d N d=d_T+d_N d = d T + d N , dense moment storage costs O ( d 2 ) O(d^2) O ( d 2 ) , and direct factorization or eigendecomposition costs O ( d 3 ) O(d^3) O ( d 3 ) per slice. Minibatch accumulation and iterative eigensolvers support larger operational subspaces; feature or score extraction remains dependent on the backbone and input size. These examples are candidate extensions for future evaluation.
Admission of new nuisance specifications. A proposed admission protocol prespecifies independent checks of nuisance relevance, task preservation, split stability, and nuisance-block conditioning. An identifiability conclusion requires all four checks to pass; a failed prerequisite is recorded as an unvalidated specification. For a finite admitted family, report the minimum certificate with uncertainty accounting for the family. The present experiments evaluate prespecified nuisance hypotheses, with implemented checks and sensitivity analyses in Appendices B.4, C, and D.5. The general admission procedure remains to be evaluated.
Ridge Regularization and the Unregularized Target
For J = ( A B B T C ) ⪰ 0 J = \begin{pmatrix} A & B \\ B^{\mathsf{T}} & C \end{pmatrix} \succeq0 J = ( A B T B C ) ⪰ 0 , with A = J T T A = J_{TT} A = J T T , B = J T N B = J_{TN} B = J T N , C = J N N ≻ 0 C = J_{NN} \succ0 C = J N N ≻ 0 , and λ r i d > 0 \lambda_{\mathrm{rid}} > 0 λ rid > 0 , define
S 0 ( J ) = A − B C − 1 B T , S r i d ( J ) = A − B ( C + λ r i d I ) − 1 B T . (46) S_0(J) = A - BC^{-1}B^{\mathsf{T}}, \qquad S_{\mathrm{rid}}(J) = A - B(C + \lambda_{\mathrm{rid}}I)^{-1}B^{\mathsf{T}}. \tag*{(46)} S 0 ( J ) = A − B C − 1 B T , S rid ( J ) = A − B ( C + λ rid I ) − 1 B T . (46)
Both are positive semidefinite. The resolvent identity, whose inverse factors commute, gives
Δ r i d ( J ) : = S r i d ( J ) − S 0 ( J ) = λ r i d B C − 1 ( C + λ r i d I ) − 1 B T ⪰ 0 , \Delta_{\mathrm{rid}}(J) := S_{\mathrm{rid}}(J) - S_0(J) = \lambda_{\mathrm{rid}}BC^{-1}(C + \lambda_{\mathrm{rid}}I)^{-1}B^{\mathsf{T}} \succeq0, Δ rid ( J ) := S rid ( J ) − S 0 ( J ) = λ rid B C − 1 ( C + λ rid I ) − 1 B T ⪰ 0 ,
∥ Δ r i d ( J ) ∥ o p ≤ λ r i d M 2 κ ( κ + λ r i d ) if C ⪰ κ I , κ > 0 , ∥ B ∥ o p ≤ M . (47) \lVert\Delta_{\mathrm{rid}}(J)\rVert_{\mathrm{op}} \le\frac{\lambda_{\mathrm{rid}}M^2}{\kappa(\kappa+ \lambda_{\mathrm{rid}})}
\quad\text{if } C \succeq\kappa I,\ \kappa> 0,\ \lVert B\rVert_{\mathrm{op}} \le M. \tag*{(47)} ∥ Δ rid ( J ) ∥ op ≤ κ ( κ + λ rid ) λ rid M 2 if C ⪰ κ I , κ > 0 , ∥ B ∥ op ≤ M . (47)
Conditioning is essential: scalar blocks A = 1 A = 1 A = 1 , B = c B = \sqrt{c} B = c , C = c > 0 C = c > 0 C = c > 0 give a positive-semidefinite joint matrix with S 0 = 0 S_0 = 0 S 0 = 0 and S r i d = λ r i d / ( c + λ r i d ) → 1 S_{\mathrm{rid}} = \lambda_{\mathrm{rid}}/(c + \lambda_{\mathrm{rid}}) \to1 S rid = λ rid / ( c + λ rid ) → 1 as c / λ r i d ↓ 0 c/\lambda_{\mathrm{rid}} \downarrow0 c / λ rid ↓ 0 . Comparisons therefore retain the declared standardized nuisance coordinates and ridge value.
Error Allowances and Lower Certificates
Proposition B.1 (Simultaneous lower certificate). Fix G G G groups and weights π g > 0 \pi_g > 0 π g > 0 summing to one. Within each group, let M g i M_{gi} M g i be i.i.d. with mean J g J_g J g , 0 ⪯ M g i ⪯ R g I d 0 \preceq M_{gi} \preceq R_g I_d 0 ⪯ M g i ⪯ R g I d , for specified finite R g R_g R g , and J ^ g = n g − 1 ∑ i M g i \widehat{J}_g = n_g^{-1}\sum_i M_{gi} J g = n g − 1 ∑ i M g i . For 0 < δ < 1 0 < \delta< 1 0 < δ < 1 , set
u δ = log ( 2 d G / δ ) , t g = R g ( 2 u δ n g + 2 u δ 3 n g ) , u_\delta= \log(2dG/\delta), \qquad t_g = R_g\left(\sqrt{\frac{2u_\delta}{n_g}} + \frac{2u_\delta}{3n_g}\right), u δ = log ( 2 d G / δ ) , t g = R g ( n g 2 u δ + 3 n g 2 u δ ) , E δ = ⋂ g { ∥ J ^ g − J g ∥ o p ≤ t g } , Pr ( E δ ) ≥ 1 − δ . (48) \mathcal{E}_\delta= \bigcap_g\{\lVert\widehat{J}_g - J_g\rVert_{\mathrm{op}} \le t_g\}, \qquad\Pr(\mathcal{E}_\delta) \ge1 - \delta. \tag*{(48)} E δ = g ⋂ {∥ J g − J g ∥ op ≤ t g } , Pr ( E δ ) ≥ 1 − δ . (48) Assume each C g ≻ 0 C_g \succ0 C g ≻ 0 . Define observable bounds and allowances by
M ‾ g = ∥ B ^ g ∥ o p + t g , κ g = λ min ( C ^ g ) − t g , \overline{M}_g = \lVert\widehat{B}_g\rVert_{\mathrm{op}} + t_g, \qquad\kappa_g = \lambda_{\min}(\widehat{C}_g) - t_g, M g = ∥ B g ∥ op + t g , κ g = λ m i n ( C g ) − t g , h r i d ( t , M ) = t + 2 M t + t 2 λ r i d + M 2 t λ r i d 2 , a g = h r i d ( t g , M ‾ g ) , h_{\mathrm{rid}}(t,M) = t + \frac{2Mt + t^2}{\lambda_{\mathrm{rid}}} + \frac{M^2t}{\lambda_{\mathrm{rid}}^2}, \qquad a_g = h_{\mathrm{rid}}(t_g,\overline{M}_g), h rid ( t , M ) = t + λ rid 2 M t + t 2 + λ rid 2 M 2 t , a g = h rid ( t g , M g ) , d g = λ r i d M ‾ g 2 κ g ( κ g + λ r i d ) when all κ g > 0. (49) d_g = \frac{\lambda_{\mathrm{rid}}\overline{M}_g^2}{\kappa_g(\kappa_g + \lambda_{\mathrm{rid}})}
\qquad\text{when all }\kappa_g > 0. \tag*{(49)} d g = κ g ( κ g + λ rid ) λ rid M g 2 when all κ g > 0. (49) Report the lower certificate in (14) with audit distribution π \pi π only when these conditioning margins are positive; otherwise record conditioning failure. Its probability of exceeding the unregularized population certificate is at most δ \delta δ . On E δ \mathcal{E}_\delta E δ , a reported level I α , π ≥ ρ > ε I_{\alpha,\pi} \ge\rho> \varepsilon I α , π ≥ ρ > ε also implies
∑ g : λ min ( S 0 ( J g ) ) ≥ ρ − ε π g ≥ α . \sum_{g:\lambda_{\min}(S_0(J_g))\ge\rho-\varepsilon}\pi_g \ge\alpha. g : λ m i n ( S 0 ( J g )) ≥ ρ − ε ∑ π g ≥ α . Proof. For X g i = M g i − J g X_{gi} = M_{gi} - J_g X g i = M g i − J g , ∥ X g i ∥ o p ≤ R g \lVert X_{gi}\rVert_{\mathrm{op}} \le R_g ∥ X g i ∥ op ≤ R g and E [ X g i 2 ] = E [ M g i 2 ] − J g 2 ⪯ R g 2 I \mathbb{E}[X_{gi}^2] = \mathbb{E}[M_{gi}^2] - J_g^2 \preceq R_g^2I E [ X g i 2 ] = E [ M g i 2 ] − J g 2 ⪯ R g 2 I . Matrix Bernstein [23 Theorem 1.4] with the displayed t g t_g t g gives failure at most δ / G \delta/G δ / G per group; the union bound allows dependence between group estimates. On this event, block extraction and Weyl give ∥ B g ∥ o p ≤ M ‾ g \lVert B_g\rVert_{\mathrm{op}} \le\overline{M}_g ∥ B g ∥ op ≤ M g and C g ⪰ κ g I C_g \succeq\kappa_gI C g ⪰ κ g I , so (B.4) bounds the ridge gap by d g d_g d g .
For the estimation error, write Q = ( C + λ r i d I ) − 1 Q = (C + \lambda_{\mathrm{rid}}I)^{-1} Q = ( C + λ rid I ) − 1 , Q ^ = ( C ^ + λ r i d I ) − 1 \widehat{Q} = (\widehat{C} + \lambda_{\mathrm{rid}}I)^{-1} Q = ( C + λ rid I ) − 1 , and Δ B = B ^ − B \Delta B = \widehat{B} - B Δ B = B − B . Positive semidefiniteness and the resolvent identity give ∥ Q ∥ , ∥ Q ^ ∥ ≤ λ r i d − 1 \lVert Q\rVert, \lVert\widehat{Q}\rVert\le\lambda_{\mathrm{rid}}^{-1} ∥ Q ∥ , ∥ Q ∥ ≤ λ rid − 1 and ∥ Q ^ − Q ∥ ≤ t / λ r i d 2 \lVert\widehat{Q} - Q\rVert\le t/\lambda_{\mathrm{rid}}^2 ∥ Q − Q ∥ ≤ t / λ rid 2 . Decompose the quadratic-block difference as Δ B Q ^ B ^ T + B Q ^ Δ B T + B ( Q ^ − Q ) B T \Delta B\widehat{Q}\widehat{B}^{\mathsf{T}} + B\widehat{Q}\Delta B^{\mathsf{T}} + B(\widehat{Q} - Q)B^{\mathsf{T}} Δ B Q B T + B Q Δ B T + B ( Q − Q ) B T . If ∥ B ∥ ≤ M \lVert B\rVert\le M ∥ B ∥ ≤ M , its three norms are bounded by t ( M + t ) / λ r i d t(M+t)/\lambda_{\mathrm{rid}} t ( M + t ) / λ rid , M t / λ r i d Mt/\lambda_{\mathrm{rid}} M t / λ rid , and M 2 t / λ r i d 2 M^2t/\lambda_{\mathrm{rid}}^2 M 2 t / λ rid 2 . Adding the task-block error t t t proves the bound h r i d ( t , M ) h_{\mathrm{rid}}(t,M) h rid ( t , M ) and hence a g a_g a g .
Writing λ g 0 = λ min ( S 0 ( J g ) ) \lambda_g^0 = \lambda_{\min}(S_0(J_g)) λ g 0 = λ m i n ( S 0 ( J g )) , Weyl now yields
λ g 0 ≥ λ min ( S r i d ( J g ) ) − d g ≥ λ min ( S r i d ( J b g ) ) − a g − d g . (50) \lambda_g^0 \ge\lambda_{\min}(S_{\mathrm{rid}}(J_g)) - d_g \ge\lambda_{\min}(S_{\mathrm{rid}}(J_{\mathrm{bg}})) - a_g - d_g. \tag*{(50)} λ g 0 ≥ λ m i n ( S rid ( J g )) − d g ≥ λ m i n ( S rid ( J bg )) − a g − d g . (50)
Since S 0 ( J g ) ⪰ 0 S_0(J_g) \succeq0 S 0 ( J g ) ⪰ 0 , clipping in (14) preserves λ g ≤ λ g 0 \lambda_g \le\lambda_g^0 λ g ≤ λ g 0 . Thus monotonicity of CVaR and reciprocation give
I α , π ≤ I α , π 0 : = [ CVaR α , g ∼ π ( ( λ g 0 + ε ) − 1 ) ] − 1 . (51) I_{\alpha,\pi} \le I_{\alpha,\pi}^0 := \left[\operatorname{CVaR}_{\alpha,g\sim\pi}\left((\lambda_g^0+\varepsilon)^{-1}\right)\right]^{-1}. \tag*{(51)} I α , π ≤ I α , π 0 := [ CVaR α , g ∼ π ( ( λ g 0 + ε ) − 1 ) ] − 1 . (51)
The information-coverage conclusion follows from Appendix § 11 .§ 5 . □
Confidence concerns held-out observations; coverage concerns the fixed weights π \pi π . A finite pre-declared nuisance family can be included in the union bound over group–specification pairs; the infimum of certificates preserves coverage separately for each specification. Appendix § 13 .§ 4 supplies matrix allowances for frozen soft gates, including random membership normalization, pooling, and support correction, to which the same deterministic argument applies.
Implemented Audit and Numerical Comparison
Tables B.1–B.2 audit uncentered feature moments of normalized scalar task and nuisance coordinates z i z_i z i , b i b_i b i , with uniform within-group weights. Separately from the finite-sample proposition, Gaussian/Wald simulation uses 10 5 10^5 1 0 5 moment draws per group at nominal 95% confidence. For general normalized weights w i w_i w i and u i = ( z i 2 , z i b i , b i 2 ) T u_i=(z_i^2,z_i b_i,b_i^2)^\mathsf{T} u i = ( z i 2 , z i b i , b i 2 ) T , draw m ∗ = ( A ∗ , B ∗ , C ∗ ) T ∼ N ( m ^ , Σ ^ m ^ ) m^*=(A^*,B^*,C^*)^\mathsf{T}\sim N(\widehat{m},\widehat{\Sigma}_{\widehat{m}}) m ∗ = ( A ∗ , B ∗ , C ∗ ) T ∼ N ( m , Σ m ) , where
m ^ = ∑ i w i u i , Σ ^ m ^ = ∑ i w i 2 ( u i − m ^ ) ( u i − m ^ ) T 1 − ∑ i w i 2 . (52) \widehat{m}=\sum_i w_i u_i,\qquad
\widehat{\Sigma}_{\widehat{m}}=\frac{\sum_i w_i^2(u_i-\widehat{m})(u_i-\widehat{m})^\mathsf{T}}{1-\sum_i w_i^2}. \tag*{(52)} m = i ∑ w i u i , Σ m = 1 − ∑ i w i 2 ∑ i w i 2 ( u i − m ) ( u i − m ) T . (52)
Uniform weights give sample covariance divided by n n n . Simulation fixes realized coordinate-standardization constants and weights, omitting evaluation-fitted standardization error. Bonferroni tail δ W = 0.05 / ( 3 G ) \delta_W=0.05/(3G) δ W = 0.05/ ( 3 G ) covers nuisance conditioning, ridge lower information, and an upper ridge gap within a run (G = 4 G=4 G = 4 Waterbirds; 18 18 18 CivilComments).
Let J ∗ J^* J ∗ have blocks ( A ∗ , B ∗ , C ∗ ) (A^*,B^*,C^*) ( A ∗ , B ∗ , C ∗ ) , and write F ^ g 0 = S 0 ( J ^ g ) \widehat{F}_g^0=S_0(\widehat{J}_g) F g 0 = S 0 ( J g ) , F ^ g r i d = S r i d ( J ^ g ) \widehat{F}_g^{\mathrm{rid}}=S_{\mathrm{rid}}(\widehat{J}_g) F g rid = S rid ( J g ) . The δ W \delta_W δ W -quantiles C g L , L g r i d , L g 0 C_g^L,L_g^{\mathrm{rid}},L_g^0 C g L , L g rid , L g 0 of C ∗ C^* C ∗ , [ S r i d ( J ∗ ) ] + [S_{\mathrm{rid}}(J^*)]_+ [ S rid ( J ∗ ) ] + , [ S 0 ( J ∗ ) ] + [S_0(J^*)]_+ [ S 0 ( J ∗ ) ] + are clipped to [ 0 , C ^ g ] [0,\widehat{C}_g] [ 0 , C g ] , [ 0 , F ^ g r i d ] [0,\widehat{F}_g^{\mathrm{rid}}] [ 0 , F g rid ] , [ 0 , F ^ g 0 ] [0,\widehat{F}_g^0] [ 0 , F g 0 ] , respectively. The ( 1 − δ W ) (1-\delta_W) ( 1 − δ W ) -quantile U g Δ U_g^\Delta U g Δ of the closed-form ridge gap is enlarged to at least its plug-in value. Lower and upper quantiles use lower and higher order statistics, respectively. The implemented lower information is
λ g W = min { [ L g r i d − U g Δ ] + , L g 0 , F ^ g 0 } . (53) \lambda_g^{\mathrm{W}}=\min\{[L_g^{\mathrm{rid}}-U_g^\Delta]_+,L_g^0,\widehat{F}_g^0\}. \tag*{(53)} λ g W = min {[ L g rid − U g Δ ] + , L g 0 , F g 0 } . (53)
A denominator C ∗ C^* C ∗ or C ∗ + λ r i d C^*+\lambda_{\mathrm{rid}} C ∗ + λ rid at or below 10 − 12 10^{-12} 1 0 − 12 gives zero for its corresponding Schur value and an infinite gap; otherwise (B.4) is used even for non-PSD draws. An infinite upper gap or C g L ≤ 10 − 12 C_g^L\le10^{-12} C g L ≤ 1 0 − 12 forces zero and an uncertifiable status; support, empirical-PSD, and identity checks can also prevent certification. With λ r i d = 10 − 3 \lambda_{\mathrm{rid}}=10^{-3} λ rid = 1 0 − 3 , certificates with superscripts 0 0 0 , rid, and W aggregate F ^ g 0 \widehat{F}_g^0 F g 0 , F ^ g r i d \widehat{F}_g^{\mathrm{rid}} F g rid , and λ g W \lambda_g^{\mathrm{W}} λ g W , respectively, using uniform environment weights, α = 0.95 \alpha=0.95 α = 0.95 , ε = 10 − 8 \varepsilon=10^{-8} ε = 1 0 − 8 , and fractional CVaR threshold-atom mass.
Table B.1. Same-cohort summaries of 75 default-ridge held-out audits. Conditioning/gap parentheses are within-run nominal Wald lower/upper values; entries are extrema across runs, not joint confidence statements across seeds or methods. Inflation is ridge minus exact plug-in certificate.
Every run passed conditioning and satisfied I 0.95 W ≤ I ^ 0.95 0 ≤ I ^ 0.95 r i d I^{W}_{0.95} \le\widehat{I}^{0}_{0.95} \le\widehat{I}^{\mathrm{rid}}_{0.95} I 0.95 W ≤ I 0.95 0 ≤ I 0.95 rid ; Waterbirds diagnostic values are 0.1504, 0.1639, and 0.1646. Across the five CivilComments diagnostic seeds, the mean ridge and unregularized certificates are 0.4993 and 0.4991, respectively. The inequality I 0.95 W ≤ I ^ 0.95 0 I^{W}_{0.95} \le\widehat{I}^{0}_{0.95} I 0.95 W ≤ I 0.95 0 is enforced by the caps and does not establish confidence coverage. Training audits use fresh models: original training runs lacked the retained environment blocks needed for checkpoint-specific auditing.
For the separate audits in Table B.2, frozen Group DRO + FG maximizes validation certificate within two percentage points of the best validation worst-group accuracy among { 0.1 , 0.3 } \{0.1, 0.3\} { 0.1 , 0.3 } ; seeds 17–21 select (0.3, 0.1, 0.3, 0.3, 0.3). Repeated frozen-feature training/audit runs at ridges 10 − 4 10^{-4} 1 0 − 4 , 10 − 3 10^{-3} 1 0 − 3 , 10 − 2 10^{-2} 1 0 − 2 give maximum group gaps 0.0001443, 0.0014374, 0.0138286 and Wald upper values 0.0001996, 0.0019880, 0.0191165.
Table B.2: Separate fresh training audits: five-seed means at the stated coefficients. Table B.1 instead includes frozen models selected over the full grid { 0.01 , 0.03 , 0.1 , 0.3 } \{0.01, 0.03, 0.1, 0.3\} { 0.01 , 0.03 , 0.1 , 0.3 } .
Table B.2.
Hidden-Slice Discovery
Appendix § 14 .§ 4 reports this protocol’s outcomes. Frozen-gate evaluation recomputes membership normalization and support correction. Prompts, model revision and execution details are in the artifact.
Gate Family and Support Constraints
For the frozen standardized 576-dimensional representation ψ ( x ) \psi(x) ψ ( x ) and task class c c c , define a logistic gate with parameters ϕ = ( a c , b c ) \phi= (a_c, b_c) ϕ = ( a c , b c ) :
h ϕ , c ( x ) = σ ( a c T ψ ( x ) + b c ) , q i 0 = h ϕ , c ( x i ) ∑ j ∈ D c h ϕ , c ( x j ) , σ ( t ) = ( 1 + e − t ) − 1 , h_{\phi,c}(x) = \sigma(a_c^\mathsf{T}\psi(x) + b_c), \qquad q_i^0 = \frac{h_{\phi,c}(x_i)}{\sum_{j \in D_c} h_{\phi,c}(x_j)}, \qquad\sigma(t) = (1 + e^{-t})^{-1}, h ϕ , c ( x ) = σ ( a c T ψ ( x ) + b c ) , q i 0 = ∑ j ∈ D c h ϕ , c ( x j ) h ϕ , c ( x i ) , σ ( t ) = ( 1 + e − t ) − 1 ,
m n o m = τ s l i c e n c , m E S S = min { n c , max { 1 , 0.8 m n o m } } , b max = min { 1 , 2 / m n o m } . m_{\mathrm{nom}} = \tau_{\mathrm{slice}} n_c, \qquad m_{\mathrm{ESS}} = \min\{n_c, \max\{1, 0.8m_{\mathrm{nom}}\}\}, \qquad b_{\max} = \min\{1, 2/m_{\mathrm{nom}}\}. m nom = τ slice n c , m ESS = min { n c , max { 1 , 0.8 m nom }} , b m a x = min { 1 , 2/ m nom } .
Here D c D_c D c contains the current sample’s examples with task label c c c , n c = ∣ D c ∣ n_c = |D_c| n c = ∣ D c ∣ , and τ s l i c e = 0.10 \tau_{\mathrm{slice}} = 0.10 τ slice = 0.10 is within-class mass. Discovery calibrates b c b_c b c to this mean membership; evaluation freezes the selected intercept, without re-centering or mass recalibration. The cap denominator is nominal slice size, not the ESS floor.
Support correction uses Euclidean projection of q 0 q^0 q 0 onto the capped simplex only if its maximum exceeds b max b_{\max} b m a x . Writing the result as q c a p q^{\mathrm{cap}} q cap and u i = 1 / n c u_i = 1/n_c u i = 1/ n c , it takes q ~ = u + t ( q c a p − u ) \widetilde{q} = u + t(q^{\mathrm{cap}} - u) q = u + t ( q cap − u ) with the largest t ∈ [ 0 , 1 ] t \in[0,1] t ∈ [ 0 , 1 ] satisfying ∑ i q ~ i 2 ≤ 1 / m E S S \sum_i \widetilde{q}_i^2 \le1/m_{\mathrm{ESS}} ∑ i q i 2 ≤ 1/ m ESS . Thus q ~ ≥ 0 \widetilde{q} \ge0 q ≥ 0 , ∑ i q ~ i = 1 \sum_i \widetilde{q}_i = 1 ∑ i q i = 1 , max i q ~ i ≤ b max \max_i \widetilde{q}_i \le b_{\max} max i q i ≤ b m a x , and n e f f = ( ∑ i q ~ i 2 ) − 1 ≥ m E S S n_{\mathrm{eff}} = \left(\sum_i \widetilde{q}_i^2\right)^{-1} \ge m_{\mathrm{ESS}} n eff = ( ∑ i q i 2 ) − 1 ≥ m ESS . These constraints do not imply nuisance conditions. Each example uses its own n c n_c n c , making q ~ \widetilde{q} q batch-dependent. Reported mass averages raw selected-class membership; ESS uses q ~ \widetilde{q} q .
Metrics and controls. Fisher and high-loss gates report accuracy, uncalibrated binary NLL (temperature one), and post-hoc composition averaged under q ~ \widetilde{q} q , without information pooling. Predicted labels use a zero-logit threshold; calibrated NLL is stored separately. Random controls weight k = ⌈ τ s l i c e n c ⌉ k = \lceil\tau_{\mathrm{slice}} n_c \rceil k = ⌈ τ slice n c ⌉ sampled rows of the Fisher-selected class uniformly; the constant gate uses 1 / n c 1/n_c 1/ n c . Controls using feature moments and centered covariances reevaluate the predictive-Fisher and high-loss gates without rediscovery.
Discovery Objective and Settings
With matrix M i M_i M i and same-class pool J ^ p o o l , c = n c − 1 ∑ i ∈ D c M i \widehat{J}_{\mathrm{pool},c}=n_c^{-1}\sum_{i\in D_c}M_i J pool , c = n c − 1 ∑ i ∈ D c M i , the ideal constrained objective searches over class–gate pairs:
J ^ ϕ , c ( η ) = ( 1 − η ) ∑ i q ~ i ( ϕ ) M i + η J ^ p o o l , c , \widehat{J}^{(\eta)}_{\phi,c}=(1-\eta)\sum_i \widetilde{q}_i(\phi)M_i+\eta\widehat{J}_{\mathrm{pool},c}, J ϕ , c ( η ) = ( 1 − η ) i ∑ q i ( ϕ ) M i + η J pool , c ,
( c ⋆ , ϕ ⋆ ) ∈ arg max c , ϕ { λ min ( S r i d ( J ^ ϕ , c ( η ) ) ) + ε } − 1 , 0 ≤ η < 1. (54) (c^\star,\phi^\star)\in\arg\max_{c,\phi}\left\{\lambda_{\min}\left(S_{\mathrm{rid}}\left(\widehat{J}^{(\eta)}_{\phi,c}\right)\right)+\varepsilon\right\}^{-1},\qquad0\leq\eta<1. \tag*{(54)} ( c ⋆ , ϕ ⋆ ) ∈ arg c , ϕ max { λ m i n ( S rid ( J ϕ , c ( η ) ) ) + ε } − 1 , 0 ≤ η < 1. (54)
Table C.1. Fixed hidden-discovery settings; sensitivity grids are declared before metadata reveal.
For the scalar task/nuisance blocks, write F ^ r i d = S r i d ( J ^ ϕ , c ( η ) ) \widehat{F}^{\mathrm{rid}}=S_{\mathrm{rid}}\left(\widehat{J}^{(\eta)}_{\phi,c}\right) F rid = S rid ( J ϕ , c ( η ) ) . Optimization and checkpoint/restart/class selection use − log ( F ^ r i d + ε ) − 10 ( h ˉ / τ s l i c e − 1 ) 2 -\log(\widehat{F}^{\mathrm{rid}}+\varepsilon)-10(\bar{h}/\tau_{\mathrm{slice}}-1)^2 − log ( F rid + ε ) − 10 ( h ˉ / τ slice − 1 ) 2 , where h ˉ = n c − 1 ∑ i h ϕ , c ( x i ) \bar{h}=n_c^{-1}\sum_i h_{\phi,c}(x_i) h ˉ = n c − 1 ∑ i h ϕ , c ( x i ) . The intercept is recalibrated before discovery evaluation; only mass-feasible finite checkpoints are retained, including a constant-gate baseline. High-loss search uses the same gate, mass and support rules, replacing negative log information with weighted, uncalibrated binary NLL. Information uses v i = ( z i , b i ) T v_i=(z_i,b_i)^{\mathsf{T}} v i = ( z i , b i ) T : z i z_i z i is the raw head logit f w ( x i ) f_w(x_i) f w ( x i ) standardized by fitting-subset moments, b i b_i b i the standardized CLIP nuisance score, and p i = σ ( f w ( x i ) / T ) p_i=\sigma(f_w(x_i)/T) p i = σ ( f w ( x i ) / T ) . Calibration changes the unnormalized predictive factor. Shrinkage pools complete matrices within the current sample’s class before taking the Schur complement. Feature controls omit p i ( 1 − p i ) p_i(1-p_i) p i ( 1 − p i ) but retain gate weights. The centered control forms gated and uniform same-class covariances separately, then pools them with weights 1 − η 1-\eta 1 − η and η \eta η before taking the Schur complement. For CLIP, normalize five prompt embeddings per concept, normalize their concept means, then normalize the water-minus-land difference. Its dot product with normalized image embeddings gives the raw nuisance score, without background/group annotations.
Fitting and Independent Evaluation
Table C.2 uses task labels and official splits throughout; background/group metadata remain sealed until all seeds finish held-out evaluation. Jaccard overlap uses k = max { 1 , ⌈ τ s l i c e n c ⌉ } k=\max\{1,\lceil\tau_{\mathrm{slice}}n_c\rceil\} k = max { 1 , ⌈ τ slice n c ⌉} largest raw memberships in each selected class, intersected on original row identities; different classes give zero. None of the ten Fisher/high-loss gates has a cutoff tie. Fold stability applies this rule to the official training data’s 15% calibration subset: average same-class fold-pair overlaps, then weight these class averages by selected-fold counts normalized over classes with at least two gates. Neither comparison uses positive-membership supports or corrected-weight ranks.
Held-Out Targets and Uncertainty
Conditional on fitting data and fitted quantities, let P c P_c P c be the selected-class distribution and n = n c n=n_c n = n c its evaluation sample size. For frozen gate h h h and matrix map M M M ,
J h = A p , A = E P c [ h M ] , p = E P c [ h ] > 0 ; J ^ h = A ^ p ^ , ( A ^ , p ^ ) = 1 n ∑ i = 1 n ( h i M i , h i ) . (55) J_h=\frac{A}{p},\qquad A=\mathbb{E}_{P_c}[hM],\qquad p=\mathbb{E}_{P_c}[h]>0;\qquad\widehat{J}_h=\frac{\widehat{A}}{\widehat{p}},\qquad(\widehat{A},\widehat{p})=\frac{1}{n}\sum_{i=1}^{n}(h_iM_i,h_i). \tag*{(55)} J h = p A , A = E P c [ h M ] , p = E P c [ h ] > 0 ; J h = p A , ( A , p ) = n 1 i = 1 ∑ n ( h i M i , h i ) . (55)
Table C.2. Fitting and evaluation sequence under the frozen protocol.
Support correction leaves all five primary Fisher gates’ test weights unchanged; for high-loss seed 19 it raises ESS from 94.48 to 102.72. Frozen memberships need not give fixed numerical weights or inactive bootstrap correction.
Proposition C.1 (Frozen-gate ratio bound). For i.i.d. evaluation observations from P c P_c P c , assume 0 ≤ h ≤ 1 0 \le h \le1 0 ≤ h ≤ 1 and 0 ⪯ M ⪯ M max I d 0 \preceq M \preceq M_{\max}I_d 0 ⪯ M ⪯ M m a x I d , with finite M max M_{\max} M m a x specified independently of evaluation data. Given 0 < δ < 1 0 < \delta< 1 0 < δ < 1 , let
u = log ( 4 d / δ ) , r A = M max ( 2 u n + 2 u 3 n ) , r p = log ( 4 / δ ) 2 n . u = \log(4d/\delta), \qquad r_A = M_{\max}\left(\sqrt{\frac{2u}{n}}+\frac{2u}{3n}\right), \qquad r_p = \sqrt{\frac{\log(4/\delta)}{2n}}. u = log ( 4 d / δ ) , r A = M m a x ( n 2 u + 3 n 2 u ) , r p = 2 n log ( 4/ δ ) . With conditional probability at least 1 − δ 1-\delta 1 − δ , simultaneously ∥ A ^ − A ∥ o p ≤ r A \lVert\widehat{A}-A\rVert_{\mathrm{op}} \le r_A ∥ A − A ∥ op ≤ r A and ∣ p ^ − p ∣ ≤ r p \lvert\widehat{p}-p\rvert\le r_p ∣ p − p ∣ ≤ r p . Whenever p = p ^ − r p > 0 p=\widehat{p}-r_p>0 p = p − r p > 0 on this event,
∥ J ^ h − J h ∥ o p ≤ r J : = r A + ∥ J ^ h ∥ o p r p p ^ − r p . (56) \lVert\widehat{J}_h-J_h\rVert_{\mathrm{op}} \le r_J := \frac{r_A+\lVert\widehat{J}_h\rVert_{\mathrm{op}}r_p}{\widehat{p}-r_p}. \tag*{(56)} ∥ J h − J h ∥ op ≤ r J := p − r p r A + ∥ J h ∥ op r p . (56) Proof. The centered matrices h i M i − A h_iM_i-A h i M i − A have norm at most M max M_{\max} M m a x and variance bounded by M max 2 I d M_{\max}^2I_d M m a x 2 I d . Matrix Bernstein [23 Theorem 1.4] gives the numerator event with failure at most δ / 2 \delta/2 δ /2 ; Hoeffding gives the denominator event with failure at most δ / 2 \delta/2 δ /2 . Their union bound and
J ^ h − J h = ( A ^ − A ) − J ^ h ( p ^ − p ) p , p ≥ p ^ − r p > 0 , \widehat{J}_h-J_h=\frac{(\widehat{A}-A)-\widehat{J}_h(\widehat{p}-p)}{p}, \qquad p\ge\widehat{p}-r_p>0, J h − J h = p ( A − A ) − J h ( p − p ) , p ≥ p − r p > 0 , prove the observable bound. The shared random denominator is therefore explicitly controlled. □
Pooling and support correction. Set J p o o l , c = E P c [ M ] J_{\mathrm{pool},c}=\mathbb{E}_{P_c}[M] J pool , c = E P c [ M ] and J h ( η ) = ( 1 − η ) J h + η J p o o l , c J_h^{(\eta)}=(1-\eta)J_h+\eta J_{\mathrm{pool},c} J h ( η ) = ( 1 − η ) J h + η J pool , c . Allocate δ r a t i o + δ p o o l = δ \delta_{\mathrm{ratio}}+\delta_{\mathrm{pool}}=\delta δ ratio + δ pool = δ , using the former in Proposition C.1. For the evaluation mean J ^ p o o l , c = n − 1 ∑ i M i \widehat{J}_{\mathrm{pool},c}=n^{-1}\sum_i M_i J pool , c = n − 1 ∑ i M i , matrix Bernstein supplies
r p o o l = M max [ 2 log ( 2 d / δ p o o l ) n + 2 log ( 2 d / δ p o o l ) 3 n ] . r_{\mathrm{pool}}=M_{\max}\left[\sqrt{\frac{2\log(2d/\delta_{\mathrm{pool}})}{n}}+\frac{2\log(2d/\delta_{\mathrm{pool}})}{3n}\right]. r pool = M m a x [ n 2 log ( 2 d / δ pool ) + 3 n 2 log ( 2 d / δ pool ) ] .
Writing J ~ h = ∑ i q ~ i M i \widetilde{J}_h=\sum_i\widetilde{q}_iM_i J h = ∑ i q i M i , the triangle inequality gives ∥ J ~ h − J ^ h ∥ o p ≤ M max ∑ i ∣ q ~ i − q i 0 ∣ = : r p r o j \lVert\widetilde{J}_h-\widehat{J}_h\rVert_{\mathrm{op}}\le M_{\max}\sum_i\lvert\widetilde{q}_i-q_i^0\rvert=:r_{\mathrm{proj}} ∥ J h − J h ∥ op ≤ M m a x ∑ i ∣ q i − q i 0 ∣ =: r proj . Hence, on the simultaneous event and when p > 0 p>0 p > 0 ,
∥ ( 1 − η ) J ~ h + η J ^ p o o l , c − J h ( η ) ∥ o p ≤ ( 1 − η ) ( r J + r p r o j ) + η r p o o l . (57) \left\lVert(1-\eta)\widetilde{J}_h+\eta\widehat{J}_{\mathrm{pool},c}-J_h^{(\eta)}\right\rVert_{\mathrm{op}}\le(1-\eta)(r_J+r_{\mathrm{proj}})+\eta r_{\mathrm{pool}}. \tag*{(57)} ( 1 − η ) J h + η J pool , c − J h ( η ) op ≤ ( 1 − η ) ( r J + r proj ) + η r pool . (57)
The union bound permits dependence between pool and ratio estimates; r p r o j r_{\mathrm{proj}} r proj covers batch-dependent support correction. If the pool is fixed from fitting data, it belongs to the conditional target and needs no evaluation-error term. Appendix B.3 converts the matrix allowance into a Schur/eigenvalue lower certificate, requiring positive lower membership mass and nuisance-conditioning margins. The target retains shrinkage; evaluation-estimated centering additionally requires first-moment control.
Reported uncertainty. Each of 2,000 within-class bootstrap replicates resamples examples with membership, coordinates, logit and label together, recomputing support-corrected weights and the same-class pool. Gate coefficients/intercept, normalizers and temperature stay frozen. The 95% interval uses the 2.5th/97.5th percentiles of the pooled predictive matrix’s ridge Schur value, retaining η = 0.10 \eta= 0.10 η = 0.10 and λ r i d = 10 − 3 \lambda_{\mathrm{rid}} = 10^{-3} λ rid = 1 0 − 3 , with linear quantile interpolation; ϵ \epsilon ϵ enters inverse information separately. For any reported metric T T T , form paired differences Δ s = T s g a t e − T s c o n t r o l \Delta_s = T^{\mathrm{gate}}_s - T^{\mathrm{control}}_s Δ s = T s gate − T s control across five seeds, then report Δ ± t 0.975 , 4 s Δ / 5 \Delta\pm t_{0.975,4}s_{\Delta}/\sqrt{5} Δ ± t 0.975 , 4 s Δ / 5 , with sample standard deviation s Δ s_{\Delta} s Δ ; single-procedure intervals use its five values. Bootstrap intervals condition on fitted quantities; seed intervals reflect shared-benchmark fitting variation. Neither validates the finite-sample radii above.
Experimental Protocols and Additional Results
Experimental Quantities and Aggregation
Table D.1 identifies the operational targets; Appendix § 12 .§ 1 defines the matrix constructions. Real-data audits use nuisance ridge 10 − 3 10^{-3} 1 0 − 3 and stabilization 10 − 8 10^{-8} 1 0 − 8 ; synthetic audits use ε = 10 − 12 \varepsilon= 10^{-12} ε = 1 0 − 12 . Certificates use α = 0.95 \alpha= 0.95 α = 0.95 . Primary predefined-group coordinates use training moments; prospective normalization is specified below. Reported evaluation Loss-CVaR averages group-mean NLLs at or above their empirical 95th percentile, weighting groups equally; synthetic experiments use environment-mean NLLs. The Loss-CVaR training baseline averages the largest ⌈ 0.1 n ⌉ \lceil0.1n\rceil ⌈ 0.1 n ⌉ individual-example NLLs.
Table D.1. Diagnostic targets and replication units.
Uncertainty. Prospective, synthetic, predefined-group, and frozen-feature training summaries report nominal 95% normal-approximation half-widths 1.96 s / 5 1.96s/\sqrt{5} 1.96 s / 5 , with sample standard deviation s s s . Hidden-slice and group-free comparisons use five-seed Student-t t t intervals; contrasts are paired within seed. The end-to-end coefficient sweep reports standard deviations, and the conditional Gaussian/Wald audit is separate (Appendix § 12 .§ 4 ). Hidden-gate bootstrap intervals condition on the frozen gate and recompute evaluation weights (Appendix § 13 ). Acquisition retains means without recovered per-seed intervals.
Seeds describe fitting/partition or control variation on fixed benchmark splits, not independent datasets. For the predefined-group Waterbirds diagnostic in Appendix § 14 .§ 5 , the classifier is deterministic after feature extraction and logistic fitting; only its random controls and reserve anchor vary across seeds. Prospective intervals use training seeds: 23,176 composites share 5,794 birds, with two donors per bird and arm. Accuracy changes are percentage points; information retains its original units.
Prospective Nuisance Intervention
Training and frozen selection. ImageNet-initialized MobileNetV3-small [10 , 24 ] is fully fine-tuned on Waterbirds [2 ] for seeds 22–26: 50 epochs, batch size 32, learning rate 10 − 4 10^{-4} 1 0 − 4 , and bf16 on an H100. A validation-fitted temperature and class-balanced logistic background probe use the pooled representation. Five probe folds are stratified by validation group (3,000 maximum iterations); features use validation standardization. Validation nuisance scores are out of fold and use validation moments; test scores use the full-validation probe and test moments. Task logits use training moments. Test normalization never selects the slice. The rule is
G e l i g i b l e = { g : Acc v ( g ) ≥ Acc v , NLL v ( g ) ≤ median h NLL v ( h ) } , \mathcal{G}_{\mathrm{eligible}} = \{g : \operatorname{Acc}_{v}(g) \ge\operatorname{Acc}_{v},\ \operatorname{NLL}_{v}(g) \le\operatorname{median}_{h}\operatorname{NLL}_{v}(h)\}, G eligible = { g : Acc v ( g ) ≥ Acc v , NLL v ( g ) ≤ median h NLL v ( h )} ,
g ⋆ ∈ arg min g ∈ G e l i g i b l e λ min ( F g G r i d ) J T T , g . (58) g^{\star} \in\operatorname*{arg\,min}_{g \in\mathcal{G}_{\mathrm{eligible}}} \frac{\lambda_{\min}\!\left(\mathbf{F}^{\mathrm{Grid}}_{g}\right)}{J_{\mathrm{TT},g}}. \tag*{(58)} g ⋆ ∈ g ∈ G eligible arg min J TT , g λ m i n ( F g Grid ) . (58)
Here v v v denotes validation and eligibility uses uncalibrated NLL. Temperature scaling [25 ] minimizes full-validation binary NLL over 400 log-spaced temperatures in [ 0.05 , 20 ] [0.05, 20] [ 0.05 , 20 ] , choosing the smaller temperature at ties. The checkpoint, temperature, probe, slice and audit constants are frozen before scoring; checkpoint mismatches abort evaluation. Table D.2 shows that LL (landbird/land) and WW (waterbird/water) qualify; WW is selected in all five seeds.
Foreground-preserving construction. Waterbirds combines CUB birds [26 ] with Places backgrounds [27 ] under soft CUB mattes. For each bird, we retain background pixels verbatim outside the dilated foreground and fill the covered region using Telea inpainting [28 ] , with a 5-pixel inpainting radius and a 5 × 5 elliptical dilation kernel. Its extracted bird layer is recomposed onto training-image donors of the same or opposite land/water background type. Donors have foreground fraction at most 0.08; a deterministic sequence tries four, taking the first with visible inpainted area at most 0.03, or the least-inpainted of four. The same-type arm undergoes identical recomposition.
Table D.2. Validation selection. Means ± nominal 95% normal-approximation half-widths across seeds 22–26; accuracy is a fraction. Information uses uncentered feature moments.
Both arms use the same foreground layer, matte, label and source birds. Opaque foreground pixels are unchanged before JPEG encoding (quality 95, no chroma subsampling); soft-edge mixtures vary with background. Two draws per arm yield 23,176 composites. Visible inpainted area has mean 4.13%, p95 8.45%, maximum 20.37%, and arm means 4.13%/4.12%. Correctness is averaged across donors within bird, then across birds.
Outcomes and confident flips. Table D.3 reports all groups. The confident half uses the within-slice median calibrated true-label probability among originally correct birds, including ties. In WW, opposite-type/control flip rates are 56.4%/3.3% overall and 41.8%/0.7% in that half; the fixed threshold p ≥ 0.9 p \ge0.9 p ≥ 0.9 selects no WW birds.
Table D.3. Held-out interventions. Accuracies and flip rates are seed means in percent; controlled drops are same-type minus opposite-type percentage points with nominal 95% normal-approximation half-widths. Panel (b) counts are mean confident-half counts.
Matched selection comparisons. Reapplying eligibility within each seed leaves LL and WW. Minimum validation accuracy and maximum uncalibrated NLL, evaluated separately on this identical set, agree with the surviving-fraction selector in all five seeds. All-group comparisons answer a different question; landbird/water determines the model-level certificate (Table D.4).
Centering and predictive Fisher. The post hoc check fixes the checkpoints, temperatures, validation coordinates v = ( z , b ) T v = (z, b)^{\mathsf{T}} v = ( z , b ) T , original scales, eligible sets and intervention outcomes. No model/probe is refitted. With ⟨ ⋅ ⟩ g \langle\cdot\rangle_g ⟨ ⋅ ⟩ g the divisor-n g n_g n g slice mean, compare
J g = ⟨ v v T ⟩ g = Σ g + μ g μ g T , Σ g = ⟨ ( v − μ g ) ( v − μ g ) T ⟩ g , μ g = ⟨ v ⟩ g . J_g = \langle vv^{\mathsf{T}}\rangle_g = \Sigma_g + \mu_g\mu_g^{\mathsf{T}}, \qquad\Sigma_g = \langle(v-\mu_g)(v-\mu_g)^{\mathsf{T}}\rangle_g, \qquad\mu_g = \langle v\rangle_g. J g = ⟨ v v T ⟩ g = Σ g + μ g μ g T , Σ g = ⟨( v − μ g ) ( v − μ g ) T ⟩ g , μ g = ⟨ v ⟩ g .
Centering does not rescale. The local model logit p θ , ξ ( x ) = f w ( x ) / T ^ + θ z ( x ) + ξ b ( x ) p_{\theta,\xi}(x) = f_w(x)/\widehat{T} + \theta z(x) + \xi b(x) p θ , ξ ( x ) = f w ( x ) / T + θ z ( x ) + ξ b ( x ) at zero has score ( Y − p ) v (Y-p)v ( Y − p ) v with conditional mean zero even when μ g ≠ 0 \mu_g \ne0 μ g = 0 . Its predictive Fisher and the common fraction are
I g = ⟨ p ( 1 − p ) v v T ⟩ g , r ρ ( A ) = 1 − A z b 2 A z z ( A b b + ρ ) , ρ = 10 − 3 . I_g = \langle p(1-p)vv^{\mathsf{T}}\rangle_g, \qquad r_\rho(A) = 1 - \frac{A_{zb}^{2}}{A_{zz}(A_{bb}+\rho)}, \qquad\rho= 10^{-3}. I g = ⟨ p ( 1 − p ) v v T ⟩ g , r ρ ( A ) = 1 − A z z ( A bb + ρ ) A z b 2 , ρ = 1 0 − 3 .
Perturbations follow calibration and profile only the declared background coordinate. The normalized-weight comparator is K g = I g / q ˉ g K_g = I_g/\bar{q}_g K g = I g / q ˉ g , q ˉ g = ⟨ p ( 1 − p ) ⟩ g \bar{q}_g = \langle p(1-p)\rangle_g q ˉ g = ⟨ p ( 1 − p ) ⟩ g , so r ρ ( K g ) = r q ˉ g ρ ( I g ) r_\rho(K_g) = r_{\bar{q}_g\rho}(I_g) r ρ ( K g ) = r q ˉ g ρ ( I g ) : normalization preserves the unregularized fraction but changes effective ridge. Table D.4 therefore reports it separately.
Table D.4: Selection and matrix sensitivity, evaluated within each seed. Panel (a) distinguishes original eligible and all-group sets; drops are mean percentage points. Panel (b) fixes the original eligible LL/WW set and interventions; fractions/drops are means ± \pm ± nominal 95% normal-approximation half-widths.
Table D.4. Selection and matrix sensitivity, evaluated within each seed. Panel (a) distinguishes original eligible and all-group sets; drops are mean percentage points. Panel (b) fixes the original eligible LL/WW set and interventions; fractions/drops are means ± \pm ± nominal 95% normal-approximation half-widths.
Centering selects LL in all seeds, whereas predictive Fisher retains WW. Slice means therefore contribute to the original ranking: for WW in seed 22, μ z 2 = 2.094 \mu_z^2 = 2.094 μ z 2 = 2.094 versus Σ z z = 0.0636 \Sigma_{zz} = 0.0636 Σ z z = 0.0636 . Table D.5 preserves the per-seed reversal; full means, covariance entries, weights, matrices and hashes are retained in the artifact.
Table D.5: Per-seed fractions on the original global scales. Each cell is LL / WW; the smaller value determines selection. Centering reverses the ranking in every seed.
Table D.5. Per-seed fractions on the original global scales. Each cell is LL / WW; the smaller value determines selection. Centering reverses the ranking in every seed.
Development chronology. The surviving-fraction rule followed an absolute-information pilot on seeds 17–21 and was frozen for newly trained seeds 22–26 on the same benchmark. Primary-cohort entries use the documented rerun; the original-versus-rerun ledger is retained in the artifact. Appendix § 15 .§ 3 gives the Fisher-Glass training comparison under this same intervention.
Raw, projected, and tail summaries. We compare four summaries of retained validation geometry: mean raw task second moment A A A , arithmetic mean projected information P P P , reciprocal mean inverse information H H H , and inverse-CVaR certificate C C C . For the four equally weighted groups, let F g = F ˉ g r i d F_g = \bar{F}^{\mathrm{rid}}_g F g = F ˉ g rid and L g = ( F g + ε ) − 1 L_g = (F_g + \varepsilon)^{-1} L g = ( F g + ε ) − 1 . Then
A = 1 4 ∑ g J ^ T T , g , P = 1 4 ∑ g F g , A = \frac{1}{4}\sum_g \widehat{J}_{TT,g}, \qquad P = \frac{1}{4}\sum_g F_g, A = 4 1 g ∑ J T T , g , P = 4 1 g ∑ F g ,
H = ( 1 4 ∑ g L g ) − 1 , C = [ CVaR 0.95 , π ( L g ) ] − 1 , π g = 1 4 . H = \left(\frac{1}{4}\sum_g L_g\right)^{-1}, \qquad C = \left[\operatorname{CVaR}_{0.95,\pi}(L_g)\right]^{-1}, \qquad\pi_g = \frac{1}{4}. H = ( 4 1 g ∑ L g ) − 1 , C = [ CVaR 0.95 , π ( L g ) ] − 1 , π g = 4 1 .
Thus H H H is the harmonic mean of the stabilized projected information values F g + ε F_g + \varepsilon F g + ε . The four summaries use uncentered feature moments; projection uses nuisance ridge 10 − 3 10^{-3} 1 0 − 3 , and H H H and C C C use reciprocal stabilization ε = 10 − 8 \varepsilon= 10^{-8} ε = 1 0 − 8 . This post hoc analysis joins validation summaries to held-out interventions in the primary (seeds 22–26) and development (17–21) cohorts. It compares summaries of existing models, not training objectives. “Selected FG” denotes the Fisher-Glass models selected within each cohort, including the accuracy-budget fallbacks listed in the caption; Appendix § 15 .§ 3 details the primary training comparison.
Table D.6: Four validation summaries and their intervention associations. Panels (a)–(b): means ± \pm ± nominal 95% normal-approximation half-widths, 1.96 s / 5 1.96s/\sqrt{5} 1.96 s / 5 ; changes are paired within seed and drops are signed percentage points. Selected FG includes accuracy-budget fallbacks (primary seeds 22, 24; development seed 20). Panel (c): descriptive Spearman coefficients for ten models arranged in five ERM/FG seed pairs per cohort, with average ranks for tied outcomes.
Table D.6. Four validation summaries and their intervention associations.
For Table D.6(c), D max = 100 max g ( A c c s a m e , g − A c c o p p o s i t e , g ) D_{\max} = 100\max_g(\mathrm{Acc}_{\mathrm{same},g} - \mathrm{Acc}_{\mathrm{opposite},g}) D m a x = 100 max g ( Acc same , g − Acc opposite , g ) and W o p p = min g A c c o p p o s i t e , g W_{\mathrm{opp}} = \min_g \mathrm{Acc}_{\mathrm{opposite},g} W opp = min g Acc opposite , g , both over all four groups. Integer correct counts recover exact outcome ties: primary ERM seeds 22 and 25 share drop 348/642, despite different floating-point subtractions. LL and LW denote landbird/land and landbird/water. Negative drops mean opposite-type recomposition improves accuracy. The group with the largest inverse-information contribution accounts for 38.3% and 40.6% of summed inverse information on average in the primary and development cohorts, respectively. The original headline rules differed: surviving fraction in the primary cohort, absolute projected information in development; this comparison consistently uses all four groups for model summaries and correlations.
Projection and aggregation give different model summaries, with rank associations varying across cohorts. For four equally weighted groups and α = 0.95 \alpha= 0.95 α = 0.95 , C = min g F g + ϵ C = \min_g F_g + \epsilon C = min g F g + ϵ ; its active group also minimizes projected information and maximizes inverse contribution. Here that group is LW, whose opposite-type recomposition improves accuracy. The prospective WW diagnosis instead uses the surviving fraction within the eligible set. This comparison does not establish superior swap-damage prediction by inverse-CVaR; paired changes in P P P , H H H , C C C all have intervals containing zero. The artifact provides per-model and per-seed values, exact-count outcomes, input hashes, and replay code.
Auxiliary flip prediction. Each originally correct source bird contributes two opposite-type donor rows. All predictor covariates are evaluated on the original, pre-intervention image. Baseline predictors are uncalibrated NLL, calibrated true-label confidence, absolute raw logit and standardized task logit z z z . Augmentation adds β g b \beta_g b β g b and ∣ z − β g b ∣ \lvert z - \beta_g b\rvert ∣ z − β g b ∣ , with β g = J ^ T N , g / ( J ^ N N , g + 10 − 3 ) \beta_g = \widehat{J}_{TN,g}/(\widehat{J}_{NN,g} + 10^{-3}) β g = J T N , g / ( J N N , g + 1 0 − 3 ) frozen from the validation block. Both models use identical five-fold, nonshuffled CUB-species GroupKFold splits, keeping species and donor pairs together. Logistic regression uses an intercept, ℓ 2 \ell_2 ℓ 2 penalty, C = 1 C = 1 C = 1 , L-BFGS, tolerance 10 − 4 10^{-4} 1 0 − 4 , maximum 5,000 iterations, seed 0 and no class weighting or tuning. All columns, including group indicators, are centered and scaled using training-fold means and population standard deviations; standard deviations below 10 − 9 10^{-9} 1 0 − 9 are replaced by one.
Pool out-of-fold predictions within seed before computing average precision (reported as AUPRC), area under the receiver operating characteristic curve (AUROC), and mean log likelihood. For seed s s s , N s N_s N s counts all retained donor rows in the analyzed population, including flips and nonflips. Let y s i f l i p = 1 y_{si}^{\mathrm{flip}} = 1 y s i flip = 1 indicate an incorrect prediction on the donor composite and y s i f l i p = 0 y_{si}^{\mathrm{flip}} = 0 y s i flip = 0 a correct prediction. The probabilities q s i A q_{si}^{A} q s i A and q s i B q_{si}^{B} q s i B are the augmented and baseline out-of-fold flip predictions. For clipped probabilities q ∈ [ 10 − 12 , 1 − 10 − 12 ] q \in[10^{-12}, 1 - 10^{-12}] q ∈ [ 1 0 − 12 , 1 − 1 0 − 12 ] ,
G s = 1 N s ∑ i = 1 N s [ L L ( y s i f l i p , q s i A ) − L L ( y s i f l i p , q s i B ) ] , L L ( y , q ) = y log q + ( 1 − y ) log ( 1 − q ) . (59) G_s = \frac{1}{N_s}\sum_{i=1}^{N_s}\left[\mathrm{LL}(y_{si}^{\mathrm{flip}},q_{si}^{A}) - \mathrm{LL}(y_{si}^{\mathrm{flip}},q_{si}^{B})\right], \qquad\mathrm{LL}(y,q) = y\log q + (1-y)\log(1-q). \tag*{(59)} G s = N s 1 i = 1 ∑ N s [ LL ( y s i flip , q s i A ) − LL ( y s i flip , q s i B ) ] , LL ( y , q ) = y log q + ( 1 − y ) log ( 1 − q ) . (59)
G s G_s G s is augmented-minus-baseline gain in nats per donor row. Table D.7 averages paired changes equally across seeds 22–26. The coordinates improve prediction without group indicators and within WW; with indicators, AUPRC decreases. This tests their incremental value, not unique attribution to the scalar information statistic. Original inputs, exact-source refits and paired records are in the artifact.
Table D.7. Incremental flip prediction. Counts and rates are seed means; parentheses count seeds with improvement. Changes are paired within seed.
Synthetic Classification and Tail Checks
Draw U e ∼ Uniform ( 0 , 1 ) U_e \sim\operatorname{Uniform}(0,1) U e ∼ Uniform ( 0 , 1 ) , set δ e = 0.35 U e 1 / γ \delta_e = 0.35U_e^{1/\gamma} δ e = 0.35 U e 1/ γ , and draw independent standard normals s , r s,r s , r for each observation. The task is z = s + δ e r z = s + \delta_e r z = s + δ e r , Y = 1 { z > 0 } Y = 1\{z > 0\} Y = 1 { z > 0 } , scored by nuisance-only logit 4 s 4s 4 s . Thus
Pr ( δ e < t ) = ( t / 0.35 ) γ ( 0 ≤ t ≤ 0.35 ) , J T T , e = 1 + δ e 2 , F e 0 = δ e 2 . (60) \Pr(\delta_e < t) = (t/0.35)^\gamma\ (0 \le t \le0.35), \qquad J_{TT,e} = 1 + \delta_e^2, \qquad F_e^0 = \delta_e^2. \tag*{(60)} Pr ( δ e < t ) = ( t /0.35 ) γ ( 0 ≤ t ≤ 0.35 ) , J T T , e = 1 + δ e 2 , F e 0 = δ e 2 . (60)
Population feature moments are exact; loss/accuracy use 256 observations per environment. The γ = 1 \gamma= 1 γ = 1 experiment uses 300,000 environments and seeds 11–15. Fit the log-survival slope over the largest 5% of inverse information; empirical CVaR averages values at or above the empirical α \alpha α -quantile. Figure A.1 regenerates seed 11 with L e = ( δ e 2 + 10 − 12 ) − 1 L_e = (\delta_e^2 + 10^{-12})^{-1} L e = ( δ e 2 + 1 0 − 12 ) − 1 , rather than recovering missing original raw output; its survival curve and the five-seed summaries have different replication levels.
Table D.8 checks Theorem 2 and the scalar portfolio law in Appendix § 11 .§ 2 . For K K K independent draws, information K − 1 ∑ j δ e , j 2 K^{-1}\sum_j \delta_{e,j}^2 K − 1 ∑ j δ e , j 2 has exponent K / 2 K/2 K /2 at γ = 1 \gamma= 1 γ = 1 ; weights affect its constant. The mass sweep predicts γ / 2 \gamma/2 γ /2 . An anchor with δ 0 = 0.5 \delta_0 = 0.5 δ 0 = 0.5 and reserve mass 0.05 yields information 0.95 δ e 2 + 0.05 ( 0.25 ) 0.95\delta_e^2 + 0.05(0.25) 0.95 δ e 2 + 0.05 ( 0.25 ) , floor 0.0125, and measured certificate 0.01260 ± \pm ± 0.000001 (five seeds; Appendix § 15 .§ 1 ).
Table D.8. Synthetic checks. Panel (a) and the branch rows are five-seed means ± \pm ± nominal 95% normal-approximation half-widths; zero widths reflect rounding. Mass-sweep rows are reported single-seed values.
Hidden-Slice Discovery Results
Appendix § 13 gives the blind protocol: task labels, frozen MobileNet features and a fixed CLIP direction are available; background/group metadata remain sealed through held-out evaluation. Results use the primary hidden-slice cohort, seeds 17–21, with full-validation gates frozen before test. Support correction may depend on the test sample but leaves all five primary CLIP gates’ test weights unchanged. Information uses same-class pooling; accuracy, uncalibrated NLL and composition use admissible within-class weights without pooling. The artifact identifies this cohort as v2 and records original hashes; the older uncorrected cohort is excluded.
All selectors share classifiers with average accuracy 76.3 ± 0.7 % 76.3 \pm0.7\% 76.3 ± 0.7% and post hoc worst-group accuracy 57.3 ± 1.3 % 57.3 \pm1.3\% 57.3 ± 1.3% . CLIP search finds 98.8 % 98.8\% 98.8% membership-weighted slice accuracy. Table D.9 includes every matched-mass comparator and paired contrast. The random-direction interval and the uncentered and centered feature-moment intervals against high-loss selection contain zero. Feature controls reevaluate gates selected using predictive Fisher without rediscovery.
Table D.9: Hidden-slice comparisons. Panel (a): means ± 95 % \pm95\% ± 95% Student-t t t half-widths across five paired seeds; predictive information retains calibration and shrinkage η = 0.10 \eta= 0.10 η = 0.10 . Panel (b): CLIP-search minus comparator, with paired 95 % 95\% 95% Student-t t t intervals; negative differences indicate lower information.
Table D.9. Hidden-slice comparisons. Panel (a): means ± 95 % \pm95\% ± 95% Student-t t t half-widths across five paired seeds; predictive information retains calibration and shrinkage η = 0.10 \eta= 0.10 η = 0.10 . Panel (b): CLIP-search minus comparator, with paired 95 % 95\% 95% Student-t t t intervals; negative differences indicate lower information.
Class selection and overlap. CLIP, random-direction, random-slice and constant-gate selectors choose landbirds in all five seeds; high loss chooses landbirds in seeds 17,18,21 and waterbirds in 19,20. The two different-class Jaccards are zero; same-class values are 0.00222, 0.00111, and 0. Their mean 0.000666, reported as 0.001, combines between-class and within-class differences under Appendix § 13 .§ 3 ’s top-count rule.
Support and sensitivity. On same-class rows from the 15% training-calibration subset, cross-fold gate-score correlation is 0.419 [0.353, 0.485] and overlap 0.258 [0.225, 0.291]. Test ESS is 414 [383, 445] using admissible weights. Raw membership mass is 0.083 [0.077, 0.089] within the selected class (target 0.10), not the full test set; landbirds contain 4,510 test examples. Validation cross-fitting, separate from final testing, gives information 0.004–0.005 over masses {0.05, 0.10, 0.20} versus high-loss 0.012–0.027, and 0.004–0.006 over shrinkages {0.05, 0.10, 0.20}.
Composition after held-out evaluation is 97.0% landbird/land for CLIP, 94.4% for random direction, and 58.4% landbird/water for high loss. The hidden slice has lower projected predictive information than the weakest predefined group in all five seeds: mean paired difference − 0.0029 -0.0029 − 0.0029 , with 95% Student-t t t interval [ − 0.0034 , − 0.0024 ] [-0.0034, -0.0024] [ − 0.0034 , − 0.0024 ] , taking the predefined-group minimum separately within each seed. Both sides use the same model, CLIP coordinate, calibrated predictive moments, pooling η = 0.10 \eta= 0.10 η = 0.10 , and ridge 10 − 3 10^{-3} 1 0 − 3 . A metadata-trained oracle probe used only after blind freeze gives information 0.003 ± 0.000, NLL 0.009 ± 0.004, and 99.1% landbird/land composition (zero width is rounding). Repair outcomes are in Appendix § 15 .§ 3 .
Predefined-Group Diagnostics
Fitting. CivilComments-WILDS [29 , 30 ] predicts comment toxicity, with y = 0 y = 0 y = 0 denoting nontoxic comments. Identity-any indicates that at least one of the 24 original identity annotation scores is at least 0.5. Training-fitted unigram/bigram TF-IDF (50,000 features, minimum document frequency 5, sublinear TF) feeds class-balanced SGD logistic task/identity-any probes (ℓ 2 = 10 − 5 \ell_2 = 10^{-5} ℓ 2 = 1 0 − 5 , 50 passes, tolerance 10 − 3 10^{-3} 1 0 − 3 ). Identity-by-label groups require 100 examples; tail regression uses the largest 50% of group inverse information, requiring eight groups and four tail points. Seed 13 gives the fixed diagnostic; seeds 13–17 give aggregates.
Waterbirds uses frozen ImageNet MobileNetV3-small features (extraction batch 64) and balanced L-BFGS logistic task/background models (1,000 maximum iterations). Four groups are reported directly, without fitting a tail exponent; only controls/anchors vary over seeds 17–21. Primary task/probe and metadata coordinates use training moments. These original runs used an i9-11900F/32 GB workstation, about one minute for CivilComments and three minutes for first Waterbirds extraction/fitting, then cached features.
Table D.10. Original predefined-group diagnostics, distinct from fresh audits in Appendix § 12 .§ 4 . Means ± nominal 95% normal-approximation half-widths; Waterbirds zeros reflect fixed-model fitting.
For fixed CivilComments, the weakest-information group is muslim/y = 0 (accuracy 60.1%); lowest accuracy is black/y = 0 (44.0%). Waterbirds’ weakest-information group is landbird/land (97.3% accuracy), whereas waterbird/land has lowest accuracy (56.5%); inverse-information p95 is 5.90.
Specification and stress controls. The prespecified semantic family is { b m e t a , b p r o b e } \{b_{\mathrm{meta}}, b_{\mathrm{probe}}\} { b meta , b probe } : standardized identity-any/background annotation and its training-fitted probe, without cross-fitting. Semantic relevance follows those annotations; no held-out relevance threshold, anchor task-preservation tolerance, fold-stability admission screen, or nuisance-conditioning threshold was applied. The ridge is 10 − 3 10^{-3} 1 0 − 3 ; family minima are 0.499 and 0.165. Reserve anchors (identity-present label-balanced CivilComments; four-group-balanced Waterbirds) are construction inputs, not admission samples. Fresh exact/ridge audits and hidden-gate stability are separate analyses.
Table D.11. Nuisance sensitivity of fixed-model diagnostics. Entries are I 0.95 F G I^{\mathrm{FG}}_{0.95} I 0.95 FG ; worst-group accuracy remains 44.0%/56.5% for CivilComments/Waterbirds.
Random nuisance is independent standard normal; under-projection sets it to zero. Over-projection uses standardized task score plus Gaussian noise of standard deviation 0.01 and deliberately erases task information. “Hidden-loss” is observed test loss with original groups fixed. The random, over-projected, and observed-loss coordinates use evaluated-array normalization and are not independently learned nuisance scores. The metadata/random contrast is 0.587/0.588 on CivilComments, versus probe/random 0.165/0.482 on Waterbirds, illustrating specification dependence.
The versioned artifact (see the [artifact availability statement](artifact availability statement)) retains execution commands, full-precision matrices and overlaps, run-history ledgers and expanded source tables, with original cohort provenance. Those records supplement the scientific definitions and controls above.
Information-Changing Interventions
This appendix gives the reserve proof, training and acquisition comparisons, and the influence derivative behind § 6 . Distinct cohorts and information targets remain separate. Expanded protocols and numerical breakdowns are retained in the versioned artifact’s docs/extended_appendix_details/, with executable reproduction guides and provenance (see the artifact availability statement).
Fix the predictor, coordinates, and normalization. Let J e , J 0 ⪰ 0 J_e, J_0 \succeq0 J e , J 0 ⪰ 0 have positive-definite nuisance blocks. The anchor is an additional observation source with F 0 0 : = S 0 ( J 0 ) ⪰ ρ 0 I F_0^0 := S_0(J_0) \succeq\rho_0 I F 0 0 := S 0 ( J 0 ) ⪰ ρ 0 I , ρ 0 > 0 \rho_0 > 0 ρ 0 > 0 . For the allocation in § 6 , each observation’s source is observed and allocation probabilities are parameter-independent. Sampling the anchor with probability r ∈ ( 0 , 1 ] r \in(0,1] r ∈ ( 0 , 1 ] therefore gives
J e r e s = ( 1 − r ) J e + r J 0 , J_e^{\mathrm{res}} = (1-r)J_e + rJ_0, J e res = ( 1 − r ) J e + r J 0 ,
S 0 ( J e r e s ) ⪰ ( 1 − r ) S 0 ( J e ) + r S 0 ( J 0 ) ⪰ r ρ 0 I . (61) S_0(J_e^{\mathrm{res}}) \succeq(1-r)S_0(J_e) + rS_0(J_0) \succeq r\rho_0 I. \tag*{(61)} S 0 ( J e res ) ⪰ ( 1 − r ) S 0 ( J e ) + r S 0 ( J 0 ) ⪰ r ρ 0 I . (61)
The same statement applies to affine allocation of a fixed operational matrix map; it is not an assertion about an unobserved mixture likelihood.
Proof. For J = ( A B B T C ) J = \begin{pmatrix} A & B \\ B^{\mathsf T} & C \end{pmatrix} J = ( A B T B C ) , C ≻ 0 C \succ0 C ≻ 0 , completing the square gives
u T S 0 ( J ) u = min v q J ( u , v ) , q J ( u , v ) = ( u − v ) T J ( u − v ) , v ∗ = C − 1 B T u . (62) u^{\mathsf T} S_0(J)u = \min_v q_J(u,v), \qquad q_J(u,v) = \begin{pmatrix} u \\ -v \end{pmatrix}^{\mathsf T} J \begin{pmatrix} u \\ -v \end{pmatrix}, \qquad v_* = C^{-1}B^{\mathsf T}u. \tag*{(62)} u T S 0 ( J ) u = v min q J ( u , v ) , q J ( u , v ) = ( u − v ) T J ( u − v ) , v ∗ = C − 1 B T u . (62)
Linearity in J J J implies
min v { ( 1 − r ) q J e ( u , v ) + r q J 0 ( u , v ) } ≥ ( 1 − r ) min v q J e ( u , v ) + r min v q J 0 ( u , v ) ≥ r ρ 0 ∥ u ∥ 2 . (63) \begin{aligned}
\min_v\{(1-r)q_{J_e}(u,v) + rq_{J_0}(u,v)\} \ge(1-r)\min_v q_{J_e}(u,v) + r\min_v q_{J_0}(u,v) \\
&\ge r\rho_0\lVert u\rVert^2. \tag*{(63)}
\end{aligned} v min {( 1 − r ) q J e ( u , v ) + r q J 0 ( u , v )} ≥ ( 1 − r ) v min q J e ( u , v ) + r v min q J 0 ( u , v ) ≥ r ρ 0 ∥ u ∥ 2 . (63)
The last inequality uses S 0 ( J e ) ⪰ 0 S_0(J_e) \succeq0 S 0 ( J e ) ⪰ 0 and the anchor bound; holding for every u u u , it proves (E.1). □ \square □
For any environment distribution, 0 < α < 1 0 < \alpha< 1 0 < α < 1 and ε ≥ 0 \varepsilon\ge0 ε ≥ 0 , CVaR monotonicity gives
L e r e s , ( ε ) ≤ ( r ρ 0 + ε ) − 1 , I α F G , r e s ( ε ) ≥ r ρ 0 + ε ≥ r ρ 0 . (64) L_e^{\mathrm{res},(\varepsilon)} \le(r\rho_0+\varepsilon)^{-1}, \qquad I_{\alpha}^{\mathrm{FG,res}}(\varepsilon) \ge r\rho_0+\varepsilon\ge r\rho_0. \tag*{(64)} L e res , ( ε ) ≤ ( r ρ 0 + ε ) − 1 , I α FG , res ( ε ) ≥ r ρ 0 + ε ≥ r ρ 0 . (64)
Here r = m / ( n + m ) r=m/(n+m) r = m / ( n + m ) for m m m anchor and n n n base observations. An estimated anchor’s positive lower bound from Appendix § 12 .§ 3 yields the same guarantee on its confidence event. The claim concerns the fixed predictor; retraining requires a new audit. Appendix § 14 .§ 3 evaluates r = .05 r=.05 r = .05 , ρ 0 = .25 \rho_0=.25 ρ 0 = .25 , and the resulting floor .0125 .0125 .0125 .
Regularized Training
For a declared group matrix J ^ g ( w ) \widehat{J}_g(w) J g ( w ) , put L ^ g ( w ) = [ λ min ( S r i d ( J ^ g ( w ) ) ) + ε ] − 1 \widehat{L}_g(w)=[\lambda_{\min}(S_{\mathrm{rid}}(\widehat{J}_g(w)))+\varepsilon]^{-1} L g ( w ) = [ λ m i n ( S rid ( J g ( w ))) + ε ] − 1 . With fixed π g ≥ 0 \pi_g \ge0 π g ≥ 0 , ∑ g π g = 1 \sum_g \pi_g=1 ∑ g π g = 1 , training solves
min w , τ R ^ b a s e ( w ) + λ F G [ τ + 1 1 − α ∑ g π g ( L ^ g ( w ) − τ ) + ] . (65) \min_{w,\tau}\widehat{R}_{\mathrm{base}}(w)+\lambda_{\mathrm{FG}}\left[\tau+\frac{1}{1-\alpha}\sum_g\pi_g(\widehat{L}_g(w)-\tau)_+\right]. \tag*{(65)} w , τ min R base ( w ) + λ FG [ τ + 1 − α 1 g ∑ π g ( L g ( w ) − τ ) + ] . (65)
Here w w w denotes predictor parameters, R ^ b a s e \widehat{R}_{\mathrm{base}} R base is the ERM or Group DRO task-risk objective, and λ F G \lambda_{\mathrm{FG}} λ FG its information-penalty coefficient. Frozen-feature Waterbirds uses uncentered second moments of standardized task logits and fixed background-probe scores, ridge 10 − 3 10^{-3} 1 0 − 3 and ε = 10 − 8 \varepsilon=10^{-8} ε = 1 0 − 8 . Training α = .75 \alpha=.75 α = .75 and audit α = .95 \alpha=.95 α = .95 both give the maximum inverse information over four equally weighted groups.
Protocol and selection. ImageNet MobileNetV3-small features and background-probe scores use training-moment standardization. A linear head receives 1,200 full-batch AdamW updates (lr = .01, weight decay 10 − 4 10^{-4} 1 0 − 4 , seeds 17–21), Gaussian N ( 0 , .01 2 ) N(0, .01^{2}) N ( 0 , .0 1 2 ) initial weights and zero intercept. Current logits use full-training population standard deviation plus 10 − 6 10^{-6} 1 0 − 6 ; training projected information is floored at ε \varepsilon ε before reciprocal stabilization. Evaluation uses fitted training-logit moments. Group DRO minimizes the maximum group mean cross-entropy; Loss-CVaR uses the largest ⌈ .1 n ⌉ \lceil.1n\rceil ⌈ .1 n ⌉ training losses, and JTT refits with weight 20 on initial ERM mistakes.
For each FG family and seed, search Λ F G = { .01 , .03 , .1 , .3 } \Lambda_{\mathrm{FG}} = \{.01, .03, .1, .3\} Λ FG = { .01 , .03 , .1 , .3 } . Among candidates whose validation worst-group accuracy is within .02 of the family’s best, maximize the validation certificate, breaking ties by higher validation worst-group accuracy. This uses validation group annotations. Historical choices are .03 for ERM + FG in all seeds and .1 or .3 for Group DRO + FG. Table E.1 retains all six methods; official test data enter only final evaluation.
Table E.1: Original frozen-feature training cohort, seeds 17–21. Both panels give means ± \pm ± nominal 95% normal-approximation half-widths; accuracy is percent and the uncentered feature–Schur ridge certificate uses α = .95 \alpha= .95 α = .95 .
Table E.1. Original frozen-feature training cohort, seeds 17–21. Both panels give means ± \pm ± nominal 95% normal-approximation half-widths; accuracy is percent and the uncentered feature–Schur ridge certificate uses α = .95 \alpha= .95 α = .95 .
FG doubles ERM’s certificate; adding it to Group DRO raises the certificate from .233 to .614 and mean worst-group accuracy from 67.3% to 69.4%. These historical summaries differ from Appendix § 12 ’s fresh ridge/exact audit cohort, which does not retroactively certify the historical checkpoints. Their original per-seed outputs were not recovered; retained reports, complete settings and independent rerun commands are in the artifact (see the artifact availability statement).
Extended Training and Selection Results
End-to-end Group DRO + Fisher-Glass coefficient sweep. MobileNetV3-small is fine-tuned for 10 epochs (batch 128, evaluation batch 512, seeds 17–21) with AdamW learning rate and weight decay 10 − 4 10^{-4} 1 0 − 4 ; the final epoch is evaluated. The reported H100 configuration uses bf16 and TF32. The nuisance is training-standardized background annotation; task logits use minibatch population standard deviation plus 10 − 6 10^{-6} 1 0 − 6 . Groups with fewer than two observations are omitted. Ridge is 10 − 3 10^{-3} 1 0 − 3 , α = .95 \alpha= .95 α = .95 , and projected information is floored at 10 − 8 10^{-8} 1 0 − 8 before another 10 − 8 10^{-8} 1 0 − 8 enters the reciprocal. Table E.2 retains the complete coefficient sweep: stronger penalties can increase information while degrading prediction.
Table E.2. End-to-end Waterbirds Group DRO + Fisher-Glass coefficient sweep. FG entries are means ± \pm ± sample standard deviations over five seeds, not confidence intervals. The Group DRO reference is the baseline from the λ F G = 10 − 4 \lambda_{\mathrm{FG}} = 10^{-4} λ FG = 1 0 − 4 sweep; each sweep reran its baseline, so this is not a shared paired reference for every coefficient.
At λ F G = 10 − 4 \lambda_{\mathrm{FG}} = 10^{-4} λ FG = 1 0 − 4 the certificate rises from .259 to .502 and inverse p95 falls from 3.68 to 2.10; at λ F G = 10 − 2 \lambda_{\mathrm{FG}} = 10^{-2} λ FG = 1 0 − 2 worst-group accuracy is only 22.7%. Historical report summaries support this table; Appendix § 12 ’s fresh audits are a separate cohort.
Prospective information training. Use Appendix § 14 .§ 2 ’s shared model, 50-epoch schedule and seeds 22–26. Backbone fine-tuning uses AdamW with learning rate and weight decay 10 − 4 10^{-4} 1 0 − 4 . FG adds coefficients {.003, .01, .03} and a background probe trained on detached features (AdamW learning rate .001, default weight decay .01), whose scores are detached in the penalty. After one penalty-free epoch, training uses α = .75 \alpha= .75 α = .75 , ridge .001, batch standard-deviation guard 10 − 6 10^{-6} 1 0 − 6 , groups of at least six observations with nuisance standard deviation at least .05, and a .05 information floor before direct inversion. This floor differs from audit stabilization.
For this prospective 50-epoch cohort, select the final model with largest validation certificate among candidates whose overall validation accuracy is at least ERM minus .01. If none qualify, maximize the certificate over all FG candidates for that seed and record the fallback. Choices are [.01, .03, .01, .003, .01], with fallbacks in seeds 22 and 24. All audit quantities are frozen before intervention scoring. In the prospective replication, FG raises the validation certificate from .289 to .330, but the controlled drop is 52.9 versus 52.6 percentage points for ERM and the originally-correct flip rate is 57.7% versus 56.4%. The accuracy constraint fails in two seeds and background sensitivity persists.
Constrained hidden-slice repair. The primary hidden-slice cohort of Appendix § 14 .§ 4 compares loss, Fisher and combined repair without opening group metadata. Each arm adds its repair penalties to ERM, includes the unchanged model, and must preserve overall, balanced and each-class validation accuracy within .02 of the base. Loss coefficients are {.1, .3, 1}, Fisher coefficients {.01, .03, .1}, and combined candidates use their Cartesian product. The same warm start and fresh AdamW optimizer (lr = .01, weight decay 10 − 4 10^{-4} 1 0 − 4 , gradient clip 10) receive at most 1,200 steps. Calibration NLL at temperature one is checked every 25 steps, with patience 12 and improvement threshold 10 − 5 10^{-5} 1 0 − 5 ; the best checkpoint is restored, with the initial model eligible for selection. The matched ERM continuation sets both penalties to zero. Base temperature remains fixed for information training, selection and subsequent audits.
For the frozen training slice, the dimensionless Fisher penalty is
P F G ( w ) = max { F i n i t , ε } F ( w ) + ε , ε = 10 − 8 , (66) P_{\mathrm{FG}}(w) = \frac{\max\{F_{\mathrm{init}},\varepsilon\}}{F(w)+\varepsilon}, \qquad\varepsilon= 10^{-8}, \tag*{(66)} P FG ( w ) = F ( w ) + ε max { F init , ε } , ε = 1 0 − 8 , (66)
Here F ( w ) F(w) F ( w ) is the frozen slice’s projected predictive information and F i n i t F_{\mathrm{init}} F init its value at the unchanged model, with shrinkage .1 .1 .1 , ridge .001 .001 .001 , and negative numerical information clipped to zero. Within the accuracy envelope, loss/Fisher repair minimizes frozen-slice loss/maximizes information. Combined selection uses the loss–information Pareto frontier, maximizing the smaller relative improvement; ties favor mean relative improvement, then the smaller coefficient sum normalized by grid maxima, λ l o s s + 10 λ F i s h e r \lambda_{\mathrm{loss}} + 10\lambda_{\mathrm{Fisher}} λ loss + 10 λ Fisher , then earlier index. Fresh rediscovery uses matched gate/fold seeds, so the reported population can change.
Table E.3. Constrained repair on the primary hidden-slice cohort, seeds 17–21. Accuracy, NLL and information are means ± 95% Student-t t t half-widths; composition is a mean percentage. Panel two audits each arm’s rediscovered CLIP-defined slice. The Group DRO oracle uses group annotations.
Fisher repair chooses the unchanged model in every seed. Paired worst-group changes versus matched continuation are −.4, .0 and −.6 points for loss, Fisher and combined repair, with intervals containing zero. The group-supervised Group DRO oracle reaches 64.8% worst-group accuracy while rediscovered information remains .005. LL denotes landbird/land.
Post hoc group-free coefficient selection. This analysis was designed after earlier group/test outcomes were inspected. Its grid { 0 , .01 , .03 , .1 , .3 } \{0, .01, .03, .1, .3\} { 0 , .01 , .03 , .1 , .3 } uses (E.6), not the known-group objective’s scale. Training is partitioned 70/15/15% for fitting/calibration/gate discovery; a frozen train-only slice supplies the penalty. All candidates share the warm start, training seed, optimizer, 1,200-step budget, checkpoint rule and base temperature above. Official validation enters only after heads are fixed; group metadata stays sealed through blind test evaluation.
Let R ^ λ \widehat{R}_{\lambda} R λ be mean fixed-temperature validation NLL, s λ = s d ( ℓ λ i ) / 1199 s_{\lambda} = s_{\mathrm{d}}(\ell_{\lambda i})/\sqrt{1199} s λ = s d ( ℓ λi ) / 1199 its within-run standard error, and λ R = arg min λ ( R ^ λ , λ ) \lambda_R = \operatorname*{arg\,min}_{\lambda}(\widehat{R}_{\lambda}, \lambda) λ R = arg min λ ( R λ , λ ) lexicographically. Here ℓ λ i \ell_{\lambda i} ℓ λi is the fixed-temperature validation NLL of example i i i , and s d i s_{\mathrm{d}i} s d i is the sample standard deviation with divisor 1198. Five task-stratified validation folds rediscover candidate-specific slices; V k , λ V_{k,\lambda} V k , λ is held-out inverse predictive information and s V , λ s_{V,\lambda} s V , λ its sample standard deviation. Selection uses
Λ r i s k = { λ : R ^ λ ≤ R ^ λ R + s λ R } , U λ a d j = V ‾ λ + t 4 , .99 s V , λ / 5 , \Lambda_{\mathrm{risk}} = \{\lambda: \widehat{R}_{\lambda} \le\widehat{R}_{\lambda_R} + s_{\lambda_R}\}, \qquad U_{\lambda}^{\mathrm{adj}} = \overline{V}_{\lambda} + t_{4,.99}s_{V,\lambda}/\sqrt{5}, Λ risk = { λ : R λ ≤ R λ R + s λ R } , U λ adj = V λ + t 4 , .99 s V , λ / 5 ,
λ ^ h i d d e n = min arg max λ ∈ Λ r i s k ( U λ a d j ) − 1 , t 4 , .99 = 3.74695. (67) \widehat{\lambda}_{\mathrm{hidden}} = \min\operatorname*{arg\,max}_{\lambda\in\Lambda_{\mathrm{risk}}} \left(U_{\lambda}^{\mathrm{adj}}\right)^{-1}, \qquad t_{4,.99} = 3.74695. \tag*{(67)} λ hidden = min λ ∈ Λ risk arg max ( U λ adj ) − 1 , t 4 , .99 = 3.74695. (67)
The risk allowance uses the minimizer’s SE for all candidates. Bonferroni adjustment covers five coefficients, but overlapping gate-training folds make this a ranking score without independent-fold coverage. Accuracy screens and paired inverse-information tests are diagnostics, not selection constraints. The group-supervised comparator maximizes validation worst-group accuracy over the same Λ r i s k \Lambda_{\mathrm{risk}} Λ risk , breaking ties toward smaller λ \lambda λ .
Matched zero continuation restores step zero in all five seeds, explaining equality with frozen ERM despite intervening optimization. Group-free selection equals plain NLL minimization in every seed; the information criterion does not change any choice. Versus matched continuation, paired mean changes are +2.1 average-accuracy points ([+ .2, +4.0]), +.006 information ([-.001, +.010]), and −12.6 worst-group points ([-21.6, −3.7]). The group-supervised comparator opens validation annotations only after blind decisions and records are fixed and is not an upper bound. The post hoc coefficient-selection results and primary repair cohort have separate artifact identifiers (see the artifact availability statement), alongside extended settings and full tables.
Table E.4. Post hoc coefficient selection, five paired seeds. Accuracies are test results; information is projected predictive information in test slices defined by validation-rediscovered, frozen gates. Brackets are 95% Student-t t t intervals. Policies selecting identical models across these seeds share rows; the text explains both coincidences. Coefficient counts are λ \lambda λ : number of seeds.
Operational choice of the penalty. For group-free selection, we propose prespecifying an information requirement and a validation-loss budget, then choosing the smallest coefficient satisfying both. If no candidate qualifies, report the requirement as unmet. With neither a specified information requirement nor a group-labeled audit set, retain λ F G = 0 \lambda_{\mathrm{FG}} = 0 λ FG = 0 . A worst-group-accuracy objective requires group labels or a justified group proxy. This threshold-based policy requires prospective validation; the reported experiment maximizes information within an NLL eligibility set.
Tail-Transverse Influence
Fix the predictor, coordinates, calibration, slice rules and finite audit weights π g ≥ 0 \pi_g \ge0 π g ≥ 0 , ∑ g π g = 1 \sum_g \pi_g = 1 ∑ g π g = 1 . An action specifies a positive-semidefinite path J g ( t , a ) = J g + t H g , a + o ( t ) J_g(t,a) = J_g + tH_{g,a} + o(t) J g ( t , a ) = J g + t H g , a + o ( t ) as t ↓ 0 t \downarrow0 t ↓ 0 , with symmetric H g , a H_{g,a} H g , a and an operator-norm remainder. Allocation, normalization and retained pooling belong in this path; audit weights stay fixed.
One differential for exact and ridge targets. Write J g = ( A g B g B g T C g ) J_g = \begin{pmatrix} A_g & B_g \\ B_g^{\mathsf T} & C_g \end{pmatrix} J g = ( A g B g T B g C g ) . Choose a fixed ρ ≥ 0 \rho\ge0 ρ ≥ 0 with C g + ρ I ≻ 0 C_g + \rho I \succ0 C g + ρ I ≻ 0 : ρ = 0 \rho= 0 ρ = 0 gives exact Schur information and ρ = λ r i d \rho= \lambda_{\mathrm{rid}} ρ = λ rid gives the ridge target. Set Q g = ( C g + ρ I ) − 1 Q_g = (C_g + \rho I)^{-1} Q g = ( C g + ρ I ) − 1 , S ρ ( J g ) = A g − B g Q g B g T S_\rho(J_g) = A_g - B_gQ_gB_g^{\mathsf T} S ρ ( J g ) = A g − B g Q g B g T , and let λ g \lambda_g λ g be its smallest eigenvalue. Since D Q g [ H N N ] = − Q g H N N Q g DQ_g[H_{NN}] = -Q_gH_{NN}Q_g D Q g [ H N N ] = − Q g H N N Q g ,
D S ρ ( J g ) [ H ] = H T T − H T N Q g B g T − B g Q g H T N T + B g Q g H N N Q g B g T , (68) DS_\rho(J_g)[H] = H_{TT} - H_{TN}Q_gB_g^{\mathsf T} - B_gQ_gH_{TN}^{\mathsf T}
+ B_gQ_gH_{NN}Q_gB_g^{\mathsf T}, \tag*{(68)} D S ρ ( J g ) [ H ] = H T T − H T N Q g B g T − B g Q g H T N T + B g Q g H N N Q g B g T , (68)
p g ( u ) = ( u − Q g B g T u ) , u T D S ρ ( J g ) [ H ] u = p g ( u ) T H p g ( u ) . p_g(u) = \begin{pmatrix} u \\ -Q_gB_g^{\mathsf T}u \end{pmatrix}, \qquad u^{\mathsf T}DS_\rho(J_g)[H]u = p_g(u)^{\mathsf T}Hp_g(u). p g ( u ) = ( u − Q g B g T u ) , u T D S ρ ( J g ) [ H ] u = p g ( u ) T H p g ( u ) .
For a simple minimum eigenvalue with unit eigenvector u g u_g u g , differentiating the eigenvalue equation therefore gives
h g , a : = D a λ g = p g T H g , a p g , p g = p g ( u g ) . (69) h_{g,a} := D_a\lambda_g = p_g^{\mathsf T}H_{g,a}p_g, \qquad p_g = p_g(u_g). \tag*{(69)} h g , a := D a λ g = p g T H g , a p g , p g = p g ( u g ) . (69)
Let s a s_a s a be the candidate’s score vector and M g , a M_{g,a} M g , a its matrix contribution to group g g g . For allocation rate η g ≥ 0 \eta_g \ge0 η g ≥ 0 , additive and normalized contributions have different derivatives:
H g , a = η g s a s a T ⟹ h g , a = η g ( p g T s a ) 2 , H_{g,a} = \eta_gs_as_a^{\mathsf T} \quad\Longrightarrow\quad h_{g,a} = \eta_g(p_g^{\mathsf T}s_a)^2, H g , a = η g s a s a T ⟹ h g , a = η g ( p g T s a ) 2 ,
J g ( t , a ) = J g + t η g M g , a 1 + t η g ⟹ H g , a = η g ( M g , a − J g ) , (70) J_g(t,a) = \frac{J_g + t\eta_gM_{g,a}}{1+t\eta_g} \quad\Longrightarrow\quad H_{g,a} = \eta_g(M_{g,a}-J_g), \tag*{(70)} J g ( t , a ) = 1 + t η g J g + t η g M g , a ⟹ H g , a = η g ( M g , a − J g ) , (70)
h g , a = η g [ p g T M g , a p g − λ g + ρ ∥ Q g B g T u g ∥ 2 ] . h_{g,a} = \eta_g\left[p_g^{\mathsf T}M_{g,a}p_g-\lambda_g+\rho\lVert Q_gB_g^{\mathsf T}u_g\rVert^2\right]. h g , a = η g [ p g T M g , a p g − λ g + ρ ∥ Q g B g T u g ∥ 2 ] .
Indeed Q g C g Q g = Q g − ρ Q g 2 Q_gC_gQ_g = Q_g-\rho Q_g^2 Q g C g Q g = Q g − ρ Q g 2 , so p g T J g p g = λ g − ρ ∥ Q g B g T u g ∥ 2 p_g^{\mathsf T}J_gp_g = \lambda_g-\rho\lVert Q_gB_g^{\mathsf T}u_g\rVert^2 p g T J g p g = λ g − ρ ∥ Q g B g T u g ∥ 2 ; the correction vanishes at ρ = 0 \rho= 0 ρ = 0 . Per unit allocation, the additive rank-one contribution is the squared nuisance-adjusted residual [ u g ⊤ ( S T , a − B g Q g S N , a ) ] 2 [u_g^\top(S_{T,a}-B_gQ_gS_{N,a})]^2 [ u g ⊤ ( S T , a − B g Q g S N , a ) ] 2 ; at ρ = 0 \rho=0 ρ = 0 , this is the efficient-score residual. Replacement uses the allocated new-minus-removed contribution. For one observation appended to group g g g , its indicator and original sample-count scaling enter η g \eta_g η g ; they cannot be omitted when claiming a normalized-augmentation derivative.
Tail sensitivity, including nonsmooth cases. Fix ε > 0 \varepsilon>0 ε > 0 and 0 < α < 1 0<\alpha<1 0 < α < 1 , and put L g = ( λ g + ε ) − 1 L_g=(\lambda_g+\varepsilon)^{-1} L g = ( λ g + ε ) − 1 . Appendix § 11 .§ 5 's finite CVaR dual uses W α , π = { ω ≥ 0 : ∑ g ω g = 1 , ω g ≤ π g / ( 1 − α ) } \mathcal{W}_{\alpha,\pi}=\{\omega\geq0:\sum_g\omega_g=1,\ \omega_g\leq\pi_g/(1-\alpha)\} W α , π = { ω ≥ 0 : ∑ g ω g = 1 , ω g ≤ π g / ( 1 − α )} . Let Ω α ⋆ ( L ) \Omega_\alpha^\star(L) Ω α ⋆ ( L ) maximize ∑ g ω g L g \sum_g\omega_gL_g ∑ g ω g L g over this set. If the maximizer ω α \omega_\alpha ω α is unique and eigenvalues are simple,
G α ( a ) : = − D a CVaR α , π ( L ) = ∑ g ω α , g h g , a ( λ g + ε ) 2 , D a I α , π = I α , π 2 G α ( a ) . (71) G_\alpha(a):=-D_a\operatorname{CVaR}_{\alpha,\pi}(L)=\sum_g\omega_{\alpha,g}\frac{h_{g,a}}{(\lambda_g+\varepsilon)^2},\qquad D_aI_{\alpha,\pi}=I_{\alpha,\pi}^2G_\alpha(a). \tag*{(71)} G α ( a ) := − D a CVaR α , π ( L ) = g ∑ ω α , g ( λ g + ε ) 2 h g , a , D a I α , π = I α , π 2 G α ( a ) . (71)
Thus G α ( a ) G_\alpha(a) G α ( a ) is the first-order decrease in inverse-information CVaR along action a a a 's specified allocation path. It follows from D a L g = − h g , a / ( λ g + ε ) 2 D_aL_g=-h_{g,a}/(\lambda_g+\varepsilon)^2 D a L g = − h g , a / ( λ g + ε ) 2 ; the dual weights include fractional threshold mass and upper-tail normalization.
For a repeated minimum eigenvalue with eigenspace E g \mathcal{E}_g E g , symmetric eigenvalue perturbation gives the first formula below. At a CVaR tie, the right derivative of a maximum is the maximum of its active directional derivatives; negation gives the second:
h g , a = min u ∈ E g ∥ u ∥ = 1 p g ( u ) ⊤ H g , a p g ( u ) , h_{g,a}=\min_{\substack{u\in\mathcal{E}_g\\\lVert u\rVert=1}}p_g(u)^\top H_{g,a}p_g(u), h g , a = u ∈ E g ∥ u ∥ = 1 min p g ( u ) ⊤ H g , a p g ( u ) ,
G α + ( a ) : = − D a + CVaR α , π ( L ) = min ω ∈ Ω α ⋆ ( L ) ∑ g ω g h g , a ( λ g + ε ) 2 . (72) G_\alpha^+(a):=-D_a^+\operatorname{CVaR}_{\alpha,\pi}(L)=\min_{\omega\in\Omega_\alpha^\star(L)}\sum_g\omega_g\frac{h_{g,a}}{(\lambda_g+\varepsilon)^2}. \tag*{(72)} G α + ( a ) := − D a + CVaR α , π ( L ) = ω ∈ Ω α ⋆ ( L ) min g ∑ ω g ( λ g + ε ) 2 h g , a . (72)
These formulas cover either or both sources of nondifferentiability; the reciprocal-certificate identity holds with right derivatives. An arbitrary fixed maximizing tail vector is only a score unless it attains this direction-dependent minimum.
For assigned label y a y_a y a , tail-transverse influence multiplies the directional information gain by current prediction loss:
TTI ( a ) = G α ( a ) ℓ w ( x a c f , y a ) , ℓ w ( x , y ) = − log p w ( y ∣ x ) . (73) \operatorname{TTI}(a)=G_\alpha(a)\ell_w(x_a^{\mathrm{cf}},y_a),\qquad\ell_w(x,y)=-\log p_w(y\mid x). \tag*{(73)} TTI ( a ) = G α ( a ) ℓ w ( x a cf , y a ) , ℓ w ( x , y ) = − log p w ( y ∣ x ) . (73)
using G α + G_\alpha^+ G α + at nonsmooth points. This ranks actions at the fixed predictor; finite augmentation, refitting and changed normalization require a new audit. The implemented residual-square surrogate G ~ α \widetilde{G}_\alpha G α below omits allocation/baseline terms and is distinguished from this derivative.
Feature-Space Acquisition Experiments
Candidates, budgets and refitting. Using training-standardized frozen MobileNetV3-small representations, let v ˉ y , b \bar{v}_{y,b} v ˉ y , b denote the official-training mean for task label y y y and background b ∈ { 0 , 1 } b\in\{0,1\} b ∈ { 0 , 1 } . Each official-validation candidate becomes
v c f = v + γ c f ( v ˉ y , 1 − b − v ˉ y , b ) , y c f = y , b c f = 1 − b . (74) v^{\mathrm{cf}}=v+\gamma_{\mathrm{cf}}(\bar{v}_{y,1-b}-\bar{v}_{y,b}),\qquad y^{\mathrm{cf}}=y,\qquad b^{\mathrm{cf}}=1-b. \tag*{(74)} v cf = v + γ cf ( v ˉ y , 1 − b − v ˉ y , b ) , y cf = y , b cf = 1 − b . (74)
This preserves the assigned label, not necessarily semantic or pixel-level content. The reported unit-strength run uses label-conditional means for every candidate. A balanced logistic background probe, fitted and standardized on training data, stays fixed and scores the translated features. Test data supply final evaluation only.
Budgets b a c q = .01 , .02 , .05 b_{\mathrm{acq}}=.01,.02,.05 b acq = .01 , .02 , .05 use k = min { max ( 1 , round ( b a c q n t r a i n ) ) , n c a n d } k=\min\{\max(1,\operatorname{round}(b_{\mathrm{acq}}n_{\mathrm{train}})),n_{\mathrm{cand}}\} k = min { max ( 1 , round ( b acq n train )) , n cand } , giving 48, 96, 240 candidates from the 1,199-example validation pool relative to 4,795 original training examples. These are formula-implied counts; historical selected-index records were not recovered. Acquired features and labels augment training, and a balanced logistic task head is fitted from scratch, without an FG penalty or special acquisition weights. Both probe and head use L-BFGS, at most 1,000 iterations, and library defaults (C = 1 C=1 C = 1 , tolerance 10 − 4 10^{-4} 1 0 − 4 , fitted intercept). Class weights are recomputed on the augmented set. New task logits are normalized by augmented-training moments, while nuisance normalization stays fixed. Acquired validation examples are no longer an independent evaluation sample. Seeds are 17–21; the historical scikit-learn version was not pinned.
Implemented score and its scope. Let ( z , q ) (z,q) ( z , q ) be the task logit and nuisance score standardized on original training data. Original validation-group moments are unweighted second moments, without p ( 1 − p ) p(1-p) p ( 1 − p ) ; put β g = J T N , g / ( J N N , g + .001 ) \beta_g=J_{TN,g}/(J_{NN,g}+.001) β g = J T N , g / ( J N N , g + .001 ) , λ g = J T T , g − J T N , g 2 / ( J N N , g + .001 ) \lambda_g=J_{TT,g}-J_{TN,g}^{2}/(J_{NN,g}+.001) λ g = J T T , g − J T N , g 2 / ( J N N , g + .001 ) , and L g = ( λ g + ε ) − 1 L_g=(\lambda_g+\varepsilon)^{-1} L g = ( λ g + ε ) − 1 with ε = 10 − 8 \varepsilon=10^{-8} ε = 1 0 − 8 . Numerical negative information is floored at zero. For the threshold-inclusive tail T = { g : L g ≥ quantile .95 ( L 1 , … , L 4 ) } T=\{g:L_g\ge\operatorname{quantile}_{.95}(L_1,\ldots,L_4)\} T = { g : L g ≥ quantile .95 ( L 1 , … , L 4 )} , the code ranks by
G ~ α ( a ) = ∑ g ∈ T ( λ g + ε ) − 2 ( z a c f − β g q a c f ) 2 max { ∑ g ∈ T ( λ g + ε ) − 2 , 10 − 12 } , T T I ~ α ( a ) = G ~ α ( a ) ℓ w ( v a c f , y a ) . (75) \widetilde{G}_{\alpha}(a)=\frac{\sum_{g\in T}(\lambda_g+\varepsilon)^{-2}(z_a^{\mathrm{cf}}-\beta_gq_a^{\mathrm{cf}})^2}{\max\left\{\sum_{g\in T}(\lambda_g+\varepsilon)^{-2},10^{-12}\right\}},\qquad\widetilde{\mathrm{TTI}}_{\alpha}(a)=\widetilde{G}_{\alpha}(a)\ell_w(v_a^{\mathrm{cf}},y_a). \tag*{(75)} G α ( a ) = max { ∑ g ∈ T ( λ g + ε ) − 2 , 1 0 − 12 } ∑ g ∈ T ( λ g + ε ) − 2 ( z a cf − β g q a cf ) 2 , TTI α ( a ) = G α ( a ) ℓ w ( v a cf , y a ) . (75)
Every tail projector is applied to every candidate, irrespective of its assigned group; for a unique worst group the inverse-square weight cancels. The score lacks group allocation, sample-count scaling, old-information subtraction and replacement terms. Its nonnegative residual square is an unnormalized rank-one contribution at fixed coordinates, not the derivative of normalized augmentation or of the post-refit certificate. Threshold-inclusive ties also differ from the fractional/direction-dependent CVaR convention in Appendix § 15 .§ 4 .
High loss uses ℓ w \ell_w ℓ w ; pure tail selects by G ~ α \widetilde{G}_{\alpha} G α alone. Rank product maps ascending argsort positions for each score evenly from 0 to 1, then multiplies their rank scores; equal values receive separate ranks. Uncertainty product multiplies G ~ α \widetilde{G}_{\alpha} G α by p ( 1 − p ) p(1-p) p ( 1 − p ) . The acquisition baseline has 78.2% average and 52.5% worst-group accuracy, certificate .209, and weakest-information group landbird/land; it is distinct from the predefined-diagnostic cohort.
Table E.5: Feature-space acquisition, means over seeds 17–21. Δ a \Delta a Δ a is worst-group accuracy gain in percentage points and Δ I \Delta I Δ I the absolute ridge-certificate gain. Panel (a) gives the complete strength-by-budget comparison; panel (b) retains pure-tail and alternative-product controls at unit strength. Only aggregate reports survive, so historical seed-level intervals are unavailable.
Table E.5. Feature-space acquisition, means over seeds 17–21. Δ a \Delta a Δ a is worst-group accuracy gain in percentage points and Δ I \Delta I Δ I the absolute ridge-certificate gain. Panel (a) gives the complete strength-by-budget comparison; panel (b) retains pure-tail and alternative-product controls at unit strength.
At γ c f = 1 \gamma_{\mathrm{cf}}=1 γ cf = 1 , the loss product exceeds high loss in mean worst-group accuracy at all budgets, while high loss gives larger certificate gains. This advantage is strength-dependent: high loss wins accuracy at every budget for .25 and 1.5, and at the 5% budget for .5. The rank product also exceeds the chosen loss product at 1% (5.45 versus 4.52 points); the chosen product is not uniformly best. Information gains and predictive gains are distinct outcomes.
Table E.5 reproduces historical reports, not fresh runs. Missing per-seed JSON/CSV, selected indices and pinned versions preclude uncertainty reconstruction. The artifact supplies rerun code, aggregate candidate score/loss summaries, allocation mechanics and protocols. Five seeds on fixed splits do not constitute five independent datasets.
Artifact Availability
Code and supporting artifacts. The public repository and versioned release camera-ready-v22 are available at https://github.com/E-zClap/fisher-glass-learning/releases/tag/camera-ready-v22 .
The release provides experiment code, dependencies, derived numerical inputs, per-seed records, checksums, and commands for reproducing the retained analyses and independently rerunning the experiments. Its reproduction guides specify data preparation, environments, and experiment settings. Its artifact manifest maps manuscript tables to the supporting files and identifies the corrected hidden-slice cohort. Expanded appendix sources retain displaced execution details and numerical records; the manuscript map records consolidated table labels. The prospective matrix and auxiliary flip-prediction reanalyses can be replayed from the included frozen numerical inputs. Selected image examples used in Figure 1 are included with source provenance. Full benchmark datasets and pretrained weights are not redistributed; dataset sources and access terms are documented separately. Historical synthetic and frozen-training summaries and the acquisition results retain their stated provenance: where original per-seed outputs are unavailable, the release includes the supporting aggregate reports and executable rerun workflows; fresh experiment replication does not imply exact replay of missing historical outputs.