betaFeedback

WorksComputer Science

Overcoming the Resolution Limit: Significance-Aware Regularization for Intersectional Fairness

Abstract

Intersectional fairness requires evaluating models across fine-grained subgroups. However, as group definitions become more granular, the sample size per subgroup shrinks, causing fairness estimates to become increasingly unstable. Existing in-processing methods are certainty-blind: they penalize observed disparities without distinguishing statistically certified unfairness from sampling noise. This leads to fairness overfitting, where models sacrifice utility to "fix" spurious disparities in sparse regimes. To address this, we introduce the significant-excess metric family, a continuous, uncertainty-aware fairness score that measures only the portion of a subgroup's disparity exceeding what sampling variation can explain at a specific confidence level. Building on this metric, we propose SAFER (Significance-Aware Fairness Regularizer), a differentiable in-processing method that gates fairness penalties using a soft-count likelihood-ratio evidence statistic. SAFER satisfies a provable safety property: when every subgroup disparity lies well below the gate threshold, the unconstrained optimum is nearly stationary for SAFER. At that optimum, the back-propagated contribution of its significance-gated component is exponentially small, and the additional fairness contribution comes from a small ungated term with a fixed weight. Mechanism ablations across three benchmark datasets and purpose-built stress variants show that removing any component of SAFER (threshold smoothing, the significance gate, or a dense-tail term) produces the failure the theory predicts on at least one stress variant. In sparse intersectional settings, SAFER achieves a better fairness-utility Pareto frontier than the baseline methods, and it preserves utility when certified unfairness is absent.

Introduction

Intersectional fairness [8, 16] requires evaluating models across conjunctions of protected attributes, but as subgroups become more granular, their sample sizes inevitably shrink. In these sparse regimes, fairness estimates become noise-dominated, and intersectional fairness reaches a resolution limit: the point at which the statistical uncertainty exceeds the disparity being measured [38, 14, 13]. Indeed, even widely adopted heuristics such as the four-fifths rule lack statistical grounding and can both miss and fabricate disparities [35]. Ignoring estimation uncertainty leads to fairness overfitting, where the model memorizes group noise in the training set rather than genuine bias [28]. Existing in-processing methods are certainty-blind: they penalize observed disparities without distinguishing certified unfairness from sampling noise, causing fairness overfitting in the sparse intersectional regime. Conversely, audit tools such as [13] incorporate uncertainty awareness, but only as a post-hoc binary accept/reject decision rather than a continuous metric or a differentiable training signal. If fairness regularization is to behave sensibly in sparse intersectional regimes, the optimization criterion itself must be able to distinguish well-supported disparities from noisy ones.

40th Conference on Neural Information Processing Systems (NeurIPS 2026). Whether gating on significance helps depends on how many training subgroups lie above the gate threshold. If none do, the gated fairness contributions are suppressed. If some do, the gate selects which disparities receive greater weight. If all do, the gate is effectively inactive.

In this context, this paper makes three main contributions:

  • Uncertainty-aware fairness measurement (Section 3). We introduce the significant-excess metric family, which measures only the portion of each subgroup’s disparity that exceeds what sampling variation can explain at a given confidence level. The construction applies to any parity criterion expressible as a continuously differentiable function of multinomial cell probabilities, including Statistical Parity [11], Disparate Impact [12], Equalized Odds and Equal Opportunity [17], and yields a continuous, uncertainty-aware fairness score that is zero when no violation is statistically certified.

  • Significance-gated optimization (Section 4). We propose SAFER (Significance-Aware Fairness Regularizer), a differentiable in-processing method that gates fairness penalties using a soft-count likelihood-ratio evidence statistic. The gate attenuates subgroup penalties according to their evidence of disparity; statistical certification comes from the held-out audit. SAFER satisfies a provable safety property: when every disparity lies well below the gate threshold, the optimum of unconstrained empirical risk minimization (ERM) is nearly stationary for SAFER. At that optimum, the back-propagated contribution of its significance-gated component is exponentially small, and the additional fairness contribution comes from a small ungated term with a fixed weight.

  • Regime characterization and empirical validation (Section 5 and Appendix F.3). We identify three operating regimes (safety, selective gating, gate always open), defined by which training subgroups cross the gate’s threshold, and test each through mechanism ablations on purpose-built stress variants and a controlled benchmark.

The claims are regime-conditional: SAFER gives a better Pareto frontier than the baselines where certified disparities exist and preserves the utility of the unconstrained model where they do not, and removing any component of SAFER produces the failure the theory predicts on at least one stress variant.

Related Work

Intersectional and subgroup fairness. Equality of Opportunity [17] and related parity criteria are the standard building blocks for group fairness, but models can satisfy coarse constraints yet violate fairness on structured intersections [25]. Differential fairness treats intersectionality as a first-class object [23]; worst-case comparisons formalize the combinatorial structure of multi-attribute subgroups [15]; and multicalibration [19] provides simultaneous calibration guarantees over rich subgroup families. Molina and Loiseau [32] bound intersectional violations via marginal constraints; Himmerleich et al. [20] formalize the tension between group granularity and statistical power. Our work targets the same setting but a different bottleneck: once many intersectional groups are audited simultaneously, their disparities become heterogeneous in statistical reliability, and this reliability must enter both evaluation and optimization.

Uncertainty-aware fairness. SAFT [13] is the work closest to ours: it tests whether each subgroup’s confidence interval excludes parity, applying multiplicity corrections across the subgroup family. This makes subgroup size and estimator variance part of the fairness decision; certainty-blind metrics [22] ignore both. We build on SAFT in two directions. First, we replace binary per-subgroup flags with a continuous significant-excess score that preserves severity information beyond the accept/reject decision, yielding a single metric for ranking models (Section 3). Second, this statistical logic motivates a differentiable evidence gate that modulates subgroup penalties during training. SAFER keeps SAFT’s intervals for the audit and trains with a soft-count likelihood-ratio statistic, because a plug-in Wald standard error is unstable on small soft counts and a posterior-sampling test is too expensive inside a training loop (Section 4.2). More broadly, a growing body of work grounds fairness evaluation in statistical inference: Cherian and Candes [7] provide finite-sample valid tests for auditing; Tang et al. [37] replace Wald statistics with empirical-likelihood tests; and Lo et al. [30] advocate classical hypothesis testing for AI fairness. On the evaluation side, Barrainkua et al. [3] show that fairness conclusions are unstable under resampling, and Rosenblatt and Witter [36] benchmark how uncertainty can reverse fairness rankings. Kuzucu et al. [27] treat predictive uncertainty itself as a fairness criterion, whereas we use statistical uncertainty of the fairness estimate to gate optimization. Besse et al. [5] were among the first to propose confidence intervals for testing disparate impact, which replaced point estimates with interval-based decisions.

Fairness-aware optimization. A large literature mitigates unfairness during or after training. Post-processing methods adjust a fixed classifier’s thresholds [17], but cannot reshape the learned representation and become impractical when the number of intersectional subgroups is combinatorially large. Among in-processing approaches, Exponentiated Gradient [1] solves a sequence of weighted learning problems, Adversarial Debiasing [40] and Prejudice Remover [24] embed fairness pressure through an adversary or a dependence penalty, and differentiable regularizers such as Bechavod and Ligett [4] and FAIRRET [6] penalize group discrepancy via smooth surrogates. Recent intersectional-specific optimizers include Lee et al. [29], who regularize over multitype attributes, and Nemeček et al. [33], who enforce exact intersectional constraints via mixed-integer programming. Both remain certainty-blind. Distributionally robust optimization (DRO) [18] and minimax group fairness [9] minimize worst-case subgroup loss, while FairDRO [34] casts group fairness as classwise robust optimization. All these methods are certainty-blind: once a subgroup disparity is observed, it contributes to the objective regardless of whether it is statistically distinguishable from sampling noise. SAFER differs by using a likelihood-ratio evidence gate to modulate disparity penalties during training, while retaining the held-out audit for statistical certification.

Measuring Intersectional Fairness

We introduce the significant-excess metric family to address the resolution limit identified in § 1. To this end, these metrics are designed to answer two questions simultaneously: does a subgroup exhibit a non-zero disparity compared to the rest of the population, and is that disparity statistically significant enough to be distinguished from sampling noise?

Setup. Let S\mathcal{S} be a family of protected subgroups, including intersectional groups formed by conjunctions of sensitive attributes. For each subgroup S∈SS \in\mathcal{S}, we compare the behavior of a model on SS and its complement Sˉ\bar{S}. Both are sets of individuals: SS collects those with the subgroup’s attribute values and Sˉ\bar{S} the rest of the population. We write ϕ(S)\phi(S) for a generic fairness functional expressible as a continuously differentiable function of multinomial cell probabilities. We write ϕn(S)\phi_n(S) for its empirical estimate on a test sample of size nn. For each subgroup disparity ϕn(S)\phi_n(S) we require a confidence interval of level (1−α)(1-\alpha), expressible by a lower and upper bound [ℓα(S),uα(S)][\ell_\alpha(S), u_\alpha(S)], which quantifies the uncertainty of the estimation. We assume ℓα(S)≤ϕn(S)≤uα(S)\ell_\alpha(S) \le\phi_n(S) \le u_\alpha(S).

Definition 1 (Significant excess). For a subgroup SS and significance level α\alpha, we define the significant excess of fairness criterion ϕ\phi as

Eαϕ(S)={max⁡(0,ℓα(S))if ϕn(S)≥0,max⁡(0,−uα(S))if ϕn(S)<0.(1)\mathcal{E}^{\phi}_{\alpha}(S) = \begin{cases} \max(0,\ell_{\alpha}(S)) & \text{if }\phi_n(S)\ge0,\\ \max(0,-u_{\alpha}(S)) & \text{if }\phi_n(S)<0. \end{cases} \tag*{(1)}

The significant excess is the distance from zero to the interval. When the interval contains zero, the disparity is not statistically certified and the contribution is exactly zero. When the interval excludes zero, Eαϕ(S)\mathcal{E}^{\phi}_{\alpha}(S) measures the portion of the observed disparity that remains after subtracting what sampling noise can plausibly explain.

Basic properties. The significant-excess score satisfies:

  1. Zero iff no certified unfairness under the interval rule. Eαϕ(S)=0\mathcal{E}^{\phi}_{\alpha}(S)=0 if and only if the interval for ϕ(S)\phi(S) contains zero.

  2. Scale preservation. Eαϕ(S)\mathcal{E}^{\phi}_{\alpha}(S) is expressed in the same units as ϕ\phi.

  3. Uncertainty awareness. The score is nonincreasing when the interval expands by inclusion.

  4. Continuity. The score is continuous in the interval endpoints.

Finally, we refer to the Mean Significant Measure Excess as the mean aggregation over the nonempty audited family of subgroups with defined estimates and intervals:

E‾αϕ(f)=1∣S∣∑S∈SEαϕ(S).(2)\overline{\mathcal{E}}^{\phi}_{\alpha}(f) = \frac{1}{|\mathcal{S}|}\sum_{S\in\mathcal{S}}\mathcal{E}^{\phi}_{\alpha}(S). \tag*{(2)}

Let us consider two fairness metrics as an example, the Statistical Parity (SP) gap and the Equal Opportunity (EO) gap:

SP(S)=P(Y^=1∣S)−P(Y^=1∣S‾),(3)SP(S) = P(\widehat{Y}=1 \mid S) - P(\widehat{Y}=1 \mid\overline{S}), \tag*{(3)}
EO(S)=P(Y^=1∣Y=1,S)−P(Y^=1∣Y=1,S‾).(4)EO(S) = P(\widehat{Y}=1 \mid Y=1,S) - P(\widehat{Y}=1 \mid Y=1,\overline{S}). \tag*{(4)}

For SP, the relevant cells are predicted positive/negative outcomes conditioned on SS and S‾\overline{S}. For EO, the same construction is applied after conditioning on the positive-label population Y=1Y=1. Both can be written as continuously differentiable functions of multinomial cell probabilities. Given a confidence level α\alpha, we can define the Mean Significant Parity Excess (MSPE) by applying Equations (1) and (2) with ϕ(S)=SP(S)\phi(S)=SP(S), and the Mean Significant Equal-Opportunity Excess (MSEE) with ϕ(S)=EO(S)\phi(S)=EO(S).

MSPEα(f)=E‾αSP(f),MSEEα(f)=E‾αEO(f).(5)MSPE_{\alpha}(f)=\overline{\mathcal{E}}_{\alpha}^{SP}(f), \qquad MSEE_{\alpha}(f)=\overline{\mathcal{E}}_{\alpha}^{EO}(f). \tag*{(5)}

We obtain the confidence intervals [ℓα(S),uα(S)][\ell_{\alpha}(S),u_{\alpha}(S)] with the SAFT framework [13], because its test adapts to both large and small subgroups.

SAFER: A Differentiable Surrogate for Significant-Excess Metrics

MSEEαMSEE_{\alpha} (§ 3) measures only the statistically certified portion of each subgroup’s disparity. It is not directly trainable, because it depends on hard predictions, which are piecewise constant in θ\theta, and on an absolute value and a positive part, which are not differentiable at zero. SAFER replaces these with differentiable approximations and is built around two ideas. First, a soft-count likelihood-ratio test (LRT) provides a differentiable significance statistic that requires no explicit variance estimation. This matters in the sparse intersectional cells, where Wald-type statistics become unreliable (§4.2). Second, a significance gate built from this statistic yields a provable safety property: when every subgroup disparity lies well below the gate threshold, the ERM optimum is nearly stationary for SAFER. At that optimum, the back-propagated contribution of the significance-gated component is exponentially small, and the additional fairness contribution is proportional to the weight of a small ungated term (§4.3). Proofs are in Appendix § 17.

Threshold-Smoothed Soft Counts

Let fθ:X→[0,1]f_{\theta}:\mathcal{X}\to[0,1] be the model score. For each subgroup S∈SS\in\mathcal{S} write ΠS\Pi_S for the conditioning population: ΠS=S\Pi_S=S for Statistical Parity, and ΠS=(S,Y=1)\Pi_S=(S,Y=1) for Equal Opportunity, matching § 3. Define three population positive rates, relative to the hard binary predictions, the raw predicted probabilities and the τ\tau smoothed sigmoid:

pShard(θ)=E[1{fθ(X)>12}∣ΠS],(6)p_{S}^{\mathrm{hard}}(\theta)=\mathbb{E}\left[1\left\{f_{\theta}(X)>\frac{1}{2}\right\}\mid\Pi_S\right], \tag*{(6)}
pSraw(θ)=E[fθ(X)∣ΠS],(7)p_{S}^{\mathrm{raw}}(\theta)=\mathbb{E}[f_{\theta}(X)\mid\Pi_S], \tag*{(7)}
pSτ(θ)=E[στ(fθ(X)−12)∣ΠS],στ(u)=σ(u/τ),(8)p_{S}^{\tau}(\theta)=\mathbb{E}\left[\sigma_{\tau}\left(f_{\theta}(X)-\frac{1}{2}\right)\mid\Pi_S\right], \qquad\sigma_{\tau}(u)=\sigma(u/\tau), \tag*{(8)}

where σ\sigma is the logistic sigmoid. The corresponding disparity is ϕS∙=pS∙−pS‾∙\phi_{S}^{\bullet}=p_{S}^{\bullet}-p_{\overline{S}}^{\bullet} for each variant ∙∈{hard,raw,τ}\bullet\in\{\mathrm{hard},\mathrm{raw},\tau\}. For example in this notation, ϕShard\phi_{S}^{\mathrm{hard}} is exactly the population functional ϕ(S)\phi(S) of § 3. Furthermore, we use hats (e.g., ϕ^Sτ\widehat{\phi}_{S}^{\tau}) to denote the empirical analogues computed from sample soft counts. The decision threshold 1/21/2 is the Bayes rule when the scores estimate class probabilities and both error types cost the same. It is a property of the deployed classifier and is fixed before training. The construction applies unchanged to any other fixed threshold tt by replacing fθ(x)−12f_{\theta}(x)-\frac{1}{2} with fθ(x)−tf_{\theta}(x)-t; Appendix § 13.§ 1 reports SAFER retrained and audited at thresholds from 0.1 to 0.9.

In particular, replacing 1{fθ(x)>12}1\{f_{\theta}(x)>\frac{1}{2}\} with στ(fθ(x)−12)\sigma_{\tau}(f_{\theta}(x)-\frac{1}{2}) yields a differentiable approximation whose quality is controlled linearly by τ\tau.

Theorem 2. If, for some δ>0\delta>0, the conditional score distributions on ΠS\Pi_S and ΠS‾\Pi_{\overline{S}} admit densities bounded by LL on {u:∣u−12∣≤δ}\{u:|u-\frac{1}{2}|\le\delta\}, then ∣ϕSτ(θ)−ϕShard(θ)∣≤4Lτlog⁡2+2e−δ/τ|\phi_{S}^{\tau}(\theta)-\phi_{S}^{\mathrm{hard}}(\theta)|\le4L\tau\log2+2e^{-\delta/\tau}.

A natural alternative, using the raw probability fθ(x)f_{\theta}(x) directly (pSrawp_{S}^{\mathrm{raw}}), does not work:

Proposition 3 (Raw soft counts are not hard-decision consistent). For every η>0\eta> 0, there exist continuous bounded-density score distributions such that ∣ϕSraw(θ)−ϕShard(θ)∣≥1−η\left|\phi^{\mathrm{raw}}_{S}(\theta)-\phi^{\mathrm{hard}}_{S}(\theta)\right|\ge1-\eta, and no constant rescaling of ϕSraw\phi^{\mathrm{raw}}_{S} matches ϕShard\phi^{\mathrm{hard}}_{S} uniformly.

Two groups sitting just across the 12\frac{1}{2} boundary from opposite sides produce a hard gap near 1 but a raw-probability gap near 0; two groups both well above the boundary produce the reverse. No single correction can fix both regimes, so raw soft counts can rank models differently from the hard-decision audit. Threshold smoothing also localizes gradients to examples near the decision boundary, where predictions can flip (Theorem 7 in Appendix § 17).

A Differentiable LRT for Subgroup Disparities

Given threshold-smoothed soft counts, we need a differentiable evidence statistic to select subgroup disparities for training. An approach would be the Bernoulli-form Wald statistic ∣ZS∣=∣ϕ^S∣/SE^Sϕ^|Z_S| = |\widehat{\phi}_S|/\widehat{SE}^{\widehat{\phi}}_S, where SE^Sϕ^\widehat{SE}^{\widehat{\phi}}_S is the two-proportion standard-error proxy of property (ii), but this has two problems in the sparse intersectional regime. First, SE^Sϕ^\widehat{SE}^{\widehat{\phi}}_S is numerically unstable when soft counts are small. The second problem matters more for optimization: the Wald loss can decrease by inflating the standard-error estimate rather than reducing the disparity, since both numerator and denominator depend on θ\theta.

SAFER instead uses the Bernoulli-form two-sample deviance statistic computed on threshold-smoothed soft counts. For subgroup SS define the soft counts n~1,S(θ)=∑i∈ΠSστ(fθ(xi)−12)\widetilde{n}_{1,S}(\theta)=\sum_{i\in\Pi_S}\sigma_\tau(f_\theta(x_i)-\frac{1}{2}) and n~0,S(θ)=∣ΠS∣−n~1,S(θ)\widetilde{n}_{0,S}(\theta)=|\Pi_S|-\widetilde{n}_{1,S}(\theta). Let n~S=n~1,S+n~0,S\widetilde{n}_S=\widetilde{n}_{1,S}+\widetilde{n}_{0,S} and define analogous quantities for S‾\overline{S}. Write p^S=n~1,S/n~S\widehat{p}_S=\widetilde{n}_{1,S}/\widetilde{n}_S, p^S‾=n~1,S‾/n~S‾\widehat{p}_{\overline{S}}=\widetilde{n}_{1,\overline{S}}/\widetilde{n}_{\overline{S}}, and p^=(n~1,S+n~1,S‾)/(n~S+n~S‾)\widehat{p}=(\widetilde{n}_{1,S}+\widetilde{n}_{1,\overline{S}})/(\widetilde{n}_S+\widetilde{n}_{\overline{S}}) for the pooled null estimate. The likelihood-ratio statistic for the soft-parity null H0:pS=pS‾H_0:p_S=p_{\overline{S}} is

ΛS(θ)=−2[n~1,Slog⁡p^p^S+n~0,Slog⁡1−p^1−p^S+n~1,S‾log⁡p^p^S‾+n~0,S‾log⁡1−p^1−p^S‾].(9)\Lambda_S(\theta)=-2\left[\widetilde{n}_{1,S}\log\frac{\widehat{p}}{\widehat{p}_S}+\widetilde{n}_{0,S}\log\frac{1-\widehat{p}}{1-\widehat{p}_S}+\widetilde{n}_{1,\overline{S}}\log\frac{\widehat{p}}{\widehat{p}_{\overline{S}}}+\widetilde{n}_{0,\overline{S}}\log\frac{1-\widehat{p}}{1-\widehat{p}_{\overline{S}}}\right]. \tag*{(9)}

For SP, these are soft counts of predicted positive/negative across SS and S‾\overline{S}; for EO, the same construction restricts to the positive-label population.

Three properties make the soft-count LRT a suitable significance statistic for intersectional fairness training:

  • (i) No variance estimation. ΛS\Lambda_S is a closed-form function of the soft counts: it never estimates or differentiates through SE^Sϕ^\widehat{SE}^{\widehat{\phi}}_S. This removes the gradient path through which the Wald loss can be lowered by inflating the standard error.

  • (ii) Boundary stability. In sparse cells the Bernoulli-form variance proxy (SE^Sϕ^)2=p^S(1−p^S)/n~S+p^S‾(1−p^S‾)/n~S‾(\widehat{SE}^{\widehat{\phi}}_S)^2=\widehat{p}_S(1-\widehat{p}_S)/\widetilde{n}_S+\widehat{p}_{\overline{S}}(1-\widehat{p}_{\overline{S}})/\widetilde{n}_{\overline{S}} is unstable: with only a few effective observations, threshold-smoothed rates can approach zero or one, the variance estimate collapses toward zero, and ∣ZS∣|Z_S| grows without bound, so a few tiny, saturated subgroups can dominate the objective. ΛS\Lambda_S is computed from the Bernoulli log-likelihood itself and contains no division by a plug-in variance estimate that can vanish, so it avoids this boundary singularity.

  • (iii) Asymptotic equivalence. For interior rates and vanishing disparities, a second-order expansion gives ΛS=ZS2(1+op(1))\Lambda_S=Z_S^2(1+o_p(1)), so the two statistics use the same reference cutoff χα\chi_\alpha. For a fixed predictor evaluated on independent observations under soft parity, ΛS⇒cχ12\Lambda_S\Rightarrow c\chi^2_1, 0≤c≤10\le c\le1, and the Bernoulli reference cutoff is asymptotically conservative (Appendix § 17.§ 6).

During training the gate is an optimization device; statistical certification is provided by the held-out hard-count audit of § 3.

LRT-based threshold scale. To express the significance threshold in the raw-disparity units of MSEαMSE_\alpha when the disparity is nonzero, we define an effective standard-error proxy from the empirical disparity and the deviance:

SE^eff,S(θ)=∣ϕ^S(θ)∣ΛS(θ).(10)\widehat{SE}_{\mathrm{eff},S}(\theta)=\frac{\left|\widehat{\phi}_S(\theta)\right|}{\sqrt{\Lambda_S(\theta)}}. \tag*{(10)}

This conversion puts the dimensionless evidence statistic on the disparity scale, so that the training threshold carries the units of ϕ\phi. Under the local conditions of property (iii), SE^eff,S=SE^Sϕ(1+op(1))\widehat{SE}_{\mathrm{eff},S}=\widehat{SE}^{\phi}_{S}(1+o_{p}(1)), recovering the Bernoulli-form normalization. Before numerical clamping, the threshold therefore scales with 1/nS+1/nSˉ\sqrt{1/n_S+1/n_{\bar S}} for SP and 1/mS+1/mSˉ\sqrt{1/m_S+1/m_{\bar S}} for EO. It thus shrinks as the inverse square root of the sample size when both comparison groups grow in proportion. The implementation regularizes the ratio and clamps it for numerical stability.

The trained objective. SAFER penalizes excess disparity above the LRT-based training threshold, with the threshold detached from the gradient computation:

Ldir(θ)=1∣S∣∑S∈Ssoftplus⁡τs(ϕ^Sτ(θ)2+ε2−zαsg⁡[SE^eff,S(θ)])(11)\mathcal{L}_{\mathrm{dir}}(\theta)=\frac{1}{|\mathcal{S}|}\sum_{S\in\mathcal{S}}\operatorname{softplus}_{\tau_s}\left(\sqrt{\widehat{\phi}_{S}^{\tau}(\theta)^2+\varepsilon^2}-z_{\alpha}\operatorname{sg}\left[\widehat{SE}_{\mathrm{eff},S}(\theta)\right]\right) \tag*{(11)}

where softplus⁡τ(u)=τlog⁡(1+eu/τ)\operatorname{softplus}_{\tau}(u)=\tau\log(1+e^{u/\tau}) and sg⁡[⋅]\operatorname{sg}[\cdot] denotes stop-gradient (the enclosed expression is treated as a constant during back-propagation). Detachment ensures that the optimizer can only reduce the loss by shrinking ϕ^Sτ\widehat{\phi}^{\tau}_{S}, not by inflating the threshold. The threshold still updates between optimizer steps as θ\theta changes, so it remains adaptive: it does not provide a gradient path. Within one update the gated term therefore gives the optimizer no first-order incentive to raise the threshold; across updates the threshold follows the predictions, which every term of the objective changes. Appendix J.9 gives a scalar example and a diagnostic over training runs, in which the mean threshold shows no upward drift. A canonical version of Eq. (11), in which the surrogate and the metric share the population Wald threshold BS=zαSESϕB_S=z_{\alpha}SE^{\phi}_{S}, approximates MSEEαpopMSEE_{\alpha}^{\mathrm{pop}} (Eq. (A.8)) pointwise with error at most ε+τslog⁡2+4Lτlog⁡2+2e−δ/τ\varepsilon+\tau_s\log2+4L\tau\log2+2e^{-\delta/\tau} (Theorem 8 in Appendix J).

Safety Property and Operating Regimes

The softplus gate in Ldir\mathcal{L}_{\mathrm{dir}} separates subgroups above the gate threshold from those below it. Let

uS(θ)=ϕ^Sτ(θ)2+ε2−zαsg⁡[SE^eff,S(θ)].u_S(\theta)=\sqrt{\widehat{\phi}_{S}^{\tau}(\theta)^2+\varepsilon^2}-z_{\alpha}\operatorname{sg}\left[\widehat{SE}_{\mathrm{eff},S}(\theta)\right].

Since the stop-gradient (sg) freezes the threshold for differentiation, ∇θuS=∇θϕ^Sτ(θ)2+ε2\nabla_{\theta}u_S=\nabla_{\theta}\sqrt{\widehat{\phi}_{S}^{\tau}(\theta)^2+\varepsilon^2}. Throughout, ∇θ\nabla_{\theta} of an expression containing sg denotes the back-propagated fairness contribution with the detached quantity held fixed; Proposition 4 bounds it. The chain rule and softplus⁡τs′(u)=σ(u/τs)\operatorname{softplus}'_{\tau_s}(u)=\sigma(u/\tau_s) give

∥∇θsoftplus⁡τs(uS)∥≤σ(uS/τs)∥∇θuS∥.(12)\left\|\nabla_{\theta}\operatorname{softplus}_{\tau_s}(u_S)\right\|\leq\sigma(u_S/\tau_s)\left\|\nabla_{\theta}u_S\right\|. \tag*{(12)}

When uS<0u_S<0 (disparity below the gate threshold), the prefactor σ(uS/τs)\sigma(u_S/\tau_s) decays exponentially as euS/τse^{u_S/\tau_s}, suppressing the fairness gradient on that subgroup.

Dense tail. Under strong regularization, active disparities may stay close to the hinge boundary where the softplus gate attenuates the gradient substantially: by Eq. (12) the gradient is multiplied by σ(uS/τs)\sigma(u_S/\tau_s), which equals 1/21/2 at the gate boundary and is already below 0.010.01 at uS=−0.05u_S=-0.05 for τs=10−2\tau_s=10^{-2}, so under strong regularization the optimization can stall before an active disparity is removed. To prevent stalling, SAFER adds a small ungated term:

Ltail(θ)=wtail∣S∣∑S∈SΛS(θ).(13)\mathcal{L}_{\mathrm{tail}}(\theta)=\frac{w_{\mathrm{tail}}}{|\mathcal{S}|}\sum_{S\in\mathcal{S}}\sqrt{\Lambda_S(\theta)}. \tag*{(13)}

where wtail≥0w_{\mathrm{tail}}\geq0. This term carries no gate multiplier, so it provides a gradient even when the softplus is saturated. Since ΛS\sqrt{\Lambda_S} locally expresses the disparity in units of the Bernoulli-form normalization, the tail weights each subgroup by its statistical support (Appendix J.7). Because the term is ungated, the safety bound below includes an additional term proportional to wtailw_{\mathrm{tail}}, while the gated part of the gradient keeps its exponential bound; with wtail=0w_{\mathrm{tail}}=0 the objective is gated only. We use the fixed value wtail=0.003w_{\mathrm{tail}}=0.003 in all runs of the full method. When active subgroups dominate, the tail is negligible relative to Ldir\mathcal{L}_{\mathrm{dir}}.

Proposition 4 (Safety at the unconstrained optimum). If uS(θERM⋆)≤−ϵ⋆u_S(\theta_{\mathrm{ERM}}^{\star})\leq-\epsilon^{\star} for every S∈SS\in\mathcal{S}, and the score and statistic gradients are bounded near θERM⋆\theta_{\mathrm{ERM}}^{\star}, then

∥∇θLdir(θERM⋆)∥=O(e−ϵ⋆/τs).(14)\left\|\nabla_{\theta}\mathcal{L}_{\mathrm{dir}}(\theta_{\mathrm{ERM}}^{\star})\right\|=O\left(e^{-\epsilon^{\star}/\tau_s}\right). \tag*{(14)}
∥∇θ(Ldir+Ltail)(θERM⋆)∥≤O(e−ϵ⋆/τs)+wtail∣S∣∑S∈S∥∇θΛS(θERM⋆)∥.(15)\left\|\nabla_{\theta}\left(\mathcal{L}_{\mathrm{dir}}+\mathcal{L}_{\mathrm{tail}}\right)(\theta_{\mathrm{ERM}}^{\star})\right\|\leq O\left(e^{-\epsilon^{\star}/\tau_s}\right)+\frac{w_{\mathrm{tail}}}{|\mathcal{S}|}\sum_{S\in\mathcal{S}}\left\|\nabla_{\theta}\sqrt{\Lambda_S(\theta_{\mathrm{ERM}}^{\star})}\right\|. \tag*{(15)}

When every subgroup lies below the gate boundary by a margin ϵ⋆\epsilon^{\star} large relative to τs\tau_s, the ERM optimum is nearly stationary for SAFER: the gated fairness contribution there is exponentially small, and the dense-tail residual (Eq. (13)) enters the full objective multiplied by λwtail\lambda w_{\mathrm{tail}}.

Three operating regimes. The gate threshold determines which subgroups contribute to the fairness gradient. Write A(θ)={S∈S:uS(θ)>0}A(\theta)=\{S\in\mathcal{S}:u_S(\theta)>0\} for the active set of training subgroups: a subgroup is active when uS>0u_S>0, which, up to the smoothing constant ϵ\epsilon and the numerical clamps, means TS=ΛS>zαT_S=\sqrt{\Lambda_S}>z_\alpha, with zα=1.96z_\alpha=1.96 at α=0.05\alpha=0.05. At a given checkpoint θ\theta, the active set places a dataset in one of three regimes:

  1. (i) Safety (A(θ)=∅A(\theta)=\varnothing): when all subgroups at the ERM point lie well below the boundary relative to τs\tau_s, the gated fairness contribution is exponentially small, and the unconstrained optimum is nearly stationary for SAFER up to the dense-tail residual.

  2. (ii) Selective gating (0<∣A(θ)∣<∣S∣0<|A(\theta)|<|\mathcal{S}|, active and inactive subgroups coexist): the gate concentrates model capacity on the disparities for which the data provide statistical support.

  3. (iii) Gate always open (A(θ)=SA(\theta)=\mathcal{S}, all subgroups above the boundary): when all lie well above it relative to τs\tau_s, the gate is effectively inactive, and SAFER approaches a certainty-blind penalty.

The regime is read off the active set at a checkpoint; when we characterize a dataset we use the checkpoint of the unconstrained model, and the regime can change during training. Whether a subgroup is active depends on its disparity, on its outcome rates, and on the counts on both sides of the comparison (Appendix § 17.8), so the boundary between regimes is set by the evidence in each subgroup. Appendix § 13.§ 3 states the hypothesis these regimes suggest and tests it on a controlled benchmark.

Full objective. For fairness criterion ϕ\phi, SAFER minimizes

LSAFER(θ)=Lpred(θ)+λ[Ldir(θ)+Ltail(θ)],(16)\mathcal{L}_{\mathrm{SAFER}}(\theta)=\mathcal{L}_{\mathrm{pred}}(\theta)+\lambda\left[\mathcal{L}_{\mathrm{dir}}(\theta)+\mathcal{L}_{\mathrm{tail}}(\theta)\right], \tag*{(16)}

where Lpred\mathcal{L}_{\mathrm{pred}} is binary cross-entropy and λ≥0\lambda\ge0 controls the fairness–utility tradeoff.

Extensions. The construction extends to any fairness functional differentiable in multinomial cell probabilities: substituting SP gives MSPE directly, Equalized Odds adds an FPR-parity hinge, and Disparate Impact uses the log-DI metric with the same SP parity null (Appendix § 16).

Experiments

The experiments serve two purposes. First, we compare SAFER on the Pareto frontier against the strongest available in-processing fairness baselines on three datasets spanning sample sizes and disparity regimes. Second, we use constructed stress variants of the datasets (low training budgets, deeper intersection levels and small validation budgets) to probe the three mechanisms predicted by § 4. Each variant targets a setting in which one SAFER component should be needed, so removing that component in an ablation should have a measurable effect. The full codebase is available at https://github.com/Ambr.../Safer.

Datasets and Stress Variants. Public benchmarks with multiple protected attributes and non-trivial intersectional structures are scarce. The main figures use the three most established ones (Adult Income [26], COMPAS [2] and German Credit [21]). Table 1 summarizes their stress variants, each of which targets one of the mechanisms of § 4. The table also reports ACS Coverage [10] and Taiwan Credit [39], which add a public-coverage task with larger samples and a credit-risk task with many intersections. Unless a train-test split or intersection level is stated explicitly, dataset names below refer to the default variants in Table 1. The default COMPAS variant uses the more severe 20/80 train-test split and is reserved for mechanism ablations; the headline COMPAS comparison instead uses a balanced 50/50 split for a reasonable training budget.

DatasetLevelSplitntrainn_{\mathrm{train}}∣S∣|\mathcal{S}|nˉS / mˉS\bar{n}_{S}\ /\ \bar{m}_{S}Targeted mechanism
Adult Income410/904,52222819.9 / 4.9threshold smoothing: high intersectionality, low budget
COMPAS320/809933528.4 / 14.2dense tail: small training set, strong regularization
German Credit350/505003215.6 / 10.9significance gate: safety regime
ACS Coverage350/5025,00071352.1 / 103.5higher-sample ACS benchmark
Taiwan Credit450/5015,00028552.6 / 11.6high-intersectionality credit risk

Table 1. Datasets, stress variants, and the mechanism each variant probes. ∣S∣|\mathcal{S}| is the number of audited intersectional subgroups at the indicated level; nˉS\bar{n}_{S} is the mean training-set subgroup size; mˉS\bar{m}_{S} is the mean positive-label subgroup size relevant to EO. All values are approximate averages over splits.

Adult probes the threshold-smoothing mechanism (almost every intersection has few samples). COMPAS probes the dense-tail mechanism under strong regularization (small training set, moderate disparity). German probes the safety regime (almost no SP/EO violation is statistically certifiable at α=0.05\alpha=0.05 under ERM in our observed splits). ACS Coverage has larger effective subgroup sizes, while Taiwan Credit tests whether the comparison extends to a larger credit-risk task with many audited intersections but a relatively small positive-label budget. Appendix § 8 details preprocessing, protected attributes, subgroup construction, and split-level audit statistics.

Protocol and Metrics Each method and λ\lambda is evaluated over at least 10 stratified shuffle splits, with at least 3 random initialization seeds per split (xref}Appendix B gives the counts per experiment). All headline neural methods share the same MLP backbone, prediction-loss schedule, and held-out validation-only model selection. We report MSEEαMSEE_{\alpha} at α=0.05\alpha= 0.05 on hard test predictions (Eq. (5)) against held-out log-loss, with standard error (SE) bars over the runs across splits and seeds. MSEE and MSPE use unadjusted per-subgroup intervals; reported violation counts use Benjamini–Hochberg (BH)-corrected subgroup pp-values. Hyperparameter grids, BCE-guarded checkpointing, and clustered-SE checks are in Appendix § 9.

Pareto Comparison Against Baselines

We compare SAFER against ERM, EO-Penalty and two FAIRRET [6] regularizers. EO-Penalty is the certainty-blind squared-disparity regularizer (Lpred+λ∣S∣∑S(p^Sraw)2\mathcal{L}_{\mathrm{pred}} + \frac{\lambda}{|\mathcal{S}|}\sum_{S}(\hat{p}^{\mathrm{raw}}_{S})^{2}) and is the closest direct comparator to the SAFER significant-excess hinge. FAIRRET Norm 1 and FAIRRET Norm 2 are ratio-normalized differentiable fairness regularizers. EO-Penalty and both FAIRRET variants operate on raw soft probabilities rather than threshold-smoothed predictions, which makes them direct empirical tests of the inconsistency claim of Proposition 3. We also report thresholded summaries for Exponentiated Gradient [1], Prejudice Remover [24], and Adversarial Debiasing [40]. Complete per-dataset details are deferred to Appendix § 11, because these methods do not all share the matched neural in-training backbone used in the headline Pareto plots.

Table 2 summarizes the thresholded fairness–utility comparison across all five datasets and all baselines. For each dataset and method, the entry is the best mean test log loss among the swept hyperparameters whose test MSEE is at least 50% below the ERM MSEE on the same dataset. A * indicates that no swept hyperparameter achieved that fairness threshold, so the method did not produce a qualifying tradeoff under this criterion. Appendix § 11 reports the complete per-dataset tables, including the lowest-MSEE point found for each ineligible * case.

Figure 1 shows the EO Pareto frontiers for Adult, COMPAS, and German. Across panels, SAFER preserves near-ERM log-loss while crossing the threshold and reaches lower MSEE than EO-Penalty or FAIRRET without comparable log-loss collapse. Both observations are consistent with the alignment argument of Theorem 2: optimizing a threshold-smoothed surrogate that approximates the hard-decision metric with an error linear in τ\tau is more effective at reducing that metric than optimizing a raw-soft-count surrogate, especially in regions where the soft-count and hard-decision quantities can disagree by an order-one amount (Proposition 3). SAFER’s frontier matches or improves on that of a baseline that optimizes the Wald statistic directly (Appendix § 13.§ 2). The per-dataset threshold summaries, including the minimum-MSEE points for methods marked ineligible in Table 2, are reported in Appendix § 11.

(a)

COMPAS Pareto frontier

(b)

German Credit Pareto frontier

(c) Held-out log-loss versus MSEE Pareto frontiers under Equal Opportunity. The vertical axis is the mean test log-loss over runs across splits and seeds and the horizontal axis is MSEE on hard test predictions; lower is better on both. Markers are the values of the fairness weight λ\lambda along each method’s sweep, annotated and connected in increasing order, with error bars of ±1\pm1 standard error in both coordinates; the star is ERM (λ=0\lambda= 0). The methods are SAFER, EO-Penalty, FAIRRET Norm 1 and FAIRRET Norm 2. Adult Income and German Credit use the default variants in Table 1; COMPAS uses the balanced 50/50 variant. The dashed vertical line marks 0.5×0.5\times the ERM MSEE, which is the eligibility threshold used in Table 2; outlined markers indicate the selected qualifying points reported in that table.

Figure 1.

Adult IncomeCOMPASGerman CreditTaiwanACS
SAFER0.3829 ±3e−4\pm3\mathrm{e}{-4}0.6252 ±7e−4\pm7\mathrm{e}{-4}0.5349 ±1e−3\pm1\mathrm{e}{-3}0.4479 ±4e−4\pm4\mathrm{e}{-4}0.5198 ±3e−4\pm3\mathrm{e}{-4}
EO-Penalty0.4106 ±9e−4\pm9\mathrm{e}{-4}0.6339 ±5e−4\pm5\mathrm{e}{-4}0.5358 ±1e−3\pm1\mathrm{e}{-3}0.4527 ±6e−4\pm6\mathrm{e}{-4}0.5223 ±4e−4\pm4\mathrm{e}{-4}
FAIRRET Norm 10.4272 ±1e−3\pm1\mathrm{e}{-3}0.6559 ±9e−4\pm9\mathrm{e}{-4}0.5409 ±2e−3\pm2\mathrm{e}{-3}0.4668 ±1e−3\pm1\mathrm{e}{-3}0.5240 ±4e−4\pm4\mathrm{e}{-4}
FAIRRET Norm 20.4618 ±3e−3\pm3\mathrm{e}{-3}0.6366 ±7e−4\pm7\mathrm{e}{-4}0.5408 ±2e−3\pm2\mathrm{e}{-3}0.4624 ±1e−3\pm1\mathrm{e}{-3}0.5314 ±4e−4\pm4\mathrm{e}{-4}
AdvDebiasing*****
ExponGrad***0.6256 ±3.5e−2\pm3.5\mathrm{e}{-2}0.6535 ±1.3e−2\pm1.3\mathrm{e}{-2}
PrejudiceRemover***0.6216 ±1.1e−2\pm1.1\mathrm{e}{-2}*

Table 2. Thresholded fairness–utility summary across datasets and baselines. Each entry is the best mean test log loss among swept hyperparameters with MSEE at least 50% below ERM MSEE; * means no qualifying hyperparameter was found.

The corresponding SP Pareto plots are in Appendix § 15. Under SP the effective per-subgroup sample sizes nSn_S are substantially larger than EO’s mSm_S, so more subgroups are certified already at ERM and the gate contributes less. SAFER nevertheless reaches lower log-loss at matched MSPE than the raw-probability penalty on all three datasets, and the difference is largest on Adult. We attribute this to the threshold-smoothing effect of Theorem 2: with the gate close to open, the hinge approaches a smoothed-disparity penalty, and SAFER’s remaining advantage comes from targeting hard-decision disparities.

Ablation study

Section 4 predicted three mechanisms: threshold smoothing (Theorems 2, 7), the gate threshold (Proposition 4), and the dense tail (Eq. (12)). Each picks out one stress variant in which it is predicted to be empirically necessary. We test all three with single-knob ablations against the SAFER backbone (NoThreshSmooth, NoSelectivity, NoDenseTail); details are in Appendix § 14.

Threshold smoothing. On Adult and COMPAS, NoThreshSmooth is Pareto-dominated across the mid-MSEE range; at matched MSEE≈10−3MSEE \approx10^{-3} on Adult, removing threshold smoothing costs roughly 0.14 nats of log-loss. This is the empirical counterpart of Theorems 2 and 7: raw probabilities target a soft-count quantity that is biased relative to the hard-decision metric and whose gradient is not concentrated where decisions can flip.

Dense-tail term. On COMPAS, NoDenseTail plateaus at MSEE≈0.06MSEE \approx0.06 from λ≥5\lambda\ge5 while SAFER reaches MSEE≈0.018MSEE \approx0.018 at λ=100\lambda= 100 (about ten standard errors apart). On Adult, NoDenseTail similarly fails to reach the low-MSEEMSEE region of SAFER’s frontier: it remains near ERM MSEEMSEE (≈0.02\approx0.02) for almost the entire λ\lambda sweep and moves toward smaller MSEEMSEE only at the largest tested λ\lambda, where it is Pareto-dominated by SAFER (Figure 2, left). The mechanism is Eq. (12): as the optimizer drives disparities toward the gate threshold, σ(uS/τs)\sigma(u_S/\tau_s) drops below 12\frac{1}{2} and the hinge becomes a vanishing fraction of the gradient; the tail’s gradient carries no σ(uS/τs)\sigma(u_S/\tau_s) factor and lets optimization continue. On German Credit, NoDenseTail tracks SAFER, because the significance gate already keeps the hinge inactive.

Figure 2. Single-knob ablations of SAFER on the default Adult Income, COMPAS, and German Credit variants from Table 1. The panels show Adult Income (left), COMPAS (center) and German Credit (right). Axes, markers and error bars are as in Figure 1. The arms are SAFER and its three single-knob removals: NoDenseTail (wtail=0w_{\mathrm{tail}} = 0), NoSelectivity (gate threshold set to zero) and NoThreshSmooth (raw probabilities instead of threshold-smoothed counts).

Gate threshold. On German Credit, almost no EO violation is statistically certified at α=0.05\alpha= 0.05 in our sweeps; SAFER preserves ERM-level log-loss (≈0.53\approx0.53) up to λ=20\lambda= 20, while NoSelectivity climbs to log-loss ≈0.69\approx0.69 by λ=20\lambda= 20. This is the failure mode that Proposition 4 predicts when the optimizer can act on threshold-smoothed but uncertified signal.

Cross-Regime Summary. Each removal produces the failure that Section 4 predicts. Without threshold smoothing, a Pareto gap opens on Adult and COMPAS, as Theorems 2 and 7 predict. Without the dense tail, the frontier stops short of low MSEEMSEE on the same two datasets, because of the hinge suppression in Eq. (12). Without the gate threshold, the log-loss rises on German Credit, as Proposition 4 predicts. NoSelectivity has no effect on COMPAS, and on Adult it tracks SAFER up to λ=1\lambda= 1 and loses utility at larger λ\lambda (Appendix § 14.§ 4). The ablation map is in Appendix § 14.§ 5.

Conclusion

Intersectional fairness auditing encounters a resolution limit: as subgroup definitions grow finer, the statistical uncertainty of each disparity estimate comes to exceed the disparity itself. We showed that ignoring this limit, as existing in-processing methods do, leads to fairness overfitting, where the optimizer spends model capacity fitting sampling noise rather than genuine disparities.

To address this, we introduced the significant-excess metric family, which continuously measures only the portion of each subgroup’s disparity that exceeds what sampling variation can explain at a given confidence level, and SAFER, a differentiable regularizer that gates fairness penalties on a soft-count likelihood-ratio evidence statistic. SAFER satisfies a provable safety property: when every subgroup disparity lies well below the gate threshold, the ERM optimum is nearly stationary for SAFER. At that optimum, the gated fairness contribution is exponentially small, and the contribution of the small ungated term is proportional to its weight.

Experiments across several benchmarks and purpose-built stress variants support a regime-conditional claim: SAFER achieves a better Pareto frontier than the baselines where certified disparities exist and preserves the utility of the unconstrained model where they do not, and removing any component of SAFER produces the failure the theory predicts on at least one stress variant.

Limitations and future work. A zero significant excess means that no excess disparity is certifiable at the chosen confidence level and sample size; a real disparity in a small subgroup can remain uncertified, and SAFER then withholds the gradient that would reduce it (Appendix § 12). The current evaluation is limited to tabular benchmarks with at most a few hundred intersectional subgroups. Scaling to high-cardinality attribute spaces and to settings where subgroup membership is uncertain or partially observed remains open. A natural extension is to include multiplicity corrections in the training-time significance test, so that SAFER’s gate also controls the family-wise error rate.

Acknowledgments and Disclosure of Funding

This work is conducted as part of the Horizon Europe project PRE-ACT (Prediction of Radiotherapy side effects using explainable AI for patient communication and treatment modification). It is supported by the European Commission through the Horizon Europe Program (Grant Agreement number 101057746), by the Swiss State Secretariat for Education, Research and Innovation (SERI) under contract number 22 00058, and by the UK government (Innovate UK) under application number 10061955. The authors are employed by Intesa Sanpaolo. The authors declare no other competing interests.

Datasets and stress variants

All experiments use the same preprocessing code as the released scripts. Protected attributes are removed from the feature matrix and kept only for subgroup construction. Subgroup masks include all conjunctions up to the specified intersection level, with groups of size less than two omitted in the corresponding split.

Adult Income. We use the OpenML version of the UCI Adult Income dataset [26]. The label is income > $50K. The protected attributes are race, sex, age binned into {<25,25–45,45–65,>65}\{< 25, 25\text{--}45, 45\text{--}65, > 65\}, and native country binarized as US/non-US. Non-protected numeric and categorical features are retained, with categorical features one-hot encoded. The stress variant in the main paper uses intersection level 4 and a 10/90 train/test split. After dropping rows with missing values, the processed data contain 45,222 examples, 55 non-protected features, and a positive-label rate of 24.8%. Over 10 stratified splits used for a dataset audit, this dataset variant has 4,522 training examples, 227.7±4.0227.7 \pm4.0 training-set subgroup masks and 256.2±2.6256.2 \pm2.6 test-set subgroup masks. This is a budget of only 19.9±0.419.9 \pm0.4 training examples per audited subgroup, and only 4.9±0.14.9 \pm0.1 positive-label examples per subgroup for EO. For the ERM model fit, the test-set EO audit has 220.7±2.6220.7 \pm2.6 EO-defined subgroups and certifies 61.4±8.861.4 \pm8.8 significant EO violations at α=0.05\alpha= 0.05 after BH correction.

COMPAS. We use the OpenML COMPAS recidivism dataset [2]. The label is two-year recidivism. The protected attributes are race (African-American/Caucasian in the OpenML version), sex, and age category (<25< 25, 25–4525\text{--}45, >45> 45). The processed data contain 4,966 examples, 5 non-protected features, and a positive-label rate of 50.0%. The headline Pareto comparison uses intersection level 3 and a 50/50 train/test split. Over ten stratified splits this gives 2,483 training examples and exactly 35 audited subgroup masks in both train and test, a budget of 70.9 training examples and 35.5 positive-label examples per audited subgroup. We run an ERM test-set EO audit with 10 splits that certifies 27.3±2.027.3 \pm2.0 significant EO violations at α=0.05\alpha= 0.05 after BH correction. The mechanism ablation uses the more severe 20/80 split: 993 training examples, the same 35 subgroup masks, and only 28.4 training examples and 14.2 positives per audited subgroup. Under this stress split, ERM certifies 29.3±2.229.3 \pm2.2 significant EO violations, making it the high-regularization stress test for the dense-tail component.

German Credit. We use the OpenML version of the Statlog German Credit dataset [21]. The label is good credit. The protected attributes are sex derived from personal_status, age binned into {<30,30–50,>50}\{< 30, 30\text{--}50, > 50\}, and foreign-worker status. The processed data contain 1,000 examples, 53 non-protected features, and a positive-label rate of 70.0%. The stress variant uses intersection level 3 and a 50/50 train/test split. Over ten stratified splits this gives 500 training examples, 32.1 ±\pm 1.4 training-set subgroup masks, and 31.1 ±\pm 0.3 test-set subgroup masks, a budget of 15.6 ±\pm 0.7 training examples and 10.9 ±\pm 0.5 positive-label examples per audited subgroup. For the ERM model fit on these same splits, the test-set EO audit has 31.1 ±\pm 0.3 EO-defined subgroups and certifies only 0.3 ±\pm 0.9 significant EO violations at α=0.05\alpha= 0.05 after BH correction, with MSEE\mathrm{MSE}_{E} 0.0007 ±\pm 0.0018. This is therefore a safety-regime test rather than a fairness-improvement test.

ACS Coverage. We use the ACS Public Coverage task from the Folktables benchmark [10], which predicts public health insurance coverage from American Community Survey records. The protected attributes are race, sex, and age. The reported variant uses intersection level 3 and a 50/50 train/test split. The data contains 50,000 examples, so each split has 25,000 training examples. Across the dataset audit there are exactly 71 training-set and test-set subgroup masks, all EO-defined. This corresponds to a budget of 352.1 training examples and 103.5 positive-label examples per audited subgroup (≈\approx 29.4% positives). Because the subgroup budget is much larger than in Adult, COMPAS, or German, this variant serves as a higher-sample ACS coverage benchmark rather than a sparse-cell stress test.

Taiwan Credit. We use the UCI Default of Credit Card Clients dataset [39], which predicts credit card default. The protected attributes are sex, education, marriage, and age. The headline variant uses intersection level 4 and a 50/50 train/test split. The processed data contain 30,000 examples, so each split has 15,000 training examples. Over ten stratified splits this gives 285.4 ±\pm 2.8 training-set subgroup masks and 285.9 ±\pm 2.3 test-set subgroup masks. This corresponds to a budget of 52.6 ±\pm 0.5 training examples and 11.6 ±\pm 0.1 positive-label examples per audited subgroup; 245.8 subgroups are EO-defined on average. This variant therefore provides a high-intersectionality credit-risk benchmark with a larger total training set than Adult but still a relatively small positive-label budget per audited subgroup.

Training details and hyperparameters

Model and optimizer. The headline neural methods use the same MLP classifier: hidden layers (64, 32), ReLU activations, dropout 0.2, Adam optimizer (lr = 10−310^{-3}, weight decay 5×10−45 \times10^{-4}), batch size 256, gradient clipping at norm 1, and at most 200 epochs. Prediction loss is binary cross-entropy. Fairness statistics are computed from full-training-set soft counts each epoch rather than mini-batches, because subgroup-level LRT and significant-excess quantities are unstable on mini-batch counts.

Splits. The main Adult L4 10/90 and COMPAS L3 50/50 Pareto comparisons use 100 stratified shuffle splits and 3 random seeds per split. The mechanism ablation uses 10 splits ×\times 3 seeds on Adult L4 10/90, 100 splits ×\times 3 seeds on COMPAS L3 20/80, and 30 splits ×\times 3 seeds on German L3 50/50. Within the training portion of every split, 10% is held out for model selection. The held-out test partition is never used for hyperparameter or checkpoint selection.

Checkpointing. SAFER and its single-knob ablations use the joint BCE-guarded training configuration. The selector first restricts attention to checkpoints whose validation BCE is within a relative tolerance of the best validation BCE, and then selects the checkpoint with the smallest guarded score, using the full-training-set fairness loss for the fairness term. This prevents high λ\lambda runs from selecting collapsed models while still allowing fairness improvements when their validation-BCE cost is small. ERM selects by validation BCE alone. The headline EO-Penalty and FAIRRET runs use their standard joint-training selectors unless otherwise stated.

SAFER hyperparameters. The main SAFER configuration is the unweighted subgroup average (use_size_weight=false) with α=0.05\alpha= 0.05, threshold smoothing temperature τ=0.05\tau= 0.05, softplus temperature τs=0.01\tau_{\mathrm{s}} = 0.01, absolute-value smoothing ϵ=10−3\epsilon= 10^{-3}, detached LRT-based SE^eff\widehat{SE}_{\mathrm{eff}}, and LRT dense-tail weight wtail=0.003w_{\mathrm{tail}} = 0.003. The NoThresholdSmooth ablation sets τ\tau to null; NoSelectivity sets the hinge threshold to zero; NoDenseTail sets wtail=0w_{\mathrm{tail}} = 0.

Baselines. ERM is the unconstrained MLP. EO-Penalty is BCE plus an unweighted squared EO-disparity penalty. FAIRRET Norm 1 and FAIRRET Norm 2 use the ratio-normalized differentiable fairness regularizers of FAIRRET [6], with the EO variant conditioning on positive labels. We also swept ExponGrad [1] with TPR-parity constraints, Prejudice Remover, and adversarial debiasing. where compatible. These additional baselines are included in the thresholded summary, but they do not all share the same neural in-training backbone as the headline Pareto comparisons.

ExperimentMethodGrid
Adult L4 10/90 mainSAFER0.05, 0.1, 0.2, 0.3, 0.5, 1, 2, 5, 10, 20, 30, 50, 80, 100
Adult L4 10/90 mainEO-Penalty10, 20, 50, 100, 200, 500, 1000, 2000, 4000
Adult L4 10/90 mainFAIRRET Norm 20.01, 0.1, 0.5, 1, 2, 5, 10, 20, 50, 100, 500, 1000
Adult L4 10/90 ablationSAFER / NoThresh / NoTail5, 10, 20, 50, 100, 200, 500, 1000
Adult L4 10/90 ablationNoSelectivity0.5, 1, 2, 5, 10, 20, 50, 100
COMPAS L3 50/50 mainSAFER0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10, 100
COMPAS L3 50/50 mainEO-Penalty0.1, 0.5, 1, 2, 5, 10, 20, 30, 50, 80, 100, 200, 500, 1000
COMPAS L3 50/50 mainFAIRRET Norm 10.001, 0.01, 0.02, 0.03, 0.04, 0.05, 0.1, 1, 10
COMPAS L3 50/50 mainFAIRRET Norm 20.005, 0.01, 0.02, 0.03, 0.06, 0.1, 0.15, 0.2, 0.3, 0.5, 1, 10, 100
COMPAS L3 20/80 ablationSAFER / NoThresh / NoTail0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10, 20, 100
COMPAS L3 20/80 ablationNoSelectivity0.05, 0.1, 0.2, 0.3, 0.4, 0.5, 1, 2, 5, 10, 100
German L3 50/50 ablationSAFER / NoThresh / NoTail1, 2, 5, 10, 20, 50, 100
German L3 50/50 ablationNoSelectivity0.01, 1, 2, 3, 4, 5, 10, 20, 50, 100

Table A.1. Regularization grids used in the Results section. Main Pareto comparisons use 100 splits × 3 seeds; Adult ablations use 10 splits × 3 seeds.

ExperimentMethodλ\lambda jobsSplit × seedruns per jobEstimated wall time
Adult L4 10/90 mainSAFER14100 × 33–6 h
Adult L4 10/90 mainEO-Penalty9100 × 31–1.5 h
Adult L4 10/90 mainFAIRRET Norm 1 / Norm 212–13100 × 320–45 min
COMPAS L3 50/50 mainSAFER9100 × 320–40 min
COMPAS L3 50/50 mainEO-Penalty14100 × 315–30 min
COMPAS L3 50/50 mainFAIRRET Norm 1 / Norm 29–13100 × 35–10 min
German L3 50/50 mainSAFER7100 × 320–30 min
German L3 50/50 mainEO-Penalty14100 × 310–20 min
German L3 50/50 mainFAIRRET Norm 1 / Norm 29–13100 × 3under 10 min
Adult L4 10/90 ablationSAFER-family arm830 × 335–55 min per arm
COMPAS L3 20/80 ablationSAFER-family arm10–11100 × 320–40 min per arm
German L3 50/50 ablationSAFER-family arm7–1030 × 35–15 min per arm

Table A.2. Estimated CPU wall-clock time for the full sweeps on the 10-core, 32 GB Apple M1 Max machine with eight joblib workers. SAFER-ablation variants (NOTHRESHSMOOTH, NOSELECTIVITY, and NODENSETAIL) have the same resource profile as SAFER up to small constant-factor changes.

Uncertainty reporting. Figures report means and ±1 standard error over all combinations of split and seed. Because three seeds share the same data split, these observations are not fully independent. We therefore treat the plotted SE as descriptive and verified that the method ordering is unchanged when seeds are averaged within split before computing SE.

Computational resources

All reported neural sweeps were run on CPU workers. The machine used for the final local runs was an Apple MacBook Pro with an Apple M1 Max processor (10 CPU cores: 8 performance and 2 efficiency cores) and 32 GB of unified memory, running macOS/Darwin on arm64. GPU execution was disabled in the experiment scripts. Independent λ\lambda jobs were parallelized across up to 8 CPU worker processes. Each worker trains one combination of method and λ\lambda over its assigned sequence of splits and seeds. Peak memory was dominated by the Python/PyTorch worker processes and the dataset matrices; on this 32 GB machine the reported CPU sweeps ran without memory pressure when running all eight processes.

Table A.2 gives wall-clock estimates for reproducing the full neural sweeps. These estimates exclude plotting and table aggregation, which take seconds to minutes, and should be interpreted as order-of-magnitude wall-clock requirements for this CPU workstation rather than hardware-independent constants.

Per-dataset thresholded baseline summaries

The following tables expand Table 2 with per-dataset details. For eligible rows, we report the best mean test log loss among swept hyperparameters whose MSEE is at least 50% below the ERM MSEE on that dataset. For ineligible rows, corresponding to * entries in the main summary table, we report the minimum-MSEE point found by the sweep.

MethodParameterEligibleLog lossAUCMSEE
SAFERλ=30\lambda= 30yes0.3829 ±0.0003\pm0.00030.8818 ±0.0003\pm0.00030.0097 ±0.0004\pm0.0004
EO-Penaltyλ=200\lambda= 200yes0.4106 ±0.0009\pm0.00090.8624 ±0.0006\pm0.00060.0093 ±0.0006\pm0.0006
FAIRRET Norm 1λ=0.02\lambda= 0.02yes0.4272 ±0.0012\pm0.00120.8561 ±0.0007\pm0.00070.0097 ±0.0006\pm0.0006
FAIRRET Norm 2λ=10\lambda= 10yes0.4618 ±0.0030\pm0.00300.8456 ±0.0022\pm0.00220.0085 ±0.0005\pm0.0005
AdvDebiasingαadv=0.5\alpha_{\mathrm{adv}} = 0.5no0.4770 ±0.0066\pm0.00660.8208 ±0.0034\pm0.00340.0157 ±0.0009\pm0.0009
ExponGradϵEG=0.05\epsilon_{\mathrm{EG}} = 0.05no0.5936 ±0.0238\pm0.02380.7373 ±0.0066\pm0.00660.0335 ±0.0018\pm0.0018
PrejudiceRemoverλ=5\lambda= 5no1.1022 ±0.1035\pm0.10350.4367 ±0.0835\pm0.08350.0165 ±0.0038\pm0.0038

Table A.3. Adult Income: Best mean test log loss with MSEE at least 50% below ERM MSEE. (ERM MSEE=0.0201, threshold=0.0101; ineligible rows show the minimum-MSEE point).

MethodParameterEligibleLog lossAUCMSEE
SAFERλ=0.5\lambda= 0.5yes0.6252 ±0.0007\pm0.00070.7148 ±0.0009\pm0.00090.0510 ±0.0013\pm0.0013
EO-Penaltyλ=10\lambda= 10yes0.6339 ±0.0005\pm0.00050.7171 ±0.0009\pm0.00090.0661 ±0.0015\pm0.0015
FAIRRET Norm 1λ=0.03\lambda= 0.03yes0.6559 ±0.0009\pm0.00090.6975 ±0.0018\pm0.00180.0614 ±0.0021\pm0.0021
FAIRRET Norm 2λ=0.1\lambda= 0.1yes0.6366 ±0.0007\pm0.00070.7117 ±0.0010\pm0.00100.0628 ±0.0015\pm0.0015
AdvDebiasingαadv=0.5\alpha_{\mathrm{adv}} = 0.5no0.7381 ±0.0449\pm0.04490.6522 ±0.0170\pm0.01700.1029 ±0.0095\pm0.0095
ExponGradϵEG=0.05\epsilon_{\mathrm{EG}} = 0.05no0.7211 ±0.0061\pm0.00610.5291 ±0.0096\pm0.00960.1259 ±0.0045\pm0.0045
PrejudiceRemoverλ=0.1\lambda= 0.1no0.6595 ±0.0037\pm0.00370.7066 ±0.0023\pm0.00230.0939 ±0.0019\pm0.0019

Table A.4. COMPAS: Best mean test log loss with MSEE at least 50% below ERM (ERM MSEE=0.1292, threshold=0.0646; ineligible rows show the minimum-MSEE point).

MethodParameterEligibleLog lossAUCMSEE
SAFERλ=30\lambda= 30yes0.5349 ±0.0014\pm0.00140.7659 ±0.0017\pm0.00170.0006 ±0.0001\pm0.0001
EO-Penaltyλ=200\lambda= 200yes0.5358 ±0.0013\pm0.00130.7554 ±0.0017\pm0.00170.0006 ±0.0001\pm0.0001
FAIRRET Norm 1λ=0.12\lambda= 0.12yes0.5409 ±0.0016\pm0.00160.7536 ±0.0019\pm0.00190.0006 ±0.0001\pm0.0001
FAIRRET Norm 2λ=0.4\lambda= 0.4yes0.5408 ±0.0016\pm0.00160.7544 ±0.0018\pm0.00180.0006 ±0.0001\pm0.0001
AdvDebiasingαadv=0.1\alpha_{\mathrm{adv}} = 0.1no1.3272 ±0.0141\pm0.01410.7247 ±0.0024\pm0.00240.0027 ±0.0003\pm0.0003
ExponGradϵEG=0.1\epsilon_{\mathrm{EG}} = 0.1no0.5664 ±0.0033\pm0.00330.7371 ±0.0027\pm0.00270.0018 ±0.0002\pm0.0002
PrejudiceRemoverλ=5\lambda= 5no0.8915 ±0.0110\pm0.01100.6380 ±0.0098\pm0.00980.0104 ±0.0009\pm0.0009

Table A.5. German Credit: Best mean test log loss with MSEE at least 50% below ERM (ERM MSEE=0.0013, threshold=0.0007; ineligible rows show the minimum-MSEE point).

MethodParameterEligibleLog lossAUCMSEE
SAFERλ=2\lambda= 2yes0.4479 ±0.0004\pm0.00040.7552 ±0.0008\pm0.00080.0021 ±0.0002\pm0.0002
EO-Penaltyλ=5\lambda= 5yes0.4527 ±0.0006\pm0.00060.7531 ±0.0010\pm0.00100.0025 ±0.0002\pm0.0002
FAIRRET Norm 1λ=0.002\lambda= 0.002yes0.4668 ±0.0010\pm0.00100.7442 ±0.0014\pm0.00140.0014 ±0.0002\pm0.0002
FAIRRET Norm 2λ=0.05\lambda= 0.05yes0.4624 ±0.0012\pm0.00120.7455 ±0.0015\pm0.00150.0021 ±0.0002\pm0.0002
AdvDebiasingαadv=5\alpha_{\mathrm{adv}} = 5no0.4569 ±0.0009\pm0.00090.7546 ±0.0013\pm0.00130.0032 ±0.0002\pm0.0002
ExponGradϵEG=0.1\epsilon_{\mathrm{EG}} = 0.1yes0.6256 ±0.0347\pm0.03470.6096 ±0.0101\pm0.01010.0027 ±0.0006\pm0.0006
PrejudiceRemoverλ=0.01\lambda= 0.01yes0.6216 ±0.0107\pm0.01070.6093 ±0.0103\pm0.01030.0017 ±0.0002\pm0.0002

Table A.6. Taiwan: Best mean test log loss with MSEE at least 50% below ERM (ERM MSEE=0.0060, threshold=0.0030; ineligible rows show the minimum-MSEE point).

Subgroups below the certification boundary

A zero significant excess for a subgroup states only that the available sample cannot certify an excess at the chosen confidence level. The level α\alpha sets the trade between reacting to sampling noise and leaving low-power disparities uncorrected; a larger α\alpha protects smaller groups at a utility cost. The size sweep of Appendix § 13.§ 3 makes the dependence concrete: the same planted gap of −0.15-0.15 is certified in about 9% of runs at mS=5m_S = 5 and in about 95% at mS=120m_S = 120. We therefore recommend reporting MSEE together with raw subgroup disparities, confidence intervals, subgroup counts and a power or minimum-detectable-effect analysis, and treating an uncertified subgroup as unresolved; possible responses are collecting more data, pooling groups coarsely or hierarchically, or policy constraints that do not rely on statistical significance alone. Where harm is suspected in a subgroup too small to certify, the appropriate evidence is of a different kind: the structure of the decision procedure, the features it uses, the causal path from the protected attribute to the outcome, and documented practice. A complete audit combines statistical outcome testing with this process evidence, and the certification boundary marks where the hand-over occurs. Medical-imaging benchmarks whose small subgroups carry genuine disparities, such as Harvard-FairVision30k [31], are a natural next test of SAFER outside the tabular setting.

MethodParameterEligibleLog lossAUCMSEE
SAFERλ=0.3\lambda= 0.3yes0.5198 ± 0.00030.7472 ± 0.00060.0236 ± 0.0009
EO-Penaltyλ=10\lambda= 10yes0.5223 ± 0.00040.7427 ± 0.00070.0383 ± 0.0009
fairret-norm1λ=0.005\lambda= 0.005yes0.5240 ± 0.00040.7401 ± 0.00060.0357 ± 0.0011
fairret-norm2λ=0.05\lambda= 0.05yes0.5314 ± 0.00040.7320 ± 0.00070.0218 ± 0.0009
AdvDebiasingαadv=0.5\alpha_{\mathrm{adv}} = 0.5no0.5299 ± 0.01110.7450 ± 0.00300.0605 ± 0.0025
ExponGradϵEG=0.005\epsilon_{\mathrm{EG}} = 0.005yes0.6535 ± 0.01340.5623 ± 0.00780.0257 ± 0.0029
PrejudiceRemoverλ=10\lambda= 10no0.6196 ± 0.00190.6401 ± 0.00180.0464 ± 0.0013

Table A.7. ACS: Best mean test log loss with MSEE at least 50% below ERM (ERM MSEE=0.0904, threshold=0.0452; ineligible rows show the minimum-MSEE point).

Additional experiments

Decision-threshold sensitivity

The threshold tt does not enter the prediction loss, which trains probabilistic scores; it defines the deployed classifier and therefore both its accuracy and its hard-decision fairness audit. We use t=0.5t = 0.5 because, when scores estimate class probabilities and false positives and false negatives cost the same, it is the Bayes-optimal decision threshold. Since tt enters SAFER’s objective, it is fixed before training: by a deployment requirement such as a cost matrix or a regulatory decision rule, or, when no policy value exists, on validation data with a preliminary ERM model, and then held fixed while all methods are trained and compared. To test the dependence on this default we retrained SAFER for every t∈{0.1,…,0.9}t \in\{0.1,\ldots,0.9\} on the three datasets of the main Pareto comparison, five seeds per threshold, and audited each model at its own threshold. Table A.8 reports the test log-loss, the paired difference ΔMSEE=MSEESAFER−MSEEERM\Delta\mathrm{MSEE} = \mathrm{MSEE}_{\mathrm{SAFER}} - \mathrm{MSEE}_{\mathrm{ERM}} against the ERM audit at the same threshold (negative values are a fairness improvement), and the accuracy. Log-loss is computed from probabilistic scores, so for a fixed model it does not depend on tt; its variation across columns measures how tt changes the retrained model through the fairness term. Log-loss is stable across tt on Adult and German and more sensitive on COMPAS. The fairness benefit persists across operationally meaningful thresholds: Adult has non-positive ΔMSEE\Delta\mathrm{MSEE} throughout, COMPAS improves for tt between 0.2 and 0.9, and the changes on German are negligible because little certified disparity is available to reduce. Extreme thresholds produce nearly constant classifiers with trivially low MSEE (accuracy 0.500 on COMPAS for t≤0.3t \le0.3, 0.700 on German for t≤0.2t \le0.2), so those columns are not useful operating points. This is a sensitivity analysis; tt is never tuned on the test set.

A baseline that optimizes the Wald statistic

Wald-Stat penalizes the standardized Wald statistic directly:

LWald-Stat=Lpred+λ∣G∣∑S∈G[softplus⁡τS(∣ZS∣−zα)2+wtail∣ZS∣],ZS=ϕ^STSE^Sϕ(17)\mathcal{L}_{\mathrm{Wald\text{-}Stat}} = \mathcal{L}_{\mathrm{pred}} + \frac{\lambda}{|\mathcal{G}|} \sum_{S \in\mathcal{G}} \left[\operatorname{softplus}_{\tau_S}\left(\lvert Z_S\rvert- z_{\alpha}\right)^2 + w_{\mathrm{tail}}\lvert Z_S\rvert\right], \qquad Z_S = \frac{\widehat{\phi}_S^T}{\widehat{SE}_S^{\phi}} \tag*{(17)}

where ϕ^ST\widehat{\phi}_S^T is the disparity computed from the same threshold-smoothed counts as SAFER and SE^Sϕ\widehat{SE}_S^{\phi} its two-proportion plug-in standard error (§ 4.2). Both stay in the computation graph, so gradients flow through the statistic, including the route that lowers the penalty by changing the estimated standard error. Wald-Stat uses the same critical value, threshold smoothing, temperatures, dense-tail weight, architecture, optimizer and checkpointing protocol as SAFER; it tests whether optimizing Wald significance directly yields a stronger fairness–utility frontier than SAFER’s LRT-based construction. Because the two objectives live on different scales, we sweep λ\lambda independently for each and match operating points by mean test log-loss after examining both sweeps; Table A.9 summarizes the matched pairs. Entries are SAFER/Wald-Stat means ± standard deviations over independent runs over splits and seeds, 10 for Adult and COMPAS and 5 for German, rounded to three decimals; several differences are small relative to this variability, so we read the frontier rather than individual rows. For the following table, the ERM reference values in the caption come from these same runs, a lighter protocol than the 100 splits with three seeds of the main sweeps, so they differ from the ERM values of the main tables.

tt0.10.20.30.40.50.60.70.80.9
Adultlog-loss.336 ± .002.337 ± .002.337 ± .003.336 ± .002.336 ± .002.336 ± .002.337 ± .002.338 ± .003.337 ± .003
Δ\DeltaMSEE−.008 ± .009−.013 ± .005−.001 ± .009−.007 ± .006−.009 ± .006−.008 ± .003−.014 ± .004−.010 ± .005−.008 ± .005
accuracy.696 ± .004.778 ± .003.822 ± .003.839 ± .002.843 ± .002.837 ± .002.817 ± .002.785 ± .004.762 ± .002
COMPASlog-loss.615 ± .008.615 ± .007.623 ± .006.633 ± .005.640 ± .008.643 ± .007.628 ± .005.620 ± .005.615 ± .005
Δ\DeltaMSEE.000 ± .000−.005 ± .004−.089 ± .019−.131 ± .021−.084 ± .018−.069 ± .018−.078 ± .012−.051 ± .015−.011 ± .005
accuracy.500 ± .000.500 ± .000.500 ± .000.535 ± .040.631 ± .016.531 ± .024.510 ± .006.505 ± .003.502 ± .001
Germanlog-loss.531 ± .035.531 ± .035.532 ± .035.532 ± .035.531 ± .035.531 ± .035.531 ± .035.532 ± .035.531 ± .035
Δ\DeltaMSEE.0000 ± .0000.0000 ± .0000.0000 ± .0000−.0005 ± .0013+.0002 ± .0014−.0005 ± .0007−.0014 ± .0018−.0041 ± .0078−.0006 ± .0027
accuracy.700 ± .000.700 ± .000.702 ± .004.722 ± .018.731 ± .019.737 ± .014.673 ± .109.590 ± .162.478 ± .103

Table A.8. SAFER retrained and audited at decision thresholds tt from 0.1 to 0.9: test log-loss, paired Δ\DeltaMSEE against ERM at the same threshold, and accuracy (mean ± standard deviation over five seeds).

(λS,λW)(\lambda_S, \lambda_W)log-loss S/WMSEE S/W
Adult(1, 0.0001).337 ± .003/.338 ± .004.028 ± .013/.040 ± .013
(5, 0.015).344 ± .004/.344 ± .005.028 ± .015/.035 ± .013
(15, 0.5).372 ± .005/.372 ± .005.025 ± .011/.025 ± .014
(30, 2).387 ± .003/.391 ± .003.005 ± .010/.002 ± .007
(50, 3).397 ± .003/.397 ± .004.0001 ± .0002/.0003 ± .0009
COMPAS(0.1, 0.001).615 ± .009/.614 ± .009.086 ± .026/.087 ± .025
(0.5, 0.005).625 ± .010/.621 ± .008.058 ± .013/.067 ± .017
(1, 0.015).647 ± .012/.648 ± .007.041 ± .015/.052 ± .019
(3, 0.05).671 ± .008/.669 ± .006.036 ± .026/.046 ± .030
(20, 1).690 ± .003/.690 ± .003.040 ± .048/.038 ± .050
German(0.25, 0.0005).531 ± .035/.531 ± .035.006 ± .005/.006 ± .005
(1, 0.01).532 ± .035/.531 ± .035.007 ± .005/.008 ± .006
(15, 0.02).532 ± .032/.532 ± .035.006 ± .006/.007 ± .006
(20, 0.1).533 ± .032/.533 ± .034.004 ± .004/.005 ± .005
(40, 3).540 ± .033/.541 ± .030.0005 ± .0007/.0015 ± .0021

Table A.9. SAFER versus Wald-Stat at matched test log-loss. Each row pairs a SAFER and a Wald-Stat fairness weight (λS,λW)(\lambda_S, \lambda_W) with similar mean test log-loss; entries are SAFER/Wald-Stat, mean ± standard deviation over runs. ERM log-loss/MSEE: Adult 0.334/0.041, COMPAS 0.613 ± 0.009/0.127 ± 0.027, German 0.531 ± 0.035/0.006 ± 0.005.

Diagnostics on Adult, stratified by the positive-label count mSm_S, explain the gap. Across the five matched points, the median of ∣ZS∣/AS|Z_S|/\sqrt{A_S} is 1.00 for mS≥30m_S \ge30. For mS<10m_S < 10, saturation drives the plug-in standard error toward zero and the ratio’s 90th percentile is 37 to 43, a roughly fortyfold inflation in the most extreme cases. Among about 1,491 pairs of subgroup and seed per point, Wald-Stat flags 41 to 57 disparities that the LRT does not, and never the reverse. These saturated groups are 2.7% of the total but absorb 99% of Wald-Stat’s penalty and 86% of its final-epoch gradient over the smallest subgroups (mS<30m_S < 30), against 43% for SAFER. At the near-ERM Adult point, the residual certified excess of the groups with mS>300m_S > 300 is 0.0029 for SAFER and 0.0214 for Wald-Stat, so SAFER preserves capacity for the better-supported groups. SAFER also has lower mean MSEE near ERM on Adult and at the intermediate COMPAS and German points, with gaps that are modest relative to the variability across splits and seeds; under strong regularization both methods converge near zero MSEE at a greater utility cost.

A controlled benchmark for the selective-gating regime

The operating regimes of § 4.3 suggest a hypothesis for the gate threshold: when both active and inactive subgroups are present, the gate gives SAFER its largest advantage over NoSelectivity, because it withholds penalties from unsupported disparities while keeping those with support. The benchmark below tests it by separating statistical evidence from disparity magnitude. It has one large reference cell and 34 probe cells. The generator is calibrated so that its population decision rule has a true-positive rate of 0.85 on the reference cell and 0.70 on every unfair probe, a planted reference gap of −0.15-0.15; null probes have zero gap. Only the positive-label count mSm_S of the unfair probes varies, and the reference count is fixed and large. Group identity is hidden from the predictor and used only by the fairness loss and the audit, which keep the SS-versus-Sˉ\bar{S} comparison of the main text. We summarize each evidence stratum by the certification statistic ΛS\sqrt{\Lambda_S} computed on the hard predictions of the final ERM model on the training split, the same statistic as in the size sweep below (Table A.10). The null and strong-evidence stratum means lie on opposite sides of the gate in every one of the 10 runs, so the benchmark is in the selective-gating regime; cell by cell the picture is noisier, with at least one null cell certified in 6 runs and one strong-evidence cell uncertified in 1 run.

probe stratumcellsplanted gapmSm_Smean ΛS\sqrt{\Lambda_S}certified fraction
null100105.90±.16.90 \pm.16.07±.07.07 \pm.07
weak evidence12−0.15-0.1561.11±.251.11 \pm.25.11±.07.11 \pm.07
boundary8−0.15-0.15341.77±.291.77 \pm.29.45±.13.45 \pm.13
strong evidence4−0.15-0.152044.72±.504.72 \pm.50.98±.08.98 \pm.08

Table A.10. Evidence strata of the probe cells for the final ERM model: cells per stratum, planted gap, mean positive-label count mSm_S in the training split, mean certification statistic ΛS\sqrt{\Lambda_S} on hard training predictions, and fraction of cells certified (ΛS>1.96\sqrt{\Lambda_S} > 1.96); per-run stratum means, reported as mean ±\pm standard deviation over 10 runs.

Generator and protocol. Every cell draws labels y∼Bernoulli⁡(0.5)y \sim\operatorname{Bernoulli}(0.5). One signal feature is distributed as N(Φ−1(TPR⁡cell),1)\mathcal{N}(\Phi^{-1}(\operatorname{TPR}_{\mathrm{cell}}),1) for positives and N(−Φ−1(0.85),1)\mathcal{N}(-\Phi^{-1}(0.85),1) for negatives, where TPR⁡cell\operatorname{TPR}_{\mathrm{cell}} is the planted true-positive rate of the cell and Φ\Phi is the standard normal distribution function; eight pure-noise N(0,1)\mathcal{N}(0,1) features are appended. The reference cell has 45,000 examples and planted rate 0.85, which pins the decision threshold of a group-blind classifier so that each cell recovers its planted rate and the complement of every probe has a true-positive rate close to 0.85. A probe cell with design size mm has 4m4m examples, so about 0.9m0.9m positives fall in the training split; the design sizes are 10, 10, 20, 20, 40, 40, 120, 120, 400, 400 for the ten null probes (planted rate 0.85), 5, 5, 6, 6, 7, 7, 8, 8, 5, 6, 7, 8 for the twelve weak-evidence probes, 20, 30, 35, 35, 40, 40, 45, 45 for the eight boundary probes and 150, 150, 300, 300 for the four strong-evidence probes (planted rate 0.70). Each of the 10 runs draws a fresh dataset of 54,832 examples, a fresh split and a fresh initialization; with a run all arms share the initialization and the dropout stream, so comparisons are paired. Half of the data is held out as test; of the rest, 90% trains the model and 10% is set aside for validation and not used. The model is an MLP with hidden layers (64, 32) and dropout 0.2, trained full-batch for 120 epochs with Adam (learning rate 10−310^{-3}, weight decay 5×10−45 \times10^{-4}); the final model is evaluated, without early stopping or checkpoint selection, so that λ\lambda traces the whole fairness–utility frontier. SAFER uses the values of the main experiments (α=0.05\alpha= 0.05, τ=0.05\tau= 0.05, τS=0.01\tau_S = 0.01, ε=10−3\varepsilon= 10^{-3}, wtail=0.003w_{\mathrm{tail}} = 0.003); NoSelectivity removes only the gate and EO-Penalty adds λ\lambda times the squared EO gap to the prediction loss. The λ\lambda grid is chosen separately for each arm, from near-ERM to a near-constant predictor. Every cell is scored on its training and its test members with hard predictions at 0.5; a cell is skipped on a split where it or its complement has fewer than two positives, which affects only the weak-evidence stratum (10 to 12 of its 12 cells scored per run). Stratum values are cell averages, reported as mean ±\pm standard deviation over the 10 runs. The generator, the harness and the per-run results are released with the code.

We compare SAFER with NoSelectivity (NS), which removes only the gate, and with EO-Penalty (EO), over independent λ\lambda sweeps. Table A.12 reports the complete sweeps: for every arm and λ\lambda, the test log-loss and the mean residual certified excess in each evidence stratum. Points whose mean test log-loss reaches 0.5 approach the log-loss 0.693 of a constant predictor, and their near-zero excess comes from degenerate predictions. From the remaining part of the frontiers we chose three approximately matched mean test-log-loss levels, reported in Table A.11 with the fairness weight and test log-loss of each arm and the mean residual certified excess in each evidence stratum, in percentage points, as means ±\pm standard deviations over 10 runs. The methods overlap at the near-ERM level A. At B and C, SAFER has the lowest mean excess in the boundary and strong-evidence strata; in original units, SAFER’s C values in these two strata are 0.0079 and 0.0227, against 0.0184 and 0.0465 for NS and 0.0108 and 0.0429 for EO. SAFER’s null-stratum mean at C is higher and the standard deviations overlap, so the evidence is directional, with no statistically significant pairwise differences at matched utility. The full sweeps show the same pattern: EO’s strong-stratum excess stays near 4 percentage points from λ=50\lambda= 50 to λ=600\lambda= 600 while its log-loss rises from 0.47 to 0.63, and NS becomes unstable above λ≈1\lambda\approx1, with large run-to-run variation of the log-loss from λ=1.25\lambda= 1.25 on. The strong-evidence gain is an indirect effect of the whole objective: the gate is already active for those probes, and withholding unsupported gradients preserves optimization capacity for well-supported ones. Table A.13 reports the raw gap ∣ϕS∣|\phi_S| on training and on test members for the non-degenerate points. On the weak-evidence stratum, relative to ERM (training gap 0.185, test gap 0.200), EO at λ=50\lambda= 50 lowers the training gap by 0.090 but the test gap by only 0.037, whereas SAFER at λ=15\lambda= 15, at similar log-loss, lowers them by 0.098 and 0.071: the certainty-blind penalty fits training-set noise in the sparse cells. An independent size sweep (Table A.14) locates the transition for this planted gap: the certified fraction rises from 0.09±0.090.09 \pm0.09 at design size m=5m = 5 to 0.52±0.150.52 \pm0.15 at m=30m = 30, where the mean statistic equals 1.96, and to 0.95±0.080.95 \pm0.08 at m=120m = 120; the value m≈30m \approx30 is specific to this gap and reference size. On the real datasets, Adult and COMPAS were built as candidates for this regime; NoSelectivity tracks SAFER across the sweep on COMPAS and up to λ=1\lambda= 1 on Adult, where it then loses utility (Appendix § 14.§ 4).

setarmλ\lambdatest log-lossnullweakboundarystrong
–ERM0.356 ±\pm .004.89 ±\pm 1.033.49 ±\pm 2.692.01 ±\pm .747.42 ±\pm 1.30
ASAFER2.362 ±\pm .0031.09 ±\pm .773.35 ±\pm 2.131.51 ±\pm .566.35 ±\pm 1.16
ANS.5.361 ±\pm .0041.09 ±\pm .763.63 ±\pm 1.901.54 ±\pm .636.58 ±\pm 1.01
AEO5.367 ±\pm .0041.07 ±\pm .723.14 ±\pm 2.161.54 ±\pm .566.47 ±\pm 1.07
BSAFER5.381 ±\pm .0061.14 ±\pm 1.062.88 ±\pm 1.881.25 ±\pm .734.90 ±\pm 1.28
BNS1.379 ±\pm .0081.23 ±\pm .923.30 ±\pm 2.481.49 ±\pm .815.63 ±\pm 1.05
BEO10.381 ±\pm .0061.17 ±\pm .863.25 ±\pm 2.241.44 ±\pm .545.98 ±\pm 1.08
CSAFER15.457 ±\pm .0311.41 ±\pm .502.51 ±\pm 1.62.79 ±\pm .762.27 ±\pm 1.29
CNS1.25.467 ±\pm .0661.06 ±\pm .843.32 ±\pm 2.241.84 ±\pm 1.264.65 ±\pm 1.29
CEO50.466 ±\pm .0111.04 ±\pm .912.61 ±\pm 1.731.08 ±\pm .834.29 ±\pm .96

Table A.11. Matched-utility comparison on the controlled benchmark: fairness weight λ\lambda, test log-loss, and mean residual certified excess per evidence stratum in percentage points (mean ±\pm standard deviation over 10 runs; lower is better).

Ablation study details

The arm definitions and detailed numbers for the mechanism ablations of § 5.2 are given below.

Arm definitions

The four arms are evaluated on the three stress variants of Table 1 under the protocols of § 5:

  • SAFER: full method, all three components active.

  • NoThreshSmooth: τ\tau disabled (raw probabilities feed the soft counts).

  • NoSelectivity: the threshold term zαSE^eff,Sz_{\alpha}\widehat{SE}_{\mathrm{eff},S} is removed from the softplus argument, leaving ϕ^Sτ2+ε2\sqrt{\widehat{\phi}_{S}^{\tau2}+\varepsilon^2} alone.

  • NoDenseTail: wtail=0w_{\mathrm{tail}} = 0.

Apart from the toggled knob, every other element of the loss, optimizer, and protocol is identical across arms.

armλ\lambdatest log-lossnullweakboundarystrong
ERM0.356±.004.356 \pm.004.89±1.03.89 \pm1.033.49±2.693.49 \pm2.692.01±.742.01 \pm.747.42±1.307.42 \pm1.30
SAFER0.5.357±.003.357 \pm.0031.08±.941.08 \pm.944.02±2.434.02 \pm2.431.91±.841.91 \pm.846.95±1.216.95 \pm1.21
SAFER1.358±.003.358 \pm.0031.13±.791.13 \pm.793.55±2.273.55 \pm2.271.76±.621.76 \pm.626.77±1.386.77 \pm1.38
SAFER1.5.360±.003.360 \pm.0031.19±.801.19 \pm.803.16±2.123.16 \pm2.121.63±.491.63 \pm.496.74±1.296.74 \pm1.29
SAFER2.362±.003.362 \pm.0031.09±.771.09 \pm.773.35±2.133.35 \pm2.131.51±.561.51 \pm.566.35±1.166.35 \pm1.16
SAFER2.5.365±.004.365 \pm.0041.02±.731.02 \pm.733.28±2.073.28 \pm2.071.32±.651.32 \pm.656.15±1.226.15 \pm1.22
SAFER5.381±.006.381 \pm.0061.14±1.061.14 \pm1.062.88±1.882.88 \pm1.881.25±.731.25 \pm.734.90±1.284.90 \pm1.28
SAFER10.423±.018.423 \pm.0181.13±.671.13 \pm.672.53±1.532.53 \pm1.53.71±.59.71 \pm.592.90±1.252.90 \pm1.25
SAFER15.457±.031.457 \pm.0311.41±.501.41 \pm.502.51±1.622.51 \pm1.62.79±.76.79 \pm.762.27±1.292.27 \pm1.29
SAFER20.487±.037.487 \pm.0371.36±1.191.36 \pm1.192.87±1.982.87 \pm1.98.63±.81.63 \pm.812.01±1.582.01 \pm1.58
SAFER30.539±.018.539 \pm.018†.75±1.04.75 \pm1.043.13±3.913.13 \pm3.91.89±.86.89 \pm.861.63±1.241.63 \pm1.24
SAFER40.581±.021.581 \pm.021†.04±.07.04 \pm.072.68±3.332.68 \pm3.33.50±.75.50 \pm.75.31±.29.31 \pm.29
SAFER50.620±.024.620 \pm.024†.54±.96.54 \pm.963.77±2.723.77 \pm2.72.53±.73.53 \pm.73.43±.74.43 \pm.74
SAFER70.665±.013.665 \pm.013†.50±.82.50 \pm.823.33±3.613.33 \pm3.61.42±.77.42 \pm.77.14±.24.14 \pm.24
SAFER100.684±.004.684 \pm.004†.90±1.02.90 \pm1.022.62±2.362.62 \pm2.36.24±.39.24 \pm.39.20±.23.20 \pm.23
SAFER150.690±.002.690 \pm.002†.20±.32.20 \pm.32.30±.52.30 \pm.52.12±.22.12 \pm.22.02±.06.02 \pm.06
SAFER200.692±.001.692 \pm.001†.03±.10.03 \pm.10.04±.12.04 \pm.12.01±.03.01 \pm.03.00±.00.00 \pm.00
SAFER300.694±.002.694 \pm.002†.00±.00.00 \pm.00.00±.00.00 \pm.00.00±.00.00 \pm.00.00±.00.00 \pm.00
SAFER500.696±.003.696 \pm.003†.00±.00.00 \pm.00.00±.00.00 \pm.00.00±.00.00 \pm.00.00±.00.00 \pm.00
NS0.25.357±.004.357 \pm.0041.09±.881.09 \pm.884.15±2.694.15 \pm2.691.83±.641.83 \pm.646.80±1.106.80 \pm1.10
NS0.5.361±.004.361 \pm.0041.09±.761.09 \pm.763.63±1.903.63 \pm1.901.54±.631.54 \pm.636.58±1.016.58 \pm1.01
NS0.75.366±.005.366 \pm.0051.05±.681.05 \pm.683.41±2.303.41 \pm2.301.38±.781.38 \pm.786.15±1.266.15 \pm1.26
NS1.379±.008.379 \pm.0081.23±.921.23 \pm.923.30±2.483.30 \pm2.481.49±.811.49 \pm.815.63±1.055.63 \pm1.05
NS1.25.467±.066.467 \pm.0661.06±.841.06 \pm.843.32±2.243.32 \pm2.241.84±1.261.84 \pm1.264.65±1.294.65 \pm1.29
NS1.5.528±.073.528 \pm.073†.57±.74.57 \pm.744.96±4.864.96 \pm4.861.73±1.211.73 \pm1.213.18±1.933.18 \pm1.93
NS1.75.562±.077.562 \pm.077†.58±.58.58 \pm.584.60±4.174.60 \pm4.17.82±.94.82 \pm.941.10±1.511.10 \pm1.51
NS2.586±.071.586 \pm.071†.86±.90.86 \pm.902.71±2.862.71 \pm2.86.55±.43.55 \pm.43.95±1.24.95 \pm1.24
NS2.25.608±.060.608 \pm.060†.48±.62.48 \pm.621.90±2.281.90 \pm2.28.40±.47.40 \pm.47.59±.78.59 \pm.78
NS2.5.627±.051.627 \pm.051†.36±.53.36 \pm.531.32±1.701.32 \pm1.70.30±.38.30 \pm.38.43±.51.43 \pm.51
NS2.75.643±.043.643 \pm.043†.30±.41.30 \pm.411.15±1.741.15 \pm1.74.13±.15.13 \pm.15.23±.38.23 \pm.38
NS3.653±.039.653 \pm.039†.37±.46.37 \pm.46.62±.85.62 \pm.85.18±.24.18 \pm.24.21±.28.21 \pm.28
NS3.5.671±.018.671 \pm.018†.14±.26.14 \pm.26.24±.55.24 \pm.55.04±.07.04 \pm.07.09±.22.09 \pm.22
NS3.75.677±.009.677 \pm.009†.14±.34.14 \pm.34.17±.45.17 \pm.45.02±.05.02 \pm.05.07±.20.07 \pm.20
NS4.681±.006.681 \pm.006†.09±.26.09 \pm.26.13±.39.13 \pm.39.01±.02.01 \pm.02.03±.08.03 \pm.08
NS4.5.685±.004.685 \pm.004†.07±.21.07 \pm.21.08±.23.08 \pm.23.00±.00.00 \pm.00.00±.00.00 \pm.00
NS5.688±.003.688 \pm.003†.03±.11.03 \pm.11.04±.12.04 \pm.12.02±.05.02 \pm.05.00±.00.00 \pm.00
NS5.4.689±.002.689 \pm.002†.02±.06.02 \pm.06.02±.07.02 \pm.07.01±.04.01 \pm.04.00±.00.00 \pm.00
EO5.367±.004.367 \pm.0041.07±.721.07 \pm.723.14±2.163.14 \pm2.161.54±.561.54 \pm.566.47±1.076.47 \pm1.07
EO10.381±.006.381 \pm.0061.17±.861.17 \pm.863.25±2.243.25 \pm2.241.44±.541.44 \pm.545.98±1.085.98 \pm1.08
EO20.409±.009.409 \pm.0091.13±1.011.13 \pm1.012.87±2.092.87 \pm2.091.36±.781.36 \pm.785.06±.915.06 \pm.91
EO50.466±.011.466 \pm.0111.04±.911.04 \pm.912.61±1.732.61 \pm1.731.08±.831.08 \pm.834.29±.964.29 \pm.96
EO60.479±.010.479 \pm.0101.03±.911.03 \pm.912.57±1.732.57 \pm1.731.04±.761.04 \pm.764.20±1.034.20 \pm1.03
EO200.561±.011.561 \pm.011†1.20±.771.20 \pm.773.27±1.913.27 \pm1.911.13±.931.13 \pm.934.37±1.234.37 \pm1.23
EO600.633±.011.633 \pm.011†.61±.76.61 \pm.762.56±1.782.56 \pm1.781.23±.791.23 \pm.793.89±1.943.89 \pm1.94
EO1000.660±.008.660 \pm.008†.54±.54.54 \pm.543.83±2.853.83 \pm2.85.93±.76.93 \pm.763.22±2.313.22 \pm2.31
EO1500.674±.006.674 \pm.006†.68±.56.68 \pm.563.20±1.593.20 \pm1.59.77±.86.77 \pm.862.20±1.722.20 \pm1.72
EO2000.681±.004.681 \pm.004†.49±.67.49 \pm.673.05±3.953.05 \pm3.95.61±.66.61 \pm.661.74±1.421.74 \pm1.42
EO2500.684±.003.684 \pm.003†.42±.76.42 \pm.763.13±3.283.13 \pm3.28.55±.90.55 \pm.901.41±1.081.41 \pm1.08
EO5000.690±.002.690 \pm.002†.41±.67.41 \pm.671.77±2.131.77 \pm2.13.50±.65.50 \pm.65.35±.52.35 \pm.52
EO7500.691±.001.691 \pm.001†.39±.71.39 \pm.712.03±2.852.03 \pm2.85.31±.46.31 \pm.46.12±.22.12 \pm.22
EO10000.692±.001.692 \pm.001†.27±.52.27 \pm.521.27±1.661.27 \pm1.66.42±.75.42 \pm.75.06±.13.06 \pm.13

Table A.12. Complete λ\lambda sweeps on the controlled benchmark: test log-loss and mean residual certified excess per evidence stratum in percentage points (mean ±\pm standard deviation over 10 runs; lower is better). NS is NoSelectivity and EO is EO-Penalty. Rows marked † have mean test log-loss at or above 0.5: these models approach the constant predictor (log-loss 0.693), and their small excess values come from degenerate predictions rather than from fairer ones. The matched levels of Table A.11 are drawn from the unmarked rows.

armλ\lambdatest log-loss\multicolumn{4}{c}{test ∣ϕS∣|\phi_S|}\multicolumn{4}{c}{train ∣ϕS∣|\phi_S|}
nullweakboundarystrongnullweakboundarystrong
ERM0.356 ± .004.048 ± .015.200 ± .027.126 ± .027.137 ± .013.054 ± .009.185 ± .031.124 ± .022.134 ± .017
SAFER0.5.357 ± .003.050 ± .015.201 ± .028.124 ± .030.131 ± .012.050 ± .011.175 ± .026.111 ± .021.125 ± .015
SAFER1.358 ± .003.050 ± .014.187 ± .027.122 ± .029.129 ± .014.046 ± .011.170 ± .027.100 ± .021.116 ± .015
SAFER1.5.360 ± .003.049 ± .014.182 ± .025.118 ± .028.128 ± .013.045 ± .009.169 ± .027.095 ± .019.110 ± .014
SAFER2.362 ± .003.047 ± .013.185 ± .024.117 ± .027.124 ± .012.043 ± .009.162 ± .026.092 ± .016.106 ± .012
SAFER2.5.365 ± .004.045 ± .011.182 ± .025.115 ± .026.121 ± .013.040 ± .008.155 ± .020.087 ± .013.103 ± .012
SAFER5.381 ± .006.041 ± .009.170 ± .023.105 ± .033.105 ± .015.038 ± .009.139 ± .018.071 ± .007.085 ± .014
SAFER10.423 ± .018.033 ± .007.143 ± .027.084 ± .026.077 ± .018.031 ± .007.099 ± .016.054 ± .010.053 ± .008
SAFER15.457 ± .031.035 ± .013.129 ± .031.080 ± .034.068 ± .022.027 ± .006.087 ± .025.044 ± .015.036 ± .007
SAFER20.487 ± .037.041 ± .023.129 ± .047.076 ± .038.065 ± .027.031 ± .014.095 ± .035.041 ± .016.034 ± .011
NS0.25.357 ± .004.050 ± .015.194 ± .033.123 ± .026.129 ± .011.046 ± .014.147 ± .027.111 ± .025.125 ± .014
NS0.5.361 ± .004.048 ± .014.191 ± .031.119 ± .028.127 ± .010.040 ± .010.128 ± .021.102 ± .024.117 ± .014
NS0.75.366 ± .005.044 ± .010.183 ± .028.115 ± .028.121 ± .013.038 ± .009.121 ± .019.094 ± .024.113 ± .015
NS1.379 ± .008.044 ± .011.171 ± .027.113 ± .030.114 ± .011.035 ± .011.106 ± .014.080 ± .015.105 ± .018
NS1.25.467 ± .066.053 ± .018.172 ± .039.120 ± .041.106 ± .017.037 ± .013.094 ± .017.080 ± .023.102 ± .029
EO5.367 ± .004.048 ± .013.182 ± .027.119 ± .030.125 ± .011.045 ± .012.127 ± .021.099 ± .018.118 ± .016
EO10.381 ± .006.046 ± .011.182 ± .026.114 ± .027.119 ± .012.042 ± .009.108 ± .015.090 ± .015.109 ± .017
EO20.409 ± .009.043 ± .011.171 ± .021.108 ± .029.108 ± .010.039 ± .008.102 ± .010.074 ± .012.099 ± .018
EO50.466 ± .011.040 ± .011.163 ± .024.099 ± .030.098 ± .010.036 ± .009.095 ± .013.064 ± .012.088 ± .022
EO60.479 ± .010.041 ± .011.157 ± .020.099 ± .030.097 ± .011.036 ± .011.096 ± .014.064 ± .012.087 ± .021

Table A.13. Raw EO gap ∣ϕS∣|\phi_S| on test and on training members, mean over the cells of each evidence stratum, for the non-degenerate points of the sweeps (test log-loss below 0.5; mean ± standard deviation over 10 runs). Train and test values come from the same model. A training gap that falls much faster than the test gap on the sparse strata (weak, boundary) indicates fitting of training-set noise.

mm5101520305080120150
mean ΛS\sqrt{\Lambda_S}1.031.291.551.491.962.152.993.574.11
certified fraction.09 ± .09.23 ± .16.33 ± .18.27 ± .18.52 ± .15.58 ± .24.85 ± .18.95 ± .08.97 ± .07
cells (all runs)586060606060606060

Table A.14. Size sweep: ERM models on a variant of the benchmark with one reference cell (n=150,000n = 150{,}000, true-positive rate 0.85) and six probe cells per design size mm, all with planted gap −0.15-0.15. For each mm, the mean certification statistic ΛS\sqrt{\Lambda_S} over the probe cells and runs, computed on hard training predictions of the final ERM model, and the fraction of probe cells certified (ΛS>1.96\sqrt{\Lambda_S} > 1.96) as mean ± standard deviation over 10 runs. At m=5m = 5, two of the 60 cells had fewer than two positives and were skipped.

Per-mechanism detailed numbers

Threshold smoothing. On Adult L4 10/90 (10 splits ×\times 3 seeds), reaching MSE≈10−3MSE \approx10^{-3} requires λ=1000\lambda= 1000 at log-loss ≈0.57\approx0.57 for NoThreshSmooth, versus λ=200\lambda= 200 at log-loss ≈0.43\approx0.43 for SAFER. At matched fairness, the difference is roughly 0.14 nats. On COMPAS L3 2080, the gap is 0.01–0.02 nats at matched MSE through the mid-range (Figure 2, center).

Dense tail. On COMPAS L3 2080, at λ≤2\lambda\le2 the four arms are within the ±\pmSE bands of one another. From λ=5\lambda= 5 onward, NoDenseTail plateaus at MSE≈0.06MSE \approx0.06; the gap to SAFER widens monotonically and reaches roughly ten standard errors at λ=100\lambda= 100 (SAFER 0.018 ± 0.001 versus NoDenseTail 0.060 ± 0.004). On Adult L4 1090 the truncation has a different shape but the predicted form: NoDenseTail stays close to ERM MSE (≈0.02\approx0.02) for almost every λ\lambda, and only at the largest tested λ\lambda does it move toward smaller MSE, where it is Pareto-dominated by SAFER. On German L3 50/50 NoDenseTail tracks SAFER, because the significance gate already keeps the hinge inactive (Proposition 4).

Gate threshold (NoSelectivity on German). SAFER preserves ERM-level log-loss (≈0.53\approx0.53) up to λ=20\lambda= 20 and degrades only mildly (0.59 at λ=100\lambda= 100). NoSelectivity climbs from log-loss 0.53 at λ=1\lambda= 1 through 0.60 at λ=5\lambda= 5 and reaches 0.69 by λ=20\lambda= 20; the error bars on its MSE become very large in this range.

NoThreshSmooth on German: no departure from ERM

NoThreshSmooth on German exhibits a different and benign failure mode from NoSelectivity: the raw soft-count gradient is too small to move the model on this small dataset, so the optimizer never leaves ERM regardless of λ\lambda. This is consistent with Proposition 3: when subgroup distributions are tightly clustered well above (or below) the decision boundary, raw probabilities and hard decisions disagree but the threshold-smoothing step that would expose the hard-decision gap is absent. This safe behavior is therefore a side effect of the small gradient and is not guaranteed by the method.

Selective-gating: what did not appear

The hypothesis for the selective-gating regime, stated in Appendix F.3, is that the gate threshold gives SAFER an advantage over an ungated alternative when active and inactive subgroups coexist. The Adult L4 1090 and COMPAS L3 2080 results in Figure 2 (left and center) do not exhibit this behavior: NoSelectivity tracks SAFER up to λ=1\lambda= 1 on Adult, where it then loses utility, and across the sweep on COMPAS. We interpret this as evidence that the gate’s empirical role is asymmetric on these two datasets: it suppresses harm in the safety regime, while the evidence for its selective-regime benefit is directional and comes from the controlled benchmark of Appendix F.3, where the evidence strata are isolated.

Cross-regime ablation map

Table A.15: Empirical effect of removing each component on each stress variant. “≈0” indicates indistinguishable from full SAFER within ±SE. “↓↓” indicates Pareto-dominated by SAFER across the regularization sweep. “↓” indicates a Pareto gap that opens at high λ\lambda. “harm” indicates that the ablation degrades utility without any certified-fairness benefit. “n/a” indicates the ablation was not run rigorously at the matched protocol.

AblationAdult L4 1090COMPAS L3 2080German L3 5050
NoThreshSmooth (Theorem 2)↓↓↓↓≈0 (no traction)
NoDenseTail ((12))↓ at high λ\lambda↓ at high λ\lambda≈0
NoSelectivity (Proposition 4)≈0, harm at high λ\lambda≈0harm

Table A.15.

The Eq. (12) mechanism predicts that NoDenseTail under-reaches SAFER at high λ\lambda on certified-violation datasets, and we observe this on both Adult L4 1090 and COMPAS L3 2080. The shape of the truncation differs: COMPAS shows a clean plateau at MSEE≈0.06MSEE \approx0.06 from λ=5\lambda= 5 onward, while Adult shows NoDenseTail stuck near ERM MSEE for almost the entire λ\lambda sweep, escaping only at the largest tested λ\lambda and remaining Pareto-dominated by SAFER (Figure 2, left). Both shapes are consistent with the same hinge-suppression argument; a more controlled comparison of where each dataset’s subgroup statistics sit relative to the gate boundary would tighten the quantitative match and is left to future work.

Statistical Parity results

The main text focuses on EO because its positive-label conditioning makes the sparse-cell issue most severe. SP has larger effective subgroup sizes because it conditions on all examples in a subgroup. Figure A.1 shows the Adult, COMPAS and German SP Pareto frontiers. The penalty baseline, SP-Penalty, is the certainty-blind squared-disparity regularizer applied to the SP gap. Results are consistent with the EO interpretation in § 5.

Other instantiations

Equalized Odds

Section 4 shows that the SAFER framework extends beyond the single-component SP/EO case to multi-component criteria, by summing componentwise hinges. We demonstrate this concretely for

Figure A.1. Pareto frontiers for Adult, COMPAS and German Credit under Statistical Parity.

Equalized Odds [17], which requires simultaneous parity of the true-positive rate (TPR) and the false-positive rate (FPR) across subgroups:

EO(S)=P(Y^=1∣Y=1,S)−P(Y^=1∣Y=1,Sˉ),(18)EO(S) = P(\hat{Y}=1 \mid Y=1, S) - P(\hat{Y}=1 \mid Y=1, \bar{S}), \tag*{(18)}
FP(S)=P(Y^=1∣Y=0,S)−P(Y^=1∣Y=0,Sˉ).(19)FP(S) = P(\hat{Y}=1 \mid Y=0, S) - P(\hat{Y}=1 \mid Y=0, \bar{S}). \tag*{(19)}

The TPR component is the Equal Opportunity criterion already treated in the main paper; FPR parity is the complementary condition on negative-label examples.

FPR soft counts. Section 4 defines TPR soft counts by restricting to positive-label examples. The FPR component requires the analogous restriction to negative-label examples:

n~1,S(−)=∑i:yi=0y^i1S(i),n~0,S(−)=∑i:yi=0(1−y^i)1S(i).(20)\tilde{n}_{1,S}^{(-)} = \sum_{i:y_i=0} \hat{y}_i 1_S(i), \qquad\tilde{n}_{0,S}^{(-)} = \sum_{i:y_i=0} (1-\hat{y}_i)1_S(i). \tag*{(20)}

These induce a differentiable FPR estimate f^S=n~1,S(−)/(n~1,S(−)+n~0,S(−))\hat{f}_S = \tilde{n}_{1,S}^{(-)} / (\tilde{n}_{1,S}^{(-)} + \tilde{n}_{0,S}^{(-)}) and, via the identical two-sample proportion construction, a differentiable FPR disparity FP^S(θ)=f^S−f^Sˉ\widehat{FP}_S(\theta) = \hat{f}_S - \hat{f}_{\bar{S}} and LRT statistic TSFPR(θ)=ΛSFPRT_S^{\mathrm{FPR}}(\theta) = \sqrt{\Lambda_S^{\mathrm{FPR}}} where ΛSFPR\Lambda_S^{\mathrm{FPR}} is the log-likelihood ratio for the null of equal FPR between SS and Sˉ\bar{S}.

Combined SAFER objective. The Equalized Odds variant penalizes both components independently:

LSAFEREqOdds(θ)=Lpred(θ)+λ∣G∣∑S∈G[Ldir,STPR(θ)+Ldir,SFPR(θ)].(21)\mathcal{L}_{\mathrm{SAFER}}^{\mathrm{EqOdds}}(\theta) = \mathcal{L}_{\mathrm{pred}}(\theta) + \frac{\lambda}{|\mathcal{G}|}\sum_{S\in\mathcal{G}}\left[\mathcal{L}_{\mathrm{dir},S}^{\mathrm{TPR}}(\theta) + \mathcal{L}_{\mathrm{dir},S}^{\mathrm{FPR}}(\theta)\right]. \tag*{(21)}

Each component uses the same direct significant-excess form as Eq. (11), but with soft counts restricted to the corresponding label stratum. The significance gate and the safety argument apply componentwise: a subgroup whose TPR and FPR components both lie below the gate threshold has only the exponentially small below-threshold gradient described by Eq. (12), plus any dense-tail residual.

Effective sample sizes. For EqOdds, the binding constraint is the smaller of the two label-conditioned counts:

mSEqOdds=min⁡(mS(+),mS(−)),mS(+)=∣{i∈S:yi=1}∣,mS(−)=∣{i∈S:yi=0}∣.m_S^{\mathrm{EqOdds}} = \min\left(m_S^{(+)},m_S^{(-)}\right), \qquad m_S^{(+)} = \left|\{i \in S : y_i = 1\}\right|,\qquad m_S^{(-)} = \left|\{i \in S : y_i = 0\}\right|.

Whenever ρ^≪1\hat{\rho} \ll1 (the typical regime for sparse intersectional subgroups), the positive-label count mS(+)≈nSρ^Sm_S^{(+)} \approx n_S\hat{\rho}_S is the binding term; when negative labels are rarer, the FPR component is binding instead.

Metric. Specializing Definition 1 to the two components yields the combined certified-excess metric:

MSE⁡αEqOdds(θ)=1∣S∣∑S∈S[EαTPR(S)+EαFPR(S)].(22)\operatorname{MSE}_{\alpha}^{\mathrm{EqOdds}}(\theta) = \frac{1}{|\mathcal{S}|}\sum_{S \in\mathcal{S}}\left[\mathcal{E}_{\alpha}^{\mathrm{TPR}}(S) + \mathcal{E}_{\alpha}^{\mathrm{FPR}}(S)\right]. \tag*{(22)}

where each component is the significant excess of Definition 1 applied to the respective label stratum. This metric is zero if and only if no subgroup has a certified violation in either the TPR or FPR component, inheriting property 1 of the significant-excess family.

Disparate Impact instantiation

Disparate Impact (DI) [12] measures the ratio of positive-prediction rates across subgroups:

DI(S)=P(Y^=1∣S)P(Y^=1∣Sˉ).DI(S) = \frac{P(\hat{Y}=1 \mid S)}{P(\hat{Y}=1 \mid\bar{S})}.

We use DI in its parity formulation, which tests whether DI(S)=1DI(S)=1, rather than the thresholded 0.8-rule used in legal practice. The parity null DI(S)=1DI(S)=1 is equivalent to P(Y^=1∣S)=P(Y^=1∣Sˉ)P(\hat{Y}=1 \mid S)=P(\hat{Y}=1 \mid\bar{S}), which is exactly the null of Statistical Parity. Consequently, test statistic and the two-sample LRT statistic for DI parity are identical to the one already defined for SP. Therefore, the SAFER training objective for DI is the same as the SP variant of Eq. (16).

Although the training objective is shared with SP, the significant-excess metric can be expressed in log-ratio units, by defining the log-DI functional ϕDI(S)=log⁡DI(S)=log⁡pS−log⁡pSˉ\phi^{DI}(S)=\log DI(S)=\log p_S-\log p_{\bar{S}}. The significant-excess is derived by following Definition 1 given a confidence interval [ℓα(S),uα(S)][\ell_{\alpha}(S),u_{\alpha}(S)] from a statistical test on ϕDI(S)\phi^{DI}(S).

Proofs for Section 4

Throughout this appendix, we write στ(u)=σ(u/τ)\sigma_{\tau}(u)=\sigma(u/\tau), where σ\sigma is the logistic sigmoid, and softplus⁡τ(u)=τlog⁡(1+eu/τ)\operatorname{softplus}_{\tau}(u)=\tau\log(1+e^{u/\tau}).

Basic sigmoid bounds

Lemma 5 (Sigmoid–step difference). Let σ\sigma be the logistic sigmoid and στ(u)=σ(u/τ)\sigma_{\tau}(u)=\sigma(u/\tau). For every u∈Ru \in\mathbb{R} and τ>0\tau>0,

∣στ(u)−1{u>0}∣=σ(−∣u∣/τ).\left|\sigma_{\tau}(u)-\mathbb{1}\{u>0\}\right|=\sigma(-|u|/\tau).

Proof. If u>0u>0, the difference is 1−σ(u/τ)=σ(−u/τ)1-\sigma(u/\tau)=\sigma(-u/\tau). If u<0u<0, it is σ(u/τ)=σ(−∣u∣/τ)\sigma(u/\tau)=\sigma(-|u|/\tau). At u=0u=0, both sides are 1/21/2.

Lemma 6 (Integral and derivative bounds). For every τ>0\tau>0,

∫−∞∞σ(−∣v∣/τ) dv=2τlog⁡2,σ′(z)≤e−∣z∣∀z∈R.\int_{-\infty}^{\infty}\sigma(-|v|/\tau)\,\mathrm{d}v=2\tau\log2,\qquad\sigma'(z)\le e^{-|z|}\quad\forall z\in\mathbb{R}.

Proof. The integral follows by the change of variables w=v/τw = v/\tau:

∫−∞∞σ(−∣v∣/τ) dv=2τ∫0∞e−w1+e−w dw=2τlog⁡2.\int_{-\infty}^{\infty} \sigma(-|v|/\tau)\,dv = 2\tau\int_{0}^{\infty} \frac{e^{-w}}{1+e^{-w}}\,dw = 2\tau\log2.

For the derivative bound, use σ′(z)=σ(z)(1−σ(z))\sigma'(z) = \sigma(z)(1-\sigma(z)). If z≥0z \ge0, then 1−σ(z)=σ(−z)≤e−z1-\sigma(z) = \sigma(-z) \le e^{-z}. Symmetry gives the case z<0z < 0.

Proof of Theorem 2

Let FSF_S be the conditional distribution of fθ(X)f_\theta(X) on the conditioning population ΠS\Pi_S, and define

pSτ=EFS[στ(fθ(X)−12)],pShard=EFS[1{fθ(X)>12}].p_S^\tau= \mathbb{E}_{F_S}\left[\sigma_\tau\left(f_\theta(X)-\frac{1}{2}\right)\right], \qquad p_S^{\mathrm{hard}} = \mathbb{E}_{F_S}\left[\mathbf{1}\left\{f_\theta(X)>\frac{1}{2}\right\}\right].

By Lemma 5,

∣pSτ−pShard∣≤∫σ(−∣u−12∣/τ) dFS(u).\left|p_S^\tau-p_S^{\mathrm{hard}}\right| \le\int\sigma\left(-\left|u-\frac{1}{2}\right|/\tau\right)\,dF_S(u).

Let N={u:∣u−12∣≤δ}N=\left\{u:\left|u-\frac{1}{2}\right|\le\delta\right\} be the interval on which, by assumption, FSF_S has a density bounded by LL, so that dFS(u)≤L dudF_S(u)\le L\,du on NN. We split the integral at NN. On NN, substituting v=u−12v=u-\frac{1}{2}, extending the range of integration to the real line and using Lemma 6,

∫Nσ(−∣u−12∣/τ) dFS(u)≤L∫−∞∞σ(−∣v∣/τ) dv=2Lτlog⁡2.\int_N \sigma\left(-\left|u-\frac{1}{2}\right|/\tau\right)\,dF_S(u) \le L\int_{-\infty}^{\infty}\sigma(-|v|/\tau)\,dv = 2L\tau\log2.

Outside NN we have ∣u−12∣>δ\left|u-\frac{1}{2}\right|>\delta, so σ(−∣u−12∣/τ)≤σ(−δ/τ)≤e−δ/τ\sigma\left(-\left|u-\frac{1}{2}\right|/\tau\right)\le\sigma(-\delta/\tau)\le e^{-\delta/\tau}, and the mass of FSF_S outside NN is at most one; hence

∫Ncσ(−∣u−12∣/τ) dFS(u)≤e−δ/τ.\int_{N^c}\sigma\left(-\left|u-\frac{1}{2}\right|/\tau\right)\,dF_S(u)\le e^{-\delta/\tau}.

Adding the two parts,

∣pSτ−pShard∣≤2Lτlog⁡2+e−δ/τ.\left|p_S^\tau-p_S^{\mathrm{hard}}\right|\le2L\tau\log2+e^{-\delta/\tau}.

The same bound holds for S‾\overline{S}. Since ϕS∙=pS∙−pS‾∙\phi_S^\bullet=p_S^\bullet-p_{\overline{S}}^\bullet, the triangle inequality gives:

∣ϕSτ−ϕShard∣≤∣pSτ−pShard∣+∣pS‾τ−pS‾hard∣≤4Lτlog⁡2+2e−δ/τ.\left|\phi_S^\tau-\phi_S^{\mathrm{hard}}\right|\le\left|p_S^\tau-p_S^{\mathrm{hard}}\right|+\left|p_{\overline{S}}^\tau-p_{\overline{S}}^{\mathrm{hard}}\right|\le4L\tau\log2+2e^{-\delta/\tau}.

The remainder is also linear in τ\tau: since e−x≤1/(ex)e^{-x}\le1/(ex) for x>0x>0, we have 2e−δ/τ≤2τ/(eδ)2e^{-\delta/\tau}\le2\tau/(e\delta), so the whole bound is at most (4Llog⁡2+2/(eδ))τ(4L\log2+2/(e\delta))\tau. Outside NN the score distribution is arbitrary; in particular it may have atoms at 0 and 1. □

Proof of Proposition 3

Fix η>0\eta>0. Choose κ<η/2\kappa<\eta/2 and let FSF_S concentrate smoothly in a small interval around 1/2+κ1/2+\kappa, while FS‾F_{\overline{S}} concentrates smoothly in a small interval around 1/2−κ1/2-\kappa. Then

ϕSraw≈2κ,ϕShard≈1,\phi_S^{\mathrm{raw}}\approx2\kappa,\qquad\phi_S^{\mathrm{hard}}\approx1,

so

∣ϕSraw−ϕShard∣≥1−η\left|\phi_S^{\mathrm{raw}}-\phi_S^{\mathrm{hard}}\right|\ge1-\eta

for sufficiently narrow intervals. The distributions may be chosen with continuous densities bounded by a finite constant.

To see that no constant rescaling works uniformly, compare this construction with a second one in which both groups are well above the threshold, with mean scores differing by Δ>0\Delta>0. Then ϕSraw≈Δ\phi_S^{\mathrm{raw}}\approx\Delta while ϕShard=0\phi_S^{\mathrm{hard}}=0. Matching the first construction as κ→0\kappa\to0 requires a rescaling constant of order 1/(2κ)1/(2\kappa), which then makes the rescaled raw disparity in the second construction diverge instead of matching zero. Hence a single constant cannot align raw soft counts with hard-decision disparities across both regimes. □

Gradient localization at the decision boundary

The following result, deferred from § 4.1, shows that threshold smoothing localizes the fairness gradient to examples near the decision boundary {x:fθ(x)=12}\{x:f_\theta(x)=\frac{1}{2}\}.

Theorem 7 (Gradient localization). For SP and EO,

∣∂ϕSτ∂fθ(x)∣≤cS(x)τexp⁡(−∣fθ(x)−12∣τ),(23)\left|\frac{\partial\phi_S^\tau}{\partial f_\theta(x)}\right| \le\frac{c_S(x)}{\tau}\exp\left(-\frac{\left|f_\theta(x)-\frac{1}{2}\right|}{\tau}\right), \tag*{(23)}

for a bounded subgroup-membership weight cS(x)c_S(x). As τ→0+\tau\to0^+, the gradient concentrates on {x:fθ(x)=12}\{x:f_\theta(x)=\frac{1}{2}\}.

Proof. For x∈ΠSx\in\Pi_S,

∂ϕSτ∂fθ(x)=1τσ′((fθ(x)−12)/τ)cS+(x),\frac{\partial\phi_S^\tau}{\partial f_\theta(x)}=\frac{1}{\tau}\sigma'\left((f_\theta(x)-\tfrac{1}{2})/\tau\right)c_S^+(x),

where cS+(x)c_S^+(x) is the appropriate conditioning-population weight. The corresponding complement term has the same form with a possibly different weight. Therefore, for a bounded subgroup-membership weight cS(x)c_S(x),

∣∂ϕSτ∂fθ(x)∣≤cS(x)τσ′((fθ(x)−12)/τ)≤cS(x)τe−∣fθ(x)−1/2∣/τ,\left|\frac{\partial\phi_S^\tau}{\partial f_\theta(x)}\right|\le\frac{c_S(x)}{\tau}\sigma'\left((f_\theta(x)-\tfrac{1}{2})/\tau\right)\le\frac{c_S(x)}{\tau}e^{-|f_\theta(x)-1/2|/\tau},

using Lemma 6. Finally, τ−1σ′((u−12)/τ)\tau^{-1}\sigma'((u-\frac{1}{2})/\tau) is an approximate identity that converges weakly to a Dirac mass at u=12u=\frac{1}{2}, yielding the distributional limit. □\square

Canonical surrogate approximation

The main text (Eq. (11)) uses an LRT-based threshold. Here we state and prove a metric-alignment bound for a canonical version in which the surrogate and the metric share the population Wald threshold BSB_S. The bound shows that the surrogate shape tracks MSEEαpopMSEE_\alpha^{\mathrm{pop}}, defined below. This population comparison does not bound the discrepancy between the implemented LRT-based training loss and the SAFT audit metric.

Let BS(θ)=zαSESϕ(θ)B_S(\theta)=z_\alpha SE_S^\phi(\theta), where SESϕSE_S^\phi is evaluated with population cell probabilities. Define

MSEEαpop(θ)=1∣S∣∑S∈S[∣ϕShard(θ)∣−BS(θ)]+,(24)MSEE_\alpha^{\mathrm{pop}}(\theta)=\frac{1}{|\mathcal{S}|}\sum_{S\in\mathcal{S}}\left[|\phi_S^{\mathrm{hard}}(\theta)|-B_S(\theta)\right]_+, \tag*{(24)}
Lcan(θ)=1∣S∣∑S∈Ssoftplus⁡τs((ϕSτ(θ))2+ε2−BS(θ)).(25)L_{\mathrm{can}}(\theta)=\frac{1}{|\mathcal{S}|}\sum_{S\in\mathcal{S}}\operatorname{softplus}_{\tau_s}\left(\sqrt{(\phi_S^\tau(\theta))^2+\varepsilon^2}-B_S(\theta)\right). \tag*{(25)}

Theorem 8 (Canonical surrogate approximation). Under Theorem 2’s density assumption, with the same LL and δ\delta, for every ε,τs,τ>0\varepsilon,\tau_s,\tau>0,

∣Lcan(θ)−MSEEαpop(θ)∣≤ε+τslog⁡2+4Lτlog⁡2+2e−δ/τ.(26)\left|L_{\mathrm{can}}(\theta)-MSEE_\alpha^{\mathrm{pop}}(\theta)\right|\le\varepsilon+\tau_s\log2+4L\tau\log2+2e^{-\delta/\tau}. \tag*{(26)}

Proof. Fix a subgroup SS, write BS=BS(θ)B_S=B_S(\theta), and define

hS=softplus⁡τs((ϕSτ)2+ε2−BS),mS=[∣ϕShard∣−BS]+.h_S=\operatorname{softplus}_{\tau_s}\left(\sqrt{(\phi_S^\tau)^2+\varepsilon^2}-B_S\right),\qquad m_S=\left[|\phi_S^{\mathrm{hard}}|-B_S\right]_+.

The elementary inequalities

∣x2+ε2−∣x∣∣≤ε,0≤softplus⁡τs(u)−[u]+≤τslog⁡2,\left|\sqrt{x^2+\varepsilon^2}-|x|\right|\le\varepsilon,\qquad0\le\operatorname{softplus}_{\tau_s}(u)-[u]_+\le\tau_s\log2,

and the Lipschitz property of x↦[x]+x\mapsto[x]_+ imply

∣hS−mS∣≤ε+τslog⁡2+∣∣ϕSτ∣−∣ϕShard∣∣.|h_S-m_S|\le\varepsilon+\tau_s\log2+\left||\phi_S^\tau|-|\phi_S^{\mathrm{hard}}|\right|.

Since ∣a∣−∣b∣≤∣a−b∣|a|-|b|\le|a-b|, Theorem 2 gives

∣hS−mS∣≤ε+τslog⁡2+4Lτlog⁡2+2e−δ/τ.|h_S-m_S|\le\varepsilon+\tau_s\log2+4L\tau\log2+2e^{-\delta/\tau}.

Averaging this bound over S∈SS\in\mathcal{S} proves Eq. (A.10). □\square

Asymptotic equivalence of the soft-count LRT and the Wald statistic

Write Qi=στ(fθ(xi)−12)∈(0,1)Q_i = \sigma_\tau(f_\theta(x_i) - \frac{1}{2}) \in(0,1) for the soft count of observation ii, so that p^S\hat{p}_S and p^Sˉ\hat{p}_{\bar{S}} are the means of the QiQ_i over ΠS\Pi_S and ΠSˉ\Pi_{\bar{S}}, p^\hat{p} is the pooled mean, and n~S=∣ΠS∣\tilde{n}_S = |\Pi_S|, n~Sˉ=∣ΠSˉ∣\tilde{n}_{\bar{S}} = |\Pi_{\bar{S}}|. With D(a∥b)=alog⁡ab+(1−a)log⁡1−a1−bD(a \mathbin{\|} b) = a \log\frac{a}{b} + (1-a) \log\frac{1-a}{1-b} the Bernoulli divergence, Eq. (9) reads

ΛS=2[n~SD(p^S∥p^)+n~SˉD(p^Sˉ∥p^)].\Lambda_S = 2\left[\tilde{n}_S D(\hat{p}_S \mathbin{\|} \hat{p}) + \tilde{n}_{\bar{S}} D(\hat{p}_{\bar{S}} \mathbin{\|} \hat{p})\right].

Set Δ=p^S−p^Sˉ\Delta= \hat{p}_S - \hat{p}_{\bar{S}} and N=n~S+n~SˉN = \tilde{n}_S + \tilde{n}_{\bar{S}}, so that p^S−p^=(n~Sˉ/N)Δ\hat{p}_S - \hat{p} = (\tilde{n}_{\bar{S}}/N)\Delta and p^Sˉ−p^=−(n~S/N)Δ\hat{p}_{\bar{S}} - \hat{p} = -(\tilde{n}_S/N)\Delta. Since D(a∥b)=(a−b)22b(1−b)+O(∣a−b∣3)D(a \mathbin{\|} b) = \frac{(a-b)^2}{2b(1-b)} + O(|a-b|^3) for bb bounded away from 0 and 1,

ΛS=Δ2p^(1−p^)(1/n~S+1/n~Sˉ)(1+O(∣Δ∣)).\Lambda_S = \frac{\Delta^2}{\hat{p}(1-\hat{p})(1/\tilde{n}_S + 1/\tilde{n}_{\bar{S}})}\left(1+O(|\Delta|)\right).

The plug-in Wald variance of property (ii) satisfies (SE^Sϕ)2=p^(1−p^)(1/n~S+1/n~Sˉ)(1+O(∣Δ∣))(\widehat{SE}_S^\phi)^2 = \hat{p}(1-\hat{p})(1/\tilde{n}_S + 1/\tilde{n}_{\bar{S}})(1+O(|\Delta|)), because p^S(1−p^S)\hat{p}_S(1-\hat{p}_S) and p^Sˉ(1−p^Sˉ)\hat{p}_{\bar{S}}(1-\hat{p}_{\bar{S}}) differ from p^(1−p^)\hat{p}(1-\hat{p}) by O(∣Δ∣)O(|\Delta|). Under the null and under local alternatives, where disparities are of the order of their standard error, Δ→0\Delta\to0 in probability; this is the range in which the gate operates. There ΛS=ZS2(1+op(1))\Lambda_S = Z_S^2(1+o_p(1)), and for nonzero Δ\Delta Eq. (10) gives SE^eff,S=SE^Sϕ(1+op(1))\widehat{SE}_{\mathrm{eff},S} = \widehat{SE}_S^\phi(1+o_p(1)).

Null law of the soft-count statistic. Fix τ>0\tau> 0 and the predictor independently of the sample. Assume independent observations within and across groups, identically distributed within each group, with a common soft mean p∈(0,1)p \in(0,1) under the null and n~S/N→ρ∈(0,1)\tilde{n}_S/N \to\rho\in(0,1). For fractional counts the Bernoulli form overstates the variance: in each group Var⁡(Qi)=p(1−p)−E[Qi(1−Qi)]\operatorname{Var}(Q_i) = p(1-p) - \mathbb{E}[Q_i(1-Q_i)], and the central limit theorem applied to Δ\Delta in the expansion above gives ΛS⇒cχ12\Lambda_S \Rightarrow c\chi_1^2 with

0≤c=(1−ρ)vS+ρvSˉp(1−p)≤1,vS=Var⁡(Qi∣i∈ΠS),vSˉ=Var⁡(Qi∣i∈ΠSˉ).0 \le c = \frac{(1-\rho)v_S + \rho v_{\bar{S}}}{p(1-p)} \le1,\qquad v_S = \operatorname{Var}(Q_i \mid i \in\Pi_S),\qquad v_{\bar{S}} = \operatorname{Var}(Q_i \mid i \in\Pi_{\bar{S}}).

The sampling standard error of the soft mean contrast is vS/n~S+vSˉ/n~Sˉ\sqrt{v_S/\tilde{n}_S + v_{\bar{S}}/\tilde{n}_{\bar{S}}}; the Bernoulli-form denominator is a proxy for it. For c>0c>0, the reference gate opens under soft parity with asymptotic probability P(χ12>zα2/c)≤αP(\chi_1^2 > z_\alpha^2/c) \le\alpha. The same factor cc scales ZS2Z_S^2, preserving the local equivalence of the two statistics. For c>0c>0, division by a consistent estimate of cc would give asymptotic nominal size under these same fixed-predictor, soft-null assumptions; SAFER retains the Bernoulli reference cutoff.

For exact hard indicators 1{fθ(X)>1/2}1\{f_\theta(X) > 1/2\}, the corresponding deviance instead has Wilks’ χ12\chi_1^2 null limit when the common hard positive rate is interior. To relate this to soft counts as τ↓0\tau\downarrow0, assume that neither conditional score distribution has mass at 1/21/2 and that the limiting hard rates are interior. Bounded convergence then gives Var⁡(Qi)/[(EQi)(1−EQi)]→1\operatorname{Var}(Q_i)/[(\mathbb{E}Q_i)(1-\mathbb{E}Q_i)] \to1 in each group, and hence c→1c \to1 along models satisfying soft parity.

At fixed τ\tau, equal hard positive rates need not give equal soft means, so the null of the gate is soft parity, and certification of hard-decision parity is the task of the held-out hard-count audit of § 3. The fixed-predictor result above applies when the statistic is evaluated on data independent of the fit. During training the sample is also used to fit fθf_\theta, and there the gate is an optimization device that uses the audit’s reference cutoff.

Gradient identities for Section 4.3

Recall from § 4.3 that uS(θ)=ϕ^S(θ)2+ε2−zα sg⁡[SE^eff,S(θ)]u_S(\theta) = \sqrt{\hat{\phi}_S(\theta)^2 + \varepsilon^2} - z_\alpha\,\operatorname{sg}[\widehat{SE}_{\mathrm{eff},S}(\theta)], where sg⁡[⋅]\operatorname{sg}[\cdot] denotes stop-gradient. Since the stop-gradient makes the threshold a constant for differentiation, ∇θuS=∇θϕ^S(θ)2+ε2\nabla_\theta u_S = \nabla_\theta\sqrt{\hat{\phi}_S(\theta)^2 + \varepsilon^2}. By the chain rule and softplus⁡τs′(u)=σ(u/τs)\operatorname{softplus}_{\tau_s}'(u) = \sigma(u/\tau_s),

∂θsoftplus⁡τs(uS(θ))=σ(uS/τs) ∂θuS(θ).\partial_\theta\operatorname{softplus}_{\tau_s}(u_S(\theta)) = \sigma(u_S/\tau_s)\,\partial_\theta u_S(\theta).

Taking norms gives Eq. (12). If uS≤0u_S \le0, then σ(uS/τs)≤1/2\sigma(u_S/\tau_s) \le1/2; if uS<0u_S < 0, then σ(uS/τs)≤exp⁡(uS/τs)\sigma(u_S/\tau_s) \le\exp(u_S/\tau_s).

The dense-tail contribution for subgroup SS is wtailΛS(θ)w_{\mathrm{tail}}\sqrt{\Lambda_S(\theta)}. Differentiating gives wtail∂θΛS(θ)w_{\mathrm{tail}}\partial_\theta\sqrt{\Lambda_S(\theta)}. This expression contains no factor σ(uS/τs)\sigma(u_S/\tau_s), because the LRT statistic is computed from the soft counts directly and does not pass through the significant-excess softplus.

Size of the suppression. With τs=10−2\tau_s = 10^{-2}, the multiplier σ(uS/τs)\sigma(u_S/\tau_s) equals 1/21/2 at the gate boundary uS=0u_S = 0, 0.00670.0067 at uS=−0.05u_S = -0.05, and 0.99330.9933 at uS=0.05u_S = 0.05. The suppression below the boundary is the intended safety mechanism, but it also acts on active subgroups that strong regularization pushes back toward the boundary, so the high-regularization frontier can stall; the dense tail supplies a weak gradient path that does not contain this multiplier.

Why ΛS\sqrt{\Lambda_S}. Three candidates have the required descent direction. An ungated raw-disparity term ∣ϕ^S∣|\widehat{\phi}_S| would be certainty-blind and would penalize noisy disparities regardless of their statistical support. The statistic ΛS\Lambda_S itself would place the tail on a squared evidence scale and produce much steeper precision-dependent gradients. The intermediate choice ΛS\sqrt{\Lambda_S} remains on the same standardized evidence scale as the gate and satisfies ΛS=(∣ϕ^S∣/SE^S)(1+op(1))\sqrt{\Lambda_S} = (|\widehat{\phi}_S|/\widehat{SE}_S)(1 + o_p(1)) near the gate boundary (property (iii) in Section 4.2), so it retains the uncertainty weighting without introducing a new estimator or threshold; its strength is governed by the single weight wtailw_{\mathrm{tail}}.

Effect on the safety guarantee. Because the tail is ungated, Eq. (15) carries an additive residual proportional to wtailw_{\mathrm{tail}}, whose contribution to the complete objective scales as λwtail\lambda w_{\mathrm{tail}}; only the gated component retains the exponentially small gradient guarantee below the gate boundary. Setting wtail=0w_{\mathrm{tail}} = 0 removes the residual and recovers the gated-only objective. All full-SAFER runs use the fixed value wtail=0.003w_{\mathrm{tail}} = 0.003; the NoDenseTail ablation sets it to zero. The proposition bounds the back-propagated fairness contribution at one point. How far training moves from the ERM optimum depends also on the curvature of the prediction loss there: for a scalar quadratic prediction loss with curvature κ\kappa, a constant residual gradient of size rr shifts the minimizer by r/κr/\kappa. The hypothesis uS≤−ϵu_S \le-\epsilon is a margin condition: an empty active set alone gives only the multiplier bound σ(uS/τs)≤1/2\sigma(u_S/\tau_s) \le1/2, and the exponential bound needs ϵ\epsilon large relative to τs\tau_s.

Operating regimes: the gate boundary

The regimes describe the state of the gate at a checkpoint. The implemented boundary is uS=0u_S = 0; in the regular unclamped case with nonzero disparity the active side uS>0u_S > 0 is TS=ΛS>zαT_S = \sqrt{\Lambda_S} > z_\alpha. Softplus makes the transition between the gate states continuous through the multiplier σ(uS/τs)\sigma(u_S/\tau_s) of Eq. (12), whose values at uS=±0.05u_S = \pm0.05 are given above and give a concrete meaning to “well below” and “well above” the boundary. There is no universal sample-size boundary. For EO the local approximation

TS≈∣ϕS∣pS(1−pS)/mS+pˉS(1−pˉS)/mSˉT_S \approx\frac{|\phi_S|}{\sqrt{p_S(1-p_S)/m_S+\bar{p}_S(1-\bar{p}_S)/m_{\bar{S}}}}

shows that certification depends jointly on the disparity magnitude, the outcome rates, and the positive-label counts mS,mSˉm_S,m_{\bar{S}} on both sides; for SP the total subgroup counts nS,nSˉn_S,n_{\bar{S}} take their place.

Stop-gradient and the threshold across updates

Write dS(θ)=ϕ^S(θ)2+ε2d_S(\theta)=\sqrt{\widehat{\phi}_S(\theta)^2+\varepsilon^2} for the smoothed disparity and bS(θ)=zαSE^eff,S(θ)b_S(\theta)=z_\alpha\widehat{SE}_{\mathrm{eff},S}(\theta) for the threshold. At iteration kk, SAFER differentiates ℓS(θ)=softplus⁡τs(dS(θ)−sg⁡[bS(θ)])\ell_S(\theta)=\operatorname{softplus}_{\tau_s}(d_S(\theta)-\operatorname{sg}[b_S(\theta)]), so that

∇θℓS(θk)=σ(dS,k−bS,kτs)∇θdS(θk).\nabla_\theta\ell_S(\theta_k)=\sigma\left(\frac{d_{S,k}-b_{S,k}}{\tau_s}\right)\nabla_\theta d_S(\theta_k).

The threshold bS,kb_{S,k} is held fixed during that backward pass only. After the parameters are updated, bS,k+1=bS(θk+1)b_{S,k+1}=b_S(\theta_{k+1}) is recomputed from the new predictions and can increase or decrease.

A scalar example. Let d(θ)=θd(\theta)=\theta, b(θ)=2θb(\theta)=2\theta and θk=1\theta_k=1, and absorb the positive softplus factor and the learning rate into an effective step of 0.10.1; the example sets the gate factor aside and shows only the direction of the two gradient routes. With stop-gradient, the direct term differentiates only dd, giving θk+1=0.9\theta_{k+1}=0.9; on the next forward pass the threshold is recomputed and changes from 22 to $1.8. Without stop-gradient, the derivative of the excess is d′(θ)−b′(θ)=−1d'(\theta)-b'(\theta)=-1, so descent instead gives θk+1=1.1\theta_{k+1}=1.1: the disparity increases from 11 to $1.1, but the threshold rises faster, from 22 to $2.2, and the excess decreases. Detachment removes this route to a lower penalty.

Threshold drift in practice. We logged the unweighted mean of bSb_S over the subgroups eligible for the EO loss (at least two positive-label examples in both SS and Sˉ\bar{S}: 156 subgroups on Adult and 35 on COMPAS) together with the complete fairness loss. Table A.16 reports means ±\pm standard deviations over 10 Adult and 5 COMPAS runs; “final” is epoch 199 on Adult and each seed’s early-stopped epoch on COMPAS. Near initialization the scores are approximately constant, so disparities and the fairness loss are small, and both grow as the prediction loss learns predictive structure. The mean threshold moves from 0.281 to 0.269 on Adult and from 0.128 to 0.110 on COMPAS. These are averages over subgroups, and individual thresholds can rise; the guarantee itself concerns one update, in which the gated term supplies no gradient through bSb_S.

dataset (runs)mean bSb_S: initial / finalfairness loss: initial / final
Adult (10).281 ±\pm .032 / .269 ±\pm .014.0015 ±\pm .0007 / .0037 ±\pm .0011
COMPAS (5).128 ±\pm .021 / .110 ±\pm .003.0034 ±\pm .0045 / .0205 ±\pm .0023

Table A.16. Mean gate threshold bSb_S over eligible subgroups and complete fairness loss at the start and end of training (mean ±\pm standard deviation over runs).

Proof of Proposition 4

At θERM∗\theta^*_{\mathrm{ERM}}, assume uS(θERM∗)≤−ϵ∗u_S(\theta^*_{\mathrm{ERM}}) \leq-\epsilon^* for all SS. By Eq. (12),

∥∂θsoftplus⁡τs(uS(θERM∗))∥≤e−ϵ∗/τs∥∂θuS(θERM∗)∥.\left\lVert\partial_\theta\operatorname{softplus}_{\tau_s}\left(u_S(\theta^*_{\mathrm{ERM}})\right) \right\rVert\leq e^{-\epsilon^*/\tau_s}\left\lVert\partial_\theta u_S(\theta^*_{\mathrm{ERM}}) \right\rVert.

The bounded-gradient regularity condition in Proposition 4 allows the finite average of ∥∂θuS∥\lVert\partial_\theta u_S\rVert to be absorbed into the constant. Summing over subgroups gives Eq. (14). Adding the dense-tail gradients and applying the triangle inequality gives Eq. (15). □

References

  1. [1]Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna Wallach. A reductions approach to fair classification. In Proceedings of the 35th International Conference on Machine Learning (ICML), volume 80 of PMLR, pages 60–69, 2018.arxiv.org/abs/1803.02453
  2. [2]Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. Machine bias. In Ethics of data and analytics, pages 254–264. Auerbach Publications, 2022.DOI
  3. [3]Ainhize Barrainkua, Paula Gordaliza, Jose A. Lozano, and Novi Quadrianto. Uncertainty matters: Stable conclusions under unstable assessment of fairness results. In Sanjoy Dasgupta, Stephan Mandt, and Yingzhen Li, editors, Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, volume 238 of Proceedings of Machine Learning Research, pages 1198–1206. PMLR, 2024.
  4. [4]Yahav Bechavod and Katrina Ligett. Penalizing unfairness in binary classification. arXiv preprint arXiv:1707.00044, 2017.arxiv.org/abs/1707.00044
  5. [5]Philippe Besse, Eustasio del Barrio, Paula Gordaliza, and Jean-Michel Loubes. Confidence intervals for testing disparate impact in fair learning. arXiv preprint arXiv:1807.06362, 2018.arxiv.org/abs/1807.06362
  6. [6]Maarten Buyl, Marybeth Defrance, and Tij De Bie. fairret: a framework for differentiable fairness regularization terms. In Proceedings of the 12th International Conference on Learning Representations (ICLR), 2024.
  7. [7]John J. Cherian and Emmanuel J. Candès. Statistical inference for fairness auditing. Journal of Machine Learning Research, 25(149):1–49, 2024.arxiv.org/abs/2305.03712
  8. [8]Kimberle Crenshaw. Mapping the margins: Intersectionality, identity politics, and violence against women of color. Stanford Law Review, 43(6):1241–1299, 1991.
  9. [9]Emily Diana, Wesley Gill, Michael Kearns, Krishnaram Kenthapadi, and Aaron Roth. Minimax group fairness: Algorithms and experiments. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES), pages 66–76, 2021.arxiv.org/abs/2011.03108
  10. [10]Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. Retiring adult: New datasets for fair machine learning. Advances in Neural Information Processing Systems, 34, 2021.arxiv.org/abs/2108.04884
  11. [11]Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, ITCS ’12, page 214–226, New York, NY, USA, 2012. Association for Computing Machinery.
  12. [12]Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. Certifying and removing disparate impact. In Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 259–268, 2015.arxiv.org/abs/1412.3756
  13. [13]Antonio Ferrara, Francesco Cozzi, Alan Perotti, André Panisson, and Francesco Bonchi. Size-adaptive hypothesis testing for fairness. In Advances in Neural Information Processing Systems (NeurIPS) 2025, 2025.arxiv.org/abs/2506.10586
  14. [14]James R Foulds, Rashidul Islam, Kamrun Naher Keya, and Shimei Pan. An intersectional definition of fairness. In 2020 IEEE 36th international conference on data engineering (ICDE), pages 1918–1921. IEEE, 2020.
  15. [15]Avijit Ghosh, Lea Genuit, and Mary Reagan. Characterizing intersectional group fairness with worst-case comparisons. In Proceedings of the 2nd Workshop on Diversity in Artificial Intelligence (AIDBEI), volume 142 of PMLR, pages 22–34, 2021.arxiv.org/abs/2101.01673
  16. [16]Usman Gohar and Lu Cheng. A survey on intersectional fairness in machine learning: Notions, mitigation, and challenges. In Edith Elkind, editor, Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23, pages 6619–6627. International Joint Conferences on Artificial Intelligence Organization, 2023. Survey Track.DOI
  17. [17]Moritz Hardt, Eric Price, and Nathan Srebro. Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems (NeurIPS), volume 29, pages 3315–3323, 2016.arxiv.org/abs/1610.02413
  18. [18]Tatsunori B. Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In Proceedings of the 35th International Conference on Machine Learning (ICML), volume 80 of PMLR, pages 1929–1938, 2018.arxiv.org/abs/1806.08010
  19. [19]Úrsula Hébert-Johnson, Michael P. Kim, Omer Reingold, and Guy N. Rothblum. Multicalibration: Calibration for the (Computationally-Identifiable) masses. In Proceedings of the 35th International Conference on Machine Learning (ICML), volume 80 of PMLR, pages 1939–1948, 2018.
  20. [20]Johannes Himmelreich, Arbie Hsu, Ellen Veomett, and Kristian Lum. The intersectionality problem for algorithmic fairness. In Miriam Rateike, Awa Dieng, Jamelle Watson-Daniels, Ferdinando Fioretto, and Golnoosh Farnadi, editors, Proceedings of the Algorithmic Fairness Through the Lens of Metrics and Evaluation, volume 279 of Proceedings of Machine Learning Research, pages 68–95. PMLR, 2025.arxiv.org/abs/2411.02569
  21. [21]Hans Hofmann. Statlog (German Credit Data). UCI Machine Learning Repository, 1994.
  22. [22]Max Hort, Zhenpeng Chen, Jie M. Zhang, Mark Harman, and Federica Sarro. Bias mitigation for machine learning classifiers: A comprehensive survey. ACM Journal on Responsible Computing, 1(2):1–52, 2024.arxiv.org/abs/2207.07068
  23. [23]Rashidul Islam, Kamrun Naher Keya, Shimei Pan, Anand D. Sarwate, and James R. Foulds. Differential fairness: An intersectional framework for fair AI. Entropy, 25(4):660, 2023.DOI
  24. [24]Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. Fairness-aware classifier with prejudice remover regularizer. In Machine Learning and Knowledge Discovery in Databases (ECML PKDD), volume 7524 of Lecture Notes in Computer Science, pages 35–50. Springer, 2012.DOI
  25. [25]Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. Preventing fairness gerrymandering: Auditing and learning for subgroup fairness. In Proceedings of the 35th International Conference on Machine Learning (ICML), volume 80 of PMLR, pages 2564–2572, 2018.arxiv.org/abs/1711.05144
  26. [26]Ron Kohavi. Scaling up the accuracy of naive-bayes classifiers: A decision-tree hybrid. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, pages 202–207, 1996.
  27. [27]Selim Kuzucu, Jiaee Cheong, Hatice Gunes, and Sinan Kalkan. Uncertainty as a fairness measure. J. Artif. Int. Res., 81, 2025.DOI
  28. [28]Firas Laakom, Haobo Chen, Jürgen Schmidhuber, and Yuheng Bu. Fairness overfitting in machine learning: An information-theoretic perspective. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 32078–32115. PMLR, 2025.arxiv.org/abs/2506.07861
  29. [29]Ho Ming Lee, Katrien Antonio, Benjamin Avanzi, Lorenzo Marchi, and Rui Zhou. Machine learning with multitype protected attributes: Intersectional fairness through regularisation. arXiv preprint arXiv:2509.08163, 2025.arxiv.org/abs/2509.08163
  30. [30]Victor Lo, Sayan Datta, and Youssouf Salami. Bringing practical statistical science to AI and predictive model fairness testing. AI and Ethics, 5(3):2149–2164, 2025.
  31. [31]Yan Luo, Muhammad Osama Khan, Yu Tian, Min Shi, Zehao Dou, Tobias Elze, Yi Fang, and Mengyu Wang. Fair Vision: Equitable deep learning for eye disease screening via fair identity scaling. arXiv preprint arXiv:2310.02492, 2023.
  32. [32]Mathieu Molina and Patrick Loiseau. Bounding and approximating intersectional fairness through marginal fairness. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Hook, NY, USA, 2022. Curran Associates Inc.arxiv.org/abs/2206.05828
  33. [33]Jiří Němeček, Mark Kozdoba, Ilia Kryvoviáz, Tomáš Pevný, and Jakub Mareček. Intersectional fairness via mixed-integer optimization. arXiv preprint arXiv:2601.19595, 2026.arxiv.org/abs/2601.19595
  34. [34]Taeеon Park, Sangwon Jung, Sanghyuk Chun, and Taesup Moon. FairDRO: Group fairness regularization via classwise robust optimization. Neural Networks, 182:106891, 2025.DOI
  35. [35]Manish Raghavan and Pauline T. Kim. Limitations of the “four-fifths rule” and statistical parity tests for measuring fairness. Geo. L. Tech. Rev., 8:93, 2024.
  36. [36]Lucas Rosenblatt and R. Teal Witter. Fairlyuncertain: A comprehensive benchmark of uncertainty in algorithmic fairness. arXiv preprint arXiv:2410.02005, 2024.arxiv.org/abs/2410.02005
  37. [37]Jie Tang, Chuanlong Xie, Xianli Zeng, and Lixing Zhu. Empirical likelihood-based fairness auditing: Distribution-free certification and flagging. arXiv preprint arXiv:2601.20269, 2026.arxiv.org/abs/2601.20269
  38. [38]Ke Yang, Joshua R. Loftus, and Julia Stoyanovich. Causal intersectionality and fair ranking. In Symposium on Foundations of Responsible Computing, 2021.arxiv.org/abs/2006.08688
  39. [39]I-Cheng Yeh and Che-hui Lien. The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients. Expert systems with applications, 36(2):2473–2480, 2009.DOI
  40. [40]Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society (AIES), pages 335–340, 2018.

Paper details

vlnc:W_dw78juna
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents