betaFeedback

WorksComputer Science

KINDER: Kernel-based Independence for Fair Representation Learning via Prototype-space Erasure

Abstract

Concept erasure is a prominent approach to achieving fairness in machine learning through the removal of sensitive attributes from learned representations. Prior concept erasure methods typically define debiasing indirectly through the failure of a chosen adversary, probe, or fairness penalty, which makes the target of erasure dependent on a particular decoder family or optimization setup. We introduce KINDER, which directly removes sensitive-attribute information from intermediate representations with respect to a chosen family of prototype functionals, and is compatible with any deep model containing an intermediate feature space, independent of the architecture, task type, or training paradigm. KINDER operates in a random Fourier feature space, where it estimates a sensitive subspace from protected group prototypes and projects representations onto its orthogonal complement. Thus, it forms an explicit representation cleaning mechanism distinct from prior fairness control studies. We conduct extensive experiments across diverse settings and show the effectiveness of KINDER on (i) supervised and self-supervised settings, (ii) classification, regression, and image segmentation tasks, and (iii) diverse data modalities, including visual, textual, and tabular data. Code and trained models are available at: https://github.com/abtinmU/KINDER

Introduction

Extensive prior work in fair representation learning (FRL) has shown effectiveness in promoting fairness by learning intermediate representations that retain task-relevant information while reducing dependence on protected attributes [19, 35, 40, 68]. This can be pursued through adversarial invariance, information-theoretic objectives, Maximum Mean Discrepancy (MMD) regularization, disentanglement, and certified downstream fairness guarantees [15, 18, 31, 41, 59]. While a stream of FRL studies optimize the representations for a specific downstream task [9, 35], numerous FRL methods learn more general representations that are fair [12, 40, 46, 68]. This distinction is important because learned representations are often reused with unknown downstream tasks, prediction heads, modalities, or deployment settings, where a fairness constraint optimized for one task may not transfer to another [41, 57, 59]. To achieve this reusable form of FRL, a substantial body of research focuses on erasing representations that exclude sensitive concepts, such as demographic attributes like gender or race [3, 12, 52]. Such methods are known as concept erasure methods [3, 38] and typically entail balancing two competing objectives: (i) removing concept-related information from the representations to ensure they are no longer predictive of the concept after erasure [51, 52], and (ii) preserving as much information as possible from the original representation space [13, 55, 59, 69].

A prominent family of concept erasure methods relies on projections [3, 47, 50, 51, 52, 56, 58], and they typically aim to identify a subspace associated with the protected concept [50, 51, 52] and remove it from the representation through an explicit projection operator [3, 47, 56, 58] (Figure 1(a)). However, a key limitation is that the target of erasure is typically defined indirectly through restricted recovery families, such as linear probes or classifiers [3, 50], linear or kernelized adversarial games [51, 52], quadratic decoders [47], or other selected decoder families, rather than through independence at the level of the cleaned representation (Observation 1 - O1). A second limitation concerns how the erasable subspace is identified. Existing projection-based methods typically recover the removable directions indirectly, through probe-based optimization [50], minmax or adversarial objectives [14, 51, 52], covariance-based closed forms [3], sufficient or spectral projections [56, 58], or kernel/rate-distortion objectives [2]. This leaves open whether group-conditioned nuisance structure can instead be estimated directly from the representation geometry itself, rather than inferred through the selected decoder family used to test erasure (O2). Moreover, concept erasure methods are often applied as post-hoc transforms on fixed representations (O3) [3, 50, 51, 56]. However, in supervised settings, once the representation is projected, the original task head is no longer matched to the cleaned features, and post-hoc projections can alter the statistical geometry of the feature distribution [28]. This impedes direct comparison to native end-to-end models and can introduce an additional utility bottleneck [13, 69]. Further, realistic fairness settings often require simultaneous control of multiple protected attributes [45, 62], including intersectional or structured subgroup settings where marginal fairness can hide subgroup violations [15, 20, 32, 64], and imbalanced groups [50]. Yet many concept erasure methods are predominantly evaluated in single-attribute regimes [3, 50, 51, 52, 56], thus calling into question whether they remain stable and effective when multiple protected attributes are removed simultaneously [58, 67] (O4).

Figure 1. Overview of prior projection-based concept erasure and KINDER. (a) Prior methods identify an erasable subspace indirectly through decoder-specific objectives. (b) KINDER instead builds protected-group prototypes in a lifted feature space and removes the directions along which those group prototypes differ. (c–d) After applying KINDER on Moji, the representation should contain less protected-group information and smaller group-level distribution differences. This is shown by lower HSIC and average gap values. Significance tests are reported in Appendix E.2.

Based on observations O1-O4, we address three main research questions: Research Question 1 (RQ1): Can a projection-based operator estimate erasable group-conditioned structure directly at the representation level? RQ2: Can projection-based concept erasure be defined at the cleaned-representation level, rather than through the failure of a selected decoder family? RQ3: Can such an operator remain useful across post-hoc and end-to-end training, diverse task types, and single, multiple, intersectional, or imbalanced protected-attribute settings?

To address these gaps, we introduce KINDER (see Figure 1), a modular projection-based concept erasure operator that (i) lifts representations into a prototype space, (ii) estimates group-conditioned nuisance structure from explicit group prototypes, and (iii) removes that structure through an orthogonal projection. Unlike prior projection-based methods whose guarantees are tied to linear probes or minmax games [50, 51], kernelized adversaries [52], closed-form linear or quadratic erasure

[3, 47], sufficient or spectral projections [56, 58], or iterative RKHS functional erasure [1], KINDER defines erasure directly at the level of the cleaned representation itself. This makes KINDER decoder-agnostic and model-agnostic, while preserving the explicit and interpretable structure of projection-based concept erasure.

Our key contributions (KC) are as follows. KC1 (addresses RQ1 and RQ2): We improve projection-based concept erasure generalizability by moving the target of removal from a restricted decoder family to the cleaned representation itself. KC2 (addresses RQ2 and RQ3): We introduce a prototype-space projection mechanism that estimates erasable group-conditioned structure directly from explicit group prototypes in a nonlinear lifted space and makes it easier to recover the shared nuisance structure across multiple, intersectional, or unevenly represented groups. KC3 (addresses RQ3): KINDER supports both end-to-end training and retrofit adaptation within a unified formulation. KC4 (addresses RQ1–RQ2): We provide a population-level theory, summarized in § 4.5, showing that KINDER removes group-specific nuisance perturbations in prototype space and reaches prototype-space independence of the cleaned representation from the protected attributes.

The simplest setting for concept erasure involves categorical concepts, such as the removal of demographic biases related to gender or race in text embeddings [2, 5, 12] and can be achieved by both adversarial training [14, 19, 34, 65] or projection-based methods [2, 5, 14, 21, 50, 51, 52, 56]. Post-hoc methods directly modify the learned representations without altering the parameters of the backbone model [1, 50]. In contrast, fine-tuned approaches require access to the model and enforce erasure by finetuning its parameters [56]. Earlier projection-based methods typically rely on linear projection techniques [5, 21, 50, 51] applied in a post-hoc manner, whereas recent work has attempted to extend them to non-linear settings [2, 14, 52, 56].

Comparative Summary: Table 1 compares KINDER against projection-based concept-erasure methods. KCE extends minimax erasure to a selected kernel family [52]; SAL removes guarded information through spectral covariance directions [56]; LEACE and QLEACE provide closed-form guarantees for linear and quadratic information [3, 47]; and SUP removes information through sufficient projection [58]. Thus, within projection-based concept erasure, existing guarantees are typically tied to restricted decoder families, post-hoc editing regimes, or limited protected-attribute evaluations. KINDER instead proves a more general, decoder-agnostic representation-level guarantee.

Table 1: Prior methods guarantee erasure only for restricted decoder families. KINDER instead proves a representation-level guarantee, R⊥FSR \perp_{\mathcal{F}} S, meaning independence (invariance) up to the degree captured by the chosen prototype space F\mathcal{F}. The all-ff guarantee assumes the affine model (Theorem C.3). Thus, under KINDER, removal works at the level of the cleaned representation itself. Further, KINDER has been evaluated on a wider set of data, tasks and settings. Abbreviations: C: classification. R: regression. S: segmentation. Set.: settings. Inter.: intersectional fairness. E: end-to-end. P: post-hoc. NR: Not Reported. Rep.: Reported. T: Text. Tab: Tabular. V: Visual.

MethodIndependence typeErasure targetSubspace sourceDataTaskSet.Inter.
INLP [50]f(R)⊥Sf(R) \perp S, ∀\forall linear ffLinear probe failureIterative probes; nullspaceTCPNR
RLACE [51]f(R)⊥Sf(R) \perp S, ∀\forall linear ffLinear adversary failureLinear minmax subspaceTCPNR
KCE [52]f(R)⊥Sf(R) \perp S, for some kernel ffKernel adversary failureKernelized minmax subspaceTCPNR
SAL [56]Cov⁡(f(R),g(S))≈0\operatorname{Cov}(f(R),g(S)) \approx0, for top spectral linear pairs (f,g)(f,g)High-covariance directionsCovariance SVD projectionTCPNR
LEACE [3]f(R)⊥Sf(R) \perp S, ∀\forall linear ffLinear classifier failureCovariance projectionTCPNR
SUP [58]f(R)⊥Sf(R) \perp S for some chosen ffUnbiased subspaceSufficient projectionTCPNR
QLEACE [47]f(R)⊥Sf(R) \perp S, ∀\forall quadratic ffQuad. decoder failureClosed-form quad. erasureTCPNR
KINDER (ours)Pop. guarantee (affine model): Theorem: R⊥FS⇒R \perp_{\mathcal{F}} S \Rightarrow Cor.: f(R)⊥FSf(R) \perp_{\mathcal{F}} S ∀\forall measurable ffPrototype-space invariancePrototype SVD projectionT; Tab; VC; R; SE; PRep.

Table 1.

Kernel and Reproducing kernel Hilbert space (RKHS) tools have also been used for fair representation learning and prediction debiasing, including fair subspaces [61], MMD-based objectives [18], and CLIP debiasing [17]. These methods show that kernel machinery is useful for fairness, but focus on statistical tests/objectives or prediction-level debiasing, whereas KINDER uses RKHS/ Random Fourier Features (RFF) prototypes to estimate and project out group-conditioned nuisance directions. Orthogonal to this comparison, recent work studies information-theoretic erasure–utility limits of perfect concept erasure [13]. KINDER does not bypass these limits, but targets a different limitation of projection-based erasure, namely, the decoder-family dependence [3, 47].

Preliminaries and Background

Notation and Problem Setting. Let x∈Xx \in\mathcal{X} be an input, fθf_\theta an encoder, and z=fθ(x)∈Rdz=f_\theta(x)\in\mathbb{R}^{d} its representation; for two-view or multimodal settings, the same notation is applied to each branch. Let S∈{1,…,G}S\in\{1,\ldots,G\} be a discrete sensitive attribute, with s(i)s^{(i)} denoting the protected-group label for sample ii. We use mini-batches {(x(i),s(i))}i=1B\{(x^{(i)},s^{(i)})\}_{i=1}^{B}, and add y(i)y^{(i)} when task supervision is available. For a batch of features Z∈RB×dZ\in\mathbb{R}^{B\times d}, we define the centering matrix HB=IB−1B11⊤H_B=I_B-\frac{1}{B}\mathbf{1}\mathbf{1}^{\top}. We define Zc=HBZZ_c=H_BZ for the column centered version of ZZ.

Reproducing Kernel Hilbert Space (RKHS). A Hilbert space H\mathcal{H} is a vector space with an inner product ⟨⋅,⋅⟩H\langle\cdot,\cdot\rangle_{\mathcal{H}}, which allows geometric notions such as distance and orthogonality to be defined in possibly infinite-dimensional spaces. An RKHS is a Hilbert space of functions where evaluation is continuous and satisfies f(x)=⟨f,k(⋅,x)⟩Hf(x)=\langle f,k(\cdot,x)\rangle_{\mathcal{H}}, where kk is the reproducing kernel. In our setting, bounded measurable kernels kZk_Z and kSk_S induce RKHSs HZ\mathcal{H}_Z and HS\mathcal{H}_S, with feature maps φZ(z)=kZ(z,⋅)\varphi_Z(z)=k_Z(z,\cdot) and φS(s)=kS(s,⋅)\varphi_S(s)=k_S(s,\cdot) for encoder features zz and sensitive attributes ss. This lifts ZZ and SS into Hilbert spaces where distributional differences can be expressed through inner products.

KINDER: Methodology

Intuitively, our KINDER block removes group-specific nuisance structure in the lifted prototype space and maps each encoder output to a “cleaned” representation retaining only the shared component of the representation (see Figure 2 – see Appendix § 10.1 for the full algorithm). We first introduce KINDER for a unimodal setting and extend it for a pair of representations (za,zb)(z_a,z_b) (corresponding to e.g., two views, augmentations, or modalities). KINDER executes three steps:

KINDER block showing unimodal use, projection and cleaning with random Fourier features, EMA group prototypes, singular value decomposition, and multimodal use

Figure 2. The KINDER block. (a) Unimodal use: KINDER is inserted between the encoder and the downstream network to transform embeddings containing sensitive variation into cleaned embeddings used by the task loss. (b) Inside the block, the encoder output ZZ is first lifted into a prototype space through random Fourier features (RFF); group-conditioned prototypes are then maintained by exponential moving averages (EMA) using the sensitive attributes, and their dominant variation is summarized by a singular value decomposition (SVD) to estimate the nuisance subspace NN. The cleaned representation is obtained by projecting onto its orthogonal complement, P⊥=I−PNP_{\perp}=I-P_N, while the removed component corresponds to the estimated group-associated nuisance structure. (c) Multimodal use: the same block is applied independently to each modality branch before fusion, so downstream layers operate on cleaned modality-specific embeddings.

  1. Step 1: Lifting the encoder output into an RKHS space (§ 4.1).

  2. Step 2: Prototype Estimation in the RKHS space (§ 4.2).

  3. Step 3: Representation projection and cleaning in the RKHS space (§ 4.3).

In § 4.5, we provide a theoretical analysis showing that, under the corresponding population-level structural model, the cleaned representation becomes independent of the protected attribute at the prototype-space level, which is certified by zero pairwise MMD across groups.

Step 1: Uplifting the Encoder Features into an RKHS

Kernels on ZZ and SS. On encoder features ZZ, we use the Gaussian RBF kernel with bandwidth γ>0\gamma> 0:

kZ(z,z′)=exp⁡(−∥z−z′∥222γ2)=⟨φZ(z),φZ(z′)⟩HZ,(1)k_Z(z,z') = \exp\left(-\frac{\lVert z-z'\rVert_2^2}{2\gamma^2}\right) = \langle\varphi_Z(z),\varphi_Z(z')\rangle_{\mathcal{H}_Z}, \tag*{(1)}

which is a standard choice because it is universal and characteristic on Rd\mathbb{R}^d, allowing the lifted representation to capture rich geometric differences between group-conditional distributions in the encoder feature space [43, 60]. On the sensitive attribute, we use the Dirac kernel: kS(s,s′)=1{s=s′}k_S(s,s') = 1\{s=s'\}, where φS(s)=es∈RG\varphi_S(s)=e_s\in\mathbb{R}^G.

For discrete sensitive attributes, this kernel coincides with the one-hot embedding and makes the RKHS mean μS=∑sπses\mu_S = \sum_s \pi_s e_s, with πs=Pr⁡(S=s)\pi_s = \Pr(S=s), and the class conditional means E[φZ(z)∣S=s]\mathbb{E}[\varphi_Z(z)\mid S=s] directly interpretable as group prototypes [43]. This also makes group-conditioned prototype statistics directly interpretable in the lifted space, which is the basis for the nuisance-subspace construction used by KINDER.

Random Fourier features for kZk_Z. We approximate kZk_Z with random Fourier features (RFF) [49]. For bandwidth γ>0\gamma> 0 and RFF dimension MM,

Φrff(Z)=2M[cos⁡(ω1⊤Z+b1),…,cos⁡(ωM⊤Z+bM)]⊤∈RM,(2)\Phi_{\mathrm{rff}}(Z)=\sqrt{\frac{2}{M}}\left[\cos(\omega_1^\top Z+b_1),\ldots,\cos(\omega_M^\top Z+b_M)\right]^\top\in\mathbb{R}^M, \tag*{(2)}

with ωj∼iidN(0,γ−2Id)\omega_j \overset{\mathrm{iid}}{\sim}\mathcal{N}(0,\gamma^{-2}I_d) and bj∼iidUnif⁡[0,2π]b_j\overset{\mathrm{iid}}{\sim}\operatorname{Unif}[0,2\pi], where IdI_d is the d×dd\times d identity matrix. For notational simplicity we write Φ\Phi for Φrff\Phi_{\mathrm{rff}} in what follows. All linear operators used in the KINDER block implementation (prototypes, projectors, and cleaned features) act in this lifted space RM\mathbb{R}^M.

Step 2: Prototype Estimator of the Sensitive Subspace via SVD

Let X=Φrff(z)∈RMX=\Phi_{\mathrm{rff}}(z)\in\mathbb{R}^M denote the lifted encoder feature. Rather than fixing the removable structure to a single statistic, KINDER estimates group-associated variation through a chosen prototype family. For an order parameter KK, let FK={ψa(q):q=1,…,K, a∈Aq}\mathcal{F}_K=\{\psi_a^{(q)}:q=1,\ldots,K,\ a\in A_q\} be a finite collection of prototype functionals, where each ψa(q)\psi_a^{(q)} maps lifted features to a vector in RM\mathbb{R}^M. For each group ss, order qq, and prototype index aa, we maintain an EMA prototype

ps,a(q)←(1−ρ)ps,a(q)+ρ⋅1∣Bs∣∑i∈Bsψa(q)(X(i);μs),(3)p_{s,a}^{(q)}\leftarrow(1-\rho)p_{s,a}^{(q)}+\rho\cdot\frac{1}{|\mathcal{B}_s|}\sum_{i\in\mathcal{B}_s}\psi_a^{(q)}\left(X^{(i)};\mu_s\right), \tag*{(3)}
Bs:={i:s(i)=s}.\mathcal{B}_s:=\{i:s^{(i)}=s\}.

Here, ρ∈(0,1]\rho\in(0,1] controls the EMA update rate, and μs=E[X∣S=s]\mu_s=\mathbb{E}[X\mid S=s] is the group-conditional mean used for central higher-order moments; for first-order prototypes, this centering argument is not needed.

For each prototype statistic, we define its empirical global average pˉa(q)=∑s=1Gπ^sps,a(q)\bar{p}_a^{(q)}=\sum_{s=1}^G\hat{\pi}_s p_{s,a}^{(q)}, where π^s\hat{\pi}_s is the empirical group frequency. The order-KK nuisance subspace is then defined as the span of all centered group-prototype differences:

NK:=span⁡{ps,a(q)−pˉa(q):s=1,…,G, q=1,…,K, a∈Aq}⊂RM.(4)\mathcal{N}_K:=\operatorname{span}\left\{p_{s,a}^{(q)}-\bar{p}_a^{(q)}:s=1,\ldots,G,\ q=1,\ldots,K,\ a\in A_q\right\}\subset\mathbb{R}^M. \tag*{(4)}

To estimate this subspace, KINDER processes the prototype orders sequentially. For each order qq, we form a centered prototype matrix P~q\widetilde{P}_q by stacking the rows (ps,a(q)−pˉa(q))⊤(p_{s,a}^{(q)}-\bar{p}_a^{(q)})^\top over all groups ss and prototype indices a∈Aqa\in A_q. For the first order, we directly compute the thin SVD of P~1\widetilde{P}_1. For higher orders, before applying SVD, we remove the directions already selected from lower-order prototypes:

P~q∣<q=P~q(IM−Q<qQ<q⊤),(5)\widetilde{P}_{q\mid<q}=\widetilde{P}_q\left(I_M-Q_{<q}Q_{<q}^\top\right), \tag*{(5)}

where Q<qQ_{<q} is the orthonormal basis selected from orders 1,…,q−11,\ldots,q-1. We then compute

P~q∣<q=U(q)Σ(q)V(q)T,(6)\widetilde{P}_{q\mid<q}=U^{(q)}\Sigma^{(q)}V^{(q)\mathsf{T}}, \tag*{(6)}

and add the leading right singular vectors to the nuisance basis. After processing orders 1,…,K1,\ldots,K, let Q≤KQ_{\le K} denote the resulting basis. We define

PN(K)=Q≤KQ≤KT,P⊥(K)=IM−PN(K).(7)P_N^{(K)}=Q_{\le K}Q_{\le K}^{\mathsf{T}},\qquad P_{\perp}^{(K)}=I_M-P_N^{(K)}. \tag*{(7)}

The selected singular vectors span the retained nuisance subspace, which coincides with NK\mathcal{N}_K when all nonzero directions are retained and otherwise provides a rank-truncated approximation. Higher-order prototypes can contribute additional nuisance directions, while the cleaning operation remains a single orthogonal projection.

For first-order KINDER, group means in the RFF space approximate kernel mean embeddings, so their differences capture nonlinear group-conditioned variation beyond ordinary feature-mean differences. Since linear directions in this lifted space correspond to nonlinear functions of the original encoder features, the SVD projection can remove the selected directions of these discrepancies using linear operations in the lifted space.

Prototype-order instantiations. The simplest instantiation is first-order KINDER, with ψ(1)(X)=X\psi^{(1)}(X)=X. At the population level, the corresponding nuisance subspace is

N1=span⁡{μs−μˉ:s=1,…,G},μˉ=∑s=1Gπsμs.(8)\mathcal{N}_1=\operatorname{span}\{\mu_s-\bar{\mu}:s=1,\ldots,G\},\qquad\bar{\mu}=\sum_{s=1}^{G}\pi_s\mu_s. \tag*{(8)}

The EMA prototypes in Eq. (3) provide the corresponding empirical estimates. Higher-order variants address residual nuisance directions through additional prototype functionals while using the same projection pipeline. Second-order prototypes capture group covariance differences, while orders q≥3q\ge3 use directional moment sketches to avoid constructing full higher-order tensors (Appendix § 10.4). A feature-level ablation over higher-order prototype families is also provided in Appendix § 13.1.

Step 3: Representation Projection and Cleaning

Given a batch of lifted features X=Φrff(Z)∈RB×M\mathbf{X}=\Phi_{\mathrm{rff}}(\mathbf{Z})\in\mathbb{R}^{B\times M}, we define the cleaned representations as

R(K)=XP⊥(K)∈RB×M.(9)\mathbf{R}^{(K)}=\mathbf{X}P_{\perp}^{(K)}\in\mathbb{R}^{B\times M}. \tag*{(9)}

Only R(K)\mathbf{R}^{(K)} is passed to the remainder of the model (e.g., a projection head, classifier, or regressor). Protected labels are used to estimate and update the nuisance projector. Once the projector is fixed, the same cleaning map is applied to all representations without requiring their individual protected labels. For the theoretical analysis, R\mathbf{R} denotes the cleaned representation of a single branch, with any view or modality index suppressed.

Training Objective (Loss)

KINDER is loss-agnostic: after cleaning, the representation is passed to the downstream head or objective without modifying the task loss. For a supervised input xx with target yy, an encoder fθf_\theta first produces z=fθ(x)∈Rdz=f_\theta(x)\in\mathbb{R}^d. KINDER then lifts and cleans this feature,

R=KINDER⁡(z,s)=Φrff(z)P⊥(K)∈RM,(10)R=\operatorname{KINDER}(z,s)=\Phi_{\mathrm{rff}}(z)P_{\perp}^{(K)}\in\mathbb{R}^{M}, \tag*{(10)}

and the downstream head hωh_\omega is trained with the original supervised objective, LKINDER=Ltask(hω(R),y)\mathcal{L}^{\mathrm{KINDER}}=\mathcal{L}_{\mathrm{task}}(h_\omega(R),y). Thus, KINDER can be used with standard classification, regression, or segmentation losses without changing the task objective. For two-view self-supervised settings, the same block is applied to each view or modality before the contrastive objective; explicit uni-/multimodal SSL formulations are provided in Appendix § 10.5.

Theoretical Analysis

In Appendix § 11, we provide rigorous theoretical guarantees for KINDER, beginning with a shared-base affine structural model. This model assumes that a generic representation ZZ is generated by applying group-specific affine transformations to a latent vector WW that is strictly independent of SS (W⊥SW \perp S). Under this model, we identify the canonical affine discrepancy subspace, Naff\mathcal{N}_{\mathrm{aff}}, which captures the precise linear directions along which group-specific affine images of the same latent point diverge. As shown in Theorem C.3, projecting ZZ onto the orthogonal complement of Naff\mathcal{N}_{\mathrm{aff}} yields a cleaned representation R∗R^* that is strictly independent of SS. We prove Naff\mathcal{N}_{\mathrm{aff}} is the minimal linear subspace whose removal equalizes all groups, thereby preserving maximum latent information.

KINDER extends this affine intuition into a reproducing kernel Hilbert space (RKHS) via prototype families. By mapping representations to a Hilbert space RM\mathbb{R}^{M} using Φrff(Z)\Phi_{\mathrm{rff}}(Z), KINDER forms a cleaned representation RR by projecting away the subspace NK\mathcal{N}_{K} spanned by the differences in group prototype means. Theorem C.5 establishes that this projection achieves prototype-level independence, i.e. the cleaned representation is invariant to the protected attribute under the linear kernel of the prototype family (R⊥FSR \perp_{\mathcal{F}} S). Further, Corollary C.6 guarantees that if the prototype functionals are rich enough to identify the relevant group laws in the cleaned representation space, zero MMD implies full probabilistic independence, R⊥SR \perp S. The scope of this independence is determined by the selected prototype family: The projection equalizes its group-conditioned prototype statistics instead of asserting that all possible dependence is removed. Although our experiments use moment-order functionals, the framework also permits richer prototype families, including non-moment functionals, to control a broader class of group-conditioned variation without changing the projection mechanism. Complete proofs for all statements, including the minimality of the discrepancy subspace, are provided in Appendix § 11.

In practice, RFF provides a finite-dimensional approximation of the Gaussian kernel, EMA estimates group-conditioned prototypes, and sequential SVD estimates the corresponding nuisance subspace. Proposition C.7 further bounds the residual prototype discrepancy from rank truncation and projector estimation, while Proposition C.8 quantifies finite-RFF approximation of the Gaussian kernel. These results show the approximation errors beyond the population-level guarantee.

Experiment settings

Datasets used: Supervised classification: Bias in Bios [16] and Moji [70] (for post-hoc evaluation), MIMIC-IV [29], Supervised regression: Communities and Crime [42], MIMIC-IV [29], Supervised segmentation: FairSeg [63], SSL classification: Harvard-FairVLMed [39] (multimodal). CelebA [36] (unimodal).

Fairness metrics: For segmentation, we report worst-group Dice for overlap fairness [44] and HD95 gap for boundary fairness [6]. For text classification, we report worst-group macro-F1 [7], worst-class TPR gap [27, 53], and RMS class TPR gap [50, 51]. For other classification tasks, we use the aggregated fairness score from [10, 11], averaging statistical parity, equal accuracy, equalized odds, and equal opportunity. For regression, we report MAE gap for group error disparity [25]. Further training, dataset, implementation, and details are provided in Appendix § 12. Hyperparameters are selected by validation utility. For KINDER, we fix the Gaussian RBF kernel and moment-based prototype family, while selecting the prototype order by the validation criterion (Appendix § 12.2).

Our goal is to evaluate how much protected-attribute dependence can be removed while preserving performance.

Results

Across Table 2–Table 7, we evaluate whether KINDER remains effective under dataset imbalance (Appendix § 12.1, Table D.8) while preserving utility and improving fairness across single-/multiple-attribute, intersectional, supervised, and self-supervised settings.

Table 2: BIOS & MOJI classification results. Abbreviations: WgF1: worst-group macro-F1. WCTG: worst-class TPR gap. RCTG: RMS class TPR gap. Arrows indicate whether higher or lower values are better. First row is the baseline model. Rows prefixed with “+” apply the corresponding method on top of the baseline. Best values in bold, second-best values underlined for all tables.

MethodBIOSMOJI
Acc ↑F1 ↑WgF1 ↑WCTG ↓RCTG ↓Acc ↑F1 ↑WgF1 ↑WCTG ↓RCTG ↓
BERT80.974.470.747.316.280.174.762.125.617.5
+ INLP80.774.270.345.514.979.874.761.525.317.9
+ RLACE79.873.169.146.814.879.173.761.023.916.9
+ LEACE80.774.370.545.214.579.974.861.126.318.6
+ KCE79.872.568.649.316.579.273.961.423.616.7
+ SUP80.874.471.047.415.979.874.661.823.816.8
+ KINDER81.977.373.941.515.279.277.565.316.714.1

Table 2.

MethodAcc ↑\uparrowRace Gap ↓\downarrowGender Gap ↓\downarrowEthnicity Gap ↓\downarrowInter. Gap ↓\downarrowWG-Inter. ↑\uparrow
FactorCL60.412.84.315.695.80.0
+ INLP60.27.95.610.376.025.0
+ RLACE60.013.74.216.995.90.0
+ LEACE57.34.33.77.384.80.0
+ KCE58.89.15.815.793.50.0
+ SUP60.612.32.317.795.80.0
+ KINDER61.26.93.012.483.741.5
CLIP73.66.24.014.280.741.9
+ INLP70.512.05.413.574.650.0
+ RLACE73.07.42.816.180.741.9
+ LEACE69.36.76.218.087.10.0
+ KCE73.46.34.118.983.644.5
+ SUP73.46.33.112.880.244.9
+ KINDER71.65.71.65.979.150.0

Table 7. FairVLMed multimodal self-supervised learning results on CLIP and FactorCLIP. Accuracy measures performance; gap and worst group intersectional accuracy measures fairness. Arrows indicate the preferred direction.

Single attribute classification results. On BIOS and MOJI (Table 2), which evaluate KINDER in post-hoc settings, KINDER gives the strongest fairness-sensitive classification results. On BIOS, it improves accuracy and worst-group F1 while reducing the worst-class TPR gap. On MOJI, the accuracy is slightly lower than the BERT baseline, but F1 worst-group, F1, and class-level disparity metrics improve by a larger margin.

Intersectional fairness results. A similar pattern appears on MIMIC-IV (Table 4), where we test the same claim in a setting with multiple protected attributes. The joint KINDER variant gives the best overall balance across the table. It preserves the baseline accuracy, improves F1, reduces the race gap, and improves worst-group accuracy. Further multi-attribute ablations of sequential attributes removal over the baseline methods are provided in Appendix § 13.3.

MethodAcc ↑F1 ↑Gender Gap ↓Race Gap ↓Age Gap ↓Inter. Gap ↓WG Inter. ↑
TabM91.939.92.123.22.546.665.8
+ INLP79.543.61.622.15.253.358.3
+ RLACE78.543.02.420.77.153.158.3
+ LEACE78.342.62.321.46.254.458.3
+ KCE80.118.00.446.24.173.635.7
+ KINDER80.644.71.222.57.146.668.7
+ KINDER (NH)92.045.50.715.43.344.868.7

Table 4. MIMIC-IV classification results using TabM. Acc and F1 are performance metrics and the remaining columns are fairness metrics. Arrows indicate the preferred direction. KINDER (NH) uses the native end-to-end task head, while KINDER replaces this head with a linear probe as an ablation comparison with other post-hoc erasure baselines. Post-hoc ablations are reported in Appendix E.3.

In the MIMIC-IV classification results we observe the potential issues that might arise from head mismatch in post-hoc projection methods. When the native KINDER head is replaced with linear probing for comparability with prior erasure baselines, performance drops sharply even though fairness remains competitive. The gap between KINDER (NH) and KINDER suggests that replacing the original task head can impose a substantial utility bottleneck on some datasets. Similar observations were made in

our sequential attribute removal ablation in Appendix E.3. This is the setting in which an end-to-end compatible erasure operator is practically advantageous.

Additional three-seed results. Additional three-seed evaluations on BIOS, MIMIC-IV classification and regression, and FairSeg are reported in Appendix E.5. Among projection-based baselines, we provide additional analysis in Appendix E.7 on SAL [56], as it uses covariance-based spectral erasure with a kernelized extension, which motivated a separate comparison with first-order KINDER. We observe that KINDER’s advantages remain against all these baselines.

Matched end-to-end results. On MIMIC-IV classification, we further compare KINDER (NH) with FR-Train, FairMixup, GroupDRO, and FairBatch using TabM as the backbone (Appendix E.6). KINDER achieves the highest F1 (46.17), compared with 44.66 for FairMixup and 41.72 for TabM. The results show that the advantage of KINDER remains under completely matched end-to-end fairness-specific baselines.

Segmentation results. On FairSeg (Table 3), the setting is different from classification as fairness improvements cannot be attributed only to threshold shifts or changes in class scores and the method must preserve key spatial information as well. KINDER improves the overall Dice and HD95 scores relative to the UNet baseline, while improving worst-group Dice and reducing the largest boundary disparities. The strongest gains appear on the intersectional metrics, where the baseline has the largest gap. This supports that KINDER is not tied to any particular classifier head or task loss.

Table 3: FairSeg segmentation results on UNet. Dice and HD95 measure performance, Dice measures overlap and HD95 measures boundary error. D-WG and H-Gap measure fairness, where D-WG denotes worst-group Dice and H-Gap denotes HD95 gap.

MethodPerf.RaceGenderEthnicityInter.
Dice ↑HD95 ↓D-WG ↑H-Gap ↓D-WG ↑H-Gap ↓D-WG ↑H-Gap ↓D-WG ↑H-Gap ↓
UNet81.46.877.44.181.30.781.31.267.530.7
+ INLP82.65.578.73.082.61.082.60.170.520.0
+ RLACE83.35.180.02.883.20.383.20.273.117.0
+ LEACE83.54.780.31.283.50.683.40.876.66.0
+ SUP83.05.579.13.783.00.882.91.468.730.8
+ KCE83.54.580.21.983.30.383.40.575.76.9
+ KINDER83.74.180.41.683.70.183.70.178.25.4

Table 3.

Regression results. On MIMIC-IV regression (Table 6), KINDER improves ResNet MAE and marginal worst-group errors, with intersectional worst-group MAE as the main exception. Similar observations appear for MLP-PLR in the same table. On Communities and Crime (Appendix Table E.13), KINDER gives the best utility and worst-group error, while remaining close to the strongest race-gap result. This is important as some post-hoc baselines, especially INLP and LEACE, reduce the representation aggressively and lose a large amount of predictive information.

MethodMAE ↓R2R^2 ↑Race WG-MAE ↓Gender WG-MAE ↓Age WG-MAE ↓Inter. WG-MAE ↓
ResNet2.590.153.242.612.785.55
+ INLP2.960.003.753.013.2810.88
+ LEACE2.770.113.252.793.015.91
+ RLACE2.770.113.272.793.025.93
+ KCE2.570.153.232.592.845.10
+ SUP2.770.113.292.793.035.90
+ KINDER2.520.133.072.532.727.01
MLP-PLR2.590.113.352.622.845.65
+ INLP2.670.113.452.722.835.87
+ LEACE2.690.093.402.732.835.25
+ RLACE2.680.093.432.712.895.05
+ KCE2.670.103.502.702.895.29
+ SUP2.680.093.502.722.885.22
+ KINDER2.590.123.302.612.856.83

Table 6. MIMIC-IV regression results on ResNet and MLP-PLR. MAE and R2R^2 measure performance; worst-group MAE columns measure fairness. Arrows indicate the preferred direction. Best values are in bold and second-best values are underlined.

Multimodal SSL results. Looking at FairVLMed (Table 7), on FactorCL, KINDER improves accuracy while reducing all reported gaps and gives the strongest worst-intersectional balanced accuracy. On CLIP, KINDER gives the best race, gender, and ethnicity gaps, and matches the best worst-intersectional balanced accuracy. CelebA (Table 5) shows the same unimodal SSL pattern: KINDER gives the best accuracy, age gap, and worst-intersectional performance, while keeping gender and intersectional gaps close to the baseline.

Table 5: I-JEPA results on CelebA. Acc. and WG-Inter. measure utility and worst-group intersectional performance, where higher is better; gap metrics measure protected-attribute disparity, where lower is better.

MethodAcc. ↑Gender Gap ↓Age Gap ↓Inter. Gap ↓WG-Inter. ↑
I-JEPA91.16.62.711.587.7
+ INLP85.73.48.513.782.7
+ LEACE90.06.35.812.086.3
+ RLACE91.86.12.610.888.5
+ KCE91.87.02.511.488.3
+ SUP91.96.02.711.088.5
+ KINDER92.26.32.311.489.4

Table 5.

Discussion and Conclusion

RQ 1: Representation-level estimation: We show that in KINDER, a projection-based operator is able to estimate erasable group-conditioned structure directly at the representation level by estimating the removable structure from the explicit protected-group prototypes in a lifted prototype space. The sequential SVD then allows higher-order prototypes to contribute residual nuisance directions beyond those identified at lower orders, without relying on a selected probe or adversary to define the erasure target. Our prototype-order ablation (App. § 13.1) further shows that higher orders reduce measured feature dependence and higher-order group discrepancies, while increasing the projection shift.

RQ 2: Generalizable concept erasure: As a result, KINDER is able to achieve generalizable concept erasure by moving the target of removal from a restricted decoder family to the cleaned representation itself. Empirically, KINDER is shown to work across a variety of training paradigms (i.e. during training and post training) and architectures (BERT, IJEPA, ResNet, UNet, CLIP, etc.). Theoretically, our analysis shows that KINDER removes the nuisance directions responsible for protected-group variation and yields independence from the protected attribute at the level identified by the prototype space. Further, under stronger prototype-exactness conditions, our prototype-level guarantee recovers probabilistic independence and thus, it separates the main target of erasure from a particular decoder family while preserving the explicit structure of projection-based concept erasure. Under such conditions, independence is also preserved under any measurable downstream transformation of the cleaned representation.

RQ 3: All round utility across a variety of training paradigms, tasks and settings: KINDER improves the empirical fairness–utility tradeoff under our utility-first selection protocol [13, 69]. During end-to-end training, the encoder can adapt to the projection and retain task-relevant signals through the remaining representation directions, rather than relying on the removed nuisance structure. KINDER achieves strong performance across supervised (e.g., MIMIC-IV dataset, Bios & Moji), self-supervised (e.g., FairVLMed, CelebA), unimodal, and multimodal settings, and consistently improves both fairness and performance (see Figure 3) for a variety of tasks even under multiple intersectional protected attributes. The computational-cost analysis further shows that KINDER maintains competitive runtime and memory requirements compared with other concept erasure methods (Appendix Table D.9). Our findings support KINDER as an all-round practical and theoretically grounded approach to generalizable task-agnostic fair representation learning.

Figure 3. Performance-fairness tradeoff plots across classification, segmentation & regression. Red triangles denote KINDER; blue circles denote baselines and competing post-hoc methods.

Limitations: Our experiments are restricted to datasets that contain sensitive attributes which is the common setting in fairness analysis. We only focus on KINDER’s moment order functionals and not other forms of functionals. Future work can extend the analysis on other datasets and new variations of prototype functionals. We provide an extended discussion of the limitations in Appendix § 14.

Acknowledgments and Disclosure of Funding

Abtin Mogharabin is supported by the Konrad Zuse School of Excellence in Learning and Intelligent Systems (ELIZA) through the DAAD programme Konrad Zuse Schools of Excellence in Artificial Intelligence, sponsored by the Federal Ministry of Education and Research. Jiaee Cheong is supported by the DAAD Fellowship and the Harvard BKC Fellowship. We also gratefully acknowledge the computational resources kindly provided by METU Image Lab and METU-ROMER, Center for Robotics and Artificial Intelligence.

More Related Work

Projection-based concept-erasure methods. INLP relies on iterative linear probes and a joint-nullspace projection for concept erasure [50]. RLACE formulates concept erasure as a linear minimax game where the concept is removed from the learned subspace [51]. KCE kernelizes the linear minimax game and provides erasure against a selected kernelized adversary [52]. LEACE is a closed-form method that prevents all linear classifiers from detecting a concept while changing the embedding as little as possible [3]. SUP obtains debiased representations by projecting them onto the subspace orthogonal to a sufficient span [58]. QLEACE extends LEACE by removing quadratically available information about a concept from a representation [47]. Obliviator targets nonlinear guardedness by minimizing statistical dependence between learned PLM representations and unwanted attributes [1]. These methods motivate Table 1, where the main distinction is whether erasure is defined through a restricted decoder/adversary family or directly at the cleaned-representation level.

Algorithm 1 KINDER for a batch: first-order prototype subspace projection.

Require: Minibatch representations Z∈RB×d\mathbf{Z} \in\mathbb{R}^{B \times d} and protected-group labels s=(s(1),…,s(B))\mathbf{s}=(s^{(1)},\ldots,s^{(B)}), with s(i)∈{1,…,G}s^{(i)} \in\{1,\ldots,G\}

Require: RFF feature map Φrff\Phi_{\mathrm{rff}} defined in Eq. (2)

Require: Removal rank dN≤G−1d_N \le G-1, EMA rate ρ∈(0,1]\rho\in(0,1]

Require: Prototype-subspace refresh interval Trefresh∈NT_{\mathrm{refresh}} \in\mathbb{N} (measured in optimization steps) and current optimization step tstep∈Nt_{\mathrm{step}} \in\mathbb{N}

Require: Empirical group frequencies (π^1,…,π^G)(\hat{\pi}_1,\ldots,\hat{\pi}_G)

Require: Stored mean prototypes {μs∈RM}s=1G\{\mu_s \in\mathbb{R}^{M}\}_{s=1}^{G} and prototype basis UN∈RM×dN\mathbf{U}_N \in\mathbb{R}^{M \times d_N} with orthonormal columns

Ensure: Cleaned representations R∈RB×M\mathbf{R} \in\mathbb{R}^{B \times M}

Step 1: RFF lift and projection

  1. X←Φrff(Z)\mathbf{X} \leftarrow\Phi_{\mathrm{rff}}(\mathbf{Z}) ▷ RFF-lifted representations from Eq. (2)

  2. P⊥←IM−UNUN⊤\mathbf{P}_{\perp} \leftarrow\mathbf{I}_M-\mathbf{U}_N\mathbf{U}_N^{\top} ▷ Projector from Eq. (7)

  3. R←XP⊥\mathbf{R} \leftarrow\mathbf{X}\mathbf{P}_{\perp} ▷ Cleaned representations from Eq. (9)

Step 2: Mean-prototype update

  1. for each protected group s=1,…,Gs=1,\ldots,G do

  2. Bs←{i:s(i)=s}\mathcal{B}_s \leftarrow\{i:s^{(i)}=s\}

  3. if ∣Bs∣>0|\mathcal{B}_s|>0 then

  4. xˉs←1∣Bs∣∑i∈BsXi,:\bar{\mathbf{x}}_s \leftarrow\frac{1}{|\mathcal{B}_s|}\sum_{i\in\mathcal{B}_s}\mathbf{X}_{i,:} ▷ Batch group mean used in Eq. (3)

  5. μs←(1−ρ)μs+ρxˉs\mu_s \leftarrow(1-\rho)\mu_s+\rho\bar{\mathbf{x}}_s ▷ EMA update from Eq. (3)

  6. end if

  7. end for

Step 3: SVD refresh of the prototype subspace

  1. if tstepmod⁡Trefresh=0t_{\mathrm{step}}\mathbin{\operatorname{mod}}T_{\mathrm{refresh}}=0 then

  2. μˉ←∑s=1Gπ^sμs\bar{\mu}\leftarrow\sum_{s=1}^{G}\hat{\pi}_s\mu_s ▷ Global mean for Eq. (4)

  3. Form P~∈RG×M\tilde{\mathbf{P}}\in\mathbb{R}^{G\times M} with rows P~s,:←(μs−μˉ)⊤\tilde{\mathbf{P}}_{s,:}\leftarrow(\mu_s-\bar{\mu})^{\top} ▷ Top right singular vectors from Eq. (6)

  4. UN←TopRightSingVecs⁡(P~,dN)\mathbf{U}_N\leftarrow\operatorname{TopRightSingVecs}(\tilde{\mathbf{P}},d_N) ▷ Centered prototype matrix for Eq. (6)

  5. end if

  6. return R\mathbf{R}

Optimization step: compute the chosen objective L(R)\mathcal{L}(\mathbf{R}) and backpropagate through the RFF map Φ\Phi and the linear cleaning map X↦XP⊥\mathbf{X}\mapsto\mathbf{X}\mathbf{P}_{\perp}.

Further KINDER Details

KINDER Algorithm

We present the full algorithm in Algorithm 1.

Supervised vs. Semi-supervised Implementation

In supervised settings, the same cleaned representations RmR_m can be fed to hh and trained with a standard loss (e.g., cross-entropy or regression loss) without modifying the KINDER block.

We use a two-stage protocol on semi-supervised settings. In Stage 1 (self-supervised pretraining), we train on all available samples using either the unimodal objective LKINDERuni-ssl\mathcal{L}_{\mathrm{KINDER}}^{\mathrm{uni\text{-}ssl}} or the multimodal objective LKINDERmm-ssl\mathcal{L}_{\mathrm{KINDER}}^{\mathrm{mm\text{-}ssl}}, while maintaining prototypes indexed only by ss in Eq. (3). In Stage 2 (supervised fine-tuning), we train a task head on the labeled subset using the supervised objective LKINDERsup\mathcal{L}_{\mathrm{KINDER}}^{\mathrm{sup}}. When discrete task labels are available, Stage 2 also admits a label-aware refinement in which prototypes are indexed by (s,y)(s,y), i.e., by replacing s(i)s^{(i)} with (s(i),y(i))(s^{(i)},y^{(i)}) in Eq. (3). This changes only the prototypes used to form P⊥P_{\perp} and leaves the training objective unchanged. The resulting label-aware version conditions the removable nuisance structure on the task labels, reducing the risk of removing task-relevant variation in supervised classification settings.

Specific Block Formulation

Unimodal version. For a mini-batch of encoder outputs Z=[z(1),…,z(B)]⊤\mathbf{Z} = [\mathbf{z}^{(1)}, \ldots, \mathbf{z}^{(B)}]^{\top} with labels s=(s(1),…,s(B))\mathbf{s} = (s^{(1)}, \ldots, s^{(B)}), we define the first-order KINDER block operator:

KINDER⁡(Z,s):=Φrff(Z)P⊥=Φrff(Z)(IM−U(N)U(N)⊤).(11)\operatorname{KINDER}(\mathbf{Z}, \mathbf{s}) := \Phi_{\mathrm{rff}}(\mathbf{Z})P_{\perp} = \Phi_{\mathrm{rff}}(\mathbf{Z})\left(I_{M} - U^{(N)}U^{(N)\top}\right). \tag*{(11)}

where U(N)=U({μs}s=1G)U^{(N)} = \mathcal{U}(\{\mu_s\}_{s=1}^{G}) denotes the top dNd_N right singular vectors obtained from the SVD in Eq. (6). The map U\mathcal{U} is defined as

U:(RM)G→RM×dN\mathcal{U} : (\mathbb{R}^{M})^{G} \to\mathbb{R}^{M \times d_N}
{μs}s=1G↦Q≤1.(12)\{\mu_s\}_{s=1}^{G} \mapsto Q_{\leq1}. \tag*{(12)}

Multimodal version. The KINDER block could also be defined for multimodal methods that benefit from a pair of views (e.g., contrastive learning methods). For the two-view case, we define each view as m∈{m1,m2}m \in\{m_1,m_2\}. Now, we can define the multi-view version of KINDER block as

KINDER⁡(Zm1,Zm2,s):=(KINDER⁡m1(Zm1,s),\operatorname{KINDER}(\mathbf{Z}_{m_1}, \mathbf{Z}_{m_2}, \mathbf{s}) := (\operatorname{KINDER}_{m_1}(\mathbf{Z}_{m_1},\mathbf{s}),
KINDER⁡m2(Zm2,s)).(13)\operatorname{KINDER}_{m_2}(\mathbf{Z}_{m_2},\mathbf{s})). \tag*{(13)}

Higher-order prototype instantiations

Section 4.2 defines KINDER for a general order-KK prototype family and introduces the first-order instantiation. Here we detail the exact second-order construction, which extends the nuisance subspace to group-conditional covariance differences in the lifted feature space.

A stronger exact second-order version additionally maintains covariance prototypes

Σs=E[(X−μs)(X−μs)⊤∣S=s],Σˉ=∑s=1GπsΣs.\Sigma_s = \mathbb{E}\left[(X-\mu_s)(X-\mu_s)^{\top} \mid S=s\right], \qquad\bar{\Sigma} = \sum_{s=1}^{G}\pi_s\Sigma_s.

The corresponding nuisance subspace is

N2exact=span⁡{μs−μˉ:s=1,…,G}+∑s=1GRange⁡(Σs−Σˉ).\mathcal{N}^{\mathrm{exact}}_2 = \operatorname{span}\{\mu_s-\bar{\mu}:s=1,\ldots,G\} + \sum_{s=1}^{G}\operatorname{Range}(\Sigma_s-\bar{\Sigma}).

Let P⊥(2)P_{\perp}^{(2)} be the orthogonal projector onto the complement of N2exact\mathcal{N}^{\mathrm{exact}}_2. By construction, for every pair of groups s,ts,t,

P⊥(2)(μs−μt)=0,P⊥(2)(Σs−Σt)P⊥(2)=0.P_{\perp}^{(2)}(\mu_s-\mu_t)=0,\qquad P_{\perp}^{(2)}(\Sigma_s-\Sigma_t)P_{\perp}^{(2)}=0.

Consequently, the cleaned representations R(2)=P⊥(2)XR^{(2)}=P_{\perp}^{(2)}X have identical group-conditional means and covariance matrices. This extends the first-order construction from mean alignment to exact first- and second-moment alignment in the cleaned RFF space.

For orders q≥3q \geq3, forming full central moment tensors is generally impractical. We therefore use sketched higher-order prototypes based on random unit directions uq,a∈RMu_{q,a} \in\mathbb{R}^{M}:

ψa(q)(X;μs)=(X−μs)(uq,a⊤(X−μs))q−1,q=3,…,K.\psi_{a}^{(q)}(X;\mu_s) = (X-\mu_s)\left(u_{q,a}^{\top}(X-\mu_s)\right)^{q-1}, \qquad q=3,\ldots,K.

These functionals provide tractable directional contractions of the group central moment tensors while keeping the prototype vectors in RM\mathbb{R}^{M}. In implementation, the current group mini-batch mean is used for centering, and the resulting prototypes are maintained by EMA and processed through the same sequential projection framework. In the reported results, prototype order is selected by validation over orders 1,…,51,\ldots,5, using the task-specific validation metric described in Appendix § 12.2.

Two-view self-supervised loss formulations

Let xax_a and xbx_b denote two views of the same instance, corresponding either to two augmentations of one modality or to two paired modalities. Let faf_a and fbf_b be the encoders, producing za=fa(xa)z_a=f_a(x_a) and zb=fb(xb)z_b = f_b(x_b). For group labels s=(s(1),…,s(B))s = (s^{(1)}, \ldots, s^{(B)}), the KINDER block maps each encoder output to cleaned lifted representations,

(Ra,Rb)=KINDER⁡(Za,Zb,s),Ra,Rb∈RB×M.(\mathbf{R}_a,\mathbf{R}_b) = \operatorname{KINDER}(\mathbf{Z}_a,\mathbf{Z}_b,s), \qquad\mathbf{R}_a,\mathbf{R}_b \in\mathbb{R}^{B \times M}.

Any two-view contrastive objective can then be applied without modification:

Luni-sslKINDER⁡=Lctr(Ra,Rb),Lmm-sslKINDER⁡=Lctr(Rm1,Rm2).\mathcal{L}^{\operatorname{KINDER}}_{\mathrm{uni\text{-}ssl}} = \mathcal{L}_{\mathrm{ctr}}(\mathbf{R}_a,\mathbf{R}_b), \qquad\mathcal{L}^{\operatorname{KINDER}}_{\mathrm{mm\text{-}ssl}} = \mathcal{L}_{\mathrm{ctr}}(\mathbf{R}_{m_1},\mathbf{R}_{m_2}).

Detailed Theoretical Results

Let κ\kappa be a bounded measurable kernel on the support of a cleaned representation RR. For probability measures P,QP,Q on that space, let μPκ\mu^\kappa_P denote the corresponding kernel mean embedding, and write

MMD⁡κ(P,Q):=∥μPκ−μQκ∥Hκ.\operatorname{MMD}_{\kappa}(P,Q) := \lVert\mu^\kappa_P - \mu^\kappa_Q \rVert_{\mathcal{H}_{\kappa}}.

When κ\kappa is characteristic, MMD⁡κ(P,Q)=0\operatorname{MMD}_{\kappa}(P,Q)=0 is equivalent to P=QP=Q [24, 43, 60].

Definition C.1 (Shared-base affine structural model). Take a generic encoder representation Z∈RdZ \in\mathbb{R}^{d}, a discrete protected attribute S∈{1,…,G}S \in\{1,\ldots,G\} with πg:=Pr⁡(S=g)>0\pi_g := \Pr(S=g)>0 for every gg, and a latent random vector W∈RdW \in\mathbb{R}^{d} satisfying

W⊥S.W \perp S.

For each group g∈{1,…,G}g \in\{1,\ldots,G\}, let Ag∈Rd×dA_g \in\mathbb{R}^{d \times d} and bg∈Rdb_g \in\mathbb{R}^{d}, and write

Z=ASW+bSalmost surely.Z = A_S W + b_S \qquad\text{almost surely}.

Let

H:=span⁡(supp⁡(W))⊆Rd\mathcal{H} := \operatorname{span}(\operatorname{supp}(W)) \subseteq\mathbb{R}^{d}

denote the linear span of the support of WW.

Definition C.2 (Canonical affine discrepancy subspace). In the setting of Definition C.1, define

Naff:=span⁡({bg−bh:g,h∈{1,…,G}}∪⋃g,h∈{1,…,G}(Ag−Ah)H).\mathcal{N}_{\mathrm{aff}} := \operatorname{span}\left(\{b_g-b_h : g,h \in\{1,\ldots,G\}\} \cup\bigcup_{g,h \in\{1,\ldots,G\}}(A_g-A_h)\mathcal{H}\right).

Equivalently,

Naff=span⁡{(Ag−Ah)w+(bg−bh):g,h∈{1,…,G}, w∈H}.\mathcal{N}_{\mathrm{aff}} = \operatorname{span}\{(A_g-A_h)w+(b_g-b_h):g,h \in\{1,\ldots,G\},\ w \in\mathcal{H}\}.

Let Paff,⊥:Rd→Naff⊥P_{\mathrm{aff},\perp}:\mathbb{R}^{d}\to\mathcal{N}_{\mathrm{aff}}^{\perp} denote the orthogonal projector onto Naff⊥\mathcal{N}_{\mathrm{aff}}^{\perp}, and define

R⋆:=Paff,⊥Z.R^\star:= P_{\mathrm{aff},\perp}Z.

Definition C.2 collects exactly the linear directions along which two group-specific affine images of the same latent point can differ.

Theorem C.3 (Canonical affine projection theorem). In the setting of Definition C.1, there exists a measurable map

T:H→Naff⊥T:\mathcal{H}\to\mathcal{N}_{\mathrm{aff}}^{\perp}

such that

R⋆=T(W)almost surely.R^\star= T(W) \qquad\text{almost surely}.

Consequently,

R⋆⊥S.R^\star\perp S.

Moreover, if M⊆RdM \subseteq\mathbb{R}^{d} is a linear subspace satisfying

PM⊥(Agw+bg)=PM⊥(Ahw+bh)for every g,h∈{1,…,G}, w∈H,P_{M^\perp}(A_gw+b_g)=P_{M^\perp}(A_hw+b_h)\qquad\text{for every }g,h\in\{1,\ldots,G\},\ w\in\mathcal{H},

then

Naff⊆M.\mathcal{N}_{\mathrm{aff}}\subseteq M.

Hence Naff\mathcal{N}_{\mathrm{aff}} is the smallest linear subspace whose removal equalizes all groups under the affine structural model.

Proof. For every g,h∈{1,…,G}g,h \in\{1,\ldots,G\} and w∈Hw \in\mathcal{H},

(Agw+bg)−(Ahw+bh)=(Ag−Ah)w+(bg−bh)∈Naff.(A_gw+b_g)-(A_hw+b_h)=(A_g-A_h)w+(b_g-b_h)\in\mathcal{N}_{\mathrm{aff}}.

Therefore

Paff,⊥(Agw+bg)=Paff,⊥(Ahw+bh).P_{\mathrm{aff},\perp}(A_gw+b_g)=P_{\mathrm{aff},\perp}(A_hw+b_h).

Fix a reference group g0∈{1,…,G}g_0 \in\{1,\ldots,G\}, and define

T(w):=Paff,⊥(Ag0w+bg0),w∈H.T(w):=P_{\mathrm{aff},\perp}(A_{g_0}w+b_{g_0}),\qquad w\in\mathcal{H}.

The preceding identity shows that this quantity does not depend on the chosen group after projection, so

Paff,⊥(Agw+bg)=T(w)for every g∈{1,…,G}, w∈H.P_{\mathrm{aff},\perp}(A_gw+b_g)=T(w)\qquad\text{for every }g\in\{1,\ldots,G\},\ w\in\mathcal{H}.

Since W∈HW\in\mathcal{H} almost surely and Z=ASW+bSZ=A_SW+b_S almost surely,

R⋆=Paff,⊥Z=Paff,⊥(ASW+bS)=T(W)almost surely.R^\star=P_{\mathrm{aff},\perp}Z=P_{\mathrm{aff},\perp}(A_SW+b_S)=T(W)\qquad\text{almost surely}.

Because W⊥SW\perp S, it follows that

R⋆⊥S.R^\star\perp S.

For minimality, let M⊆Rd\mathcal{M}\subseteq\mathbb{R}^d satisfy the stated equalization property. Then

PM⊥((Ag−Ah)w+(bg−bh))=0for every g,h∈{1,…,G}, w∈H,P_{\mathcal{M}^{\perp}}\bigl((A_g-A_h)w+(b_g-b_h)\bigr)=0\qquad\text{for every }g,h\in\{1,\ldots,G\},\ w\in\mathcal{H},

hence

(Ag−Ah)w+(bg−bh)∈Mfor every g,h,w.(A_g-A_h)w+(b_g-b_h)\in\mathcal{M}\qquad\text{for every }g,h,w.

Taking w=0w=0 gives bg−bh∈Mb_g-b_h\in\mathcal{M}. Subtracting those vectors then gives

(Ag−Ah)w∈Mfor every g,h∈{1,…,G}, w∈H.(A_g-A_h)w\in\mathcal{M}\qquad\text{for every }g,h\in\{1,\ldots,G\},\ w\in\mathcal{H}.

Thus M\mathcal{M} contains all generators of Naff\mathcal{N}_{\mathrm{aff}}, so

Naff⊆M.\mathcal{N}_{\mathrm{aff}}\subseteq\mathcal{M}.

Theorem C.3 establishes the geometric basis for removing group-dependent directions under the shared-base affine model. We next analyze the prototype-based projection used by KINDER, directly through the group-conditioned statistics defined in Section 4.2.

Definition C.4 (KINDER prototype-space invariance). Fix the order-KK prototype family FK\mathcal{F}_K from Section 4.2, and let X=Φrff(Z)∈RMX=\Phi_{\mathrm{rff}}(Z)\in\mathbb{R}^M. For the population analysis, form NK\mathcal{N}_K as in Eq. (4), using population conditional expectations in place of EMA prototypes and population group frequencies in place of empirical frequencies. Let P⊥(K)P_{\perp}^{(K)} project onto NK⊥\mathcal{N}_K^\perp, with all nonzero nuisance directions retained, and set

R=P⊥(K)X.R=P_{\perp}^{(K)}X.

For each selected order qq, index a∈Aqa\in\mathcal{A}_q, and protected group ss, define the population prototype

ms,a(q):=E[ψa(q)(X;μs)∣S=s].m_{s,a}^{(q)}:=\mathbb{E}[\psi_a^{(q)}(X;\mu_s)\mid S=s].

We write R⊥FKSR\perp_{\mathcal{F}_K}S when the selected projected prototypes agree across groups:

P⊥(K)mg,a(q)=P⊥(K)mh,a(q)for every g,h, q=1,…,K, a∈Aq.P_{\perp}^{(K)}m_{g,a}^{(q)}=P_{\perp}^{(K)}m_{h,a}^{(q)} \qquad\text{for every }g,h,\ q=1,\ldots,K,\ a\in\mathcal{A}_q.

Equivalently, each selected pair of projected prototype distributions has zero MMD under the linear kernel ℓ(u,v)=u⊤v\ell(u,v)=u^\top v on RM\mathbb{R}^M. For first-order prototypes, this is zero pairwise linear-kernel MMD between the group-conditional laws of RR; higher orders control their corresponding selected projected prototype statistics.

Theorem C.5 (KINDER prototype-space independence theorem). In the setting of Definition C.4,

R⊥FKS.R\perp_{\mathcal{F}_K}S.

Proof. For every selected qq, aa, the population prototype differences ms,a(q)−∑tπtmt,a(q)m^{(q)}_{s,a}-\sum_t \pi_t m^{(q)}_{t,a} belong to NK\mathcal{N}_K by construction. Hence their projections vanish, and

P⊥(K)mg,a(q)=P⊥(K)mh,a(q)for all g,h,q,a.P_{\perp}^{(K)}m^{(q)}_{g,a}=P_{\perp}^{(K)}m^{(q)}_{h,a}\qquad\text{for all }g,h,q,a.

These quantities are the conditional means of the projected prototype vectors P⊥(K)ψa(q)(X;μs)P_{\perp}^{(K)}\psi^{(q)}_a(X;\mu_s). Their equality gives zero pairwise MMD under the linear kernel ℓ\ell for every selected prototype, proving R⊥FKSR\perp_{F_K}S. □\square

Theorem C.5 establishes the exact population prototype-space guarantee for the selected family. Its first-order case matches group-conditional means of the cleaned representation. The exact second-order covariance construction is detailed in Appendix § 10.4.

Corollary C.6 (Exact independence under law-identifying prototypes). In the setting of Theorem C.5, let PFK\mathcal{P}_{F_K} be a class of laws on the lifted feature space containing each PX∣S=sP_{X\mid S=s}. Suppose the selected projected prototypes identify the cleaned laws on this class. Meaning, for any P,Q∈PFKP,Q\in\mathcal{P}_{F_K} with μP=EX∼P[X]\mu_P=\mathbb{E}_{X\sim P}[X] and μQ=EX∼Q[X]\mu_Q=\mathbb{E}_{X\sim Q}[X], assume that

P⊥(K)EX∼P[ψa(q)(X;μP)]=P⊥(K)EX∼Q[ψa(q)(X;μQ)]for all q,aP_{\perp}^{(K)}\mathbb{E}_{X\sim P}[\psi^{(q)}_a(X;\mu_P)]=P_{\perp}^{(K)}\mathbb{E}_{X\sim Q}[\psi^{(q)}_a(X;\mu_Q)]\qquad\text{for all }q,a

implies equality of the laws of P⊥(K)XP_{\perp}^{(K)}X under PP and QQ. Then

R⊥S.R\perp S.

Proof. Theorem C.5 equalizes all selected projected prototypes of the group-conditional laws PX∣S=gP_{X\mid S=g} and PX∣S=hP_{X\mid S=h}. The law-identifying property therefore makes the corresponding cleaned laws PR∣S=gP_{R\mid S=g} and PR∣S=hP_{R\mid S=h} equal for every pair g,hg,h. Since SS is discrete, these equal conditional laws give R⊥SR\perp S. □\square

Theorem C.5 describes the population projector for a fixed RFF map. We now quantify the effect of estimating its nuisance subspace and retaining a finite number of singular directions.

For each order qq, let P~q\widetilde{P}_q be the population centered prototype matrix from § 4.2, formed from the conditional means ms,a(q)m^{(q)}_{s,a} in Definition C.4. Let P~^q\widehat{\widetilde{P}}_q be its EMA-based estimate, including the empirical group frequencies. Write Π≤q\Pi_{\leq q} and Π^≤q\widehat{\Pi}_{\leq q} for the cumulative population and empirical nuisance projectors constructed by the sequential SVD in Eqs. (6)–(7), with Π≤0=Π^≤0=0\Pi_{\leq0}=\widehat{\Pi}_{\leq0}=0.

Proposition C.7 (Finite-sample prototype-space discrepancy). At each order qq, let

Aq=P~q(IM−Π<q),A^q=P~^q(IM−Π^<q),A_q=\widetilde{P}_q(I_M-\Pi_{<q}),\qquad\widehat{A}_q=\widehat{\widetilde{P}}_q(I_M-\widehat{\Pi}_{<q}),

where Π<q=Π≤q−1\Pi_{<q}=\Pi_{\leq q-1} and similarly for its estimate. Suppose rqr_q nonzero right-singular directions are selected at order qq. Define

ϵq=∥P~^q−P~q∥op,Δq=σrq(Aq)−σrq+1(Aq).\epsilon_q=\left\|\widehat{\widetilde{P}}_q-\widetilde{P}_q\right\|_{\mathrm{op}},\qquad\Delta_q=\sigma_{r_q}(A_q)-\sigma_{r_q+1}(A_q).

Set b0=0b_0=0 and, recursively,

ζq=ϵq+∥P~^q∥opbq−1,bq=bq−1+4ζqΔq−ζq.\zeta_q=\epsilon_q+\left\|\widehat{\widetilde{P}}_q\right\|_{\mathrm{op}}b_{q-1},\qquad b_q=b_{q-1}+\frac{4\zeta_q}{\Delta_q-\zeta_q}.

If Δq>2ζq\Delta_q>2\zeta_q for every selected order, then

∥Π^≤K−Π≤K∥op≤bK.\left\|\widehat{\Pi}_{\leq K}-\Pi_{\leq K}\right\|_{\mathrm{op}}\leq b_K.

Consequently, for every group pair g,hg,h and selected prototype q,aq,a, writing dg,h(q,a)=mg,a(q)−mh,a(q)d^{(q,a)}_{g,h}=m^{(q)}_{g,a}-m^{(q)}_{h,a},

∥(IM−Π^≤K)dg,h(q,a)∥2≤∥(IM−Π≤K)dg,h(q,a)∥2⏟rank-truncation residual\left\|(I_M-\widehat{\Pi}_{\leq K})d^{(q,a)}_{g,h}\right\|_2 \leq \underbrace{\left\|(I_M-\Pi_{\leq K})d^{(q,a)}_{g,h}\right\|_2}_{\text{rank-truncation residual}}
+bK∥dg,h(q,a)∥2.(14){}+b_K\left\|d^{(q,a)}_{g,h}\right\|_2. \tag*{(14)}

The left-hand side is the pairwise linear-kernel MMD between the corresponding projected group-conditional prototype distributions. When all nonzero nuisance directions are retained, the first term vanishes.

Proof. Let Πq=Π≤q−Π<q\Pi_q = \Pi_{\leq q} - \Pi_{<q} be the projector onto the new directions selected at order qq, and define Π^q\widehat{\Pi}_q analogously. The residualized matrices satisfy

∥A^q−Aq∥op≤ϵq+∥P~^q∥op∥Π^<q−Π<q∥op.\lVert\widehat{A}_q - A_q\rVert_{\mathrm{op}} \leq\epsilon_q + \lVert\widehat{\widetilde{P}}_q\rVert_{\mathrm{op}}\lVert\widehat{\Pi}_{<q} - \Pi_{<q}\rVert_{\mathrm{op}}.

By induction, this difference is at most ζq\zeta_q. Wedin’s singular-subspace perturbation inequality and Δq>2ζq\Delta_q > 2\zeta_q give the conservative estimate

∥Π^q−Πq∥op≤4ζqΔq−ζq.\lVert\widehat{\Pi}_q - \Pi_q\rVert_{\mathrm{op}} \leq\frac{4\zeta_q}{\Delta_q - \zeta_q}.

Because the selected directions are orthogonal across orders, Π≤q=∑j=1qΠj\Pi_{\leq q} = \sum_{j=1}^{q} \Pi_j and Π^≤q=∑j=1qΠ^j\widehat{\Pi}_{\leq q} = \sum_{j=1}^{q} \widehat{\Pi}_j. The triangle inequality therefore yields ∥Π^≤q−Π≤q∥op≤bq\lVert\widehat{\Pi}_{\leq q}-\Pi_{\leq q}\rVert_{\mathrm{op}} \leq b_q. Applying it again to (IM−Π^≤K)dg,h(q,a)(I_M-\widehat{\Pi}_{\leq K})d^{(q,a)}_{g,h} proves Eq. (C.14). For the linear kernel, MMD is the norm of the conditional mean difference. If all nonzero singular directions are selected at every order, the cumulative population subspace is NK\mathcal{N}_K, which contains each dg,h(q,a)d^{(q,a)}_{g,h}.

The perturbations ϵq\epsilon_q can be related to group sample sizes. Suppose all selected prototype vectors have norm at most BB, and put

e=max⁡s,q,a∥ps,a(q)−ms,a(q)∥2,ν=∑s=1G∣π^s−πs∣.e = \max_{s,q,a}\lVert p^{(q)}_{s,a} - m^{(q)}_{s,a}\rVert_2,\qquad\nu= \sum_{s=1}^{G}\lvert\widehat{\pi}_s-\pi_s\rvert.

The weighted centering step gives, for each order,

ϵq≤G∣Aq∣(2e+Bν).(15)\epsilon_q \leq\sqrt{G\lvert A_q\rvert}(2e+B\nu). \tag*{(15)}

For a fixed encoder and RFF map, suppose each group provides at least nmin⁡n_{\min} independent observations. If the EMA and evolving-feature tracking error relative to the corresponding empirical prototype means is at most β\beta, bounded-vector concentration gives, with probability at least 1−α1-\alpha,

e≤β+4Blog⁡(2J/α)nmin⁡,J=G∑q=1K∣Aq∣.(16)e \leq\beta+ 4B\sqrt{\frac{\log(2J/\alpha)}{n_{\min}}},\qquad J = G\sum_{q=1}^{K}\lvert A_q\rvert. \tag*{(16)}

Thus, subgroup size controls the sampling term, while β\beta and ν\nu account for EMA tracking and empirical group-frequency error.

Random Fourier feature approximation

The preceding result is conditional on the realized finite RFF map. The following bound quantifies its approximation of the Gaussian kernel used in § 4.1.

Proposition C.8 (Gaussian-to-RFF kernel approximation). Let kZk_Z be the Gaussian kernel defined in § 4.1, and define

kM(z,z′)=⟨Φrff(z),Φrff(z′)⟩.k_M(z,z') = \langle\Phi_{\mathrm{rff}}(z),\Phi_{\mathrm{rff}}(z')\rangle.

On a compact domain Z⊂Rd\mathcal{Z} \subset\mathbb{R}^d, put

ϵM=sup⁡z,z′∈Z∣kM(z,z′)−kZ(z,z′)∣.\epsilon_M = \sup_{z,z'\in\mathcal{Z}}\lvert k_M(z,z')-k_Z(z,z')\rvert.

For any probability distributions P,QP,Q supported on Z\mathcal{Z},

∣MMD⁡kM2(P,Q)−MMD⁡kZ2(P,Q)∣≤4ϵM.(17)\left|\operatorname{MMD}_{k_M}^{2}(P,Q)-\operatorname{MMD}_{k_Z}^{2}(P,Q)\right| \leq4\epsilon_M. \tag*{(17)}

and therefore

∣MMD⁡kM(P,Q)−MMD⁡kZ(P,Q)∣≤2ϵM.\left|\operatorname{MMD}_{k_M}(P,Q)-\operatorname{MMD}_{k_Z}(P,Q)\right| \leq2\sqrt{\epsilon_M}.

For any fixed z,z′z,z', the RFF construction in Eq. (2) also satisfies

Pr⁡(∣kM(z,z′)−kZ(z,z′)∣>ϵ)≤2exp⁡(−Mϵ28).\Pr\left(\left|k_M(z,z')-k_Z(z,z')\right|>\epsilon\right)\leq2\exp\left(-\frac{M\epsilon^2}{8}\right).

Proof. The squared MMD is the sum of within-distribution kernel expectations minus twice the cross-distribution kernel expectation. Each changes by at most εM\varepsilon_M when kZk_Z is replaced by kMk_M, yielding Eq. (C.17). The unsquared bound follows from ∣a−b∣≤∣a−b∣|\sqrt{a}-\sqrt{b}| \le\sqrt{|a-b|} for a,b≥0a,b \ge0. For fixed z,z′z,z', the RFF kernel is an average of MM independent, unbiased terms of the form 2cos⁡(ωjTz+bj)cos⁡(ωjTz′+bj)2\cos(\omega_j^{\mathsf T}z+b_j)\cos(\omega_j^{\mathsf T}z'+b_j), each in [−2,2][-2,2]. Hoeffding’s inequality gives the stated tail bound.

Proposition C.8 bounds approximation of the base Gaussian kernel by the finite RFF map. For that realized map, Theorem C.5 gives exact population prototype-space invariance when the full nuisance subspace is removed; Proposition C.7 quantifies the additional residual from rank truncation and empirical prototype estimation.

More on Experimental Settings

Datasets

Bias for Bios. We use BIOS for occupation classification from free-text biographies. BIOS was introduced to study bias in occupation prediction from semantic representations, and is particularly well suited for concept-erasure evaluation because the target task is strongly correlated with demographic cues such as gender. [16]

Moji. We use Moji for binary sentiment classification from tweets. The dataset is designed to study social bias in language representations by comparing sentiment prediction across dialect groups. The protected attribute is dialect, distinguishing African-American English from Standard American English. This setting is useful for evaluating whether a model’s sentiment predictions rely on dialectal cues rather than task-relevant semantic content. [4]

Harvard-FairVLMed. We use Harvard-FairVLMed for multimodal vision-language learning. The task is binary glaucoma classification from paired Scanning Laser Ophthalmoscopy fundus images and de-identified ophthalmology clinical notes. The dataset contains 10,000 image–text pairs with official training, validation, and test splits. For fairness evaluation, we use race, gender, age group, and ethnicity as protected attributes [39].

MIMIC-IV. We use MIMIC-IV for both classification and regression experiments. MIMIC-IV is a large deidentified electronic health record database collected from Beth Israel Deaconess Medical Center and contains rich clinical information including demographics, diagnoses, procedures, laboratory measurements, treatments, and notes. Its breadth makes it a useful benchmark for studying fairness in realistic clinical prediction settings. [29]

FairSeg. We evaluate segmentation on FairSeg, a medical-image fairness benchmark built from 10,000 fundus images with pixel-level disc and cup masks and associated demographic attributes. FairSeg was introduced to support fairness research in medical segmentation, where both predictive quality and group-level equity are important. [63]

CelebA. We use CelebA for unimodal face-attribute prediction. The task is binary smile classification from face images. Images and attribute labels are taken from the original CelebA dataset, using its standard train/validation/test partition. For fairness evaluation, we use age group and gender as protected attributes [36].

Communities and Crime. We use Communities and Crime for tabular regression. This dataset combines community-level socioeconomic variables, law-enforcement statistics, and crime outcomes across U.S. communities, making it a standard benchmark for studying regression under sensitive demographic structure. [42]

Dataset split imbalance. Table D.8 summarizes the imbalance structure of the evaluation splits under the same protected attributes used in the reported fairness metrics. The table separates marginal protected-group imbalance from the more difficult label-conditioned or target-conditioned subgroup imbalance. This distinction is important because several datasets that appear moderate under a single protected attribute become substantially more skewed after conditioning on the task label or on protected-attribute intersections. The table should be read as a characterization of the statistical stress imposed by the benchmark: classification settings test whether errors and true-positive behavior remain stable across label-conditioned protected groups; segmentation settings test whether spatial prediction quality remains stable across protected groups and their intersections; and regression settings test whether prediction error remains stable across groups with different target distributions.

These split properties are closely related to the design of KINDER. Because KINDER estimates protected-group structure through group prototypes in the representation space, it is intended to remove demographic variation without tying the erasure operation to a particular prediction head or task loss. The imbalance summary therefore helps explain why the experiments emphasize not only marginal protected attributes, but also multi-attribute and intersectional settings. In these regimes, fairness failures can remain hidden when only aggregate performance or single-attribute summaries are considered. The empirical results broadly consistent with this motivation, especially in the text, MIMIC-IV classification, FairSeg, and FairVLMed settings, where improvements are observed on fairness metrics that directly depend on class-conditioned, multi-attribute, or intersectional subgroup behavior. At the same time, the regression results indicate that severe intersectional sparsity remains a difficult case.

Dataset and asset licenses. All datasets and baseline assets are used according to their original research-use terms. MIMIC-IV is used under the PhysioNet credentialed-access data use agreement. CelebA is used under its non-commercial research-use terms. BIOS, Moji, FairSeg, Harvard-FairVLMed, and Communities and Crime are public research datasets used according to their original release terms. We do not redistribute any dataset.

Implementation Details

Software and optimization. Unless specified otherwise by a baseline paper, all models were implemented in PyTorch. Models were optimized with Adam [33] or AdamW [37], matching the optimizer used by the corresponding baseline trainer when applicable. Hyperparameters were selected

DatasetTask / targetNNOutcomeSmallest group per attributeWorst split imbalance
FairSegSegmentation disc/cup mask9,432mask targetrace: Asian, 9.2%, R=8.3×R=8.3\times gender: male, 41.5%, R=1.4×R=1.4\times ethnicity: Hispanic, 3.9%, R=24.4×R=24.4\timesAsian ×\times male ×\times Hispanic < 0.1%, R=4051.0×R=4051.0\times
FairVLMedBinary classification glaucoma9,460positive 50.6%race: Asian, 8.3%, R=9.3×R=9.3\times gender: male, 43.6%, R=1.3×R=1.3\times ethnicity: Hispanic, 4.2%, R=22.8×R=22.8\timespositive ×\times Asian ×\times male ×\times Hispanic < 0.1%, R=2089.0×R=2089.0\times
MIMIC-IV ICU mortalityBinary classification hospital mortality72,114positive 10.1%gender: female, 45.0%, R=1.2×R=1.2\times race: NHPI, 0.2%, R=483.3×R=483.3\times age: ≤50\le50, 16.4%, R=5.1×R=5.1\timespositive ×\times female ×\times AIAN ×\times ≤50\le50 < 0.1%, R=24464.0×R=24464.0\times
MIMIC-IV ICU LOSRegression ICU length of stay56,527median 2.36 IQR 1.55–4.29gender: female, 44.2%, R=1.3×R=1.3\times race: NHPI, 0.2%, R=524.8×R=524.8\times age: 18–39, 7.9%, R=4.5×R=4.5\timesintersectional target-mean shift 5.8
Bias for BiosMulticlass classification profession396,18928 classes class R=84.2×R=84.2\timesgender: group 1, 46.1%, R=1.2×R=1.2\timesprofession 23 ×\times group 1 < 0.1%, R=476.6×R=476.6\times
Moji 90%Binary classification sentiment100,000positive 50.0%dialect: protected=0, 50.0%, R=1.0×R=1.0\timesnegative ×\times protected=1 5.0%, R=9.0×R=9.0\times
Communities and CrimeRegression violent crime rate1,994median 0.15 IQR 0.07–0.33race: Asian, 4.4%, R=17.9×R=17.9\timesrace target-mean shift 0.392
MethodMethod fitting (s) ↓\downarrowTotal pipeline (s) ↓\downarrowPeak GPU (GB) ↓\downarrowPeak RAM (GB) ↓\downarrow
TabM–21.60.1471.832
INLP171.7195.10.1473.654
LEACE1.424.80.1473.129
SAL7.230.20.1473.943
KCE150.1173.20.6826.721
RLACE217.4240.80.1475.677
KINDER-order 12.925.60.1472.876
KINDER-order 27.930.20.1472.885
KINDER-order 39.031.30.1472.902

Table D.8. Dataset split imbalance summary. RR denotes the max/min ratio over nonzero groups or cells. For classification datasets, the final column reports the smallest nonzero label-by-intersectional protected cell. For segmentation, it reports the smallest intersectional protected group. For regression, it reports the largest target-mean shift across protected groups or intersections. Non-specific protected labels such as unknown/other are excluded.

according to validation task utility, using AUC for classification, MAE for regression, and Dice score for segmentation.

For all experiments, the main hyperparameter tuned for KINDER was the Gamma parameter, searched over Γ∈{0.1,0.5,1.0,1.5,2.0,10,15,50}\Gamma\in\{0.1, 0.5, 1.0, 1.5, 2.0, 10, 15, 50\}. We also performed a smaller grid search over the random Fourier feature dimension, mRFF∈{128,256,512,1024}m_{\mathrm{RFF}} \in\{128, 256, 512, 1024\}. All remaining hyperparameters were kept identical to those of the corresponding baseline model.

Compute resources. All experiments were run on a cluster using a single NVIDIA A100 80GB GPU per run. Individual training runs usually took up to 3 GPU-hours depending on the task, dataset, and backbone. Hyperparameter grid searches required approximately 1–48 GPU-hours depending on the size of the search space and the task.

Self-supervised learning experiments. For SSL methods, we swept a 3×33 \times3 grid over learning rates {10−5,10−4,10−3}\{10^{-5}, 10^{-4}, 10^{-3}\} and weight-decay coefficients {10−6.5,10−5,10−3.5}\{10^{-6.5}, 10^{-5}, 10^{-3.5}\}. These values correspond to three log-spaced points spanning the same decade ranges as in prior work [8]. Objective-specific architectural hyperparameters were fixed to their default values for each loss and architecture. Within this grid, SSL models were trained for 10 epochs. We used a batch size of 512 for multimodal runs and 1024 for unimodal runs.

Text classification experiments. For text classification, we used a batch size of 512, a learning rate of 2×10−52 \times10^{-5}, a weight decay of 10−210^{-2}, and early stopping patience of 3. These hyperparameters were kept fixed across the corresponding baseline and KINDER experiments.

Regression experiments. For regression tasks, we used a batch size of 1024, an evaluation batch size of 4096, and a maximum training length of 50 epochs. Model selection was performed using validation MAE, and the same optimization and model-selection protocol was used for both the baseline and KINDER experiments.

Segmentation experiments. For segmentation tasks, we used a batch size of 64, a learning rate of 10−310^{-3}, a weight decay of 10−410^{-4}, and early stopping patience of 3. These hyperparameters were kept fixed across all segmentation experiments. Importantly, prior concept-erasure methods were not directly compatible with segmentation tasks, since they typically require removing or retraining the higher-level learned weights after the layer to which the erasure method is applied. To adapt these methods to segmentation, we used the following protocol: first, we trained the baseline model; second, we applied the concept-erasure method to the encoder; and third, we retrained the decoder on the cleaned representations while keeping the encoder frozen.

Representation learning backbones. For Harvard-FairVLMed multimodal experiments, we report CLIP and FactorCL backbones from the FairVLMed/FairCLIP setting [39, 48]. For MIMIC-IV tabular classification, the backbone is TabM [22]. For tabular regression, we report ResNet [26] and periodic-embedding MLPs (MLP-PLR) [23] on MIMIC-IV, and MLP-PLR on Communities and Crime. For Bias in Bios and Moji, we use the BERT Transformer encoder from HuggingFace [66]. For FairSeg, we use U-Net [54] following the reported FairSeg benchmark configuration [63].

Protected attributes are used only in fairness evaluation and in the prototype-indexed KINDER variants.

Compared Method Details

We compare KINDER-augmented training against standard post-training concept erasure and projection baselines applied to frozen features before probing. The compared baselines are INLP [50], RLACE [51], KCE [52], LEACE [3], and SUP [58]. For each method, we follow the implementation and optimization details reported in the corresponding paper whenever applicable, and tune only the main erasure-strength hyperparameter used by that method: the projection count for INLP, the removed dimension for RLACE, the kernel family and kernel bandwidth γ\gamma for KCE, the sufficient-projection dimension qq for SUP, and the removed rank for LEACE. All baseline hyperparameters are selected by validation task utility rather than fairness or erasure scores, using the task-specific validation criteria described in Appendix § 12.2. This matches our research question on how much protected-attribute information can be removed while preserving strong downstream performance.

Definitions of Metrics Used

Let D={(xi,yi,y^i,si)}i=1n\mathcal{D} = \{(x_i, y_i, \hat{y}_i, s_i)\}_{i=1}^{n} denote the test set, where yiy_i is the ground-truth target, y^i\hat{y}_i is the model prediction, and si∈Gs_i \in\mathcal{G} denotes the protected-group membership. The group variable sis_i may denote either a single protected attribute or an intersectional group obtained by taking the Cartesian product of multiple protected attributes. Thus, all group-wise metrics below are computed in the same way for both marginal and intersectional fairness evaluations.

Let YY denote the ground-truth target, Y^\hat{Y} the model prediction, and S∈GS \in\mathcal{G} the protected-group variable. The group variable SS can represent either a single protected attribute or an intersectional attribute formed from multiple protected attributes; all group-wise metrics are computed with respect to the resulting group set G\mathcal{G}.

Classification. For non-text classification tasks, we follow prior work [10] and report an aggregate fairness metric based on statistical parity, equal opportunity, equalized odds, and equal accuracy. Let

SR⁡(g)=P(Y^=1∣S=g),(18)\operatorname{SR}(g) = \mathbb{P}(\hat{Y}=1 \mid S=g), \tag*{(18)}
TPR⁡(g)=P(Y^=1∣Y=1,S=g),(19)\operatorname{TPR}(g) = \mathbb{P}(\hat{Y}=1 \mid Y=1, S=g), \tag*{(19)}
FPR⁡(g)=P(Y^=1∣Y=0,S=g),(20)\operatorname{FPR}(g) = \mathbb{P}(\hat{Y}=1 \mid Y=0, S=g), \tag*{(20)}
Acc⁡(g)=P(Y^=Y∣S=g).(21)\operatorname{Acc}(g) = \mathbb{P}(\hat{Y}=Y \mid S=g). \tag*{(21)}

For a group-wise quantity m(g)m(g), define its group disparity as

ΔG(m)=max⁡g∈Gm(g)−min⁡g∈Gm(g).(22)\Delta_{\mathcal{G}}(m) = \max_{g\in\mathcal{G}} m(g) - \min_{g\in\mathcal{G}} m(g). \tag*{(22)}

We then compute

SP⁡gap=ΔG(SR⁡),(23)\operatorname{SP}_{\mathrm{gap}} = \Delta_{\mathcal{G}}(\operatorname{SR}), \tag*{(23)}
EOpp⁡gap=ΔG(TPR⁡),(24)\operatorname{EOpp}_{\mathrm{gap}} = \Delta_{\mathcal{G}}(\operatorname{TPR}), \tag*{(24)}
EOdd⁡gap=max⁡{ΔG(TPR⁡),ΔG(FPR⁡)},(25)\operatorname{EOdd}_{\mathrm{gap}} = \max\{\Delta_{\mathcal{G}}(\operatorname{TPR}), \Delta_{\mathcal{G}}(\operatorname{FPR})\}, \tag*{(25)}
EAcc⁡gap=ΔG(Acc⁡).(26)\operatorname{EAcc}_{\mathrm{gap}} = \Delta_{\mathcal{G}}(\operatorname{Acc}). \tag*{(26)}

The aggregate fairness gap is

FairGap⁡=14(SP⁡gap+EOpp⁡gap+EOdd⁡gap+EAcc⁡gap).(27)\operatorname{FairGap} = \frac{1}{4}\left(\operatorname{SP}_{\mathrm{gap}} + \operatorname{EOpp}_{\mathrm{gap}} + \operatorname{EOdd}_{\mathrm{gap}} + \operatorname{EAcc}_{\mathrm{gap}}\right). \tag*{(27)}

Lower values indicate smaller group disparity. When reporting worst-group performance, we use

WGA⁡=min⁡g∈GAcc⁡(g),(28)\operatorname{WGA} = \min_{g\in\mathcal{G}} \operatorname{Acc}(g), \tag*{(28)}

where higher values indicate better worst-group utility.

Text classification. For text classification benchmarks, we follow the text fairness literature and report worst-group macro-F1 [7], worst-class TPR gap [27, 53], and RMS class TPR gap [50, 51]. These metrics are used because text benchmarks such as occupation and sentiment classification are commonly evaluated through class-wise group disparities rather than only aggregate binary fairness gaps.

Let CC be the label set. For each group gg and class cc, let F1g,c\mathrm{F1}_{g,c} denote the class-cc F1 score computed over all samples belonging to group gg. The group macro-F1 is

MacroF1(g)=1∣C∣∑c∈CF1g,c.(29)\mathrm{MacroF1}(g) = \frac{1}{|C|}\sum_{c \in C}\mathrm{F1}_{g,c}. \tag*{(29)}

Worst-group macro-F1 is

WgF1=min⁡g∈GMacroF1(g).(30)\mathrm{WgF1} = \min_{g \in\mathcal{G}}\mathrm{MacroF1}(g). \tag*{(30)}

Higher values indicate better worst-group text classification performance.

For class-wise group recall disparity, define

TPRg,c=P(Y^=c∣Y=c,S=g).(31)\mathrm{TPR}_{g,c} = \mathbb{P}(\hat{Y} = c \mid Y = c, S = g). \tag*{(31)}

The TPR gap for class cc is

TPRGapc=max⁡g∈GTPRg,c−min⁡g∈GTPRg,c.(32)\mathrm{TPRGap}_{c} = \max_{g \in\mathcal{G}}\mathrm{TPR}_{g,c} - \min_{g \in\mathcal{G}}\mathrm{TPR}_{g,c}. \tag*{(32)}

The worst-class TPR gap is

WCTG=max⁡c∈CTPRGapc.(33)\mathrm{WCTG} = \max_{c \in C}\mathrm{TPRGap}_{c}. \tag*{(33)}

The RMS class TPR gap is

RCTG=1∣C∣∑c∈C(TPRGapc)2.(34)\mathrm{RCTG} = \sqrt{\frac{1}{|C|}\sum_{c \in C}(\mathrm{TPRGap}_{c})^{2}}. \tag*{(34)}

Lower WCTG and RCTG indicate smaller class-wise group disparity.

Regression. For regression, we report mean absolute error (MAE) as the main utility metric and MAE gap for group error disparity [25]. The overall MAE is

MAE=E[∣Y−Y^∣].(35)\mathrm{MAE} = \mathbb{E}\left[|Y - \hat{Y}|\right]. \tag*{(35)}

The group-specific MAE is

MAE(g)=E[∣Y−Y^∣∣S=g].(36)\mathrm{MAE}(g) = \mathbb{E}\left[|Y - \hat{Y}| \mid S = g\right]. \tag*{(36)}

Worst-group MAE and MAE gap are defined as

WG-MAE=max⁡g∈GMAE(g),(37)\mathrm{WG\text{-}MAE} = \max_{g \in\mathcal{G}}\mathrm{MAE}(g), \tag*{(37)}
MAEgap=max⁡g∈GMAE(g)−min⁡g∈GMAE(g).(38)\mathrm{MAE}_{\mathrm{gap}} = \max_{g \in\mathcal{G}}\mathrm{MAE}(g) - \min_{g \in\mathcal{G}}\mathrm{MAE}(g). \tag*{(38)}

Lower values indicate smaller prediction error and smaller group error disparity. We also report R2R^2 as a regression utility metric:

R2=1−∑i(yi−y^i)2∑i(yi−yˉ)2.(39)R^2 = 1 - \frac{\sum_i(y_i - \hat{y}_i)^2}{\sum_i(y_i - \bar{y})^2}. \tag*{(39)}

Segmentation. For segmentation, we report Dice and HD95 as utility metrics. Dice measures region overlap between the predicted and ground-truth masks, while HD95 measures boundary error using the 95th-percentile Hausdorff distance. We additionally report worst-group Dice for overlap fairness [44] and HD95 gap for boundary fairness [6].

Let PP denote the predicted segmentation mask and TT the ground-truth mask. The Dice score is

Dice(P,T)=2∣P∩T∣∣P∣+∣T∣.(40)\mathrm{Dice}(P,T) = \frac{2|P \cap T|}{|P| + |T|}. \tag*{(40)}
RepresentationProjection ShiftRBF-HSIC(RR, SS) ↓Mean Gap ↓Cov. Gap ↓3rd-Moment Gap ↓Removed Rank
Raw encoder zz––1.1641.4571.7930
Raw RFF Φ(z)\Phi(z)0.0001.0001.0001.0001.0000
KINDER-order 10.1550.9750.9980.9800.9311
KINDER-order 20.3620.6870.6790.2790.2633
KINDER-order 30.5400.4330.5630.1240.0654

Table E.10. Feature-level KINDER order ablation on MOJI using BERT. The RFF representation Φ(z)\Phi(z) is used as the pre-projection reference. Higher prototype orders add residual nuisance directions beyond the lower-order subspace. Projection shift is ∥R−Φ(z)∥F/∥Φ(z)∥F\lVert R-\Phi(z)\rVert_F/\lVert\Phi(z)\rVert_F.

For each group gg, let Dice⁡(g)\operatorname{Dice}(g) denote the average Dice score over samples with S=gS=g. Worst-group Dice is

WG-Dice⁡=min⁡g∈GDice⁡(g).(41)\operatorname{WG\text{-}Dice}=\min_{g\in\mathcal{G}}\operatorname{Dice}(g). \tag*{(41)}

Higher values indicate better worst-group overlap quality.

For HD95, let ∂P\partial P and ∂T\partial T denote the boundaries of the predicted and ground-truth masks. For point sets AA and BB, define the directed 95th-percentile boundary distance as

h95(A,B)=Q0.95({min⁡b∈B∥a−b∥2:a∈A}),(42)h_{95}(A,B)=Q_{0.95}\left(\left\{\min_{b\in B}\lVert a-b\rVert_2:a\in A\right\}\right), \tag*{(42)}

where Q0.95Q_{0.95} denotes the 95th percentile. The symmetric HD95 is

HD95⁡(P,T)=max⁡{h95(∂P,∂T),h95(∂T,∂P)}.(43)\operatorname{HD95}(P,T)=\max\left\{h_{95}(\partial P,\partial T),h_{95}(\partial T,\partial P)\right\}. \tag*{(43)}

For each group gg, let HD95⁡(g)\operatorname{HD95}(g) denote the average HD95 over samples with S=gS=g. The HD95 gap is

HD95⁡gap=max⁡g∈GHD95⁡(g)−min⁡g∈GHD95⁡(g).(44)\operatorname{HD95}_{\mathrm{gap}}=\max_{g\in\mathcal{G}}\operatorname{HD95}(g)-\min_{g\in\mathcal{G}}\operatorname{HD95}(g). \tag*{(44)}

Lower HD95 and lower HD95 gap indicate better boundary quality and smaller boundary disparity.

Additional Results

Prototype-order ablation

This ablation studies how the order of the prototype family affects feature-level sensitive-attribute removal on MOJI. The raw RFF representation is used as the reference, and higher-order KINDER variants add residual nuisance directions beyond the lower-order prototype subspace. As shown in Table E.10, first-order prototypes induce only a small projection shift but still remove mean-level variation. Second- and third-order prototypes remove substantially more dependence and reduce HSIC along with covariance and third-moment gaps, at the cost of larger representation shifts and higher removed rank. This supports the use of prototype order as a controllable erasure-strength parameter.

Feature-level bootstrap significance tests

To assess whether the feature-level differences in Figure 1(c–Figure 1d) are significant, we performed paired bootstrap tests for KINDER vs the reported baselines on Moji dataset in Table E.11. For each bootstrap sample, we recompute the normalized RBF-HSIC and average moment-gap metrics. The improvement is defined as the baseline value minus the KINDER value, so positive values favor KINDER. We report one-sided bootstrap pp-values for positive improvement and Benjamini–Hochberg corrected qq-values across the tested comparisons. We observe that KINDER’s shown improvements in Figure 1(c–Figure 1d) are mostly significant, with the exception of KINDER vs KCE average gaps where despite the 95% CI being largely positive, it still holds zero inside it.

Sequential MIMIC-IV classification ablation

Table E.12 reports a sequential single-attribute removal ablation for MIMIC-IV classification. In the main experiment, Table 4, KINDER removes the joint protected-attribute structure directly. Here, we instead apply each erasure method repeatedly, removing one protected attribute at a time in sequence, and then evaluate the final representation using the same marginal and intersectional fairness metrics. This tests whether methods originally designed around a single protected attribute remain stable when extended to multiple protected attributes through repeated erasure. The results show that repeated post-hoc removal can reduce some marginal gaps, but often introduces large utility loss or unstable intersectional behavior. In contrast, KINDER-seq (NH) preserves the baseline accuracy, gives the best F1 and worst-group accuracy, and obtains the lowest intersectional gap. The KINDER-seq vs KINDER-seq (NH) variants further show the head-mismatch effect. Replacing the native task head with a linear probe improves some gap values but substantially reduces utility.

Table E.11: Paired bootstrap tests for the two feature-level summary metrics used in Figure 1(c–Figure 1d). Lower values are better. Improvement is baseline minus KINDER, so positive values favor KINDER.

BaselineMetricBaseline valueKINDER valueImprovement95% CIpp one-sidedqq BHSignif.
SUPRBF-HSIC0.890.460.43[0.33, 0.70]0.0030.005**
SUPAverage gap0.960.270.69[0.64, 0.80]0.0030.005**
INLPRBF-HSIC0.560.460.09[0.05, 0.46]0.0100.013*
INLPAverage gap0.890.270.62[0.56, 0.75]0.0030.005**
RLACERBF-HSIC1.010.460.55[0.42, 0.77]0.0030.005**
RLACEAverage gap0.960.270.69[0.63, 0.79]0.0030.005**
KCERBF-HSIC1.100.460.64[0.50, 0.85]0.0030.005**
KCEAverage gap0.300.270.03[−0.03, 0.13]0.1130.123

Table E.11.

Table E.12: Sequential post-hoc MIMIC-IV classification ablation using ResNet. Higher is better for accuracy, F1, and worst-group accuracy; lower is better for gap measures. Best values in bold and second-best values are <u>underlined</u>.

MethodAcc ↑F1 ↑Gender Gap ↓Race Gap ↓Age Gap ↓Inter. Gap ↓WorstGroupAcc ↑
ResNet0.920.400.020.230.030.47<u>0.66</u>
+ INLP-seq0.780.430.020.220.060.530.57
+ RLACE-seq0.780.430.030.230.070.530.58
+ LEACE-seq<u>0.81</u><u>0.44</u>0.010.250.050.730.38
+ KCE-seq<u>0.80</u>0.200.02<u>0.19</u><u>0.04</u>0.760.00
+ KINDER-seq0.790.430.020.160.08<u>0.46</u>0.50
+ KINDER-seq (NH)0.920.460.010.310.030.440.69

Table E.12.

Communities and Crime Dataset

Table E.13 reports the Communities and Crime regression results using MLP-PLR. This dataset tests whether concept erasure remains useful in a low-dimensional tabular regression setting where protected-group structure is tied to community-level socioeconomic variables. KINDER achieves the best overall utility, with the lowest MAE and highest R2R^2, and also gives the best race worst-group error. SUP and RLACE obtain slightly lower race-gap values, but KINDER remains close while preserving stronger predictive performance. This result is consistent with the main regression findings. Aggressive post-hoc removal can reduce some fairness gaps, but often loses substantial task information, while KINDER gives a stronger utility-preserving tradeoff.

Three-seed downstream results

The following tables report three-seed downstream results. These results are reported separately from the main tables because the revised evaluation selected configurations using validation performance averaged across three seeds. Significance marks indicate one-sided paired tt-tests, with *p<0.10p < 0.10, **p<0.05p < 0.05, and ***p<0.01p < 0.01.

Matched end-to-end fairness interventions

We additionally compare KINDER with end-to-end fairness interventions using the same TabM backbone, native task head, and matched random seeds. These methods are reported separately from the post-hoc concept-erasure baselines.

Table E.13: Communities and Crime regression results using MLP-PLR. Lower is better for MAE, worst-group MAE, and MAE gap; higher is better for R2R^2. Best values in bold and second-best values are <u>underlined</u>.

MethodMAE ↓\downarrowR2R^2 (%) ↑\uparrowRace WG ↓\downarrowRace Gap ↓\downarrow
MLP-PLR<u>0.093</u><u>63.8</u>0.1540.090
+ INLP0.14818.20.2880.164
+ LEACE0.15711.70.3590.225
+ RLACE0.094<u>63.8</u><u>0.152</u><u>0.071</u>
+ KCE0.09660.20.1900.113
+ SUP0.094<u>63.8</u><u>0.149</u>0.070
+ KINDER0.09265.30.1450.079

Table E.13.

Table E.14: MIMIC-IV classification three-seed results using TabM. Acc and F1 are performance metrics and the remaining columns are fairness metrics. Arrows indicate the preferred direction. Entries show the reported mean ±\pm standard deviation. Best values are in bold, second-best values are <u>underlined</u>.

MethodAcc ↑\uparrowF1 ↑\uparrowGender Gap ↓\downarrowRace Gap ↓\downarrowAge Gap ↓\downarrowInter. Gap ↓\downarrowWG Inter. ↑\uparrow
TabM<u>91.95 ±\pm 0.07</u>41.72 ±\pm 1.491.15 ±\pm 0.3320.63 ±\pm 1.82<u>3.25 ±\pm 1.01</u>70.83 ±\pm 0.0050.00 ±\pm 0.00
+ INLP77.67 ±\pm 3.5941.95 ±\pm 2.961.70 ±\pm 0.7921.48 ±\pm 2.315.62 ±\pm 0.9149.63 ±\pm 3.9757.82 ±\pm 10.80
+ RLACE77.65 ±\pm 2.9342.14 ±\pm 2.181.73 ±\pm 0.5222.38 ±\pm 3.047.95 ±\pm 0.9147.98 ±\pm 2.0457.64 ±\pm 8.87
+ LEACE77.82 ±\pm 2.5242.20 ±\pm 1.671.98 ±\pm 0.3920.88 ±\pm 1.476.77 ±\pm 1.2244.72 ±\pm 2.6856.06 ±\pm 8.37
+ KCE78.83 ±\pm 2.0417.46 ±\pm 1.181.26 ±\pm 0.5216.96 ±\pm 6.684.34 ±\pm 0.3574.42 ±\pm 4.0446.67 ±\pm 7.45
+ SAL77.01 ±\pm 2.3441.66 ±\pm 1.611.85 ±\pm 0.6021.63 ±\pm 3.087.02 ±\pm 0.6947.41 ±\pm 3.4951.82 ±\pm 6.89
KINDER post-hoc78.81 ±\pm 1.0543.01 ±\pm 1.001.37 ±\pm 0.3320.63 ±\pm 3.066.82 ±\pm 0.9846.69 ±\pm 4.6157.58 ±\pm 13.12
+ KINDER79.82 ±\pm 1.49<u>44.48 ±\pm 1.27</u>{sup}*`<u>1.14 ±\pm 0.19</u>{sup}*`<u>20.51 ±\pm 2.74</u>{sup}**`7.17 ±\pm 0.9546.59 ±\pm 0.0063.64 ±\pm 0.00
+ KINDER (NH)92.05 ±\pm 0.07{sup}***`46.17 ±\pm 1.35{sup}*`0.88 ±\pm 0.39{sup}*`22.10 ±\pm 8.282.23 ±\pm 0.27{sup}***`72.22 ±\pm 2.4150.00 ±\pm 0.00

Table E.14.

Multi-seed results and SAL comparison

We additionally compare KINDER directly with SAL across the four settings. SAL [56] is a kernelized spectral projection baseline for attribute removal, and it has partial methodological overlap limited to a special case of first-order KINDER. For categorical protected attributes, SAL’s cross-covariance directions align with prevalence-weighted group-mean deviations under matched feature maps, encodings, centering, weighting, and retained rank. The three-seed results show improvements in the main utility metrics across all four settings and several fairness metrics. For MIMIC-IV classification, we distinguish the post-hoc and native-head (NH) configurations.

MethodMAE ↓\downarrowR2↑R^2 \uparrowRace WG-MAE ↓\downarrowGender WG-MAE ↓\downarrowAge WG-MAE ↓\downarrowInter. WG-MAE ↓\downarrow
MLP-PLR2.62 ±\pm 0.050.12 ±\pm 0.00<u>3.39</u> ±\pm 0.06<u>2.64</u> ±\pm 0.052.88 ±\pm 0.06<u>8.00</u> ±\pm 1.21
+ INLP2.73 ±\pm 0.130.09 ±\pm 0.053.50 ±\pm 0.132.76 ±\pm 0.142.98 ±\pm 0.168.29 ±\pm 1.87
+ LEACE2.70 ±\pm 0.010.08 ±\pm 0.023.48 ±\pm 0.062.74 ±\pm 0.01<u>2.84</u> ±\pm 0.068.49 ±\pm 2.16
+ SUP2.68 ±\pm 0.000.08 ±\pm 0.033.51 ±\pm 0.052.71 ±\pm 0.022.95 ±\pm 0.087.40 ±\pm 1.96
+ SAL2.70 ±\pm 0.010.08 ±\pm 0.033.54 ±\pm 0.072.71 ±\pm 0.022.92 ±\pm 0.098.27 ±\pm 2.14
+ KINDER2.54 ±\pm 0.04**<u>0.11</u> ±\pm 0.00*3.31 ±\pm 0.06***2.55 ±\pm 0.05**2.78 ±\pm 0.07**8.80 ±\pm 0.16

Table E.15. MIMIC-IV regression three-seed results using MLP-PLR. MAE and R2R^2 measure performance; worst-group MAE columns measure fairness. Arrows indicate the preferred direction. Entries show the reported mean ±\pm standard deviation. Best values are in bold, second-best values are <u>underlined</u>.

Method\multicolumn{2}{c}{Perf.}\multicolumn{2}{c}{Race}\multicolumn{2}{c}{Gender}\multicolumn{2}{c}{Ethnicity}\multicolumn{2}{c}{Inter.}
Dice ↑\uparrowHD95 ↓\downarrowD-WG ↑\uparrowDice Gap ↓\downarrowD-WG ↑\uparrowDice Gap ↓\downarrowD-WG ↑\uparrowDice Gap ↓\downarrowD-WG ↑\uparrowDice Gap ↓\downarrow
Backbone83.12 ±\pm 1.225.12 ±\pm 1.1978.25 ±\pm 0.974.03 ±\pm 0.2982.93 ±\pm 1.170.37 ±\pm 0.2482.02 ±\pm 0.531.60 ±\pm 0.3465.88 ±\pm 2.6216.00 ±\pm 10.18
+ INLP83.42 ±\pm 1.074.82 ±\pm 0.7278.43 ±\pm 1.453.85 ±\pm 0.6683.22 ±\pm 0.910.50 ±\pm 0.3582.38 ±\pm 0.871.32 ±\pm 0.8266.53 ±\pm 4.3313.35 ±\pm 5.81
+ RLACE83.70 ±\pm 0.724.72 ±\pm 0.6478.97 ±\pm 1.013.80 ±\pm 0.6783.58 ±\pm 0.690.20 ±\pm 0.1282.52 ±\pm 0.961.42 ±\pm 0.8667.92 ±\pm 3.8311.50 ±\pm 3.99
+ LEACE83.68 ±\pm 0.814.60 ±\pm 0.7379.10 ±\pm 1.283.18 ±\pm 1.3283.52 ±\pm 0.810.32 ±\pm 0.1982.52 ±\pm 1.111.58 ±\pm 0.5669.22 ±\pm 5.348.22 ±\pm 2.42
+ SUP83.60 ±\pm 0.824.90 ±\pm 0.8878.68 ±\pm 1.123.97 ±\pm 0.3483.50 ±\pm 0.700.30 ±\pm 0.3682.55 ±\pm 0.841.60 ±\pm 0.2265.65 ±\pm 2.8616.98 ±\pm 9.37
+ KCE84.05 ±\pm 0.37<u>4.53</u> ±\pm 0.05<u>79.25</u> ±\pm 0.663.50 ±\pm 1.0783.88 ±\pm 0.390.25 ±\pm 0.1083.00 ±\pm 0.27<u>1.38</u> ±\pm 0.5967.58 ±\pm 5.5410.95 ±\pm 3.27
+ SAL83.97 ±\pm 0.684.43 ±\pm 0.5978.63 ±\pm 0.864.03 ±\pm 0.0683.63 ±\pm 0.500.40 ±\pm 0.1782.27 ±\pm 0.782.07 ±\pm 0.1565.87 ±\pm 0.61<u>9.57</u> ±\pm 1.55
+ KINDER84.00 ±\pm 0.224.18 ±\pm 0.29*79.40 ±\pm 0.71*<u>3.38</u> ±\pm 1.23<u>83.78</u> ±\pm 0.15<u>0.25</u> ±\pm 0.13<u>82.80</u> ±\pm 0.621.45 ±\pm 0.9170.18 ±\pm 5.38***7.43 ±\pm 1.52**

Table E.16. FairSeg segmentation three-seed results. Dice and HD95 measure performance; D-WG denotes worst-group Dice and Dice Gap denotes subgroup Dice disparity. Arrows indicate the preferred direction. Entries show the reported mean ±\pm standard deviation. Best values are in bold, second-best values are <u>underlined</u>.

MethodAcc ↑\uparrowF1 ↑\uparrowWgF1 ↑\uparrowWCTG ↓\downarrowRCTG ↓\downarrow
BERT<u>80.12</u> ±\pm 0.04<u>70.98</u> ±\pm 0.0566.42 ±\pm 0.0752.40 ±\pm 0.1819.58 ±\pm 0.11
+ INLP79.04 ±\pm 0.0568.53 ±\pm 0.0466.91 ±\pm 0.1146.64 ±\pm 0.2214.65 ±\pm 0.21
+ RLACE79.82 ±\pm 0.0670.52 ±\pm 0.1167.70 ±\pm 0.0151.28 ±\pm 0.6516.22 ±\pm 0.07
+ LEACE79.38 ±\pm 0.0470.08 ±\pm 0.09<u>67.05</u> ±\pm 0.05<u>47.90</u> ±\pm 0.1512.96 ±\pm 0.07
+ KCE78.22 ±\pm 2.2570.40 ±\pm 2.8966.63 ±\pm 3.5049.78 ±\pm 2.9716.25 ±\pm 3.10
+ SUP79.52 ±\pm 0.0870.18 ±\pm 0.1465.82 ±\pm 0.1851.35 ±\pm 0.4518.31 ±\pm 0.20
+ SAL79.11 ±\pm 0.0169.95 ±\pm 0.0866.95 ±\pm 0.13<u>47.81</u> ±\pm 0.30<u>13.05</u> ±\pm 0.20
+ KINDER82.08 ±\pm 0.07***72.72 ±\pm 0.20***71.65 ±\pm 0.32***47.98 ±\pm 2.32*14.28 ±\pm 0.69***

Table E.17. BIOS classification three-seed results using BERT. WgF1 denotes worst-group macro-F1, WCTG denotes worst-class TPR gap, and RCTG denotes RMS class TPR gap. Arrows indicate the preferred direction. Entries show the reported mean ±\pm standard deviation. Best values are in bold, second-best values are <u>underlined</u>.

MethodAcc ↑\uparrowF1 ↑\uparrowGender Gap ↓\downarrowRace Gap ↓\downarrowAge Gap ↓\downarrowInter. Gap ↓\downarrowWG Inter. ↑\uparrow
TabM<u>91.95</u> ±\pm 0.0741.72 ±\pm 1.491.15 ±\pm 0.3320.63 ±\pm 1.823.25 ±\pm 1.01<u>70.83</u> ±\pm 0.0050.00 ±\pm 0.00
+ FR-Train91.67 ±\pm 0.0538.39 ±\pm 3.350.34 ±\pm 0.20<u>21.61</u> ±\pm 3.313.45 ±\pm 0.99<u>70.83</u> ±\pm 0.0050.00 ±\pm 0.00
+ FairMup80.59 ±\pm 0.59<u>44.66</u> ±\pm 0.421.34 ±\pm 0.6224.48 ±\pm 1.347.38 ±\pm 0.5150.02 ±\pm 2.6660.91 ±\pm 6.10
+ GroupDRO91.31 ±\pm 0.2740.58 ±\pm 3.731.07 ±\pm 0.5526.36 ±\pm 11.44<u>3.11</u> ±\pm 1.0671.00 ±\pm 2.66<u>56.00</u> ±\pm 13.42
+ FairBatch91.65 ±\pm 0.1339.66 ±\pm 3.220.47 ±\pm 0.3329.96 ±\pm 6.904.11 ±\pm 1.3073.33 ±\pm 2.2850.00 ±\pm 0.00
+ KINDER (NH)92.05 ±\pm 0.07**46.17 ±\pm 1.35**0.88 ±\pm 0.3922.10 ±\pm 8.282.23 ±\pm 0.27*72.22 ±\pm 2.4150.00 ±\pm 0.00

Table E.18. MIMIC-IV classification matched end-to-end results using TabM and the native task head. Acc and F1 are performance metrics and the remaining columns are fairness metrics. Arrows indicate the preferred direction. Entries show the reported three-seed mean ±\pm standard deviation. Best values are in bold, second-best values are <u>underlined</u>.

MethodAcc ↑\uparrowF1 ↑\uparrowGender Gap ↓\downarrowRace Gap ↓\downarrowAge Gap ↓\downarrowInter. Gap ↓\downarrowWG Inter. ↑\uparrow
+ SAL77.01 ±\pm 2.3441.66 ±\pm 1.611.85 ±\pm 0.60<u>21.63</u> ±\pm 3.087.02 ±\pm 0.6947.41 ±\pm 3.4951.82 ±\pm 6.89
+ KINDER post-hoc (Order 1)80.22 ±\pm 1.0744.32 ±\pm 0.93<u>1.22</u> ±\pm 0.6821.15 ±\pm 2.57<u>6.97</u> ±\pm 0.6556.53 ±\pm 15.33<u>55.56</u> ±\pm 9.62
+ KINDER (Order 1)<u>80.72</u> ±\pm 1.52<u>45.38</u> ±\pm 1.161.51 ±\pm 0.2322.61 ±\pm 3.086.24 ±\pm 0.52<u>49.81</u> ±\pm 3.1364.65 ±\pm 1.75
+ KINDER (NH, Order 1)91.93 ±\pm 0.0248.62 ±\pm 1.501.17 ±\pm 0.3328.08 ±\pm 3.752.87 ±\pm 0.4275.00 ±\pm 0.0050.00 ±\pm 0.00

Table E.19. MIMIC-IV classification first-order KINDER and SAL comparison. Entries show the reported three-seed mean ±\pm standard deviation. Best values are in bold, second-best values are <u>underlined</u>.

MethodMAE ↓\downarrowR2↑R^2 \uparrowRace WG-MAE ↓\downarrowGender WG-MAE ↓\downarrowAge WG-MAE ↓\downarrowInter. WG-MAE ↓\downarrow
+ SAL2.70 ± 0.010.08 ± 0.033.54 ± 0.072.71 ± 0.022.92 ± 0.098.27 ± 2.14
+ KINDER (Order 1)2.56 ± 0.090.12 ± 0.003.37 ± 0.162.58 ± 0.102.81 ± 0.10<u>10.12 ± 0.25</u>

Table E.20. MIMIC-IV regression first-order KINDER and SAL comparison using MLP-PLR. Entries show the reported three-seed mean ± standard deviation. Best values are in bold, second-best values are <u>underlined</u>.

MethodAcc ↑\uparrowF1 ↑\uparrowWgF1 ↑\uparrowWCTG ↓\downarrowRCTG ↓\downarrow
+ SAL<u>79.11 ± 0.01</u><u>69.95 ± 0.08</u><u>66.95 ± 0.13</u><u>47.81 ± 0.30</u>13.05 ± 0.20
+ KINDER (Order 1)82.00 ± 0.0473.54 ± 0.2670.99 ± 0.3147.79 ± 2.54<u>13.98 ± 0.68</u>

Table E.21. BIOS classification first-order KINDER and SAL comparison using BERT. Entries show the reported three-seed mean ± standard deviation. Best values are in bold, second-best values are <u>underlined</u>.

MethodPerf. Dice ↑\uparrowPerf. HD95 ↓\downarrowRace D-WG ↑\uparrowRace Dice Gap ↓\downarrowGender D-WG ↑\uparrowGender Dice Gap ↓\downarrowEthnicity D-WG ↑\uparrowEthnicity Dice Gap ↓\downarrowInter. D-WG ↑\uparrowInter. Dice Gap ↓\downarrow
+ SAL<u>83.97 ± 0.68</u><u>4.43 ± 0.59</u><u>78.63 ± 0.86</u><u>4.03 ± 0.06</u><u>83.63 ± 0.50</u><u>0.40 ± 0.17</u><u>82.27 ± 0.78</u><u>2.07 ± 0.15</u><u>65.87 ± 0.61</u><u>9.57 ± 1.55</u>
+ KINDER (Order 1)84.00 ± 0.224.18 ± 0.2979.40 ± 0.713.38 ± 1.2383.78 ± 0.150.25 ± 0.1382.80 ± 0.621.45 ± 0.9170.18 ± 5.387.43 ± 1.52

Table E.22. FairSeg segmentation first-order KINDER and SAL comparison. D-WG denotes worst-group Dice and Dice Gap denotes subgroup Dice disparity. Entries show the reported three-seed mean ± standard deviation. Best values are in bold, second-best values are <u>underlined</u>.

Limitations

KINDER assumes access to protected-attribute labels during training or post-hoc prototype estimation, which is standard in fairness evaluation but may not hold in all deployment settings. Its effectiveness also depends on the chosen prototype functionals and RFF approximation. Richer prototypes can capture more group-conditioned structure, but may increase computation and remove more task-relevant variation if the protected attributes are strongly entangled with the target. In addition, while our theory gives prototype-space independence, the full probabilistic independence requires the much stronger condition that the prototype family is defined in a way that identifies the relevant group laws. Future work should explore additional prototype families, settings with missing or noisy sensitive attributes, and larger-scale evaluations under severe intersectional sparsity.

Impact Statement

Our work recognizes that ML models can cause negative societal impacts if not addressed. Thus, we present a method, KINDER to promote fairer machine learning (ML) methods that can be fully generalizable across a range of tasks, data types and training settings. By developing KINDER, we aim to reduce the ML model performance gaps across different demographic groups at both the group and intersectional level, and thus generating a positive societal impact. We acknowledge that the sensitive attributes may not capture all dimensions of identities and future work should take these into consideration.

NeurIPS Paper Checklist

Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction state four contributions. Our first key contribution is outlined in § 4.3, our second contribution is outlined in § 4.2, 4.3. The claim is that KINDER is generalizable and works across diverse settings, tasks and modalities in Section 6.

Guidelines:

  • The answer [N/A] means that the abstract and introduction do not include the claims made in the paper.

  • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A [No] or [N/A] answer to this question will not be perceived well by the reviewers.

  • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

  • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Discussion and Conclusion Section (Section 7) has a dedicated coverage of the limitations of our work which we further expand on in Appendix § 14.

Guidelines:

  • The answer [N/A] means that the paper has no limitation while the answer [No] means that the paper has limitations, but those are not discussed in the paper.

  • The authors are encouraged to create a separate “Limitations” section in their paper.

  • The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

  • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

  • The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

  • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

  • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

  • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The theoretical discussions in the paper list the assumptions made.

Guidelines:

  • The answer [N/A] means that the paper does not include theoretical results.

  • All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.

  • All assumptions should be clearly stated or referenced in the statement of any theorems.

  • The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

  • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

  • Theorems and Lemmas that the proof relies upon should be properly referenced.

Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: All experimental details are included. Furthermore, reproduction codes and trained models are available at: https://github.com/abtinmU/KINDER

Guidelines:

  • The answer [N/A] means that the paper does not include experiments.

  • If the paper includes experiments, a [No] answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

  • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

  • Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

  • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example (a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully. (c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: All datasets used are publicly available. The code that can be used to reproduce our results is shared at: https://github.com/abtinmU/KINDER

Guidelines:

  • The answer [N/A] means that paper does not include experiments requiring code.

  • Please see the NeurIPS code and data submission guidelines (https://neurips.cc/public/guides/CodeSubmissionPolicy) for more details.

  • While we encourage the release of code and data, we understand that this might not be possible, so [No] is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

  • The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://neurips.cc/public/guides/CodeSubmissionPolicy) for more details.

  • The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

  • The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

  • At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

  • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer) necessary to understand the results?

Answer: [Yes]

Justification: All the training setup are included inside the appendix.

Guidelines:

  • The answer [N/A] means that the paper does not include experiments.

  • The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

  • The full details can be provided either with the code, in appendix, or as supplemental material.

Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Significant test results are included inside the appendix.

Guidelines:

  • The answer [N/A] means that the paper does not include experiments.

  • The authors should answer [Yes] if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

  • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

  • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

  • The assumptions made should be given (e.g., Normally distributed errors).

  • It should be clear whether the error bar is the standard deviation or the standard error of the mean.

  • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

  • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g., negative error rates).

  • If error bars are reported in tables or plots, the authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Details are included inside the appendix.

Guidelines:

  • The answer [N/A] means that the paper does not include experiments.

  • The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

  • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

  • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn’t make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?

Answer: [Yes]

Justification: The research conducted in this paper conforms with the NeurIPS Code of Ethics.

Guidelines:

  • The answer [N/A] means that the authors have not reviewed the NeurIPS Code of Ethics.

  • If the authors answer [No], they should explain the special circumstances that require a deviation from the Code of Ethics.

  • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: A dedicated impact section has been included under Section G.

Guidelines:

  • The answer [N/A] means that there is no societal impact of the work performed.

  • If the authors answer [N/A] or [No], they should explain why their work has no societal impact or why the paper does not address societal impact.

  • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

  • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate Deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

  • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

  • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pre-trained language models, image generators, or scraped datasets)?

Answer: [N/A]

Justification: We do not release data or generative models.

Guidelines:

  • The answer [N/A] means that the paper poses no such risks.

  • Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

  • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

  • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All datasets used are publicly available for research purposes. All datasets and baseline models have been cited properly.

Guidelines:

  • The answer [N/A] means that the paper does not use existing assets.

  • The authors should cite the original paper that produced the code package or dataset.

  • The authors should state which version of the asset is used and, if possible, include a URL.

  • The name of the license (e.g., CC-BY 4.0) should be included for each asset.

  • For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

  • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

  • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

  • If this information is not available online, the authors are encouraged to reach out to the asset’s creators.

New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: Along with the paper, we release a detailed Readme file accompanying the code and trained models.

Guidelines:

  • The answer [N/A] means that the paper does not release new assets.

  • Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

  • The paper should discuss whether and how consent was obtained from people whose asset is used.

  • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [N/A]

Justification: Not applicable.

Guidelines:

  • The answer [N/A] means that the paper does not involve crowdsourcing nor research with human subjects.

  • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

  • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [N/A]

Justification: Not Applicable.

Guidelines:

  • The answer [N/A] means that the paper does not involve crowdsourcing nor research with human subjects.

  • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

  • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

  • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigor, or originality of the research, declaration is not required.

Answer: [N/A]

Justification: We have used an LLM only for paraphrasing text to improve the overall clarity.

Guidelines:

  • The answer [N/A] means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

  • Please refer to our LLM policy in the NeurIPS handbook for what should or should not be described.

References

  1. [1]Ramin Akbari, Milad Afshari, and Vishnu Naresh Bodd̄eti. Obliviator reveals the cost of nonlinear guardedness in concept erasure. arXiv preprint arXiv:2603.07529, 2026.arxiv.org/abs/2603.07529
  2. [2]Somnath Basu Roy Chowdhury, Nicholas Monath, Kumar Avinava Dubey, Amr Ahmed, and Snigdha Chaturvedi. Robust concept erasure via kernelized rate-distortion maximization. In Advances in Neural Information Processing Systems, volume 36, pages 43284–43306, 2023.arxiv.org/abs/2312.00194
  3. [3]Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Biderman. LEACE: Perfect linear concept erasure in closed form. In Advances in Neural Information Processing Systems, volume 36, pages 66044–66063, 2023.arxiv.org/abs/2306.03819
  4. [4]Su Lin Blodgett, Lisa Green, and Brendan O’Connor. Demographic dialectal variation in social media: A case study of African-American English. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1119–1130, 2016.arxiv.org/abs/1608.08868
  5. [5]Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in Neural Information Processing Systems, volume 29, pages 4349–4357, 2016.arxiv.org/abs/1607.06520
  6. [6]Kate Čevora, Ben Glocker, and Wenjia Bai. Quantifying the impact of population shift across age and sex for abdominal organ segmentation. In MICCAI Workshop on Fairness of AI in Medical Imaging, pages 88–97. Springer, 2024.arxiv.org/abs/2408.04610
  7. [7]Ilias Chalkidis, Tommaso Pasini, Sheng Zhang, Letizia Tomada, Sebastian Schwemer, and Anders Søgaard. Fairlex: A multilingual benchmark for evaluating fairness in legal text processing. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4389–4406, 2022.
  8. [8]Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 1597–1607. PMLR, 2020.arxiv.org/abs/2002.05709
  9. [9]Jiaee Cheong, Sinan Kalkan, and Hatice Gunes. Fairrefuse: referee-guided fusion for multi-modal causal fairness in depression detection. In Proceedings of the 33rd International Joint Conference on Artificial Intelligence, pages 7224–7232, 2024.DOI
  10. [10]Jiaee Cheong, Abtin Mogharabin, Paul Liang, Hatice Gunes, and Sinan Kalkan. Fairwell: Fair multimodal self-supervised learning for wellbeing prediction. arXiv preprint arXiv:2508.16748, 2025.
  11. [11]Jiaee Cheong, Abtin Mogharabin, Paul Pu Liang, Hatice Gunes, and Sinan Kalkan. FairSSL: Fair multimodal self-supervised learning. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pages 19236–19266. PMLR, 2026.arxiv.org/abs/2508.16748
  12. [12]Somnath Basu Roy Chowdhury and Snigdha Chaturvedi. Learning fair representations via rate-distortion maximization. Transactions of the Association for Computational Linguistics, 10:1159–1174, 2022.arxiv.org/abs/2202.00035
  13. [13]Somnath Basu Roy Chowdhury, Kumar Avinava Dubey, Ahmad Beirami, Rahul Kidambi, Nicholas Monath, Amr Ahmed, and Snigdha Chaturvedi. Fundamental limits of perfect concept erasure. In Proceedings of the 28th International Conference on Artificial Intelligence and Statistics, volume 258 of Proceedings of Machine Learning Research, pages 901–909. PMLR, 2025.arxiv.org/abs/2503.20098
  14. [14]Somnath Basu Roy Chowdhury, Sayan Ghosh, Yiyuan Li, Junier Oliva, Shashank Srivastava, and Snigdha Chaturvedi. Adversarial scrubbing of demographic information for text classification. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 550–562, 2021.arxiv.org/abs/2109.08613
  15. [15]Elliot Creager, David Madras, Jörn-Henrik Jacobsen, Marissa Weis, Kevin Swersky, Toniann Pitassi, and Richard Zemel. Flexibly fair representation learning by disentanglement. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 1436–1445. PMLR, 2019.arxiv.org/abs/1906.02589
  16. [16]Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. Bias in bios: A case study of semantic representation bias in a high-stakes setting. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 120–128, 2019.
  17. [17]Sepehr Dehdashtian, Lan Wang, and Vishnu Bodd eti. FairerCLIP: Debiasing CLIP’s zero-shot predictions using functions in RKHSS. In International Conference on Learning Representations, 2024.arxiv.org/abs/2403.15593
  18. [18]Namrata Deka and Danica J. Sutherland. Mmd-b-fair: Learning fair representations with statistical testing. In Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, volume 206 of Proceedings of Machine Learning Research, pages 9564–9576. PMLR, 2023.arxiv.org/abs/2211.07907
  19. [19]Harrison Edwards and Amos Storkey. Censoring representations with an adversary. arXiv preprint arXiv:1511.05897, 2015.arxiv.org/abs/1511.05897
  20. [20]James R. Foulds, Rashidul Islam, Kamrun Naher Keya, and Shimei Pan. An intersectional definition of fairness. In 2020 IEEE 36th international conference on data engineering (ICDE), pages 1918–1921. IEEE, 2020.
  21. [21]Hila Gonen and Yoav Goldberg. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 609–614, 2019.
  22. [22]Yury Gorishniy, Akim Kotelnikov, and Artem Babenko. TabM: Advancing tabular deep learning with parameter-efficient ensembling. In International Conference on Learning Representations, 2025.arxiv.org/abs/2410.24210
  23. [23]Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. Revisiting deep learning models for tabular data. In Advances in Neural Information Processing Systems, volume 34, pages 18932–18943, 2021.arxiv.org/abs/2106.11959
  24. [24]Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(25):723–773, 2012.DOI
  25. [25]Xudong Han, Aili Shen, Trevor Cohn, Timothy Baldwin, and Lea Frermann. Systematic evaluation of predictive fairness. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 68–81, 2022.arxiv.org/abs/2210.08758
  26. [26]Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016.arxiv.org/abs/1512.03385
  27. [27]Léo Hemamou and William Coleman. Delivering fairness in human resources ai: Mutual information to the rescue. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 867–882, 2022.DOI
  28. [28]Richard Johansson. What happens to a dataset transformed by a projection-based concept removal method? In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 17486–17492, 2024.arxiv.org/abs/2403.16142
  29. [29]Alistair Johnson, Lucas Bulgarelli, Tom Pollard, Steven Horng, Leo Anthony Celi, and Roger Mark. Mimic-iv. PhysioNet, 2021.
  30. [31]Nikola Jovanovic, Mislav Balunovic, Dimitar Iliev Dimitrov, and Martin Vechev. FARE: Provably fair representation learning with practical certificates. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 15401–15420. PMLR, 2023.arxiv.org/abs/2210.07213
  31. [32]Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. Preventing fairness gerrymandering: Auditing and learning for subgroup fairness. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2564–2572. PMLR, 2018.arxiv.org/abs/1711.05144
  32. [33]Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015.arxiv.org/abs/1412.6980
  33. [34]Yitong Li, Timothy Baldwin, and Trevor Cohn. Towards robust and privacy-preserving text representations. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 25–30, 2018.arxiv.org/abs/1805.06093
  34. [35]Ji Liu, Zenan Li, Yuan Yao, Feng Xu, Xiaoxing Ma, Miao Xu, and Hanghang Tong. Fair representation learning: An alternative to mutual information. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1088–1097, 2022.
  35. [36]Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE International Conference on Computer Vision, pages 3730–3738, 2015.arxiv.org/abs/1411.7766
  36. [37]Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019.
  37. [38]Shilin Lu, Zilan Wang, Leyang Li, Yanzhu Liu, and Adams Wai-Kin Kong. Mace: Mass concept erasure in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6430–6440, 2024.arxiv.org/abs/2403.06135
  38. [39]Yan Luo, Min Shi, Muhammad Osama Khan, Muhammad Muneeb Afzal, Hao Huang, Shua||||hang Yuan, Yu Tian, Luo Song, Ava Kouhana, Tobias Elze, et al. Fairclip: Harnessing fairness in vision-language learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12289–12301, 2024.arxiv.org/abs/2403.19949
  39. [40]Yuhong Luo, Austin Hoag, Xintong Wang, Philip S Thomas, and Przemyslaw A Grabowicz. Fair representation learning with controllable high confidence guarantees via adversarial inference. In Advances in Neural Information Processing Systems, volume 38, 2025.arxiv.org/abs/2510.21017
  40. [41]David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Learning adversarially fair and transferable representations. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 3384–3393. PMLR, 2018.arxiv.org/abs/1802.06309
  41. [42]Lawrence McClendon and Natarajan Meghanathan. Using machine learning algorithms to analyze crime data. Machine Learning and Applications: An International Journal, 2(1):1–12, 2015.
  42. [43]Krikamol Muandet, Kenji Fukumizu, Bharath Sriperumbudur, Bernhard Schölkopf, et al. Kernel mean embedding of distributions: A review and beyond. Foundations and Trends in Machine Learning, 10(1-2):1–141, 2017.DOI
  43. [44]Fernando Pérez-García, Harshita Sharma, Sam Bond-Taylor, Kenza Bouzid, Valentina Salvatielli, Maximilian Ilse, Shruthi Bannur, Daniel C Castro, Anton Schwaighofer, Matthew P Lungren, et al. Exploring scalable medical image encoders beyond text supervision. Nature Machine Intelligence, 7(1):119–130, 2025.
  44. [45]Dana Pessach and Erez Shmueli. A review on fairness in machine learning. ACM Computing Surveys, 55(3):1–44, 2022.DOI
  45. [46]Amisha Priyadarshini and Sergio Gago-Masague. Learning fair representations with kolmogorov-arnold networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 24864–24872, 2026.
  46. [47]Lucia Quirke and Nora Belrose. Slowing learning by erasing simple features. arXiv preprint arXiv:2502.02820, 2025.arxiv.org/abs/2502.02820
  47. [48]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 8748–8763. PMLR, 2021.arxiv.org/abs/2103.00020
  48. [49]Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. In Advances in Neural Information Processing Systems, volume 20, pages 1177–1184, 2007.
  49. [50]Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. Null it out: Guarding protected attributes by iterative nullspace projection. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7237–7256, 2020.
  50. [51]Shauli Rayfogel, Michael Twiton, Yoav Goldberg, and Ryan D Cotterell. Linear adversarial concept erasure. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 18400–18421. PMLR, 2022.
  51. [52]Shauli Ravfogel, Francisco Vargas, Yoav Goldberg, and Ryan Cotterell. Adversarial concept erasure in kernel space. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6034–6055, 2022.
  52. [53]Alexey Romanov, Maria De-Arteaga, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, Anna Rumshisky, and Adam Kalai. What’s in a name? reducing bias in bios without access to protected attributes. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4187–4195, 2019.arxiv.org/abs/1904.05233
  53. [54]Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, pages 234–241. Springer, 2015.arxiv.org/abs/1505.04597
  54. [55]Antoine Saillenfest and Pirmin Lemberger. Nonlinear concept erasure: A density matching approach. In ECAI 2025: 28th European Conference on Artificial Intelligence, volume 413 of Frontiers in Artificial Intelligence and Applications, pages 2138–2145. IOS Press, 2025.arxiv.org/abs/2507.12341
  55. [56]Shun Shao, Yftah Ziser, and Shay B Cohen. Gold doesn’t always glitter: Spectral removal of linear and nonlinear guarded attribute information. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1611–1622, 2023.
  56. [57]Xudong Shen, Yongkang Wong, and Mohan Kankanhalli. Fair representation: Guaranteeing approximate multiple group fairness for unknown tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1):525–538, 2023.DOI
  57. [58]Enze Shi, Lei Ding, Linglong Kong, and Bei Jiang. Debiasing with sufficient projection: A general theoretical framework for vector representations. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 5960–5975, 2024.DOI
  58. [59]Jiaming Song, Pratyusha Kalluri, Aditya Grover, Shengjia Zhao, and Stefano Ermon. Learning controllable fair representations. In Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, volume 89 of Proceedings of Machine Learning Research, pages 2164–2173. PMLR, 2019.arxiv.org/abs/1812.04218
  59. [60]Bharath K Sriperumbudur, Kenji Fukumizu, and Gert RG Lanckriet. Universality, characteristic kernels and RKHS embedding of measures. Journal of Machine Learning Research, 12(70):2389–2410, 2011.arxiv.org/abs/1003.0887
  60. [61]Zilong Tan, Samuel Yeom, Matt Fredrikson, and Ameet Talwalkar. Learning fair representations for kernel models. In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pages 155–166. PMLR, 2020.arxiv.org/abs/1906.11813
  61. [62]Huan Tian, Bo Liu, Tianqing Zhu, Wanlei Zhou, and Philip S Yu. Multifair: Model fairness with multiple sensitive attributes. IEEE Transactions on Neural Networks and Learning Systems, 36(3):5654–5667, 2025.DOI
  62. [63]Yu Tian, Min Shi, Yan Luo, Ava Kouhana, Tobias Elze, and Mengyu Wang. Fairseg: A large-scale medical image segmentation dataset for fairness learning using segment anything model with fair error-bound scaling. In International Conference on Learning Representations, 2024.arxiv.org/abs/2311.02189
  63. [64]Steven Vethman, Quirine TS Smit, Nina M van Liebergen, and Cor J Veenman. Fairness beyond the algorithmic frame: actionable recommendations for an intersectional approach. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, pages 3276–3290, 2025.DOI
  64. [65]Liwen Wang, Yuanmeng Yan, Keqing He, Yanan Wu, and Weiran Xu. Dynamically disentangling social bias from task-oriented representations with adversarial attack. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3740–3750, 2021.DOI
  65. [66]Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. HuggingFace’s transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, 2020.arxiv.org/abs/1910.03771
  66. [67]Changdi Yang, Zheng Zhan, Ci Zhang, Yifan Gong, Yize Li, Zichong Meng, Jun Liu, Xuan Shen, Hao Tang, Geng Yuan, Pu Zhao, Xue Lin, and Yanzhi Wang. FairSMOE: Mitigating multi-attribute fairness problem with sparse mixture-of-experts. In Proceedings of the 34th International Joint Conference on Artificial Intelligence, pages 610–618, 2025.DOI
  67. [68]Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. In Proceedings of the 30th International Conference on Machine Learning, volume 28 of Proceedings of Machine Learning Research, pages 325–333. PMLR, 2013.
  68. [69]Han Zhao and Geoffrey J. Gordon. Inherent tradeoffs in learning fair representations. Journal of Machine Learning Research, 23(57):1–26, 2022.
  69. [70]Xianda Zhou and William Yang Wang. Mojitalk: Generating emotional responses at scale. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1128–1137, 2018.arxiv.org/abs/1711.04090

Paper details

vlnc:W_chxgjx2c
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

an LLM

View statement
Looking for related papers…
Contents