WorksComputer Science

DecisionUnlearn-Bench: Unlearning Probabilistic Decisions under Changing Schemas

Published Oct 28, 2026Fresh — added to the Hub within the last 30 daysv185 min read

AI contributions

Not recorded for this paper

Abstract

Machine unlearning aims to reproduce the behavior of a model trained without designated data. For probabilistic decision models, this target also depends on the candidate options supplied at inference. We introduce DecisionUnlearn-Bench, an empirical framework that compares full, unlearned, and retain-only models across deletion granularities, label renderings, and candidate sets. Across 20 eligible model configurations on the BANKING77 dataset, full-to-oracle displacement increases consistently from example to association to class deletion. Replacing semantic option labels with opaque codes nearly eliminates the retrained oracle's recognition of deleted classes, even with successful acquisition of the full model, whereas changing the candidate set alters both the oracle and the measured unlearning progress. SimNPO satisfies the screening criterion in 19 of 20 configurations, but its gains vary substantially across seeds and deletion targets. Development tuning on the CLINC150 dataset does not establish a positive pooled effect across 48 intents. At approximately matched suppression, methods can differ markedly in oracle fidelity. Recovery experiments further distinguish initial forgetting from subsequent adaptation and show conclusions that depend on both architecture and metric. These results show that evaluating decision-model unlearning relative to a schema-specific retraining target, with separate reporting of effect size, utility, robustness, and recoverability.

Introduction

Machine unlearning asks how to remove the influence of designated training data in a trained model without repeating the entire training process [6] (Bourtoule et al. 2021). A central reference is the model obtained

by training on the remaining data. This reference makes the objective more precise than simply reducing accuracy on examples associated with a deletion request. A relevant question is whether the unlearned model would have behaved the same way without those examples.

This distinction matters for models that return probabilities over a specified set of decisions. For example, in intent classification, a request is evaluated against a set of named candidate intents. The model can use both its learned mapping from utterances to intents and the meanings of the supplied labels. Removing every training sample of an intent therefore need not eliminate recognition of that intent. The remaining data, pretrained knowledge, and candidate descriptions may still support the correct decision.

Existing language-model unlearning benchmarks examine forgotten knowledge, retained utility, privacy, and robustness through complementary evaluations [9, 10, 3]. Preference-based methods also address the instability of directly maximizing the loss on data that needs to be forgotten [11, 4]. These developments leave an additional evaluation question for decision models:

How should fidelity to retraining be interpreted when the information supplied by the decision schema changes?

A low forgotten-class accuracy can agree with one retraining target and disagree with another. Similarly, a normalized improvement can change because the unlearned model changes, the oracle changes, or the original deletion effect becomes small.

Three distinctions are consequently necessary. First, the counterfactual effect of deletion must be measured before interpreting an intervention. Second, the amount of probability removed from forgotten options must be separated from where that probability is redistributed. Third, high performance after further adaptation must be separated from the behavior already present before adaptation. Without these distinctions, a method can appear to forget successfully, transfer reliably, or relearn rapidly for reasons that the reported endpoint does not isolate.

We address these questions with DecisionUnlearn-Bench,

Copyright 2026 by the author(s). a study of probabilistic decision unlearning on the BANKING77 and CLINC150 datasets [2, 8]. The benchmark compares a full model, an unlearned model, and a retain-only retrained oracle using their complete option distributions. It separates: (i) example, association, and class deletion, (ii) development selection and subsequent evaluation, and (iii) broad model screening and targeted replication. Controlled changes to label rendering and candidate composition test how the decision interface changes the target itself.

The results identify both useful approximation and clear limits. Class deletion creates the largest distributional shift across all models in the 20-model granularity panel. SimNPO provides reliable gains across the reference BANKING77 seed-bundle grid, yet its performance under the frozen setting is less stable across other seeds and across CLINC150 targets. Opaque labels almost eliminate oracle recognition, but do not consistently increase normalized unlearning progress. Reduced candidate sets can also expose poor fidelity when labels are opaque, showing that schema sensitivity extends beyond residual semantic recognition. Recovery analyses produce different conclusions for Apertus-8B and Qwen3.5-9B models once starting behavior and remaining headroom are separated.

Our contributions are as follows:

An evaluation design that measures displacement and approximation induced by deletion to a retraining target specific to the schema on both deleted training rows and unseen related rows.

An empirical account of how label semantics and candidate composition alter oracle behavior and the interpretation of unlearning fidelity.

A combined analysis of method transfer, probability redistribution, and recovery that separates statistical acceptance from effect size and identifies the limits of conclusions drawn from a fixed decision interface.

Related Work

Unlearning as approximation to retraining. Data-deletion formulations and efficient retraining schemes establish the importance of specifying the learning procedure whose output is to be reproduced [6] (Bourtoule et al. 2021). More recent oracle-matching approaches explicitly target output distributions rather than only the correctness of forgotten predictions [5]. We adopt retraining as an operational reference for supervised task adaptation. Our experiments do not remove the dataset from the pre-training or establish equality between the distributions over the trained model parameters.

Language-model unlearning methods and evaluation.

TOFU [9] studies synthetic author knowledge with retain-only references, whereas MUSE [10] evaluates several aspects of forgetting and retained utility. The OpenUnlearning [3] framework consolidates these two methods and evaluations to examine the reliability across different metrics. NPO [11] and SimNPO [4] modify the forgetting objective to improve optimization behavior. RMU (?) instead intervenes on internal representations. Our contribution is an empirical investigation of these approaches in a decision setting, rather than a new unlearning optimizer.

Decision schemas and recoverability.

Intent datasets provide explicit candidate meanings and natural distinctions between removing individual records and removing all examples of an intent [2, 8]. We vary those candidate meanings and the supplied option set while retaining the same evaluation inputs. Relearning is already an established unlearning evaluation, including in the SimNPO study [4]. Here, it compares adaptation from unlearned and retrained checkpoints while distinguishing starting-point differences, subsequent gains, and transfer through a newly initialized head.

Problem Formulation and Evaluation

A schema-specific retraining target

Let $\mathcal{D}$ be the supervised training set, $\mathcal{D}_{F} \subset\mathcal{D}$ a deletion set, and $\mathcal{D}_{R} = \mathcal{D} \setminus\mathcal{D}_{F}$ the retained set. A decision schema $\mathcal{S} = (\mathcal{C}, \ell)$ specifies an ordered candidate set $\mathcal{C}$ and its textual rendering $\ell$ , including option names and descriptions. Model $M$ returns a categorical distribution

\mathbf{p}_{M}(x;\mathcal{S}) = \left(p_{M}(c \mid x;\mathcal{S})\right)_{c \in\mathcal{C}}, \qquad\sum_{c \in\mathcal{C}} p_{M}(c \mid x;\mathcal{S}) = 1. \tag*{(1)}

The term decision model refers to this probabilistic interface throughout the paper.

For a training procedure $\mathcal{A}$ , pretrained initialization $M_{0}$ , and training seed $\xi$ , we compare

\begin{aligned} M_{\mathrm{full}} &= \mathcal{A}(M_{0}, \mathcal{D}, \mathcal{S}_{\mathrm{train}};\xi),\\ M_{\mathrm{O}} &= \mathcal{A}(M_{0}, \mathcal{D}_{R}, \mathcal{S}_{\mathrm{train}};\xi),\\ M_{\mathrm{U}} &= \mathcal{U}(M_{\mathrm{full}}, \mathcal{D}_{F}, \mathcal{D}_{R}), \tag*{(2)} \end{aligned}

where $\mathcal{U}$ denotes an approximate unlearning procedure. The retrained oracle $M_{\mathrm{O}}$ is trained without the deletion set under the task-training recipe. The no-op baseline is $M_{\mathrm{full}}$ . Both references are evaluated under the same schema as $M_{\mathrm{U}}$ . In particular, deleted classes remain available as options: deleting their training examples does not remove their outputs from the decision space.

The label intervention changes $\mathcal{S}_{\mathrm{train}}$ and the corresponding evaluation rendering, requiring separately trained full and oracle models. The candidate-set intervention changes the evaluation schema of existing checkpoints and performs a new forward pass for each reduced option set. These interventions address different sources of schema dependence.

Deletion granularity and evaluation views

We study three deletion requests. (i) Example deletion removes a random 5% of training rows. (ii) Association deletion removes selected sub-intent clusters while retaining other examples of their parent intents. And (iii) Class deletion removes all training rows of selected classes. The primary BANKING77 comparison uses ten association clusters or four classes, whereas CLINC150 class bundles contain eight intents.

We distinguish two evaluation views. The (i) forget-training view $\mathcal{E}_{F}$ contains deleted training rows. The (ii) forget-generalization view $\mathcal{E}_{G}$ contains unseen evaluation rows associated with the deleted units. For class and association deletion, these are rows of the corresponding class or cluster. For example deletion, these are evaluation rows whose nearest training row belongs to $\mathcal{D}_{F}$ . Thus, $\mathcal{G}$ does not denote a second deletion set or a generic robustness benchmark. Retained utility is evaluated on $\mathcal{E}_{R}$ .

Distance, progress, and feasibility

For $v \in\{F,G\}$ , the oracle distance is the mean base-2 Jensen–Shannon divergence:

\operatorname{OD}_{v}(M;\mathcal{S}) = \frac{1}{\lvert\mathcal{E}_{v}\rvert}\sum_{x \in\mathcal{E}_{v}} \operatorname{JS}_{2}\left(p_{M}(x;\mathcal{S}),p_{M_{O}}(x;\mathcal{S})\right). \tag*{(3)}

We use raw option probabilities, with numerical stabilization at $10^{-12}$ . We define

\begin{aligned} D_{0,v} &= \operatorname{OD}_{v}(M_{\mathrm{full}};\mathcal{S}),\\ \Delta\operatorname{OD}_{v} &= D_{0,v} - \operatorname{OD}_{v}(M_{U};\mathcal{S}),\\ P_{v} &= \frac{\Delta\operatorname{OD}_{v}}{D_{0,v}} = 1 - \frac{\operatorname{OD}_{v}(M_{U};\mathcal{S})}{D_{0,v}}. \tag*{(4)} \end{aligned}

Here $D_{0,v}$ is deletion displacement, $\Delta\operatorname{OD}_{v}$ is the absolute gain over no-op, and $P_{v}$ is normalized oracle progress. Positive gain means closer agreement with the oracle. Progress equals zero for no-op and one for exact agreement on the evaluated distributions, and negative values indicate movement away from the oracle. Because the denominator depends on the deletion request and schema, we report $D_{0,v}$ and absolute gains alongside progress. We do not interpret ratios with negligible displacement as stable effect sizes.

A configuration is feasible when retained accuracy decreases by at most 0.02 and retained expected calibration error increases by at most 0.02 relative to the corresponding full model:

\begin{aligned} \operatorname{Acc}_{R}(M_{U}) &\ge\operatorname{Acc}_{R}(M_{\mathrm{full}}) - 0.02,\\ \operatorname{ECE}_{R}(M_{U}) &\le\operatorname{ECE}_{R}(M_{\mathrm{full}}) + 0.02. \tag*{(5)} \end{aligned}

Protocol acceptance additionally requires positive gains with positive lower 95% confidence bounds on both views. It specifies no minimum progress fraction and does not imply equivalence to retraining.

Selection, replication, and uncertainty

Development data determine target eligibility, method configurations, and suppression matching. Primary class targets require full-model development accuracy of at least 0.70 and gold probability of at least 0.50. Association targets use an accuracy threshold of 0.65. Targets are selected in hash order among eligible units at the primary seed. Replication preserves target selection rather than silently replacing difficult seed-specific cases. One semantic BANKING77 bundle-02 class falls below the accuracy threshold at seed 42. We disclose this exception in Appendix § 12.

Within a fixed trained configuration, estimates use 5000 paired row-bootstrap draws, with forgotten-class or association-cluster bootstraps as sensitivity analyses. Each progress ratio is recomputed inside the resample. The frozen confirmation family reports Holm-adjusted intersection–union tests based on the larger of the two one-sided view-specific $p$ -values. Architecture-screening decisions use the stored per-model rule without correction across models. Reference-model cross-bundle summaries use the stored hierarchical bootstrap over seeds, bundles, and units. Row intervals are conditioned on trained checkpoints and do not quantify uncertainty over future training seeds.

Experimental Design

BANKING77 provides 77 fine-grained banking intents, while CLINC150 spans 150 in-scope intents across ten domains (Casanu[e]va et al., [2]; Larson et al. [8]). Our CLINC150 interface contains 151 options, including out-of-scope options. The model panel contains 20 eligible configurations. InternLM3-8B was excluded because checkpoint reloading was not reproducible. Qwen3.5-0.8B is the reference model for method selection and the most extensive replication. We use model configuration to denote a named backbone and its size. Multiple configurations can belong to the same model family.

We evaluate gradient ascent (GradAscent), gradient difference with retained-data supervision (GradDiff), its KL-regularized variant (GradDiff-KL), NPO, SimNPO, RMU, and a uniformization control in the experimental blocks, each of which was selected or specified. These are not a complete method-by-model factorial comparison. The main confirmation compares six class-deletion arms, with the broad transfer panel evaluating the frozen reference-model SimNPO configuration. RMU is included in the suppression diagnostic. Temperature scaling provides an additional control for CLINC150.

The main evaluation design separates three questions. First, the granularity screen measures the changes in retain-only retraining across all eligible models. Second, method confirmation and a seed-bundle replication test approximate that target. Third, schema, suppression, and recovery interventions examine why a favorable fixed-interface result may not persist. Table 1 summarizes observed evaluation coverage. Supplementary tables preserve the full numerical results and the detailed distinction among the pilot, development, confirmation, screening, replication, and exploratory stages.

Experiment	Observed coverage	Role
Granularity	BANKING77; 20 eligible models; seed 17; three deletion requests	Descriptive full–oracle comparison
Method comparison	Qwen3.5-0.8B; six class-confirmation arms; four methods across a 3-seed × 3-bundle grid	Development selection, confirmation, replication
Model transfer	Frozen SimNPO; 20-model panel including the reference; five-model seed/bundle comparison	Single-seed screening and targeted replication
Dataset transfer	CLINC150; frozen transfer, then tuned SimNPO on seven cells across bundles 02–07	Confirmation plus intent-level replication
Label rendering	Qwen3.5-0.8B; seven semantic/opaque pairs: four BANKING77, three CLINC150	Mechanistic contrasts
Candidate sets	Three models; full, neighbor, and equal-size random-distractor schemas; semantic and selected opaque checkpoints	Evaluation-only rescoring
Suppression	Reference model; BANKING77 target $s = 0.6$ , CLINC150 target $s = 1$ on development data	Held-out diagnostic
Recovery	Three models; one trained seed each; three exposure budgets, two adaptation seeds, three head seeds	Exploratory; controls for 0.8B and Apertus only

Table 1. Observed evaluation design. Repeated evaluations of one checkpoint are not independent training runs.

Results

Deletion granularity determines the size of the counterfactual shift

Full-to-oracle displacement increases from example to association, then to class deletion in every configuration in the BANKING77 screen (Figure 1). The panel medians are respectively 0.035, 0.150, and 0.477 on $\mathcal{F}$ , and 0.021, 0.149, and 0.396 on $\mathcal{G}$ . These are descriptive medians over model configurations, not population estimates.

Schematic and log-scale plots showing oracle displacement for example, association, and class deletion settings — Figure 1. Deletion granularity changes the retraining target. (A) Full, unlearned, and retain-only models. (B,C) BANKING77 displacement for 20 eligible configurations, seed 17, on $\mathcal{F}$ and $\mathcal{G}$ . Thin lines connect each model’s three deletion settings: black marks show the median and interquartile range, not confidence intervals; gray marks show the range of same-data full-model divergence for Qwen3.5-0.8B only. Example, association, and class deletion increase displacement in every model. The schematic’s association description refers to selected parent intents, not all 77 intents.

Same-data replicas are available for Qwen3.5-0.8B at seeds 17, 42, and 99. For this model, example-deletion displacement is approximately within twice the observed replica noise. This supports the protocol’s decision not to advance example-deletion methods beyond selection. It does not establish that deleted records were never memorized, or that retaining the full model meets a privacy requirement. The replica comparison is also specific to this reference model.

Association deletion produces a larger target shift, and its final cross-bundle sensitivity analysis finds modest SimNPO gains: $\Delta ODF = 0.019$ [0.007, 0.034] and $\Delta ODG = 0.018$ [0.005, 0.035]. We retain this result while distinguishing its selection pathway from the primary class-deletion confirmation. The strongest and most extensively replicated intervention evidence concerns class deletion.

Label meaning changes the oracle, but does not guarantee easier unlearning

We replace option names and descriptions with fixed opaque codes while preserving utterances, target identities, option order, and the availability of forgotten options. Full models pass the opaque-label acquisition checks. On BANKING77, oracle accuracy on deleted training rows falls from 0.209–0.315 with semantic labels to zero across both bundles and both seeds. The corresponding unseen-row accuracies are also zero under opaque labels. Forgotten-option mass remains small but nonzero, approximately $4.7 \times10^{-4}$ to $1.8 \times10^{-3}$ on $\mathcal{F}$ (Figure 2A,B).

Decision schema comparison across label meanings, deletion displacement, oracle recognition, and candidate-dependent progress — Figure 2. The decision schema changes the oracle and measured progress. (A) Oracle recognition under semantic and opaque labels; the narrow strip reports the lowest full-model development accuracy among forgotten classes. (B) Each condition’s displacement denominator. (C) Oracle $\mathcal{F}$ accuracy under full, neighbor, and random-distractor schemas. (D) BANKING77 SimNPO $P_F$ with row-bootstrap 95% intervals, dashed open markers denote opaque labels. Reduced schemas are genuinely rescored and retain every forgotten option. Circles and squares in A,B indicate $\mathcal{F}$ and $\mathcal{G}$ , option counts appear below C.

The CLINC150 contrasts show the same qualitative oracle effect. Semantic-label oracle accuracy on $\mathcal{F}$ is 0.661, 0.505, and 0.781 for bundles 02, 03, and 06. With opaque labels, only one of 791 rows is correct on bundle 02, and none is correct on bundles 03 or 06. Full-to-oracle displacement increases to 0.983, 0.993, and 0.999. Thus, the retrained model’s residual recognition depends strongly on the information in the supplied labels in these configurations.

Unlearning progress does not respond uniformly. On BANKING77 bundle 01, SimNPO reaches $P_{\mathcal{F}}$ of 0.60–0.61 under opaque labels, compared with 0.55–0.59 under semantic labels. On bundle 02, progress decreases from 0.70 to 0.45 at seed 17 and from 0.47 to 0.40 at seed 42. CLINC150 progress changes from 0.22 to 0.28 on bundle 02, from 0.22 to 0.13 on bundle 03, and from -0.24 to 0.02 on bundle 06. The last opaque run is statistically accepted but reproduces only a small fraction of the deletion shift.

The intervention therefore changes the oracle more consistently than it improves approximation to the oracle. Removing semantic labels does not by itself resolve CLINC150’s unlearning difficulty. These are mechanistic contrasts, including previously inspected targets, rather than an independent estimate of cross-target success.

Candidate composition changes both recognition and fidelity

We next rescore the existing checkpoints under the full option set ( $S_0$ ), the set of forgotten classes with semantic neighbors ( $S_1$ ), and the set of forgotten classes with randomly retained distractors ( $S_2$ ). The two reduced schemas have equal cardinality, with 12 options in BANKING77 and 24 in CLINC150. This comparison controls for subset size between $S_1$ and $S_2$ . Comparison with $S_0$ changes both size and composition.

For semantic BANKING77 bundle 01, Qwen3.5-0.8B’s oracle accuracy on $\mathcal{F}$ increases from 0.269 under $S_0$ to 0.460 under $S_1$ and 0.799 under $S_2$ . On $\mathcal{G}$ , the corresponding values are 0.306, 0.519, and 0.756. The deletion displacement on $\mathcal{F}$ decreases from 0.608 to 0.438 and 0.148. SimNPO’s progress changes from 0.59 to 0.66 and then to 0.32. Other selected methods also lose progress with random distractors. Apertus-8B and Qwen3.5-9B retain positive row-level progress under $S_2$ , although class-level uncertainty is wider (Figure 2C,D).

Opaque labels separate this effect from semantic oracle recognition. On BANKING77 bundle 01, oracle recognition remains near zero under reduced schemas, yet SimNPO progress falls from 0.61 under $S_0$ to 0.00 under $S_1$ and 0.12 under $S_2$ . Bundle 02 shows the same qualitative failure to preserve full-schema progress (0.45 to 0.04 and 0.12). Reduced-schema sensitivity therefore persists even when the oracle does not substantially re-recognize the forgotten classes.

Reliable reference-model gains do not imply uniform transfer

In the reference class-deletion confirmation, SimNPO, GradAscent, GradDiff, and NPO satisfy the feasibility and gain criteria (Table 2). SimNPO has the largest point gains, $\Delta O_{\mathrm{DF}} = 0.359$ and $\Delta O_{\mathrm{DG}} = 0.280$ , while GradDiff-KL and Uniformization fail the utility/calibration gate despite positive distance gains. These results concern selected configurations, not an exhaustive ranking of optimization algorithms.

Method	$\Delta O_{\mathrm{DF}}$	$\Delta O_{\mathrm{DG}}$	$P_F/P_G$	Decision
Phase 1: seed 17, bundle 01; $D_{0,F} = 0.608$ , $D_{0,G} = 0.495$
SimNPO	0.359	0.280	0.59/0.57	Accepted
GradAscent	0.332	0.241	0.55/0.49	Accepted
GradDiff	0.274	0.209	0.45/0.42	Accepted
NPO	0.236	0.176	0.39/0.36	Accepted
GradDiff-KL	0.188	0.139	0.31/0.28	Infeasible
Uniformization	0.160	0.102	0.26/0.21	Infeasible
Phase 2: seeds 17/42/99; bundles 01–03; nine cells per method
SimNPO	0.337	0.302	—	Replicates
GradAscent	0.114	0.081	—	Does not replicate
GradDiff	0.187	0.160	—	Infeasible cell(s)
NPO	0.168	0.142	—	Infeasible cell(s)

Table 2. Reference-model class deletion on BANKING77. Gains are no-op minus unlearned oracle distance. Intervals in the upper block use paired rows; the lower block reports the stored hierarchical seed–bundle–class analysis.

Across the reference model’s nine seed–bundle cells, SimNPO retains positive hierarchical gains on both views, remains feasible in every cell, and survives the stored leave-one-seed/bundle-out checks. Its pooled gains are 0.337 [0.238,0.443] on $\mathcal{F}$ and 0.302 [0.208,0.412] on $\mathcal{G}$ . GradAscent does not yield positive aggregate gains, whereas GradDiff and NPO do, but both fail the full replication rule because feasibility is not maintained in every cell.

The frozen-setting panel contains 20 configurations, including the reference model. 19 satisfy the stored screening criterion, while Qwen3-4B is positive but unconfirmed. Progress on $\mathcal{F}$ ranges from 0.20 to 0.63, with a median of 0.46. However, additional seeds reveal substantial variation (Figure 3). At seed 42, $P_{\mathcal{F}}$ is 0.17 for Apertus-8B, 0.14 for Qwen3.5-9B, 0.04 for OLMo-3-7B, and -0.18 for Qwen3-4B. The reference model retains 0.55. Results for seed 17 on bundle 02 are positive across all five replicated models. This purposive, incomplete panel supports within-model contrasts, not a population variance decomposition or a 95% generalization claim.

Architecture screen and seed / bundle replication plots — Figure 3. Frozen-setting transfer and within-model variation. (A) BANKING77 bundle 01, seed 17, for 20 configurations including the reference model: circles $P_F$ , squares $P_G$ , with conditional row-bootstrap 95% intervals. Status follows the stored screening rule without correction across models. (B) Five-model replication panel. Solid guides mark seed changes, dashed guides mark bundle changes. Small positive effects can satisfy acceptance, while seed-specific effects can be weak or negative.

Phase 1 uses 428 F and 160 G rows. The full model’s retained accuracy is 93.7%, and ECE is 0.026. Retained accuracy changes for the six methods are -0.1,-1.0,+0.2,-2.2,-1.0,+2.2 percentage points; ECE changes are +0.001,+0.013,+0.011,+0.003,+0.035,+0.046. The supplied confirmation table reports Holm-adjusted $p = 0.0005$ for the first five arms; it does not provide an adjusted value for Uniformization. Feasibility is required independently of gain. Phase 2 progress is not pooled; individual results and ranges are in the supplement. success probability.

Dataset transfer is less favorable. On CLINC150 bundle 01, the BANKING77 SimNPO settings increase oracle distance: gains are -0.191 on $\mathcal{F}$ and -0.237 on $\mathcal{G}$ . Development tuning produces accepted results on bundles 02 and 03, but later bundles include unconfirmed and negative outcomes. Across 48 intents in six bundles, the pooled gains are 0.022 [-0.045,0.090] on $\mathcal{F}$ and 0.022 [-0.044,0.094] on $\mathcal{G}$ (Table 3). These intervals do not establish a positive average effect across targets. They also do not establish equivalence to no-op. CLINC150 bundle 08 remains unused.

Configuration	$\Delta OD_F$	$\Delta OD_G$	$P_F/P_G$
Frozen transfer, b01/s17	-0.191	-0.237	-0.46/-0.67
Tuned, b02/s17	0.065	0.088	0.22/0.28
Tuned, b02/s42	0.062	0.040	0.18/0.11
Tuned, b03/s17	0.102	0.083	0.22/0.20
Tuned, b04/s17	0.061	0.033	0.13/0.08
Tuned, b05/s17	0.025	0.022	0.07/0.07
Tuned, b06/s17	-0.050	-0.020	-0.24/-0.09
Tuned, b07/s17	-0.071	-0.051	-0.24/-0.17
Pooled, 48 intents	0.022	0.022	Not pooled
Opaque-label contrasts: same tuned configuration, seed 17
Opaque, b02	0.275	0.287	0.28/0.29
Opaque, b03	0.132	0.111	0.13/0.11
Opaque, b06	0.023	0.024	0.02/0.02

Table 3. CLINC150 transfer and label-semantic contrasts. Positive row-level gains in selected bundles do not establish a pooled gain across intents. All methods in this table are SimNPO on Qwen3.5-0.8B. Bundle-wise intervals resample rows; the pooled row resamples intents after averaging seeds within an intent. Opaque contrasts are exploratory and are excluded from that pooled semantic-label estimate. Their $D_{0,F}$ values are 0.983, 0.993, and 0.999, versus 0.298, 0.454, and 0.207 with semantic labels. Bundle 08 remains reserved.

Suppression magnitude is insufficient to characterize fidelity

Let $C_F \subset C$ be the forgotten classes and $m_M(x) = \sum_{c \in C_F} p_M(c \mid x; S)$ their total probability mass. The suppression ratio

s = \frac{\sum_{x \in E_F \cup E_G} [m_{M_{\mathrm{full}}}(x) - m_{M_U}(x)]}{\sum_{x \in E_F \cup E_G} [m_{M_{\mathrm{full}}}(x) - m_{M_O}(x)]} \tag*{(6)}

compares the amount removed with the oracle’s amount. It is distinct from $P_v$ , which uses the complete distribution.

Configurations selected to approximately match suppression on development data differ on held-out data (Figure 4).

Suppression and distributional fidelity across held-out suppression ratios and oracle-aligned diagnostics — Figure 4. *Figure 4.* Suppression and distributional fidelity. (A) Held-out suppression $s$ against $P_F$ , with open markers locating development suppression for the same configuration and dotted horizontal guides showing drift. The vertical coordinate of each open marker remains the held-out progress; it is not a development-performance measurement. Intervals are conditional row-bootstrap 95% intervals. (B) Pooled oracle-aligned coefficient and relative residual from per-row projections. Matching is approximate; infeasible configurations remain visible. The projection identity is exact for squared L2, not JS.

On BANKING77, SimNPO, GradAscent, and RMU yield $s = 0.78$ , 0.68, and 0.72, respectively, but $P_F = 0.56$ , 0.55, and 0.04. Therefore, similar removal of forgotten-option mass does not ensure similar oracle fidelity. This comparison is approximate, since development matching does not enforce identical held-out suppression.

To describe distributional movement, define per-row changes $\Delta_U = \mathbf{p}_{M_U} - \mathbf{p}_{M_{\mathrm{full}}}$ and $\Delta_O = \mathbf{p}_{M_O} - \mathbf{p}_{M_{\mathrm{full}}}$ . Their projection gives

\begin{aligned} a &= \frac{\langle\Delta_U,\Delta_O\rangle}{\lVert\Delta_O\rVert^2},\\ r &= \Delta_U-a\Delta_O,\\ \lVert\Delta_U-\Delta_O\rVert^2 &= (a-1)^2\lVert\Delta_O\rVert^2+\lVert r\rVert^2. \tag*{(7)} \end{aligned}

The identity describes squared Euclidean error, not a decomposition of JS divergence. RMU’s pooled aligned coefficient is 0.40, compared with 0.67 for matched SimNPO. These diagnostics support a distinction between suppression and movement toward the oracle, and they do not identify a causal mediator.

On CLINC150, three settings near $s = 1$ on development data yield held-out ratios of 1.93, 1.61, and 1.18, with corresponding $P_F = -0.39$ , $-0.17$ , and 0.17. Oversuppression and declining fidelity co-occur in this comparison. Increasing update strength is consequently not a general correction for weak progress.

Recovery depends on the starting checkpoint and the metric

Recovery uses BANKING77 bundle 01, training seed 17, with $k \in\{1, 5, 20\}$ forgotten training examples per class and two adaptation seeds. We adapt the decision head and LoRA parameters (Hu et al., 2022) under matched settings and evaluate at $t \in\{0, 1, 5, 10, 25\}$ updates on 160 unseen rows from four forgotten classes. Let $\overline{p}_M(t)$ be the mean gold probability. Normalized trapezoidal area is $A_M = \sum_t w_t\overline{p}_M(t)$ , with weights $(0.02, 0.10, 0.18, 0.40, 0.30)$ .

Define

\begin{aligned} G_{\mathrm{abs}} &= A_{M_U} - A_{M_O},\\ G_{\mathrm{adj}} &= [A_{M_U} - \bar{p}_{M_U}(0)] - [A_{M_O} - \bar{p}_{M_O}(0)],\\ H_M &= \frac{A_M - \bar{p}_M(0)}{1-\bar{p}_M(0)}, \qquad G_H = H_{M_U} - H_{M_O}. \tag*{(8)} \end{aligned}

Hence $G_{\mathrm{abs}} = \Delta\bar{p}(0) + G_{\mathrm{adj}}$ , where $\Delta\bar{p}(0) = \bar{p}_{M_U}(0) - \bar{p}_{M_O}(0)$ . Headroom normalization is a secondary sensitivity analysis and does not identify an intrinsic learning rate.

Apertus-8B starts below its oracle (0.326 versus 0.429) but has positive baseline-adjusted recovery in every configuration, with pooled $G_{\mathrm{adj}} \approx0.16$ [0.12,0.20]. The advantage also survives headroom normalization. Its adjusted recovery exceeds a retained label-swap control by approximately 0.18 [0.14,0.22], with a class-bootstrap interval of [0.08,0.27] (Figure 5). This specificity is relative to a control that overrides an existing label mapping and starts near zero gold probability.

Recovery relative to retraining across update steps, exposure budgets, model checkpoints, adaptation seeds, area contrasts, adjusted recovery, and fresh-head initializations — Figure 5. *Figure 5.* Recovery relative to retraining. (A) Gold probability by update, exposure budget, checkpoint, and adaptation seed. (B) Absolute, baseline-adjusted, and headroom-normalized area contrasts; configuration dots and forgotten-class diamonds are descriptive, with pooled row and class intervals. (C) Adjusted recovery on unseen rows, exposure-excluded seen rows, and the retained label-swap control, plus direct contrasts. (D) Three fresh-head initializations per frozen backbone. Repeated evaluations share the same trained checkpoints, so the intervals are conditional on those checkpoints. Qwen3.5-9B has no control arm.

For Qwen3.5-0.8B, pooled adjusted recovery is approximately 0.04 [0.00,0.09] and its contrast with the label-swap control is -0.02 [-0.06,0.03]. The controls therefore do not establish forgotten-target specificity for this model. Neither control model shows a clear seen-row advantage over unseen rows; this is not an equivalence result.

Qwen3.5-9B starts above its oracle (0.614 versus 0.477). Its positive absolute area contrast coexists with a negative adjusted contrast of approximately -0.05 and a positive headroom-normalized contrast of approximately 0.10. Its apparent recovery advantage therefore depends on the endpoint. No specificity controls were run for this model.

Finally, new heads trained on retained classes atop frozen backbones give higher forgotten-class accuracy point estimates for unlearned classes than oracle backbones. Initialization-level variation and broad class intervals limit this evidence. Because semantic labels remain available, the experiment measures transfer through the frozen representations under this interface, and it does not demonstrate extractable deleted records.

Discussion and Limitations

The results support evaluating unlearning as an approximation to a specified counterfactual, including the decision schema. The oracle may retain substantial recognition through semantic labels; changing the offered alternatives can alter that recognition without changing model weights. Conversely, opaque labels can eliminate oracle recognition while leaving the unlearned model poorly aligned under a reduced option set. Therefore, a fixed-schema score does not establish that unlearning is robust to schema.

Three distinctions prevent stronger conclusions than the evidence supports. Statistical acceptance does not guarantee substantial progress, as illustrated by opaque CLINC150 bundle 06. A broad single-seed screen does not establish reliable transfer across training seeds or deletion targets. Higher recovery after adaptation does not, by itself, indicate faster learning, particularly when initial gold probabilities differ. The probability-geometry analysis is descriptive, and imperfect suppression matching does not isolate a unique mechanism.

The study is limited to two intent-classification datasets and supervised task adaptation from pretrained backbones. It does not establish erasure from pretraining, behavior in open-ended generation, or guarantees against privacy attacks. Granularity comparisons change the membership and organization of deleted rows and do not isolate granularity from every data-composition effect. A single label-code map per dataset cannot separate all consequences of changed label representations and training history. The architecture panel is purposively selected, recovery uses one trained seed per model, and class-bootstrap sensitivity is coarse with four forgotten classes. The retained label-swap control differs from forgotten-target adaptation in its initial difficulty and objective. Finally, checkpoint-level summaries do not recover the distribution over all possible retrained models.

Conclusion

We introduced DecisionUnlearn-Bench to examine how deletion changes probabilistic decision models and how well approximate unlearning reproduces those changes. Deletion granularity, label meaning, and candidate composition all affect the retraining target. SimNPO provides reproducible reference-model gains on BANKING77, but those gains do not transfer uniformly across seeds, targets, or datasets. Suppression and recovery diagnostics further show why low forgotten-class accuracy and high post-adaptation performance are insufficient summaries. Reporting the schema-specific oracle, absolute and normalized fidelity, retained utility, and controlled recovery yields a more precise account of what decision-model unlearning achieves.

Impact Statement

This work studies how to evaluate the removal of supervised training influence from probabilistic decision models. More

accurate evaluation can help avoid unsupported claims that a model has forgotten private information. The experiments use intent-classification benchmarks and do not establish a privacy guarantee, successful record extraction, or the removal of knowledge acquired during pretraining. Recovery analyses are intended to characterize limitations of behavioral evaluation. Any release should respect the licenses of the datasets and base models and distinguish empirical fidelity results from guarantees about individuals’ data.

Notation and Scope of the Retraining Reference

Throughout the manuscript, $\mathcal{D}_F$ is deleted training data, $\mathcal{D}_R$ is retained training data, and $\mathcal{E}_F$ , $\mathcal{E}_G$ , $\mathcal{E}_R$ are evaluation views. A deletion bundle is a fixed collection of target units, not a collection of independently trained models. Training seeds index the construction of full and retain-only checkpoints; adaptation seeds index subsequent recovery runs; head seeds index fresh-head initialization. These sources of variation are not interchangeable.

The full model $M_{\mathrm{full}}$ and oracle $M_O$ are both derived from the same pretrained initialization under the task-training procedure. The oracle is a reference for deleting supervised adaptation data. It may retain relevant pretrained information and information inferred from the retained task data or the option descriptions. Its residual recognition is therefore compatible with the specified retraining counterfactual. It is not, by itself, evidence that the oracle was trained on $\mathcal{D}_F$ .

The schema $S = (C, \ell)$ includes both candidate membership and the textual information supplied about candidates. We suppress $S$ in formulas when all models being compared use the same schema. An oracle trained with opaque labels is a different checkpoint from the semantic-label oracle. By contrast, changing the candidate set at evaluation reuses checkpoints and supplies them with a different schema.

The supplied plots use $D_0$ , $P_F$ , $P_G$ , $s$ , $a$ , and $G_{\mathrm{adj}}$ ; these symbols have the same meanings here. Source-table strings dOD_F and dOD_G are rendered as $\Delta$ ODF and $\Delta$ ODG. A change in retained accuracy is reported in percentage points when explicitly marked pp; probability and ECE differences are on the $[0, 1]$ scale.

Data, Target Construction, and Experimental Coverage

Datasets and deletion units

BANKING77 contains 77 banking intents ([2]). CLINC150 contains 150 in-scope intents, with an additional out-of-scope option in our interface ([8]). The benchmark uses the existing intent annotations. No inference about a person’s identity or private attribute is an evaluation target in this study.

Example deletion removes 5% of training rows. Association deletion removes sub-intent clusters from selected parent intents, leaving other training examples of those intents. The primary BANKING77 association bundle contains ten clusters, whereas the supplied replication tables contain six and eight clusters in bundles 02 and 03, respectively. Class deletion removes all training rows associated with the selected class. BANKING77 class bundles contain four classes, whereas CLINC150 class bundles contain eight intents.

Target eligibility is evaluated on full-model development predictions. Class targets require accuracy of at least 0.70 and gold probability of at least 0.50, whereas association targets have a 0.65 accuracy threshold. Selection proceeds in hash order among eligible targets at the primary seed. Targets are not reselected to make subsequent seeds favorable. This produces one documented exception, where the semantic Qwen3.5-0.8B at seed 42 has a development accuracy of 0.65 on transfer_timing in the BANKING77 bundle 02. The corresponding opaque configuration passes its acquisition check.

Evaluation populations

For the primary BANKING77 class bundle, $\mathcal{E}_F$ contains 428 deleted training rows and $\mathcal{E}_G$ contains 160 unseen rows. The reference model’s second bundle has 382 deleted training rows. Row counts can differ across model-specific bundle-02 records. Therefore, a shared bundle label is not sufficient evidence that every architecture used identical deleted rows. The supplied results preserve the observed counts.

For class deletion, $\mathcal{E}_G$ comprises unseen rows of the deleted classes. Association-level generalization uses unseen rows of the deleted clusters. At the example level, evaluation rows are associated with deleted training rows through a nearest-training-row relation. Retained utility is computed on retained evaluation rows from the same configuration.

Observed coverage and stages

The 20-model granularity panel contains 120 full/oracle evaluations across three deletion requests. These are evaluation records, not necessarily 120 separately trained checkpoints, because full models may be reused. The 21st configured model, InternLM3-8B, was excluded due to a checkpoint-reload reproducibility issue.

The class-method confirmation uses Qwen3.5-0.8B, seed 17, bundle 01. Four methods continue to nine seed–bundle cells, with seeds 17, 42, and 99 and class bundles 01–03. The broad frozen-setting panel contains the reference model plus 19 additional model configurations. Four additional models join the reference in the targeted seed/bundle comparison: Apertus-8B, OLMo-3-7B, Qwen3-4B, and Qwen3.5-9B.

The label study contains seven semantic/opaque pairs: BANKING77 bundles 01 and 02 at seeds 17 and 42, and CLINC150 bundles 02, 03, and 06 at seed 17. The opaque bundle-06 result appears in the detailed tables despite a stale not-run entry in the supplied design table. CLINC150 bundle 08 remains reserved. The candidate-set study additionally includes opaque BANKING77 bundles 01 and 02.

The run registry contains 358 displayed records. Its footer reports 1,043 evaluation records in the wider collection, including 646 development-sweep candidates, eight development-reference records, and 31 synthetic fixtures. The omitted records are not independent experimental replications. Table S1 summarizes the registry, and its CSV companion preserves every displayed row, including shortened keys that are not globally unique. The complete source paths and granularity fields must accompany any public machine-readable release.

Dataset	Labels	Recorded stage	Evaluations
banking77	opaque	exploratory	12
banking77	semantic	confirmation	6
banking77	semantic	confirmation_sensitivity	29
banking77	semantic	exploratory	14
banking77	semantic	pilot	15
banking77	semantic	reference	170
banking77	semantic	replication	40
banking77	semantic	screening	19
clinc150	opaque	exploratory	9
clinc150	semantic	confirmation	2
clinc150	semantic	control	3
clinc150	semantic	exploratory	3
clinc150	semantic	reference	24
clinc150	semantic	replication	5
clinc150	semantic	transfer	7

Table S1. Observed evaluation registry, summarized by dataset, label scheme, and recorded stage.

Model Training, Methods, and Configuration Reporting

The study compares probabilistic decision checkpoints through their complete option distributions. Approximate unlearning begins from the full checkpoint, whereas the oracle is constructed by applying the task-training procedure to the retained data from the shared pretrained initialization. Recovery explicitly updates the decision head and LoRA parameters; fresh-head analysis freezes the corresponding backbone and replaces the head.

GradAscent reverses the supervised objective on deleted examples. GradDiff combines forgetting pressure with retained-data supervision, and GradDiff-KL uses a retained-distribution regularizer. NPO and SimNPO use preference-inspired forgetting objectives [11, 4]. RMU intervenes on representations (?). Uniformization is an erasure control. These descriptions identify the evaluated method families, and their exact decision-probability losses and implementation-specific normalization must match the released code.

The CLINC150 tuning figure identifies the selected configuration as learning rate $3 \times10^{-5}$ , $\beta= 10$ , retained-objective weight $\alpha= 0.5$ , and 10 epochs. Its development objective is approximately 0.641, versus 0.945 for no-op. The selected point lies on the tested grid boundary. The result is reported as the selected tested setting, without asserting that it is an optimum beyond the grid. Dataset tuning used the development bundle, and the subsequent semantic-label evaluations use frozen settings.

Recovery uses learning rate $10^{-4}$ , three exposure budgets, and two adaptation seeds. Conditional seed-42 strength diagnostics use learning rate $6 \times10^{-5}$ on Apertus-8B and Qwen3.5-9B. The exact frozen BANKING77 selection and all full/oracle/head-training settings are referenced by a configuration CSV in the original asset notes, but that CSV is absent from this archive.

Metrics and Statistical Analysis

Distributional metrics

For distributions $\mathbf{p}$ , $\mathbf{q}$ and $\mathbf{m} = (\mathbf{p} + \mathbf{q})/2$ ,

\mathrm{JS}_2(\mathbf{p}, \mathbf{q}) = \tfrac{1}{2}\mathrm{KL}_2(\mathbf{p}\Vert\mathbf{m}) + \tfrac{1}{2}\mathrm{KL}_2(\mathbf{q}\Vert\mathbf{m}). \tag*{(9)}

Distances lie between zero and one. Numerical stabilization uses $\epsilon= 10^{-12}$ . Oracle distance is an average of row divergences. Normalized progress is a ratio of aggregate distances, not an average of row-wise progress ratios. Each schema and label condition uses its own full/oracle denominator.

Retained accuracy uses the model’s highest-probability option. ECE compares confidence with empirical correctness within bins. The standard form is

\mathrm{ECE}_{\mathrm{R}} = \sum_{j=1}^{J} \frac{\lvert B_j\rvert}{\lvert\mathcal{E}_{\mathrm{R}}\rvert}\left\lvert\operatorname{Acc}(B_j) - \operatorname{conf}(B_j)\right\rvert. \tag*{(10)}

The specific binning procedure is part of the implementation and must be documented; no new calibration transformation is applied to the JS probabilities for this draft.

Within-configuration inference

Paired resampling keeps full, unlearned, and oracle predictions aligned for the same evaluation rows. The default analysis uses 5000 bootstrap replicates and percentile 95% intervals. Class- or cluster-level resampling assesses sensitivity to the unit of deletion. With only four forgotten classes in BANKING77, these sensitivity intervals have limited resolution and do not characterize arbitrary new class selections.

Both-view acceptance is an intersection criterion: both $\Delta\mathrm{OD}_{\mathcal{F}}$ and $\Delta\mathrm{OD}_{\mathcal{G}}$ must have positive point estimates and positive lower bounds, and the retained-utility constraints must hold. Positive, unconfirmed denotes positive point gains with at least one nonpositive lower bound. No benefit and infeasible are retained as distinct reported outcomes. Statistical acceptance is not an equivalence test and does not require $P_v$ to exceed a practical-effect threshold.

The frozen confirmation family reports a one-sided intersection-union $p$ -value formed from the larger of the two view-specific values, followed by Holm adjustment across the candidate family. Screening decisions are uncorrected across model configurations. The reported $p$ -values and intervals retain their original roles, and we do not recompute them from rounded table entries.

Cross-bundle summaries

Reference-model Phase 2 uses stored hierarchical resampling over seeds, bundles, and deletion units. A method replicates under the protocol only if the aggregate lower bound is positive on both views, feasibility holds in every observed cell, and the result survives the specified leave-one-seed/bundle-out checks. Positive aggregate distance gains can therefore coexist with a failed replication decision.

For CLINC150, the pooled analysis averages seeds within each intent and considers 48 intents from six bundles. The intent-level estimate is not formed by selectively pooling successful bundles. On $\mathcal{F}$ , the reported intent-bootstrap interval is [-0.045,0.090], and the bundle-then-intent interval is [-0.054,0.107] around the same point gain of 0.022. Both retain substantial uncertainty. Mean per-intent progress is omitted because some displacement denominators are small.

Complete Method and Transfer Results

Table S3 retains all 150 displayed intervention/control rows, including pilots and infeasible settings. The pilot configurations use default settings and must not replace selected confirmation configurations in the main comparison. In particular, example-deletion pilot results exist even though no example-deletion method advanced through selection to confirmation. The supplied main-table note saying that no method was run is too broad.

The final association-level sensitivity results also require separate treatment. SimNPO has modest positive cross-bundle gains and satisfies the stored replication rule. GradDiff does not establish a positive $\mathcal{G}$ aggregate. Uniformization is reported as a control without a replication decision. Association bundles have different numbers of clusters, and individual seed–bundle results remain visible.

The architecture results in Table S4 include both views, class sensitivity intervals, and observed seed/bundle differences. The reference configuration is included in the total of 20. Thus, 19 accepted configurations correspond to the reference plus 18 of 19 additional configurations. This is a descriptive count for the selected panel, not a probability that an arbitrary model will satisfy the criterion.

Table S7 preserves the CLINC150 comparisons and controls. Its source omits the deletion-granularity field, leaving two visibly identical Qwen3.5-0.8B bundle-01 SimNPO keys with different denominators (0.058 and 0.416). The complete results table distinguishes their unit counts (18 versus 8) and row populations. We retain both records with explicit row identifiers and do not merge them. The main class-deletion transfer result is the eight-intent record with $D_{0,\mathcal{F}} = 0.416$ .

Label and Candidate-Schema Interventions

Opaque rendering replaces every option name and description with a fixed code. The original map contains 77 codes for BANKING77 and 151 for CLINC150, including out-of-scope. Input utterances, target identities, option identifiers, and option order are unchanged. Forgotten options remain present, and their logits are not masked. The intervention is therefore a change in the information supplied by labels, together with the corresponding full/oracle training under that rendering.

Table S5 reports acquisition, oracle accuracy, forgotten-option mass, displacement, and progress. Rounded zero accuracy does not imply exactly zero option probability: scientific notation is retained for the mass values. The semantic/opaque effect is strong for oracle recognition, but SimNPO’s normalized progress varies with bundle and dataset. A single fixed code map per dataset leaves code-assignment variability untested.

Candidate schemas are fixed within a bundle. $S_0$ supplies all options. $S_1$ supplies the forgotten classes and two retained semantic neighbors per forgotten class, and $S_2$ supplies the forgotten classes with the same number of random retained distractors. Observed subset cardinalities are 12 for BANKING77 and 24 for CLINC150. The exact membership lists and selection code should accompany the release.

All models are rescored under the reduced input schema. This is not slicing and renormalizing a stored full-schema vector. Table S6 reports both views and the new denominator. For example, semantic CLINC150 bundle 02 has $D_{0,F}=0.039$ with random distractors, compared with 0.298 under the full schema. The smaller denominator changes the interpretation and uncertainty of normalized progress.

Suppression and Probability Geometry

The suppression ratio aggregates forgotten-option mass over the relevant rows before dividing. A value $s=1$ matches the oracle’s net amount of removed mass; it does not specify which retained options receive that mass or whether individual forgotten classes are suppressed similarly. A value above one indicates greater net mass removal than the oracle.

Development matching uses target $s=0.6$ on BANKING77 and $s=1$ on CLINC150. Table S8 preserves the observed development and held-out values, including configurations whose utility gate fails on held-out data. Uniformization has no feasible development checkpoint near the matched target and is absent from that comparison. The matched SimNPO configuration is distinct from the main confirmation checkpoint, so its $P_F=0.56$ should not be substituted for the main confirmation value 0.59.

For row $i$ with $\Delta_{O,i}\ne0$ , orthogonality gives

\langle r_i,\Delta_{O,i}\rangle=0,\quad\lVert p_{U,i}-p_{O,i}\rVert^2=(a_i-1)^2\lVert\Delta_{O,i}\rVert^2+\lVert r_i\rVert^2. \tag*{(11)}

Summing these per-row identities is exact. The pooled coefficient displayed in the figures is

\bar{a}=\frac{\sum_i\langle\Delta_{U,i},\Delta_{O,i}\rangle}{\sum_i\lVert\Delta_{O,i}\rVert^2},\quad\rho_r=\sqrt{\frac{\sum_i\lVert r_i\rVert^2}{\sum_i\lVert\Delta_{O,i}\rVert^2}}. \tag*{(12)}

The pooled $\bar{a}$ must not replace every $a_i$ in the exact summed decomposition. The matched-suppression table reports identity errors at or below $2.0\times10^{-16}$ and no rows with zero oracle displacement. The wider diagnostic figure includes additional runs with a broader numerical error range.

The magnitude-only and residual-only JS diagnostics use probability vectors that are projected back onto the simplex. This projection and the nonlinear JS metric prevent interpreting them as an additive JS decomposition. Correlations between these diagnostics and progress also share underlying probability vectors. They are descriptive associations, not independent causal evidence about the optimization mechanism.

Recovery, Controls, and Fresh Heads

Adaptation and numerical summaries

Full, SimNPO, and retrained checkpoints are adapted using the same exposure examples, update schedule, and evaluation rows within each recovery configuration. Exposure budgets are $k=1,5,20$ rows per forgotten class, and adaptation seeds are 0 and 1. Evaluation occurs at updates 0,1,5,10,25. Trapezoidal integration divided by the horizon of 25 yields weights 0.02,0.10,0.18,0.40,0.30. Consequently, each area has the same probability scale as a time-averaged gold probability.

Table S9 provides all 18 summaries across model, budget, and adaptation seed. Its configuration-level identity residuals are at most approximately $10^{-16}$ . The six settings for each model share the same underlying trained checkpoints and four forgotten classes. They are not six independent replications of model training.

Shared row resampling preserves dependence across checkpoints, time points, budgets, and adaptation seeds wherever the same evaluation row is reused. Class intervals condition on the four tested forgotten classes and the reused backbones.

Threshold times are the first evaluated update whose mean gold probability reaches the threshold; they are grid-based observations, not interpolated continuous crossing times. An unreached threshold is right-censored at 25 updates.

Starting point and headroom

Absolute area contains both the initial probability difference and the integrated difference in subsequent gains. Subtracting the initial value removes that additive offset. Headroom normalization instead asks what fraction of $1 - \bar{p}(0)$ is recovered. The principal headroom analysis uses a ratio of aggregate means; averaging row-level ratios yields a different estimand and can place large weight on rows starting near one. No equivalence between these aggregations is assumed.

Apertus-8B’s adjusted gains are positive across all configurations. Qwen3.5-9B’s adjusted gains are negative for every configuration, whereas its headroom point contrasts are positive. This metric dependence is part of the result rather than a basis for selecting one favorable summary. Neither adjustment alone identifies a model-intrinsic learning rate.

Specificity and exposure controls

Controls are available for Qwen3.5-0.8B and Apertus-8B. For an evaluation population $q$ , let

R_q = [A_{U,q} - \bar{p}_{U,q}(0)] - [A_{O,q} - \bar{p}_{O,q}(0)]. \tag*{(13)}

The specificity contrast is $R_{\mathrm{unseen}} - R_{\mathrm{swap}}$ ; the exposure contrast is $R_{\mathrm{seen}} - R_{\mathrm{unseen}}$ . These are direct differences of paired advantages, not comparisons of whether separate intervals include zero.

In the retained label-swap arm, a retained class $X$ is relabeled as $Y$ and adapted using the same budget and schedule. This arm starts with a near-zero target probability and reverses a previously correct mapping. Therefore, a positive specificity contrast establishes a difference relative to this particular control, not exclusion of all forms of general post-unlearning plasticity.

Seen evaluation rows belong to the original deletion set but are excluded from the new exposure. Figure S12 reports 4 exposed and 424 remaining seen rows for $k = 1$ , 20 and 408 for $k = 5$ , and 80 and 348 for $k = 20$ , in both control models and adaptation seeds. The supplied audit reports verified disjointness. With no clear positive seen-minus-unseen advantage, these experiments do not establish a record-specific recovery effect; they also do not prove the absence of such an effect.

Recovery controls and exposure exclusion. Top panels: Qwen3.5-0.8B and Apertus-8B, with y-axis labels specificity, $k = 1$; specificity, $k = 5$; specificity, $k = 20$; specificity, pooled; seen − unseen, $k = 1$; seen − unseen, $k = 5$; seen − unseen, $k = 20$; seen − unseen, pooled. The x-axis is contrast in baseline-adjusted area. Exposure exclusion audit: Qwen3.5-0.8B: verified; k1_s0: 4 exposed / 424 seen; k1_s1: 4 exposed / 424 seen; k5_s0: 20 exposed / 408 seen; k5_s1: 20 exposed / 408 seen; k20_s0: 80 exposed / 348 seen; k20_s1: 80 exposed / 348 seen. Apertus-8B: verified; k1_s0: 4 exposed / 424 seen; k1_s1: 4 exposed / 424 seen; k5_s0: 20 exposed / 408 seen; k5_s1: 20 exposed / 408 seen; k20_s0: 80 exposed / 348 seen; k20_s1: 80 exposed / 348 seen. Bottom panels: forgotten, unseen; forgotten, seen; label-swap control. The x-axis is starting gold probability p(0), and the y-axis is cumulative share of rows. Legend: SimNPO, oracle, Qwen3.5-0.8B, Apertus-8B. — Figure S12. **Recovery controls and exposure exclusion.** For Qwen3.5-0.8B and Apertus-8B, forgotten-unseen recovery is contrasted directly with retained label-swap recovery, and seen forgotten rows are contrasted with unseen rows. Exposure rows are excluded from seen-row evaluation: the remaining counts are 424, 408, and 348 for $k = 1, 5, 20$ . Shared bootstrap draws preserve pairing across conditions. Apertus’s adjusted advantage exceeds this control, while Qwen3.5-0.8B’s does not clearly do so. The control begins near zero gold probability and changes a correct mapping; it is not matched on starting difficulty.

Fresh-head transfer

The fresh-head study freezes each backbone and trains a newly initialized decision head on retained-class rows, with three matched head initializations. The full backbone is a reference ceiling, and the oracle backbone is the counterfactual comparison. Figure S13 reports forgotten-class accuracy, gold probability, and retained accuracy separately for every initialization.

Three-panel scatter plot showing forgotten-class accuracy, gold probability, and retained accuracy for Qwen3.5-0.8B, Apertus-8B, and Qwen3.5-9B — Figure S13. *Fresh heads on frozen representations.* Forgotten-class accuracy, gold probability, and retained performance are reported for three fresh-head initializations trained on retained-class examples only. Each initialization is shown separately. Across-initialization ranges are not confidence intervals. Reported row-pooled accuracy differences are borderline, and class intervals include zero. Recognition under semantic labels demonstrates transfer from frozen representations under the tested readout; it does not demonstrate extraction of deleted training records.

Higher forgotten-class recognition with an unlearned backbone can indicate distinct transfer properties, while semantic option labels remain available. The experiment does not isolate storage of individual deleted records. The supplied caption states that pooled row intervals exclude zero, whereas class intervals do not; the exact pooled fresh-head and control tables are not included in the archive, so the manuscript does not invent additional numerical interval bounds.

Optimization Diagnostics and Conditional Strength Reruns

The seed comparison reveals several distinct patterns (Table S10). For Apertus-8B, suppression on $\mathcal{F}$ falls from 1.07 to 0.39 between seeds 17 and 42; for Qwen3.5-9B it falls from 0.57 to 0.15. Their progress also falls. In OLMo-3-7B, suppression is much more similar (0.38 versus 0.35), while progress decreases from 0.34 to 0.04. For Qwen3-4B, suppression is 0.60 versus 0.61, but progress changes from 0.28 to -0.18. Therefore, insufficient suppression is not a complete explanation for all seed effects.

Model / seed	$P_{\mathrm{F}}/S_{\mathrm{F}}$	Full gold p / confidence / margin	Oracle accuracy	Forget loss: first / last 10	Retain loss: first / last 10	Gradient norm: first / last 10	Source
qwen3.5-0.8b seed 17	+0.59 / 0.96	0.967 / 0.977 / 0.959	0.27	0.291 / 0.082	0.057 / 0.146	0.822 / 1.314	recorded training log
qwen3.5-0.8b seed 42	+0.55 / 0.86	0.955 / 0.971 / 0.947	0.32	0.277 / 0.103	0.170 / 0.180	0.952 / 1.483	recorded training log
apertus-8b seed 17	+0.55 / 1.07	0.934 / 0.956 / 0.920	0.39	0.268 / 0.053	0.131 / 0.214	0.448 / 0.678	recorded training log
apertus-8b seed 42	+0.17 / 0.39	0.943 / 0.970 / 0.947	0.41	0.266 / 0.116	0.172 / 0.264	0.564 / 0.622	recorded training log
qwen3.5-9b seed 17	+0.54 / 0.57	0.962 / 0.976 / 0.958	0.48	0.290 / 0.178	0.073 / 0.109	0.504 / 1.096	recorded training log
qwen3.5-9b seed 42	+0.14 / 0.15	0.969 / 0.988 / 0.979	0.42	0.285 / 0.251	0.150 / 0.141	0.669 / 1.038	recorded training log
olmo3-7b seed 17	+0.34 / 0.38	0.956 / 0.969 / 0.945	0.35	0.283 / 0.196	0.063 / 0.135	0.322 / 0.532	recorded training log
olmo3-7b seed 42	+0.04 / 0.35	0.962 / 0.979 / 0.964	0.45	0.276 / 0.156	0.136 / 0.193	0.395 / 0.603	recorded training log
qwen3-4b seed 17	+0.28 / 0.60	0.955 / 0.968 / 0.941	0.58	0.283 / 0.198	0.069 / 0.168	0.445 / 1.039	recorded training log
qwen3-4b seed 42	-0.18 / 0.61	0.953 / 0.970 / 0.945	0.56	0.271 / 0.120	0.174 / 0.211	0.547 / 0.910	recorded training log

Table S10. Seed-17 and seed-42 optimisation diagnostics (frozen class SimNPO, BANKING77 bundle 01).

The recorded training logs provide forget loss, retain loss, and gradient norms. Initial gradient norms are not uniformly smaller for the weaker seed. Parameter-update norms were not recorded, and the figures do not reconstruct a historical trajectory that was never saved.

Conditional stronger-update reruns are exploratory. At learning rate $6 \times10^{-5}$ , the best displayed Apertus checkpoint reaches approximately $P_{\mathcal{F}} = 0.42$ , whereas the Qwen3.5-9B trace reaches only approximately 0.19 and later oversuppresses. The displayed best epochs were chosen after observing these traces; they do not replace the frozen-transfer estimates or establish a validated adjustment rule.

Compute, Integrity, and Reproducibility

Table S11 reproduces logged GPU-hour summaries by evaluation category. These totals are partial: full-model evaluations record zero and do not include the training time stored with the full checkpoint. The table also does not establish complete recovery-job accounting. We therefore do not report the sum as the study’s total training cost or infer an unlearning speedup over full retraining.

Experiment	Stage	Evaluations	GPU hours (logged)	Models
architecture_replication	replication	8	40.7	4
architecture_screen	screening	19	97.8	19
association_methods	confirmation_sensitivity	29	23.7	1
clinc150_frozen_transfer	transfer	7	9.6	2
clinc150_tuned	confirmation	2	5.9	1
clinc150_tuned	control	3	4.5	1
clinc150_tuned	replication	5	25.1	1
counterfactual_reference	reference	194	545.7	20
development_selection	development	646	705.9	1
development_selection	development_reference	8	17.6	1
label_semantics_banking77	exploratory	12	16.0	1
label_semantics_clinc150	exploratory	9	36.9	1
matched_suppression	exploratory	7	16.9	1
method_comparison	confirmation	6	6.0	1
method_comparison	replication	32	34.2	1
pilot_milestone	pilot	15	13.3	1
seed42_strength	exploratory	10	45.0	2
test_fixture	excluded	31	0.0	1

Table S11. Compute by experiment and stage, and frozen configurations.

The supplied audit reports exact recomputation of oracle distance for 358 evaluations and point-estimate agreement for 252 bootstrap comparisons. It reports identical intervals when the number of bootstrap draws matches, and differences of up to 0.0075 for stored summaries of 2000 versus 5000 draws, without category changes. These are reported upstream checks; the current manuscript package contains figure/table exports, not the raw probabilities needed to repeat them.

The audit also documents repaired filename collisions between semantic and opaque analyses and between training seeds, a reporting-label collision between GradDiff and GradDiff-KL, and recovery reruns after out-of-memory failures. No repaired or post hoc results are treated as additional independent replications. The separate S2-Rank privacy experiment stopped at its acquisition gate and did not test the proposed attack; it provides no evidence of privacy robustness.

Supplementary Figures

The following figures retain the supplied vector assets. Their captions distinguish supported conclusions from the limits of interpretation. Production identifiers S1–S15 correspond to the original asset grouping.

Supplementary Tables

The numerical entries below are transcribed from the supplied LaTeX exports. Formatting and explicit row identifiers have been added; no confidence interval is recalculated from rounded entries. Tables with many columns are divided into readable panels keyed by the same row identifier. The machine-readable transcriptions preserve the original fields.

Observed experimental coverage by model and experiment family — Figure S1. **Observed experimental coverage.** Cells indicate available evaluations by model and experiment family, including the reference-model replication grid, semantic/opaque comparisons, candidate-set rescoring, and recovery analyses. Coverage is purposive and incomplete across combinations; a filled cell does not imply independent replication of every factor. InternLM3-8B was excluded because checkpoint reload was not reproducible. The registry and machine-readable companion provide the available record keys.

Deletion granularity across eligible architectures, showing example deletion, association deletion, class deletion, and utility for 20 architectures — Figure S2. **Deletion granularity across eligible architectures.** Full-to-oracle displacement on deleted training rows (F), unseen related rows (G), and retained utility is shown for all 20 eligible architectures on BANKING77 at seed 17. Example, association, and class deletion differ in the scope of the target and the amount of task structure removed. The same-data noise reference is available only for Qwen3.5-0.8B. Displacement orders consistently by granularity; this is a statement about the tested decision distributions, not a privacy guarantee.

Four-panel scatter plot showing oracle progress and retained accuracy changes for BANKING77 and CLINC150 under PF and PG configurations — Figure S3. **Oracle progress and retained utility across evaluated configurations.** Each point is a recorded method configuration, with normalized progress plotted against retained-accuracy change. Calibration is an additional feasibility requirement even where only accuracy is plotted. Exploratory and screening points do not inherit confirmatory status. Extreme utility-loss runs are retained in the display; the complete table separates point estimates, uncertainty, and recorded acceptance category.

Label semantics and the retraining target across seven semantic/opaque pairs — Figure S4. **Label semantics and the retraining target.** The seven semantic/opaque pairs report acquisition, forgotten-class recognition and probability mass, displacement, and normalized SimNPO progress. All opaque full models pass the acquisition check. Opaque oracles have approximately zero forgotten-class recognition despite successful full-model acquisition. The semantic seed-42 BANKING77 bundle-02 model contains one class below the acquisition threshold; targets were selected at the primary seed and were not reselected. The CLINC150 bundle-06 opaque result is positive but small ( $P_F \simeq0.02$ ).

Figure S5. numbers under each schema: no-op displacement D0 on F rows (the denominator of P). **All evaluated candidate schemas.** Each checkpoint is re-scored under the original option set ( $S_0$ ), forgotten classes plus nearby retained alternatives ( $S_1$ ), and an equally sized set with random retained alternatives ( $S_2$ ). Oracle recognition, raw displacement, and method progress are shown for the available semantic and opaque conditions. Row-bootstrap intervals are conditional on the fitted checkpoints, and the ratio is recomputed in each replicate. Small displacement can make normalized progress sensitive; the underlying displacement must be read alongside the ratio.

Three-panel sensitivity plots comparing paired differences in P across architectures and a reference model across training seeds and bundles — Figure S6. *Figure S6.* **Seed and target-bundle sensitivity.** The purposive architecture panel contrasts seed 42 with seed 17 on bundle 01 and bundle 02 with bundle 01 at seed 17. The reference model additionally has a three-seed, three-bundle grid. Guides connect paired comparisons, not a temporal trajectory. Seed labels are identifiers. These sparse contrasts demonstrate sensitivity within an architecture but do not identify a population variance decomposition.

Two-panel forest plot of tuned SimNPO effects by intent bundle for ODF and ODG — Figure S7. *Figure S7.* **CLINC150 transfer and heterogeneity across intent bundles.** Frozen BANKING77 settings and development-tuned SimNPO are distinguished. Tuned effects vary in sign across held-out bundles, and the intent-pooled estimate includes zero. The reported fraction of intents with a positive point estimate does not imply a positive pooled effect. Bundle 08 remains unused. Intervals and the unit of pooling are described in Appendix § 14.

CLINC150 development tuning heatmap showing development ODF + ODG values across epochs and parameter settings — Figure S8. *Figure S8.* **CLINC150 development tuning.** The grid displays the development objective and diagnostics over evaluated settings. Selection uses development data, and the chosen setting is on the tested boundary ( $\beta= 10$ , 10 epochs, learning rate $3 \times10^{-5}$ , retain weight $\alpha= 0.5$ ). The grid was not extended. This landscape is a development diagnostic rather than held-out evidence; unavailable or infeasible cells should be interpreted using the source records.

Six-panel probability-redistribution diagnostics showing oracle-aligned coefficient, relative residual, Jensen–Shannon divergence of residual-only and magnitude-only counterfactuals, mean squared L2 error, and maximum identity error across runs — Figure S9. **Probability-redistribution diagnostics across recorded runs.** The oracle-aligned projection and off-direction residual describe Euclidean changes in the option-probability vector. Per-row orthogonality yields the exact squared-distance identity in Appendix § 17. The displayed all-run numerical-error diagnostic is distinct from the matched-suppression subset in Table S8; its precision range should not be replaced by that subset’s $2 \times10^{-15}$ bound. Simplex-projected magnitude-only and residual-only diagnostics are not an additive decomposition of Jensen–Shannon divergence.

Development matching and held-out suppression for BANKING77 and CLINC150 — Figure S10. Development matching and held-out suppression. Configurations were chosen on development data to approach suppression targets of 0.6 on BANKING77 and 1 on CLINC150, subject to feasibility. Their held-out suppression can differ substantially. Open development coordinates and held-out coordinates belong to the same selected configuration; matching is approximate. Held-out fidelity cannot be inferred from development suppression alone.

Figure S11. **Relearning for every exposure budget.** Forgotten-class accuracy and gold-probability changes are shown for $k \in\{1, 5, 20\}$ exposed deleted-training rows per class and adaptation seeds 0 and 1. Evaluation uses 160 unseen rows from four forgotten classes. Each curve uses one trained backbone seed; repeated budgets and adaptation seeds are not independent backbone replications. Starting probability, absolute recovery, adjusted recovery, and headroom-normalized recovery address different comparisons.

Optimization diagnostics for Qwen3.5-0.8B, Apertus-8B, OLMo-3-7B, Qwen3.5-9B, and Qwen3-4B, showing forget loss, gradient norm, and decision statistics for seeds 17 and 42 — Figure S14. **Optimization diagnostics for the seed-42 comparison.** Initial decision statistics, forget and retain losses, gradient norms, suppression, and progress are shown for paired model configurations. Apertus-8B and Qwen3.5-9B exhibit weaker suppression at seed 42, but OLMo-3-7B and Qwen3-4B lose fidelity despite similar suppression. No single insufficient-update explanation accounts for all models. Parameter-update norms were not recorded.

Figure S15. *Exploratory stronger-update reruns.* Additional traces examine whether changing update strength can recover progress in selected seed-42 failures. Apertus-8B improves at an observed intermediate checkpoint, while Qwen3.5-9B remains limited and later oversuppresses. Best displayed epochs were selected after inspecting these traces. These results motivate a prospective adjustment rule; they do not replace the frozen-setting replication or establish a validated repair.

The complete 358-row registry, including model, seed, bundle, run label, and probability hash, is preserved in data/S01_run_registry.csv. These are evaluations, not independent training replicates. Several displayed keys lack deletion granularity; source-row identifiers preserve distinct records. The upstream audit reports a broader total of 1,043 evaluations, including 646 development, 8 development-reference, and 31 fixture evaluations outside this registry.

Exported quantity	Definition	Upstream source (not included)
F view (E1, forget-train)	Rows of the deletion set D_F itself (training rows of the forgotten examples / association / classes).	src/evaluation/views.py
G view (E2, forget-gen)	Unseen evaluation-split rows of the forgotten class / association; at example level, test rows whose nearest training row is a forgotten row.	src/evaluation/views.py
Oracle distance OD_v	Mean over rows of the view of JS(p_model, p_oracle), base-2 Jensen-Shannon on raw (uncalibrated) option probabilities, eps 1e-12; $0 \leq\mathrm{OD} \leq1$ .	src/evaluation/oracle_distance.py
Displacement D0_v	OD_v of the full (no-op) model against the retrain oracle of the same seed and forget set.	record.json of ’full’
Gain dOD_v	D0_v - OD_v(method); positive = closer to the oracle than no-op; paired on rows.	PROTOCOL Amendment 2-3
Normalised progress P_v	$1 - \mathrm{OD}_v(\mathrm{method}) / \mathrm{D0}_v$ from aggregate distances (never a mean of row ratios); recomputed per bootstrap replicate.	PROTOCOL Amendment 3
Feasibility	Acc_R ≥ Acc_R(full) - 0.02 and ECE ≤ ECE(full) + 0.02 on retained rows of the same evaluation.	PROTOCOL Amendment 1-2
Acceptance (Phase 1 rule)	accepted: both gains > 0 with row-bootstrap L95 > 0, feasible; positive, unconfirmed: both gains > 0, a bound ≤ 0; otherwise no benefit / infeasible.	PROTOCOL Amendment 3

Table S2. Metric definitions, gates and uncertainty units.

Exported quantity	Definition	Upstream source (not included)
Multiplicity	Holm-adjusted intersection-union p (max of the two one-sided p) within the frozen candidate family of a confirmation; screening tests per model are uncorrected.	PROTOCOL Amendment 3; screen record
Bootstrap	5000 paired draws; rows (primary, seeds 1 F / 2 G) and forgotten classes or association clusters (sensitivity, seed + 100); percentile 95% CI; one-sided p from centred resamples.	scripts/confirm_bootstrap.py
Cross-bundle replication	Hierarchical bootstrap: seeds, then bundles, then units within bundle; replicates if aggregate L95 > 0 on both views, feasible in every cell, survives leave-one-bundle/seed-out.	PROTOCOL Phase 2
Noise floor	JS between same-data full models of different seeds (Qwen3.5-0.8B, seeds 17/42/99) on the same rows.	scripts/displacement.py
Acquisition gate	Each candidate forgotten class: full-model development accuracy ≥ 0.70 and gold probability ≥ 0.50 (association: accuracy ≥ 0.65); targets selected in hash order among eligible ones at the	src/datasets/targets.py
Suppression ratio s	Pooled forgotten-class mass removed by the method relative to the oracle over F and G rows: $\operatorname{sum}(m_\mathrm{full} - m_\mathrm{run}) / \operatorname{sum}(m_\mathrm{full} - m_\mathrm{oracle})$ .	scripts/matched_suppression.py
Projection a, residual r	Per row $\Delta_U = p_U - p_\mathrm{full}$ , $\Delta_O = p_\mathrm{oracle} - p_\mathrm{full}$ , $a = \frac{\Delta_U \cdot\Delta_O}{\lVert\Delta_O\rVert_2^2}$ ; $r = \Delta_U - a\Delta_O$ ; pooled $a = \frac{\sum\Delta_U \cdot\Delta_O}{\sum\lVert\Delta_O\rVert_2^2}$ ; relative residual $= \sqrt{\frac{\sum\lVert r\rVert^2}{\sum\lVert\Delta_O\rVert^2}}$ ; exact $L_2$ identity checked per row.	scripts/redistribution_decomposition.py
Recovery areas	Trapezoid over $t = 0, 1, 5, 10, 25$ updates, weights normalised to sum to one; absolute $A = \sum_t w_t p(t)$ ; adjusted $A_\mathrm{adj} = A - p(0)$ ; headroom $H = \frac{A_\mathrm{adj}}{1-p(0)}$ (ratio of means; mean of row ratios as a second aggregation).	scripts/recovery-uncertainty.py

Table S2 (PDF p. 33). Metric definitions, gates and uncertainty units (continued).

The manuscript uses $D_{0,v}$ , $\Delta\mathrm{OD}_v$ , and $P_v$ consistently for the exported D0, dOD, and P fields. File paths are provenance pointers from the source table, not files distributed with this draft.

ID	Configuration	$D_{0,F} / D_{0,G}$	$\Delta O_{DF}$ [rows]; unit CI	$\Delta O_{DG}$ [rows]; unit CI	$P_F/P_G$	Status
1	apertus-8b BANKING77 sem. s17 b01 SimNPO / screen	0.479 0.400	+0.263 [+0.226, +0.301] units: [+0.141, +0.434]	+0.231 [+0.164, +0.298] units: [+0.059, +0.464]	0.55 / 0.58	accepted
2	apertus-8b BANKING77 sem. s17 b02 SimNPO / replic.	0.565 0.552	+0.318 [+0.289, +0.347] units: [+0.108, +0.516]	+0.316 [+0.272, +0.361] units: [+0.100, +0.500]	0.56 / 0.57	accepted
3	apertus-8b BANKING77 sem. s42 b01 SimNPO / explor.	0.471 0.405	+0.196 [+0.153, +0.239] units: [-0.069, +0.514]	+0.184 [+0.111, +0.256] units: [-0.136, +0.498]	0.42 / 0.45	accepted
4	apertus-8b BANKING77 sem. s42 b01 SimNPO / explor.	0.471 0.405	+0.145 [+0.096, +0.195] units: [-0.091, +0.533]	+0.137 [+0.051, +0.222] units: [-0.145, +0.494]	0.31 / 0.34	accepted
5	apertus-8b BANKING77 sem. s42 b01 SimNPO / explor.	0.471 0.405	+0.115 [+0.085, +0.146] units: [-0.034, +0.293]	+0.121 [+0.071, +0.170] units: [-0.062, +0.240]	0.24 / 0.30	accepted
6	apertus-8b BANKING77 sem. s42 b01 SimNPO / explor.	0.471 0.405	+0.141 [+0.098, +0.187] units: [-0.161, +0.487]	+0.170 [+0.095, +0.244] units: [-0.212, +0.513]	0.30 / 0.42	accepted
7	apertus-8b BANKING77 sem. s42 b01 SimNPO / explor.	0.471 0.405	+0.159 [+0.110, +0.208] units: [-0.128, +0.524]	+0.140 [+0.056, +0.223] units: [-0.188, +0.510]	0.34 / 0.35	accepted
8	apertus-8b BANKING77 sem. s42 b01 SimNPO / replic.	0.471 0.405	+0.078 [+0.047, +0.112] units: [-0.121, +0.316]	+0.102 [+0.048, +0.154] units: [-0.142, +0.276]	0.17 / 0.25	accepted
9	falcon-h1-3b BANKING77 sem. s17 b01 SimNPO / screen	0.501 0.466	+0.181 [+0.156, +0.206] units: [+0.068, +0.403]	+0.195 [+0.154, +0.238] units: [+0.074, +0.333]	0.36 / 0.42	accepted
10	gemma4-e2b BANKING77 sem. s17 b01 SimNPO / screen	0.474 0.394	+0.195 [+0.167, +0.222] units: [+0.090, +0.381]	+0.172 [+0.128, +0.217] units: [+0.024, +0.321]	0.41 / 0.44	accepted
11	granite4-micro BANKING77 sem. s17 b01 SimNPO / screen	0.468 0.401	+0.216 [+0.188, +0.245] units: [+0.109, +0.380]	+0.220 [+0.176, +0.265] units: [+0.091, +0.336]	0.46 / 0.55	accepted
12	lfm2.5-1.2b BANKING77 sem. s17 b01 SimNPO / screen	0.539 0.434	+0.160 [+0.116, +0.202] units: [+0.038, +0.308]	+0.098 [+0.027, +0.169] units: [-0.106, +0.318]	0.30 / 0.23	accepted
13	llama3.2-1b BANKING77 sem. s17 b01 SimNPO / screen	0.626 0.502	+0.355 [+0.324, +0.386] units: [+0.243, +0.470]	+0.271 [+0.224, +0.320] units: [+0.210, +0.314]	0.57 / 0.54	accepted
14	llama3.2-3b BANKING77 sem. s17 b01 SimNPO / screen	0.558 0.461	+0.280 [+0.248, +0.314] units: [+0.168, +0.461]	+0.259 [+0.203, +0.316] units: [+0.113, +0.436]	0.50 / 0.56	accepted
15	ministral3-3b BANKING77 sem. s17 b01 SimNPO / screen	0.464 0.377	+0.131 [+0.104, +0.160] units: [+0.055, +0.262]	+0.079 [+0.027, +0.131] units: [+0.013, +0.148]	0.28 / 0.21	accepted

Table S3. Complete method results: oracle fidelity and recorded status. Row IDs link the utility panel and CSV.

ID	Configuration	$D_{0,F} / D_{0,G}$	$\Delta$ ODF [rows]; unit CI	$\Delta$ ODG [rows]; unit CI	$P_F/P_G$	Status
16	nemotron-nano-9b BANKING77 sem. s17 b01 SimNPO / screen	0.472 0.369	+0.218 [+0.186, +0.250] units: [+0.080, +0.463]	+0.194 [+0.140, +0.248] units: [+0.036, +0.416]	0.46 / 0.53	accepted
17	olmo3-7b BANKING77 sem. s17 b01 SimNPO / screen	0.545 0.447	+0.186 [+0.158, +0.215] units: [+0.099, +0.371]	+0.156 [+0.107, +0.204] units: [+0.078, +0.210]	0.34 / 0.35	accepted
18	olmo3-7b BANKING77 sem. s17 b02 SimNPO / replic.	0.659 0.643	+0.328 [+0.297, +0.360] units: [+0.071, +0.591]	+0.312 [+0.265, +0.359] units: [+0.078, +0.546]	0.50 / 0.49	accepted
19	olmo3-7b BANKING77 sem. s42 b01 SimNPO / replic.	0.464 0.388	+0.017 [-0.021, +0.054] units: [-0.196, +0.301]	+0.001 [-0.055, +0.055] units: [-0.290, +0.206]	0.04 / 0.00	positive, unconfirmed
20	phi4-mini BANKING77 sem. s17 b01 SimNPO / screen	0.396 0.321	+0.187 [+0.158, +0.216] units: [+0.069, +0.352]	+0.163 [+0.114, +0.213] units: [+0.037, +0.357]	0.47 / 0.51	accepted
21	qwen2.5-1.5b BANKING77 sem. s17 b01 SimNPO / screen	0.539 0.417	+0.243 [+0.214, +0.273] units: [+0.096, +0.402]	+0.227 [+0.181, +0.275] units: [+0.072, +0.374]	0.45 / 0.54	accepted
22	qwen2.5-3b BANKING77 sem. s17 b01 SimNPO / screen	0.457 0.378	+0.228 [+0.196, +0.260] units: [+0.083, +0.452]	+0.228 [+0.180, +0.276] units: [+0.091, +0.365]	0.50 / 0.60	accepted
23	qwen2.5-7b BANKING77 sem. s17 b01 SimNPO / screen	0.418 0.351	+0.263 [+0.232, +0.295] units: [+0.146, +0.472]	+0.222 [+0.176, +0.268] units: [+0.081, +0.363]	0.63 / 0.63	accepted
24	qwen3-4b BANKING77 sem. s17 b01 SimNPO / screen	0.318 0.238	+0.091 [+0.063, +0.117] units: [+0.024, +0.155]	+0.044 [-0.001, +0.091] units: [-0.090, +0.173]	0.28 / 0.19	positive, unconfirmed
25	qwen3-4b BANKING77 sem. s17 b02 SimNPO / replic.	0.614 0.595	+0.314 [+0.281, +0.348] units: [+0.094, +0.578]	+0.300 [+0.250, +0.351] units: [+0.089, +0.546]	0.51 / 0.50	accepted
26	qwen3-4b BANKING77 sem. s42 b01 SimNPO / replic.	0.340 0.298	-0.060 [-0.109, -0.011] units: [-0.434, +0.272]	-0.031 [-0.106, +0.042] units: [-0.447, +0.258]	-0.18 / -0.10	no benefit
27	qwen3.5-0.8b BANKING77 opaque s17 b01 SimNPO / explor.	0.943 0.881	+0.571 [+0.539, +0.603] units: [+0.431, +0.732]	+0.549 [+0.495, +0.604] units: [+0.392, +0.706]	0.61 / 0.62	accepted
28	qwen3.5-0.8b BANKING77 opaque s17 b02 SimNPO / explor.	0.912 0.872	+0.410 [+0.376, +0.445] units: [+0.234, +0.571]	+0.371 [+0.322, +0.423] units: [+0.207, +0.475]	0.45 / 0.43	accepted
29	qwen3.5-0.8b BANKING77 opaque s42 b01 SimNPO / explor.	0.934 0.878	+0.560 [+0.532, +0.587] units: [+0.467, +0.666]	+0.508 [+0.461, +0.554] units: [+0.428, +0.647]	0.60 / 0.58	accepted
30	qwen3.5-0.8b BANKING77 opaque s42 b02 SimNPO / explor.	0.911 0.902	+0.362 [+0.326, +0.398] units: [+0.118, +0.611]	+0.345 [+0.293, +0.398] units: [+0.095, +0.594]	0.40 / 0.38	accepted

Table S3 (PDF p. 35). Complete method results (continued): oracle fidelity, IDs 16–30.

ID	Configuration	$D_{0,F} / D_{0,G}$	$\Delta ODF$ [rows]; unit CI	$\Delta ODG$ [rows]; unit CI	$P_F/P_G$	Status
31	qwen3.5-0.8b BANKING77 sem. s17 b01 GradAscent / confirm.	0.608 0.495	+0.332 [+0.297, +0.366] units: [+0.224, +0.422]	+0.241 [+0.194, +0.290] units: [+0.192, +0.290]	0.55 / 0.49	accepted
32	qwen3.5-0.8b BANKING77 sem. s17 b01 GradDiff / confirm.	0.608 0.495	+0.274 [+0.241, +0.308] units: [+0.183, +0.379]	+0.209 [+0.165, +0.255] units: [+0.145, +0.283]	0.45 / 0.42	accepted
33	qwen3.5-0.8b BANKING77 sem. s17 b01 GradDiff-KL / confirm.	0.608 0.495	+0.188 [+0.131, +0.242] units: [+0.006, +0.431]	+0.139 [+0.053, +0.228] units: [-0.092, +0.338]	0.31 / 0.28	infeasible
34	qwen3.5-0.8b BANKING77 sem. s17 b01 NPO / confirm.	0.608 0.495	+0.236 [+0.205, +0.266] units: [+0.143, +0.342]	+0.176 [+0.135, +0.220] units: [+0.100, +0.252]	0.39 / 0.36	accepted
35	qwen3.5-0.8b BANKING77 sem. s17 b01 SimNPO / confirm.	0.608 0.495	+0.359 [+0.319, +0.396] units: [+0.297, +0.479]	+0.280 [+0.224, +0.336] units: [+0.207, +0.381]	0.59 / 0.57	accepted
36	qwen3.5-0.8b BANKING77 sem. s17 b01 Uniformization / confirm.	0.608 0.495	+0.160 [+0.144, +0.175] units: [+0.112, +0.211]	+0.102 [+0.082, +0.124] units: [+0.079, +0.127]	0.26 / 0.21	infeasible
37	qwen3.5-0.8b BANKING77 sem. s17 b01 GradDiff-KL / confirmation-sensitivity	0.193 0.148	-0.005 [-0.024, +0.014] units: [-0.076, +0.057]	-0.049 [-0.076, -0.023] units: [-0.125, +0.019]	-0.03 / -0.33	infeasible
38	qwen3.5-0.8b BANKING77 sem. s17 b01 SimNPO / confirmation-sensitivity	0.193 0.148	+0.040 [+0.033, +0.047] units: [+0.007, +0.083]	+0.044 [+0.031, +0.057] units: [+0.003, +0.091]	0.21 / 0.29	accepted
39	qwen3.5-0.8b BANKING77 sem. s17 b01 Uniformization / confirmation-sensitivity	0.193 0.148	+0.030 [+0.025, +0.035] units: [+0.013, +0.045]	+0.023 [+0.017, +0.030] units: [+0.004, +0.044]	0.16 / 0.16	accepted
40	qwen3.5-0.8b BANKING77 sem. s17 b01 GradDiff / confirmation-sensitivity	0.193 0.148	+0.007 [-0.007, +0.021] units: [-0.018, +0.038]	-0.014 [-0.033, +0.003] units: [-0.032, +0.004]	0.04 / -0.10	no benefit
41	qwen3.5-0.8b BANKING77 sem. s17 b01 NPO / confirmation_sensitivity	0.193 0.148	-0.003 [-0.024, +0.018] units: [-0.097, +0.068]	-0.050 [-0.081, -0.019] units: [-0.160, +0.040]	-0.02 / -0.34	no benefit
42	qwen3.5-0.8b BANKING77 sem. s17 b01 GradAscent / pilot	0.193 0.148	-0.168 [-0.203, -0.132] units: [-0.303, -0.033]	-0.241 [-0.296, -0.187] units: [-0.420, -0.084]	-0.87 / -1.63	infeasible
43	qwen3.5-0.8b BANKING77 sem. s17 b01 GradAscent / pilot	0.608 0.495	+0.168 [+0.110, +0.225] units: [-0.011, +0.409]	+0.111 [+0.022, +0.201] units: [-0.125, +0.318]	0.28 / 0.22	infeasible
44	qwen3.5-0.8b BANKING77 sem. s17 b01 GradDiff / pilot	0.193 0.148	-0.132 [-0.164, -0.100] units: [-0.249, -0.023]	-0.199 [-0.250, -0.149] units: [-0.373, -0.061]	-0.68 / -1.35	infeasible
45	qwen3.5-0.8b BANKING77 sem. s17 b01 GradDiff / pilot	0.608 0.495	+0.133 [+0.082, +0.184] units: [-0.003, +0.370]	+0.07 [-0.007, +0.147] units: [-0.120, +0.287]	0.22 / 0.14	infeasible

Table S3 (PDF p. 36). Table S3. Complete method results (continued): oracle fidelity, IDs 31–45.

ID	Configuration	$D_{0,F} / D_{0,G}$	$\Delta O_{DF}$ [rows]; unit CI	$\Delta O_{DG}$ [rows]; unit CI	$P_F/P_G$	Status
46	qwen3.5-0.8b BANKING77 sem. s17 b01 GradDiff / explor.	0.608 0.495	+0.113 [+0.058, +0.167] units: [-0.050, +0.361]	+0.015 [-0.070, +0.099] units: [-0.208, +0.229]	0.19 / 0.03	infeasible
47	qwen3.5-0.8b BANKING77 sem. s17 b01 GradDiff-KL / explor.	0.608 0.495	+0.293 [+0.260, +0.325] units: [+0.201, +0.372]	+0.215 [+0.172, +0.261] units: [+0.172, +0.255]	0.48 / 0.43	accepted
48	qwen3.5-0.8b BANKING77 sem. s17 b01 RMU / explor.	0.608 0.495	+0.026 [+0.002, +0.048] units: [-0.034, +0.135]	-0.009 [-0.051, +0.033] units: [-0.093, +0.087]	0.04 / -0.02	no benefit
49	qwen3.5-0.8b BANKING77 sem. s17 b01 SimNPO / explor.	0.608 0.495	+0.339 [+0.305, +0.372] units: [+0.223, +0.468]	+0.262 [+0.213, +0.312] units: [+0.190, +0.372]	0.56 / 0.53	accepted
50	qwen3.5-0.8b BANKING77 sem. s17 b01 NPO / pilot	0.193 0.148	-0.145 [-0.178, -0.111] units: [-0.277, -0.014]	-0.201 [-0.252, -0.151] units: [-0.384, -0.046]	-0.75 / -1.36	infeasible
51	qwen3.5-0.8b BANKING77 sem. s17 b01 NPO / pilot	0.608 0.495	+0.183 [+0.132, +0.233] units: [+0.028, +0.392]	+0.106 [+0.029, +0.184] units: [-0.075, +0.288]	0.30 / 0.21	infeasible
52	qwen3.5-0.8b BANKING77 sem. s17 b01 RMU / pilot	0.193 0.148	-0.000 [-0.001, +0.000] units: [-0.002, +0.001]	-0.000 [-0.001, +0.001] units: [-0.002, +0.001]	-0.00 / -0.00	no benefit
53	qwen3.5-0.8b BANKING77 sem. s17 b01 RMU / pilot	0.608 0.495	+0.002 [+0.001, +0.003] units: [-0.002, +0.006]	+0.001 [+0.000, +0.003] units: [-0.001, +0.004]	0.00 / 0.00	accepted
54	qwen3.5-0.8b BANKING77 sem. s17 b01 Uniformization / pilot	0.193 0.148	-0.437 [-0.466, -0.409] units: [-0.567, -0.303]	-0.466 [-0.508, -0.426] units: [-0.591, -0.337]	-2.26 / -3.16	infeasible
55	qwen3.5-0.8b BANKING77 sem. s17 b01 Uniformization / pilot	0.608 0.495	-0.135 [-0.173, -0.099] units: [-0.286, -0.008]	-0.198 [-0.255, -0.141] units: [-0.350, -0.119]	-0.22 / -0.40	infeasible
56	qwen3.5-0.8b BANKING77 sem. s17 b02 GradAscent / replic.	0.644 0.636	+0.185 [+0.152, +0.218] units: [-0.001, +0.374]	+0.162 [+0.117, +0.210] units: [-0.004, +0.327]	0.29 / 0.25	infeasible
57	qwen3.5-0.8b BANKING77 sem. s17 b02 GradDiff / replic.	0.644 0.636	+0.185 [+0.153, +0.218] units: [-0.000, +0.375]	+0.164 [+0.119, +0.213] units: [-0.002, +0.330]	0.29 / 0.26	infeasible
58	qwen3.5-0.8b BANKING77 sem. s17 b02 NPO / replic.	0.644 0.636	+0.204 [+0.170, +0.237] units: [+0.005, +0.406]	+0.199 [+0.151, +0.249] units: [+0.013, +0.384]	0.32 / 0.31	infeasible
59	qwen3.5-0.8b BANKING77 sem. s17 b02 SimNPO / replic.	0.644 0.636	+0.449 [+0.412, +0.487] units: [+0.143, +0.759]	+0.467 [+0.408, +0.524] units: [+0.142, +0.743]	0.70 / 0.73	accepted
60	qwen3.5-0.8b BANKING77 sem. s17 b02 SimNPO / confirmation-sensitivity	0.225 0.223	+0.024 [+0.017, +0.032] units: [+0.001, +0.045]	+0.021 [+0.010, +0.035] units: [+0.000, +0.041]	0.11 / 0.10	accepted

Table S3 (PDF p. 37). Complete method results (continued): oracle fidelity, IDs 46–60.

ID	Configuration	$D_{0,F} / D_{0,G}$	$\Delta ODF$ [rows]; unit CI	$\Delta ODG$ [rows]; unit CI	$P_F/P_G$	Status
61	qwen3.5-0.8b BANKING77 sem. s17 b02 Uniformization / confirmation sensitivity	0.225 0.223	+0.016 [+0.012, +0.020] units: [+0.008, +0.023]	+0.018 [+0.011, +0.025] units: [+0.006, +0.027]	0.07 / 0.08	accepted
62	qwen3.5-0.8b BANKING77 sem. s17 b02 GradDiff / confirmation sensitivity	0.225 0.223	+0.082 [+0.061, +0.103] units: [+0.004, +0.185]	+0.055 [+0.029, +0.084] units: [+0.003, +0.122]	0.36 / 0.25	accepted
63	qwen3.5-0.8b BANKING77 sem. s17 b03 GradAscent / replic.	0.465 0.440	-0.341 [-0.382, -0.300] units: [-0.582, +0.017]	-0.375 [-0.443, -0.306] units: [-0.638, -0.022]	-0.73 / -0.85	infeasible
64	qwen3.5-0.8b BANKING77 sem. s17 b03 GradDiff / replic.	0.465 0.440	+0.075 [+0.040, +0.111] units: [-0.146, +0.336]	+0.048 [-0.014, +0.112] units: [-0.213, +0.318]	0.16 / 0.11	infeasible
65	qwen3.5-0.8b BANKING77 sem. s17 b03 NPO / replic.	0.465 0.440	+0.159 [+0.135, +0.186] units: [+0.047, +0.318]	+0.156 [+0.111, +0.203] units: [+0.027, +0.298]	0.34 / 0.36	accepted
66	qwen3.5-0.8b BANKING77 sem. s17 b03 SimNPO / replic.	0.465 0.440	+0.282 [+0.248, +0.317] units: [+0.089, +0.473]	+0.274 [+0.213, +0.338] units: [+0.081, +0.468]	0.61 / 0.62	accepted
67	qwen3.5-0.8b BANKING77 sem. s17 b03 SimNPO / confirmation sensitivity	0.087 0.049	+0.005 [+0.003, +0.009] units: [+0.000, +0.012]	+0.003 [+0.001, +0.005] units: [+0.000, +0.006]	0.06 / 0.05	accepted
68	qwen3.5-0.8b BANKING77 sem. s17 b03 Uniformization / confirmation sensitivity	0.087 0.049	+0.011 [+0.007, +0.014] units: [+0.003, +0.020]	+0.007 [+0.003, +0.012] units: [+0.001, +0.017]	0.12 / 0.15	accepted
69	qwen3.5-0.8b BANKING77 sem. s17 b03 GradDiff / confirmation sensitivity	0.087 0.049	+0.012 [+0.005, +0.020] units: [-0.000, +0.029]	+0.005 [-0.001, +0.015] units: [-0.001, +0.014]	0.13 / 0.10	positive, unconfirmed
70	qwen3.5-0.8b BANKING77 sem. s17 ex_seed17 GradAscent / pilot	0.030 0.032	-0.030 [-0.044, -0.016] units: [-0.044, -0.016]	-0.030 [-0.052, -0.012] units: [-0.053, -0.012]	-0.99 / -0.93	infeasible
71	qwen3.5-0.8b BANKING77 sem. s17 ex_seed17 GradDiff / pilot	0.030 0.032	-0.016 [-0.024, -0.009] units: [-0.024, -0.009]	-0.012 [-0.023, -0.004] units: [-0.023, -0.004]	-0.53 / -0.37	infeasible
72	qwen3.5-0.8b BANKING77 sem. s17 ex_seed17 NPO / pilot	0.030 0.032	-0.016 [-0.024, -0.009] units: [-0.024, -0.009]	-0.011 [-0.022, -0.003] units: [-0.022, -0.003]	-0.53 / -0.35	infeasible
73	qwen3.5-0.8b BANKING77 sem. s17 ex_seed17 RMU / pilot	0.030 0.032	-0.000 [-0.000, +0.000] units: [-0.000, +0.000]	+0.000 [-0.000, +0.000] units: [+0.000, +0.000]	-0.00 / 0.00	no benefit
74	qwen3.5-0.8b BANKING77 sem. s17 ex_seed17 Uniformization / pilot	0.030 0.032	-0.310 [-0.321, -0.300] units: [-0.320, -0.300]	-0.311 [-0.325, -0.296] units: [-0.325, -0.296]	-10.34 / -9.58	infeasible
75	qwen3.5-0.8b BANKING77 sem. s42 b01 GradAscent / replic.	0.558 0.470	+0.165 [+0.135, +0.196] units: [+0.066, +0.353]	+0.119 [+0.075, +0.166] units: [+0.006, +0.238]	0.30 / 0.25	infeasible

Table S3 (PDF p. 38). Complete method results (continued): oracle fidelity, IDs 61–75.

ID	Configuration	$D_{0,F}$ / $D_{0,G}$	$\Delta$ ODF [rows]; unit CI	$\Delta$ ODG [rows]; unit CI	$P_F/P_G$	Status
76	qwen3.5-0.8b BANKING77 sem. s42 b01 GradDiff / replic.	0.558 0.470	+0.161 [+0.132, +0.191] units: [+0.062, +0.346]	+0.115 [+0.072, +0.161] units: [+0.001, +0.230]	0.29 / 0.24	infeasible
77	qwen3.5-0.8b BANKING77 sem. s42 b01 NPO / replic.	0.558 0.470	+0.153 [+0.126, +0.181] units: [+0.072, +0.289]	+0.105 [+0.070, +0.142] units: [+0.021, +0.189]	0.27 / 0.22	accepted
78	qwen3.5-0.8b BANKING77 sem. s42 b01 SimNPO / replic.	0.558 0.470	+0.307 [+0.265, +0.347] units: [+0.186, +0.518]	+0.240 [+0.174, +0.304] units: [+0.116, +0.363]	0.55 / 0.51	accepted
79	qwen3.5-0.8b BANKING77 sem. s42 b01 SimNPO / confirmation-sensitivity	0.208 0.186	+0.040 [+0.032, +0.049] units: [+0.004, +0.080]	+0.050 [+0.035, +0.066] units: [+0.004, +0.098]	0.19 / 0.27	accepted
80	qwen3.5-0.8b BANKING77 sem. s42 b01 Uniformization / confirmation-sensitivity	0.208 0.186	+0.032 [+0.027, +0.037] units: [+0.014, +0.049]	+0.026 [+0.018, +0.034] units: [+0.008, +0.044]	0.15 / 0.14	accepted
81	qwen3.5-0.8b BANKING77 sem. s42 b01 GradDiff / confirmation-sensitivity	0.208 0.186	+0.054 [+0.034, +0.074] units: [+0.017, +0.106]	+0.032 [+0.004, +0.061] units: [-0.005, +0.078]	0.26 / 0.17	accepted
82	qwen3.5-0.8b BANKING77 sem. s42 b02 GradAscent / replic.	0.676 0.660	+0.257 [+0.222, +0.293] units: [+0.002, +0.518]	+0.250 [+0.200, +0.304] units: [-0.002, +0.502]	0.38 / 0.38	infeasible
83	qwen3.5-0.8b BANKING77 sem. s42 b02 GradDiff / replic.	0.676 0.660	+0.256 [+0.221, +0.292] units: [+0.002, +0.516]	+0.249 [+0.198, +0.302] units: [-0.001, +0.499]	0.38 / 0.38	infeasible
84	qwen3.5-0.8b BANKING77 sem. s42 b02 NPO / replic.	0.676 0.660	+0.290 [+0.256, +0.326] units: [+0.015, +0.571]	+0.277 [+0.226, +0.331] units: [+0.005, +0.548]	0.43 / 0.42	accepted
85	qwen3.5-0.8b BANKING77 sem. s42 b02 SimNPO / replic.	0.676 0.660	+0.319 [+0.283, +0.355] units: [+0.008, +0.636]	+0.297 [+0.245, +0.350] units: [+0.005, +0.589]	0.47 / 0.45	accepted
86	qwen3.5-0.8b BANKING77 sem. s42 b02 SimNPO / confirmation-sensitivity	0.170 0.133	+0.019 [+0.014, +0.025] units: [+0.002, +0.035]	+0.016 [+0.009, +0.025] units: [-0.000, +0.030]	0.11 / 0.12	accepted
87	qwen3.5-0.8b BANKING77 sem. s42 b02 Uniformization / confirmation-sensitivity	0.170 0.133	+0.018 [+0.014, +0.023] units: [+0.006, +0.029]	+0.013 [+0.007, +0.020] units: [+0.000, +0.024]	0.11 / 0.10	accepted
88	qwen3.5-0.8b BANKING77 sem. s42 b02 GradDiff / confirmation-sensitivity	0.170 0.133	-0.009 [-0.032, +0.014] units: [-0.033, +0.010]	-0.041 [-0.080, -0.002] units: [-0.105, +0.009]	-0.06 / -0.31	no benefit
89	qwen3.5-0.8b BANKING77 sem. s42 b03 GradAscent / replic.	0.435 0.398	-0.156 [-0.192, -0.120] units: [-0.425, +0.119]	-0.210 [-0.272, -0.147] units: [-0.474, +0.068]	-0.36 / -0.53	infeasible
90	qwen3.5-0.8b BANKING77 sem. s42 b03 GradDiff / replic.	0.435 0.398	+0.147 [+0.121, +0.173] units: [+0.015, +0.316]	+0.129 [+0.083, +0.175] units: [-0.007, +0.267]	0.34 / 0.32	accepted

Table S3 (PDF p. 39). Table S3. Complete method results (continued): oracle fidelity, IDs 76–90.

ID	Configuration	$D_{0,F}$ / $D_{0,G}$	$\Delta$ ODF [rows]; unit CI	$\Delta$ ODG [rows]; unit CI	$P_F/P_G$	Status
91	qwen3.5-0.8b BANKING77 sem. s42 b03 NPO / replic.	0.435 0.398	-0.042 [-0.071, -0.012] units: [-0.261, +0.176]	-0.087 [-0.140, -0.034] units: [-0.314, +0.140]	-0.10 / -0.22	infeasible
92	qwen3.5-0.8b BANKING77 sem. s42 b03 SimNPO / replic.	0.435 0.398	+0.209 [+0.182, +0.237] units: [+0.057, +0.359]	+0.195 [+0.152, +0.241] units: [+0.078, +0.313]	0.48 / 0.49	accepted
93	qwen3.5-0.8b BANKING77 sem. s42 b03 SimNPO / confirmation-sensitivity	0.093 0.075	+0.002 [-0.001, +0.005] units: [-0.002, +0.007]	-0.002 [-0.004, +0.000] units: [-0.004, -0.000]	0.02 / -0.03	no benefit
94	qwen3.5-0.8b BANKING77 sem. s42 b03 Uniformization / confirmation-sensitivity	0.093 0.075	+0.007 [+0.004, +0.011] units: [+0.002, +0.014]	+0.006 [+0.002, +0.010] units: [-0.000, +0.013]	0.08 / 0.08	accepted
95	qwen3.5-0.8b BANKING77 sem. s42 b03 GradDiff / confirmation-sensitivity	0.093 0.075	+0.004 [-0.003, +0.011] units: [-0.003, +0.013]	-0.003 [-0.009, +0.002] units: [-0.005, -0.001]	0.04 / -0.04	no benefit
96	qwen3.5-0.8b BANKING77 sem. s99 b01 GradAscent / replic.	0.610 0.495	+0.136 [+0.109, +0.166] units: [+0.033, +0.291]	+0.119 [+0.076, +0.165] units: [+0.027, +0.217]	0.22 / 0.24	accepted
97	qwen3.5-0.8b BANKING77 sem. s99 b01 GradDiff / replic.	0.610 0.495	+0.138 [+0.110, +0.167] units: [+0.034, +0.291]	+0.123 [+0.080, +0.167] units: [+0.028, +0.218]	0.23 / 0.25	accepted
98	qwen3.5-0.8b BANKING77 sem. s99 b01 NPO / replic.	0.610 0.495	+0.140 [+0.113, +0.167] units: [+0.037, +0.291]	+0.131 [+0.090, +0.174] units: [+0.034, +0.223]	0.23 / 0.26	accepted
99	qwen3.5-0.8b BANKING77 sem. s99 b01 SimNPO / replic.	0.610 0.495	+0.236 [+0.204, +0.267] units: [+0.095, +0.515]	+0.240 [+0.191, +0.290] units: [+0.101, +0.405]	0.39 / 0.48	accepted
100	qwen3.5-0.8b BANKING77 sem. s99 b01 SimNPO / confirmation-sensitivity	0.230 0.186	+0.055 [+0.043, +0.069] units: [+0.004, +0.137]	+0.061 [+0.040, +0.084] units: [+0.002, +0.148]	0.24 / 0.33	accepted
101	qwen3.5-0.8b BANKING77 sem. s99 b01 Uniformization / confirmation-sensitivity	0.230 0.186	+0.024 [+0.020, +0.028] units: [+0.008, +0.043]	+0.027 [+0.021, +0.034] units: [+0.009, +0.048]	0.10 / 0.15	accepted
102	qwen3.5-0.8b BANKING77 sem. s99 b01 GradDiff / confirmation-sensitivity	0.230 0.186	+0.017 [+0.003, +0.032] units: [+0.003, +0.036]	+0.023 [-0.001, +0.048] units: [+0.002, +0.055]	0.07 / 0.12	positive, unconfirmed
103	qwen3.5-0.8b BANKING77 sem. s99 b02 GradAscent / replic.	0.676 0.648	+0.185 [+0.154, +0.218] units: [+0.004, +0.370]	+0.207 [+0.158, +0.259] units: [+0.006, +0.408]	0.27 / 0.32	accepted
104	qwen3.5-0.8b BANKING77 sem. s99 b02 GradDiff / replic.	0.676 0.648	+0.185 [+0.153, +0.218] units: [+0.004, +0.370]	+0.206 [+0.157, +0.258] units: [+0.006, +0.406]	0.27 / 0.32	accepted
105	qwen3.5-0.8b BANKING77 sem. s99 b02 NPO / replic.	0.676 0.648	+0.196 [+0.164, +0.229] units: [+0.005, +0.390]	+0.212 [+0.163, +0.264] units: [+0.010, +0.414]	0.29 / 0.33	accepted

Table S3 (PDF p. 40). Complete method results (continued): oracle fidelity, IDs 91–105.

ID	Configuration	$D_{0,F}$ / $D_{0,G}$	$\Delta$ ODF [rows]; unit CI	$\Delta$ ODG [rows]; unit CI	$P_F/P_G$	Status
106	qwen3.5-0.8b BANKING77 sem. s99 b02 SimNPO / replic.	0.676 0.648	+0.469 [+0.432, +0.507] units: [+0.150, +0.707]	+0.492 [+0.433, +0.550] units: [+0.169, +0.740]	0.69 / 0.76	accepted
107	qwen3.5-0.8b BANKING77 sem. s99 b02 SimNPO / confirmation-sensitivity	0.197 0.183	+0.016 [+0.010, +0.023] units: [+0.001, +0.030]	+0.019 [+0.010, +0.031] units: [+0.002, +0.032]	0.08 / 0.10	accepted
108	qwen3.5-0.8b BANKING77 sem. s99 b02 Uniformization / confirmation-sensitivity	0.197 0.183	+0.014 [+0.011, +0.017] units: [+0.007, +0.022]	+0.012 [+0.006, +0.018] units: [+0.002, +0.021]	0.07 / 0.06	accepted
109	qwen3.5-0.8b BANKING77 sem. s99 b02 GradDiff / confirmation-sensitivity	0.197 0.183	+0.031 [+0.019, +0.044] units: [+0.000, +0.075]	+0.011 [-0.007, +0.029] units: [+0.002, +0.020]	0.15 / 0.06	positive, unconfirmed
110	qwen3.5-0.8b BANKING77 sem. s99 b03 GradAscent / replic.	0.469 0.454	+0.206 [+0.174, +0.240] units: [+0.004, +0.408]	+0.218 [+0.160, +0.279] units: [-0.006, +0.442]	0.44 / 0.48	infeasible
111	qwen3.5-0.8b BANKING77 sem. s99 b03 GradDiff / replic.	0.469 0.454	+0.188 [+0.158, +0.219] units: [+0.012, +0.363]	+0.194 [+0.141, +0.251] units: [+0.006, +0.383]	0.40 / 0.43	infeasible
112	qwen3.5-0.8b BANKING77 sem. s99 b03 NPO / replic.	0.469 0.454	+0.119 [+0.096, +0.143] units: [+0.021, +0.239]	+0.113 [+0.079, +0.150] units: [+0.028, +0.198]	0.25 / 0.25	accepted
113	qwen3.5-0.8b BANKING77 sem. s99 b03 SimNPO / replic.	0.469 0.454	+0.231 [+0.202, +0.260] units: [+0.051, +0.410]	+0.236 [+0.187, +0.286] units: [+0.055, +0.416]	0.49 / 0.52	accepted
114	qwen3.5-0.8b BANKING77 sem. s99 b03 SimNPO / confirmation-sensitivity	0.106 0.088	+0.004 [+0.002, +0.007] units: [+0.001, +0.008]	+0.001 [-0.003, +0.004] units: [-0.002, +0.005]	0.04 / 0.01	positive, unconfirmed
115	qwen3.5-0.8b BANKING77 sem. s99 b03 Uniformization / confirmation-sensitivity	0.106 0.088	+0.007 [+0.004, +0.010] units: [+0.001, +0.014]	+0.002 [-0.003, +0.006] units: [-0.004, +0.008]	0.07 / 0.02	positive, unconfirmed
116	qwen3.5-0.8b BANKING77 sem. s99 b03 GradDiff / confirmation-sensitivity	0.106 0.088	-0.001 [-0.008, +0.005] units: [-0.011, +0.010]	-0.002 [-0.011, +0.007] units: [-0.012, +0.007]	-0.01 / -0.02	no benefit
117	qwen3.5-2b BANKING77 sem. s17 b01 SimNPO / screen	0.497 0.399	+0.231 [+0.197, +0.266] units: [+0.107, +0.351]	+0.195 [+0.133, +0.256] units: [+0.053, +0.351]	0.46 / 0.49	accepted
118	qwen3.5-4b BANKING77 sem. s17 b01 SimNPO / screen	0.505 0.391	+0.099 [+0.074, +0.126] units: [+0.042, +0.150]	+0.071 [+0.027, +0.115] units: [-0.003, +0.146]	0.20 / 0.18	accepted
119	qwen3.5-9b BANKING77 sem. s17 b01 SimNPO / screen	0.414 0.353	+0.223 [+0.194, +0.254] units: [+0.112, +0.422]	+0.206 [+0.164, +0.249] units: [+0.082, +0.329]	0.54 / 0.58	accepted
120	qwen3.5-9b BANKING77 sem. s17 b02 SimNPO / replic.	0.599 0.591	+0.261 [+0.222, +0.300] units: [+0.036, +0.538]	+0.255 [+0.203, +0.310] units: [+0.036, +0.507]	0.44 / 0.43	accepted

Table S3 (PDF p. 41). Complete method results (continued): oracle fidelity, IDs 106–120.

ID	Configuration	$D_{0,F} / D_{0,G}$	$\Delta O_{\mathrm{DF}}$ [rows]; unit CI	$\Delta O_{\mathrm{DG}}$ [rows]; unit CI	$P_F/P_G$	Status
121	qwen3.5-9b BANKING77 sem. s42 b01 SimNPO / explor.	0.511 0.425	+0.094 [+0.057, +0.135] units: [+0.015, +0.212]	+0.093 [+0.038, +0.147] units: [-0.035, +0.225]	0.18 / 0.22	accepted
122	qwen3.5-9b BANKING77 sem. s42 b01 SimNPO / explor.	0.511 0.425	+0.008 [-0.038, +0.057] units: [-0.085, +0.173]	-0.033 [-0.109, +0.043] units: [-0.191, +0.236]	0.02 / -0.08	no benefit
123	qwen3.5-9b BANKING77 sem. s42 b01 SimNPO / explor.	0.511 0.425	+0.014 [-0.030, +0.061] units: [-0.081, +0.149]	-0.027 [-0.094, +0.041] units: [-0.207, +0.190]	0.03 / -0.06	no benefit
124	qwen3.5-9b BANKING77 sem. s42 b01 SimNPO / explor.	0.511 0.425	+0.084 [+0.067, +0.103] units: [+0.043, +0.156]	+0.084 [+0.060, +0.110] units: [+0.037, +0.132]	0.16 / 0.20	accepted
125	qwen3.5-9b BANKING77 sem. s42 b01 SimNPO / explor.	0.511 0.425	+0.068 [+0.023, +0.117] units: [-0.042, +0.232]	+0.022 [-0.056, +0.099] units: [-0.130, +0.277]	0.13 / 0.05	positive, unconfirmed
126	qwen3.5-9b BANKING77 sem. s42 b01 SimNPO / replic.	0.511 0.425	+0.070 [+0.055, +0.087] units: [+0.045, +0.116]	+0.076 [+0.051, +0.103] units: [+0.031, +0.141]	0.14 / 0.18	accepted
127	smollm3-3b BANKING77 sem. s17 b01 SimNPO / screen	0.414 0.319	+0.205 [+0.174, +0.237] units: [+0.081, +0.337]	+0.148 [+0.097, +0.199] units: [+0.063, +0.232]	0.50 / 0.46	accepted
128	llama3.2-1b CLINC150 sem. s17 b01 SimNPO / transfer	0.453 0.404	-0.079 [-0.125, -0.034] units: [-0.424, +0.281]	-0.082 [-0.159, -0.006] units: [-0.403, +0.241]	-0.18 / -0.20	no benefit
129	qwen3.5-0.8b CLINC150 opaque s17 b02 SimNPO / explor.	0.983 0.976	+0.275 [+0.247, +0.304] units: [+0.079, +0.538]	+0.287 [+0.237, +0.340] units: [+0.023, +0.560]	0.28 / 0.29	accepted
130	qwen3.5-0.8b CLINC150 opaque s17 b03 SimNPO / explor.	0.993 0.973	+0.132 [+0.111, +0.153] units: [+0.007, +0.342]	+0.111 [+0.076, +0.148] units: [+0.002, +0.299]	0.13 / 0.11	accepted
131	qwen3.5-0.8b CLINC150 opaque s17 b06 SimNPO / explor.	0.999 0.989	+0.023 [+0.017, +0.031] units: [+0.006, +0.046]	+0.024 [+0.012, +0.039] units: [+0.009, +0.043]	0.02 / 0.02	accepted
132	qwen3.5-0.8b CLINC150 sem. s17 b01 GradAscent / transfer	0.416 0.356	-0.185 [-0.217, -0.152] units: [-0.404, -0.001]	-0.168 [-0.228, -0.109] units: [-0.387, +0.021]	-0.44 / -0.47	no benefit
133	qwen3.5-0.8b CLINC150 sem. s17 b01 GradDiff / transfer	0.416 0.356	-0.102 [-0.130, -0.075] units: [-0.261, -0.005]	-0.073 [-0.121, -0.025] units: [-0.239, +0.027]	-0.25 / -0.20	no benefit
134	qwen3.5-0.8b CLINC150 sem. s17 b01 NPO / transfer	0.416 0.356	-0.183 [-0.214, -0.151] units: [-0.395, -0.011]	-0.158 [-0.216, -0.099] units: [-0.369, +0.022]	-0.44 / -0.44	no benefit
135	qwen3.5-0.8b CLINC150 sem. s17 b01 SimNPO / transfer	0.058 0.079	+0.002 [+0.000, +0.005] units: [+0.000, +0.005]	+0.006 [+0.001, +0.011] units: [-0.002, +0.014]	0.04 / 0.07	accepted

Table S3 (PDF p. 42). Complete method results (continued): oracle fidelity, IDs 121–135.

ID	Configuration	$D_{0,F}$ / $D_{0,G}$	$\Delta O_{DF}$ [rows]; unit CI	$\Delta O_{DG}$ [rows]; unit CI	$P_F/P_G$	Status
136	qwen3.5-0.8b CLINC150 sem. s17 b01 SimNPO / transfer	0.416 0.356	-0.191 [-0.242, -0.138] units: [-0.577, +0.216]	-0.237 [-0.328, -0.148] units: [-0.628, +0.182]	-0.46 / -0.67	no benefit
137	qwen3.5-0.8b CLINC150 sem. s17 b01 Uniformization / transfer	0.058 0.079	+0.003 [+0.002, +0.005] units: [+0.001, +0.007]	+0.014 [+0.007, +0.023] units: [+0.002, +0.029]	0.06 / 0.18	accepted
138	qwen3.5-0.8b CLINC150 sem. s17 b02 SimNPO / explor.	0.298 0.319	-0.117 [-0.152, -0.083] units: [-0.297, +0.080]	-0.111 [-0.178, -0.044] units: [-0.383, +0.186]	-0.39 / -0.35	no benefit
139	qwen3.5-0.8b CLINC150 sem. s17 b02 SimNPO / explor.	0.298 0.319	-0.051 [-0.082, -0.020] units: [-0.219, +0.138]	-0.040 [-0.100, +0.022] units: [-0.290, +0.241]	-0.17 / -0.13	no benefit
140	qwen3.5-0.8b CLINC150 sem. s17 b02 SimNPO / explor.	0.298 0.319	+0.049 [+0.025, +0.074] units: [-0.085, +0.205]	+0.074 [+0.025, +0.124] units: [-0.119, +0.309]	0.17 / 0.23	accepted
141	qwen3.5-0.8b CLINC150 sem. s17 b02 temperature control / control	0.298 0.319	+0.020 [+0.018, +0.023] units: [+0.009, +0.033]	+0.032 [+0.025, +0.041] units: [+0.015, +0.050]	0.07 / 0.10	accepted
142	qwen3.5-0.8b CLINC150 sem. s17 b02 SimNPO / control	0.298 0.319	-0.018 [-0.046, +0.011] units: [-0.171, +0.157]	+0.009 [-0.048, +0.067] units: [-0.218, +0.270]	-0.06 / 0.03	no benefit
143	qwen3.5-0.8b CLINC150 sem. s17 b02 SimNPO / confirm.	0.298 0.319	+0.065 [+0.046, +0.085] units: [-0.032, +0.181]	+0.088 [+0.049, +0.129] units: [-0.058, +0.271]	0.22 / 0.28	accepted
144	qwen3.5-0.8b CLINC150 sem. s17 b03 temperature control / control	0.454 0.407	+0.024 [+0.021, +0.027] units: [+0.007, +0.046]	+0.030 [+0.023, +0.037] units: [+0.013, +0.055]	0.05 / 0.07	accepted
145	qwen3.5-0.8b CLINC150 sem. s17 b03 SimNPO / confirm.	0.454 0.407	+0.102 [+0.082, +0.121] units: [+0.001, +0.289]	+0.083 [+0.052, +0.115] units: [-0.006, +0.249]	0.22 / 0.20	accepted
146	qwen3.5-0.8b CLINC150 sem. s17 b04 SimNPO / replic.	0.481 0.410	+0.061 [+0.034, +0.089] units: [-0.192, +0.317]	+0.033 [-0.012, +0.080] units: [-0.200, +0.274]	0.13 / 0.08	positive, unconfirmed
147	qwen3.5-0.8b CLINC150 sem. s17 b05 SimNPO / replic.	0.344 0.322	+0.025 [-0.003, +0.052] units: [-0.122, +0.220]	+0.022 [-0.032, +0.078] units: [-0.182, +0.276]	0.07 / 0.07	positive, unconfirmed
148	qwen3.5-0.8b CLINC150 sem. s17 b06 SimNPO / replic.	0.207 0.233	-0.050 [-0.070, -0.030] units: [-0.234, +0.074]	-0.020 [-0.057, +0.016] units: [-0.176, +0.093]	-0.24 / -0.09	no benefit
149	qwen3.5-0.8b CLINC150 sem. s17 b07 SimNPO / replic.	0.289 0.301	-0.071 [-0.087, -0.054] units: [-0.148, +0.001]	-0.051 [-0.080, -0.021] units: [-0.137, +0.011]	-0.24 / -0.17	no benefit
150	qwen3.5-0.8b CLINC150 sem. s42 b02 SimNPO / replic.	0.346 0.349	+0.062 [+0.042, +0.083] units: [-0.012, +0.140]	+0.040 [+0.006, +0.074] units: [-0.071, +0.126]	0.18 / 0.11	accepted

Table S3 (PDF p. 43). Complete method results (continued): oracle fidelity, IDs 136–150. Intervals on the first line are paired-row intervals; unit intervals resample forgotten classes or association clusters. Configuration labels are abbreviated; IDs identify the complete original run labels in the CSV. Some source keys omit granularity. Screening status is not a simultaneous guarantee across architectures.

ID	$\Delta\mathrm{Acc}_{\mathrm{R}}$ (pp)	$\mathrm{ECE}_{\mathrm{R}}$	$\Delta\mathrm{ECE}_{\mathrm{R}}$	Rows F / G	Units
1	-0.3	0.014	-0.005	428/160	4
2	-0.1	0.021	-0.001	359/160	4
3	-0.4	0.029	+0.001	428/160	4
4	+0.0	0.022	-0.006	428/160	4
5	-0.3	0.034	+0.006	428/160	4
6	-0.6	0.025	-0.003	428/160	4
7	-0.3	0.020	-0.008	428/160	4
8	+0.0	0.028	-0.000	428/160	4
9	+0.3	0.025	-0.003	428/160	4
10	+1.0	0.024	-0.007	428/160	4
11	-0.6	0.033	+0.004	428/160	4
12	-0.3	0.041	+0.010	428/160	4
13	-0.2	0.026	+0.003	428/160	4
14	-0.4	0.031	+0.003	428/160	4
15	+0.4	0.032	+0.003	428/160	4
16	+0.7	0.027	-0.004	428/160	4
17	+0.1	0.018	+0.000	428/160	4
18	+0.0	0.023	-0.001	382/160	4
19	+0.4	0.026	-0.002	428/160	4
20	+1.4	0.020	-0.007	428/160	4
21	+0.2	0.035	+0.004	428/160	4
22	+0.1	0.024	-0.001	428/160	4
23	+0.0	0.034	+0.006	428/160	4
24	+0.1	0.022	-0.005	428/160	4
25	-0.1	0.025	-0.003	359/160	4
26	+0.1	0.016	-0.007	428/160	4
27	+0.3	0.033	+0.003	428/160	4
28	-0.3	0.040	+0.006	382/160	4
29	-1.9	0.028	-0.001	428/160	4
30	-0.7	0.025	-0.005	382/160	4

Table S3 (PDF p. 44). Complete method results (continued): retained utility and evaluation counts. IDs 1–30.

ID	$\Delta$ Acc $_{\mathrm{R}}$ (pp)	ECE $_{\mathrm{R}}$	$\Delta$ ECE $_{\mathrm{R}}$	Rows F / G	Units
31	-0.1	0.039	+0.013	428/160	4
32	+0.2	0.037	+0.011	428/160	4
33	-2.1	0.061	+0.035	428/160	4
34	+0.2	0.029	+0.003	428/160	4
35	-0.1	0.028	+0.001	428/160	4
36	+0.2	0.072	+0.046	428/160	4
37	-2.1	0.047	+0.019	427/175	10
38	-0.1	0.023	-0.005	427/175	10
39	-0.2	0.016	-0.012	427/175	10
40	-0.9	0.045	+0.017	427/175	10
41	-2.0	0.031	+0.003	427/175	10
42	-9.6	0.126	+0.098	427/175	10
43	-2.2	0.064	+0.038	428/160	4
44	-4.2	0.085	+0.057	427/175	10
45	-1.4	0.065	+0.039	428/160	4
46	-3.0	0.072	+0.046	428/160	4
47	+0.0	0.023	-0.003	428/160	4
48	-0.2	0.024	-0.002	428/160	4
49	+0.1	0.027	+0.000	428/160	4
50	-4.0	0.071	+0.043	427/175	10
51	-1.0	0.050	+0.024	428/160	4
52	-0.2	0.028	-0.000	427/175	10
53	-0.1	0.024	-0.002	428/160	4
54	-0.8	0.181	+0.153	427/175	10
55	+0.7	0.079	+0.053	428/160	4
56	-0.5	0.062	+0.031	382/160	4
57	-0.4	0.057	+0.027	382/160	4
58	-0.4	0.052	+0.022	382/160	4
59	+0.3	0.018	-0.012	382/160	4
60	-0.2	0.033	+0.001	407/146	6

Table S3 (PDF p. 45). Complete method results (continued): retained utility and evaluation counts. IDs 31–60.

ID	$\Delta$ Acc $_{\mathrm{R}}$ (pp)	ECE $_{\mathrm{R}}$	$\Delta$ ECE $_{\mathrm{R}}$	Rows F / G	Units
61	+0.0	0.015	-0.017	407/146	6
62	+0.3	0.034	+0.002	407/146	6
63	-0.7	0.638	+0.609	494/160	4
64	-0.4	0.144	+0.115	494/160	4
65	-0.3	0.037	+0.008	494/160	4
66	-0.1	0.039	+0.010	494/160	4
67	+0.5	0.024	-0.001	386/149	8
68	+0.3	0.016	-0.009	386/149	8
69	+0.5	0.037	+0.012	386/149	8
70	-1.4	0.057	+0.030	386/147	386
71	-0.5	0.051	+0.024	386/147	386
72	-0.5	0.051	+0.024	386/147	386
73	+0.0	0.029	+0.002	386/147	386
74	+0.6	0.491	+0.464	386/147	386
75	+0.4	0.050	+0.024	428/160	4
76	+0.2	0.047	+0.022	428/160	4
77	+0.1	0.039	+0.014	428/160	4
78	-0.9	0.021	-0.004	428/160	4
79	+0.4	0.035	+0.007	427/175	10
80	+0.3	0.013	-0.015	427/175	10
81	+0.5	0.022	-0.006	427/175	10
82	-0.7	0.058	+0.028	382/160	4
83	-0.7	0.056	+0.026	382/160	4
84	-0.6	0.039	+0.010	382/160	4
85	+0.9	0.031	+0.001	382/160	4
86	+0.3	0.027	+0.000	407/146	6
87	+0.0	0.010	-0.017	407/146	6
88	+0.1	0.045	+0.018	407/146	6
89	-1.6	0.317	+0.290	494/160	4
90	-0.4	0.040	+0.013	494/160	4

Table S3 (PDF p. 46). Table S3. Complete method results (continued): retained utility and evaluation counts. IDs 61–90.

ID	$\Delta\mathrm{Acc}_{\mathrm{R}}$ (pp)	$\mathrm{ECE}_{\mathrm{R}}$	$\Delta\mathrm{ECE}_{\mathrm{R}}$	Rows F / G	Units
91	-1.4	0.218	+0.191	494/160	4
92	-0.1	0.036	+0.009	494/160	4
93	+0.2	0.030	+0.006	386/149	8
94	-0.1	0.011	-0.013	386/149	8
95	+0.6	0.032	+0.008	386/149	8
96	+0.1	0.055	+0.020	428/160	4
97	+0.0	0.049	+0.014	428/160	4
98	+0.1	0.032	-0.002	428/160	4
99	+0.1	0.015	-0.020	428/160	4
100	+0.0	0.033	+0.000	427/175	10
101	-0.5	0.022	-0.010	427/175	10
102	-1.0	0.049	+0.016	427/175	10
103	-0.5	0.050	+0.013	382/160	4
104	-0.4	0.050	+0.013	382/160	4
105	+0.3	0.043	+0.006	382/160	4
106	-0.9	0.030	-0.007	382/160	4
107	+0.0	0.038	+0.001	407/146	6
108	-0.3	0.020	-0.017	407/146	6
109	-0.1	0.042	+0.005	407/146	6
110	-0.8	0.069	+0.034	494/160	4
111	-0.7	0.059	+0.025	494/160	4
112	-0.9	0.041	+0.006	494/160	4
113	-1.1	0.031	-0.004	494/160	4
114	-0.7	0.031	-0.000	386/149	8
115	-0.5	0.021	-0.010	386/149	8
116	-0.3	0.045	+0.014	386/149	8
117	-0.8	0.040	+0.016	428/160	4
118	-0.6	0.026	+0.003	428/160	4
119	+0.4	0.026	+0.001	428/160	4
120	+0.4	0.021	-0.007	382/160	4

Table S3 (PDF p. 47). Complete method results (continued): retained utility and evaluation counts. IDs 91–120.

ID	$\Delta$ AccR (pp)	ECER	$\Delta$ ECER	Rows F / G	Units
121	-1.2	0.028	+0.001	428/160	4
122	-0.7	0.022	-0.005	428/160	4
123	-0.9	0.022	-0.005	428/160	4
124	-1.0	0.023	-0.004	428/160	4
125	-0.5	0.023	-0.004	428/160	4
126	-0.9	0.021	-0.005	428/160	4
127	-0.2	0.030	+0.000	428/160	4
128	-0.2	0.010	-0.011	785/240	8
129	-0.1	0.022	-0.005	791/240	8
130	+0.2	0.021	-0.008	780/240	8
131	+0.5	0.015	-0.015	789/240	8
132	-1.0	0.041	+0.017	785/240	8
133	-1.1	0.040	+0.015	785/240	8
134	-1.1	0.040	+0.016	785/240	8
135	+0.0	0.025	+0.001	768/236	18
136	-0.4	0.015	-0.009	785/240	8
137	-0.1	0.021	-0.003	768/236	18
138	-0.9	0.022	-0.001	791/240	8
139	-0.2	0.024	+0.001	791/240	8
140	+0.3	0.015	-0.009	791/240	8
141	+0.0	0.026	+0.002	791/240	8
142	+0.3	0.013	-0.010	791/240	8
143	+0.4	0.011	-0.012	791/240	8
144	+0.0	0.027	+0.003	780/240	8
145	+0.1	0.021	-0.002	780/240	8
146	+0.3	0.014	-0.010	778/240	8
147	+0.1	0.019	-0.001	790/240	8
148	+0.1	0.020	-0.005	789/240	8
149	+0.3	0.018	-0.005	784/240	8
150	+0.0	0.017	-0.004	791/240	8

Table S3 (PDF p. 48). Complete method results (continued): retained utility and evaluation counts. IDs 121–150.

IDs match the preceding panel; no estimate is recomputed from rounded entries.

Model	Seed / bundle; stage	$P_F$ [rows]; class CI	$P_G$ [rows]	$\Delta P_F/\Delta P_G$ vs. seed 17, b01	$\Delta\mathrm{Acc}_R$ (pp)	Status
Apertus-8B	17 / b01 screening	0.55 [0.50, 0.60] classes: [0.32, 0.74]	0.58 [0.47, 0.66]	— / —	-0.3	accepted
Apertus-8B	42 / b01 replication	0.17 [0.10, 0.23] classes: [-0.33, 0.61]	0.25 [0.13, 0.35]	-0.38 / -0.33	+0.0	accepted
Apertus-8B	17 / b02 replication	0.56 [0.53, 0.59] classes: [0.47, 0.62]	0.57 [0.52, 0.62]	+0.02 / -0.00	-0.1	accepted
Falcon-H1-3B	17 / b01 screening	0.36 [0.32, 0.41] classes: [0.16, 0.69]	0.42 [0.35, 0.49]	— / —	+0.3	accepted
Gemma-4-E2B	17 / b01 screening	0.41 [0.36, 0.46] classes: [0.25, 0.59]	0.44 [0.35, 0.52]	— / —	+1.0	accepted
Granite-4.0-Micro	17 / b01 screening	0.46 [0.42, 0.51] classes: [0.34, 0.59]	0.55 [0.48, 0.62]	— / —	-0.6	accepted
LFM2.5-1.2B	17 / b01 screening	0.30 [0.23, 0.36] classes: [0.09, 0.43]	0.23 [0.07, 0.36]	— / —	-0.3	accepted
Llama-3.2-1B	17 / b01 screening	0.57 [0.53, 0.60] classes: [0.46, 0.71]	0.54 [0.47, 0.61]	— / —	-0.2	accepted
Llama-3.2-3B	17 / b01 screening	0.50 [0.46, 0.55] classes: [0.31, 0.70]	0.56 [0.48, 0.63]	— / —	-0.4	accepted
Ministral-3-3B	17 / b01 screening	0.28 [0.23, 0.34] classes: [0.13, 0.47]	0.21 [0.08, 0.33]	— / —	+0.4	accepted
Nemotron-Nano-9B	17 / b01 screening	0.46 [0.41, 0.51] classes: [0.21, 0.72]	0.53 [0.42, 0.62]	— / —	+0.7	accepted
OLMo-3-7B	17 / b01 screening	0.34 [0.30, 0.39] classes: [0.18, 0.61]	0.35 [0.26, 0.44]	— / —	+0.1	accepted
OLMo-3-7B	42 / b01 replication	0.04 [-0.05, 0.11] classes: [-0.62, 0.51]	0.00 [-0.16, 0.13]	-0.30 / -0.35	+0.4	positive, unconfirmed
OLMo-3-7B	17 / b02 replication	0.50 [0.46, 0.54] classes: [0.14, 0.73]	0.49 [0.42, 0.55]	+0.16 / +0.14	+0.0	accepted
Phi-4-mini	17 / b01 screening	0.47 [0.42, 0.52] classes: [0.28, 0.68]	0.51 [0.40, 0.60]	— / —	+1.4	accepted

Table S4. Architecture screening and seed / bundle replication of frozen class SimNPO (BANKING77).

Model	Seed / bundle; stage	$P_F$ [rows]; class CI	$P_G$ [rows]	$\Delta P_F/\Delta P_G$ vs. seed 17, b01	$\Delta\mathrm{Acc}_R$ (pp)	Status
Qwen2.5-1.5B	17 / b01 screening	0.45 [0.41, 0.50] classes: [0.18, 0.68]	0.54 [0.47, 0.62]	— / —	+0.2	accepted
Qwen2.5-3B	17 / b01 screening	0.50 [0.45, 0.55] classes: [0.25, 0.74]	0.60 [0.52, 0.68]	— / —	+0.1	accepted
Qwen2.5-7B	17 / b01 screening	0.63 [0.58, 0.67] classes: [0.51, 0.79]	0.63 [0.56, 0.70]	— / —	+0.0	accepted
Qwen3-4B	17 / b01 screening	0.28 [0.21, 0.35] classes: [0.10, 0.41]	0.19 [-0.01, 0.34]	— / —	+0.1	positive, unconfirmed
Qwen3-4B	42 / b01 replication	-0.18 [-0.34, -0.03] classes: [-1.90, 0.58]	-0.10 [-0.41, 0.13]	-0.46 / -0.29	+0.1	no benefit
Qwen3-4B	17 / b02 replication	0.51 [0.47, 0.55] classes: [0.35, 0.64]	0.50 [0.44, 0.56]	+0.23 / +0.32	-0.1	accepted
Qwen3.5-0.8B	17 / b01 confirmation	0.59 [0.55, 0.63] classes: [0.49, 0.76]	0.57 [0.49, 0.64]	— / —	-0.1	accepted
Qwen3.5-0.8B	42 / b01 replication	0.55 [0.50, 0.60] classes: [0.40, 0.76]	0.51 [0.40, 0.60]	-0.04 / -0.06	-0.9	accepted
Qwen3.5-0.8B	17 / b02 replication	0.70 [0.66, 0.73] classes: [0.44, 0.83]	0.73 [0.68, 0.78]	+0.11 / +0.17	+0.3	accepted
Qwen3.5-2B	17 / b01 screening	0.46 [0.41, 0.51] classes: [0.23, 0.62]	0.49 [0.37, 0.59]	— / —	-0.8	accepted
Qwen3.5-4B	17 / b01 screening	0.20 [0.15, 0.24] classes: [0.10, 0.32]	0.18 [0.07, 0.28]	— / —	-0.6	accepted
Qwen3.5-9B	17 / b01 screening	0.54 [0.49, 0.59] classes: [0.35, 0.76]	0.58 [0.51, 0.65]	— / —	+0.4	accepted
Qwen3.5-9B	42 / b01 replication	0.14 [0.11, 0.17] classes: [0.07, 0.25]	0.18 [0.13, 0.24]	-0.40 / -0.40	-0.9	accepted
Qwen3.5-9B	17 / b02 replication	0.44 [0.38, 0.49] classes: [0.07, 0.78]	0.43 [0.36, 0.51]	-0.10 / -0.15	+0.4	accepted
SmolLM3-3B	17 / b01 screening	0.50 [0.44, 0.55] classes: [0.24, 0.66]	0.46 [0.34, 0.57]	— / —	-0.2	accepted

Table S4 (PDF p. 49). Architecture screening and seed / bundle replication (continued).

Dataset / bundle / seed	Labels; status	Full accuracy F / G	Oracle accuracy F / G	Oracle forgotten mass F / G	$D_{0,F}/D_{0,G}$	$P_F/P_G$	Opaque acquisition
clinc150 b02, seed 17	semantic found	0.995 / 0.983	0.6612 / 0.6333	6.400e-01 / 6.079e-01	0.298 / 0.319	0.22 / 0.28	—
clinc150 b02, seed 17	opaque found	0.992 / 0.971	0.0013 / 0.0000	3.153e-03 / 2.200e-03	0.983 / 0.976	0.28 / 0.29	pass; min class acc. 0.85 (semantic 0.85), min gold p 0.85
clinc150 b03, seed 17	semantic found	0.996 / 0.983	0.5051 / 0.5333	5.047e-01 / 5.316e-01	0.454 / 0.407	0.22 / 0.20	—
clinc150 b03, seed 17	opaque found	0.994 / 0.971	0.0000 / 0.0000	6.001e-04 / 7.282e-04	0.993 / 0.973	0.13 / 0.11	pass; min class acc. 0.89 (semantic 0.74), min gold p 0.87
clinc150 b06, seed 17	semantic found	0.999 / 0.979	0.7807 / 0.7375	7.705e-01 / 7.241e-01	0.207 / 0.233	-0.24 / -0.09	—
clinc150 b06, seed 17	opaque found	0.996 / 0.983	0.0000 / 0.0000	1.247e-03 / 1.292e-03	0.999 / 0.989	0.02 / 0.02	pass; min class acc. 0.94 (semantic 0.89), min gold p 0.93

Table S5 (PDF p. 50). Semantic versus opaque decision labels (continued): CLINC150.

All seven opaque acquisition checks pass. The semantic seed-42 b02 condition includes one primary-seed-selected class below the acquisition threshold at replication. Masses use the precision available in the source export.

ID	Configuration	Schema / view	$D_{0,v}$	$P_v$ [rows]	Class CI	Oracle accuracy
1	apertus-8b BANKING77 sem. _01 SimNPO	S0_original 77 options; F	0.479	0.55 [0.50, 0.60]	[0.31, 0.72]	0.395
2	apertus-8b BANKING77 sem. _01 SimNPO	S0_original 77 options; G	0.400	0.58 [0.48, 0.66]	[0.15, 0.77]	0.450
3	apertus-8b BANKING77 sem. _01 SimNPO	S1_neighbors 12 options; F	0.204	0.40 [0.32, 0.47]	[0.22, 0.61]	0.682
4	apertus-8b BANKING77 sem. _01 SimNPO	S1_neighbors 12 options; G	0.132	0.36 [0.14, 0.52]	[0.09, 0.57]	0.775
5	apertus-8b BANKING77 sem. _01 SimNPO	S2_distractors 12 options; F	0.122	0.55 [0.48, 0.61]	[-0.12, 0.68]	0.827
6	apertus-8b BANKING77 sem. _01 SimNPO	S2_distractors 12 options; G	0.150	0.68 [0.60, 0.75]	[-0.45, 0.77]	0.775
7	qwen3.5-0.8b BANKING77 sem. _01 SimNPO	S0_original 77 options; F	0.608	0.59 [0.55, 0.63]	[0.39, 0.73]	0.269
8	qwen3.5-0.8b BANKING77 sem. _01 NPO	S0_original 77 options; F	0.608	0.39 [0.34, 0.43]	[0.21, 0.59]	0.269
9	qwen3.5-0.8b BANKING77 sem. _01 GradAscent	S0_original 77 options; F	0.608	0.55 [0.50, 0.59]	[0.41, 0.62]	0.269
10	qwen3.5-0.8b BANKING77 sem. _01 GradDiff	S0_original 77 options; F	0.608	0.45 [0.41, 0.50]	[0.31, 0.60]	0.269
11	qwen3.5-0.8b BANKING77 sem. _01 SimNPO	S0_original 77 options; G	0.495	0.57 [0.49, 0.64]	[0.40, 0.71]	0.306
12	qwen3.5-0.8b BANKING77 sem. _01 NPO	S0_original 77 options; G	0.495	0.36 [0.28, 0.43]	[0.18, 0.52]	0.306
13	qwen3.5-0.8b BANKING77 sem. _01 GradAscent	S0_original 77 options; G	0.495	0.49 [0.41, 0.56]	[0.38, 0.58]	0.306
14	qwen3.5-0.8b BANKING77 sem. _01 GradDiff	S0_original 77 options; G	0.495	0.42 [0.35, 0.50]	[0.27, 0.57]	0.306

Table S6. Candidate-schema results with paired uncertainty.

ID	Configuration	Schema / view	$D_{0,v}$	$P_v$ [rows]	Class CI	Oracle accuracy
15	qwen3.5-0.8b BANKING77 sem. _01 SimNPO	S1_neighbors 12 options; F	0.438	0.66 [0.61, 0.70]	[0.50, 0.74]	0.460
16	qwen3.5-0.8b BANKING77 sem. _01 NPO	S1_neighbors 12 options; F	0.438	0.41 [0.36, 0.47]	[0.14, 0.65]	0.460
17	qwen3.5-0.8b BANKING77 sem. _01 GradAscent	S1_neighbors 12 options; F	0.438	0.62 [0.57, 0.67]	[0.50, 0.67]	0.460
18	qwen3.5-0.8b BANKING77 sem. _01 GradDiff	S1_neighbors 12 options; F	0.438	0.47 [0.41, 0.52]	[0.24, 0.66]	0.460
19	qwen3.5-0.8b BANKING77 sem. _01 SimNPO	S1_neighbors 12 options; G	0.339	0.61 [0.51, 0.70]	[0.41, 0.71]	0.519
20	qwen3.5-0.8b BANKING77 sem. _01 NPO	S1_neighbors 12 options; G	0.339	0.38 [0.28, 0.48]	[0.07, 0.60]	0.519
21	qwen3.5-0.8b BANKING77 sem. _01 GradAscent	S1_neighbors 12 options; G	0.339	0.54 [0.44, 0.63]	[0.35, 0.67]	0.519
22	qwen3.5-0.8b BANKING77 sem. _01 GradDiff	S1_neighbors 12 options; G	0.339	0.43 [0.33, 0.52]	[0.14, 0.61]	0.519
23	qwen3.5-0.8b BANKING77 sem. _01 SimNPO	S2_distractors 12 options; F	0.148	0.32 [0.27, 0.37]	[0.13, 0.45]	0.799
24	qwen3.5-0.8b BANKING77 sem. _01 NPO	S2_distractors 12 options; F	0.148	0.12 [0.07, 0.17]	[0.02, 0.20]	0.799
25	qwen3.5-0.8b BANKING77 sem. _01 GradAscent	S2_distractors 12 options; F	0.148	0.17 [0.12, 0.23]	[0.11, 0.23]	0.799
26	qwen3.5-0.8b BANKING77 sem. _01 GradDiff	S2_distractors 12 options; F	0.148	0.12 [0.07, 0.18]	[0.03, 0.19]	0.799
27	qwen3.5-0.8b BANKING77 sem. _01 SimNPO	S2_distractors 12 options; G	0.133	0.43 [0.35, 0.51]	[0.16, 0.56]	0.756
28	qwen3.5-0.8b BANKING77 sem. _01 NPO	S2_distractors 12 options; G	0.133	0.13 [0.03, 0.22]	[-0.10, 0.21]	0.756
29	qwen3.5-0.8b BANKING77 sem. _01 GradAscent	S2_distractors 12 options; G	0.133	0.17 [0.09, 0.27]	[-0.06, 0.26]	0.756
30	qwen3.5-0.8b BANKING77 sem. _01 GradDiff	S2_distractors 12 options; G	0.133	0.13 [0.03, 0.23]	[-0.12, 0.20]	0.756

Table S6 (PDF p. 52). Candidate-schema results (continued): BANKING77: alternative candidates.

ID	Configuration	Schema / view	$D_{0,v}$	$P_v$ [rows]	Class CI	Oracle accuracy
31	qwen3.5-0.8b BANKING77 opaque _01 SimNPO	S0_original 77 options; F	0.943	0.61 [0.57, 0.64]	[0.44, 0.76]	0.000
32	qwen3.5-0.8b BANKING77 opaque _01 SimNPO	S0_original 77 options; G	0.881	0.62 [0.57, 0.68]	[0.44, 0.75]	0.000
33	qwen3.5-0.8b BANKING77 opaque _01 SimNPO	S1_neighbors 12 options; F	0.464	0.00 [-0.03, 0.03]	[-0.15, 0.15]	0.016
34	qwen3.5-0.8b BANKING77 opaque _01 SimNPO	S1_neighbors 12 options; G	0.489	0.01 [-0.03, 0.04]	[-0.09, 0.09]	0.050
35	qwen3.5-0.8b BANKING77 opaque _01 SimNPO	S2_distractors 12 options; F	0.583	0.12 [0.07, 0.16]	[-0.08, 0.46]	0.000
36	qwen3.5-0.8b BANKING77 opaque _01 SimNPO	S2_distractors 12 options; G	0.553	0.14 [0.07, 0.20]	[-0.11, 0.48]	0.000
37	qwen3.5-0.8b BANKING77 opaque _02 SimNPO	S0_original 77 options; F	0.913	0.45 [0.41, 0.49]	[0.26, 0.71]	0.000
38	qwen3.5-0.8b BANKING77 opaque _02 SimNPO	S0_original 77 options; G	0.872	0.43 [0.37, 0.49]	[0.25, 0.61]	0.000
39	qwen3.5-0.8b BANKING77 opaque _02 SimNPO	S1_neighbors 12 options; F	0.714	0.04 [0.03, 0.05]	[0.01, 0.12]	0.000
40	qwen3.5-0.8b BANKING77 opaque _02 SimNPO	S1_neighbors 12 options; G	0.691	0.04 [0.03, 0.06]	[0.01, 0.09]	0.000
41	qwen3.5-0.8b BANKING77 opaque _02 SimNPO	S2_distractors 12 options; F	0.530	0.12 [0.11, 0.14]	[0.01, 0.17]	0.005
42	qwen3.5-0.8b BANKING77 opaque _02 SimNPO	S2_distractors 12 options; G	0.507	0.13 [0.10, 0.15]	[0.03, 0.17]	0.019
43	qwen3.5-9b BANKING77 sem. _01 SimNPO	S0_original 77 options; F	0.414	0.54 [0.49, 0.59]	[0.33, 0.72]	0.484
44	qwen3.5-9b BANKING77 sem. _01 SimNPO	S0_original 77 options; G	0.352	0.58 [0.51, 0.66]	[0.37, 0.70]	0.487
45	qwen3.5-9b BANKING77 sem. _01 SimNPO	S1_neighbors 12 options; F	0.187	0.53 [0.45, 0.60]	[0.16, 0.75]	0.745
46	qwen3.5-9b BANKING77 sem. _01 SimNPO	S1_neighbors 12 options; G	0.139	0.57 [0.43, 0.69]	[0.12, 0.73]	0.787
47	qwen3.5-9b BANKING77 sem. _01 SimNPO	S2_distractors 12 options; F	0.167	0.43 [0.36, 0.50]	[0.03, 0.59]	0.771
48	qwen3.5-9b BANKING77 sem. _01 SimNPO	S2_distractors 12 options; G	0.174	0.54 [0.45, 0.63]	[0.09, 0.63]	0.738

Table S6 (PDF p. 53). Candidate-schema results (continued): BANKING77: opaque labels and 9B.

ID	Configuration	Schema / view	$D_{0,v}$	$P_v$ [rows]	Class CI	Oracle accuracy
49	qwen3.5-0.8b CLINC150 sem. _02 SimNPO	S0_original 151 options; F	0.298	0.22 [0.16, 0.27]	[-0.15, 0.43]	0.661
50	qwen3.5-0.8b CLINC150 sem. _02 SimNPO	S0_original 151 options; F	0.298	-0.06 [-0.17, 0.04]	[-0.91, 0.35]	0.661
51	qwen3.5-0.8b CLINC150 sem. _02 SimNPO	S0_original 151 options; G	0.319	0.28 [0.17, 0.37]	[-0.29, 0.51]	0.633
52	qwen3.5-0.8b CLINC150 sem. _02 SimNPO	S0_original 151 options; G	0.319	0.03 [-0.17, 0.19]	[-1.44, 0.48]	0.633
53	qwen3.5-0.8b CLINC150 sem. _02 SimNPO	S1_neighbors 24 options; F	0.151	0.20 [0.10, 0.28]	[-0.54, 0.45]	0.827
54	qwen3.5-0.8b CLINC150 sem. _02 SimNPO	S1_neighbors 24 options; F	0.151	-0.38 [-0.62, -0.19]	[-2.09, 0.28]	0.827
55	qwen3.5-0.8b CLINC150 sem. _02 SimNPO	S1_neighbors 24 options; G	0.199	0.32 [0.18, 0.43]	[-0.56, 0.59]	0.762
56	qwen3.5-0.8b CLINC150 sem. _02 SimNPO	S1_neighbors 24 options; G	0.199	-0.02 [-0.32, 0.19]	[-2.59, 0.56]	0.762
57	qwen3.5-0.8b CLINC150 sem. _02 SimNPO	S2_distractors 24 options; F	0.039	0.06 [0.03, 0.10]	[0.02, 0.14]	0.957
58	qwen3.5-0.8b CLINC150 sem. _02 SimNPO	S2_distractors 24 options; F	0.039	0.06 [0.03, 0.10]	[0.02, 0.14]	0.957
59	qwen3.5-0.8b CLINC150 sem. _02 SimNPO	S2_distractors 24 options; G	0.029	0.12 [0.04, 0.21]	[0.04, 0.24]	0.963
60	qwen3.5-0.8b CLINC150 sem. _02 SimNPO	S2_distractors 24 options; G	0.029	0.15 [0.05, 0.25]	[0.04, 0.29]	0.963
61	qwen3.5-0.8b CLINC150 sem. _03 SimNPO	S0_original 151 options; F	0.454	0.22 [0.19, 0.26]	[0.01, 0.54]	0.505
62	qwen3.5-0.8b CLINC150 sem. _03 SimNPO	S0_original 151 options; G	0.407	0.20 [0.13, 0.28]	[-0.02, 0.55]	0.533
63	qwen3.5-0.8b CLINC150 sem. _03 SimNPO	S1_neighbors 24 options; F	0.348	0.22 [0.18, 0.26]	[0.01, 0.51]	0.601
64	qwen3.5-0.8b CLINC150 sem. _03 SimNPO	S1_neighbors 24 options; G	0.272	0.26 [0.16, 0.35]	[-0.02, 0.60]	0.683
65	qwen3.5-0.8b CLINC150 sem. _03 SimNPO	S2_distractors 24 options; F	0.027	-0.00 [-0.01, 0.00]	[-0.01, 0.00]	0.971
66	qwen3.5-0.8b CLINC150 sem. _03 SimNPO	S2_distractors 24 options; G	0.038	-0.03 [-0.09, 0.01]	[-0.10, 0.02]	0.963

Table S6 (PDF p. 54). Candidate-schema results (continued): CLINC150.

Candidates are re-scored by forward passes. The source bundle label _01 is retained as exported; it denotes the bundle-01 comparison. Ratios are recomputed within each bootstrap, rather than averaging row-level ratios.

ID	Configuration	$D_{0,\mathrm{F}}$	$\Delta O\!D_{\mathrm{F}}$ [rows]; intent CI	$\Delta O\!D_{\mathrm{G}}$ [rows]; intent CI	$P_{\mathrm{F}}/P_{\mathrm{G}}$	$\Delta\mathrm{Acc}_{\mathrm{R}}$ (pp)	Status
1	llama3.2-1b seed 17, b01 transfer_simnpo transfer	0.453	-0.079 [-0.125, -0.034] intents: [-0.424, +0.281]	-0.082 [-0.159, -0.006] intents: [-0.403, +0.241]	-0.18 / -0.20	-0.2	no benefit
2	qwen3.5-0.8b seed 17, b01 transfer_simnpo transfer	0.058	+0.002 [+0.000, +0.005] intents: [+0.000, +0.005]	+0.006 [+0.001, +0.011] intents: [-0.002, +0.014]	0.04 / 0.07	+0.0	accepted
3	qwen3.5-0.8b seed 17, b01 transfer_uniformization transfer	0.058	+0.003 [+0.002, +0.005] intents: [+0.001, +0.007]	+0.014 [+0.007, +0.023] intents: [+0.002, +0.029]	0.06 / 0.18	-0.1	accepted
4	qwen3.5-0.8b seed 17, b01 transfer_grad_ascent transfer	0.416	-0.185 [-0.217, -0.152] intents: [-0.404, -0.001]	-0.168 [-0.228, -0.109] intents: [-0.387, +0.021]	-0.44 / -0.47	-1.0	no benefit
5	qwen3.5-0.8b seed 17, b01 transfer_grad_diff transfer	0.416	-0.102 [-0.130, -0.075] intents: [-0.261, -0.005]	-0.073 [-0.121, -0.025] intents: [-0.239, +0.027]	-0.25 / -0.20	-1.1	no benefit
6	qwen3.5-0.8b seed 17, b01 transfer_npo transfer	0.416	-0.183 [-0.214, -0.151] intents: [-0.395, -0.011]	-0.158 [-0.216, -0.099] intents: [-0.369, +0.022]	-0.44 / -0.44	-1.1	no benefit
7	qwen3.5-0.8b seed 17, b01 transfer_simnpo transfer	0.416	-0.191 [-0.242, -0.138] intents: [-0.577, +0.216]	-0.237 [-0.328, -0.148] intents: [-0.628, +0.182]	-0.46 / -0.67	-0.4	no benefit

Table S7. CLINC150: every observed bundle, frozen transfer, tuned confirmation and replication.

ID	Configuration	$D_{0,\mathrm{F}}$	$\Delta O\!D_{\mathrm{F}}$ [rows]; intent CI	$\Delta O\!D_{\mathrm{G}}$ [rows]; intent CI	$P_{\mathrm{F}}/P_{\mathrm{G}}$	$\Delta\mathrm{Acc}_{\mathrm{R}}$ (pp)	Status
8	qwen3.5-0.8b seed 17, b02 matched s1__simnpo_b2.5_ep7 exploratory	0.298	-0.117 [-0.152, -0.083] intents: [-0.297, +0.080]	-0.111 [-0.178, -0.044] intents: [-0.383, +0.186]	-0.39 / -0.35	-0.9	no benefit
9	qwen3.5-0.8b seed 17, b02 matched s1__simnpo_b4.5_ep9 exploratory	0.298	-0.051 [-0.082, -0.020] intents: [-0.219, +0.138]	-0.040 [-0.100, +0.022] intents: [-0.290, +0.241]	-0.17 / -0.13	-0.2	no benefit
10	qwen3.5-0.8b seed 17, b02 matched s1__simnpo_b6.5_ep10 exploratory	0.298	+0.049 [+0.025, +0.074] intents: [-0.085, +0.205]	+0.074 [+0.025, +0.124] intents: [-0.119, +0.309]	0.17 / 0.23	+0.3	accepted
11	qwen3.5-0.8b seed 17, b02 temperature_control control	0.298	+0.020 [+0.018, +0.023] intents: [+0.009, +0.033]	+0.032 [+0.025, +0.041] intents: [+0.015, +0.050]	0.07 / 0.10	+0.0	accepted
12	qwen3.5-0.8b seed 17, b02 transfer_simnpo transfer	0.298	-0.018 [-0.046, +0.011] intents: [-0.171, +0.157]	+0.009 [-0.048, +0.067] intents: [-0.218, +0.270]	-0.06 / 0.03	+0.3	no benefit
13	qwen3.5-0.8b seed 17, b02 tuned_simnpo confirmation	0.298	+0.065 [+0.046, +0.085] intents: [-0.032, +0.181]	+0.088 [+0.049, +0.129] intents: [-0.058, +0.271]	0.22 / 0.28	+0.4	accepted

Table S7 (PDF p. 55). CLINC150 results (continued): bundle-02 tuning and controls.

ID	Configuration	$D_{0,F}$	$\Delta O D_F$ [rows]; intent CI	$\Delta O D_G$ [rows]; intent CI	$P_F/P_G$	$\Delta Acc_R$ (pp)	Status
14	qwen3.5-0.8b seed 17, b03 temperature_control control	0.454	+0.024 [+0.021, +0.027] intents: [+0.007, +0.046]	+0.030 [+0.023, +0.037] intents: [+0.013, +0.055]	0.05 / 0.07	+0.0	accepted
15	qwen3.5-0.8b seed 17, b03 tuned_simnpo confirmation	0.454	+0.102 [+0.082, +0.121] intents: [+0.001, +0.289]	+0.083 [+0.052, +0.115] intents: [-0.006, +0.249]	0.22 / 0.20	+0.1	accepted
16	qwen3.5-0.8b seed 17, b04 tuned_simnpo replication	0.481	+0.061 [+0.034, +0.089] intents: [-0.192, +0.317]	+0.033 [-0.012, +0.080] intents: [-0.200, +0.274]	0.13 / 0.08	+0.3	positive, unconfirmed
17	qwen3.5-0.8b seed 17, b05 tuned_simnpo replication	0.344	+0.025 [-0.003, +0.052] intents: [-0.122, +0.220]	+0.022 [-0.032, +0.078] intents: [-0.182, +0.276]	0.07 / 0.07	+0.1	positive, unconfirmed
18	qwen3.5-0.8b seed 17, b06 tuned_simnpo replication	0.207	-0.050 [-0.070, -0.030] intents: [-0.234, +0.074]	-0.020 [-0.057, +0.016] intents: [-0.176, +0.093]	-0.24 / -0.09	+0.1	no benefit
19	qwen3.5-0.8b seed 17, b07 tuned_simnpo replication	0.289	-0.071 [-0.087, -0.054] intents: [-0.148, +0.001]	-0.051 [-0.080, -0.021] intents: [-0.137, +0.011]	-0.24 / -0.17	+0.3	no benefit
20	qwen3.5-0.8b seed 42, b02 tuned_simnpo replication	0.346	+0.062 [+0.042, +0.083] intents: [-0.012, +0.140]	+0.040 [+0.006, +0.074] intents: [-0.071, +0.126]	0.18 / 0.11	+0.0	accepted

Table S7 (PDF p. 56). CLINC150 results (continued): confirmation and replication.

Some visible source keys repeat with different displacement and row counts; the export omits a distinguishing granularity field. They are retained as separate source rows, without assigning a missing label. The main-text class result is the bundle-01 row with $D_{0,F} = 0.416$ .

ID	Dataset / run	View	Dev. $s$	Held-out $s$ [rows]; class CI	Aligned $a$ [rows]	Relative residual [rows]	$P_F$ ; gate	Identity error; zero directions
1	banking77 matched_simnpo	FG	0.64	0.78 [0.74, 0.83] classes: [0.48, 1.07]	0.67 [0.63, 0.71]	0.36 [0.32, 0.39]	0.56 feasible: True	1.1e-15 zero rows: 0
2	banking77 matched_simnpo	F	0.64	0.79 [0.74, 0.84] classes: [0.48, 1.07]	0.68 [0.63, 0.72]	0.36 [0.32, 0.39]	0.56 feasible: True	1.1e-15 zero rows: 0
3	banking77 matched_simnpo	G	0.64	0.77 [0.68, 0.86] classes: [0.45, 1.09]	0.65 [0.57, 0.73]	0.35 [0.29, 0.41]	0.56 feasible: True	1.1e-15 zero rows: 0
4	banking77 confirm_npo	FG	0.59	0.40 [0.36, 0.44] classes: [0.21, 0.60]	0.39 [0.35, 0.44]	0.15 [0.12, 0.18]	0.39 feasible: True	1.3e-15 zero rows: 0
5	banking77 confirm_npo	F	0.59	0.42 [0.37, 0.47] classes: [0.22, 0.64]	0.41 [0.36, 0.46]	0.16 [0.12, 0.19]	0.39 feasible: True	8.9e-16 zero rows: 0
6	banking77 confirm_npo	G	0.59	0.36 [0.28, 0.44] classes: [0.17, 0.57]	0.36 [0.27, 0.44]	0.12 [0.05, 0.19]	0.39 feasible: True	1.3e-15 zero rows: 0
7	banking77 confirm_grad_ascent	FG	0.54	0.68 [0.63, 0.72] classes: [0.48, 0.86]	0.62 [0.58, 0.66]	0.32 [0.28, 0.35]	0.55 feasible: True	1.1e-15 zero rows: 0
8	banking77 confirm_grad_ascent	F	0.54	0.70 [0.65, 0.75] classes: [0.49, 0.86]	0.64 [0.60, 0.69]	0.32 [0.29, 0.36]	0.55 feasible: True	1.1e-15 zero rows: 0
9	banking77 confirm_grad_ascent	G	0.54	0.61 [0.52, 0.70] classes: [0.45, 0.88]	0.56 [0.47, 0.64]	0.29 [0.22, 0.35]	0.55 feasible: True	8.9e-16 zero rows: 0
ID	Dataset / run	View	Dev. $s$	Held-out $s$ [rows]; class CI	Aligned $a$ [rows]	Relative residual [rows]	$P_F$ ; gate	Identity error; zero directions
10	banking77 matched_grad_diff	FG	0.65	1.11 [1.04, 1.19] classes: [0.60, 1.60]	0.74 [0.70, 0.78]	0.50 [0.47, 0.54]	0.19 feasible: False	2.0e-15 zero rows: 0
11	banking77 matched_grad_diff	F	0.65	1.10 [1.02, 1.18] classes: [0.61, 1.55]	0.74 [0.70, 0.78]	0.50 [0.46, 0.54]	0.19 feasible: False	2.0e-15 zero rows: 0
12	banking77 matched_grad_diff	G	0.65	1.16 [1.01, 1.33] classes: [0.59, 1.81]	0.75 [0.66, 0.83]	0.51 [0.43, 0.59]	0.19 feasible: False	1.3e-15 zero rows: 0
13	banking77 matched_grad_diff_kl	FG	0.63	0.53 [0.49, 0.57] classes: [0.36, 0.72]	0.49 [0.45, 0.53]	0.20 [0.17, 0.23]	0.48 feasible: True	8.9e-16 zero rows: 0
14	banking77 matched_grad_diff_kl	F	0.63	0.55 [0.50, 0.60] classes: [0.37, 0.73]	0.51 [0.46, 0.56]	0.21 [0.18, 0.24]	0.48 feasible: True	8.9e-16 zero rows: 0
15	banking77 matched_grad_diff_kl	G	0.63	0.46 [0.38, 0.54] classes: [0.32, 0.71]	0.43 [0.35, 0.51]	0.16 [0.10, 0.22]	0.48 feasible: True	8.9e-16 zero rows: 0
16	banking77 matched_rmu	FG	0.58	0.72 [0.67, 0.76] classes: [0.43, 0.95]	0.40 [0.38, 0.43]	0.36 [0.34, 0.38]	0.04 feasible: True	1.1e-15 zero rows: 0
17	banking77 matched_rmu	F	0.58	0.72 [0.66, 0.77] classes: [0.41, 0.95]	0.40 [0.37, 0.43]	0.36 [0.34, 0.37]	0.04 feasible: True	1.1e-15 zero rows: 0
18	banking77 matched_rmu	G	0.58	0.71 [0.62, 0.82] classes: [0.51, 0.95]	0.40 [0.35, 0.46]	0.37 [0.33, 0.41]	0.04 feasible: True	8.9e-16 zero rows: 0
19	clinc150 matched_s1__simnpo_b2.5_ep7	FG	1.01	1.93 [1.80, 2.06] classes: [1.32, 3.30]	0.80 [0.77, 0.83]	0.67 [0.63, 0.71]	-0.39 feasible: True	1.6e-15 zero rows: 0
20	clinc150 matched_s1__simnpo_b2.5_ep7	F	1.01	1.94 [1.80, 2.10] classes: [1.38, 3.13]	0.81 [0.77, 0.85]	0.67 [0.63, 0.72]	-0.39 feasible: True	1.3e-15 zero rows: 0
21	clinc150 matched_s1__simnpo_b2.5_ep7	G	1.01	1.87 [1.63, 2.18] classes: [1.21, 3.74]	0.78 [0.71, 0.84]	0.64 [0.57, 0.73]	-0.39 feasible: True	1.6e-15 zero rows: 0
22	clinc150 matched_s1__simnpo_b4.5_ep9	FG	1.00	1.61 [1.51, 1.73] classes: [0.99, 2.89]	0.73 [0.69, 0.77]	0.54 [0.51, 0.58]	-0.17 feasible: True	1.6e-15 zero rows: 0
23	clinc150 matched_s1__simnpo_b4.5_ep9	F	1.00	1.63 [1.50, 1.77] classes: [1.03, 2.78]	0.74 [0.69, 0.79]	0.55 [0.51, 0.60]	-0.17 feasible: True	1.6e-15 zero rows: 0
24	clinc150 matched_s1__simnpo_b4.5_ep9	G	1.00	1.55 [1.34, 1.81] classes: [0.91, 3.16]	0.71 [0.62, 0.81]	0.51 [0.44, 0.59]	-0.17 feasible: True	1.1e-15 zero rows: 0

Table S8. Matched suppression (development vs held-out) and oracle-direction geometry.

ID	Dataset / run	View	Dev.	Held-out $s$ [rows]; $s$ class CI	Aligned $a$ [rows]	Relative residual [rows]	$P_F$ ; gate	Identity error; zero directions
25	clinc150 matched_s1__simnpo_b6.5_ep10	FG	0.94	1.18 [1.11, 1.27] classes: [0.74, 2.00]	0.61 [0.57, 0.64]	0.37 [0.34, 0.40]	0.17 feasible: True	8.9e-16 zero rows: 0
26	clinc150 matched_s1__simnpo_b6.5_ep10	F	0.94	1.18 [1.09, 1.28] classes: [0.76, 1.91]	0.60 [0.55, 0.64]	0.38 [0.35, 0.41]	0.17 feasible: True	8.9e-16 zero rows: 0
27	clinc150 matched_s1__simnpo_b6.5_ep10	G	0.94	1.19 [1.04, 1.37] classes: [0.74, 2.24]	0.64 [0.55, 0.72]	0.36 [0.29, 0.43]	0.17 feasible: True	6.7e-16 zero rows: 0
28	clinc150 tuned__simnpo	FG	—	0.81 [0.75, 0.88] classes: [0.46, 1.46]	0.42 [0.39, 0.46]	0.26 [0.24, 0.29]	0.22 feasible: True	1.1e-15 zero rows: 0
29	clinc150 tuned__simnpo	F	—	0.80 [0.73, 0.87] classes: [0.46, 1.39]	0.41 [0.37, 0.44]	0.26 [0.23, 0.29]	0.22 feasible: True	1.1e-15 zero rows: 0
30	clinc150 tuned__simnpo	G	—	0.86 [0.73, 1.00] classes: [0.51, 1.66]	0.47 [0.39, 0.55]	0.27 [0.21, 0.33]	0.22 feasible: True	8.9e-16 zero rows: 0

Table S8 (PDF p. 58). Matched suppression and oracle-direction geometry (continued).

The largest identity error in this table is $2 \times10^{-15}$ . This precision statement applies to these matched-suppression rows; the broader all-run diagnostic figure has a different displayed numerical range.

ID	Model / configuration	$p(0)$ : U / O	$G_{\mathrm{abs}}$ [rows]	$G_{\mathrm{adj}}$ [rows]; class CI	$G_{\mathrm{H}}$ [rows]
1	qwen3.5-0.8b k1_s0	0.364 / 0.313	+0.139 [+0.092, +0.187]	+0.087 [+0.043, +0.131] classes: [+0.039, +0.139]	+0.178 [+0.119, +0.237]
2	qwen3.5-0.8b k1_s1	0.364 / 0.313	+0.073 [+0.039, +0.107]	+0.022 [-0.029, +0.073] classes: [-0.049, +0.088]	+0.081 [+0.031, +0.130]
3	qwen3.5-0.8b k5_s0	0.364 / 0.313	+0.080 [+0.059, +0.104]	+0.029 [-0.016, +0.073] classes: [-0.087, +0.120]	+0.093 [+0.062, +0.127]
4	qwen3.5-0.8b k5_s1	0.364 / 0.313	+0.100 [+0.077, +0.126]	+0.049 [+0.001, +0.099] classes: [-0.074, +0.136]	+0.128 [+0.095, +0.164]
5	qwen3.5-0.8b k20_s0	0.364 / 0.313	+0.079 [+0.057, +0.103]	+0.028 [-0.017, +0.075] classes: [-0.090, +0.122]	+0.097 [+0.064, +0.132]
6	qwen3.5-0.8b k20_s1	0.364 / 0.313	+0.095 [+0.073, +0.116]	+0.043 [+0.002, +0.087] classes: [-0.045, +0.129]	+0.117 [+0.086, +0.148]
7	apertus-8b k1_s0	0.326 / 0.429	+0.056 [+0.024, +0.090]	+0.159 [+0.114, +0.205] classes: [+0.093, +0.225]	+0.139 [+0.089, +0.195]
8	apertus-8b k1_s1	0.326 / 0.429	+0.057 [+0.034, +0.080]	+0.160 [+0.114, +0.207] classes: [+0.028, +0.272]	+0.140 [+0.100, +0.184]
9	apertus-8b k5_s0	0.326 / 0.429	+0.033 [+0.009, +0.058]	+0.136 [+0.094, +0.178] classes: [+0.052, +0.219]	+0.100 [+0.064, +0.139]
10	apertus-8b k5_s1	0.326 / 0.429	+0.066 [+0.044, +0.089]	+0.169 [+0.127, +0.213] classes: [+0.061, +0.279]	+0.154 [+0.118, +0.194]
11	apertus-8b k20_s0	0.326 / 0.429	+0.075 [+0.056, +0.096]	+0.178 [+0.140, +0.217] classes: [+0.078, +0.278]	+0.167 [+0.142, +0.196]
12	apertus-8b k20_s1	0.326 / 0.429	+0.065 [+0.043, +0.087]	+0.168 [+0.127, +0.210] classes: [+0.087, +0.249]	+0.150 [+0.121, +0.184]
13	qwen3.5-9b k1_s0	0.614 / 0.477	+0.116 [+0.070, +0.162]	-0.021 [-0.057, +0.015] classes: [-0.104, +0.068]	+0.135 [+0.061, +0.210]
14	qwen3.5-9b k1_s1	0.614 / 0.477	+0.120 [+0.080, +0.166]	-0.016 [-0.068, +0.032] classes: [-0.180, +0.167]	+0.151 [+0.074, +0.229]
15	qwen3.5-9b k5_s0	0.614 / 0.477	+0.066 [+0.044, +0.092]	-0.070 [-0.115, -0.029] classes: [-0.202, +0.063]	+0.043 [-0.002, +0.090]
16	qwen3.5-9b k5_s1	0.614 / 0.477	+0.072 [+0.044, +0.102]	-0.065 [-0.112, -0.021] classes: [-0.194, +0.100]	+0.048 [-0.013, +0.110]
17	qwen3.5-9b k20_s0	0.614 / 0.477	+0.072 [+0.050, +0.093]	-0.065 [-0.108, -0.025] classes: [-0.165, +0.053]	+0.068 [+0.025, +0.108]
18	qwen3.5-9b k20_s1	0.614 / 0.477	+0.106 [+0.083, +0.129]	-0.031 [-0.070, +0.008] classes: [-0.118, +0.097]	+0.132 [+0.089, +0.174]

Table S9. Recovery contrasts for every exposure budget and adaptation seed. Each interval is conditional on the two adaptation seeds and the reused trained backbones. Four-class intervals are coarse. The ratio-of-means headroom definition is used here.

ID	$A_{\mathrm{abs}}$ : U / O	$A_{\mathrm{adj}}$ : U / O	First $t$ : $p \ge0.5$ , U / O	First $t$ : $p \ge0.8$ , U / O	Identity residual
1	0.801 / 0.662	0.437 / 0.349	1 / 5	5 / not reached by $t = 25$ (censored)	6e-17
2	0.784 / 0.711	0.420 / 0.398	5 / 5	10 / 25	6e-17
3	0.798 / 0.717	0.433 / 0.404	5 / 10	10 / 10	6e-17
4	0.848 / 0.748	0.484 / 0.435	5 / 5	10 / 10	1e-16
5	0.847 / 0.768	0.483 / 0.455	5 / 5	10 / 10	1e-16
6	0.820 / 0.725	0.456 / 0.412	5 / 10	10 / 10	8e-17
7	0.845 / 0.789	0.519 / 0.360	1 / 1	5 / 10	6e-17
8	0.850 / 0.793	0.524 / 0.364	1 / 5	5 / 10	6e-17
9	0.839 / 0.806	0.513 / 0.377	5 / 5	5 / 10	6e-17
10	0.854 / 0.788	0.528 / 0.359	1 / 5	5 / 10	6e-17
11	0.867 / 0.793	0.541 / 0.363	5 / 5	5 / 10	6e-17
12	0.862 / 0.797	0.536 / 0.368	5 / 5	5 / 10	6e-17
13	0.873 / 0.758	0.260 / 0.281	0 / 1	5 / 10	0e+00
14	0.883 / 0.763	0.269 / 0.286	0 / 1	5 / 25	0e+00
15	0.877 / 0.810	0.263 / 0.333	0 / 1	5 / 10	0e+00
16	0.869 / 0.797	0.255 / 0.320	0 / 1	5 / 10	0e+00
17	0.898 / 0.826	0.284 / 0.349	0 / 1	5 / 10	0e+00
18	0.896 / 0.790	0.282 / 0.313	0 / 5	5 / 10	0e+00

Table S9 (PDF p. 60). Recovery summaries (continued): component areas, threshold crossings, and algebraic residual.

Unreached thresholds are censored at $t = 25$ ; no extrapolated crossing time is assigned.

Logged evaluation costs are incomplete. In particular, zero entries for full-model evaluations omit full training, and recovery is not fully accounted for. These numbers do not establish the total compute cost or a retraining speedup.

Item	Upstream audit record
Acquisition	Opaque full models: all seven checks pass (gate 0.70 / 0.50). Semantic seed-42 full model: forgotten class transfer_timing (BANKING77 bundle 02) at 0.65 development accuracy, below the 0.70 class threshold; targets were gated at the primary seed only.
Privacy audit	S2-Rank stopped at its preregistered acquisition gate at seed 17 (G1 0.550, G4 0.555); the inverse-rank attack was not tested.
Missing / completed coverage	Uniformization at matched suppression (no feasible development checkpoint). CLINC150 bundle 06 under opaque labels was added on 2026-10-18 after the figure brief named it as a contrast bundle (acquisition passed); coverage of the planned designs is complete.
Reserved	CLINC150 class 05 bundle 08 held back unseen; no file of it is read by the build (the collector stops if one appears).
Excluded model	checkpoint reload not reproducible (results/model_checks/internlm3-8b_reload_diagnosis.json; docs/HANDOVER.md); jobs blocked by design
Reconciliation	Row-level OD recomputed for 358 evaluations: max —recomputed - record.json— = 0.0e+00. Paired bootstrap: point estimates exact over 252 comparisons; intervals exact where n_boot matches (5000); stored 2000-draw summaries differ by at most 0.0075 (Monte Carlo); no category changes.
Filename collisions	Seed-42 and opaque outputs once overwrote seed-17 / semantic analysis files; repaired by the seed / opaque run tag and regeneration (PROTOCOL tooling notes). The build keys every result by dataset, model, label scheme, seed, forget set and run label and asserts uniqueness.
Reporting-code fix	The paper collector initially mapped ’grad_diff_kl’ labels to GradDiff by prefix; fixed (longest match) before any asset was finalised; no stored result was affected.
Recovery reruns	8-9B recovery runs initially ran out of memory at the second checkpoint (model not released); fixed and rerun per checkpoint. Row-level reruns for the Qwen3.5-0.8B and the Apertus-8B controls were added to provide uncertainty for the control contrasts.
Selection notes	CLINC150 tuning ran without the intervention gate (no same-data replicas); the selected point lies on the grid edge (beta 10, 10 epochs), extending it was held. The best matched-strength epoch (seed-42 diagnostic) is post hoc.
Records	CLINC150 bundles 04-07 results are recorded in the analysis files and this build; PROTOCOL.md pre-registered the analysis (2026-10-14) but has no results entry for them.

Table S12. Audit record: gates, missing or reserved results, reconciliation and repairs.

The opaque-check count is corrected to seven using Table S5. Other upstream integrity checks and protocol claims are reported rather than independently reproduced from this export-only archive. The unchanged source table and CSV preserve the original wording.

Dataset / bundle / seed	Labels; status	Full accuracy F / G	Oracle accuracy F / G	Oracle forgotten mass F / G	$D_{0,F}/D_{0,G}$	$P_F/P_G$	Opaque acquisition
banking77 b01, seed 17	semantic found	0.981 / 0.912	0.2687 / 0.3063	3.043e-01 / 3.262e-01	0.608 / 0.495	0.59 / 0.57	—
banking77 b01, seed 17	opaque found	0.953 / 0.906	0.0000 / 0.0000	1.126e-03 / 9.742e-04	0.943 / 0.881	0.61 / 0.62	pass; min class acc. 0.80 (semantic 0.90), min gold p 0.77
banking77 b01, seed 42	semantic found	0.974 / 0.906	0.3154 / 0.3625	3.419e-01 / 3.789e-01	0.558 / 0.470	0.55 / 0.51	—
banking77 b01, seed 42	opaque found	0.960 / 0.894	0.0000 / 0.0000	4.706e-04 / 5.307e-04	0.934 / 0.878	0.60 / 0.58	pass; min class acc. 0.84 (semantic 0.84), min gold p 0.83
banking77 b02, seed 17	semantic found	0.940 / 0.925	0.2199 / 0.2000	2.209e-01 / 2.108e-01	0.644 / 0.636	0.70 / 0.73	—
banking77 b02, seed 17	opaque found	0.966 / 0.931	0.0000 / 0.0000	6.479e-04 / 3.491e-04	0.912 / 0.872	0.45 / 0.43	pass; min class acc. 0.74 (semantic 0.70), min gold p 0.69
banking77 b02, seed 42	semantic found	0.958 / 0.956	0.2094 / 0.2062	2.160e-01 / 2.124e-01	0.676 / 0.660	0.47 / 0.45	—
banking77 b02, seed 42	opaque found	0.955 / 0.950	0.0000 / 0.0000	1.836e-03 / 1.780e-03	0.911 / 0.902	0.40 / 0.38	pass; min class acc. 0.71 (semantic 0.65), min gold p 0.70

Table S5. Semantic versus opaque decision labels: acquisition, oracle recognition, displacement and SimNPO progress.

References

[2]Casanueva, I., Temčinas, T., Gerz, D., Henderson, M., and Vulić, I. Efficient intent detection with dual sentence encoders. In Wen, T.-H., Celikyilmaz, A., Yu, Z., Papangelis, A., Eric, M., Kumar, A., Casanueva, I., and Shah, R. (eds.), Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI, pp. 38–45, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.nlp4convai-1.5. URL https://aclanthology.org/2020.nlp4convai-1.5/.DOI
[3]Dorna, V., Mekala, A. R., Zhao, W., McCallum, A., Kolter, J. Z., Lipton, Z. C., and Maini, P. Openunlearning: Accelerating LLM unlearning via unified benchmarking of methods and metrics. In The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2026. URL https://openreview.net/forum?id=Gy67Zh5X1i.arxiv.org/abs/2506.12618
[4]Fan, C., Liu, J., Lin, L., Jia, J., Zhang, R., Mei, S., and Liu, S. Simplicity prevails: Rethinking negative preference optimization for LLM unlearning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https://openreview.net/forum?id=JbvSQm5h1l.arxiv.org/abs/2410.07163
[5]Georgiev, K., Rinberg, R., Park, S. M., Garg, S., Ilyas, A., Madry, A., and Neel, S. Machine unlearning via simulated oracle matching. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=3vXpZpOn29.
[6]Ginart, A., Guan, M. Y., Valiant, G., and Zou, J. Making AI forget you: data deletion in machine learning. Curran Associates Inc., Red Hook, NY, USA, 2019.arxiv.org/abs/1907.05012
[8]Larson, S., Mahendran, A., Peper, J. J., Clarke, C., Lee, A., Hill, P., Kummerfeld, J. K., Leach, K., Laurenzano, M. A., Tang, L., and Mars, J. An evaluation dataset for intent classification and out-of-scope prediction. In Inui, K., Jiang, J., Ng, V., and Wan, X. (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 1311–1316, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v1/D19-1131. URL https://aclanthology.org/D19-1131/.DOI
[9]Maini, P., Feng, Z., Schwarzschild, A., Lipton, Z. C., and Kolter, J. Z. TOFU: A task of fictitious unlearning for LLMs. In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=B41hNBoWLo.arxiv.org/abs/2401.06121
[10]Shi, W., Lee, J., Huang, Y., Malladi, S., Zhao, J., Holtzman, A., Liu, D., Zettlemoyer, L., Smith, N. A., and Zhang, C. MUSE: Machine unlearning six-way evaluation for language models. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=TArmA033BU.arxiv.org/abs/2407.06460
[11]Zhang, R., Lin, L., Bai, Y., and Mei, S. Negative preference optimization: From catastrophic collapse to effective unlearning. In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=MXLBXjQkmb.arxiv.org/abs/2404.05868