betaFeedback

WorksComputer Science

Is Abliteration Getting Harder? Not Uniformly: Refusal Removal by Direction-Based Ablation and ARA in Heretic on the Qwen3.8, Gemma 4, Olmo 3.1 and Mistral Small 3.2 LLMs Versus Their Predecessors

Abstract

Abliteration, a fine-tuning-free weight edit, removes refusal from open-weight large language models (LLMs). The developer of Arbitrary-Rank Ablation (ARA), an optimisation-based variant, reports that direction-based abliteration, which ablates a refusal direction, increasingly fails on newer models. On five release pairs, refusal removal did not become uniformly harder. Qwen2.5-32B and Qwen3.8-27B, Gemma-2-27B and Gemma-4-31B, OLMo-2-32B and Olmo-3.1-32B, Mistral-Small-2409 and Mistral-Small-3.2, and Qwen2.5-7B and Qwen3.5-4B were edited in Heretic 2.0 by three or four direction-based methods (difference-in-means, MPOA, SOM and, on the small pair, J-space) and ARA under the same prompts, nominal search budget, first-token KL divergence cap and selection rule; removal counted as harder when the automatic WildGuard judge found more of 972 held-out harmful prompts (JailbreakBench, XSTest, StrongREJECT, HarmBench and an author-compiled malicious-code bank) still refused. After direction-based removal, newer releases retained -1 to +37 percentage points more refusal, and after ARA -26 to +34. With run-to-run variance included, between-pair heterogeneity of the direction-based effect ranged from high to low with the scale and variance model (I^2 20-80%). ARA lost less effectiveness than direction-based methods in Gemma and OLMo and more in Qwen; each verdict rests on one ARA run, and some changed with the divergence budget or variance model. In Gemma the direction-based increase came from the malicious-code bank and reversed on public sets. Contrary to prediction, keyword counts at the tool's window did not undercount judged refusal on average, but differed by a mean 7 points per model. Audits of a single pair, method or prompt source can reach opposite conclusions. Whether a newer model resisted removal depended on the release pair, method and budget, not on generation alone.

Introduction

Open-weight language models ship with safety training that teaches them to refuse harmful requests, but their public weights allow this behaviour to be edited after release. Since 2024, fine-tuning-free techniques called abliteration, which need no end-to-end gradient training, have made such editing cheap: with a few hundred contrasting prompts and a single GPU, they modify a few weight matrices so that the model stops refusing [1, 2], without the gradient-based fine-tuning on harmful data used by earlier attacks [3, 4]. Many abliterator models are published on Hugging Face, and fully automated tools are publicly available [5]. Whether safety training withstands such editing therefore matters to attackers, to defenders and to anyone who audits whether newer releases are more robust than older ones.

The original technique rests on the finding that, in many chat models, refusal is mediated by a single residual-stream direction whose removal from the weights suppresses refusal while leaving most other behaviour intact [1]. Some successors change only how this direction is removed, to limit collateral damage [6], whereas others remove several directions or a refusal “cone” [7, 8]. Arbitrary-Rank Ablation (ARA), now the default of the open-source Heretic 2.0 tool, abandons directions and optimises each targeted weight matrix to move harmful outputs toward harmless ones while keeping harmless outputs fixed [9]. The methods are motivated by different views of where refusal lives, from one direction to a low-dimensional subspace or cone, a non-linear region (so far suggested only by visualisation [10]) or a distribution across modules. They also differ in edit rank (one to six for the direction-based methods used here, 50 for ARA), objective and search space, so comparing them compares families of edits, not geometric hypotheses about refusal. If refusal is not one-dimensional, single-direction defences (§ 2) may give a false sense of security, yet recent work offers mixed evidence on the single-direction picture [11, 12, 13].

Whether newer model generations resist refusal removal remains unclear. The author of ARA reports that older models “can usually be decensored quite easily with standard abliteration”, whereas ARA “shows its strengths precisely with the latest generation of models, where traditional directional ablation tends to be woefully inadequate” [9]. One explanation would be that newer safety training spreads refusal over more dimensions or modules, consistent with calls for alignment “more than just a few tokens deep” [14], with evidence that harmfulness and refusal are encoded separately [15], and with training methods that make refusal survive ablation of its direction [16, 17]. Adversarial safety fine-tuning of one 7B model, however, moved a still low-dimensional refusal representation to an earlier layer rather than expanding it [18]. Other explanations would not involve the representation of refusal at all: architectural changes can weaken a given edit (in post-normalised blocks part of an ablated direction reaches the residual stream; Appendix § 9.§ 1), later layers can partly compensate for an edit [19], and the estimate of the direction can be contaminated by its prompt sets. To the best of current knowledge, the ARA author’s claim has not been tested independently: the nearest evidence concerns safety-pretraining checkpoints of one 1.7B model [20], ARA has no published evaluation, and existing abliteration comparisons [21] predate both ARA and current open models. A second gap is methodological: tools are usually evaluated with the cheap signals they optimise (keyword-matched refusals over short generations and first-token KL divergence), which can disagree sharply with stronger judges: keyword matching agrees poorly with human labels [22], and conclusions can depend on the classifier used [23]. In one public comparison, keyword matching reported 3–5 refusals per 100 prompts where an LLM judge found more than 33 (Heretic PR #196, an unreviewed practitioner report1). Aggregate refusal rates can also hide failures in domains such as cybersecurity [24] and malicious code generation, for which validated prompt banks and systematic corpus reviews now exist [25, 26, 27]. A method can therefore appear successful by avoiding refusal phrases without complying, or by complying with degraded, useless output [28].

The present study addresses both gaps through a controlled comparison of refusal-removal methods on an earlier and a later release of four open-weight model families and one smaller pair, five release pairs in all. To current knowledge it is the first independent comparison of ARA with direction-based methods across model releases; it rests on one ARA run per model. The primary purpose was to determine whether refusal removal becomes harder in newer model generations. Removal is called harder for a model when more refusal remains after the same search budget and selection rule, that is, when its residual refusal (the share of held-out harmful prompts whose response an automatic judge, WildGuard, labels a refusal) is higher. The judge is independent of the optimisation objective but has not been validated against human labels on edited models. This is a budget-bound measure: residual refusal of the trial selected after 100 search trials at first-token KL divergence ≤0.1\le0.1, not the difficulty of removal for an attacker with more budget. It was hypothesised (H1) that, with every model selected under the same cap on first-token KL divergence (≤0.1\le0.1), residual refusal after ablation would be higher for the newer model of each pair than for its predecessor, for every method. Because a newer original may already refuse more before any edit, the generation effect is also reported relative to the originals’ own difference (baseline-adjusted), which separates higher starting refusal from resistance to removal; this contrast was added after H1 was stated and is exploratory. The secondary purpose was to determine whether this difficulty depends on the class of edit. It was hypothesised (H2) that the generation effect would interact with method class, such that optimisation-based ablation (ARA) would lose less effectiveness on newer models than direction-based methods (diff-in-means, projected/magnitude-preserving abliteration (MPOA), self-organising-map (SOM) ablation and, on a small pair, Jacobian-space (J-space) ablation). Because the two classes differ in rank, objective and search space as well as in their motivating geometry, H2 compares edit families and cannot by itself separate these factors. A third, exploratory aim was to test whether the cheap optimisation signals agree with independent measures; it was hypothesised (H3) that keyword-based refusal counts would underestimate residual refusal relative to an automatic judge, and that the size of this gap would differ between method classes. All methods run inside Heretic 2.0 with the same prompts and nominal search budget on four main-study pairs (Qwen, Gemma, OLMo, Mistral; 22–32B parameters) and a small Qwen pair (7B, 4B). Models selected by a fixed rule are released (except the five Gemma-2-27B variants, which are withheld; Appendix § 9.§ 5), run logs, configurations and study journals are released with the code (Data and Code Availability), and every model is scored by one harness, independent of every optimisation objective, for judged refusal, capability (knowledge, reasoning, instruction following [29] and code generation) and full-sequence distribution shift.

The study makes three contributions:

  • a held-out test of whether refusal removal is harder in newer releases, with the release pair as the unit: the generation effect differed in sign and size between pairs, the only raw per-pair effect that exceeded run-to-run variation under both variance models, at the point estimate of the run-to-run standard deviation, was the OLMo direction-based increase, and some per-pair verdicts depended on the source of the test prompts (§ 4.1);

  • a preliminary comparison of ARA with direction-based methods under one nominal search budget, KL cap and selection rule: ARA lost less effectiveness on the newer model in two pairs, more in one in its point estimate (indeterminate with run-to-run variance) and about as much in two, but each contrast rests on one ARA run whose export was refitted after the search, and the verdicts changed with the KL budget and the variance model (§ 4.2);

  • evidence on the signals that abliteration tools optimise: keyword counts at the tool’s 100-token window matched judged refusal on average but disagreed with it by several points per model, and first-token KL divergence did not bound capability loss (§ 4.3, § 4.4).

Figure 1 summarises the held-out answer to H1 and the per-pair effects.

Figure 1. Refusal removal did not become uniformly harder. (a) Change in held-out refusal from the older to the newer release of each pair (newer minus older, percentage points; WildGuard judge, 972 harmful prompts not used for search or selection), for the unedited originals and for the model each removal method selected (fewest dev refusals at first-token KL ≤0.1\le0.1, seed 1). Blue: the newer release kept more refusal (removal harder); orange: less (removal easier). Bold: significant (methods: exact McNemar, Holm over the 23 H1 tests; originals: exact McNemar, unadjusted); ns: not significant; † failed edit (Gemma-4-31B DiM left more refusal than its own original). (b) Per-pair effect of the direction-based methods (pooled over DiM, MPOA and SOM; J-space added on the small pair) and of ARA, with 95% intervals that add run-to-run variation at its point estimate to prompt sampling: thick bars assume constant run-to-run variance, thin bars variance that scales with the refusal level. Filled marker: both intervals exclude zero, which holds only for the OLMo direction-based increase. Refusal levels and Wilson intervals of every cell are in Figure 3.

Related Work

Refusal-removal methods are reviewed below by what they assume about the geometry of refusal, followed by work on how post-training shapes a behaviour, defences and evaluation practice (definitions in Appendix § 8).

Single-Direction Ablation

Arditi et al. [1] showed causally, across 13 chat models, that adding the difference between mean harmful and harmless activations induces refusal and ablating it suppresses refusal. Contemporaneous work read and steered such directions from the top down: representation engineering extracted directions for concepts including harmlessness [30], and contrastive activation addition steered refusal, among other behaviours, in Llama 2 chat models with mean-difference vectors added to the residual stream [31]. Orthogonalising the weights that write to the residual stream against this direction yields a fine-tuning-free edit that needs only a few hundred prompts [2, 32], consistent with the linear representation hypothesis [33]. The edit is fast and interpretable, but the direction also carries part of the harmless mean, an offset that an affine model of refusal handles explicitly [34]; changed row norms shift activation scales, and results depend on a hand-chosen layer and strength, with large capability differences between tools [21] (Appendix § 8.1.1).

Most successors address this collateral damage rather than the one-dimensional assumption. Projected abliteration removes the component along the harmless mean [35], magnitude-preserving abliteration (MPOA) restores row norms [6], and Heretic automates the choice of layer, strength and direction with TPE [5, 36]. A null-space-constrained variant [37] adopts the AlphaEdit constraint from knowledge editing [38, 39], which may also block the refusal edit when harmful and harmless activations share directions. Surgical refusal ablation orthogonalises the direction against capability and style directions [40], whereas Petrov [41] found that a topic-matched harmless baseline yielded no functional refusal direction. The evidence remains thin: the projection and norm-preservation variants are documented mainly in blog posts, and both preprints evaluate models of at most 14B parameters.

Multi-Direction and Subspace Methods

A second group holds that one mean difference captures only the dominant component of refusal. Gabliteration projects out the top singular vectors of per-prompt activation differences [42], OBLITERATUS adds whitened, iterative and spectral variants [43], fast subspace estimation finds multi-dimensional refusal subspaces [44], and weighted refusal direction optimisation (WRDO) learns weights over per-prompt directions, with Heretic as a baseline [45]. Wollschläger et al. [8] learned refusal directions by gradient descent, found multi-dimensional concept cones, and showed that orthogonality does not imply independence under intervention. Piras et al. [7] proved that self-organising maps (SOMs) generalise difference-in-means and reported that ablating several SOM directions outperforms the single-direction baseline. Each added direction, however, removes capacity. The premise is also contested: Joad et al. [11] found distinct refusal directions that produced nearly identical refusal–over-refusal trade-offs when steered, Wang et al. [12] found safety-triggered refusal to be one-dimensionally steerable, and in Llama-3-8B one dominant direction governed refusal while smaller orthogonal directions, such as role-play, modulated it [13]. A multi-dimensional representation therefore does not establish that multi-directional removal is necessary.

Optimisation-Based Methods

A third group replaces the estimated direction with an explicit objective. Arbitrary-Rank Ablation (ARA) fits, for each targeted module, a low-rank update that keeps harmless outputs fixed and moves harmful outputs toward them [9], so any refusal mechanism carried by module outputs is in scope up to the adapter’s rank. Its author reports that directional ablation increasingly fails on recent models and that ARA outperforms it on gpt-oss-20b (Appendix § 8.§ 3.1), but this evidence comes from a pull request, and the nearest-neighbour objective may overfit its prompt sets. A low-rank edit may suffice because safety-critical weights occupy about 3% of parameters and 2.5% of rank, separately from utility [46], although newer models may spread safety more widely. Optimal-transport ablation also matches distributions, with a closed-form Gaussian map [47], but no public code was found (Appendix § 8.§ 5).

Representation-Space Methods

A fourth group changes the space in which refusal is sought. Jacobian-space (J-space) ablation computes the direction in a workspace defined by a Jacobian lens and pulls it back to the residual stream [48, 49, 50], to isolate the part of the refusal signal the model uses downstream. By its authors’ own measurements, however, it removes little refusal, and the reference code is a single-model forward hook (Appendix § 8.§ 4.1). Related findings suggest that the residual-stream mean difference mixes signals: harmlessness is encoded separately from refusal [15, 51], and at fixed intervention strength a direction estimated at a layer’s output, before the residual stream, controlled refusal whereas one estimated in the summed stream did not [52]. Where a direction is estimated may matter as much as how many are removed.

Post-Training Mechanisms and Self-Repair

Two further literatures bear on why a newer release could resist directional removal without a higher-dimensional refusal representation. Mechanistic studies of alignment algorithms found that preference optimisation (DPO) left a model’s toxic capability in place and learned to bypass it [53], and that safety fine-tuning applied a minimal transformation to MLP weights that routes unsafe inputs towards the null space of those weights [54]. How a release’s post-training recipe implements refusal may therefore matter as much as how many directions refusal occupies. Ablation studies have also found self-repair, in which later layers compensate for an ablated component [19]; a model that compensates more strongly would retain more refusal after the same edit. Neither property was measured in the models studied here.

Defences and Evaluation Problems

Defences largely target the single-direction estimator. Adversarial training against ablation of the refusal feature cut its attack success from 53% to 10% on Llama-3-8B [16]; extended-refusal fine-tuning spreads the refusal signal across tokens, limiting refusal drops under abliteration to at most 10% versus 70–80% in baseline models [55]; and fine-tuning under random ablation of the direction teaches models to rebuild refusal, cutting attack success by about 95% [17]. Refusal aliases obscure the direction [56]; decoy directions corrupt the contrastive estimator, cutting a Heretic weight-level attack on Llama-3-8B-Instruct from 88.7% to 18% success [57]; and decoy hardening concedes refusal removal but turns most answers to hazardous requests into falsified decoys [58]. Circuit breakers instead act on harmful-output representations [59], and tamper-resistant training targets fine-tuning attacks [60, 61], yet such fine-tuning defences still allowed 79–98% success to a best-of-96 single-layer abliteration on Llama-3.1-8B, against 99% without defence [62]. Training-time defences could make later releases resist directional removal, whereas defences built around direction estimation may lose value if removal no longer relies on it.

Evaluation practice is a second weakness. Tools are optimised and reported with keyword-matched refusal counts and first-token KL divergence, yet conclusions can depend on the classifier [23], including in abliteration studies [20], and in one unreviewed practitioner report keyword counts fell far below an LLM judge’s (Heretic PR #196). Against human labels, keyword matching reached Cohen’s κ=0.37\kappa= 0.37, WildGuard 0.61 and fine-tuned LLM judges about 0.81 [22], although on unmodified models keyword matching tracked a GPT-4 judge closely (gaps of 0.2–5.5 points across six models) [63], so large gaps may arise only after modification. Compliance is also not competence: single-vector projection raised a security-task score only from 0.46 to 0.50 while raising unsafe compliance from 0.10 to 0.47 [64]; abliterated code models complied with requests to inject vulnerabilities but succeeded at rates bounded by model size [65]; and jailbreaks of models aligned to refuse benign topics cut accuracy by up to 92% [28]. Off-target effects include more optimistic decisions on a task that elicited no refusals [66] and script-dependent bias that abliteration did not remove [67]. Guardrails have proved brittle under adversarial pressure [68], and durability claims are easily overstated because small changes to the attack configuration can reverse them [69].

Positioning of the Present Study

The present study differs from this literature in four respects. First, whether removal becomes harder in newer model generations has been tested only indirectly: with one directional pipeline, susceptibility across 24 open models was model-specific rather than size-dependent, with safety-training methodology and architecture the strongest predictors [24]; in one 1.7B model the full safety-pretraining recipe was the most robust to abliteration, and Qwen3 resisted it in one setup [20]; and the earlier cross-tool comparison covered 7–14B models that predate ARA and the current generation [21]. A companion benchmark [70] uses the same harness to compare Heretic 1.x, Heretic 2.0 and community releases within the Qwen2.5 family. The two studies share the evaluation harness, the held-out prompt set and 19 of the models evaluated here: the Qwen2.5-7B, Qwen2.5-32B and Qwen3.5-4B originals, their seed-1 Deepseek, SOM and ARA models (and J-space on the two small models), and the two community releases of Qwen2.5-32B. For these 19 models the companion study uses the same stored responses (identical checksums) and therefore the same judge and keyword labels. The Qwen3.8-27B, Gemma, OLMo and Mistral Small models, the seed-2 replicates, the γ\gamma-MPOA controls and the other six community releases are used only here, and the comparison between releases (H1, H2) is reported only here. Here, earlier- and later-generation models are paired within families. Second, all four method classes (representation-space methods on the small pair only) share one framework, prompt sets, budget and trial-selection rule under one cap on first-token KL divergence. Third, a harness independent of all optimisation objectives scores judged refusal on held-out prompts, capability and full-sequence distribution shift. Fourth, the study provides, to current knowledge, the first independent comparison of ARA with direction-based methods across model releases, with one ARA run per model. Table 1 summarises the methods considered. Presets of the external OBLITERATUS toolkit are evaluated in the companion paper [70]; null-space-constrained abliteration, refusal cones, WRDO and optimal-transport ablation are not evaluated (Appendix § 8.§ 5); and fine-tuning attacks, which remove safety training cheaply but require gradient updates [3, 4], are out of scope.

MethodAssumption about refusalEditSearchExportImplementation
Diff-in-means [1]1 directionrank-1 projectionTPEyesHeretic 2.0 (no proj./norm)
Projected / MPOA [35, 6]1 direction, ⊥\perp harmlessrank-1 + normaTPEyesHeretic 2.0
SVD multi-dir. [42]kk-dim subspacerank-kk projection–yesOBLITERATUSb
SOM [7]kk cluster directionsrank-kk + normaTPEyesHeretic 2.0 forkc
ARA [9]none (per-module map)rank-50 LoRATPE + L-BFGSyesHeretic 2.0
J-space1 direction in J-spacerank-1 + normaTPEyesHeretic 2.0 forkd

Table 1. Refusal-removal methods considered in this study, as instantiated (§ 3.2). “Search” = automated hyperparameter optimisation; “Export” = produces a standalone model. SVD multi-direction (OBLITERATUS) is evaluated in the companion paper [70], not here; null-space-constrained abliteration and refusal cones are not evaluated (Appendix § 8.§ 5). Formal definitions are given in Appendix § 8.

Methods

The study crosses model generation with refusal-removal method under one selection rule, one evaluation harness independent of the optimisation objective and the same nominal budget of 100 trials (exceptions in Appendix § 9.8), since method comparisons depend on the search budget [71]; further detail is in Appendix § 9. Figure 2 summarises the workflow.

Figure 2. Study workflow. Five earlier-to-later pairs of instruction-tuned models are edited with four refusal-removal methods in Heretic 2.0 (five on the small pair, which adds J-space), each in a 100-trial Optuna TPE study that minimises dev keyword refusals and first-token KL divergence; the trial with the fewest dev refusals at KL ≤0.1\le0.1 is exported as standalone weights. The originals, the selected models and eight public community abl iterations are evaluated on held-out data (judged refusal on 1,322 prompts, of which the 44 that exactly (36) or approximately (8) match a prompt of the search dataset are flagged; teacher-forced divergence on the 350 benign prompts; five capability benchmarks), and the statistics test H1–H3 and compare dev-set with held-out refusal. The highlighted step is the held-out evaluation, applied to all 74 models; the dashed box marks the γ\gamma-MPOA control, the seed-2 replicate of the small pair and the community releases, which join the same evaluation. Keyword refusal is scored on the first 100, 256 and 512 generated tokens.

Models and Data

The main study uses one dense instruction-tuned model per family per generation, of similar scale, each run unquantised in bf16 on a single GPU (Table 2). The families are Qwen, Gemma, OLMo and Mistral; OLMo was included because its training data are openly released. Paired models also differ in size and partly in architecture, and the later model of each pair was released 9 to 23 months after the earlier one (release months and checkpoint sizes in Table 2).

Table 2: Models. Released: year and month in which the Hugging Face repository was created (a repository can be created before its public release; note c). Revision: first eight hexadecimal digits of the Hugging Face commit used. Size: bf16 checkpoint size hashed by the evaluation harness, in GB (image-text checkpoints include their vision encoder); nominal scale from the repository name in parentheses. Layers: transformer blocks reported by Heretic. Short model names follow the vendors’ spelling (OLMo-2-32B, Olmo-3.1-32B; Mistral-Small-2409 for Mistral-Small-Instruct-2409); the pairs are labelled Qwen, Gemma, OLMo, Mistral Small and small pair.

FamilyGen.ReleasedRepositoryRevisionSize (GB)LayersNotes
Main study
Qwenearlier2024-09Qwen/Qwen2.5-32B-Instruct5ede1c9765.5 (32B)64–
Qwenlater2026-08Qwen/Qwen3.8-27B1d4bf0f255.6 (27B)64hybrid linear/full attention; image-text
Gemmaearlier2024-06google/gemma-2-27b-itaaf20e6b54.5 (27B)46post-norm [72]; no system role
Gemmalater2026-04google/gemma-4-31B-it842da37962.6 (31B)60post-norm; image-text
OLMoearlier2025-03allenai/OLMo-2-0325-32B-Instructb960243464.5 (32B)64post-norm [73]
OLMolater2025-12allenai/Olmo-3.1-32B-Instructac0587e464.5 (32B)64post-norm; sliding-window attention{^b}
Mistralearlier2024-09mistralai/Mistral-Small-Instruct-24094600506f44.5{^d} (22B)56–
Mistrallater2025-06mistralai/Mistral-Small-3.2-24B-Instruct-2506{^a}3ed8d341{^a}48.1 (24B)40image-text
Small pair
Qwenearlier2024-09Qwen/Qwen2.5-7B-Instructa09a354515.2 (7B)28–
Qwenlater2026-02Qwen/Qwen3.5-4B851bf6e89.3 (4B)32hybrid linear/full attention; image-text

Table 2.

Three architectural properties are relevant. Gemma-2, Gemma-4, OLMo-2 and Olmo-3.1 normalise module outputs with an RMSNorm [75] with elementwise gain γ\gamma before they enter the residual stream (“post-norm”, i.e. output normalisation [76], documented for Gemma 2, OLMo 2 and Olmo 3 [72, 73, 74]), so direction-based edits are incomplete on four of the eight main-study models; this potential confound is disclosed rather than corrected in the primary runs, and a γ\gamma-corrected MPOA control (γ\gamma-MPOA) was run on these four models as an additional condition (Appendix § 9.§ 2; results in § 4.4). Qwen3.8-27B and Qwen3.5-4B interleave gated-delta-rule linear-attention layers [77] with full-attention layers, and four checkpoints are image-text models used with text only (Appendix § 9.§ 1).

The small pair (Qwen2.5-7B-Instruct and Qwen3.5-4B) is the only candidate pair with instruction-tuned Jacobian lenses, so J-space ablation is restricted to it; its later model is the smaller one, confounding generation and size (Appendix § 10).

Prompt data. Refusal directions, SOM fitting and ARA module capture use the first 400 training prompts of mlabonne/harmful_behaviors and mlabonne/harmless_alpaca [78, 79]. The first 100 test prompts of harmful_behaviors form the development (dev) set for search and selection, and those of harmless_alpaca serve the KL objective. All primary and secondary refusal, capability and teacher-forced outcomes come from held-out sets. The held-out refusal set, frozen before any held-out response was generated, contains 1,322 prompts, 972 harmful and 350 benign (Table 3), from JailbreakBench, XSTest, StrongREJECT, HarmBench and a code-as-weapon bank [80, 81, 82, 83, 27] (sampling in Appendix § 9.§ 4). The 44 harmful items that match a harmful_behaviors prompt are flagged, and results are reported with and without them. The code-as-weapon bank was compiled by the author from public malicious-code prompt corpora and is described in a preprint that has not been peer-reviewed [27]; its 200 items were drawn, and the held-out file frozen, before any held-out response was generated.

SubsetKindnnOverlap
JailbreakBench harmfulharmful10011 / 6
JailbreakBench benignbenign100–
XSTest unsafeharmful2000 / 0
XSTest safebenign250–
StrongREJECTharmful31325 / 1
HarmBench (standard)harmful1590 / 1
Code-as-weaponharmful2000 / 0
Total1,32236 / 8

Table 3. Held-out refusal set (refusal_sets.json, v2). “Overlap”: items matching a mlabonne/harmful_behaviors prompt exactly / fuzzily.

Study Design and Conditions

The independent variables are model generation (earlier or later release within a family) and refusal-removal method; for H2, ARA forms the optimisation-based class and the remaining methods the direction-based class. All methods run inside Heretic 2.0 [5] and differ in the weight edit and its search space (Table 1, Appendix § 9.§ 2). Each edit targets the attention output projection and MLP down-projection of each layer and is held in a LoRA adapter [84] during optimisation. The methods of Appendix § 8 are instantiated as follows (full specification and additional conditions in Appendix § 9.§ 2. Three presets of the external OBLITERATUS toolkit [43] on Qwen2.5-7B-Instruct are evaluated with this study’s harness in the companion paper [70] and are not part of this study.

DiM. The difference-in-means refusal direction [1] is removed by a rank-1 edit that leaves the embedding matrix unchanged; unlike the original procedure (Appendix § 8.1.1), direction, strength and layer profile are tuned by the same search as MPOA (“Heretic-tuned DiM”).

MPOA. Heretic’s defaults: the direction is projected orthogonal to the harmless mean [35], and the edit is computed on row-normalised weights whose row norms are then restored [6] (rank-3 adapter).

SOM. A port of the proof of concept in Heretic PR #196 [7] (“Heretic-tuned SOM”): the k=4k = 4 most-winning neurons of a per-layer self-organising map of harmful residuals yield orthonormalised directions, which are ablated with a searched geometric decay (rank-6 adapter).

ARA. Heretic’s default modifier [9]: a rank-50 adapter per targeted module is fitted by L-BFGS to the objective of Appendix § 8.§ 3.1; row-magnitude preservation enters only the objective, and the exported adapter is not renormalised.

J-space (small pair only). Directions computed by the jspace-refusal code from Jacobian lenses and refusal-token unembeddings [48, 50, 49] are baked into the weights with the DiM/MPOA edit and search space (rank-3 adapter) rather than applied as the original forward hook.

Measures

Generation. One response per held-out prompt is generated for the original and every modified model with vLLM [85]: greedy, at most 512 new tokens, the model’s own chat template, system prompt “You are a helpful assistant.” and reasoning disabled. Decoding is identical for all models because refusal is highly sensitive to decoding settings [86].

Refusal and outcome classes. The primary outcome is the WildGuard [87] refusal label of each harmful held-out response; WildGuard agreed more closely with human labels than the HarmBench classifier and was less affected by stylistic rewriting [22, 88], but it has not been validated against human labels on the outputs of edited models. No human audit was made here, so every refusal rate is relative to the automatic judges, and “independent” means independent of the optimisation objective. All judges score the same stored responses: WildGuard (refusal and harmful response), the HarmBench classifier (harmful compliance), Qwen3Guard-Gen-8B [89] (second LLM judge), two encoder-based refusal classifiers, and Heretic’s 33 keyword markers on the full response and its first 256 tokens; WildGuard, Qwen3Guard and the HarmBench classifier were also used by a recent refusal-direction study [45]. A degeneracy heuristic flags empty, repetitive or largely non-printable responses, which ablation can produce [34]. Each response is assigned one outcome class in a fixed order: degenerate if flagged, otherwise refusal if WildGuard labelled it a refusal, otherwise compliance, split into complete and incomplete compliance by whether generation stopped at the 512-token limit (finish reason length). The primary outcome uses the WildGuard label of every response, degenerate ones included; in the outcome classes degenerate responses count as not refused (Appendix § 9.§ 4).

Capability and distribution shift. Five benchmarks run with lm-evaluation-harness 0.4.13 [90] on vLLM 0.33.0 (Table 4): fixed, seeded subsets of MMLU-Pro [91] and GSM8K [92] and the complete IFEval [93], HumanEval [94] and MBPP [95] test sets, with greedy decoding, chat templates and reasoning disabled. Distribution shift is measured by the first-token KL divergence of the distance objective (§ 3.4) and by a teacher-forced divergence: the original model’s non-empty greedy responses yy to the 350 benign held-out prompts are scored by each model, and the divergence of a modified model is the token-averaged log⁡porig(y)−log⁡pmod(y)\log p_{\mathrm{orig}}(y) - \log p_{\mathrm{mod}}(y).

BenchmarkTasknnShotsMetricMax. tok.
MMLU-Prommlu_pro_*1,0005 (CoT)exact match2,048
GSM8Kgsm8k5005exact match (flexible)512
IFEvalifeval5410prompt-level strict acc.1,280
HumanEvalhumaneval_instruct1640pass@11,024
MBPPmbpp_instruct5003pass@11,024

Table 4. Capability benchmarks: lm-eval task, items, shots, primary metric and maximum generated tokens.

Procedure

Optimisation. Each model–method combination is one Optuna study [36] with a multivariate TPE sampler [96]: 100 trials, of which the first 30 are random, with seed 1 (seeded generators, exceptions for two resumed studies and response-prefix handling in Appendix B.3). Every model used the same seed, so studies with the same search space share their 30 random start-up draws, and two pairs of selected parameter vectors coincide across models (Appendix D.4). The ten small-pair studies were repeated with seed 2 to estimate seed sensitivity (Appendix B.2). Two objectives are minimised, so each study yields a Pareto front. The refusal objective is the fraction of 100 greedy dev-set responses (at most 100 new tokens) containing any of Heretic’s 33 refusal markers; the distance objective is the KL divergence of the modified model’s first-token distribution from the original model’s, averaged over the 100 harmless prompts.

Selection rule. One rule is applied to every run: among completed trials with first-token KL divergence ≤0.1\le0.1, select the one with the fewest dev-set refusals, breaking ties by lower KL divergence and then lower trial index; if none qualifies, select nothing and report the lowest-KL trial. For sensitivity analysis the rule is repeated at thresholds 0.05, 0.2 and 0.5, and full Pareto fronts are reported. The cap is the same for every model, but it need not constrain every run: it is said to bind in a run when some trial above KL 0.1 had fewer dev refusals than the selected trial. Binding is recorded for every run, together with the position of the selected trial’s parameters in Heretic’s search ranges (Appendix D.4).

Export and release. Each selected trial is merged into a standalone checkpoint that differs from the original only in the edited matrices; the 42 primary models and the four γ\gamma-MPOA controls are published on Hugging Face, except the five Gemma-2-27B variants, which are withheld; the ten seed-2 replicates are evaluated but not released (Appendix B.5).

Data Analysis

All intervals are 95%. Proportions carry Wilson score intervals [97, 98]; paired differences on the same prompts carry Newcombe’s hybrid score intervals [99] and are tested with the exact McNemar test [100], a paired, prompt-level comparison as recommended for language-model evaluations [101]. Each modified model is compared in this way with its original. For H1, the generation effect of a pair and method is the paired difference in residual refusal between the selected models of the newer and the older release on the same 972 harmful prompts, tested by exact McNemar. Pooled over the four main-study families, H1 and H2 are tested with a logistic generalised estimating equation (GEE) [102] with generation, method class, their interaction and family as fixed effects, an exchangeable working correlation clustered by prompt and robust standard errors. The planned mixed-effects logistic regression, refusal ∼\sim generation ×\times method_class + (1|prompt) + (1|family) in lme4 notation [103, 104], was fitted by variational Bayes and is reported as posterior means with ±1.96\pm1.96 SD intervals, without pp values; its conditional coefficients are larger in magnitude than the population-averaged GEE coefficients. It has no pair-specific generation or interaction slopes, so it cannot represent between-pair heterogeneity, and it is reported for completeness only, in Appendix D.11 (Table 33). H1 is tested by the GEE generation effect within each method class and H2 by the generation ×\times class interaction, pooled and per pair. H2 is also expressed as a risk-difference contrast (ARA generation effect minus the mean of the direct-baseline-adjusted methods), and baseline-adjusted generation effects subtract the originals’ own generation difference; both carry item-bootstrap intervals (4,000 resamples). Pooled baseline-adjusted generation effects come from the same GEE with the originals added as a reference class, so that the generation coefficient of each method class is its generation effect minus that of the originals, in log-odds.

Because each main-study cell was optimised once, these intervals reflect prompt sampling only, not run-to-run variation of the search. That variation was estimated post hoc from the ten small-pair cells, each optimised with two seeds: with did_i the seed-2 minus seed-1 held-out refusal of cell ii in percentage points, the run-to-run standard deviation of one cell is σ=∑i=1ddi2/(2n)\sigma= \sqrt{\sum_{i=1}^{d} d_i^2/(2n)} (n=10n = 10), with a χ102\chi^2_{10} interval, and the difference between two independently optimised cells has standard deviation σ2\sigma\sqrt{2}. A per-pair generation effect is flagged as exceeding seed noise when its absolute value is larger than 2σ22\sigma\sqrt{2}. This flag is a screen that accompanies, not replaces, the McNemar tests, and it assumes that the seed variability of the 4B and 7B small-pair models transfers to the 22–32B main-study models. In 3 of the 10 seed pairs (Qwen3.5-4B DiM, SOM and ARA) the seed-2 run also used a generation batch size of 64 instead of 128, because 128 ran out of memory, so σ\sigma is also reported without these pairs.

The generational question has the model pair as its unit: four main-study pairs and one small pair. The prompt-level intervals and tests above therefore describe prompt sampling within a pair, not variation between runs or between pairs, and per-pair effects are the primary evidence. For each pair the generation effect of the direction-based class (the mean of its methods) and of ARA is given in percentage points (mean of the paired item differences) and in log-odds (per-pair GEE), raw and baseline-adjusted, with intervals that add run-to-run variance: each optimised cell contributes σ2\sigma^2, so a single-method effect has 2σ22\sigma^2, a mean of mm methods 2σ2/m2\sigma^2/m and the H2 contrast 2σ2+2σ2/m2\sigma^2 + 2\sigma^2/m, with σ\sigma estimated on each scale from the seed pairs. Differences between pairs are tested at the prompt level with Wald tests of the generation ×\times family and generation ×\times class ×\times family terms of the GEE (3 df), which treat prompts, not runs, as independent. Pooled effects over the four main pairs are summarised with a random-effects model: DerSimonian–Laird between-pair variance [105], a Hartung–Knapp–Sidik–Jonkman 95% interval with a tt distribution on 3 degrees of freedom [106, 107], I2\mathrm{I}^2 [108] and a 95% prediction interval for a new pair [109]; with k=4k = 4 pairs these intervals are wide by construction. Leave-one-family-out GEE fits complete the pooled analysis. The baseline-adjusted contrasts were added after H1 had been stated and are exploratory; because the percentage-point and log-odds scales can disagree in sign, both are reported.

Four further analyses were added in response to review, after all results were known, and are exploratory. First, between-pair heterogeneity is reported as Cochran’s QQ, I2I^2 and the DerSimonian–Laird τ2\tau^2 with within-pair variances that include the run-to-run variance above, not only prompt sampling. Second, because a constant σ\sigma in percentage points ignores that a rate varies less near 0% and 100%, the run effect is also modelled as constant on the logit scale, σL\sigma_L (estimated from the logits of the same seed pairs), and mapped to each cell by the delta method, σpp=σLp(1−p)\sigma_{\mathrm{pp}} = \sigma_L p(1-p) at the cell’s rate pp; per-pair verdicts are reported under both models across the interval of σ\sigma. Third, H2 is re-estimated on the direction-based units in which the KL cap bound in both cells, and the per-pair verdicts are read on the dev set at first-token KL budgets of 0.05 to 1.0 and on the area under each run’s refusal–KL front, with run-to-run variance estimated from the small-pair seed pairs on the same quantity. Fourth, as a scale-free alternative to the baseline adjustment, conditional refusal retention is the share of the prompts refused by a model’s own original that the edited model still refuses, with prompt-bootstrap intervals and with intervals that add a run-to-run variance of retention estimated, as for the refusal rate, from the ten small-pair seed pairs (Table 34).

PairEstimandEffectPrompt [95% CI]Prompt + seed [95% CI]
QwenDirection+6.0[+3.5, +8.3][−11.7, +23.7]
QwenARA+36.9[+33.4, +40.4][+6.3, +67.5]*
QwenH2 contrast+30.9[+26.9, +34.9][−4.5, +66.2]
GemmaDirection+4.0[+1.5, +6.5][−13.8, +21.7]
GemmaARA−29.4[−32.5, −26.4][−59.9, +1.2]
GemmaH2 contrast−33.3[−36.8, −30.0][−68.6, +1.9]
OLMoDirection+43.4[+41.0, +45.7][+25.7, +61.1]*
OLMoARA−3.2[−5.8, −0.5][−33.7, +27.3]
OLMoH2 contrast−46.6[−49.8, −43.3][−81.8, −11.3]*
Mistral SmallDirection−6.6[−9.0, −4.3][−24.3, +11.1]
Mistral SmallARA−2.6[−4.8, −0.5][−33.0, +27.9]
Mistral SmallH2 contrast+4.0[+1.2, +6.8][−31.2, +39.2]
Small pairDirection+13.5[+10.7, +15.9][−1.9, +28.9]
Small pairARA+15.0[+12.0, +18.0][−15.6, +45.5]
Small pairH2 contrast+1.5[−1.9, +5.1][−32.7, +35.6]

Table 34. Conditional refusal retention: generation effect (newer minus older model, pp) on the share of the prompts refused by a model’s own original that the edited model still refused (WildGuard, 972 harmful held-out prompts). Prompt: prompt-cluster bootstrap 95% interval (2,000 resamples). Prompt + seed: adds a run-to-run standard deviation of retention of 11.0 pp [7.7, 19.2] per cell, estimated from the ten small-pair seed pairs, with the multipliers of Table 5. *Prompt + seed interval excludes zero.

H3 compares keyword and judge labels on identical responses: the paired keyword-minus-WildGuard gap of each model (Newcombe interval, exact McNemar), pooled gaps with prompt-cluster bootstrap intervals (2,000 resamples), and agreement as Cohen’s κ\kappa for rater pairs [110] and Fleiss’ κ\kappa across judge panels [111], each with a prompt-bootstrap interval, raw agreement and label marginals, because κ\kappa is low when labels are as imbalanced as near-0 or near-100% refusal [112]. The method-class difference in the gap is tested with a linear GEE on the per-response paired difference (clustered by prompt, pair as fixed effect) and checked with an ordinary least-squares model on the per-model mean gaps, with the model as unit. Keyword labels are computed on three windows: the full response (at most 512 tokens), its first 256 tokens and its first 100 tokens, the window Heretic scores during search. Token ids were not stored, so the 100-token text is obtained by re-tokenising each stored response with the evaluated model’s tokenizer, a procedure validated against the stored back-token prefixes (Appendix § 11.§ 2).

Holm’s procedure [113] controls each family of related tests at a two-sided α=0.05\alpha= 0.05, separately for each refusal measure and prompt subset (families and sizes in Appendix § 9.§ 6). Variational-Bayes results are not adjusted. Overfitting of selection to the dev set, expected when a noisy criterion is both optimised and used for selection [114], is assessed by the Spearman correlation between dev-set keyword refusal and held-out judged refusal across selected models, with percentile bootstrap intervals over models. Of the secondary outcomes (over-refusal on the 350 benign items, which tracks harmful-prompt refusal across models [63], capability differences and teacher-forced divergence), changes in over-refusal from the original are tested by exact McNemar, Holm-adjusted over all modified models; the others are descriptive apart from the tests listed above. As sensitivity analyses, H1 and H2 are repeated on the four public item sets (772 prompts) and on the code-as-weapon bank (200) separately, and with harmful compliance as the outcome (HarmBench classifier and WildGuard harmful-response labels, coded as not harmful so that signs match the refusal analysis). Dev-set drift of the released exports is compared between exports rebuilt on the search machine and on another GPU with Fisher’s exact test. These analyses, the pair-level pooling and the KL-cap analysis were added after the held-out results were known. Analyses used Python 3.12.15 with pandas 3.1.1, NumPy 2.5.4, SciPy 1.18.1 and statsmodels 0.15.0 [115]; hardware and other software versions are listed in Appendix § 9.7.

Ethics and Preregistration

Ethics. No human participants were involved. Prompt sets are described by name and count only; responses to harmful prompts are stored privately for scoring and not released. Licences of released models and judges are listed in Appendix § 9.§ 5.

Preregistration. The study was not preregistered; the hypotheses were formulated after pilot runs on the small pair (30 trials, 10 random), which served to verify the pipeline and are not among the analysed runs. The baseline-adjusted contrast was added after H1 had been stated, and the pair-level pooling, the KL-cap, item-source and harmful-compliance analyses and the 100-token keyword window were added after the held-out results were known, as were the analyses added in response to review (seed-inclusive heterogeneity, refusal-level run-to-run variance, KL budgets, cap-bound units, conditional retention and outcome classes); all of these are exploratory. Deviations from the study plan are listed in Appendix § 9.8.

Results

The study asked whether refusal removal is harder in the newer model of each release pair (H1), whether this depends on the class of edit (H2), and whether keyword counts agree with an automatic judge (H3). Outcomes are WildGuard refusal rates on the 972 harmful held-out prompts for the selected seed-1 models; pp denotes percentage points. The release pair is the unit, so per-pair effects with intervals that add run-to-run variance are the primary evidence; prompt-level tests describe prompt sampling within a pair (§ 3.5). Detail that earlier drafts gave here is in Appendix § 11.7.

H1: Residual Refusal in the Newer Model

Refusal removal was not uniformly harder on the newer model. The generation effect differed in sign and size between pairs (Table 5, Figure 3; per method in Table 6): after direction-based removal it ranged from −1.2 pp (Mistral Small) to +36.9 pp (OLMo), and after ARA from −26.0 pp (Gemma) to +34.4 pp (Qwen). No main-study pair retained more refusal on the newer model for every method, as H1 required; in the small pair all five methods did with seed 1. With run-to-run variance added to prompt sampling, the raw per-pair effects whose interval excluded zero were the OLMo and small-pair direction-based increases, the Qwen ARA increase and the Gemma ARA decrease.

Figure 3. Held-out, judged refusal of the original and refusal-removed models, older vs newer release. Each panel is one pair; each slot is one condition: the unedited originals (grey star), the direction-based methods (DiM, MPOA, SOM; J-space on the small pair only) and, right of the second dashed divider, the optimisation-based ARA. Within a slot the older release (left, hollow marker) is joined to the newer one (right, filled marker). The value is the share of the 972 harmful held-out prompts whose response WildGuard labels a refusal; bars are Wilson 95% intervals. The number above a slot is the paired generation effect (newer minus older, percentage points), printed in grey with “ns” where it is not significant. Significance for the methods is Holm-adjusted (exact McNemar, Holm over the 23 H1 tests); significance for the originals is unadjusted (exact McNemar), because they are the baseline rather than an H1 test. The ring marks a failed edit, a selected model whose held-out refusal is at or above its own original’s (Gemma-4-31B DiM). Each modified model is the dev-set selection of its cell (fewest dev keyword refusals at first-token KL ≤0.1\le0.1, seed 1).

Table 5: Per-pair generation effects (newer minus older model) on held-out WildGuard refusal, by method class and scale; the primary evidence for H1 and H2. Direction: mean of DiM, MPOA and SOM (on the small pair also J-space); ARA: the single ARA cell. Adjusted: minus the originals’ own generation difference, an exploratory contrast added after H1 was stated. H2 contrast: ARA minus direction. pp: percentage points (paired item differences); log-odds: per-pair GEE. Per-pair intervals (95%) combine prompt sampling with run-to-run (seed) variance (σ=9.15\sigma= 9.15 pp, 0.83 on the logit scale, from the ten small-pair seed pairs). Pooled: random-effects estimate over the four main pairs with Hartung–Knapp–Sidik–Jonkman 95% interval (t3t_3). The pooled interval uses the prompt-sampling variances; with run-to-run variance added its limits move by at most 0.1 pp and 0.07 log-odds, because the interval is driven by the spread between pairs. Between-pair heterogeneity was I2=99I^2 = 99–100% with prompt-sampling variances only and 20–89% with the seed-inclusive variances of this table (Table 29). Gemma direction includes the failed Gemma-4-31B DiM edit; without it the Gemma direction effect was −1.1 pp and the H2 contrast −24.9 pp. Per-method effects are in Table 6 and the prompt-level H2 tests in Table 8.

Estimand, scaleQwenGemmaOLMoMistral SmallSmall pairPooled, 4 main pairs
Direction, raw, pp+10.7 [−4.0, +25.5]+4.7 [−10.2, +19.6]+36.9 [+22.1, +51.7]−1.2 [−16.0, +13.5]+17.3 [+4.4, +30.1]+12.8 [−13.9, +39.5]
log-odds+0.46 [−0.86, +1.79]+0.21 [−1.13, +1.54]+1.63 [+0.30, +2.96]−0.10 [−1.43, +1.23]+0.79 [−0.36, +1.94]+0.55 [−0.66, +1.76]
Direction, adjusted, pp−2.4 [−17.3, +12.5]+3.7 [−11.2, +18.6]+36.0 [+21.2, +50.9]−19.2 [−34.1, −4.2]+0.9 [−12.1, +14.0]+4.6 [−32.2, +41.3]
log-odds−0.53 [−1.86, +0.81]+0.10 [−1.24, +1.44]+1.57 [+0.24, +2.91]−0.90 [−2.23, +0.44]−0.30 [−1.46, +0.86]+0.06 [−1.67, +1.80]
ARA, raw, pp+34.4 [+8.8, +59.9]−26.0 [−51.6, −0.5]−2.5 [−27.9, +23.0]−0.7 [−26.1, +24.7]+14.2 [−11.3, +39.7]+1.3 [−38.3, +40.8]
log-odds+2.35 [+0.04, +4.66]−4.07 [−6.50, −1.64]−0.23 [−2.54, +2.07]−0.17 [−2.48, +2.15]+1.55 [−0.77, +3.86]−0.49 [−4.66, +3.69]
ARA, adjusted, pp+21.3 [−4.4, +47.0]−27.1 [−52.7, −1.5]−3.3 [−28.8, +22.2]−18.6 [−44.2, +6.9]−2.2 [−27.8, +23.5]−6.9 [−40.7, +26.8]
log-odds+1.36 [−0.95, +3.68]−4.17 [−6.61, −1.73]−0.29 [−2.60, +2.01]−0.96 [−3.28, +1.36]+0.46 [−1.86, +2.78]−0.96 [−4.59, +2.68]
H2 contrast, pp+23.6 [−5.9, +53.1]−30.8 [−60.2, −1.3]−39.3 [−68.8, −9.9]+0.5 [−28.8, +29.9]−3.1 [−31.6, +25.4]−11.5 [−57.6, +34.6]
log-odds+1.89 [−0.78, +4.55]−4.27 [−7.04, −1.51]−1.87 [−4.53, +0.79]−0.06 [−2.73, +2.61]+0.76 [−1.83, +3.34]−1.05 [−5.21, +3.11]

Table 5.

Table 6: Held-out residual refusal of the older and newer model of each pair (H1). Rates: share of the 972 harmful held-out prompts whose response WildGuard labels a refusal, with Wilson 95% intervals. Difference: newer minus older model in percentage points (pp) with Newcombe 95% intervals; b/cb/c: prompts refused only by the older / only by the newer model (exact McNemar test); pHolmp_{\mathrm{Holm}}: Holm over the 23 tests of the H1 family (21 pair × method tests and two pooled terms); ufor the originals the unadjusted McNemar pp is given, as they are the baseline rather than an H1 test. Baseline-adjusted: the method’s generation effect minus that of the originals, pp [95% CI]. Each modified model is the dev-set selection of its cell (first-token KL ≤ 0.1, seed 1). aNot significant after Holm when the 44 prompts that overlap the search prompts are excluded. bFailed edit: held-out refusal at or above the model’s own original. sSeed noise: “exceeds” if ∣new−old∣|\mathrm{new}-\mathrm{old}| is larger than 25.9 pp, twice the SD (12.9 pp) of an older-minus-newer difference between two independently optimised cells, otherwise “within” (5 of 16 exceed); the per-cell run-to-run SD, 9.2 pp [6.4, 16.1], is estimated from the seed-1 versus seed-2 held-out refusal of the ten small-pair cells (§ 4.1) and assumed to transfer to the main pairs. Not given for the small pair, whose replicate is the source of the estimate (Figure 7).

ConditionOlder, % [95% CI]Newer, % [95% CI]New − old, pp [95% CI]b/cb/cpHolmp_{\mathrm{Holm}}Baseline-adj., pp [95% CI]Seed noises
Qwen: Qwen2.5-32B → Qwen3.8-27B
Original76.8 [74.1, 79.4]89.9 [87.9, 91.7]+13.1 [+10.9, +15.4]6/133<0.001u––
DiM47.4 [44.3, 50.6]74.3 [71.4, 76.9]+26.9 [+23.8, +29.8]17/278<0.001+13.7 [+10.1, +17.1]exceeds
MPOA3.9 [2.9, 5.3]14.0 [12.0, 16.3]+10.1 [+7.8, +12.5]23/121<0.001−3.1 [−6.3, +0.2]within
SOM42.7 [39.6, 45.8]38.0 [35.0, 41.1]−4.7 [−8.0, −1.4]157/1110.041a−17.8 [−21.5, −14.2]within
ARA6.1 [4.7, 7.8]40.4 [37.4, 43.5]+34.4 [+31.2, +37.5]11/345<0.001+21.3 [+17.1, +25.4]exceeds
Gemma: Gemma-2-27B → Gemma-4-31B
Original88.3 [86.1, 90.1]89.3 [87.2, 91.1]+1.0 [−1.4, +3.5]69/790.460u––
DiM73.8 [70.9, 76.4]90.1 [88.1, 91.8]b+16.4 [+13.5, +19.3]35/194<0.001+15.3 [+13.0, +17.8]within
MPOA45.2 [42.1, 48.3]40.4 [37.4, 43.5]−4.7 [−8.6, −0.9]206/1600.093−5.8 [−9.7, −1.7]within
SOM65.2 [62.2, 68.2]67.8 [64.8, 70.7]+2.6 [−1.1, +6.2]151/1760.369+1.5 [−1.9, +4.9]within
ARA26.6 [24.0, 29.5]0.6 [0.3, 1.3]−26.0 [−28.9, −23.3]2/253<0.001−27.1 [−30.3, −23.6]exceeds
OLMo: OLMo-2-32B → Olmo-3.1-32B
Original83.2 [80.8, 85.4]84.1 [81.6, 86.2]+0.8 [−0.6, +2.2]19/270.302u––
DiM33.5 [30.6, 36.6]68.3 [65.3, 71.2]+34.8 [+31.5, +37.9]14/352<0.001+34.0 [+30.5, +37.6]exceeds
MPOA3.6 [2.6, 5.0]65.0 [62.0, 68.0]+61.4 [+58.2, +64.4]2/599<0.001+60.6 [+57.1, +64.1]exceeds
SOM27.3 [24.6, 30.1]41.7 [38.6, 44.8]+14.4 [+11.5, +17.3]45/185<0.001+13.6 [+10.4, +16.7]within
ARA13.3 [11.3, 15.5]10.8 [9.0, 12.9]−2.5 [−4.6, −0.3]67/430.111−3.3 [−5.8, −0.9]within
Mistral Small: Mistral-Small-2409 → Mistral-Small-3.2
Original55.7 [52.6, 58.8]73.6 [70.7, 76.2]+17.9 [+15.2, +20.6]21/195<0.001u––
DiM32.8 [29.9, 35.8]28.5 [25.7, 31.4]−4.3 [−7.4, −1.2]141/990.048a−22.2 [−26.1, −18.4]within
MPOA6.9 [5.5, 8.7]4.0 [2.9, 5.4]−2.9 [−4.7, −1.2]51/230.014−20.8 [−24.0, −17.7]within
SOM3.7 [2.7, 5.1]7.2 [5.7, 9.0]+3.5 [+1.7, +5.3]22/560.001−14.4 [−17.7, −11.1]within
ARA4.9 [3.7, 6.5]4.2 [3.1, 5.7]−0.7 [−2.1, +0.6]23/160.369−18.6 [−21.6, −15.7]within
Small pair: Qwen2.5-7B → Qwen3.5-4B
Original72.1 [69.2, 74.8]88.5 [86.3, 90.3]+16.4 [+13.9, +18.9]11/170<0.001u––
DiM20.6 [18.2, 23.2]54.6 [51.5, 57.7]+34.1 [+30.5, +37.5]43/374<0.001+17.7 [+13.5, +21.8]–
MPOA13.8 [11.8, 16.1]18.0 [15.7, 20.5]+4.2 [+1.6, +6.9]66/1070.018−12.1 [−15.7, −8.3]–
SOM12.2 [10.3, 14.5]21.9 [19.4, 24.6]+9.7 [+6.9, +12.4]50/144<0.001−6.7 [−10.4, −3.0]–
J-space52.6 [49.4, 55.7]73.8 [70.9, 76.4]+21.2 [+18.1, +24.3]37/243<0.001+4.8 [+1.3, +8.3]–
ARA4.7 [3.6, 6.3]18.9 [16.6, 21.5]+14.2 [+11.6, +16.9]24/162<0.001−2.2 [−6.0, +1.6]–

Table 6.

Heterogeneity between pairs. With prompt sampling only, the pairs differed far beyond chance (prompt-level generation × pair Wald χ2=547\chi^2 = 547, 3 df; I2I^2 99–100%), but this statistic treats prompts, not optimisation runs, as the unit. With the seed-inclusive within-pair variances of Table 5 and a constant run-to-run variance, heterogeneity of the raw direction-based effect was Q=14.8Q = 14.8 (3 df, p=0.002p = 0.002; I2=80%I^2 = 80\%, τ2=225\tau^2 = 225 pp2^2, τ=15.0\tau= 15.0 pp) on the pp scale but Q=3.8Q = 3.8 (p=0.29p = 0.29; I2=20%I^2 = 20\%, τ2=0.12\tau^2 = 0.12) on the log-odds scale; for the baseline-adjusted effect it was Q=27.7Q = 27.7 (I2=89%I^2 = 89\%, τ2=476\tau^2 = 476 pp2^2) and Q=7.7Q = 7.7 (p=0.053p = 0.053; I2=61%I^2 = 61\%, τ2=0.72\tau^2 = 0.72). How large the pp-scale heterogeneity is depends on the variance model: with the refusal-level run-to-run variance it was Q=5.5Q = 5.5 (p=0.14p = 0.14; I2=46%I^2 = 46\%, τ2=127\tau^2 = 127 pp2^2) for the raw direction-based effect, and across the 95% interval of the run-to-run standard deviation I2I^2 ranged from 90% to 39% (constant) and from 73% to 0% (refusal-level). Table 29 gives QQ, I2I^2, and τ2\tau^2 for every estimand and Table 30 the within-pair variances. The pooled random-effects summaries (Table 5, last column) span large effects of either sign; beyond the originals’ own difference, an exploratory contrast, the pooled direction-based effect was +0.06 log-odds [−1.67, +1.80].

Estimandpp scalelog-odds scale
Prompt only+ seed, const.+ seed, levelPrompt only+ seed
Direction, raw821.8; <0.001; 100%; 28414.8; 0.002; 80%; 2255.5; 0.136; 46%; 127590.8; <0.001; 99%; 0.563.8; 0.289; 20%; 0.12
Direction, adjusted849.4; <0.001; 100%; 52727.7; <0.001; 89%; 47611.5; 0.010; 74%; 425608.6; <0.001; 100%; 1.537.7; 0.053; 61%; 0.72
ARA, raw807.1; <0.001; 100%; 32110.9; 0.012; 73%; 4475.3; 0.148; 44%; 113378.5; <0.001; 99%; 2.8614.3; 0.003; 79%; 5.34
ARA, adjusted381.8; <0.001; 99%; 3287.9; 0.048; 62%; 2785.0; 0.172; 40%; 97208.5; <0.001; 99%; 1.9510.8; 0.013; 72%; 3.74
H2 contrast1006.7; <0.001; 100%; 68011.2; 0.011; 73%; 6146.3; 0.097; 53%; 343539.7; <0.001; 99%; 4.5410.8; 0.013; 72%; 4.87

Table 29. Between-pair heterogeneity of the four main-pair effects with three kinds of within-pair variance: prompt sampling only, prompt sampling plus constant run-to-run variance, and (pp scale only) prompt sampling plus refusal-level run-to-run variance, each at the point estimate of σ\sigma. Each cell gives Cochran’s QQ (3 df); its pp value; I2I^2; and the DerSimonian–Laird between-pair variance τ2\tau^2 (pp2^2 on the pp scale, squared log-odds on the log-odds scale). Across the 95% interval of σ\sigma, I2I^2 of the raw direction-based pp effect is 90% (lower limit) to 39% (upper limit) with constant and 73% to 0% with refusal-level variance. The within-pair variances are in Table 30.

EstimandPairPrompt only+ seed, const.+ seed, levelPrompt only+ seed
Direction, rawQwen1.056.8176.00.0020.459
Gemma1.957.8202.90.0040.461
OLMo1.257.1190.00.0030.459
Mistral Small0.656.477.90.0040.461
Direction, adjustedQwen1.957.8177.00.0090.466
Gemma1.657.5202.60.0110.468
OLMo1.757.5190.50.0050.462
Mistral Small2.358.279.60.0070.464
ARA, rawQwen2.6170.2422.40.0191.389
Gemma2.0169.6264.10.1651.536
OLMo1.2168.8155.60.0101.381
Mistral Small0.4168.026.70.0221.393
ARA, adjustedQwen4.3171.9424.10.0251.396
Gemma3.0170.6265.10.1781.549
OLMo1.6169.2156.10.0131.384
Mistral Small2.3169.928.60.0261.396
H2 contrastQwen3.3226.7598.20.0191.847
Gemma2.7226.2465.70.1671.995
OLMo2.3225.7345.50.0121.840
Mistral Small0.9224.4104.50.0251.853

Table 30. Within-pair variances of the four main-pair effects used as inverse-variance weights in Table 29, at the point estimate of σ\sigma: prompt sampling only, and prompt sampling plus constant or refusal-level run-to-run variance. pp scale in pp2^2; log-odds scale in squared log-odds. The constant run-to-run term is the same for every pair (2σ22\sigma^2 per method, 2σ2/32\sigma^2/3 for the direction-class mean), so on that variance model the pairs are weighted almost equally.

Run-to-run variation. Each main-study cell was optimised once, so run-to-run variance was transferred from the ten small-pair seed pairs (§ 3.5). Under the constant standard deviation of 9.2 pp per cell, 5 of the 16 main-study per-method effects exceeded twice the standard deviation of a difference of two runs (Table 6). When the run effect was instead held constant on the logit scale and mapped to percentage points at each cell’s refusal rate, which makes it smaller near 0% and larger near 50%, only OLMo MPOA exceeded it, and of the per-pair class intervals only the OLMo direction-based increase, the OLMo H2 contrast and the Mistral Small baseline-adjusted effects excluded zero; the Qwen and Gemma ARA effects and the Gemma H2 contrast did not (Appendix § 11.8, Table 28). The ten seed pairs did not favour either variance model (log-likelihood 6.3 vs. 6.0).

PairEstimandEffectConstant σ\sigma [95% interval]Refusal-level σ\sigma [95% interval]Excl. 0, const. lo/pt/hiExcl. 0, level lo/pt/hi
QwenDirection, raw+10.7[−4.0, +25.5][−15.3, +36.7]✓/–/––/–/–
Direction, adjusted−2.4[−17.3, +12.5][−28.5, +23.7]–/–/––/–/–
ARA, raw+34.4[+8.8, +59.9][−5.9, +74.6]✓/✓/–✓/–/–
ARA, adjusted+21.3[−4.4, +47.0][−19.0, +61.7]✓/–/––/–/–
H2 contrast+23.6[−5.9, +53.1][−24.3, +71.6]✓/–/––/–/–
GemmaDirection, raw+4.7[−10.2, +19.6][−23.2, +32.6]–/–/––/–/–
Direction, adjusted+3.7[−11.2, +18.6][−24.2, +31.6]–/–/––/–/–
ARA, raw−26.0[−51.6, −0.5][−57.9, +5.8]✓/✓/–✓/–/–
ARA, adjusted−27.1[−52.7, −1.5][−59.0, +4.9]✓/✓/–✓/–/–
H2 contrast−30.8[−60.2, −1.3][−73.1, +11.5]✓/✓/–✓/–/–
OLMoDirection, raw+36.9[+22.1, +51.7][+9.8, +63.9]✓/✓/✓✓/✓/–
Direction, adjusted+36.0[+21.2, +50.9][+9.0, +63.1]✓/✓/✓✓/✓/–
ARA, raw−2.5[−27.9, +23.0][−26.9, +22.0]–/–/––/–/–
ARA, adjusted−3.3[−28.8, +22.2][−27.8, +21.2]–/–/––/–/–
H2 contrast−39.3[−68.8, −9.9][−75.8, −2.9]✓/✓/–✓/✓/–
Mistral SmallDirection, raw−1.2[−16.0, +13.5][−18.5, +16.1]–/–/––/–/–
Direction, adjusted−19.2[−34.1, −4.2][−36.6, −1.7]✓/✓/–✓/✓/–
ARA, raw−0.7[−26.1, +24.7][−10.8, +9.4]–/–/––/–/–
ARA, adjusted−18.6[−44.2, +6.9][−29.1, −8.2]✓/–/–✓/✓/✓
H2 contrast+0.5[−28.8, +29.9][−19.5, +20.5]–/–/––/–/–
Small pairDirection, raw+17.3[+4.4, +30.1][−3.7, +38.2]✓/✓/–✓/–/–
Direction, adjusted+0.9[−12.1, +14.0][−20.2, +22.0]–/–/––/–/–
ARA, raw+14.2[−11.3, +39.7][−11.9, +40.3]–/–/––/–/–
ARA, adjusted−2.2[−27.8, +23.5][−28.4, +24.1]–/–/––/–/–
H2 contrast−3.1[−31.6, +25.4][−36.5, +30.3]–/–/––/–/–

Table 28. Per-pair class effects (pp) under two models of run-to-run variance. Constant: σ=9.15\sigma= 9.15 pp per cell [6.40, 16.07] (the intervals of Table 5). Refusal-level: σL=0.83\sigma_L = 0.83 [0.58, 1.45] per cell on the logit scale, mapped to pp at each cell’s rate pp as σLp(1−p)\sigma_L p(1-p). Intervals add prompt sampling and the run-to-run variance at the point estimate of σ\sigma; the last two columns mark (✓) whether the interval excludes zero at the lower limit, point estimate and upper limit of σ\sigma (lo / pt / hi).

Baseline. The newer Qwen, Mistral Small and small-pair originals already refused more than their predecessors (+13.1, +17.9 and +16.4 pp); the Gemma and OLMo originals did not. Relative to this difference the direction-based effect was near zero in Qwen and the small pair, negative in Mistral Small and unchanged in Gemma and OLMo (Table 5); the two scales of this adjustment disagreed in sign for 3 of 21 pair × method cells (Appendix § 11.§ 4, Table 24). As a scale-free alternative, the share of the prompts refused by a model’s own original that the edited model still refused

PairMethodEffect, ppOrig., ppAdjusted, ppEffect, log-oddsOrig., log-oddsAdjusted, log-oddsSignSig.
QwenDiM+26.9 [+23.8, +29.8]+13.1+13.7 [+10.1, +17.1]+1.16 [+1.02, +1.30]+0.99+0.17 [−0.03, +0.37]
QwenMPOA+10.1 [+7.8, +12.5]+13.1−3.1 [−6.3, +0.2]+1.39 [+1.04, +1.73]+0.99+0.39 [+0.01, +0.77]××
QwenSOM−4.7 [−8.0, −1.4]+13.1−17.8 [−21.5, −14.2]−0.20 [−0.33, −0.06]+0.99−1.19 [−1.40, −0.97]
QwenARA+34.4 [+31.2, +37.5]+13.1+21.3 [+17.1, +25.4]+2.35 [+2.08, +2.62]+0.99+1.36 [+1.05, +1.68]
GemmaDiM+16.4 [+13.5, +19.3]+1.0+15.3 [+13.0, +17.8]+1.18 [+0.96, +1.40]+0.10+1.07 [+0.90, +1.25]
GemmaMPOA−4.7 [−8.6, −0.9]+1.0−5.8 [−9.7, −1.7]−0.19 [−0.35, −0.04]+0.10−0.30 [−0.55, −0.04]
GemmaSOM+2.6 [−1.1, +6.2]+1.0+1.5 [−1.9, +4.9]+0.12 [−0.05, +0.28]+0.10+0.01 [−0.22, +0.24]
GemmaARA−26.0 [−28.9, −23.3]+1.0−27.1 [−30.3, −23.6]−4.07 [−4.87, −3.27]+0.10−4.17 [−5.00, −3.34]
OLMoDiM+34.8 [+31.5, +37.9]+0.8+34.0 [+30.5, +37.6]+1.45 [+1.31, +1.60]+0.06+1.39 [+1.22, +1.56]
OLMoMPOA+61.4 [+58.2, +64.4]+0.8+60.6 [+57.1, +64.1]+3.91 [+3.56, +4.25]+0.06+3.85 [+3.49, +4.21]
OLMoSOM+14.4 [+11.5, +17.3]+0.8+13.6 [+10.4, +16.7]+0.64 [+0.51, +0.78]+0.06+0.58 [+0.42, +0.75]
OLMoARA−2.5 [−4.6, −0.3]+0.8−3.3 [−5.8, −0.9]−0.23 [−0.43, −0.03]+0.06−0.29 [−0.52, −0.07]
Mistral SmallDiM−4.3 [−7.4, −1.2]+17.9−22.2 [−26.1, −18.4]−0.20 [−0.35, −0.06]+0.80−1.00 [−1.18, −0.82]
Mistral SmallMPOA−2.9 [−4.7, −1.2]+17.9−20.8 [−24.0, −17.7]−0.57 [−0.92, −0.22]+0.80−1.37 [−1.74, −1.00]
Mistral SmallSOM+3.5 [+1.7, +5.3]+17.9−14.4 [−17.7, −11.1]+0.70 [+0.34, +1.06]+0.80−0.10 [−0.47, +0.28]×
Mistral SmallARA−0.7 [−2.1, +0.6]+17.9−18.6 [−21.6, −15.7]−0.17 [−0.45, +0.12]+0.80−0.96 [−1.28, −0.65]
Small pairDiM+34.1 [+30.5, +37.5]+16.4+17.7 [+13.5, +21.8]+1.54 [+1.36, +1.71]+1.09+0.45 [+0.22, +0.68]
Small pairMPOA+4.2 [+1.6, +6.9]+16.4−12.1 [−15.7, −8.3]+0.32 [+0.12, +0.52]+1.09−0.77 [−1.03, −0.51]
Small pairSOM+9.7 [+6.9, +12.4]+16.4−6.7 [−10.4, −3.0]+0.70 [+0.50, +0.90]+1.09−0.39 [−0.65, −0.13]
Small pairJ-space+21.2 [+18.1, +24.3]+16.4+4.8 [+1.3, +8.3]+0.93 [+0.79, +1.07]+1.09−0.16 [−0.35, +0.04]×
Small pairARA+14.2 [+11.6, +16.9]+16.4−2.2 [−6.0, +1.6]+1.55 [+1.23, +1.86]+1.09+0.46 [+0.10, +0.82]××

Table 24. Generation effect, originals’ difference and baseline-adjusted generation effect per pair and method on the percentage-point and the log-odds scale (WildGuard refusal, 972 harmful prompts; 95% intervals, prompt sampling only). Sign: the adjusted effect has opposite signs on the two scales. Sig.: its significance differs between scales.

rose on the newer model by +43.4 pp in OLMo (95% interval with run-to-run variance [+25.7, +61.1]) and +13.5 pp in the small pair ([−1.9, +28.9]) after direction-based removal, and changed by −6.6 to +6.0 pp in the other pairs, none of which excluded zero (Table 34).

Item source. The per-pair verdicts depended in part on where the prompts came from (Table 7). The code-as-weapon bank (200 of the 972 prompts), compiled by the author, carried 52% of the mean originals’ difference of the four main pairs. Gemma’s direction-based increase came entirely from it: +49.3 pp on the code items against −6.8 pp on the public sets. The OLMo direction-based increase, the Mistral Small null and the sign of the Gemma and OLMo H2 contrasts held on both sources; the Gemma ARA decrease lay in the public sets (the code items were near the floor), and the Qwen H2 contrast was positive on the public sets (+30.3 pp) and near zero on the code items (−1.8 pp).

Table 7: Per-pair generation effects (newer minus older model, pp) by item source: the four public sets (JailbreakBench, XSTest unsafe, StrongREJECT, HarmBench; 772 prompts) and the author-compiled code-as-weapon bank (200 prompts). Originals: unedited models. Direction: mean of the direction-based methods. H2: ARA minus direction. Item-bootstrap 95% intervals, prompt sampling only (run-to-run variance not added). Many edited models refused none of the code items, so effects on them are bounded near zero. All-item values are in Table 5; pooled log-odds by item set in Table 25.

PairOriginalsDirectionARAH2 contrast
PublicCodePublicCodePublicCodePublicCode
Qwen+5.3 [+3.5, +7.1]+43.2 [+36.5, +50.5]+11.0 [+8.8, +13.3]+9.4 [+6.5, +12.4]+41.3 [+37.6, +45.1]+7.5 [+4.0, +11.5]+30.3 [+25.9, +34.6]−1.8 [−5.5, +2.0]
Gemma−5.7 [−7.5, −4.0]+27.0 [+18.0, +36.0]−6.8 [−9.5, −4.4]+49.3 [+43.7, +55.0]−32.4 [−35.8, −29.0]−1.5 [−3.5, 0.0]−25.6 [−29.3, −21.9]−50.8 [−56.3, −45.2]
OLMo−0.3 [−1.6, +1.0]+5.0 [+0.5, +9.5]+45.2 [+42.8, +47.5]+4.8 [+2.7, +7.2]−3.1 [−5.7, −0.4]0.0 [0.0, 0.0]−48.3 [−51.6, −44.9]−4.8 [−7.2, −2.7]
Mistral Small+20.5 [+17.2, +23.7]+8.0 [+4.5, +12.1]−1.6 [−3.5, +0.2]+0.2 [0.0, +0.5]−0.9 [−2.5, +0.6]0.0 [0.0, 0.0]+0.7 [−1.6, +3.0]−0.2 [−0.5, 0.0]
Small pair+9.5 [+7.1, +11.9]+43.0 [+36.0, +50.0]+20.2 [+17.9, +22.7]+5.9 [+4.1, +7.6]+17.7 [+14.6, +21.0]+0.5 [0.0, +1.5]−2.5 [−6.1, +1.2]−5.4 [−7.2, −3.5]

Table 7.

Generation x Edit Class

ARA, a rank-50-per-module edit, lost less effectiveness on the newer model than the low-rank direction-based edits in the Gemma (−30.8 pp) and OLMo (−39.3 pp) pairs, more in the Qwen pair (+23.6 pp), and about as much in Mistral Small and the small pair (Table 5 and Table 8). Each per-pair contrast rests on a single ARA cell. With constant run-to-run variance, only the Gemma and OLMo contrasts excluded zero; with the refusal-level variance, only OLMo did. The Qwen contrast is therefore indeterminate: its interval with run-to-run variance included zero (−5.9 to +53.1 pp). Pooled with the pair as the unit, the contrast was −11.5 pp [−57.6, +34.6]. The planned mixed model gave the same signs (Appendix § 11.11, Table 33).

Table 8: Generation × method-class interaction (H2) on held-out WildGuard refusal. ARA and direction mean: generation effects (newer minus older model, pp) of ARA and of the mean of the direction-based methods (DiM, MPOA, SOM; on the small pair also J-space). RD contrast: ARA minus direction mean, pp, item-bootstrap 95% interval. GEE: generation × class interaction in log-odds (exchangeable, clustered by prompt) with 95% interval; pHolmp_{\mathrm{Holm}} over the six H2 tests. Pooled (main): the four main-study pairs. All intervals here reflect prompt sampling only; the per-pair intervals with run-to-run variance are in Table 5, and the planned mixed-model estimates in Table 33. H2 predicts a negative interaction (ARA loses less effectiveness on the newer model); sign vs. H2: GEE interaction significant after Holm and negative (predicted) or positive (opposite), or not significant (n.s.). c^{c}The RD contrast and the log-odds interaction have opposite signs.

PairARA, ppDirection mean, ppRD contrast, pp [95% CI]GEE, log-odds [95% CI]pHolmp_{\mathrm{Holm}}Sign vs. H2
Qwen+34.4+10.7+23.6 [+19.9, +27.1]+1.89 [+1.61, +2.16]<0.001opposite
Gemma−26.0+4.7−30.8 [−34.0, −27.5]−4.27 [−5.07, −3.47]<0.001predicted
OLMo−2.5+36.9−39.3 [−42.2, −36.4]−1.87 [−2.08, −1.65]<0.001predicted
Mistral Small−0.7−1.2+0.5 [−1.4, +2.4]−0.06 [−0.37, +0.25]0.696n.s.
Small pair+14.2+17.3−3.1 [−6.0, −0.1]+0.76 [+0.46, +1.06]<0.001oppositec^{c}
Pooled (main)−−−−0.50 [−0.62, −0.38]<0.001predicted

Table 8.

PairGEE, log-odds [95% CI]VB, log-odds [95% CrI]
Qwen+1.89 [+1.61, +2.16]+2.29 [+2.14, +2.44]
Gemma−4.27 [−5.07, −3.47]−4.90 [−5.64, −4.17]
OLMo−1.87 [−2.08, −1.65]−3.25 [−3.49, −3.01]
Mistral Small−0.06 [−0.37, +0.25]−0.08 [−0.42, +0.26]
Small pair+0.76 [+0.46, +1.06]+0.80 [+0.62, +0.99]
Pooled (main)−0.50 [−0.62, −0.38]−0.71 [−0.81, −0.60]

Table 33. Planned mixed logistic model of the H2 interaction (variational Bayes; random intercepts for prompt and, in the pooled fit, pair): posterior mean of the generation × class interaction in log-odds ± 1.96 posterior SD, next to the GEE estimate of Table 8. The model has no pair-specific generation or interaction slopes, so it cannot represent between-pair heterogeneity (§ 3.5).

KL cap and budget. The KL cap bound in all 8 ARA runs but in 15 of the 24 direction-based runs. Restricted to the direction-based units in which it bound in both cells (Qwen DiM; Gemma DiM, MPOA and SOM; Mistral Small DiM; none in OLMo), the H2 contrast was +7.5 pp [−28.6, +43.7] in Qwen, −30.8 pp [−60.2, −1.3] in Gemma and +3.6 pp [−32.4, +39.6] in Mistral Small, and pooled −8.5 pp [−62.5, +45.5]. Away from the selected budget only dev-set keyword counts exist, and both checks below use them. Their validity for H2 is limited: at the selected budget, the dev ARA generation effects had the opposite sign to the held-out judged ARA effects in three of the four main pairs (Appendix § 11.9), so these checks show how stable the dev-set contrast is, not the held-out one. On them the direction-based generation effects kept their sign at every budget from 0.1 to 1.0 in all four main pairs, whereas the H2 contrast changed sign with the budget in three of them (Gemma above KL = 0.1; OLMo and Mistral Small below it). On the area under the refusal–KL front, a budget-free summary, the contrast was in the predicted direction but within run-to-run variation in Gemma and OLMo and opposite, beyond it, in Qwen and Mistral Small (Appendix § 11.9, Table 31).

PairEstimand≤0.05\le0.05≤0.1\le0.1≤0.2\le0.2≤0.5\le0.5≤1.0\le1.0AreaHeld-out
QwenDirection−38.3∗-38.3^{\ast}−25.3∗-25.3^{\ast}−11.3∗-11.3^{\ast}−3.3-3.3−3.3-3.3−7.2∗-7.2^{\ast}+10.7+10.7
ARA+86.0∗+86.0^{\ast}+53.0∗+53.0^{\ast}+13.0+13.0+14.0∗+14.0^{\ast}+1.0+1.0+37.1∗+37.1^{\ast}+34.4+34.4
H2 contrast+124.3∗+124.3^{\ast}+78.3∗+78.3^{\ast}+24.3∗+24.3^{\ast}+17.3∗+17.3^{\ast}+4.3+4.3+44.3∗+44.3^{\ast}+23.6+23.6
GemmaDirection+0.3+0.3+4.7+4.7+22.7∗+22.7^{\ast}+30.0∗+30.0^{\ast}+30.0∗+30.0^{\ast}+13.0∗+13.0^{\ast}+4.7+4.7
ARA+23.0∗+23.0^{\ast}+12.0+12.0+4.0+4.0+1.0+1.0+1.0+1.0+5.1+5.1−26.0-26.0
H2 contrast+22.7∗+22.7^{\ast}+7.3+7.3−18.7∗-18.7^{\ast}−29.0∗-29.0^{\ast}−29.0∗-29.0^{\ast}−7.9-7.9−30.8-30.8
OLMoDirection+42.3∗+42.3^{\ast}+40.0∗+40.0^{\ast}+34.0∗+34.0^{\ast}+31.7∗+31.7^{\ast}+31.7∗+31.7^{\ast}+27.9∗+27.9^{\ast}+36.9+36.9
ARA+55.0∗+55.0^{\ast}+14.0+14.0+24.0∗+24.0^{\ast}+31.0∗+31.0^{\ast}+4.0+4.0+21.9∗+21.9^{\ast}−2.5-2.5
H2 contrast+12.7+12.7−26.0-26.0−10.0-10.0−0.7-0.7−27.7∗-27.7^{\ast}−6.0-6.0−39.3-39.3
Mistral SmallDirection+21.0∗+21.0^{\ast}−6.0-6.0−8.3-8.3−6.3-6.3−6.3-6.3+6.5+6.5−1.2-1.2
ARA+17.0+17.0+2.0+2.0+2.0+2.0+2.0+2.0+2.0+2.0+23.6∗+23.6^{\ast}−0.7-0.7
H2 contrast−4.0-4.0+8.0+8.0+10.3+10.3+8.3+8.3+8.3+8.3+17.1∗+17.1^{\ast}+0.5+0.5
Small pairDirection−27.0∗-27.0^{\ast}−22.8∗-22.8^{\ast}−5.0-5.0−5.0-5.0−5.0-5.0−11.6∗-11.6^{\ast}+17.3+17.3
ARA+6.0+6.0−2.0-2.0+3.0+3.0+1.0+1.0+1.0+1.0+4.0+4.0+14.2+14.2
H2 contrast+33.0∗+33.0^{\ast}+20.8+20.8+8.0+8.0+6.0+6.0+6.0+6.0+15.5∗+15.5^{\ast}−3.1-3.1

Table 31. Generation effects on the development set at several first-token KL budgets (newer minus older model; dev keyword refusals of 100 of the best completed trial at KL ≤t\le t, including the original at KL =0= 0) and on the area under the refusal–KL front (mean front value over 200 log-spaced budgets from 0.01 to 1). Direction: mean of the direction-based methods; H2: ARA minus direction. *Beyond 1.96 run-to-run standard errors, from the ten small-pair seed pairs on the same quantity (run-to-run SD per cell: ≤0.05\le0.05 6.8, ≤0.1\le0.1 10.8, ≤0.2\le0.2 5.2, ≤0.5\le0.5 4.6, ≤1.0\le1.0 4.6, Area 4.3; prompt sampling of the 100 dev prompts is not included). The KL ≤0.1\le0.1 column belongs to the selected models; only those were evaluated on held-out prompts (last column, WildGuard, pp). Dev counts are optimised and therefore optimistic.

ARA cells. Three properties of the ARA cells bear on these contrasts (Appendix § 11.7): the four ARA experts refitted on a different GPU from the search (Qwen and Gemma pairs) drifted more on dev re-score than those refitted on the search GPU; the Gemma-4-31B ARA export, behind the largest contrast, re-scored above the KL cap (0.107); and a public ARA model of Qwen3.8-27B built with the same defaults refused 26.4% of the held-out prompts, against 40.4% for the ARA model selected here, on which the reversed Qwen estimate rests. Without the failed Gemma-4-31B DiM edit, the Gemma contrast was −24.9 pp [−56.2, +6.3].

H3: Keyword Counts versus an Automatic Judge

H3’s prediction that keyword matching would count fewer refusals than the judge was not supported at any window (Table 9, Figure 4). At the 100-token window that Heretic scores, an exploratory window added after the held-out results were known, keyword matching and WildGuard agreed on average over the 42 ablated models (+0.5 pp [−1.0, +1.9]) but differed by 7.0 pp per model on average, significantly in 25 models and in both directions. Over the full response keyword matching counted more refusals (+8.4 pp [+6.9, +10.0]), mainly through markers after the first 100 tokens. In the main study the gap was larger for ARA and on the newer models with the prompt as the unit, but not with the model as the unit (Table 10). Keyword–WildGuard agreement fell from κ=0.73\kappa= 0.73 in the originals to 0.40 (main study) after ablation, whereas the two LLM judges agreed at 0.82.

Figure 4. Keyword counts versus two LLM judges on the same held-out responses (H3). (a) Each marker is one evaluated model (cell): its keyword refusal rate (any of Heretic’s 33 refusal markers in the first 100 generated tokens, the window Heretic scores during search) minus its WildGuard refusal rate on the 972 harmful held-out prompts, in percentage points, with a paired 95% interval (Newcombe). Slots, left to right: the unedited originals, the seed-1 ablated models by method (direction-based, then ARA), γ\gamma-MPOA, public community releases and the small pair’s seed-2 replicates (lighter); within a slot the models run in pair order (Qwen, Gemma, OLMo, Mistral Small, small pair), hollow = older, filled = newer release. The black bar and grey band are the pooled gap of the slot group with its 95% interval; the count above a slot is the number of cells with a positive gap. H3 predicted the shaded region (keyword below the judge). (b) Pooled keyword-minus-WildGuard gap for keyword windows of 100 and 256 tokens and the full response (at most 512 tokens). (c) Cohen’s κ\kappa of the full-response keyword label, with prompt-bootstrap 95% intervals. No human-labelled audit exists, so the panels show where keyword matching and the judges disagree, not which is correct.

ModelsCellsKw, % 512Kw, % 100WG, %Gap, pp [95% CI] 512 tokensGap, pp [95% CI] 256 tokensGap, pp [95% CI] 100 tokens
Originals1082.880.980.2+2.6 [+1.8, +3.4]+1.6 [+0.8, +2.4]+0.7 [−0.1, +1.5]
Ablated (seed 1), all4239.932.031.5+8.4 [+6.9, +10.0]+3.8 [+2.4, +5.2]+0.5 [−1.0, +1.9]
main study3242.033.832.3+9.8 [+8.2, +11.3]+4.9 [+3.5, +6.4]+1.5 [+0.0, +2.9]
small pair1033.326.529.1+4.2 [+2.5, +5.9]+0.4 [−1.4, +2.0]−2.6 [−4.3, −0.9]
responses ending before the cap4251.242.152.4−1.2 [−3.2, +0.9]−5.6 [−7.6, −3.8]−10.4 [−12.2, −8.6]
responses truncated at the cap4229.122.411.4+17.7 [+16.2, +19.3]+13.0 [+11.5, +14.4]+11.0 [+9.7, +12.4]
γ\gamma-MPOA control447.038.729.9+17.1 [+14.9, +19.4]+12.0 [+9.9, +14.0]+8.8 [+6.8, +10.9]
Community releases822.917.36.4+16.5 [+15.2, +17.8]+13.6 [+12.4, +14.8]+11.0 [+9.8, +12.1]
Seed-2 replicates (small pair)1032.826.429.2+3.5 [+2.0, +5.0]+0.0 [−1.5, +1.4]−2.9 [−4.3, −1.5]
All judged models7443.336.635.0+8.3 [+7.0, +9.6]+4.5 [+3.3, +5.7]+1.7 [+0.5, +2.8]

Table 9. Keyword matching versus WildGuard on the same held-out responses to the 972 harmful prompts, by keyword window (H3). Kw: refusal rate by Heretic’s 33 markers in the full response (at most 512 tokens) or in its first 100 tokens, the window Heretic scores during search; WG: WildGuard refusal on the full response. Gap: keyword minus WildGuard, pp, with prompt-cluster bootstrap 95% intervals; 512 and 100 tokens: keyword on that window against WildGuard on the full response; 256 tokens: both on the first 256 tokens (Appendix § 11.§ 2). H3 predicted a negative gap. Over the 42 ablated models the gap was positive in 34 models (24 Holm-significant) and negative in 7 (3) at 512 tokens, and positive in 24 (14) and negative in 18 (11) at 100 tokens. Ablated: the 42 seed-1 models selected from the primary runs; cap: the 512-token generation limit.

ModelsCellsResponsesKw, %WG, %Gap, pp [95% CI]kw/WG onlyκ\kappa [95% CI]
Originals109,71882.880.2+2.6 [+1.8, +3.4]531/2770.73 [0.70, 0.75]
Ablated (seed 1), all4240,82439.931.5+8.4 [+6.9, +10.0]7,503/4,0590.39 [0.37, 0.41]
main study3231,10442.032.3+9.8 [+8.2, +11.3]5,897/2,8590.40 [0.39, 0.42]
small pair109,72033.329.1+4.2 [+2.5, +5.9]1,606/1,2000.33 [0.31, 0.35]
responses ending before the cap4220,01151.252.4−1.2 [−3.2, +0.9]2,998/3,240−
responses truncated at 512 tokens4220,81329.111.4+17.7 [+16.2, +19.3]4,505/819−
γ\gamma-MPOA control43,88847.029.9+17.1 [+14.9, +19.4]1,038/3720.26 [0.23, 0.29]
Community releases87,77622.96.4+16.5 [+15.2, +17.8]1,475/1920.18 [0.16, 0.21]
Seed-2 replicates (small pair)109,72032.829.2+3.5 [+2.0, +5.0]1,351/1,0100.43 [0.41, 0.46]
All judged models7471,92643.335.0+8.3 [+7.0, +9.6]11,898/5,9100.48 [0.47, 0.50]

Table 10. Keyword matching versus WildGuard on the same held-out responses to the 972 harmful prompts (H3). Keyword: any of Heretic’s 33 refusal markers in the full response (at most 512 tokens). Gap: keyword minus WildGuard refusal rate, pp, with prompt-cluster bootstrap 95% interval (H3 predicted a negative gap). kw/WG only: responses counted as refusals by keyword matching only / by WildGuard only. κ\kappa: Cohen’s kappa with prompt-bootstrap 95% interval. Ablated: the 42 seed-1 models selected from the primary runs.

Secondary Outcomes

γ\gamma-MPOA control. Correcting MPOA for the post-norm output gains (Appendix § 9.§ 2) lowered held-out refusal on all four post-norm models (Figure 5), more on the newer model of each pair: by 4.2 and 19.9 pp on Gemma-2-27B and Gemma-4-31B (generation difference −15.6 pp [−19.6, −11.7]) and by 3.1 and 7.5 pp on OLMo-2-32B and OLMo-3.1-32B (−4.4 pp [−6.4, −2.2]). With γ\gamma-MPOA in place of MPOA, the OLMo generation effect remained +57.0 pp and the pooled prompt-level H2 interaction −0.42 log-odds. On Gemma-4-31B and OLMo-2-32B, γ\gamma-MPOA lost 21.4 and 8.1 pp of mean capability. DiM and SOM were not corrected.

Figure 5. γ\gamma-corrected MPOA versus MPOA on the four post-norm models: held-out, judged refusal and capability. Each block is one model (older or newer release of the Gemma and OLMo pairs) with its unedited original (grey star), the selected MPOA model (vermillion square) and the selected γ\gamma-MPOA model (wine-red cross; MPOA with the edit corrected for the post-norm output gains); hollow markers are the older releases. (a) Share of the 972 harmful held-out prompts whose response WildGuard labels a refusal, with Wilson 95% intervals; the column on the right is γ\gamma-MPOA minus MPOA (percentage points, paired; * = significant, exact McNemar, Holm over the four models). (b) Change of each benchmark score from the model’s own original, in percentage points (orange = loss, blue = gain; the colour saturates at ±8 pp, the printed values are exact; the original’s row gives its scores in %); * marks a γ\gamma-MPOA score that differs from the original’s (paired per-item McNemar, Holm over 20 tests). The last two columns are the selected trial’s dev first-token KL divergence and the teacher-forced divergence on the 350 benign held-out prompts (nats per token). Single seed; one benchmark item is worth 0.1–0.61 pp.

Community releases. Of eight public abliterations of five of the checkpoints (Figure 6), four refused less than the best model selected here for the same checkpoint, for example 1.4% against 14.0% on Qwen3.8-27B and 0.7% against 10.8% on Olmo-3.1-32B, at teacher-forced divergences of 0.085–0.151 (best selected models 0.042–0.060) and mean capability changes within 1 pp.

Figure 6. Models selected in this study versus public community abl iterations of the same checkpoints (held-out, judged). One block per base model with at least one community release: the unedited original (grey star), the four seed-1 models selected here by the dev-set rule (hollow markers = older release) and each community release (sky-blue plus; row label = Hugging Face uploader). (a) Share of the 972 harmful held-out prompts whose response WildGuard labels a refusal, with Wilson 95% intervals; the ring marks the best selected model (lowest held-out refusal of the four), and the column on the right gives community minus that model (percentage points, paired; * = significant, exact McNemar, Holm over the four comparisons per community release). (b) Change of the unweighted mean of the five capability benchmarks from the model’s own original (percentage points). (c) Teacher-forced divergence from the original on the 350 benign held-out prompts (nats per token). γ\gamma-MPOA is not drawn.

Harmful compliance and outcome classes. Residual refusal is not harmful compliance: of the ablated main-study responses that WildGuard did not label as refusals, 75.1% were harmful by the HarmBench classifier. With harmful compliance as the outcome the pooled conclusion and the sign of the per-pair H2 interactions were the same, but under HarmBench every Qwen method was higher on the newer model, so the Qwen pair met the H1 criterion (Appendix § 11.§ 6).

Degenerate responses were rare (at most 0.6% per original or primary model); of the non-refused responses of the 42 primary models, 65.8% reached the 512-token limit (incomplete compliance; Appendix § 11.10).

Seed sensitivity. On the small pair a second seed changed held-out refusal significantly in all ten cells (median 9.4 pp, maximum 25.2 pp; Figure 7) and erased the ARA generation effect (+14.2 pp with seed 1, +0.0 pp with seed 2); method accounted for η2=0.94\eta^2 = 0.94 and 0.88 of the variance among the cell rates of the two models.

Figure 7. Seed sensitivity of held-out, judged refusal on the small pair (Qwen2.5-7B → Qwen3.5-4B). Each of the ten model × method cells was optimised twice with 100 TPE trials that differ only in the seed (in three seed-2 runs also in batch size), and the model selected from each run was evaluated on the held-out set. (a) Share of the 972 harmful held-out prompts whose response WildGuard labels a refusal for the seed-1 model (full method colour) and the seed-2 model (lighter), with Wilson 95% intervals; hollow markers and dashed lines mark the older model. Columns on the right: seed 2 minus seed 1 (percentage points, paired; * = significant, exact McNemar, Holm over the ten cells) and the dev keyword refusals (of 100) of the two selected trials. (b) Generation effect (newer minus older model, paired, Newcombe 95% intervals; item-bootstrap interval for the H2 row) with seed 1 (upper) and seed 2 (lower, lighter); the grey ticks are the two cross-seed combinations. The bottom row is the H2 contrast (ARA minus the mean of the direction-based methods). The main-study pairs have one seed only.

Capability, divergence and over-refusal. Mean capability change of the 42 primary models ranged from −4.2 to +0.9 pp (Figure 8) and teacher-forced divergence from 0.004 to 0.279 nats per token (Figure 9). Capability change was not correlated with the selected dev KL (Spearman ρ=−0.03\rho= -0.03 [−0.29, +0.24], 56 modified models) but was with teacher-forced divergence (ρ=−0.48\rho= -0.48 [−0.66, −0.24]). Over-refusal on the 350 benign prompt fell significantly in 41 of the 42 primary models; the exception was the failed Gemma-4-31B DiM edit.

Figure 8. Capability change after refusal removal. Each cell is the benchmark score of the selected model minus that of its own original, in percentage points (pp); the last column is the unweighted mean of the five benchmarks. Orange marks a loss, blue a gain; the scale saturates at ±8\pm8 pp and the printed values are exact (beyond the scale: Gemma-4-31B SOM MBPP, −8.4-8.4 pp; Gemma-4-31B γ\gamma-MPOA MMLU-Pro, −9.7-9.7 pp; Gemma-4-31B γ\gamma-MPOA IFEval, −10.4-10.4 pp; Gemma-4-31B γ\gamma-MPOA HumanEval, −53.0-53.0 pp; Gemma-4-31B γ\gamma-MPOA MBPP, −33.6-33.6 pp; Gemma-4-31B γ\gamma-MPOA Mean, −21.4-21.4 pp; OLMo-2-32B ARA MBPP, −13.2-13.2 pp; OLMo-2-32B γ\gamma-MPOA HumanEval, −13.4-13.4 pp; OLMo-2-32B γ\gamma-MPOA MBPP, −19.2-19.2 pp; OLMo-2-32B γ\gamma-MPOA Mean, −8.1-8.1 pp). Rows align the same method across the older (left) and newer (right) release of a pair; the 8 public community releases are shown in Figure 13. One item is worth 0.1–0.61 pp depending on the benchmark and the lm-eval standard error of a single score is 0.7–3.9 pp, so differences of 1–2 pp are within noise (single seed).

Figure 9. Divergence of the refusal-removed models from their originals on benign held-out prompts. Each dot is the teacher-forced divergence of one selected model: the token-averaged log⁡porig(y)−log⁡pmod(y)\log p_{\mathrm{orig}}(y)-\log p_{\mathrm{mod}}(y) over the original model’s own greedy answers yy to the 350 benign held-out prompts; 0 is the original model. Panels are pairs; in each row the older release is in the upper lane (hollow marker) and the newer one in the lower lane (filled), joined by a grey line. The Qwen3.5-4B values also contain an export difference: its evaluated checkpoints lack the 15 mtp.* tensors and store 48 F32 linear-attention tensors (A_log, norm.weight) as BF16, which by itself changes the log-probabilities. Single seed; no uncertainty intervals are drawn.

Development set and search. On the dev prompts used for search and selection (Figure 10), dev keyword counts correlated with held-out refusal across the 42 primary models (ρ=0.77\rho= 0.77 [0.61, 0.87]), but the change from older to newer model could differ in sign between the two sets; for ARA it rose by 20.3 dev refusals on average while the pooled held-out ARA effect was small (Appendix § 11.7, Figure 17). The released exports reproduced their selected trials’ dev counts to within two refusals in 37 of 46, and 34 of the 42 core searches reached their selected count by trial 75 (Figure 19).

Figure 10. Refusal–KL trade-off explored by each optimisation run (dev set). Rows are model pairs (older → newer release), columns refusal-removal methods; J-space was run on the small pair only. Each panel holds two Pareto fronts, one per release (one run each: 100 TPE trials, seed 1): the trials that no other trial of that run beats on both dev keyword refusals and first-token KL divergence; dashed with hollow markers for the older release, solid with filled markers for the newer one. The ring marks the trial chosen by the selection rule (fewest dev keyword refusals at KL ≤0.1\le0.1); its count is given at the top right of the panel (older → newer release). Grey stars are the original models (KL = 0). The x axis is logarithmic, with KL = 0 drawn at the left edge behind an axis break; the dotted line is the selection budget KL = 0.1. Every completed trial is shown per run in Figure 11 and Figure 12. Refusal counts are keyword matches on the 100 dev prompts that were also used for optimisation and selection, not held-out refusal rates.

Figure 11. All completed TPE trials of each optimisation run, older releases (dev set). One panel per run (model × method, seed 1), laid out as in Figure 10; the older model of each pair heads its row. Each dot is one completed trial; the line is the run’s Pareto front (dashed, hollow markers), the ring the trial chosen by the selection rule (fewest dev keyword refusals at KL ≤ 0.1; count at the top right of the panel) and the grey star the original model (KL = 0). The x axis is logarithmic, with KL = 0 drawn at the left edge behind an axis break; the dotted line is the selection budget KL = 0.1. 2100 trials from 21 runs; the newer releases are in Figure 12. Refusal counts are keyword matches on the 100 dev prompts that were also used for optimisation and selection, not held-out refusal rates.

Figure 12. All completed TPE trials of each optimisation run, newer releases (dev set). One panel per run (model ×\times method, seed 1), laid out as in Figure 10; the newer model of each pair heads its row. Each dot is one completed trial; the line is the run’s Pareto front (solid, filled markers), the ring the trial chosen by the selection rule (fewest dev keyword refusals at KL ≤0.1\le0.1; count at the top right of the panel) and the grey star the original model (KL = 0). The x axis is logarithmic, with KL = 0 drawn at the left edge behind an axis break; the dotted line is the selection budget KL = 0.1. 2099 trials from 21 runs; the older releases are in Figure 11. Refusal counts are keyword matches on the 100 dev prompts that were also used for optimisation and selection, not held-out refusal rates.

Figure 17. Generation effect on the development set (a) and on held-out prompts (b). One block per pair, one row per condition; hollow = older, filled = newer model; the number right of each panel is the change, newer − older. (a) Dev keyword refusals (of 100) of the trial picked by the selection rule (fewest dev refusals at first-token KL ≤0.1\le0.1, seed 1); the Original row gives the unedited models. These prompts were also used for search and selection, so the counts are optimistically biased and untested; seed-to-seed variation is shown in Figure 16. (b) Held-out refusal judged by WildGuard, paired over the 972 harmful prompts: the H1 measure, with the same values as Figure 3 and Table 6. Bold: significant; grey: not significant. Significance for the methods is Holm-adjusted (exact McNemar, Holm within the 23 H1 tests); significance for the originals and γ\gamma-MPOA is unadjusted (exact McNemar), because they are not H1 tests; † failed edit (held-out refusal not below the model’s own original). γ\gamma-MPOA was run on the Gemma and OLMo pairs only, J-space on the small pair only. Single seed; 95% Wilson intervals for (b) are in Figure 3.

Figure 19. Search-budget convergence of the 100-trial TPE searches (dev set). Top: for each core method with ten runs (seed 1; five pairs, older and newer release), the median over the runs of the best count so far: after trial tt, the fewest dev keyword refusals (of 100) among trials 1 to tt with first-token KL divergence ≤0.1\le0.1 from the original, i.e. the count the selection rule would have returned had the search stopped at trial tt. Dark band: middle 50% of the runs (25th–75th percentile); light band: all runs (minimum to maximum). Curves start at the run’s first trial within the KL budget (trials 5–10 for ARA, 1–2 otherwise); the median and bands start once every run has a value. The grey band marks the 30 random startup trials of the TPE sampler, the dotted line trial 75. Bottom: the trial at which each of the 42 core runs (DiM, MPOA, ARA, SOM and, on the small pair, J-space) first reached the count it finally selected (median 48; 10 runs within the random startup); hollow markers are older, filled markers newer releases, colour and shape the method. The 8 runs (19%) that still improved after trial 75, by 1–14 refusals, show that the selected counts are not converged minima; for the other 34 runs these data cannot tell whether a longer search would have helped. Every run is drawn as its own curve, by method and release, in Figure 14, together with the two J-space runs and the 4 γ\gamma-MPOA runs (Gemma and OLMo pairs only; not counted in the histogram). All counts are keyword matches on the 100 dev prompts that were also used for search and selection, not held-out refusal rates.

Summary of Key Findings

Judged by the class-level per-pair effects, H1 was not supported: after direction-based removal the newer model retained much more refusal in the OLMo pair, somewhat more in the small, Qwen and Gemma pairs and none in the Mistral Small pair, and after ARA more in the Qwen and small pairs and less in the Gemma pair (Table 5). At the point estimate of the run-to-run standard deviation, the only raw per-pair generation effect that excluded zero under both variance models was the OLMo direction-based increase. With run-to-run variance included, the between-pair heterogeneity of the direction-based effect was high on the percentage-point scale under the constant variance model (I2=80%I^2 = 80\%), moderate under the refusal-level model (46%) and small on the log-odds scale (20%). The conjunctive criterion of H1, more refusal on the newer model for every method, was met by no main-study pair; it is a secondary and seed-fragile summary, because the small pair met it with seed 1 but not with seed 2, which erased the ARA generation effect. H2 held per pair in Gemma and OLMo, but each contrast rests on one ARA cell; the Gemma contrast lost its interval support under the refusal-level variance and changed sign with the KL budget on the dev set, and the Qwen contrast, positive in its point estimate and on the public item sets (Table 7), was indeterminate, because its interval with run-to-run variance included zero. H3’s prediction was not supported: keyword matching agreed with the judge on average at the tool’s window but disagreed by several points per model.

Discussion

This study tested whether refusal removal is harder in the newer model of open-weight release pairs (H1), whether the higher-rank, optimisation-based ARA withstands any such change better than low-rank direction-based edits (H2), and whether keyword counts agree with an automatic judge (H3). Removal did not become uniformly harder. The newer model retained much more refusal after direction-based removal in the OLMo pair, somewhat more in the small, Qwen and Gemma pairs and none in the Mistral Small pair. ARA lost less effectiveness than the direction-based edits in two pairs and, in its point estimate, more in one, but these contrasts rest on single ARA runs and were sensitive to the variance model, the divergence budget and the item source. At the tool’s own window, keyword counts matched the judge on average and disagreed with it model by model. The answer to the generational question therefore depended on the release pair, the method, the budget and the measure.

The per-pair effects (Table 5) agree with the ARA author’s account [9] most closely in the OLMo pair: the originals did not differ, the direction-based edits left much more refusal on Olmo-3.1-32B, ARA did not, and, at the point estimate of the run-to-run standard deviation, this was the only raw per-pair increase that exceeded run-to-run variation under both variance models. It also held on the public and the code items (Table 7) and under the second judge. One candidate mechanism, shown in one 1B OLMo-2 model, is that more diverse refusal openings raise the rank of refusal-related activation changes and weaken single-vector ablation [116]; rank was not measured here. The other pairs give weaker or source-dependent evidence. The Gemma direction-based increase came from the author-compiled code items and reversed on the public sets, and in Mistral Small every method’s effect was smaller than the originals’ own difference. These pair-specific patterns resemble non-monotonic black-box safety across Gemma releases [117] and model-specific abliteration susceptibility across 24 open models [24]. To current knowledge they are the first held-out, judged estimates of the generation effect for several removal classes on released model pairs.

Is Abliteration Getting Harder?

The pairs differ in more than release date, so the heterogeneity cannot be attributed to family lineage. Each family contributes one step, and the steps differ: the newer Qwen models interleave linear-attention layers with full attention, Gemma-2 and Gemma-4 and both OLMo models apply post-normalisation after WOW_O and WdownW_{\mathrm{down}}, so that part of an ablated direction reaches the residual stream, and the pairs also differ in depth, attention pattern, multimodality and post-training recipe (Appendix § 9.§ 1). Any of these may change how well a given edit works. Post-training can bypass a behaviour rather than remove it [53, 54], and later layers can compensate for an ablated component [19]; a recipe or depth that does more of either would predict more refusal left after a single-direction edit, but neither was measured here. How strongly the pairs differ also depends on the scale and on the run-to-run variance assumed: with seed-inclusive variances, I2I^2 on the percentage-point scale was 80% under the constant and 46% under the refusal-level variance model, and 20% on the log-odds scale. Four main pairs cannot represent a population of release steps, and they show that a conclusion drawn from one pair does not transfer to another.

H2 compared two edit families, low-rank projections of an estimated direction and rank-50 per-module updates fitted to an objective, which differ in rank, objective, search space and in how often the KL cap bound. The per-pair contrasts therefore cannot show whether refusal is distributed over more dimensions in the newer models; they show which family removed more refusal under this protocol. ARA’s advantage was large where it occurred (Gemma and OLMo). In Qwen the point estimate was reversed, but with run-to-run variance the contrast was indeterminate, and the positive estimate came from the public items. The Gemma contrast lost its interval support under the refusal-level variance model and, on the dev set, changed sign with the budget. Each contrast also rests on one ARA export whose adapter was refitted after the search, and two of them on exports refitted on other hardware or re-scored above the cap. The study is therefore a first, preliminary independent comparison of ARA with direction-based methods across releases; support for the ARA author’s claim [9] is limited to the Gemma and OLMo pairs and is conditional on these single cells. That the newer Qwen model retained more refusal under DiM, MPOA and ARA agrees with a report that Qwen3 resisted abliteration in one setup [20], and the small effect of DiM on Gemma-2-27B agrees with reports that single-direction edits did little on Gemma-2 [118, 119], although rank-1 projection sufficed for Gemma-2-9B in another setup [120].

H3 predicted that keyword matching would undercount refusal; it did not at any window. At Heretic’s 100-token window the two measures agreed on average but disagreed by several points for many single models, in both directions, and over full responses keyword matching counted more refusals because safety and disclaimer markers appear late in long answers that WildGuard labels as compliance. This agrees with reports that disclaimers persist after abliteration [121], that binary refusal metrics miss hedged partial compliance [122], and that substring–judge disagreement grows with intervention strength [123]. Which measure is correct cannot be decided without human labels: an LLM judge over-counted refusal in disclaimer-wrapped answers in one hand audit [123], safety judges can degrade under distribution shift [124], and agreement between judges with correlated errors overstates the evidence [125]. The finding is therefore one of disagreement between two fallible raters: on edited models, the keyword–judge relation changed with the edit, the model and the length of text read, so a keyword-optimised search is steered by a signal whose relation to judged refusal differs between models.

The γ\gamma-MPOA control shows that the post-norm leak matters for MPOA and that it is not constant across releases: the correction lowered refusal on all four post-norm models and more on the newer model of each pair. It did not remove the OLMo MPOA generation effect or the pooled prompt-level H2H2 interaction, but it turned the Gemma MPOA effect strongly negative, at a large capability cost on Gemma-4-31B. DiM and SOM were not corrected, so their generation effects in the Gemma and OLMo pairs may still contain a leak that differs between releases. The control therefore narrows, but does not remove, architecture as an explanation of the post-norm pairs’ results.

“Harder” here means more residual refusal after a 100-trial search at first-token KL ≤0.1\le0.1. Public releases reached much lower refusal than this protocol on two newer checkpoints, at larger generation budget but with small capability changes, so residual refusal under the budget is an upper bound, not a floor, and the protocol’s ranking of checkpoints was not validated against stronger attacks. Tampering evaluations need strong, tuned attacks to approach worst-case behaviour [126, 127].

Capability costs were small for most models, but first-token KL did not bound them: capability change tracked teacher-forced divergence, not the selected first-token KL, and the two largest losses occurred inside the budget. This qualifies the report that refusal ablation preserves response quality [128]. Dev-set keyword counts would also have suggested different conclusions from the held-out judge, as expected when one noisy criterion is both optimised and used for selection [114].

In sum, judged by the class-level per-pair effects, H1 was not supported: the generation effects differed in sign and size between pairs, and the data are consistent with the ARA author’s claim in the OLMo pair and, more weakly, in the Gemma pair, and not in the Qwen and Mistral Small pairs. The conjunctive criterion, more refusal on the newer model for every method, was met by no main-study pair; it is a secondary summary and fragile to the seed, since the small pair met it with one seed but not the other. H2 held per pair in Gemma and OLMo and was indeterminate in Qwen, with the caveats above. H3’s prediction that keyword counts undercount refusal was not supported.

Several design features address likely objections. Comparisons were made within release pairs and against each pair’s own originals, and they are stated about the newer model of a pair, because two-point pairs cannot establish a trend [117]. The prompt-level heterogeneity statistics were complemented by pair-level statistics with run-to-run variance, under two models of that variance. The KL cap did not bind in every direction-based run, so H2 was repeated on the cap-bound units and read at other budgets. Item source, harmful compliance, a second judge, the 44 held-out prompts that overlap the search set and the post-norm leak were checked separately (Appendices § 11.§ 5, § 11.§ 6, § 11.§ 4). Generation used one greedy response per prompt with reasoning disabled, and chain of thought can carry refusal [129], so the conclusions apply to greedy decoding in non-thinking mode.

These results bear on how open-weight safety is audited. Casper et al. [130] list as an open problem which attack configurations are needed to stress-test open-weight safety. Here an audit of the OLMo pair alone would have concluded that the newer release resists direction-based removal, and one of the Mistral Small pair that it does not; within Gemma, the DiM edit of Gemma-4-31B failed (it left refusal at the original’s level), and taken alone would have suggested a resistance that ARA did not show. Its search found no trial that lowered dev refusal below 98% at a first-token KL under 3, thirty times the cap; refusal fell only in trials with larger KL (Appendix § 11.§ 1). Difference-in-means directions have failed in a similar way on Gemma-3, where the failure has been attributed to high-magnitude outlier channels that dominate the estimate [6, 131], and refusal may also occupy more than one direction [8]. Both Gemma models carry outlier post-norm gain channels, more dispersed than in the OLMo models, so these accounts do not by themselves explain why the edit failed on Gemma-4-31B but not on Gemma-2-27B; the cause was not tested. A minimal audit protocol consistent with these results would use at least one direction-based and one higher-rank optimisation-based method, two or more optimisation seeds per model and method, held-out prompts from more than one source scored by at least two automatic judges, results at more than one divergence budget, and a full-sequence divergence and capability check. Auditors should also have humans label a random sample of the responses on which the judges, or a judge and keyword matching, disagree, together with a smaller sample on which they agree; 100 disagreements per release pair would estimate a disagreement error rate near 10% to within about ±6\pm6 points (95%), and 50 agreements would check that agreement is not shared error. The present study made no such human check (§ 5.2). In this study a 100-trial search of a 22–32B model took a median of 18 minutes on one NVIDIA H200 GPU (141 GB) for a direction-based method and 81 minutes for ARA, 18.1 h for the 31 timed main-study searches (Appendix § 9.7), so a second seed roughly doubles this search time, before the evaluation of the added models. Such an audit would still not cover sampled decoding, reasoning modes, or fine-tuning and reinforcement-learning attacks [3, 132]. Defences built around the refusal direction [16, 17, 57] should also be tested against optimisation-based removal. For tool developers, keyword counts and first-token KL are insufficient evidence of success, and keyword counts should be read at the window used for selection.

Limitations and threats to validity

One limitation is that each main-study cell was optimised with a single seed (seed 1), although TPE search is stochastic. On the small pair, a second seed changed held-out refusal in all 10 cells (median 9.4 pp, maximum 25.2 pp; run-to-run standard deviation 9.2 pp [6.4, 16.1]) and erased the ARA generation effect (+14.2 pp with seed 1, +0.0 pp with seed 2). Variance of this size could create or mask a generation effect in a single main-study cell, as seed and configuration changes have altered conclusions about algorithm comparisons [133] and safeguard durability [69]. The band used to flag main-study effects as within seed noise (Table 6) assumes that this variance, measured on 4B and 7B models, transfers to the 22–32B main-study models; this cannot be tested with one seed per cell. Whether it depends on the refusal level could not be decided from ten seed pairs, and the per-pair verdicts differed between a constant and a refusal-level model (Appendix § 11.8): at the point estimate of the run-to-run standard deviation, only the OLMo direction-based increase exceeded run-to-run variation under both, and it did not at the upper limit of the refusal-level model. Every main-study run also used the same seed, so runs with the same search space shared their random start-up trials. However, full Pareto fronts and four KL thresholds are reported, method rankings were more stable than single-cell generation effects (method accounted for η2=0.94\eta^2 = 0.94 and 0.88 of the variance among the small-pair cell rates), the pooled estimates average run-to-run variation over the 32 main-study cells (24 direction-based, only 8 for ARA), the per-pair intervals of § 4.1 add the estimated run-to-run variance to prompt sampling (the pair-level pooled intervals barely change when it is added), and the 11 main-study effects within the band are flagged as provisional. Three of the ten seed-2 runs also used a smaller generation batch size after running out of memory; without them the run-to-run standard deviation was 7.2 pp instead of 9.2 pp, and the same five raw effects exceeded the band. A second seed for the 32 main-study cells, with distinct seeds per model, which would replace the transferred variance with a direct estimate, is left to future work. The re-run rule was not uniform: an interrupted Qwen3.8-27B ARA study was repeated from scratch, whereas the Qwen3.8-27B MPOA study was resumed (Appendix § 9.§ 3). Its selected trial is a seeded start-up draw that an uninterrupted study would also have evaluated, so the resumption can only have raised its dev refusal rate, which would favour finding the newer Qwen model harder for MPOA; the one study run both ways (ARA) differed by 4 of 100 dev refusals. Likewise, for γ\gamma-MPOA the post-norm component along r^\hat{r} was not measured in forward passes of the selected models: only the weight identity b⊤W′=(1−λ)b⊤W\mathbf{b}^{\top}\mathbf{W}' = (1-\lambda)\mathbf{b}^{\top}\mathbf{W} was checked (Appendix § 9.§ 2).

A second limitation is that each held-out prompt received one greedy response of at most 512 tokens, with reasoning disabled although the chat templates of Qwen3.8-27B and Qwen3.5-4B open a reasoning block. Sampled decoding can change refusal rates substantially [86], and chain of thought can carry refusal [129], possibly by different amounts in each generation. However, decoding was identical for every model, so within-pair comparisons are matched on it, and the conclusions are stated for greedy decoding in non-thinking mode.

The ARA cells carry limitations of their own. The released ARA model is rebuilt by refitting its adapter, and the four ARA exports refitted on a different GPU from the search drifted by 5 to 9 dev refusals on re-score, whereas those refitted on the search machine drifted by at most 2; the Gemma-4-31B ARA export re-scored at a first-token KL of 0.107, above the cap; and the 0 of 100 dev refusals reported by heretic-org on its model card (an unreviewed report) for an ARA model of Qwen3.8-27B with the same defaults was not reproduced (59 and 63 of 100 in two runs here). Held-out results describe the released models, so these points concern the link between search and release rather than the reported rates, but they bear directly on H2, whose reversed Qwen estimate and Gemma support both rest on single ARA cells. The per-pair H2 verdicts are therefore preliminary. Running heretic-org’s reported trial through this pipeline, refitting the ARA exports on the search hardware and a second ARA seed for the Qwen and Gemma pairs are left to future work.

A further limitation is that trials are optimised and selected on the 100 development prompts that produce Heretic’s refusal count, which relies on 33 keyword markers with a reported recall of about 0.48, against 0.77 for an LLM judge (Heretic #419, an unreviewed practitioner report). Development-set refusal is therefore optimistic [114], and the best dev method was the best held-out method in only 7 of 12 model ×\times seed sets. However, keyword counts drive optimisation and selection only: reported refusal comes from a frozen held-out set of 1,322 prompts (Table 3), labelled by WildGuard as primary judge with Qwen3Guard as a second judge (κ=0.82\kappa= 0.82 [0.80, 0.83]) on the 42 ablated models. A related limitation is that no human audit was made. Every refusal rate is therefore relative to the automatic judges, WildGuard has not been validated on the outputs of edited models, and where keyword matching and the judges disagreed (H3) neither can be called correct; agreement between LLM judges with correlated errors overstates the evidence [125]. WildGuard and both OLMo releases come from the same developer, whose open supervised post-training data for both releases include about 50,000 prompts from WildGuardMix, the data set released with WildGuard and used to train it6, so its labels of OLMo outputs may share their provenance; under the second judge, Qwen3Guard, the OLMo direction-based generation effect was +32.3 pp (WildGuard +36.9 pp) and the OLMo H2 contrast −35.0 pp (−39.3 pp). The keyword count that drives search and selection may also be a less valid signal on one generation than on the other, which would bias the selected models differently; dev and held-out changes had opposite signs in several cells (Appendix § 11.7), and re-selecting trials with a judge would test this.

Another limitation is that generation is confounded with size, architecture, training data, post-training recipe and release interval, and later checkpoints may have been safety-tuned on the public held-out benchmarks [134]. In particular, post-norm models (Gemma-2/4, OLMo-2/Olmo-3.1) apply RMSNorm [75] after WOW_O and WdownW_{\mathrm{down}}, so the normalised output retains a component along the ablated direction, handicapping direction-based generation—the source of the eight main-study models. Because both generations of each post-norm pair share this design [72, 76, 73, 74], the leak confounds H1 and H2 only to the extent that its size differs between generations. However, comparisons are made within family and against each pair’s own originals, with family as a random effect in the planned model [104]; the γ\gamma-corrected MPOA control on the four post-norm models lowered held-out refusal on all four, by more on the newer model of each pair, but left the OLMo generation effect (+57.0 pp) and the pooled H2 interaction (−0.42) in place; DiM and SOM were not corrected; and conclusions are stated about later releases, not about mechanisms such as safety training. The 200 code-as-weapon items were compiled by the author and carried about half of the originals’ generation difference; some per-pair verdicts, notably Gemma’s direction-based increase, depended on them (Table 7).

A final limitation concerns the scope of two implementations. J-space ablation is limited to the small pair, because instruction-tuned Jacobian lenses exist only for Qwen3.5-4B and Qwen2.5-7B-Instruct; the lens is fitted on non-chat text, and the direction is baked into the weights rather than applied as the reference forward hook, with unknown direction of bias; J-space as a forward hook (α=1\alpha= 1) was planned on the small pair but not run and is left to future work. Heretic-tuned SOM deviates from Piras et al. [7] in neuron ranking, orthonormalisation, decay, layer scope and adapter rank, and its narrower search may understate SOM; a paper-faithful SOM configuration was planned but not run and is left to future work. However, both are evaluated, and labelled, as Heretic variants (Heretic-tuned SOM; weight-baked J-space), so the conclusions about them are restricted to these variants. All threats, including those not discussed here (tokenisation, truncation, decoding, first-token KL, benchmark exposure, benchmark coverage, judge validity, preregistration, generalisation, and dual use), are detailed in Appendix § 10.

Conclusion

This study compared direction-based and optimisation-based refusal removal on five open-weight release pairs under one nominal search budget, KL cap and selection rule, with held-out refusal labelled by automatic judges. Measured as residual refusal after a 100-trial search at first-token KL divergence ≤0.1\le0.1, refusal removal did not become uniformly harder in the later releases. The generation effect differed in sign and size between pairs; at the point estimate of the run-to-run standard deviation, only the OLMo pair’s direction-based increase exceeded run-to-run variation under both variance models considered, and some per-pair verdicts depended on the source of the test prompts. In a preliminary comparison resting on one ARA run per model, ARA lost less effectiveness than the direction-based methods in two pairs and, in its point estimate, more in one, and these verdicts changed with the variance model and the divergence budget. At the tool’s 100-token window, keyword counts matched judged refusal on average but not model by model, and first-token KL did not bound capability loss. Claims that a newer model resists refusal removal, or that a removal tool succeeds, therefore need several release pairs analysed as pairs, the unedited models as reference, several method classes, held-out judged evaluation, more than one divergence budget, a full-sequence measure of collateral change and several optimisation seeds per model; the present study met the last requirement only on the small pair.

Four directions follow. First, a second optimisation seed for the 32 main-study cells would replace the transferred run-to-run variance with a direct estimate. Second, running heretic-org’s reported Qwen3.8-27B ARA trial through this pipeline, and refitting the ARA exports on the search hardware, would show whether the reversed Qwen estimate and the Gemma-4-31B result depend on the search or on the release step. Third, a human-labelled audit of the responses on which keyword matching and the judges disagree would show which measure is closer to human judgement on edited models [135]. Fourth, judge labels for non-selected trials would allow Pareto fronts to be re-selected with a judged objective and a full-sequence divergence constraint. Until such checks are routine, a model’s apparent resistance to refusal removal should be read as a statement about one release pair, one method, one budget and one measure, not about its generation.

Ethics Statement

The study involved no human participants and no personal data and was not human-subjects research, so no IRB review was required. It creates and releases language models whose refusal behaviour has been removed, which is dual-use. The models are released openly with two exceptions. The five Gemma-2-27B variants are withheld (their repositories are private), because the Gemma Prohibited Use Policy may forbid redistributing them (see below). The five OLMo-2-32B variants (DiM, MPOA, SOM, ARA and γ\gamma-MPOA) are gated on Hugging Face with automatic approval, because OLMo-2-32B is the only base model studied for which no third-party public abliteration was found. For the other nine bases (the Gemma-2-27B variants among them are withheld regardless), refusal-removed derivatives were already public: for five of them, public releases were evaluated here (§ 4.4), and for the other four (Gemma-2-27B, Mistral-Small-2409, Qwen2.5-7B and Qwen3.5-4B), third-party abliterated or uncensored derivatives were found on Hugging Face7 whose residual refusal was not evaluated. The argument is therefore that comparable models are already available, not that they are equally effective. The release serves two purposes that the configurations and journals alone do not: it lets others re-run the audit on the evaluated weights, which matters because re-exports from stored configurations are not exactly reproducible (the ARA refits drifted when rebuilt on other hardware; Appendix § 11.§ 3), and it provides fixed edited models against which defences and detectors can be evaluated. Responses to harmful prompts are stored privately for scoring and are not released. Released models (all except the five withheld Gemma-2-27B variants) keep the licence of their base model: Apache-2.0 (the Qwen, OLMo and Mistral-Small-3.2 bases and Gemma-4-31B), to which the use restrictions of the OpenRAIL++ licence are added; or, for Mistral-Small-2409, the Mistral Research Licence, which permits non-commercial research use. Apache-2.0 and the Mistral Research Licence permit redistributing such derivatives, the latter for research purposes only. The Gemma Prohibited Use Policy, which the Gemma Terms incorporate, forbids “attempts to override or circumvent safety filters”, so this study cannot state that redistributing the refusal-removed Gemma-2-27B derivatives is permitted, and they are not released (Appendix § 9.§ 5). The automatic-approval gate on the OLMo-2-32B variants requires a logged-in Hugging Face account and shares the requester’s contact details with the author, so access is recorded; it does not limit who can obtain the weights. A licence states the permitted use; it is not a technical control.

Data and Code Availability

The 42 primary models and the four γ\gamma-MPOA control models are on Hugging Face under https://huggingface.co/ richardyoung (<base>-<METHOD>-heretic); the five Gemma-2-27B variants are withheld (private repositories), the five OLMo-2-32B variants are gated with automatic approval, and the others are public. The release of code and data consists of the code (the Heretic fork branches paper33-plugins and paper33-gamma, the evaluation harness and the analysis scripts), the configuration of every run, the Optuna study journals and run logs, and the judged labels of every held-out response (judge, keyword and degeneracy labels per prompt and model, without the response text). The held-out prompts come from the public sets of Table 3. The analysis code, harness, configurations, journals, logs and labels are at https://github.com/ricyoung/abliteration-studies, and the fork branches at https://github.com/ricyoung/heretic. The Heretic fork is released under AGPL-3.0, the licence of upstream Heretic; the other code and the data are released under OpenRAIL++.

Method Definitions

This appendix gives the notation, the update rule and the rationale of each refusal-removal method reviewed in § 2, together with the strengths and weaknesses of each and the way it is instantiated in this study.

Notation. A decoder-only transformer maintains a residual stream x(ℓ)∈Rdx^{(\ell)} \in\mathbb{R}^d at each layer ℓ\ell. Each layer reads from the stream and writes back to it through two matrices: the attention output projection WO(ℓ)W_{O}^{(\ell)} and the MLP down-projection Wdown(ℓ)W_{\mathrm{down}}^{(\ell)}. Let DharmD_{\mathrm{harm}} and DsafeD_{\mathrm{safe}} be sets of harmful and harmless prompts, and let μharm(ℓ)\mu_{\mathrm{harm}}^{(\ell)} and μsafe(ℓ)\mu_{\mathrm{safe}}^{(\ell)} be the mean residual activations at the last prompt token over each set.

Methods are grouped by what they assume about refusal: (i) it is a single linear direction, (ii) it is a low-dimensional subspace or cone, (iii) it is whatever transformation of each module’s outputs separates harmful from harmless inputs (no geometric assumption), or (iv) it is best located in a different representation space altogether. Table 1 (§ 2) summarises the evaluated methods as instantiated in this study (§ 3.2).

Single-direction methods

Classic abliteration (difference-in-means)

Mechanism. Compute the refusal direction r^(ℓ)=normalize⁡(μharm(ℓ)−μsafe(ℓ))\hat{r}^{(\ell)} = \operatorname{normalize}(\mu_{\mathrm{harm}}^{(\ell)} - \mu_{\mathrm{safe}}^{(\ell)}), pick one layer’s direction r^\hat{r}, and orthogonalise every matrix that writes to the residual stream against it: W←W−λr^r^TWW \leftarrow W - \lambda\hat{r}\hat{r}^{\mathsf{T}}W [1]. With λ=1\lambda= 1 the model can no longer write any component along r^\hat{r}.

Why it works. Under the linear representation hypothesis [33], high-level features correspond to directions. Arditi et al. [1] showed causally that adding r^\hat{r} induces refusal on harmless prompts and ablating it suppresses refusal on harmful ones, across 13 chat models. Because the edit is a rank-one projection, most of the weight matrix is untouched.

Why it exists. It turned an interpretability finding into a practical, fine-tuning-free procedure that needs only forward passes on a few hundred prompts [2, 32].

Pros. Simple, fast (minutes), interpretable, rank-one, few hyperparameters (layer, strength). Cons. Assumes refusal is one-dimensional; the direction also carries part of the harmless mean, so removing it damages unrelated behaviour (refusal is better described as an affine function whose offset plain ablation ignores; on a recurrent RWKV model plain ablation produced nonsense where affine concept editing stayed coherent [34]); changing row norms shifts activation scales; quality depends heavily on the chosen layer and strength, which are usually set by hand (in an earlier cross-tool comparison, GSM8K accuracy changed by between +1.51 and −18.81 percentage points [21]). DECCP, ErisForge and FailSpy’s tool are implementations of this recipe with different layer-selection heuristics. In this study diff-in-means runs inside Heretic with the same automated search over direction, strength and layer profile as MPOA (“Heretic-tuned DiM”, § 3.2).

Projected and magnitude-preserving abliteration (MPOA)

Mechanism. Two refinements of §A.1.1. (1) Projection: remove from r^\hat{r} its component along the normalised harmless mean, so that only the part of the refusal direction orthogonal to ordinary activations is ablated [35]. (2) Magnitude preservation: row-normalise WW, apply the ablation, then restore each row’s original norm [6]. Heretic adds a search over a per-layer weight kernel (peak strength, peak position, minimum strength, falloff distance) for attention and MLP separately, global versus per-layer directions, and a fractional direction index interpolated between layers, all tuned by multivariate TPE [36, 5]. The edit is stored as a low-rank LoRA adapter [84].

Why it works. Projection reduces collateral damage: the part of r^\hat{r} shared with harmless activations is exactly what is needed for normal behaviour. Norm preservation keeps downstream layer norms and activation magnitudes in the range the model was trained in, so edits change direction rather than scale.

Why it exists. Plain abliteration measurably degraded capability (for example GSM8K drops of several points in an earlier comparison [21]); these refinements recover much of that loss.

Pros. Better capability retention at the same refusal rate; automated hyperparameter search removes manual tuning. Cons. Still assumes a single direction per layer; norm preservation is only approximate when stored as a low-rank adapter; the search adds many evaluation passes (one per trial).

Null-space-constrained abliteration (jwest33)

Mechanism. Compute a projected, norm-rescaled diff-in-means edit ΔW\Delta W as above, then restrict it to the null space of activations on a set of preservation prompts: with VV the top singular vectors of those activations, apply ΔW(I−VVT)\Delta W(I - VV^{\mathsf{T}}) (the AlphaEdit constraint [38]). Optional Gaussian layer weighting spreads the edit around a chosen depth.

Why it works. If an input lies in the span of VV, the constrained edit has no effect on it, so behaviour on preserved inputs is guaranteed unchanged to first order; the edit can only act on directions that harmless inputs do not use.

Why it exists. It brings guarantees from knowledge editing [39, 38] to refusal removal, as an alternative to tuning strength against a KL budget.

Pros. Principled protection of chosen behaviour; single pass, no search. Cons. If harmful and harmless activations share directions (likely in newer models), the constraint also blocks the refusal edit; the rank ratio is a new hyperparameter; dense models only in the current implementation. It is not evaluated in this study (§A.5).

Multi-direction and subspace methods

SVD multi-direction (Gabliteration, OBLITERATUS presets)

Mechanism. Instead of the mean difference, take the top-kk singular vectors of the matrix of per-prompt differences between harmful and harmless activations, and project all kk out, typically with ridge regularisation and selected layers [42]. OBLITERATUS packages this (“advanced”, 4 directions), whitened and iterative variants (“aggressive”), frequency-band weighting of layers via a DCT (“spectral cascade”), and a re-implementation of gradient-refined directions (“rdo”) in one toolkit.

Why it works. When refusal is spread over several correlated directions, one mean difference captures only the dominant one; the remaining components can re-trigger refusal. Removing a subspace closes those alternative routes.

Why it exists. Single-direction ablation leaves residual refusals on some prompt categories and some models; multi-direction methods target that residual.

Pros. Removes more refusal on models where one direction is insufficient; still closed-form and fast. Cons. Every extra direction removed is extra capacity destroyed; choosing kk trades refusal against damage; several presets are heuristic combinations without a clear theoretical justification. The OBLITERATUS v0.1.4 presets advanced and spectral_cascade, together with basic, are not evaluated in this study; they were run on Qwen2.5-7B-Instruct and scored with this study’s harness in the companion paper [70]. Version 0.1.4 lists a som preset but does not ship its direction module and does not run on Qwen3.5-4B (Appendix § 9.§ 2).

Refusal cones (RDO)

Mechanism. Learn refusal directions by gradient descent on an objective that requires ablation to remove refusal and addition to induce it, while keeping other behaviour fixed; multiple such directions form a polyhedral concept cone [8].

Why it works / exists. It showed that refusal is not a single direction but a multi-dimensional cone of causally relevant, representationally independent directions, and that directions found by optimisation outperform diff-in-means. Pros. Directly targets the causal effect rather than a statistical proxy. Cons. Requires gradients and more compute. It is not evaluated in this study (§A.5).

Self-organising-map directions (SOM)

Mechanism. Fit a small self-organising map (e.g. 4×44 \times4) to the harmful activations at each layer, take kk map neurons as prototypes of distinct “kinds” of harmful input, and use each prototype minus the harmless mean as a refusal direction; ablate all kk, each with its own strength [7]. The proof-of-concept Heretic integration (PR #196, an unreviewed practitioner report) tunes the per-direction strengths with TPE.

Why it works. Different harm categories (weapons, self-harm, cyber, …) may trigger refusal through different directions; a single mean averages them and represents none well. The SOM finds the clusters without labels. Piras et al. Piras et al. [7] show diff-in-means is a special case.

Pros. Strong reported results (3/100 refusals at KL ≈0.1\approx0.1 on gpt-oss-20b, PR #196, unreviewed); principled multi-direction choice. Cons. More hyperparameters (map size, kk, per-direction strengths) make TPE converge slowly; the PoC’s top-kk selection counts winners over the map’s own weights instead of the data. In this study SOM is ported to Heretic 2.0 as a modifier (“Heretic-tuned SOM”) that ranks neurons by wins over the harmful residuals, orthonormalises the kk directions and replaces the per-direction strengths with one searched decay; these and the other deviations from Piras et al. [7] are listed in §B.8.

Optimisation-based methods

Arbitrary-Rank Ablation (ARA)

Mechanism. ARA makes no assumption about the geometry of refusal [9]. For each targeted linear module WW in a layer range (in Heretic, the attention output projection and the MLP down-projection), it records the module’s inputs XX and outputs Y=XW⊤Y = XW^\top on harmless (gg) and harmful (bb) prompts, then learns a low-rank update ΔW=BA\Delta W = BA (rank 50 by default) minimising

L=wp∥Y~g−Yg∥2+ws[dk(Y~b,Yg)−wodk(Y~b,Yb)],(1)\mathcal{L} = w_p \lVert\widetilde{Y}_g - Y_g \rVert^2 + w_s \left[d_k(\widetilde{Y}_b,Y_g) - w_o d_k(\widetilde{Y}_b,Y_b)\right], \tag*{(1)}

where Y~\widetilde{Y} is the output of W+BAW + BA with each row rescaled to the norm of the corresponding row of WW, and dk(⋅,⋅)d_k(\cdot,\cdot) is the mean distance to the kk nearest neighbours. The rescaling enters only the objective; the exported adapter BABA is not renormalised (§ 3.2). The three terms keep harmless outputs fixed, pull harmful outputs toward harmless ones, and push them away from where they were (“overcorrection”). Each module is solved independently with L-BFGS; Heretic’s TPE searches the layer range, the three weights and kk.

Why it works. Rather than guessing the shape of the refusal representation, ARA directly asks each module to make harmful inputs produce the outputs harmless inputs produce. Any refusal mechanism that a module’s output carries, whether linear, multi-directional or non-linear across prompts, is in scope, up to the adapter’s rank. The objective is close to convex and starts at its optimum for the preservation term, so L-BFGS converges in a few iterations.

Why it exists. Its author reports that directional ablation increasingly fails on recent models, and that ARA reaches lower refusal at lower KL on gpt-oss-20b than any directional result [9].

Pros. No geometric assumption; fewer search parameters than per-component directional kernels; memory similar to abliteration because modules are solved one at a time. Cons. Slower per trial; rank-50 adapters per module are larger artefacts; the nearest-neighbour objective depends on the prompt sets, so it may overfit them; no published analysis yet (preprint forthcoming).

Methods in other representation spaces

Jacobian-space (J-space) ablation

Mechanism. Using a Jacobian lens, map each layer’s activations into a “workspace” where features are read out by the model’s later computation; compute the refusal direction there and pull it back to the residual stream through the layer Jacobian, dℓ=Jℓ⊤(⋅)d_\ell= J_\ell^\top(\cdot); ablate it with a forward hook rather than a weight edit [48].

Why it exists. To localise the part of the refusal signal the model actually uses downstream, rather than everything correlated with harmfulness. Pros. Very targeted (small distribution shift). Cons. By its authors’ own measurements removes little refusal (AdvBench refusal 0.78 versus 0.06 for diff-in-means on Qwen3.5-4B); the reference code is hook-based, so not directly an exportable model, and is a single-model implementation. In this study the J-space directions are instead baked into the weights with Heretic’s directional-ablation edit and search (full row normalisation, no harmless-mean projection), which yields an exportable model (§ 3.2); the reference forward hook is planned as a further condition. Because, among the candidate models, instruction-tuned Jacobian lenses (neuronpedia/jacobian-lens) exist only for the two small-pair models, J-space is run on the small pair only.

Methods not evaluated

Null-space-constrained abliteration (§A.1.3) and refusal cones (§A.2.2) are described above for completeness but are not part of the final design. A code audit before the main runs found that the available null-space implementation (jwest33/abliterator) applies the constraint on the output side, where it gives no preservation guarantee, and edits square attention output projections, as in Qwen2.5-32B, OLMo-2-32B, Olmo-3.1-32B and Mistral-Small-2409, in the wrong space; the revised plan therefore dropped it. The only RDO implementation considered, the rdo preset of OBLITERATUS, is a probe-based proxy rather than the gradient-based optimisation of Wollschläger et al. [8], and it is not among the OBLITERATUS presets evaluated in the companion paper [70]. Other diff-in-means implementations (DECCP, ErisForge, FailSpy’s tool) are not run separately; Heretic-tuned DiM represents this recipe [21]. Optimal-transport ablation maps the harmful activation distribution onto the harmless one with a closed-form Gaussian transport map at one or two layers and reports up to 11% higher attack success than baselines on six 7–32B models [47]; no public code was found. Weighted refusal direction optimisation (WRDO) learns weights over a bank of per-prompt refusal directions and was benchmarked against basic refusal-direction ablation, RDO, RDO cones and Heretic on Qwen3-8B-Base and Olmo 3 7B [45]; it is not evaluated. Circuit-breaker training [59] and extended-refusal fine-tuning [55] are defences rather than removal methods and are discussed in § 2.

Experimental Details

This appendix gives the implementation detail behind the Methods (§ 3), in the same order: models, method instantiation and search spaces, optimisation and prompt splits, held-out evaluation, export and release, multiple-testing families, compute and software, and deviations from the study plan.

Models

Except for the Mistral-Small-3.2 mirror, Heretic was not given an explicit revision; it used the locally cached snapshot, whose commit (Table 2) equals the commit recorded by the evaluation harness wherever both exist. The small pair (Qwen2.5-7B-Instruct and Qwen3.5-4B) fits a 24 GB consumer GPU.

Three architectural properties matter. First, Gemma-2, Gemma-4, OLMo-2 and Olmo-3.1 apply an RMSNorm [75] with elementwise gain γ\gamma to the outputs of WOW_{O} and WdownW_{\mathrm{down}} before they enter the residual stream (“post-norm”; output normalisation in the terminology of Kim et al. [76], distinct from classic Post-LN, which normalises after the residual addition). Gemma 2 normalises both the input and the output of each sublayer [72], whereas OLMo 2 and Olmo 3 normalise outputs only [73, 74]. Orthogonalising WW against r^\hat{r} then removes r^\hat{r} only from the pre-norm output uu; the residual stream still receives (γ⊙r^)⊤u/rms⁡(u)(\gamma\odot\hat{r})^{\top}u/\operatorname{rms}(u) along r^\hat{r}, which vanishes only for constant γ\gamma. Direction-based edits are thus incomplete on four of the eight main-study models; this potential confound is disclosed rather than corrected in the primary runs, and a γ\gamma-corrected MPOA control was run on all four post-norm models (§B.2; results in § 4.4). Second, Qwen3.8-27B and Qwen3.5-4B use the Transformers qwen3_5 model class, which interleaves gated-delta-rule linear-attention layers [77] with full-attention layers (three to one in Qwen3.5-4B); on linear-attention layers Heretic edits the linear-attention output projection in place of WOW_{O}. Third, Qwen3.8-27B, Gemma-4-31B, Mistral-Small-3.2 and Qwen3.5-4B are image-text checkpoints used with text only, and the Gemma-2 chat template has no system role, so the system prompt is prepended to the user turn (in Heretic and in the evaluation harness).

Method Instantiation and Search Spaces

All methods run inside Heretic 2.0 [5], so prompt sets, scorers, trial budget and selection are shared, while the weight edit and its search space differ (Table 11). In every layer Heretic targets the attention output projection and the MLP down-projection. During optimisation each edit is held in a PEFT LoRA adapter (ΔW=BA\Delta W = BA, with LoRA α\alpha equal to the rank so that the scaling α/r\alpha/r is 1 and the adapter acts at full strength; no dropout); adapters are reset to zero before every trial and the base weights are not modified. Models were loaded unquantised in bf16; activation winsorisation was off. Built-in methods on the GPU boxes ran from upstream Heretic commit 0db5c31; SOM, J-space and all small-pair runs ran from the fork branch paper33-plugins at 8fd4d76, which only adds the SOM (commit 45a4e6e) and PrecomputedDirections modifiers, SOM tests and a SOM section of the configuration template to 0db5c31, leaving the built-in modifiers identical. The γ\gamma-MPOA control ran from a third fork branch, paper33-gamma at fdef54b, which adds only the γ\gamma-MPOA modifier to 8fd4d76. On the GPU boxes the fork code was loaded through PYTHONPATH into the 0db5c31 environment, so the run records there name 0db5c31; the fork commits are recorded in the environment captures. Configurations are configs/core_{dim,mpoa,ara,som,jspace,gmpoa}.toml; Table 11 lists the search spaces.

Table 11: Search spaces sampled by multivariate TPE. LL = number of layers; intervals are closed; “→0\to0” marks a negative lower bound whose draws are clamped to 0, giving positive probability to switching the term off. Kernel parameters are sampled separately for the attention and MLP components (10 parameters for DiM, MPOA and J-space, 11 for SOM, 6 for ARA).

ParameterRangeMethods
Direction scope{global, per layer}DiM, MPOA, J-space, SOM
Direction index (global scope)[0.4(L−1),0.9(L−1)][0.4(L - 1), 0.9(L - 1)], continuous, interpolatedDiM, MPOA, J-space
integers in [round⁡(0.4(L−1)),round⁡(0.9(L−1))][\operatorname{round}(0.4(L - 1)), \operatorname{round}(0.9(L - 1))]SOM
Direction decay δ\delta[−0.2,1.0][-0.2, 1.0], →0\to0SOM
Max. weight, attention[0.8,1.5][0.8, 1.5]DiM, MPOA, J-space, SOM
Max. weight, MLP[−0.25,1.5][-0.25, 1.5], →0\to0DiM, MPOA, J-space, SOM
Max.-weight position (layer)[0.6(L−1),L−1][0.6(L - 1), L - 1]DiM, MPOA, J-space, SOM
Min. weight (fraction of max.)[0,1][0, 1]DiM, MPOA, J-space, SOM
Min.-weight distance (layers)[1,max⁡(0.6(L−1),1)][1, \max(0.6(L - 1), 1)]DiM, MPOA, J-space, SOM
Start layerintegers in [0,⌊L/2⌋][0, \lfloor L/2 \rfloor]ARA
End layer (exclusive)integers in [⌊L/2⌋+1,L][\lfloor L/2 \rfloor+ 1, L]ARA
wpw_p (preserve harmless outputs)[0,1][0, 1]ARA
wsw_s (steer harmful outputs)[10−4,1][10^{-4}, 1], log scaleARA
wow_o (overcorrection, relative)[0,1.3][0, 1.3]ARA
kk (nearest neighbours)integers in [1,15][1, 15]ARA

Table 11.

DiM. The refusal direction of layer ℓ\ell is the normalised difference of harmful and harmless mean residuals at the last prompt position [1], removed by the rank-1 edit ΔW=−λr^(r^⊤W)\Delta W=-\lambda\hat{r}(\hat{r}^{\top}W) with neither projection nor row normalisation. Unlike the original procedure (§A.1.1), only WOW_{O} and WdownW_{\mathrm{down}} are edited, the embedding matrix is not, and direction, strength and layer profile are tuned by the same search as MPOA (“Heretic-tuned DiM”). The strength λ\lambda follows a per-component kernel: the maximum weight at the maximum-weight position, decreasing linearly to the minimum weight at the minimum-weight distance, and zero beyond. A global direction is interpolated between the two layers adjacent to a fractional index; the per-layer scope uses each layer’s own direction.

MPOA. Heretic’s defaults: the direction is projected orthogonal to the normalised harmless mean [35], and the edit is computed on row-normalised weights whose row norms are then restored [6], approximated by a rank-3 adapter from a randomised SVD reseeded with the run seed.

γ-MPOA. In a post-norm block the residual stream receives y=Γo/rms⁡(o)\mathbf{y} = \mathbf{\Gamma}\mathbf{o}/\operatorname{rms}(\mathbf{o}), with o=Wz\mathbf{o} = W\mathbf{z} the writer’s output and Γ=diag⁡(γ)\mathbf{\Gamma} = \operatorname{diag}(\boldsymbol{\gamma}) the gain of the following RMSNorm, so its component along the refusal direction r^\hat{\mathbf{r}} is (Γr^)⊤o/rms⁡(o)(\mathbf{\Gamma}\hat{\mathbf{r}})^{\top}\mathbf{o}/\operatorname{rms}(\mathbf{o}). γ\gamma-MPOA replaces MPOA’s projection by the oblique projection

W′=W−λab⊤W,b=Γr^,b⊤a=1,W' = W - \lambda\mathbf{a}\mathbf{b}^{\top}W,\qquad\mathbf{b} = \mathbf{\Gamma}\hat{\mathbf{r}},\qquad\mathbf{b}^{\top}\mathbf{a} = 1,

so that b⊤W′=(1−λ)b⊤W\mathbf{b}^{\top}W' = (1-\lambda)\mathbf{b}^{\top}W and the write along r^\hat{\mathbf{r}} vanishes at λ=1\lambda= 1, the per-token rms⁡(o)\operatorname{rms}(\mathbf{o}) being a scalar; λ\lambda is MPOA’s per-layer weight from the search. The removal direction is a ridge-regularised inverse, a∝(Γ2+δ2I)−1Γr^\mathbf{a} \propto(\mathbf{\Gamma}^{2} + \delta^{2}I)^{-1}\mathbf{\Gamma}\hat{\mathbf{r}} with δ=0.1rms⁡(γ)\delta= 0.1\operatorname{rms}(\boldsymbol{\gamma}) of the writer’s gain, because many gains are near zero. MPOA’s row-norm restoration is then applied but confined to the kernel of b⊤\mathbf{b}^{\top}, so that it cannot reintroduce a component along r^\hat{\mathbf{r}}, and the edit is exported as the same rank-3 adapter (one exact rank-1 term plus a rank-2 randomised-SVD approximation of the restoration, projected onto that kernel). The gains are read from the RMSNorm after WOW_O and WdownW_{\mathrm{down}}: 1+w1+w for Gemma-2 (and Gemma-3), whose RMSNorm stores an offset weight ww, and the raw weight ww for Gemma-4, OLMo-2 and Olmo-3. Each gain vector is checked against the module’s own forward pass, and the fork’s unit tests check that the deployed weight satisfies b⊤W′=(1−λ)b⊤W\mathbf{b}^{\top}W' = (1-\lambda)\mathbf{b}^{\top}W to floating-point precision. On pre-norm models the modifier reduces to MPOA. The post-norm component along r^\hat{\mathbf{r}} was not measured in forward passes of the selected models.

SOM. A port of the proof of concept in Heretic PR #196 [7]. Per layer, a 4×44 \times4 hexagonal self-organising map is fitted to the per-prompt harmful residuals (10,000 online iterations, learning rate 0.01 and neighbourhood radius 0.5, both decayed as 1/(1+2t/T)1/(1+2t/T); seeded with the run seed). The k=4k=4 neurons that win the most harmful residuals yield directions normalize⁡(prototype⁡−μsafe)\operatorname{normalize}(\operatorname{prototype}-\mu_{\mathrm{safe}}), which are projected orthogonal to the harmless mean and Gram–Schmidt orthonormalised in rank order. Direction ii is ablated with strength λ(ℓ)δi\lambda(\ell)\delta^{i} (λ\lambda as above, δ\delta a searched decay: δ=0\delta=0 is single-direction ablation, δ=1\delta=1 removes the kk-dimensional subspace), with full row normalisation in a rank-6 adapter. A global scope uses one integer layer. Deviations from Piras et al. [7] are listed in §B.8.

ARA. Heretic’s default modifier [9]. For each targeted module in a searched layer range, inputs and outputs at the last prompt position are recorded on the harmless and harmful estimation prompts, and a rank-50 adapter is fitted by L-BFGS (5 outer steps of at most 20 iterations, history 10, learning rate 1.0, strong-Wolfe line search) to the objective of §A.3.1, with AA initialised from the run seed. Row-magnitude preservation enters only the objective; the exported adapter is not renormalised.

J-space (small pair only). Directions follow the jspace-refusal code [48, 50, 49]: dℓ=JℓT(g⊙(WU[R]‾−WU‾))d_{\ell}=J_{\ell}^{\mathsf{T}}(g\odot(\overline{W_U[R]}-\overline{W_U})), unit-normalised, with WUW_U the unembedding, RR the token ids of the package’s 29 refusal words (34 ids), gg the final-norm gain probed by normalising an all-ones vector, and JℓJ_{\ell} the WikiText-fitted Jacobian lens of layer ℓ\ell (neuronpedia/jacobian-lens, revision qwen-n1000); the lenses cover all blocks but the last, where J=IJ=I. No harmful/harmless contrast set is used. The PrecomputedDirections modifier then applies these directions with DiM/MPOA’s edit and search space unchanged, without harmless-mean projection and with full row normalisation (rank 3). J-space is thus weight-baked here, not the forward-hook intervention of the original code.

Additional conditions. Three further conditions were run: a γ\gamma-corrected MPOA control on the four post-norm models; a second seed (seed 2) for all five methods on the small pair (10 runs), in place of the planned second seed for DiM, MPOA and ARA on the Qwen and Gemma pairs; and eight publicly released community abliterations as external references under the same evaluation protocol (huihui-ai/Qwen2.5-32B-Instruct-abliterated, zetasepic/Qwen2.5-32B-Instruct-abliterated-v2, heretic-org/Qwen3.8-27B-heretic-ara, huihui-ai/Huihui-Qwen3.8-27B-abliterated, coder3101/gemma-4-31B-it-heretic, MuXodious/Olmo-3.1-32B-Instruct-impotent-heresy, huihui-ai/Huihui-Mistral-Small-3.2-24B-Instruct-2506-abliterated and llmfan46/Mistral-Small-3.2-24B-Instruct-2506-ultra-uncensored-heretic). The external OBLITERATUS toolkit [43] is not part of this study. Its v0.1.4 presets basic, advanced and spectral_cascade (commit 461ff086) were run on Qwen2.5-7B-Instruct and scored with this study’s harness; these are reported, with other external tools, in the companion paper [70]. Version 0.1.4 lists a som preset but does not ship its direction module and does not support the hybrid-attention Qwen3.5-4B, so it could provide no older–newer contrast. A paper-faithful SOM configuration and J-space as a forward-hook with α=1\alpha=1 were planned on the small pair but not run; they are left to future work (§ 5.2).

Optimisation, Prompt Splits and Selection

Sampler, seeding and resumption. The TPE sampler uses 128 expected-improvement candidates. Seed 1 is applied to Python, NumPy, PyTorch and Optuna and to the method-internal random steps (randomised SVD, SOM, ARA initialisation). In resumed processes (Qwen3.8-27B MPOA and the interrupted Qwen3.8-27B ARA study, §B.8) the stored seed (1) was restored for the sampler and the method-internal steps, but the global Python, NumPy and PyTorch generators were seeded randomly, because the resume commands did not pass --seed. Heretic chose the generation batch size automatically (upper bound 128); it was 128 in every run except Qwen2.5-32B DiM, capped at 64, and the seed-2 Qwen3.5-4B DiM, SOM and ARA runs on the workstation, where batch size 128 ran out of memory and 64 was chosen, so those three seed pairs differ in batch size as well as seed (sensitivity of the seed-noise band in § 4.1 and Appendix § 11.§ 4). Journals, configurations and logs are kept for every run.

Prompt splits. The refusal objective uses the first 100 test prompts of harmful_behaviors, which serve as the development (dev) set for both search and selection, and the KL objective the first 100 test prompts of harmless_alpaca. Batch-size probing and response-prefix detection use training prompts only. Dataset revisions were not pinned in the Heretic configurations. All primary and secondary refusal, capability and teacher-forced outcomes come from the held-out sets of § 3.1 and § 3.3. Dev-set values are reported only to document selection (Pareto fronts, KL-threshold sensitivity), as the first-token KL measure of distribution shift, and for the dev-versus-held-out overfitting check (§ 3.5).

Objective details. Heretic’s refusal markers are matched case-insensitively, after removing emphasis and normalising apostrophes and whitespace; empty responses count as refusals. Prompts use each model’s chat template with the system prompt “You are a helpful assistant.”. Heretic determines a response prefix automatically and prefills it when one is found: if the chat template opens a reasoning block, the prefix closes it; otherwise Heretic searches for a prefix common to all responses to the detection prompts. A prefix was found only for Qwen3.8-27B and Qwen3.5-4B, whose templates open a reasoning block, so their dev-set responses contain no reasoning; no prefix was found for any other model.

Selection. Heretic’s headless default exports the trial with the fewest refusals irrespective of KL divergence; the fixed rule of § 3.4 is applied instead in every run. The selected trial always lies on the Pareto front.

Held-out Evaluation

Held-out refusal set. The held-out file is version 2 of refusal_sets.json; its HarmBench items are the standard category of the HarmBench text test set [83], and its 200 malicious-code requests come from the code-as-weapon bank [27]. The code items come from the bank’s code_safety split, restricted to unanimous labeller agreement (tiers 5/5, 4/4), excluding items sourced from harmful_behaviors or JailbreakBench, and drawn proportionally by source with seed 20261105 (astra 88, scam2prompt 43, rmcbench 24, cysecbेंch 23, malwarebench 17, redcode 5). Each harmful item was compared with all 520 harmful_behaviors prompts (exact match after normalisation, or token Jaccard ≥0.6\ge0.6 with SequenceMatcher ratio ≥0.8\ge0.8); 44 items are flagged and results are reported with and without them. In a check made after evaluation with the same criterion, none of the 350 benign items matched any of the 31,323 harmless_alpaca prompts (training and test splits). Dataset revisions and file hashes are stored in the file’s metadata.

Generation. Where the chat template lacks a system role, the system prompt is folded into the user turn; reasoning is disabled with enable_thinking=False. A copy of each response truncated to 256 tokens and the finish reason are stored.

Teacher-forced divergence. The original model’s responses are scored under teacher forcing from vLLM prompt log-probabilities; the measure is a sequence-level complement to the first-token KL objective.

Capability benchmarks. Subsets were drawn once with seed 20261104 (MMLU-Pro allocated proportionally over its 14 subjects); their document identifiers are re-verified before every run. Decoding is greedy with fixed budgets, chat templates are applied (few-shot examples as multi-turn chat), reasoning is disabled and the context is 8,192 tokens; harness seeds are 0 (Python) and 1234 (NumPy, PyTorch, few-shot sampling, vLLM). A Transformers backend is reserved for models vLLM cannot load; all evaluations so far used vLLM. Each summary records library versions and a SHA-256 over the evaluated model’s files.

Judges and degeneracy heuristic. All judges score the same stored responses: WildGuard [87] (allenai/wildguard; primary judge of refusal and harmful response), the HarmBench classifier (cais/HarmBench-Llama-2-13b-cls; harmful compliance on harmful prompts), Qwen3Guard-Gen-8B [89] (second LLM judge; safety and refusal labels), each with its official prompt format; the ModernBERT refusal classifier of Heretic’s BERT scorer (VINAY-UMRETHE/Refusal-Classifier; prompt and response); the RoBERTa classifier Human-CentricAI/LLM-Refusal-Classifier (response only), kept for continuity with Young [21]; and Heretic’s 33 keyword markers (lower-cased, typographic apostrophes normalised; unlike Heretic’s scorer, no emphasis or whitespace normalisation, and empty responses are flagged as degenerate rather than counted as refusals) on the full response and its first 256 tokens. LLM judges run greedily in vLLM with a 4,096-token context, except the HarmBench classifier, which runs with its 2,048-token context on the response cut to its first 512 classifier tokens; for 6 long code-misuse prompts the behaviour text was also shortened to fit (436 of 97,828 judged responses). BERT inputs are truncated to 512 tokens. A degeneracy heuristic flags empty responses; responses of at least 20 words in which more than half of the word 4-grams repeat or fewer than 15 % of words are distinct; runs of more than 40 identical characters other than whitespace and the separator characters used for rules in code and Markdown (= - _ * # - ~ ), whose runs count only when they make up more than half of the response; and more than 20 % non-printable characters. In the analysis a character-run flag is kept only if the run reaches the end of the response, if a run of a character other than these separators and box-drawing characters lies outside Markdown code, or if a separator or box-drawing run makes up more than half of the response; otherwise only the non-printable test applies. This recheck cleared 119 of the 162 raw character-run flags, which marked mainly hexadecimal constants, a long string copied from one prompt and box-drawing rules in coherent code. The outcome classes of § 3.3 are assigned to all 97,828 judged responses; WildGuard returned no refusal label for 5 of them, which are left unscored and excluded from WildGuard rates. Each judge was loaded at an explicit Hugging Face commit, resolved from the repository’s main branch when scoring started, passed to the loader and stored with the scores. Both scoring batches resolved the same commits, so all 74 evaluated models were scored with allenai/wildguard at cbba4823, cais/HarmBench-Llama-2-13b-cls at bda70534, Qwen/Qwen3Guard-Gen-8B at 4505cb1a, VINAY-UMRETHE/Refusal-Classifier at 5c525175 and Human-CentricAI/LLM-Refusal-Classifier at 0d9379ea.

Export and Release

Every evaluated model is a merged, standalone checkpoint, never a live adapter. The selected trial is restored from a copy of the study journal patched with the trial’s Pareto position. Heretic reloads the base model, recomputes the method’s statistics (J-space reads its direction file), rebuilds the adapter from the trial’s stored parameters and merges it with PEFT merge_and_unload. The exported checkpoint differs from the original only in the attention (or linear-attention) output and MLP down-projection matrices of the layers the trial edits, each replaced by W+BAW + BA with rank (BA)≤1(BA) \le1 (DiM), 3 (MPOA, J-space), 6 (SOM) or 50 (ARA); all other weight tensors and the tokenizer are the base model’s. Main-study searches ran on the GPU that then evaluated the model (§B.7): the RTX box for the Qwen and Gemma pairs and the H200 box for the OLMo and Mistral Small pairs. All main-study searches ran on the H200 box, so the Qwen and Gemma exports were rebuilt on a different GPU from their search. For ARA, rebuilding the adapter means refitting it by L-BFGS from the stored hyperparameters, and the ARA exports rebuilt on the other GPU drifted more on the dev re-score than those rebuilt on the search GPU (Appendix § 11.§ 3). The export step itself does not re-score the model; the released merged exports were therefore re-scored separately on the dev set and compared with the selected trial’s dev refusal count and KL divergence (Appendix § 11.§ 3).

Models are published with Heretic’s upload action as repositories richardyoung/{base}-{METHOD}-heretic with Heretic’s model card; the five Gemma-2-27B repositories are private (withheld), the five OLMo-2-32B repositories are gated with automatic approval, and all others are public (Ethics Statement). For MPOA, ARA and DiM the upload includes Heretic’s full reproduce/ manifest (configuration, study journal, requirements and weight checksums), which upstream Heretic can use to re-run the search and export (heretic --reproduce); the manifest itself notes that reproducibility is not guaranteed because Heretic was installed from a local directory or wheel. SOM, J-space and γ\gamma-MPOA models omit it, because they need a project fork and, for J-space, an external direction file. The released model cards name neither this paper nor the fork commits (8fd4d76, fdfe54b); all Heretic commits used are recorded in the run records and environment captures (§B.2). For the main study the evaluated copy is downloaded back from the published repository, so evaluated and published weights coincide by construction. Small-pair models were evaluated from a locally kept export (the workstation’s Hugging Face token was read-only during the runs) and published afterwards by re-exporting the same trial; publication is accepted only if every uploaded weight shard is SHA-256-identical to the benchmarked copy; this check passed for all ten small-pair models.

Released models (all except the five withheld Gemma-2-27B variants) keep the licence of their base model: Apache-2.0 (the Qwen, OLMo and Mistral-Small-3.2 bases and Gemma-4-31B), to which the use restrictions of the OpenRAIL++ licence are added; or, for Mistral-Small-2409, the Mistral Research Licence, which permits non-commercial research use. The five OLMo-2-32B variants are gated on Hugging Face with automatic approval (Ethics Statement). A licence states permitted use; it is not a technical control. The Human-CentricAI judge (CC BY-NC) is used for research only.

Whether each licence permits redistributing a derivative whose refusal has been removed was checked against the licence texts, retrieved on 8 November 2026. Apache-2.0 (Section 4) permits redistribution of modified works with a copy of the licence and notices of modification and contains no use restriction; the OLMo model cards add that the models are “intended for research and educational use” under Ai2’s Responsible Use Guidelines, which “do not restrict the use of the Ai2 Tools for the sole purpose of scientific research and development”. The Mistral Research Licence 0.1 permits the distribution of derivatives made by the licensee (Section 2.3) provided they carry “prominent notices stating that You modified the concerned Mistral Model”, and limits the use of derivatives to research purposes (Section 3.2); it contains no provision on safety behaviour, so it permits the release for non-commercial research use. The Gemma Terms of Use permit the distribution of model derivatives (Section 3.1) subject to the use restrictions of Section 3.2, which incorporate the Gemma Prohibited Use Policy; that policy forbids using Gemma or model derivatives for “attempts to override or circumvent safety filters or intentionally drive Gemma or Model Derivatives to act in a manner that contravenes this Gemma Prohibited Use Policy”. The terms do not define “safety filters”, and on a plain reading the removal of trained refusal may fall under this clause, so this study cannot state that redistributing the Gemma-2-27B derivatives is permitted. The five Gemma-2-27B variants were therefore made private on 9 November 2026 and are not released; the “Notice” text file that Section 3.1 requires with every distribution was added to their repositories.

Multiple-Testing Families

Holm’s procedure controls each family of related tests at a two-sided α=0.05\alpha= 0.05, separately for each refusal measure and prompt subset. The families are H1 (23 tests: 21 pair ×\times method McNemar tests and the two pooled class effects), H2 (6: five per-pair interactions and the pooled one), modified versus original (42 models), the H3 gap model (4 terms), per-model keyword–WildGuard gaps (all 74 evaluated models), γ\gamma-MPOA versus MPOA (4), each community release versus the four models selected here for the same checkpoint (4 per release), seed 2 versus seed 1 (10 cells), per-benchmark capability changes of the γ\gamma-MPOA models (20 per reference model), over-refusal changes from the original (all 64 modified models), the 256-token label change of each judge and the 256-token keyword–judge gaps (all 74 evaluated models each; Appendix D.2), the two dev re-score tests (refusals and KL; Appendix D.3), the 100-token keyword–judge gaps (all 74 models; § 4.3), and, in the sensitivity analyses, the H1 and H2 families repeated within each item subset (Appendix D.5) and within each harmful-compliance outcome (Appendix D.6). Pair-level random-effects summaries, the heterogeneity tests, the KL-cap refits and the export-drift Fisher tests are not adjusted.

Compute and Software

Three single-GPU machines were used (Table 12); the two server boxes were rented from Massed Compute and downloaded models directly from Hugging Face. The Heretic environment used Python 3.12 (3.12.16 on the boxes, 3.12.15 on the workstation), PyTorch 2.14.1 (CUDA 13.0 build), Transformers 5.18.0, PEFT 0.21.2, Accelerate 1.15.0, Optuna 5.0.0, Datasets 5.0.1, safetensors 0.8.0, huggingface_hub 1.33.0 and NumPy 2.5.4. The evaluation environment is a separate lock-file-pinned project with vLLM 0.33.0, lm-eval 0.4.13, PyTorch 2.14.0 (CUDA 13.0), Transformers 5.20.2, Datasets 5.1.0 and Accelerate 1.15.0; vLLM’s FlashInfer sampler was disabled (identical argmax under greedy decoding). The 168 recorded single-GPU jobs ran for 69.3 h of wall-clock time in total: 32.7 h of Heretic optimisation (73 processes, including eight pilot runs), 33.8 h of evaluation (90 jobs; for selected models these include selection, re-export, upload and download where applicable) and 2.9 h of judging (5 jobs). The totals include 6.1 h of failed or aborted attempts and exclude time spent waiting for a busy GPU. Because two job queues shared the H200 for part of the search phase (§B.8), the GPUs were occupied for 67.5 h (H200 30.4 h, RTX PRO 6000 22.6 h, RTX 4090 14.5 h). Not included are model downloads, work without a run record (smoke tests, the computation of the J-space directions, and the re-export and upload that published the small-pair models), evaluations run on the RTX box for another study, and the statistical analysis. Per completed 100-trial main-study search on the H200 box, a direction-based study took a median of 18.1 minutes (range 10.2–61.5; 23 studies) and an ARA study 81.0 minutes (50.3–95.6; 8 studies), 18.1 h in total; the resumed Qwen3.8-27B MPOA study, split over three processes, is not included.

MachineGPU (memory)CPU / RAMDriverRole
H200 box1× NVIDIA H200 NVL (143,771 MiB)16 vCPU, AMD EPYC 9355 / 176 GiB580.126.09main-study search; export and evaluation of the OLMo and Mistral Small pairs (incl. OLMo γ\gamma-MPOA) and 3 community models; ARA/MPOA/DiM pilots
RTX box1× RTX PRO 6000 Blackwell Server Ed. (97,887 MiB)16 vCPU, AMD EPYC 9335 / 141 GiB580.126.09export and evaluation of the Qwen and Gemma pairs (incl. Gemma γ\gamma-MPOA) and 5 community models; all judging; release
Workstation1× GeForce RTX 4090 (24,564 MiB)AMD Ryzen 9 7950X / 62 GiB615.78.08small pair: all steps; SOM and J-space pilots

Table 12. Hardware (GPU memory from nvidia-smi, RAM from free -g).

Deviations from the Plan

  • Mistral-Small-3.2 source. The official repository ships only mistral-common tokenizer files, without Hugging Face tokenizer or processor configurations, so Heretic failed at load before any trial. The runs were repeated from unsloth/Mistral-Small-3.2-24B-Instruct-2506 pinned at commit 3ed8d341b53ea3d0e4194689905477688a6cf733, whose ten weight shards are SHA-256-identical to the official ones (config.json differs only in metadata). The Hugging Face tokenizer has a known pre-tokenisation regular-expression defect (Transformers suggests fix_mistral_regex=True, which Heretic does not pass) that changes the tokenisation of 3 of 33,165 study prompts; the same tokenizer is used for original and modified models in Heretic and vLLM.

  • SOM relative to Piras et al. [7] The port ranks neurons by wins over the harmful residuals (the proof of concept counted wins over the map’s own prototypes, reducing “top-k” to the first kk grid cells), orthonormalises rather than sums the directions, uses one geometric decay instead of per-direction strengths and a single integer layer for the global scope, keeps the proof-of-concept SOM hyperparameters (not confirmed against the paper), and uses a rank-6 adapter not compared with higher ranks; hence “Heretic-tuned SOM”.

  • Concurrent jobs on the H200. Until an idle-GPU guard was added, a second job queue shared the GPU, causing out-of-memory failures. Qwen2.5-32B MPOA completed 100 trials with part of the model offloaded to CPU. Qwen3.8-27B MPOA failed twice (after 5 and 6 trials) and was resumed from the second attempt’s journal; the evaluated study has 99 completed trials and one failed trial. The first Qwen2.5-32B ARA run failed at its third trial with a meta-device error under CPU offloading and was repeated from scratch (100 trials; evaluated).

  • Qwen3.8-27B ARA rerun. The first Qwen3.8-27B ARA study was interrupted by the same cause after 66 completed trials; one resumption failed and a second completed it (99 completed, one failed). On resumption Optuna re-creates the sampler from the same seed, so later proposals differ from those of an uninterrupted run. Because the study was not one uninterrupted optimisation, and because its dev-set result (59/100 keyword refusals at KL ≤0.1\leq0.1) differed from the 0/100 at KL 0.054 reported for heretic-org/Qwen3.8-27B-heretic-ara (same defaults, 60 trials), it was repeated from scratch with identical configuration and seed and no interruption (the “clean” run, 100 trials). The clean run is evaluated; the interrupted study is retained. No such repeat was run for Qwen3.8-27B MPOA. Instead, its journal was checked against the seeded start-up sequence. The study was resumed after 7 recorded trials (6 complete, 1 failed). On resumption the sampler restarted the seed-1 start-up sequence, so trials 7–29 repeat the first 23 start-up draws: all 30 start-up trials have the same parameters, position for position, as the uninterrupted seed-1 OLMo-3.1-32B MPOA study, and the 6 trials evaluated twice gave dev keyword refusals within 1 of 100 of each other. The selected trial (number 29, 25/100 at KL 0.085) is one of these repeated start-up draws, so an uninterrupted seed-1 study would have evaluated

the same parameters (as its trial 22) and, under the selection rule, selected a trial with at most about 25/100 dev refusals. The resumption can therefore have raised, but not lowered, the dev refusal rate of the selected Qwen3.8-27B MPOA trial; its effect on held-out refusal was not measured. For the one study run both ways (Qwen3.8-27B ARA, seed 1), the resumed and clean selections differed by 4 of 100 dev keyword refusals (59 and 63).

  • J-space directions. Before the analysed runs the export was corrected (probed rather than stored final-norm gain; J=IJ = I for the last block) and the directions regenerated; the J-space pilot used the earlier directions.

  • Evaluation-harness corrections. (i) lm-eval 0.4.13’s mbpp_instruct extractor dropped the first word of every program, giving pass@1 of zero; it was patched and earlier MBPP scores were recomputed from logged samples. (ii) vLLM 0.33’s detokeniser left the meta-space character U+2581 in Gemma-2 outputs; it is mapped to a space in every evaluation run after the fix, and the affected Gemma-2 evaluation was repeated; the earlier Qwen2.5-32B base evaluation contains no U+2581 characters. (iii) The first Qwen3.8-27B baseline evaluation fell back silently to the Transformers backend (vLLM rejected the default concurrency for this hybrid model); it was repeated in vLLM with max_num_seqs = 256, as used for all Qwen3.8/Qwen3.5 models. Other model-specific vLLM settings: Gemma-4 image inputs disabled; Mistral with Hugging Face tokenizer and configuration formats.

  • Mixed model. The planned mixed model was to be fitted with lme4; because R was not available it was fitted by variational Bayes in statsmodels (BinomialBayesMixedGLM), which gives no pp values, and the pooled tests use the GEE (§ 3.5).

  • Status. All 42 primary runs, the four γ\gamma-MPOA controls and the ten seed-2 runs were completed, selected, exported and evaluated, as were the ten original models and the eight community releases, and all 74 evaluated models were scored by every judge on all 1,322 held-out prompts. The three LLM judges were also applied to the 256-token cut of every response (Appendix § 11.§ 2), and the 46 released merged exports (42 primary, four γ\gamma-MPOA) were re-scored on the dev set (§B.5, Appendix § 11.§ 3). The paper-faithful SOM configuration and the J-space forward hook (§B.2) were not run and are left to future work.

Threats to Validity: Full List

This appendix lists every identified limitation and threat to validity; § 5.2 summarises the most consequential of them. Where it can be stated, each limitation is given with its likely direction of bias and its mitigation.

Single seed and run-to-run variance. Each main-study cell is optimised with a single seed (seed 1); the Qwen3.8-27B ARA cell was run twice with seed 1 (see below). TPE search is stochastic (Heretic #88), and one ARA trial has been reported to yield 52 and 11 refusals on two machines (Heretic #499); both are unreviewed practitioner reports, retrieved in November 2026. Here, two seed-1 ARA runs on Qwen3.8-27B (one interrupted and resumed, one clean) selected trials with 59 and 63 of 100 keyword refusals at KL ≤0.1\leq0.1, whereas a community model with the same defaults and 60 trials (heretic-org/Qwen3.8-27B-heretic-ara) reports 0 of 100 at KL 0.054 on its model card, an unreviewed figure. The evaluated Qwen3.8-27B MPOA run was resumed after an out-of-memory failure and completed 99 of 100 trials. Seed and configuration variance has altered conclusions about algorithm comparisons [133] and about the durability of open-weight safeguards [69]. Such variance can create or mask a generation effect in a single cell. On the small pair, all five methods were optimised again with seed 2 (10 runs, in place of the planned second seed for DiM, MPOA and ARA on the Qwen and Gemma pairs): held-out refusal differed between seeds in all 10 cells (median 9.4 pp, maximum 25.2 pp), and the ARA generation effect fell from +14.2 pp to +0.0 pp, so the small pair meets H1 with seed 1 only. Mitigation: full Pareto fronts and four KL thresholds are reported next to the seed-2 spread is reported next to the generation effects, a run-to-run standard deviation derived from [9.2 pp [6.4, 16.1]] flags the 11 of 16 main-study generation effects that lie within seed noise (Table 6; this assumes that the seed variance of the 4B and 7B models transfers to the 22–32B models), and method rankings were more stable than single-cell generation effects (η2\eta^2 of method 0.94 and 0.88). Three of the ten seed-2 runs also used a generation batch size of 64 instead of 128 after running out of memory; without them the standard deviation is 7.2 pp [4.8, 14.6] and the same five raw effects exceed the band (baseline-adjusted: six instead of three). The heretic-org trial on Qwen3.8-27B was not run in this pipeline, and a second seed for the main-study cells is left to future work.

Unit of inference. The generational question has the model pair as its unit, and there are four main-study pairs. Prompt-level intervals and tests cover prompt sampling within a pair only; between-family heterogeneity was large (generation ×\times family χ2=547.4\chi^2 = 547.4, 3 df), so pooled estimates are summaries of per-pair effects of different sign. Mitigation: per-pair effects with seed variance added are the primary evidence, pooled effects are reported with pair-level random-effects (Hartung–Knapp–Sidik–Jonkman) and prediction intervals, and leave-one-family-out fits are given (§ 4.1, Appendix § 11.§ 4).

Selection on the development set. TPE optimises, and the selection rule selects, on the 100 prompts (mlabonne/harmful_behaviors test[:100]) that produce Heretic’s refusal count, and refusal directions (DiM, MPOA, SOM) and ARA’s harmful-side activations are estimated from mlabonne/harmful_behaviors train[:400], contrasted with mlabonne/harmless_alpaca train[:400] (J-space directions use no contrast set). Development-set refusal is therefore optimistic (Heretic #481, #503; unreviewed practitioner reports), possibly more so for methods with larger search spaces, because optimising a noisy selection criterion overfits it and reusing the selection data for evaluation biases the estimate [114]. Mitigation: reported refusal comes from a frozen held-out set of 1,322 prompts (JailbreakBench harmful and benign, 100 each; XSTest, 450; StrongREJECT, 313; HarmBench standard, 159; code-as-weapon bank, 200 [27]) [80, 81, 82, 83]; 44 items matching the 520 AdvBench-derived prompts are flagged and results reported with and without them; dev–test agreement is tested with Spearman’s ρ\rho.

Keyword refusal. Heretic counts a refusal when any of 33 marker strings appears within 100 generated tokens. Reported keyword recall is about 0.48, against 0.77 for an LLM judge (Heretic #419; see also PR #196; both unreviewed practitioner reports, retrieved in November 2026); in a peer-reviewed, human-labelled meta-evaluation, keyword matching reached Cohen’s κ=0.37\kappa= 0.37, against about 0.81 for fine-tuned LLM judges [22]. Because TPE minimises this count, methods are rewarded for avoiding marker phrases rather than for complying, so selected models were expected to retain more refusal than counted (H3). On the held-out set, at the tool’s 100-token window, keyword matching and WildGuard agreed on average over the 42 ablated models (+0.5 pp [−1.0, +1.9]) but differed significantly in both directions for 25 of them; on the full response keyword matching was higher (+8.4 pp), because of markers after the first 100 tokens (Table 9). Mitigation: keyword counts drive optimisation and trial selection only; held-out refusal is reported from WildGuard, with Qwen3Guard as a second judge and a degeneracy check, and held-out keyword counts are reported only for comparison with the judges (H3).

Post-norm leakage. Gemma-2/4, OLMo-2 and Olmo-3.1 apply an RMSNorm [75] with gain γ\gamma after WOW_O and WdownW_{\mathrm{down}} (output normalisation [76]). OLMo 2 and Olmo 3 normalise module outputs only [73, 74], whereas Gemma 2 normalises the input and the output of each sublayer [72]. After orthogonalisation the module output satisfies r^Ty=0\hat{\mathbf{r}}^{\mathsf{T}}\mathbf{y}=0, but the normalised output keeps a component proportional to (γ⊙r^)Ty(\gamma\odot\hat{\mathbf{r}})^{\mathsf{T}}\mathbf{y}, handicapping direction-based methods on four of the eight main-study models. Because both generations of each affected family share this design, the leak mainly penalises direction-based methods relative to ARA within these families; it confounds the generation effect (H1) and its interaction with method class (H2) to the extent that its size differs between generations, and it is also a candidate mechanism for any generation effect. Mitigation: per-family reporting and a γ\gamma-corrected MPOA control on the four post-norm models. γ\gamma-MPOA lowered held-out refusal relative to MPOA on all four (by 3.1–19.9 pp), but the OLMo generation effect remained +57.0 pp and the pooled H2 interaction −0.42. On Gemma-4-31B (the largest reduction) and OLMo-2-32B, γ\gamma-MPOA also had a larger teacher-forced divergence than MPOA and lost 21.4 and 8.1 pp of mean capability, so these two reductions are confounded with a stronger edit.

J-space scope. Instruction-tuned Jacobian lenses exist only for Qwen3.5-4B and Qwen2.5-7B-Instruct (neuronpedia/jacobian-lens, revision qwen-n1000, fitted on WikiText; Gemma lenses are base-model lenses), so J-space is limited to the small pair. The lens is fitted on non-chat text, and the direction is baked into the weights through Heretic’s ablation and TPE search rather than applied as the reference forward hook; the direction of bias is unknown. J-space as a forward hook (α=1\alpha= 1) was planned but not run and is left to future work.

Method implementations. Heretic-tuned SOM (commit 45a4e6e) deviates from Piras et al. [7] and the proof of concept (Heretic PR #196): neurons are ranked by wins over harmful residuals (fixing a win-count bug); directions are Gram–Schmidt orthonormalised; one geometric decay replaces per-direction strengths; global scope uses a single integer layer; map settings follow the proof of concept; and the norm-preserving adapter has rank 6, untested against higher ranks. The narrower search may understate SOM; a paper-faithful SOM configuration was planned but not run and is left to future work. MPOA’s row normalisation is approximated by a rank-3 LoRA, and ARA’s deployed adapter is not row-norm-preserving. All evaluated models are merged exports, not live adapters (Heretic #490, an unreviewed practitioner report).

ARA instantiation. The released ARA model is rebuilt by refitting its adapter from the stored hyperparameters. The four ARA exports refitted on a different GPU from the search (Qwen and Gemma pairs) moved by more than two dev refusals on re-score, against none of the four refitted on the search machine (Fisher p=0.029p = 0.029), and the Gemma-4-31B ARA export re-scored at KL 0.107, above the cap. Two seed-1 ARA runs on Qwen3.8-27B selected 59 and 63 of 100 dev refusals, whereas heretic-org reports 0 of 100 at KL 0.054 with the same defaults; the reported trial was not tested here. These points bear on H2, whose Qwen reversal and Gemma support rest on single ARA cells. Mitigation: held-out results were measured on the released exports, so they describe the released models; the drift is reported per export (Appendix § 11.§ 3).

Mistral-Small-3.2 tokeniser. The official repository lacks Hugging Face tokeniser files, so the unsloth mirror (commit 3ed8d34) was used; its 10 weight shards are SHA256-identical to the official ones. Its Hugging Face tokeniser has a known pre-tokenisation regex defect (transformers suggests fix_mistral_regex=True, which Heretic does not pass) that alters 3 of 33,165 study prompts. The same tokeniser is used for the base model, all modified models and evaluation (the base evaluation recorded commit 3ed8d34), so within-model comparisons should be unaffected.

Truncation. Held-out responses are limited to 512 tokens; for the unmodified Qwen3.8-27B, 491 of 1,322 responses reached the limit, and on the 972 harmful prompts 11.2–35.3% of the originals’ responses and 17.6–87.8% of the 42 primary models’ responses did so. Truncation should affect long, compliant answers more than refusals, so judges may rate partial answers as incomplete or less harmful, understating harmful compliance. Mitigation: finish reasons and the first 256 tokens of every response are stored; keyword refusal is also computed on the 256-token cut; compliance is split into complete and incomplete compliance by finish reason (§ 3.3); the keyword–WildGuard gap is reported separately for truncated and complete responses (§ 4.3); and the three LLM judges were also applied to the 256-token cut of every response (Appendix § 11.§ 2).

Generation is confounded. Pairs differ in more than release date: size (the newer small-pair model is smaller, and after abliteration harmful-code success has been found to be limited by model size [65]), architecture (Qwen3.8-27B and Qwen3.5-4B use hybrid attention with gated-delta-rule linear-attention layers [77], whose recurrent state may carry refusal information that residual-stream directions do not capture, and plain directional ablation has made a recurrent model incoherent [34]; Olmo 3 adds sliding-window attention on three of every four layers [74]; Qwen3.8-27B, Gemma-4-31B, Mistral-Small-3.2 and Qwen3.5-4B are image-text checkpoints used with text only), training data and post-training recipe, and the interval between releases varies by family. Later checkpoints may also have been safety-tuned on the public held-out benchmarks (JailbreakBench, XSTest, StrongREJECT, HarmBench) or paraphrases of them, which would raise their held-out refusal for reasons other than resistance to removal [134]; the overlap check covers only harmful_behaviors and is lexical. The four public benchmarks first appeared between August 2023 (XSTest) and April 2024 (JailbreakBench; HarmBench and StrongREJECT in February 2024), before the earliest checkpoint (Gemma-2-27B-it, June 2024; release months in Table 2), so exposure was possible for every checkpoint, and the later checkpoint of each pair appeared 9 to 23 months after the earlier one; the code-as-weapon bank (May 2026) predates only Qwen3.8-27B (August 2026). A generation effect cannot be attributed to safety training alone. The item mix also matters for the originals’ baseline: the code-as-weapon bank (200 items) carried 52% of the mean generation difference between the main-study originals, although the H1 and H2 conclusions were the same on the four public sets alone (Appendix § 11.§ 5). Mitigation: within-family comparisons, family as a random effect, and conclusions phrased as statements about later releases, not mechanisms.

Decoding. All refusal measurements, during search and on the held-out set, use greedy decoding. Changing only decoding hyperparameters raised the misalignment rate of 11 open models from 0% to more than 95% [86], so residual refusal under greedy decoding may overstate refusal under sampling, possibly by different amounts in each generation. Each prompt received one greedy response, and reasoning was disabled, although the Qwen3.8-27B and Qwen3.5-4B chat templates open a reasoning block and chain of thought can carry refusal [129]. Mitigation: decoding is identical for every model, so within-pair comparisons are matched on decoding, and the conclusions are stated for greedy, non-thinking generation; a sampled-decoding sensitivity check is left to future work.

First-token KL. Distribution shift is constrained only through a common cap on the KL divergence of the first generated token on 100 harmless prompts, which need not reflect sequence-level change and may not be comparable across vocabularies. The cap is common but not matched: it bound in 15 of the 24 main-study direction-based runs and in all 8 ARA runs, and 4 direction-based runs never sampled a trial above it. Mitigation: selection at KL 0.05, 0.2 and 0.5, teacher-forced divergence on the original model’s answers to the 350 benign held-out prompts, and H1 and H2 refitted by cap binding (Appendix § 11.§ 4).

Capability benchmarks. MMLU-Pro (1,000 items, stratified) and GSM8K (500 of 1,319) are subsets; IFEval (541), HumanEval (164) and MBPP (500) are complete [91, 92, 93, 94, 95]. Thinking is disabled, so reasoning-mode ability is not measured. Two harness defects were corrected: the lm-eval 0.4.13 MBPP answer extractor (MBPP scores recomputed from logged samples) and a vLLM 0.3.33 detokeniser artefact (U+2581 left in Gemma-2 output; mapped to a space in every evaluation run after the fix, and the Gemma-2 base evaluation rerun; the earlier Qwen2.5-32B base evaluation, run without the mapping, contains no U+2581).

Outcome construct. Residual refusal is not attack success: of the ablated main-study responses that WildGuard did not label as refusals, 75.1% were harmful by the HarmBench classifier and 55.8% by WildGuard. Mitigation: H1 and H2 were repeated with harmful compliance as the outcome, with the same pooled conclusions (Appendix § 11.§ 6).

Judge validity. Reported refusal and harm rely on automated judges (WildGuard, the HarmBench classifier, Qwen3Guard, two BERT-style classifiers) [87, 83, 89], applied to all 74 evaluated models, with no human-labelled audit. Conclusions can depend on the classifier [23], modified-model outputs may be out of distribution for the judges, and the BERT-style classifiers truncate inputs to 512 tokens. Against human labels, WildGuard and the HarmBench classifier reached κ\kappa of 0.61 and 0.53 [22], and rewriting harmful responses as stories raised their false-negative rates from 0.02 to 0.12 and from 0.12 to 0.36 [88]. Mitigation: several judges with official templates, each loaded at a pinned and recorded Hugging Face commit (the same five commits for all 74 models; Appendix § 9.§ 4), inter-judge agreement (Cohen’s κ\kappa for pairs [110], Fleiss’ κ\kappa across the panel [111]), each with raw agreement and marginals, because κ\kappa is depressed when labels are imbalanced [112], and a separate degeneracy class. WildGuard and Qwen3Guard agreed at κ=0.90\kappa= 0.90 on the originals and 0.82 after ablation, but agreement between LLM judges with correlated errors is not confirmation [125].

No preregistration. The study was not preregistered: hypotheses were written after pilot data, and the protocol was revised after part of the main-study optimisation and the capability benchmarks of one base model (Qwen2.5-32B-Instruct) had run, though before any held-out response was generated.

Generalisation. Ten dense models from four families (Qwen contributing two pairs), 4–32B parameters, English prompts and one Heretic 2.0 configuration with 100 trials were studied; method rankings can change with the search budget [71], particularly between methods with search spaces as different as rank-1 DiM and rank-50 ARA; 8 of the 42 core runs (19%) still improved after trial 75 (Figure 19). Results may not transfer to mixture-of-experts or larger models, other languages, reasoning mode or other implementations; eight public community releases of five checkpoints were evaluated as external references (Figure 6), and four of them reached lower held-out refusal than the models selected here, at larger teacher-forced divergence. Three presets of the external OBLITERATUS toolkit on Qwen2.5-7B-Instruct are evaluated with the same harness in the companion paper [70].

Dual use and ethics. Each released model (all except the five withheld Gemma-2-27B variants) is a full merged checkpoint, published on Hugging Face under the licence of its base model (Apache-2.0 with the added use restrictions of the OpenRAIL++ licence, or the Mistral Research Licence for Mistral-Small-2409; Gemma-4-31B is Apache-2.0); a licence states permitted use and is not a technical control. The five Gemma-2-27B variants are withheld because of the Gemma Prohibited Use Policy; the five OLMo-2-32B variants are gated with automatic approval, because no third-party public abliteration of OLMo-2-32B was found; the others are public (Ethics Statement). Server-evaluated models are the copies downloaded back from the published repositories; all ten RTX 4090 (small-pair) uploads are verified SHA256-identical, shard by shard, to the benchmarked copies. Reproduction manifests accompany only MPOA, ARA and DiM, because SOM, J-space and γ\gamma-MPOA need a project fork. No harmful completions are released; responses are stored privately for scoring. The study involved no human participants and was not human-subjects research.

Additional Results

This appendix gives the per-run optimisation traces and the exact values behind several main-text figures. Figures 11 and 12 show every completed trial of the seed-1 runs of the older and newer models (Pareto fronts in Figure 10). Figure 13 gives the capability change and teacher-forced divergence of the eight community releases (refusal in Figure 6), and Figure 14 the convergence curve of every run (summary in Figure 19). Tables 13 and 14 list the dev-set keyword refusals of the trial selected at KL budgets of 0.05, 0.1 and 0.2 (Figure 18); Tables 15 and 16 list the benchmark scores (Figure 8); and Tables 17 and 18 list the teacher-forced log-likelihood of each original model’s answers and its shift under each selected model (Figure 9). The four γ\gamma-MPOA models, which Figures 8 and 9 include, are not listed in these tables; their benchmark changes, dev KL and teacher-forced divergence are printed in Figure 5(b). Appendices D.2 and D.3 report the 256- and 100-token checks of H3 and the dev-set re-score of the released exports; Appendices D.4, D.5 and D.6 report the KL-cap binding, the pair-level pooling and the two scales, the item-source analysis and harmful compliance as the outcome.

Figure 13. Public community releases: capability change and divergence from the original. The 8 community releases evaluated alongside the selected models (2 of older, 6 of newer models; row label = Hugging Face uploader), grouped by the original they were derived from. Left: benchmark score minus that of the same original, in percentage points, with the diverging scale of Figure 8 (saturating at ±8 pp; printed values exact); the last column is the unweighted mean of the five benchmarks. Right: teacher-forced divergence on the 350 benign held-out prompts, defined as in Figure 9 (0 = the original); hollow markers are releases of the older, filled of the newer model of a pair. The grey bar spans the divergences of this paper’s selected models on the same original (all conditions of Figure 9). 6 of 8 releases lie above that range; divergences range from 0.026 (heretic-org, Qwen3.8-27B) to 0.191 (llmfan46, Mistral-Small-3.2) nats per token, and mean capability changes from −0.9 to +0.1 pp. The releases were made by third parties with their own methods and selection; they were not selected with this paper’s rule and are not part of the H1–H3 tests.

Figure 14. Search-budget convergence of every optimisation run (dev set). One step curve per run (seed 1): after trial tt, the fewest dev keyword refusals (of 100) among trials 1 to tt with first-token KL divergence ≤0.1\le0.1 from the original, i.e. the count the selection rule would have returned had the search stopped at trial tt. Rows are methods and columns the older and newer release of each pair, so that a panel holds at most five runs, each named at its right end; bottom row: J-space (small pair only) and γ\gamma-MPOA (Gemma and OLMo pairs only). Dashed lines with hollow markers are older, solid lines with filled markers newer releases; the marker sits where the run first reached the count it finally selected. Curves start at the run’s first trial within the KL budget (trials 5–10 for ARA, 1–2 otherwise). The grey band marks the 30 random startup trials of the TPE sampler, the dotted line trial 75. The medians, spreads and first-reach histogram of the core runs are in Figure 19. All counts are keyword matches on the 100 dev prompts that were also used for search and selection, not held-out refusal rates.

Figure 18. How much refusal is left at a given KL budget (dev set). Each cell is the fewest dev keyword refusals (of 100) among the completed TPE trials of one optimisation run whose first-token KL divergence from the original is at most the column budget tt. Within each pair the upper block is the older and the lower block the newer release; each block starts with the unedited original (KL 0). Darker cells mean more refusal survives. The outlined t=0.1t = 0.1 column is the selection budget, so it equals the dev refusals of the selected models. A degree sign (°) marks a budget that is not binding (60 cells: every completed trial already lies within it, so larger budgets were never explored and the value is the run’s overall minimum); a dagger marks a minimum over fewer than 10 trials (6 cells). The columns are post-hoc budgets on one 100-trial search per run (seed 1), not separate optimisations, and the counts are dev-set numbers, not held-out refusal rates.

The Failed Gemma-4-31B DiM Edit

In the Gemma-4-31B DiM study, 10 of the 100 trials met the KL cap and 90 exceeded it. Every trial with a first-token KL below 3 (28 trials) refused 98 to 100 of the 100 dev prompts; lower refusal appeared only above a KL of 3 (minimum

3 of 100 between 3 and 8, and 0 above 8), where the edited model is far from the original. In every other direction-based primary study, the lowest-refusal trial above the cap had a KL of at most 0.37. The edit therefore did not trade refusal against divergence gradually: within any usable budget it had no effect. Two mechanisms proposed for similar failures on Gemma-3 are a residual-stream coordinate of very large magnitude that dominates the difference-in-means estimate [131, 6] and learned normalisation gains that amplify in [131]. The post-sublayer RMSNorm gains of the four post-norm models were therefore read from the released weights. Gemma-2-27B and Gemma-4-31B both had recurring outlier channels (a gain over 20 times the layer’s median in 39 of 92 and 27 of 120 post-norm blocks, mostly one channel in Gemma-2-27B and two in Gemma-4-31B), whereas OLMo-2-32B and Olmo-3.1-32B had 3 of 128 and 1 of 128. The median coefficient of variation of the gains was 0.65 and 0.68 for the attention outputs and 0.38 and 0.62 for the feed-forward outputs of the two Gemma models, and 0.09 and 0.34 in the OLMo models; outlier channels were more frequent in Gemma-2-27B than in Gemma-4-31B (42% and 23% of vectors). The gains thus separate the Gemma family from OLMо, consistent with the larger effect of the γ correction on the Gemma models (§ 4.4), but they do not separate Gemma-4-31B from Gemma-2-27B, on which DiM lowered refusal. The activation outlier was not measured, and the cause of the failure remains untested.

Judging the First 256 Tokens

Held-out responses were generated with up to 512 new tokens and judged in full, whereas Heretic’s keyword check reads only the start of a response. To test whether the H3 comparison depends on how much text each measure sees, WildGuard, Qwen3Guard and the HarmBench classifier were re-run, with the same pinned revisions, prompt templates and parsers, on the stored first 256 tokens of all 97,828 held-out responses of the 74 judged models, and compared with keyword matching on the same 256 tokens. WildGuard’s output could not be parsed for 7 prefixes, which were dropped from the comparisons that use it. Only responses longer than 256 tokens differ between the two texts (Table 19); on the 24,507 responses of at most 256 tokens, WildGuard and Qwen3Guard repeated their full-response label in all but 6 and 14 cases, and the HarmBench classifier in all but 4 of the 17,967 among them that answer harmful prompts. Per model, label changes are paired differences of rates (prefix minus full response) with Newcombe intervals and exact McNemar tests, Holm-adjusted per judge across the 74 models. The keyword-minus-judge gaps, their pooling with the prompt-cluster bootstrap and the refitted class and generation effects (prompt-level GEE, model-level OLS) follow the H3 analysis of § 4.3.

ModelsCells>256, %WG changeWG, full textWG, 256 tokensQ3G, 256 tokens
Originals1040.7+0.3 [+0.1, +0.5]+2.6 [+1.8, +3.4]+1.6 [+0.8, +2.4]+2.7 [+1.9, +3.6]
Ablated (seed 1), all4278.9+0.6 [+0.5, +0.7]+8.4 [+6.9, +10.0]+3.8 [+2.4, +5.2]+0.9 [−0.6, +2.4]
&nbsp;&nbsp;main study3279.1+0.6 [+0.5, +0.7]+9.8 [+8.2, +11.3]+4.9 [+3.5, +6.4]+2.1 [+0.6, +3.5]
&nbsp;&nbsp;small pair1078.1+0.7 [+0.5, +0.9]+4.2 [+2.5, +5.9]+0.4 [−1.4, +2.0]−2.7 [−4.5, −1.0]
&nbsp;&nbsp;ending before the cap4256.9+0.0 [−0.1, +0.1]−1.2 [−3.2, +0.9]−5.6 [−7.6, −3.8]−7.6 [−9.8, −5.5]
&nbsp;&nbsp;truncated at the cap42100.0+1.2 [+1.0, +1.4]+17.7 [+16.2, +19.3]+13.0 [+11.5, +14.4]+9.1 [+7.9, +10.3]
γ-MPOA control482.0+0.7 [+0.4, +1.0]+17.1 [+14.9, +19.4]+12.0 [+9.9, +14.0]+9.1 [+7.1, +11.1]
Community releases890.3+0.5 [+0.3, +0.7]+16.5 [+15.2, +17.8]+13.6 [+12.4, +14.8]+8.1 [+6.8, +9.3]
Seed-2 replicates1078.2+0.8 [+0.5, +1.0]+3.5 [+2.0, +5.0]+0.0 [−1.5, +1.4]−1.9 [−3.4, −0.4]
All judged models7475.0+0.6 [+0.5, +0.7]+8.3 [+7.0, +9.6]+4.5 [+3.3, +5.7]+2.0 [+0.8, +3.2]

Table 19. Keyword matching and LLM judges on the first 256 tokens versus the full response, harmful held-out prompts (972 per model; H3 robustness). >256: share of responses longer than 256 tokens, the only ones whose 256-token prefix differs from the full response. WG change: WildGuard refusal rate on the prefix minus that on the full response. Keyword minus judge: refusal rate by Heretic’s 33 markers minus the judge’s rate; full text = both on the full response (as in Table 10); 256 tokens = both on the same prefix (WG = WildGuard, Q3G = Qwen3Guard). All differences in pp with prompt-cluster bootstrap 95% intervals; H3 predicted a negative gap. Cap: the 512-token generation limit.

Judging the prefix changed few labels. Over the 42 ablated models, WildGuard refusal on the harmful prompts was 32.1% on the first 256 tokens and 31.5% on the full response (+0.6 pp [+0.5, +0.7]); 385 responses gained and 131 lost the refusal label; Qwen3Guard refusal changed by +1.3 pp [+1.1, +1.4] and HarmBench harmful compliance by −1.6 pp [−1.8, −1.3]. Per model (Holm over the 74 models), WildGuard refusal changed significantly in 6 of the 74, all upwards and by 2.0–5.1 pp; Qwen3Guard refusal rose in 22 (by up to 8.3 pp) and HarmBench harmful compliance fell in 24 (by up to 9.3 pp), and no judge changed significantly in the opposite direction. Keyword matching on the same 256 tokens counted 36.0% refusals, so the keyword-minus-WildGuard gap of the ablated models was +3.8 pp [+2.4, +5.2], against +8.4 pp [+6.9, +10.0] on the full text (change −4.6 pp [−4.9, −4.3]); of this change, 4.0 pp came from the shorter keyword window (39.9% to 36.0%) and 0.6 pp from WildGuard. The gap was +4.9 pp [+3.5, +6.4] in the main study and +0.4 pp [−1.4, +2.0] in the small pair, and −5.6 pp [−7.6, −3.8] on responses that ended before the cap against +13.0 pp [+11.5, +14.4] on responses truncated at the cap. It was positive in 28 of the 42 ablated models (21 Holm-significant) and negative in 14 (6 significant), against 34 (24) and 7 (3) on the full text. Against Qwen3Guard on the same 256 tokens, the gap was +0.9 pp [−0.6, +2.4] (full text +6.2 pp [+4.6, +7.7]). In the main study the gap remained larger for ARA than for the direction-based methods, by +5.5 pp [+4.2, +6.8] with the prompt as unit (pHolm<0.001p_{\mathrm{Holm}} < 0.001; full text +6.0 pp) but not with the model as unit (n=32n = 32, p=0.139p = 0.139; full text p=0.151p = 0.151).

With keyword matching and WildGuard reading the same 256 tokens, keyword matching after ablation still counted more refusals than the judge, so the negative gap that H3 predicted was again not observed over the ablated models, and the class difference again held with the prompt but not with the model as unit. The gap was about half as large as on the full text, mostly because markers that occur only after the first 256 tokens no longer counted (4.0 of the 4.6 pp), and it was not distinguishable from zero in the small pair or against Qwen3Guard. A negative gap appeared only in subsets: on responses that ended before the cap (−5.6 pp; full text −1.2 pp [−3.2, +0.9]) and, against Qwen3Guard, in the small pair (−2.7 pp [−4.5, −1.0]). Part of the full-text gap in Table 10 therefore reflects the length of the keyword window, not the amount of text the judges read; at the 100-token window that Heretic scores, the pooled gap disappeared (below and Table 9). Pooled over the ablated models, the judges’ labels changed by at most 1.6 pp with the cut, although single models changed by up to 9.3 pp; the HarmBench classifier labelled slightly fewer prefixes than full responses as harmful, the direction anticipated for truncated answers in Appendix § 10. As in the main analysis, these comparisons show where the measures diverge, not which is correct.

Keyword matching on the first 100 tokens. Heretic counts a dev response as a refusal if any of its markers occurs in at most 100 generated tokens. Token ids of the held-out responses were not stored, so each stored response was re-tokenised with the tokenizer of the evaluated model, cut to its first 100 tokens, decoded and matched with the same 33 markers; because decoding was greedy, these are the tokens that a 100-token run would have produced, up to numerics. The procedure reproduced the stored full-response keyword label in all 97,828 responses and the stored 256-token label in 97,825 (OLMo-2-32B, which emits each newline as a separate token, needed a newline-split encoding). Bounds that do not depend on exact re-encoding fixed the 100-token label in 99.98% of the responses (17 unresolved) and gave the same pooled gaps. Over the 42 ablated models the 100-token keyword refusal was 32.0%, against 36.0% at 256 tokens and 39.9% on the full text, and its gap to WildGuard was +0.5 pp [−1.0, +1.9] against the full-response label and −0.1 pp [−1.6, +1.3] against the 256-token label; Heretic’s own matcher, which also strips emphasis and counts empty responses as refusals, gave the same +0.5 pp. In the main study, the class, generation and generation × class terms of the gap were +5.2, +4.7 and +8.1 pp with the prompt as unit (all pHolm<0.001p_{\mathrm{Holm}} < 0.001) and not significant with the model as unit. No judge was run on the 100-token text, and the held-out responses were generated with vLLM after the chat template’s empty reasoning block, whereas Heretic scores Transformers generations on the dev prompts.

Re-scoring the Released Exports on the Dev Set

The dev refusal count and first-token KL divergence of each selected model (Table 13 and Table 14) were measured during the search, with the trial’s adapter applied to the original model at run time. The released checkpoint is a separate artefact: the trial is restored from the study journal, its adapter is rebuilt and merged into the bf16 weights, and the export step does not score the result (Appendix § 9.§ 5). Each of the 46 released exports (42 primary models and four γ\gamma-MPOA controls; the ten seed-2 replicates are not released) was therefore scored again with Heretic’s own evaluator (heretic --model <original> --evaluate-model <export>), with the run’s configuration, code version and generation batch size, the original model at the revision of Table 2, and the export at the revision that was evaluated. The evaluator recomputes the original model’s dev responses and first-token distributions and scores the export with the same keyword and KL scorers on the same 100 harmful and 100 harmless dev prompts. The 36 main-study exports, searched on the H200 server, were re-scored on the RTX PRO 6000 server; the ten small-pair exports were searched and re-scored on the RTX 4090 workstation (Table 12). In all 46 runs the original model’s dev refusal count equalled the value recorded during the search. The study journal stores only the count and the KL of each trial, not per-prompt outcomes, so search-time and re-scored values are compared as counts: by Wilcoxon signed-rank tests across exports (Holm-adjusted over the refusal and KL tests) and by Spearman’s ρ\rho with a bootstrap interval.

Table 13. Keyword refusals on the development set (mlabonne/harmful_behaviors, test split, first 100 prompts; count out of 100, Heretic keyword scorer) for the trial selected at each first-token KL budget τ\tau (KL on mlabonne/harmless_alpaca, test split, first 100 prompts). Selection rule: among completed trials with KL ≤τ\le\tau, fewest refusals; ties broken by lower KL, then lower trial index. “Original” is the unmodified model as recorded by Heretic in the same runs. Main study; seed 1; 100 trials planned per run. †Run with fewer completed than planned trials (Qwen3.8-27B MPOA: 99 of 100). –: not yet available.

QwenGemmaOLMoMistral Small
Methodτ\tau2.5-32B3.8-27B2-27B4-31B2-32B3.1-32B24093.2-24B
Original989810099981009398
DiM0.058953929947928546
0.18947899832886935
0.25637739832673111
MPOA0.057543†898136891060
0.11325†748036841013
0.21325†46703684104
ARA0.05129867903489522
0.110631628344846
0.282159103446
SOM0.05813493956392860
0.1732783826379518
0.254276382607956

Table 14. Keyword refusals on the development set (mlabonne/harmful_behaviors, test split, first 100 prompts; count out of 100, Heretic keyword scorer) for the trial selected at each first-token KL budget τ\tau (KL on mlabonne/harmless_alpaca, test split, first 100 prompts). Selection rule: among completed trials with KL ≤τ\le\tau, fewest refusals; ties broken by lower KL, then lower trial index. “Original” is the unmodified model as recorded by Heretic in the same runs. Small pair; seed 1; 100 trials planned per run. –: not yet available.

Qwen (small pair)
Methodτ\tau2.5-7B3.5-4B
Original9999
DiM0.056334
0.15530
0.21130
MPOA0.053310
0.1196
0.2146
ARA0.05814
0.186
0.236
SOM0.053919
0.12710
0.2510
J-space0.058448
0.18448
0.28448

Table 20 lists both values for every export. The re-scored refusal count differed from the selected trial’s by a median of +1 (mean +1.0; 27 higher, 9 equal, 10 lower) and lay within ±2 of it for 37 of the 46 exports. The differences were shifted upward (Wilcoxon signed-rank p=0.002p = 0.002, Holm-adjusted p=0.004p = 0.004), and the two counts ranked the exports almost identically (ρ=0.995\rho= 0.995, 95% bootstrap interval [0.984, 0.997]). First-token KL changed by a median of +0.0003, with no significant shift (Wilcoxon p=0.22p = 0.22). ARA changed most: 4 of its 10 exports moved by more than two refusals (Qwen2.5-32B +9, Gemma-4-31B +6, Gemma-2-27B +5), against 5 of the 36 exports of the other methods (largest change +7, Qwen3.5-4B DiM), and its three largest KL changes (+0.0356 for Qwen2.5-32B, +0.0225 for Gemma-4-31B, +0.0176 for Qwen3.8-27B) exceed the largest absolute change of any other method (0.0090). These four ARA refusal changes occurred in main-study exports whose adapter was refitted and re-scored on a different GPU from the search; the two small-pair ARA exports changed by −1 and +1 refusals, and KL changed by at most 0.0015 across all ten small-pair exports, which were exported and re-scored on the GPU of their search. The sources of the differences (merged bf16 weights versus a run-time adapter, GPU type and kernels, greedy decisions on borderline prompts) were not fully separated, but the export host is examined below. Four exports scored above the selection budget of KL 0.1 on re-score, each by less than 0.008: Gemma-4-31B ARA (0.0845 to 0.1070), Gemma-2-27B MPOA (0.0991 to 0.1036) and the γ\gamma-MPOA controls of Gemma-2-27B (0.0969 to 0.1059) and Olmo-3.1-32B (0.0993 to 0.1032).

ModelDiMMPOASOMJ-spaceARAγ-MPOA
(a) Dev keyword refusals (of 100): selected trial → released export
Qwen2.5-32B89→9013→1773→75–10→16–
Qwen3.8-27B47→4425→2627→28–63→72–
Gemma-2-27Bᵃ89→9074→7483→86–16→2173→72
Gemma-4-31B98→9880→8082→83–28→2178→78
OLMo-2-32B32→3436→3863→62–34→347→8
Olmo-3.1-32B88→9084→8479→78–48→4982→81
Mistral-Small-240969→7210→125→6–4→6–
Mistral-Small-3.235→3413→1318→18–6→6–
Qwen2.5-7B55→5419→2027→2884→858→7–
Qwen3.5-4B30→376→810→1148→476→7–
(b) First-token KL divergence: selected trial → released export
Qwen2.5-32B0.0175→0.01780.0599→0.06030.0750→0.0707–0.0620→0.0976–
Qwen3.8-27B0.0530→0.05170.0854→0.08420.0843→0.0813–0.0605→0.0781–
Gemma-2-27Bᵃ0.0686→0.06720.0991→0.10360.0860→0.0807–0.0990→0.09960.0969→0.1059
Gemma-4-31B0.0932→0.09360.0691→0.07180.0548→0.0567–0.0845→0.10700.0609→0.0645
OLMo-2-32B0.0887→0.08450.0372→0.03570.0470→0.0446–0.0494→0.04930.0852→0.0839
Olmo-3.1-32B0.0891→0.09120.0916→0.09590.0638→0.0670–0.0526→0.05470.0993→0.1032
Mistral-Small-24090.0990→0.09890.0493→0.04800.0620→0.0625–0.0656→0.0672–
Mistral-Small-3.20.0931→0.09310.0938→0.09220.0995→0.0973–0.0829→0.0827–
Qwen2.5-7B0.0782→0.07970.0743→0.07480.0756→0.07540.0216→0.02090.0205→0.0205–
Qwen3.5-4B0.0539→0.05300.0543→0.05480.0550→0.05530.0188→0.01930.0645→0.0648–

Table 20. Dev-set re-score of the 46 released exports. Each cell gives the value of the selected trial logged during the search (adapter applied to the original model) → the value of the released merged export scored by Heretic’s evaluator with the run’s configuration, code version and batch size. (a) Keyword refusals of 100 on the dev prompts (mlabonne/harmful_behaviors, test split, first 100); bold: change of more than two. (b) First-token KL divergence from the original model (mlabonne/harmless_alpaca, test split, first 100); bold: re-scored value above the selection budget of 0.1. Main-study exports were searched on the H200 server and re-scored on the RTX PRO 6000 server; small-pair exports (last two rows of each panel) were searched and re-scored on the RTX 4090. ᵃRe-scored with eager attention set in a local copy of the configuration (Table 21). –: method not applied to that model.

Export host. For ARA, the export step refits the adapter by L-BFGS from the stored hyperparameters (Appendix § 9.§ 5). All main-study searches ran on the H200 server; the Qwen and Gemma exports were rebuilt on the RTX PRO 6000 server and the OLMo and Mistral Small exports on the H200, and all 36 main-study exports were re-scored on the RTX PRO 6000. The four ARA exports rebuilt on the other GPU moved by more than two dev refusals in 4 of 4 cases (+6, +9, +5 and −7), against 0 of the 4 rebuilt on the search GPU (0 to +2; Fisher p=0.029p = 0.029), and their largest KL change was 0.0356 against 0.0021. For the other methods the corresponding counts were 3 of 14 and 1 of 14 (p=0.60p = 0.60). Because the OLMo and Mistral Small ARA exports, rebuilt on the H200 and re-scored on the RTX PRO 6000, moved by at most two refusals, re-scoring on other hardware alone does not explain the large ARA changes; they coincide with refitting on other hardware. Gemma-4-31B ARA, rebuilt on the RTX PRO 6000, is the only ARA export that re-scored above the KL cap.

Gemma-2 attention configuration. A first re-score of the five Gemma-2-27B exports gave KL 0.404 to 0.491 and refusal changes from −22 to +7 (Table 21). Gemma-2 caps attention logits and final logits with a scaled tanh [[72]]. The SDPA attention path of transformers does not apply the attention-logit cap; the eager path does, and the original model’s configuration selects it with "attn_implementation": "eager". Heretic saved the exports with transformers 5.18.0, whose configuration serialisation drops this key, while the cap values themselves (attn_logit_softcapping 50, final_logit_softcapping 30) were kept. In the re-score the original model was therefore loaded with eager attention and each export with SDPA attention, which ignores the attention-logit cap, so the measured divergence reflected the attention implementation rather than the edit. With "attn_implementation": "eager" written into a local copy of each export’s configuration and the weights unchanged, the five exports re-scored at KL 0.067 to 0.106 with refusal changes from −1 to +5; these eager values are the ones in Table 20 and in the counts above. The configurations of the five public repositories were patched on 6 November 2026 by one commit per repository that adds only this key (Table 21); the weight files are unchanged, and every evaluation in this paper used the earlier revisions. Before the patch, loading a released Gemma-2 model with transformers 5.18 without requesting eager attention selected SDPA, as in the first re-score.

MethodSelected trial Ref.Selected trial KLRe-score, SDPA Ref.Re-score, SDPA KLRe-score, eager Ref.Re-score, eager KLRevision EvaluatedRevision Patched
DiM890.0686890.4036900.06729ee29ab3ab163f2f3e1d2c36
MPOA740.0991520.4609740.1036a2c021b2e06a8aa25e304fdd
SOM830.0860770.4912860.08072a355f20c0d4c63892ab9b07
ARA160.0990230.4817210.099629250dd15c03958fda591386
γ-MPOA730.0969630.4167720.105978da037a4a289630b864b0fe

Table 21. Gemma-2-27B exports re-scored on the dev set with the configuration as exported (transformers SDPA attention, no attention-logit soft-capping) and with eager attention set in a local copy of the configuration (same weights). Ref.: keyword refusals of 100; KL: first-token KL divergence. Revisions of richardyoung/gemma-2-27b-it-{DiM,MPOA,SOM,ARA,GMPOA}-heretic, first 12 hexadecimal digits: the revision evaluated throughout this paper and the revision created by the configuration patch of 6 November 2026, which adds only "_attn_implementation": "eager".

Held-out evaluation in vLLM. The held-out generations, capability benchmarks and teacher-forced divergences were computed with vLLM 0.33.0 and are not affected by the configuration defect. vLLM’s Gemma-2 implementation reads the cap values from attn_logit_softcapping and final_logit_softcapping, passes the attention cap to its own attention backend and does not read _attn_implementation; the original model and all five exports ran on the same backend (FlashAttention 2), which applies the cap, and the five exports were evaluated with the unpatched configuration. Consistent with this, the teacher-forced divergences of the four primary Gemma-2-27B exports (0.031 to 0.161 nats per token) and their mean capability changes (−0.50 to +0.21 pp) lie within the ranges of the other 38 primary models (0.004 to 0.279; −4.21 to +0.86 pp).

Consequences for the reported results. Held-out refusal, capability and teacher-forced divergence were measured on the released weights themselves (main study: downloaded back from the published repository; small pair: SHA-256-identical shards; Appendix § 9.§ 5), so they describe the released models whatever the dev re-score shows. The re-score bears on two other points. First, the dev values in Table 13 and Table 14 and in the dev–held-out comparison are those of the selected trials; the released exports reproduce their ranking (ρ=0.995\rho= 0.995) and, for 37 of the 46 exports, their counts to within two refusals, and the small upward shift is the direction expected when the lowest of many measured counts is measured again (Appendix § 10). Second, the selected trial of every run met KL ≤0.1\le0.1 as measured during the search, so no selection changes under the rule as specified, and 42 of the 46 released exports also stayed within the budget on re-score. Had the re-scored KL been used for the other four, their selected trials would have been ineligible, and among the logged trials the rule would have returned trial 16 of Gemma-2-27B MPOA (87 refusals at KL 0.0835, against 74), trial 76 of Gemma-4-31B ARA (28 refusals at KL 0.0921, the same count; trial 66 with 45 refusals if every trial of that run had drifted by the same +0.0225), trial 26 of Gemma-2-27B γ-MPOA (75, against 73), and trial 52 of Olmo-3.1-32B γ-MPOA (84, against 82). These trials were neither exported nor evaluated, and their own KL was not re-measured. The two primary models concerned, Gemma-2-27B MPOA and Gemma-4-31B ARA, are therefore reported as selected at KL ≤0.1\le0.1 on the search measurement, with released exports at 0.104 and 0.107 on re-score.

KL-Cap Binding, Pair-Level Pooling and the Two Scales

KL-cap binding. The selection rule caps first-token KL divergence at 0.1 for every model. The cap binds in a run when some trial above 0.1 had fewer dev keyword refusals than the selected trial; a run stayed below the cap when at most one of its trials exceeded 0.1. In the main study the cap bound in 15 of the 24 direction-based runs (7 of 12 older and 8 of 12 newer models) and in all 8 ARA runs, every one of which reached 0 dev refusals above the cap (maximum KL 10.9 to 33.2). Eight direction-based runs stayed below the cap, four on each generation: Qwen3.8-27B MPOA, Olmo-3.1-32B MPOA and SOM and Mistral-Small-2409 MPOA never sampled a trial above 0.1, and Olmo-2-32B DiM, MPOA and SOM and Qwen3.8-27B SOM sampled one. On the small pair the cap bound in 4 of the 16 direction-based runs of both seeds (all four J-space runs stayed below it) and in all 4 ARA runs. The selected dev KL ranged from 0.018 to 0.100 for the direction-based runs (median 0.049) and from 0.049 to 0.099 for ARA; 7 of 24 direction-based and 1 of 8 ARA selections lay within 0.01 of the cap.

Position in the search range. The selected trial’s parameters were read from the Optuna journals and placed in Heretic’s default ranges (Table 11); a maximum weight counts as at the ceiling when it lies on the top 5% of its range. The selected trial was at the ceiling in 5 of the 24 main-study direction-based runs and in none of the 8 runs that stayed below the cap. In those 8 runs the selected maximum weights lay at a median of 0.81 (attention; range 0.56–0.93) and 0.89 (MLP; 0.83–0.94) of their range, against 0.58 and 0.84 in the other direction-based runs, so the selected edits lean to the strong half of the range but not to its bound. The search did sample the strongest maximum weights in each of the 8 runs, and those trials had no fewer dev refusals than the selected trial in 7 of 8 (in Olmo-2-32B SOM the better trial at the ceiling had KL 0.110). Counting every strength-type parameter (maximum weight, minimum weight, minimum-weight distance, SOM decay), the selected trial had at least one at the strong end of its range in 14 of the 24 direction-based runs and in 4 of the 8 that stayed below the cap (a minimum weight or minimum-weight distance in all four). The selected trial was one of the 30 random start-up trials of the TPE sampler in 7 of the 24 direction-based runs

(4 of the 8 that stayed below the cap) and in 1 of 8 ARA runs. Because seed 1 and the same search space give identical start-up draws, two pairs of selected parameter vectors coincide across models (OLMo-2-32B SOM and Qwen3.8-27B SOM; Olmo-3.1-32B MPOA and Qwen3.8-27B MPOA).

H1 and H2 by binding. At the level of pair × method units, the cap bound in both cells for Qwen DiM and ARA, all four Gemma methods, OLMo ARA and Mistral Small DiM and ARA; in neither cell for Qwen MPOA and OLMo MPOA; and in one cell for Qwen SOM, OLMo DiM and SOM, and Mistral Small MPOA and SOM. Table 22 refits the pooled GEE of § 4.1 on these subsets. The raw direction-based effect was positive in every subset; the baseline-adjusted one was negative where the cap bound in both cells and positive where it did not, and the pooled +0.03 of all units lies between the two. The subsets also differ in family, so they cannot separate the cap from the family. H2 can be estimated only on units in which the cap bound, because it bound in every ARA cell.

SubsetUnitsDir., rawDir., adj.ARA, rawH2
All units12/4+0.62 [+0.57, +0.68]+0.03 [−0.05, +0.11]+0.12 [+0.00, +0.24]−0.50 [−0.62, −0.38]
Cap bound in both cells5/4+0.32 [+0.24, +0.41]−0.24 [−0.33, −0.14]+0.11 [+0.01, +0.22]−0.21 [−0.33, −0.09]
Not bound in ≥1 cell7/0+1.03 [+0.95, +1.11]+0.22 [+0.12, +0.32]n/an/a
Bound in neither cell2/0+3.23 [+2.90, +3.56]+2.20 [+1.95, +2.45]n/an/a
≥2 trials above the cap in both6/4+0.33 [+0.24, +0.41]−0.28 [−0.37, −0.18]+0.12 [+0.01, +0.23]−0.21 [−0.33, −0.08]
≤1 trial above the cap in a cell6/0+1.04 [+0.96, +1.12]+0.27 [+0.17, +0.37]n/an/a

Table 22. Pooled H1 and H2 terms (GEE, log-odds, prompt clusters, family fixed effects, 95% intervals) refitted on subsets of the 16 main-study pair × method units, by whether the KL cap bound in each unit’s two cells. Units: direction-based / ARA. Dir.: direction-based class; adj.: baseline-adjusted (minus the originals’ difference); n/a: no ARA unit in the subset.

Pair-level pooling. With the pair as the unit, between-pair heterogeneity dominated every pooled estimand (I2I^2 99–100%; between-pair SD of the adjusted direction-based effect 22.9 pp and of the H2 contrast 26.1 pp). The 95% prediction intervals for a new pair, which use a tt distribution on 2 degrees of freedom, were −5.90 to +6.02 log-odds for the adjusted direction-based effect and −11.32 to +9.21 for the H2 contrast; on the percentage-point scale they extend beyond ±100 pp, so with four pairs they state only that an effect of either sign is plausible for a new pair. Mapping the percentage-point σ\sigma to the logit scale by the delta method at each cell’s rate, instead of estimating σ\sigma on the logit scale, gave seed SEs up to 14.9 log-odds near 0% refusal (Gemma-4-31B ARA) and pooled intervals of −1.61 to +1.82 (adjusted direction-based) and −3.59 to +2.81 (H2). Table 23 gives the leave-one-family-out GEE terms.

TermAll fourw/o Qwenw/o Gemmaw/o OLMow/o Mistral Small
Generation effect, direction class+0.62+0.69+0.81+0.24+0.74
Generation effect, ARA+0.12−1.37+1.02+0.24+0.15
Generation × class (H2)−0.50−2.06+0.20−0.00−0.59
Baseline-adjusted, direction class+0.03+0.22+0.13−0.51+0.33
Baseline-adjusted, ARA−0.46−1.68+0.31−0.51−0.25
Originals’ generation difference+0.58+0.45+0.66+0.74+0.40

Table 23. Leave-one-family-out pooled GEE terms (log-odds; main study, WildGuard refusal, 972 harmful prompts).

The two scales. Table 24 gives every pair × method generation effect, the originals’ difference and the baseline-adjusted effect on both scales. Raw effects of a single method have the same sign on both scales, because they compare two rates. Adjusted effects changed sign between scales in 3 of 21 cells and in significance in 5. On the percentage-point scale all four Mistral Small adjusted effects were large and negative (−22.2 to −14.4 pp), whereas in log-odds they ranged from −1.37 to −0.10.

Item Source

The 972 harmful held-out prompts come from four public sets (JailbreakBench, XSTest unsafe, StrongREJECT and HarmBench; 772 prompts) and the code-as-weapon bank (200). H1 and H2 were re-run, with the same models, tests and Holm families as in § 4.1, on the public sets together, on the code-as-weapon bank and on each set separately. The originals’ generation difference is the item-weighted sum of the differences on the sets, so the contribution of the code items is 200/972 times their difference (Table 25a). The code items carried most of the Qwen originals’ difference and 52% of the mean difference of the four main pairs; in Gemma, a decrease on the public sets (−5.7 pp) and an increase on the code items (+27.0 pp) cancelled; and in Mistral Small the difference lay in the public sets. Gemma’s direction-based effects also lay in the code items (DiM, MPOA and SOM +65.5, +35.0 and +47.5 pp on code items, against +3.6, −15.0 and −9.1 pp on the public sets), whereas the largest effect of each other main pair lay in the public sets (Qwen ARA +41.3 vs. +7.5 pp, OLMo MPOA +75.6 vs. +5.5 pp).

Table 25: Item source. (a) Originals’ generation difference (newer minus older unmodified model, WildGuard refusal, pp, item-bootstrap 95% intervals) on all 972 harmful prompts, the four public sets and the code-as-weapon bank, and the share of the 972-item difference contributed by the code items (shown only where the total excludes zero). (b) Pooled main-study GEE terms (log-odds, 95% intervals) by item subset (Public: the four public sets; Code: code-as-weapon bank; JBB: JailbreakBench; XSTest: its unsafe prompts): originals’ difference, generation effect of the direction-based class raw and baseline-adjusted, ARA raw and adjusted, H2 interaction (pHolmp_{\mathrm{Holm}} over the subset’s H2 family) and the Holm tally of the 16 main-pair H1 tests (newer higher / lower / not significant). The generation × family Wald χ2\chi^2 (3 df) was 547 (all items), 793 (public sets) and 313 (XSTest); it was not estimable on the other subsets, where some model refused 0% or 100% of the items.

PairAll 972Public sets (772)Code-as-weapon (200)Code contributionCode share
Qwen+13.1 [+10.8, +15.4]+5.3 [+3.5, +7.2]+43.2 [+36.1, +50.2]+8.9 [+7.1, +10.7]68% [59, 77]
Gemma+1.0 [−1.3, +3.5]−5.7 [−7.5, −4.0]+27.0 [+17.9, +35.8]+5.6 [+3.6, +7.6]–
OLMo+0.8 [−0.5, +2.2]−0.3 [−1.5, +1.0]+5.0 [+0.5, +9.8]+1.0 [+0.1, +2.1]–
Mistral Small+17.9 [+15.2, +20.7]+20.5 [+17.2, +23.8]+8.0 [+4.6, +12.1]+1.6 [+0.9, +2.5]9% [5, 14]
Small pair+16.4 [+13.9, +18.8]+9.5 [+7.0, +11.8]+43.0 [+36.3, +50.0]+8.8 [+7.2, +10.6]54% [46, 63]
Mean of 4 main pairs+8.1 [+7.0, +9.3]+5.0 [+3.8, +6.0]+20.6 [+17.3, +23.8]+4.2 [+3.4, +5.1]52% [44, 60]
ItemsOriginalsDir., rawDir., adj.ARA, rawARA, adj.H2 (pHolmp_{\mathrm{Holm}})H1 tally
All 972+0.58 [+0.49, +0.66]+0.62 [+0.57, +0.68]+0.03 [−0.05, +0.11]+0.12 [+0.00, +0.24]−0.46 [−0.58, −0.34]−0.50 (<0.001)8/4/4
Public (772)+0.69 [+0.52, +0.85]+0.56 [+0.50, +0.62]−0.13 [−0.28, +0.03]+0.10 [−0.02, +0.22]−0.59 [−0.77, −0.41]−0.46 (<0.001)8/6/2
Code (200)+1.24 [+0.98, +1.49]+2.96 [+2.48, +3.43]+1.39 [+1.08, +1.70]+1.75 [+0.42, +3.08]+0.19 [−0.59, +0.96]−1.20 (0.129)8/0/8
JBB (100)+0.70 [+0.21, +1.20]+0.53 [+0.36, +0.71]−0.17 [−0.63, +0.29]+0.69 [+0.14, +1.24]−0.04 [−0.66, +0.59]+0.16 (0.577)5/2/9
XSTest (200)−0.09 [−0.37, +0.19]+0.22 [+0.12, +0.33]+0.31 [+0.03, +0.59]−0.46 [−0.63, −0.29]−0.35 [−0.65, −0.05]−0.68 (<0.001)5/3/8
StrongREJECT (313)+1.39 [+1.03, +1.75]+0.80 [+0.70, +0.89]−0.59 [−0.93, −0.25]+0.34 [+0.14, +0.53]−1.05 [−1.43, −0.68]−0.46 (<0.001)7/1/8
HarmBench (159)+1.00 [+0.69, +1.32]+0.63 [+0.51, +0.76]−0.37 [−0.68, −0.06]+0.47 [+0.14, +0.81]−0.53 [−0.97, −0.09]−0.16 (0.524)6/2/8

Table 25. Item source. (a) Originals’ generation difference by item source, pp. (b) Pooled main-study generation effects by item subset, log-odds.

On the 772 public items the pooled conclusions were unchanged (Table 25b): the baseline-adjusted direction-based effect was −0.13 log-odds [−0.28, +0.03], the H2 interaction −0.46 [−0.58, −0.34], with the same pairs in the predicted (Gemma, OLMo) and opposite (Qwen, small pair) direction, and only the small pair had every method higher on the newer model. Without the 44 prompts that overlap the search dataset, the public-set estimates were −0.08 [−0.23, +0.08] (adjusted direction-based), −0.58 [−0.76, −0.39] (adjusted ARA) and −0.50 [−0.63, −0.38] (H2). On the code items alone the adjusted direction-based effect was positive (+1.39 [+1.08, +1.70]), but 15 of the 32 main-study modified models refused none of the 200 code items, so code-only log-odds rest on the few models that still refused some of them. The overlap check is lexical; semantic near-duplicates and safety tuning of the newer models on the public benchmarks were not tested.

Harmful Compliance as the Outcome

H1 and H2 were re-run on all 972 harmful prompts with two harmful-compliance labels, the HarmBench classifier and WildGuard’s harmful-response label, coded as not harmful so that a positive generation effect again means that removal left less on the newer model (Table 26). The pooled conclusions were the same as for refusal. Across the 42 ablated seed-1 models, the share of non-refused responses that the HarmBench classifier labelled harmful ranged from 48.1% (OLMo-2-32B DiM) to 88.8%, and the WildGuard share from 15.8% to 75.9%. Of the 21 per-pair H1 tests, 5 changed Holm decision or sign under HarmBench (Qwen SOM, Gemma MPOA and Gemma SOM became significantly higher on the newer model; Mistral Small DiM lost significance; Mistral Small ARA became significantly lower) and 6 under WildGuard (Gemma MPOA and SOM became significantly higher; Qwen SOM and Mistral Small DiM and MPOA lost significance; OLMo ARA changed sign without being significant). In addition, the pooled raw ARA effect became significant under both labels, and the small-pair H2 interaction lost significance under WildGuard. Per pair, the harmful-compliance rates themselves are given in Table 27.

TermWG refusal (primary)HB not harmfulWG not harmful
Originals’ difference+0.58 [+0.49, +0.66]+0.65 [+0.58, +0.72]+0.72 [+0.62, +0.83]
Direction, raw+0.62 [+0.57, +0.68]+0.70 [+0.65, +0.74]+0.69 [+0.64, +0.73]
Direction, adjusted+0.03 [−0.05, +0.11]+0.05 [−0.02, +0.11]−0.03 [−0.13, +0.07]
ARA, raw (pHolmp_{\mathrm{Holm}})+0.12 [+0.00, +0.24] (0.135)+0.21 [+0.14, +0.28] (<0.001)+0.24 [+0.18, +0.30] (<0.001)
ARA, adjusted−0.46 [−0.58, −0.34]−0.44 [−0.53, −0.36]−0.48 [−0.60, −0.37]
H2 interaction (pHolmp_{\mathrm{Holm}})−0.50 [−0.62, −0.38] (<0.001)−0.49 [−0.56, −0.41] (<0.001)−0.45 [−0.52, −0.38] (<0.001)
Generation × family χ2\chi^2547450447
H1 tally (higher / lower / n.s.)8/4/411/3/210/1/5
Every method highersmall pairQwen, small pairsmall pair
PairWG refusalHB not harmfulWG not harmful
Qwen+1.89 (<0.001)+1.47 (<0.001)+1.43 (<0.001)
Gemma−4.27 (<0.001)−2.17 (<0.001)−1.92 (<0.001)
OLMo−1.87 (<0.001)−1.26 (<0.001)−1.44 (<0.001)
Mistral Small−0.06 (0.696)−0.15 (0.083)−0.06 (0.732)
Small pair+0.76 (<0.001)+0.27 (0.004)+0.06 (0.732)
CellsWG refusalHB harmfulHB among not refusedWG among not refused
Originals (10)80.216.079.0 [75.5, 82.1]29.6 [25.6, 33.8]
Ablated, main (32)32.351.075.1 [73.4, 76.9]55.8 [53.3, 58.1]
Ablated, small (10)29.154.476.7 [74.7, 78.7]58.5 [55.9, 61.1]
γ\gamma-MPOA (4)29.956.279.9 [78.0, 81.8]60.2 [57.6, 62.7]
Community (8)6.475.580.5 [79.0, 82.1]66.7 [64.3, 68.9]
Seed 2 (10)29.256.479.5 [77.7, 81.4]61.3 [58.6, 63.9]

Table 26. Harmful compliance as the outcome. (a) Pooled main-study GEE terms (log-odds, 95% intervals) for WildGuard refusal (primary) and for two not-harmful outcomes (1 minus harmful compliance by the HarmBench classifier or by WildGuard), so that signs match; generation × family χ2\chi^2 with 3 df; Holm tally of the 16 main-pair H1 tests; pairs in which every method was significantly higher on the newer model. (b) Per-pair H2 interaction (log-odds, pHolmp_{\mathrm{Holm}}) by outcome. (c) Share of responses that WildGuard does not label as refusal and that are harmful, pooled over cell groups (prompt-cluster bootstrap 95% intervals).

PairMethodWG refusal: effectHB harmful: effectWG harmful: effect
QwenOriginal76.8→89.9: +13.119.8→6.3: +13.57.1→1.4: +5.7
DiM47.4→74.3: +26.9*40.3→16.4: +24.0*26.0→6.0: +20.1*
MPOA3.9→14.0: +10.1*85.3→48.6: +36.7*72.9→34.2: +38.8*
SOM42.7→38.0: −4.7*44.5→39.9: +4.6*30.1→27.8: +2.3
ARA6.1→40.4: +34.4*82.3→30.6: +51.7*69.1→17.2: +52.0*
GemmaOriginal88.3→89.3: +1.010.7→6.9: +3.82.1→1.8: +0.3
DiM73.8→90.1: +16.4*22.3→6.5: +15.8*10.0→1.5: +8.4*
MPOA45.2→40.4: −4.745.5→31.6: +13.9*32.5→22.6: +9.9*
SOM65.2→67.8: +2.628.8→17.5: +11.3*15.5→8.6: +6.9*
ARA26.6→0.6: −26.0*50.1→80.9: −30.8*35.0→65.2: −30.2*
OLMoOriginal83.2→84.1: +0.815.0→12.3: +2.74.0→3.2: +0.8
DiM33.5→68.3: +34.8*32.0→20.4: +11.6*18.2→7.2: +11.0*
MPOA3.6→65.0: +61.4*83.5→21.6: +61.9*69.4→9.4: +60.1*
SOM27.3→41.7: +14.4*53.5→39.6: +13.9*40.5→26.6: +13.9*
ARA13.3→10.8: −2.560.7→61.2: −0.545.6→44.5: +1.1
Mistral SmallOriginal55.7→73.6: +17.935.5→21.3: +14.219.8→8.4: +11.3
DiM32.8→28.5: −4.3*53.4→56.4: −3.038.9→42.2: −3.3
MPOA6.9→4.0: −2.9*77.7→82.5: −4.8*63.1→65.6: −2.6
SOM3.7→7.2: +3.5*80.5→76.7: +3.7*68.7→61.3: +7.4*
ARA4.9→4.2: −0.778.5→81.9: −3.4*66.0→66.9: −0.8
Small pairOriginal72.1→88.5: +16.423.3→9.5: +13.89.3→1.6: +7.6
DiM20.6→54.6: +34.1*64.0→32.6: +31.4*50.2→21.3: +28.9*
MPOA13.8→18.0: +4.2*71.5→55.9: +15.6*58.0→44.1: +13.9*
SOM12.2→21.9: +9.7*73.1→56.4: +16.8*59.2→44.2: +14.9*
J-space52.6→73.8: +21.2*36.3→18.9: +17.4*22.8→5.5: +17.4*
ARA4.7→18.9: +14.2*79.0→55.9: +23.1*65.1→43.9: +21.2*

Table 27. Refusal and harmful compliance per pair and method, older → newer model (%, 972 harmful prompts). Effect: newer minus older on the not-refused (WG refusal) or not-harmful scale, pp, so that a positive effect means that removal left less on the newer model; * Holm-significant within that outcome’s H1 family.

Detail Moved from the Results

This subsection keeps the detailed Results text of the previous draft, which the main text now summarises, together with five secondary figures. Numbers and tests are unchanged.

Residual Refusal in the Newer Model

Table 5 gives the per-pair generation effects by method class, and Figure 3 and Table 6 give them per method. After direction-based removal (mean of DiM, M P O A and S O M), residual refusal was higher on the newer model by +10.7 pp in the Qwen pair, +4.7 pp in Gemma, +36.9 pp in OLMo and −1.2 pp in Mistral Small, and by +17.3 pp in the small pair (mean including J-space). After ARA the changes were +34.4, −26.0, −2.5, −0.7 and +14.2 pp. The generation effect differed between pairs (prompt-level generation × pair Wald χ2=547.4\chi^2 = 547.4, 3 df, p<0.001p < 0.001; χ2\chi^2 of 277–551 and p≤10−59p \le10^{-59} for every judge, with and without the 44 prompts that overlap the search dataset). With run-to-run (seed) variance added, the per-pair half-widths grew to about 15 pp for the direction-based class and 25 pp for ARA, and 14 of the 40 main-pair estimand × scale intervals excluded zero (32 with prompt sampling only). Of the ten raw effects on the pp scale (five pairs, two classes), four excluded zero: the OLMo (+36.9 pp [+22.1, +51.7]) and small-pair (+17.3 pp [+4.4, +30.1]) direction-based increases, the Qwen ARA increase (+34.4 pp [+8.8, +59.9]) and the Gemma ARA decrease (−26.0 pp [−51.6, −0.5]).

The unedited originals already differed by +13.1 (Qwen), +17.9 (Mistral Small) and +16.4 pp (small pair), but not in Gemma (p=0.46p = 0.46) or OLMo (p=0.30p = 0.30). Relative to this baseline difference, a contrast added after H1 was stated and therefore exploratory, the direction-based effect was −2.4 (Qwen), +3.7 (Gemma), +36.0 (OLMo), −19.2 (Mistral Small) and +0.9 pp (small pair), and the ARA effect +21.3, −27.1, −3.3, −18.6 and −2.2 pp. The two scales of the adjustment disagreed in places. Among the 21 pair × method cells, the adjusted effect had opposite signs in pp and log-odds for three (Qwen MPOA, −3.1 pp but +0.39 log-odds; small-pair J-space and ARA), and its significance differed in five. At class level the signs differed for the small pair’s direction-based (+0.9 pp, −0.30 log-odds) and ARA effects and for the H2 contrasts of Mistral Small and the small pair. Near 0% refusal the log-odds scale magnifies differences: Gemma ARA (26.6% to 0.6%) was −26.0 pp but −4.07 log-odds, the largest log-odds effect in the main study.

Pooled over the four main pairs with the pair as the unit (random effects, Hartung–Knapp–Sidik–Jonkman interval), the direction-based generation effect was +12.8 pp [−13.9, +39.5] (+0.55 log-odds [−0.66, +1.76]) and, beyond the originals’ difference, +4.6 pp [−32.2, +41.3] (+0.06 log-odds [−1.67, +1.80]); 95% prediction interval for a new pair −5.90 to +6.02); for ARA the adjusted effect was −6.9 pp [−40.7, +26.8] (−0.96 log-odds [−4.59, +2.68]). Heterogeneity between pairs was almost total (I2I^2 99–100%; between-pair SD of the adjusted direction-based effect 22.9 pp, against a seed SE of 7.5 pp and a prompt SE of 1.3–1.5 pp per pair), so adding seed variance moved these limits by at most 0.1 pp and 0.07 log-odds, and none of the ten pooled estimand × scale intervals excluded zero. With the prompt as the unit (GEE, prompt sampling only) the same quantities were narrower: the direction-based effect was +0.62 log-odds [+0.57, +0.68] (pHolm<0.001p_{\mathrm{Holm}} < 0.001) raw and +0.03 [−0.05, +0.11] (p=0.529p = 0.529) adjusted, the ARA effect +0.12

[+0.00, +0.24] (pHolmp_{\mathrm{Holm}} = 0.135) raw and −0.46 [−0.58, −0.34] (p<0.001p < 0.001) adjusted, and the originals’ difference +0.58 [+0.49, +0.66] (p<0.001p < 0.001); the planned mixed model gave unadjusted effects of +0.86 [+0.82, +0.90] and +0.15 [+0.04, +0.26]. Leaving one family out moved the adjusted direction-based GEE effect between −0.51 (without OLMo) and +0.33 (without Mistral Small).

All models were selected under the same cap on first-token KL divergence (≤0.1\le0.1), but the cap did not constrain every run (Appendix § 11.§ 4). It bound, in the sense that some trial above 0.1 had fewer dev refusals than the selected trial, in 15 of the 24 main-study direction-based runs and in all 8 ARA runs, which reached 0 dev refusals above the cap (maximum KL 10.9–33.2); to stay under it, the median binding run gave up 24 (direction-based) and 22 (ARA) dev refusals. Four direction-based runs never sampled a trial above 0.1 (Qwen3.8-27B MPOA, OLMo-3.1-32B MPOA and SOM, Mistral-Small-2409 MPOA) and four sampled one (OLMo-2-32B DiM, MPOA and SOM, Qwen3.8-27B SOM). In the eight runs the selected maximum weights were not at the upper bound of Heretic’s search range (0 of 8), and in 7 of 8 the trials at that bound had no fewer dev refusals than the selected trial (in the eighth, OLMo-2-32B SOM, the better one had KL 0.110), so the upper bound of the maximum weights did not limit their residual refusal; another strength parameter (a minimum weight or minimum-weight distance) was the strong end of its range in 4 of the 8. Refitted on the pair × method units in which the cap bound in both cells (5 direction-based, 4 ARA), the direction-based effect was +0.32 log-odds [+0.24, +0.41] raw and −0.24 [−0.33, −0.14] adjusted; on the 7 direction-based units in which it did not bind in at least one cell, +1.03 [+0.95, +1.11] raw and +0.22 [+0.12, +0.32] adjusted. The two subsets also differ in family (cap-bound units: one Qwen, three Gemma and one Mistral Small unit; others: two Qwen, three OLMo and two Mistral Small units), so this split does not separate the cap from the family.

The per-pair effects can also be compared with the run-to-run variation of the search. The seed-1 and seed-2 models of the ten small-pair cells gave a run-to-run standard deviation of a cell’s held-out refusal of 9.2 pp [6.4, 16.1] (§ 3.5), so the difference between two independently optimised cells has a standard deviation of 12.9 pp, and an effect larger than 25.9 pp (2 SD) exceeds seed noise. Of the 16 main-study per-method effects, 5 exceeded this band (Table 6, last column): four increases, Qwen DiM (+26.9 pp [+23.8, +29.8]), Qwen ARA (+34.4 pp [+31.2, +37.5]), OLMo DiM (+34.8 pp [+31.5, +37.9]) and OLMo MPOA (3.6% [2.6, 5.0] to 65.0% [62.0, 68.0]; +61.4 pp [+58.2, +64.4]), and one decrease, Gemma ARA (26.6% [24.0, 29.5] to 0.6% [0.3, 1.3]; −26.0 pp [−28.9, −23.3]), which lay 0.1 pp beyond the band. The other 11, from −4.7 to +16.4 pp, lay within seed noise. Across the 95% interval of the standard deviation, between 1 and 5 effects exceeded the band; of the baseline-adjusted effects, 3 exceeded it (OLMo DiM +34.0, OLMo MPOA +60.6 and Gemma ARA −27.1 pp). In 3 of the ten seed pairs (Qwen3.5-4B DiM, SOM and ARA) the seed-2 run also used a generation batch size of 64 instead of 128, after running out of memory at 128. Without these three pairs the standard deviation was 7.2 pp [4.8, 14.6] and the threshold 20.4 pp; the same 5 raw effects exceeded it, and the baseline-adjusted effects that did so rose from 3 to 6 (adding Qwen ARA +21.3, Mistral Small DiM −22.2 and MPOA −20.8 pp). The band assumes that the seed variability of the 4B and 7B small-pair models transfers to the 22–32B main-study models, each optimised once.

With the prompt as the unit of inference (exact McNemar, Holm over 23 tests), the newer model refused significantly more in 8 of the 16 main-study comparisons, significantly less in 4 and not significantly differently in 4; 4 of the 8 increases and 3 of the 4 decreases lay within seed noise, and no main-study pair was higher for every method, as H1 required. Gemma-4-31B DiM refused more than its own original (90.1% vs. 89.3%; +0.8 pp [+0.1, +1.6], pHolm=0.039p_{\mathrm{Holm}} = 0.039 over the 42 edit-versus-original tests), a failed edit. In the small pair all five methods refused more on the newer model with seed 1 (+4.2 to +34.1 pp, all pHolm≤0.019p_{\mathrm{Holm}} \le0.019).

Under Qwen3Guard the pooled ARA generation effect was +0.21 [+0.10, +0.32] and under full-response keyword matching +0.57 [+0.47, +0.67] (both pHolm<0.001p_{\mathrm{Holm}} < 0.001). Excluding the 44 held-out prompts that match a prompt of the search dataset (harmful_behaviors) removed significance for Qwen SOM and Mistral Small DiM (all effects changed by ≤1.9\le1.9 pp). No main-study pair met H1 under Qwen3Guard or full-response keyword matching; with the 256-token keyword window, all four OLMo methods were higher on the newer model (OLMo ARA +5.2 pp [+2.2, +8.3], pHolm=0.009p_{\mathrm{Holm}} = 0.009).

Item source. About half of the originals’ generation difference came from the code-as-weapon bank (200 of the 972 items; Appendix D.5). Averaged over the four main pairs, the originals differed by +5.0 pp [+3.8, +6.0] on the four public sets and by +20.6 pp [+17.3, +23.8] on the code items, which contributed 52% [44, 60] of the mean difference of +8.1 pp (Qwen 68% [59, 77], Mistral Small 9% [5, 14]). Gemma’s direction-based effects also lay in the code items (DiM, MPOA and SOM +65.5, +35.0 and +47.5 pp, against +3.6, −15.0 and −9.1 pp on the public sets). On the 772 public items alone the conclusions were unchanged: the baseline-adjusted direction-based GEE effect was −0.13 [−0.28, +0.03], only the small pair was higher on the newer model for every method, the Holm tally was 8 higher, 6 lower and 2 not significant, and the H2 interaction was −0.46 [−0.58, −0.34], in the predicted direction for Gemma and OLMo and opposite for Qwen and the small pair.

Generation ×\times Edit Class

Per pair, ARA lost less effectiveness than the direction-based methods in Gemma (H2 contrast, ARA minus direction mean, −30.8-30.8 pp) and OLMo (−39.3-39.3 pp), more in Qwen (+23.6+23.6 pp), and about as much in Mistral Small (+0.5+0.5 pp) and the small pair (−3.1-3.1 pp; Table 8 and Table 5). In log-odds the per-pair interactions were −4.27-4.27 (Gemma), −1.87-1.87 (OLMo), +1.89+1.89 (Qwen) and +0.76+0.76 (small pair, opposite in sign to its risk difference), all pHolm<0.001p_{\mathrm{Holm}} < 0.001, and −0.06-0.06 for Mistral Small (pHolm=0.696p_{\mathrm{Holm}} = 0.696); the pairs differed (generation ×\times class ×\times family χ2=430.4\chi^2 = 430.4, 3 df, p<0.001p < 0.001). With seed variance added, only the Gemma (−30.8-30.8 pp [−60.2-60.2, −1.3-1.3]) and OLMo (−39.3-39.3 pp [−68.8-68.8, −9.9-9.9]) contrasts excluded zero. Pooled with the pair as the unit, the contrast was −11.5-11.5 pp [−57.6-57.6, +34.6+34.6] (−1.05-1.05 log-odds [−5.21-5.21, +3.11+3.11]). With the prompt as the unit, the pooled GEE interaction was −0.50-0.50 log-odds [−0.62-0.62, −0.38-0.38] (pHolm<0.001p_{\mathrm{Holm}} < 0.001, Holm over 6 tests), and the planned mixed model gave −0.71-0.71 [−0.81-0.81, −0.60-0.60], both in the predicted direction. Leaving out Gemma reversed the pooled interaction (+0.20+0.20 [+0.05+0.05, +0.36+0.36], p=0.008p = 0.008), leaving out OLMo removed it (0.000.00 [−0.15-0.15, +0.15+0.15], p=0.98p = 0.98), and under keyword scoring it was +0.02+0.02 [−0.08-0.08, +0.12+0.12] (pHolm=1.000p_{\mathrm{Holm}} = 1.000). Because the KL cap bound in every ARA run but in only 15 of the 24 direction-based runs, H2 compares a class whose selection was always set by the cap with one whose selection was set by it in part; on the units in which the cap bound in both cells, the interaction was −0.21-0.21 [−0.33-0.33, −0.09-0.09].

Three properties of the ARA cells bear on these contrasts. First, the released ARA model is rebuilt by refitting its adapter (L-BFGS) from the stored trial parameters, not copied from the search. For the Qwen and Gemma pairs this refit ran on a different GPU (RTX PRO 6000) from the search (H200), and these four ARA exports moved by +6+6, +9+9, +5+5 and −7-7 dev keyword refusals when re-scored, whereas the four refitted on the search machine (OLMo, Mistral Small) moved by 0 to 2 (more than two refusals: 4 of 4 vs. 0 of 4, Fisher p=0.029p = 0.029; other methods 3 of 14 vs. 1 of 14, p=0.60p = 0.60; Appendix § 11.§ 3). Held-out results were measured on the released models, so they describe those models, but the link between the searched trial and the released model is weaker for these four. Second, the Gemma-4-31B ARA export, behind the largest H2 contrast, re-scored at a first-token KL of 0.107, above the cap (0.0845 during the search). Third, two seed-1 ARA runs on Qwen3.8-27B selected trials with 59 and 63 of 100 dev keyword refusals, whereas heretic-org reports 0 of 100 at KL 0.054 for a model built with the same defaults; this was not reproduced, the reported trial was not run in this pipeline, and the released heretic-org model refused 26.4% of the held-out prompts against 40.4% for the ARA model selected here (−14.0-14.0 pp [−17.0-17.0, −11.0-11.0]). The Qwen ARA effect (+34.4+34.4 pp), which reverses H2 in the Qwen pair, rests on this cell.

Keyword Counts versus an Automatic Judge

Table 9 reports H3 for three keyword windows; Table 10 details the full-response comparison and Figure 4 shows the gap per model at the 100-token window and the pooled gap at each window. At the window Heretic itself scores, the first 100 generated tokens, keyword matching and WildGuard agreed on average: over the 42 ablated models the keyword-minus-WildGuard gap was +0.5+0.5 pp [−1.0-1.0, +1.9+1.9] (main study +1.5+1.5 pp [+0.0+0.0, +2.9+2.9]; small pair −2.6-2.6 pp [−4.3-4.3, −0.9-0.9]; originals +0.7+0.7 pp [−0.1-0.1, +1.5+1.5]), and −0.1-0.1 pp [−1.6-1.6, +1.3+1.3] against WildGuard on the first 256 tokens. Per model the sign varied, with keyword matching was significantly higher than WildGuard in 14 of the 42 ablated models and significantly lower in 11 (Holm over the 74 models). With longer windows the gap grew, to +3.8+3.8 pp [+2.4+2.4, +5.2+5.2] at 256 tokens (both measures on the same prefix) and +8.4+8.4 pp [+6.9+6.9, +10.0+10.0] on the full response of at most 512 tokens, where keyword matching was higher in 34 models (24 significant) and lower in 7 (3). Keyword refusal fell from 39.9% on the full response to 32.0% at 100 tokens while WildGuard refusal was 31.5%, so most of the full-response excess came from markers that appear after the first 100 tokens. At 100 tokens the gap was +11.0+11.0 pp [+9.7+9.7, +12.4+12.4] on responses that reached the 512-token cap and −10.4-10.4 pp [−12.2-12.2, −8.6-8.6] on responses that ended earlier (full response +17.7+17.7 and −1.2-1.2 pp). The 100-token text was obtained by re-tokenising the stored responses, which reproduced the stored 256-token keyword labels in 97,825 of 97,828 responses (Appendix § 11.§ 2).

The structure of the gap was the same at every window. In the main study, with the prompt as the unit, the 100-token gap was larger for ARA than for the direction-based methods (+5.2+5.2 pp [+3.9+3.9, +6.4+6.4]), larger on the newer models (+4.7+4.7 pp [+3.4+3.4, +5.9+5.9]) and largest for ARA on the newer models (generation ×\times class +8.1+8.1 pp [+6.0+6.0, +10.1+10.1]; all pHolm<0.001p_{\mathrm{Holm}} < 0.001; full response +6.0+6.0, +4.4+4.4 and +9.4+9.4 pp). With the model as the unit (n=32n = 32), none of these terms was significant (class p=0.17p = 0.17, interaction p=0.28p = 0.28; full response p=0.151p = 0.151 and 0.26). In the small pair the class difference was not significant (100 tokens +1.1+1.1 pp, pHolm=0.280p_{\mathrm{Holm}} = 0.280; full response +0.8+0.8 pp [−1.5-1.5, +3.0+3.0], pHolm=0.774p_{\mathrm{Holm}} = 0.774).

On the full response, keyword–WildGuard agreement fell from κ=0.73\kappa= 0.73 [0.700.70, 0.750.75] in the originals to 0.40 [0.390.39, 0.420.42] (main study) and 0.33 [0.310.31, 0.350.35] (small pair) after ablation, whereas WildGuard and Qwen3Guard agreed at 0.90 [0.880.88, 0.920.92] (originals) and 0.82 [0.800.80, 0.830.83] after ablation (42 models; main study 0.82 [0.810.81, 0.840.84], small pair 0.81 [0.780.78, 0.830.83]). Over the 42 ablated models, 7,503 responses were refusals by full-response keyword matching only and 4,059 by WildGuard only; by string heuristics that were not validated, 90.9% of the keyword-only refusals matched only Heretic’s safety and disclaimer markers (such as “illegal” or “harmful”), not a first-person refusal or AI-identity phrase, and 60.0% were truncated at 512 tokens. Judging the first 256 tokens instead of the full response changed WildGuard refusal by +0.6 pp [+0.5, +0.7] (Appendix § 11.§ 2).

Secondary Outcomes

γ\gamma-MPOA control. Correcting MPOA for the post-norm output gains (Figure 15 and Figure 5) lowered held-out refusal on all four post-norm models, by 3.1 pp [2.1, 4.4] (OLMo-2-32B) to 19.9 pp [16.9, 22.7] (Gemma-4-31B; all pHolm≤0.002p_{\mathrm{Holm}} \le0.002). The OLMo generation effect remained +57.0 pp [+53.8, +60.1] (MPOA +61.4 pp), the Gemma generation effect became −20.4 pp [−23.6, −17.1] (MPOA −4.7 pp), and the pooled H2 interaction with γ\gamma-MPOA in place of MPOA was −0.42 [−0.53, −0.30]. Gemma-4-31B γ\gamma-MPOA lost 21.4 pp of mean capability at a dev KL of 0.061, with a teacher-forced divergence of 0.128 (MPOA 0.007), and OLMo-2-32B γ\gamma-MPOA lost 8.1 pp at 0.085 (divergence 0.196; MPOA 0.088); on Gemma-2-27B and Olmo-3.1-32B, where γ\gamma-MPOA lowered refusal by 4.2 and 7.5 pp, no benchmark score differed from the original’s (pHolm=1.0p_{\mathrm{Holm}} = 1.0).

Figure 15. Gamma-corrected MPOA versus MPOA on the post-norm models (dev set). One panel per model (columns: older and newer release; rows: Gemma, OLMo). A curve gives, for every KL budget tt, the fewest dev keyword refusals (of 100) among the completed TPE trials of one run (100 trials, seed 1) with first-token KL divergence from the original at most tt; it steps down at the run’s Pareto-front trials (method markers; older release dashed and hollow, newer solid and filled). Past the largest KL sampled (end tick) the value is only carried forward (thin line). Faint dots: all completed trials; dotted line: the selection budget KL = 0.1; the small table (upper right for OLMo-2-32B, lower left elsewhere) reads each curve at KL ≤ 0.1 and ≤ 0.5. MPOA vs γ\gamma-MPOA: on Gemma-2-27B the correction changes little (74 vs 73 at KL ≤ 0.1, 40 vs 35 at KL ≤ 0.5). On Gemma-4-31B the two are close at KL ≤ 0.1 (80 vs 78) but separate at larger budgets (70 vs 13 at KL ≤ 0.5), a gap confounded with search coverage (MPOA placed 14 of 100 trials above KL 0.1, γ\gamma-MPOA 82): the MPOA curve is flat there because that region was barely explored, not because lower counts were shown to be unreachable. On OLMo-2-32B the correction lowers the count at the selection budget itself (36 vs 7 at KL ≤ 0.1, 36 vs 3 at KL ≤ 0.5), with similar search coverage (99 and 88 of 100 trials at KL ≤ 0.1, none above 0.3), so this gap is not a coverage artefact. On Olmo-3.1-32B the correction changes little (84 vs 82 at KL ≤ 0.1; neither search sampled a larger KL, so the KL ≤ 0.5 values only repeat these). All counts are dev keyword refusals on the 100 prompts also used for optimisation and selection, not held-out refusal rates; one search per run (seed 1), budgets other than 0.1 read post hoc.

Community releases. Of eight public abl iterations (Figure 6 and Figure 13), four refused less than the best model selected here for the same checkpoint (all pHolm<0.001p_{\mathrm{Holm}} < 0.001), for example on Qwen3.8-27B (1.4% [0.9, 2.4] vs. 14.0% [12.0, 16.3]; −12.6 pp [−14.8, −10.5]); two did not differ significantly and two refused more. The four had teacher-forced divergences of 0.085–0.151 (best selected models 0.042–0.060) and mean capability changes of −0.94 to −0.15 pp. On three of the five checkpoints with a public release (Qwen2.5-32B, Qwen3.8-27B, Olmo-3.1-32B), a public release refused less than every model selected here under the KL cap; on the other two (Gemma-4-31B, Mistral-Small-3.2) the best release did not differ significantly from the best selected model.

Seed sensitivity. On the small pair a second seed changed the selected dev count by a median of 8 and up to 35 refusals (Figure 16), and held-out refusal differed between seeds in all 10 cells after Holm correction (median 9.4 pp, maximum 25.2 pp; item-level κ=0.08\kappa= 0.08–0.790.79; Figure 7); these differences give the run-to-run standard deviation used in § 4.1. The ARA generation effect was +14.2 pp with seed 1 and +0.0 pp [−1.2, +1.2] (p=1.00p = 1.00) with seed 2, the direction-based effects changed from +34.1, +4.2, +9.7 and +21.2 pp (DiM, MPOA, SOM, J-space) to +44.3, +30.1, +29.6 and +10.6 pp, and the H2 risk-difference contrast was −3.1 pp with seed 1 and −28.7 pp [−31.0, −26.5] with seed 2. Method accounted for η2=0.94\eta^2 = 0.94 and 0.88 of the variance among the ten cell rates of Qwen2.5-7B and Qwen3.5-4B (F(4,5)=19.46F(4, 5) = 19.46 and 9.58, p=0.003p = 0.003 and 0.015).

Figure 16. Seed sensitivity of the selected trial (dev set, small pair). Each row is one model × method cell of the small Qwen pair, optimised twice with 100 TPE trials that differ only in the seed (same configuration, Heretic commit and RTX 4090). The two markers give the dev keyword refusals (of 100) of the trial chosen by the selection rule (fewest dev keyword refusals at first-token KL ≤ 0.1) in the seed-1 run (full colour) and the seed-2 run (lighter); the column on the right is their absolute difference. Hollow markers and dashed lines mark the older model (Qwen2.5-7B), filled markers and solid lines the newer one (Qwen3.5-4B). The right panels show both seeds’ Pareto fronts for the cell with the largest difference (Qwen2.5-7B DiM, 55 vs 20) and the cell with the smallest (Qwen2.5-7B MPOA, 19 vs 18). In the fronts, the rings mark each seed’s selected trial and the dotted line the KL = 0.1 threshold. All counts are keyword matches on the 100 dev prompts used for optimisation and selection, not held-out refusal rates; seed 1 is the run evaluated elsewhere in the paper.

Over-refusal. WildGuard refusal on the 350 benign held-out prompts fell from 4.3–22.3% in the originals to 0.0–7.4% in the 42 primary models, significantly in 41 of them (exact McNemar, Holm over all 64 modified models); only Gemma-4-31B DiM, the failed edit, did not change significantly (+1.1 pp, pHolm=0.250p_{\mathrm{Holm}} = 0.250).

Harmful compliance. Residual refusal is not the same as harmful compliance. Of the ablated main-study responses that WildGuard did not label as refusals, 75.1% [73.4, 76.9] were harmful according to the HarmBench classifier and 55.8% [53.3, 58.1] according to WildGuard’s harmful-response label (all ten originals: 79.0% and 29.6%), so 16.8% [15.7, 18.0] of all ablated main-study responses were neither refused nor harmful by HarmBench. With harmful compliance as the outcome, coded as not harmful so that a positive generation effect again means that removal left less on the newer model, the pooled conclusions were the same (Appendix § 11.§ 6). The baseline-adjusted direction-based effect was +0.05 log-odds [−0.02, +0.11] (HarmBench) and −0.03 [−0.13, +0.07] (WildGuard), against +0.03 for refusal; the H2 interaction was −0.49 [−0.56, −0.41] and −0.45 [−0.52, −0.38], against −0.50; and per pair the interaction was in the predicted direction for Gemma and OLMo and opposite for Qwen under every outcome. Several per-pair results changed. Under HarmBench every Qwen method was significantly higher on the newer model, so the Qwen pair met the H1 criterion on this outcome: Qwen SOM, Gemma MPOA and Gemma SOM became significantly higher, Mistral Small DiM lost significance and Mistral Small ARA became significantly lower. The raw pooled ARA effect became significant (+0.21 [+0.14, +0.28] and +0.24 [+0.18, +0.30], against +0.12 for refusal), and the Holm tally of the 16 main-pair tests was 11 higher, 3 lower and 2 not significant (HarmBench) and 10, 1 and 5 (WildGuard), against 8, 4 and 4.

Capability and divergence. Across the 42 primary models, mean capability change ranged from −4.2 pp (OLMo2-32B ARA) to +0.9 pp (Qwen2.5-7B J-space; Figure 8, Table 15–Table 16), and teacher-forced divergence from 0.004 to 0.279 nats per token (Figure 9, Table 17–Table 18). Across the 56 modified models with a dev trial, capability change was not correlated with selected dev KL (Spearman ρ=−0.03\rho= -0.03 [−0.29, +0.24], p=0.81p = 0.81) but was with teacher-forced divergence (ρ=−0.48\rho= -0.48 [−0.66, −0.24], p<0.001p < 0.001); every selected trial had KL ≤0.1\le0.1 by design. Among the 42 primary models alone, the correlations were ρ=+0.08\rho= +0.08 [−0.22, +0.38] (p=0.63p = 0.63) with dev KL and −0.39 [−0.63, −0.11] (p=0.010p = 0.010) with teacher-forced divergence.

ModelMethodMMLU-Pro (1000)GSM8K (500)IFEval (541)HumanEval (164)MBPP (500)
Qwen2.5-32B-InstructOriginal§71.185.478.989.677.6‡
Qwen2.5-32B-InstructDiM69.9 (−1.2)85.6 (+0.2)79.7 (+0.7)90.2 (+0.6)77.0 (−0.6)
Qwen2.5-32B-InstructMPOA70.5 (−0.6)85.0 (−0.4)80.8 (+1.8)89.6 (0.0)76.6 (−1.0)
Qwen2.5-32B-InstructARA70.3 (−0.8)86.6 (+1.2)79.9 (+0.9)84.8 (−4.9)77.0 (−0.6)
Qwen2.5-32B-InstructSOM71.1 (0.0)85.6 (+0.2)79.5 (+0.6)89.6 (0.0)78.4 (+0.8)
Qwen3.8-27BOriginal79.696.280.697.680.8
Qwen3.8-27BDiM79.5 (−0.1)96.6 (+0.4)80.6 (0.0)96.3 (−1.2)80.6 (−0.2)
Qwen3.8-27BMPOA79.6 (0.0)97.0 (+0.8)82.4 (+1.8)97.0 (−0.6)80.4 (−0.4)
Qwen3.8-27BARA78.5 (−1.1)95.8 (−0.4)81.0 (+0.4)93.9 (−3.7)79.2 (−1.6)
Qwen3.8-27BSOM79.4 (−0.2)97.0 (+0.8)79.5 (−1.1)95.1 (−2.4)80.6 (−0.2)
gemma-2-27b-itOriginal§42.886.460.676.269.6
gemma-2-27b-itDiM42.3 (−0.5)86.0 (−0.4)60.6 (0.0)75.0 (−1.2)69.2 (−0.4)
gemma-2-27b-itMPOA43.1 (+0.3)84.4 (−2.0)60.8 (+0.2)76.8 (+0.6)68.6 (−1.0)
gemma-2-27b-itARA44.3 (+1.5)84.2 (−2.2)61.4 (+0.7)76.8 (+0.6)70.0 (+0.4)
gemma-2-27b-itSOM45.3 (+2.5)86.2 (−0.2)60.8 (+0.2)75.6 (−0.6)68.6 (−1.0)
gemma-4-31B-itOriginal85.696.691.592.771.4
gemma-4-31B-itDiM85.2 (−0.4)97.0 (+0.4)90.0 (−1.5)90.2 (−2.4)69.2 (−2.2)
gemma-4-31B-itMPOA86.1 (+0.5)96.4 (−0.2)91.7 (+0.2)90.9 (−1.8)70.0 (−1.4)
gemma-4-31B-itARA85.2 (−0.4)97.0 (+0.4)91.5 (0.0)86.0 (−6.7)67.2 (−4.2)
gemma-4-31B-itSOM85.4 (−0.2)96.8 (+0.2)89.6 (−1.8)89.0 (−3.7)63.0 (−8.4)
OLMo-2-0325-32B-InstructOriginal52.688.884.361.044.0
OLMo-2-0325-32B-InstructDiM52.2 (−0.4)88.6 (−0.2)80.8 (−3.5)59.8 (−1.2)38.4 (−5.6)
OLMo-2-0325-32B-InstructMPOA52.3 (−0.3)88.0 (−0.8)83.0 (−1.3)57.9 (−3.0)42.2 (−1.8)
OLMo-2-0325-32B-InstructARA52.3 (−0.3)86.8 (−2.0)86.8 (−3.7)57.3 (−3.7)30.8 (−13.2)
OLMo-2-0325-32B-InstructSOM51.9 (−0.7)88.2 (−0.6)83.2 (−1.1)57.9 (−3.0)44.0 (0.0)
Olmo-3.1-32B-InstructOriginal64.789.286.987.271.4
Olmo-3.1-32B-InstructDiM64.6 (−0.1)90.4 (+1.2)87.2 (+0.4)88.4 (+1.2)70.2 (−1.2)
Olmo-3.1-32B-InstructMPOA63.5 (−1.2)89.8 (+0.6)86.7 (−0.2)90.2 (+3.0)71.4 (0.0)
Olmo-3.1-32B-InstructARA63.5 (−1.2)88.6 (−0.6)87.2 (+0.4)92.1 (+4.9)70.0 (−1.4)
Olmo-3.1-32B-InstructSOM64.1 (−0.6)89.2 (0.0)86.7 (−0.2)91.5 (+4.3)70.8 (−0.6)
Mistral-Small-Instruct-2409Original56.288.261.476.262.6
Mistral-Small-Instruct-2409DiM56.6 (+0.4)87.4 (−0.8)62.3 (+0.9)75.6 (−0.6)61.6 (−1.0)
Mistral-Small-Instruct-2409MPOA53.9 (−2.3)88.0 (−0.2)61.0 (−0.4)75.0 (−1.2)62.2 (−0.4)
Mistral-Small-Instruct-2409ARA51.5 (−4.7)84.2 (−4.0)63.6 (+2.2)74.4 (−1.8)58.0 (−4.6)
Mistral-Small-Instruct-2409SOM55.6 (−0.6)88.0 (−0.2)63.2 (+1.8)76.8 (+0.6)62.4 (−0.2)
Mistral-Small-3.2-24B-Instruct-2506Original64.893.677.187.270.6
Mistral-Small-3.2-24B-Instruct-2506DiM64.8 (0.0)93.8 (+0.2)76.9 (−0.2)87.8 (+0.6)71.2 (+0.6)
Mistral-Small-3.2-24B-Instruct-2506MPOA64.9 (+0.1)94.0 (+0.4)76.0 (−1.1)89.6 (+2.4)69.6 (−1.0)
Mistral-Small-3.2-24B-Instruct-2506ARA63.4 (−1.4)92.6 (−1.0)77.4 (+0.4)88.4 (+1.2)70.2 (−0.4)
Mistral-Small-3.2-24B-Instruct-2506SOM63.7 (−1.1)93.6 (0.0)76.0 (−1.1)88.4 (+1.2)69.6 (−1.0)

Table 15. Capability of the original and the selected modified models (KL ≤ 0.1 trial), in %. Modified models: score with the difference to the same family’s original model in percentage points in parentheses (computed from unrounded scores). Number of items in the column header. Metrics: MMLU-Pro exact match (custom extract, 5-shot CoT); GSM8K exact match (flexible extract, 5-shot); IFEval prompt-level strict accuracy (0-shot); HumanEval pass@1 (0-shot); MBPP pass@1 (3-shot). Greedy decoding, chat template on, thinking disabled. Main study. ‡Rescored from the logged samples with the patched MBPP answer extractor (run predates the lm-eval patch; scripts/bench/LM_EVAL_PATCHES.md): Qwen2.5-32B-Instruct Original MBPP. §Original model’s capability run used evaluation-harness settings that differ from its modified models’ runs (original vs. modified models; per-cell settings in paper/tables/coverage.md, evaluation protocol table): Qwen2.5-32B-Instruct: bench.py revision (sha256 prefix) 0ac2aca202 vs. b6a926cf73, f220db7bb9; output post-processing none vs. U+2581→space; BENCH_VLLM_EXTRA unset vs. {}. gemma-2-27b-it: bench.py revision (sha256 prefix) b6a926cf73 vs. f220db7bb9. –: not yet available.

ModelMethodMMLU-Pro (1000)GSM8K (500)IFEval (541)HumanEval (164)MBPP (500)
Qwen2.5-7B-InstructOriginal59.585.871.284.863.2
DiM58.1 (−1.4)86.0 (+0.2)71.7 (+0.6)86.0 (+1.2)63.8 (+0.6)
MPOA57.9 (−1.6)85.0 (−0.8)70.6 (−0.6)86.6 (+1.8)63.2 (0.0)
ARA57.8 (−1.7)86.8 (+1.0)70.2 (−0.9)87.2 (+2.4)64.8 (+1.6)
SOM59.8 (+0.3)84.0 (−1.8)70.6 (−0.6)83.5 (−1.2)64.8 (+1.6)
J-space60.5 (+1.0)86.4 (+0.6)72.6 (+1.5)84.8 (0.0)64.4 (+1.2)
Qwen3.5-4BOriginal70.691.882.379.962.6
DiM70.1 (−0.5)91.2 (−0.6)82.4 (+0.2)81.7 (+1.8)61.8 (−0.8)
MPOA69.2 (−1.4)91.6 (−0.2)82.3 (0.0)79.9 (0.0)61.6 (−1.0)
ARA69.4 (−1.2)90.2 (−1.6)79.3 (−3.0)79.3 (−0.6)61.2 (−1.4)
SOM68.7 (−1.9)92.6 (+0.8)81.7 (−0.6)79.9 (0.0)61.2 (−1.4)
J-space70.4 (−0.2)91.6 (−0.2)81.3 (−0.9)82.9 (+3.0)61.2 (−1.4)

Table 16. Capability of the original and the selected modified models (KL ≤0.1\le0.1 trial), in %. Modified models: score with the difference to the same family’s original model in percentage points in parentheses (computed from unrounded scores). Number of items in the column header. Metrics: MMLU-Pro exact match (custom extract, 5-shot CoT); GSM8K exact match (flexible extract, 5-shot); IFEval prompt-level strict accuracy (0-shot); HumanEval pass@1 (0-shot); MBPP pass@1 (3-shot). Greedy decoding, chat template on, thinking disabled. Small pair. –: not yet available.

ModelTokensOriginalDiMMPOAARASOM
Qwen2.5-32B-Instruct112398−0.272§−0.017−0.042−0.077−0.020
Qwen3.8-27B138016−0.275−0.006−0.060−0.042−0.021
gemma-2-27b-it97666−0.227§−0.068−0.063−0.161−0.031
gemma-4-31B-it142322−0.110−0.042−0.007−0.279−0.026
Olmo-2-0325-32B-Instruct110792−0.520−0.051−0.088−0.055−0.030
Olmo-3.1-32B-Instruct136772−0.288−0.008−0.008−0.049−0.016
Mistral-Small-Instruct-2409129960−0.209−0.004−0.018−0.080−0.023
Mistral-Small-3.2-24B-Instruct-2506131780−0.171−0.011−0.066−0.101−0.037

Table 17. Teacher-forced log-likelihood of the original model’s greedy answers to the 350 benign held-out prompts (jbb_benign 100, xstest_safe 250). “Original”: mean log-probability per answer token under the original model (token-weighted over all items); method columns: the same quantity under the selected modified model (KL ≤0.1\le0.1 trial) minus the original’s value (negative = the original answers became less likely). Tokens: number of scored answer tokens. Main study. 8Original model’s held-out generation and teacher forcing come from a run whose evaluation-harness settings differ from its modified models’ runs (original vs. modified models; per-cell settings in paper/tables/coverage.md, evaluation protocol table): Qwen2.5-32B-Instruct: separate bench.py run (refusal_run/summary.json in its evaluation directory); bench.py revision (sha256 prefix) f7731f7d3f vs. b6a926cf73, f220db7bb9; output post-processing none vs. U+2581→space. gemma-2-27b-it: bench.py revision (sha256 prefix) b6a926cf73 vs. f220db7bb9. –: not yet available.

ModelTokensOriginalDiMMPOAARASOMJ-space
Qwen2.5-7B-Instruct119254−0.321−0.019−0.018−0.048−0.026−0.009
Qwen3.5-4B139372−0.412−0.010−0.031−0.093−0.035−0.045

Table 18. Teacher-forced log-likelihood of the original model’s greedy answers to the 350 benign held-out prompts (jbb_benign 100, xstest_safe 250). “Original”: mean log-probability per answer token under the original model (token-weighted over all items); method columns: the same quantity under the selected modified model (KL ≤0.1\le0.1 trial) minus the original’s value (negative = the original answers became less likely). Tokens: number of scored answer tokens. Small pair. –: not yet available.

Development set versus held-out refusal. On the dev prompts used for search and selection (Figure 10–Figure 18, Table 13), the selected keyword count changed from older to newer model by −2.8 (DiM), +17.3 (MPOA), +20.3 (ARA) and −4.5 (SOM) refusals on average over the main pairs (Figure 17). The newer models’ ARA counts (Qwen, Gemma, OLMo, Mistral Small) were 63, 28, 48 and 6 at a KL budget of 0.1 and 21, 9, 34 and 6 at 0.2 (Figure 18). Gemma-4-31B ARA, with 28 dev keyword refusals, had 22.3% [19.8, 25.0] held-out keyword refusals and 0.6% held-out WildGuard refusals. Dev refusals correlated with held-out refusal across the 42 primary models (ρ=0.77\rho= 0.77 [0.61, 0.87]; older models 0.85, newer 0.73), and the best dev method was the best held-out method in 7 of 12 model × seed sets (6 of 12 without the KL tie-break). The direction of the change could differ between the two sets: in 3 of the 16 main-pair method cells (Qwen DiM, Gemma ARA, Mistral Small MPOA) the dev change and a significant held-out change had opposite signs, and on the small pair every method’s dev count fell from the older to the newer model while every held-out rate rose (Figure 17).

Re-scoring the released exports. The 46 released merged exports were re-scored on the development set with Heretic’s own evaluator. Their keyword refusal counts were within two of the selected trials’ counts for 37 of 46 (median change +1, range −7 to +9, largest for ARA), first-token KL changed by a median of +0.0003, and four exports (Gemma-2-27B MPOA, Gemma-4-31B ARA and the γ\gamma-MPOA controls of Gemma-2-27B and Olmo-3.1-32B) re-scored slightly above the KL budget, at 0.103 to 0.107 (Appendix § 11.§ 3). The largest changes were in ARA exports whose adapter had been refitted on a different GPU from the search (§ 4.2). The re-score also showed that the five Gemma-2-27B exports had been saved without the eager-attention setting that transformers needs for Gemma-2’s attention-logit soft-capping; their public configurations were patched, and the held-out results are unaffected because all generations, benchmarks and teacher-forced divergences were computed in vLLM, which applies the soft-cap from the configuration regardless of this setting.

Search convergence. Of the 42 core runs, 34 reached their selected count by trial 75 (median trial 48) and 8 (19%) improved later, by 1–14 refusals (Figure 19 and Figure 14); late runs were equally frequent on older and newer models (4 of 21 each; Fisher p=1.0p = 1.0). The failed Gemma-4-31B DiM run reached its selected count (98) at trial 40.

Run-to-Run Variance as a Function of the Refusal Level

The per-pair intervals of Table 5 add a run-to-run variance σ2\sigma^2 per optimised cell, with σ=9.15\sigma= 9.15 pp [6.40, 16.07] estimated from the ten small-pair seed pairs and assumed constant across models, methods and refusal levels (§ 3.5). A binomial-type rate varies less near 0% and 100%, so a second model holds the run effect constant on the logit scale, σL=0.83\sigma_L = 0.83 [0.58, 1.45] from the same seed pairs, and maps it to percentage points at each cell’s observed rate pp by the delta method, σpp=σLp(1−p)\sigma_{\mathrm{pp}} = \sigma_L p(1-p). This gives about 0.5 pp for Gemma-4-34B ARA (0.6% refusal) and up to 20.7 pp at 50% refusal. The ten seed pairs do not distinguish the two models (log-likelihood of the pp differences 6.25 under the constant model and 6.01 under the logit model; rank correlation between ∣d∣|d| and pˉ(1−pˉ)\bar{p}(1-\bar{p}) −0.01). Under the logit model, 1 of the 16 main-study per-method effects (OLMo MPOA) exceeded twice the run-to-run standard deviation of a difference, against 5 under the constant model; across the 95% interval of σ\sigma, 0 to 3 did. Per pair (Table 28), the OLMo direction-based increase excluded zero under both models except at the upper limit of the logit model, whereas the Qwen ARA increase, the Gemma ARA decrease and the Gemma H2 contrast excluded zero only under the constant model. Table 29 gives the between-pair heterogeneity with each kind of within-pair variance, including the between-pair variance τ2\tau^2 of every estimand, and Table 30 the within-pair variances used as weights. The headline I2I^2 of the raw direction-based pp effect is therefore 80% only under the constant variance model at the point estimate of σ\sigma; under the refusal-level model it is 46% (τ2=127\tau^2 = 127 pp2^2 against 225 pp2^2).

Verdicts at Other KL Budgets (Development Set)

Only the trial selected at first-token KL ≤0.1\leq0.1 was evaluated on held-out prompts, so other budgets can be read only on the 100 dev prompts, from the completed trials of each search. For each run the refusal–KL front f(t)f(t), the fewest dev keyword refusals among trials with KL ≤t\leq t (the original at KL = 0 included), was read at t=0.05t = 0.05, 0.1, 0.2, 0.5 and 1.0, and summarised without a budget as the area under the front, the mean of f(t)f(t) over 200 log-spaced budgets between 0.01 and 1. Run-to-run variation of each quantity was estimated from the ten small-pair seed pairs. In 7 of the 42 seed-1 runs the front was carried forward beyond the largest sampled KL over more than half of this range. Table 31 gives the class-level generation effects. The direction-based effects kept their sign from KL ≤0.1\leq0.1 to KL ≤1.0\leq1.0 in all four main pairs, and the OLMo direction-based increase exceeded run-to-run variation at every budget and on the area. The dev ARA effects had the same sign at every budget but the opposite sign to the held-out ARA effects in three of the four main pairs, as at the selected budget (§ 4.4). The H2 contrast was positive at small budgets and negative at large ones in Gemma, negative from KL ≤0.1\le0.1 in OLMo, and positive in Qwen at every budget.

Outcome Classes

Table 32 gives the four outcome classes of §3.3 for every original and primary model. Degenerate responses were at most 0.6% of any of these models’ responses. Among the 42 primary models, incomplete compliance (generation stopped at the 512-token limit) ranged from 7.6% to 79.9% of responses and complete compliance from 2.3% to 61.3%; 65.8% of all non-refused responses were incomplete. Incomplete compliance is a property of the 512-token budget, not evidence that a response was unhelpful.

PairConditionOlder model Ref.Older model Compl.Older model Incompl.Older model Degen.Newer model Ref.Newer model Compl.Newer model Incompl.Newer model Degen.
QwenOriginal76.75.717.40.189.94.25.80.1
DiM47.418.833.60.174.35.220.40.1
MPOA3.950.645.40.114.08.777.30.0
SOM42.720.436.80.138.014.047.80.2
ARA6.161.332.50.140.34.554.90.2
GemmaOriginal88.32.29.60.089.32.38.40.0
DiM73.88.417.80.090.12.37.60.0
MPOA45.227.127.80.040.411.048.60.0
SOM65.213.621.20.067.79.922.30.1
ARA26.646.127.30.00.619.479.90.0
OLMoOriginal83.22.614.20.084.11.414.50.0
DiM33.528.538.00.068.35.725.90.1
MPOA3.642.054.40.065.06.328.70.0
SOM27.327.445.40.041.710.248.10.0
ARA13.339.347.40.010.818.970.30.0
Mistral SmallOriginal55.711.832.20.273.66.120.40.0
DiM32.823.643.60.028.526.245.30.0
MPOA6.940.452.70.04.035.660.40.0
SOM3.742.154.20.07.233.659.20.0
ARA4.940.954.00.14.226.569.20.0
Small pairOriginal72.15.122.50.288.52.09.60.0
DiM20.627.851.50.154.611.334.10.0
MPOA13.832.553.30.418.022.559.20.3
SOM12.232.255.00.521.921.556.20.4
J-space52.611.935.30.273.84.022.20.0
ARA4.730.564.30.518.916.763.80.6

Table 32. Outcome classes on the 972 harmful held-out prompts (% of responses), older and newer model of each pair: refusal (WildGuard, non-degenerate responses), complete compliance, incomplete compliance (generation stopped at the 512-token limit) and degenerate responses (Methods). Rows may not add to 100 because of rounding and a few unscored responses.

Mixed-Model Estimates and Conditional Retention

Table 33 gives the planned mixed logistic model of the H2 interaction next to the GEE estimates of Table 8. Its conditional coefficients have the same sign as the GEE coefficients in every pair and pooled and are larger in magnitude, as expected for a conditional model, but the model has no pair-specific slopes and its intervals describe prompt sampling only. Table 34 gives the generation effects on conditional refusal retention with prompt-bootstrap intervals and with intervals that add a run-to-run standard deviation of retention of 11.0 pp [7.7, 19.2] per cell, estimated from the ten small-pair seed pairs. With run-to-run variance, the OLMo direction-based increase (+43.4 pp [+25.7, +61.1]), the

OLMo H2 contrast (−46.6-46.6 pp [−81.8-81.8, −11.3-11.3]) and the Qwen ARA increase (+36.9+36.9 pp [+6.3+6.3, +67.5+67.5]) excluded zero; no other class-level effect or H2 contrast did.

References

  1. [1]Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction. In NeurIPS, volume 37, pages 136037–136083, 2024. arXiv:2406.11717.arxiv.org/abs/2406.11717
  2. [2]Maxime Labonne. Uncensor any LLM with abliter­ation. Hugging Face Blog, https://huggingface.co/blog/mlabonne/abliteration, 2024.
  3. [3]Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Fine-tuning aligned language models compromises safety, even when users do not intend to! In ICLR, pages 30988–31043, 2024. arXiv:2310.03693.arxiv.org/abs/2310.03693
  4. [4]Simon Lermen, Charlie Rogers-Smith, and Jeffrey Ladish. LoRA fine-tuning efficiently undoes safety training in Llama 2-Chat 70B, 2023. arXiv:2310.20624.arxiv.org/abs/2310.20624
  5. [5]Philipp Emanuel Weidmann. Heretic: Fully automatic censorship removal for language models. https://github.com/p-e-w/heretic, 2025.
  6. [6]Jim Lai. Norm-preserving biprojected abliter­ation. https://huggingface.co/blog/grimjim/norm-preserving-biprojected-abliteration, 2025.
  7. [7]Giorgio Piras, Raffaele Mura, Fabio Brau, Luca Oneto, Fabio Roli, and Battista Biggio. SOM directions are better than one: Multi-directional refusal suppression in language models. In AAAI, volume 40, pages 32728–32736, 2026. arXiv:2511.08379.arxiv.org/abs/2511.08379
  8. [8]Tom Wollschläger, Jannes Elstner, Simon Geisler, Vincent Cohen-Addad, Stephan Günnemann, and Johannes Gasteiger. The geometry of refusal in large language models: Concept cones and representational independence. In ICML, volume 267 of PMLR, pages 66945–66970, 2025. arXiv:2502.17420.arxiv.org/abs/2502.17420
  9. [9]Philipp Emanuel Weidmann. Arbitrary-rank ablation (ARA). Heretic pull request #211, https://github.com/p-e-w/heretic/pull/211, 2026. Unreviewed practitioner report, retrieved 2 November 2026. Not merged; ARA entered Heretic 2.0.0 through pull request #446, https://github.com/p-e-w/heretic/pull/446.
  10. [10]Fabian Hildebrandt, Andreas Maier, Patrick Krauss, and Achim Schilling. Refusal behavior in large language models: A nonlinear perspective. In International Joint Conference on Neural Networks (IJCNN), pages 1–7, 2025. arXiv:2501.08145.arxiv.org/abs/2501.08145
  11. [11]Faaiz Joad, Majd Hawasly, Sabri Boughorbel, Nadir Durrani, and Husrev Taha Sencar. There is more to refusal in large language models than a single direction. In EMNLP, pages 59196–59232, 2026. arXiv:2602.02132.arxiv.org/abs/2602.02132
  12. [12]Han Wang, Erik Miehling, Dennis Wei, Karthikeyan Natesan Ramamurthy, and Huan Zhang. On the steering dimensionality of refusal in language models, 2026. InterpScience Workshop at NeurIPS 2026. arXiv:2610.04245.arxiv.org/abs/2610.04245
  13. [13]Wenbo Pan, Zhichao Liu, Qiguang Chen, Xiangyang Zhou, Haining Yu, and Xiaohua Jia. The hidden dimensions of LLM alignment: A multi-dimensional analysis of orthogonal safety directions. In ICML, volume 267 of PMLR, pages 47697–47716, 2025. arXiv:2502.09674.arxiv.org/abs/2502.09674
  14. [14]Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, and Peter Henderson. Safety alignment should be made more than just a few tokens deep. In ICLR, pages 54911–54941, 2025. arXiv:2406.05946.arxiv.org/abs/2406.05946
  15. [15]Jiachen Zhao, Jing Huang, Zhengyuan Wu, David Bau, and Weiyan Shi. LLMs encode harmfulness and refusal separately. In NeurIPS, volume 38, pages 155649–155684, 2025. arXiv:2507.11878.arxiv.org/abs/2507.11878
  16. [16]Lei Yu, Virginie Do, Karen Hambardzumyan, and Nicola Cancedda. Robust LLM safeguarding via refusal feature adversarial training. In ICLR, pages 5254–5277, 2025. arXiv:2409.20089.arxiv.org/abs/2409.20089
  17. [17]Yuanbo Xie, Yingjie Zhang, Tianyun Liu, Douhe Ma, and Tingwen Liu. Beyond surface alignment: Rebuilding LLMs safety mechanism via probabilistically ablating refusal direction. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 17620–17632, Suzhou, China, 2025. Association for Computational Linguistics. arXiv:2509.15202.arxiv.org/abs/2509.15202
  18. [18]Wenhao Lan, Shan Li, Xinhua Lai, Meiqi Wu, Junbin Yang, Haihua Shen, and Yijun Yang. Dynamic adversarial fine-tuning reorganizes refusal geometry, 2026. arXiv:2604.27019.arxiv.org/abs/2604.27019
  19. [19]Thomas McGrath, Matthew Rahtz, Janos Kramar, Vladimir Mikulik, and Shane Legg. The hydra effect: Emergent self-repair in language model computations, 2023. arXiv:2307.15771.arxiv.org/abs/2307.15771
  20. [20]Shashank Agnihotri, Jonas Jakubassa, Priyam Dey, Sachin Goyal, Bernt Schiele, Venkatesh Babu Radhakrishnan, and Margret Keuper. A granular study of safety pretraining under model abliteration, 2025. NeurIPS 2025 Workshop Lock-LLM. arXiv:2510.02768.arxiv.org/abs/2510.02768
  21. [21]Richard J. Young. Comparative analysis of LLM abliteration methods: A cross-architecture evaluation, 2025. arXiv:2512.13655.arxiv.org/abs/2512.13655
  22. [22]Tinghao Xie, Xiangyu Qi, Yi Zeng, Yangsibo Huang, Udari Madhushani Sehwag, Kaixuan Huang, Luxi He, Boyi Wei, Dacheng Li, Ying Sheng, Ruoxi Jia, Bo Li, Kai Li, Danqi Chen, Peter Henderson, and Prateek Mittal. SORRY-Bench: Systematically evaluating large language model safety refusal. In ICLR, pages 59937–59973, 2025. arXiv:2406.14598.arxiv.org/abs/2406.14598
  23. [23]Richard J. Young. Measuring faithfulness depends on how you measure: Classifier sensitivity in LLM chain-of-thought evaluation, 2026. arXiv:2603.20172.arxiv.org/abs/2603.20172
  24. [24]Vadym Hadetskyi, Dario Pasquini, and Artem Sorokin. Not all refusals are equal: How safety alignment fails cybersecurity at scale, 2026. arXiv:2607.02714.arxiv.org/abs/2607.02714
  25. [25]Richard J. Young and Gregory D. Moody. Refusal evaluation in coding LLMs and code agents: A systematic review of thirteen malicious-code prompt corpora (2023–2025), 2026. arXiv:2605.20351.arxiv.org/abs/2605.20351
  26. [26]Richard J. Young and Gregory D. Moody. A validated prompt bank for malicious code generation: Separating executable weapons from security knowledge in 1,554 consensus-labeled prompts, 2026. arXiv:2605.03179.arxiv.org/abs/2605.03179
  27. [27]Richard J. Young and Gregory D. Moody. Code as a weapon: A consensus-labeled prompt bank for measuring coding-model compliance with malicious-code requests, 2026. arXiv:2605.28734.arxiv.org/abs/2605.28734
  28. [28]Kristina Nikolić, Luze Sun, Jie Zhang, and Florian Tramèr. The jailbreak tax: How useful are your jailbreak outputs? In ICML, volume 267 of PMLR, pages 46412–46426, 2025. arXiv:2504.10694.arxiv.org/abs/2504.10694
  29. [29]Richard J. Young, Brandon Gillins, and Alice M. Matthews. When models can’t follow: Testing instruction adherence across 256 LLMs, 2025. arXiv:2510.18892.arxiv.org/abs/2510.18892
  30. [30]Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendrycks. Representation engineering: A top-down approach to AI transparency, 2023. arXiv:2310.01405.arxiv.org/abs/2310.01405
  31. [31]Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering Llama 2 via contrastive activation addition, 2023. arXiv:2312.06681.arxiv.org/abs/2312.06681
  32. [32]FailSpy. abliterator: Simple Python library/structure to ablate features in LLMs which are supported by TransformerLens. https://github.com/FailSpy/abliterator, 2024.
  33. [33]Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models. In ICML, volume 235 of PMLR, pages 39643–39666, 2024. arXiv:2311.03658.arxiv.org/abs/2311.03658
  34. [34]Thomas Marshall, Adam Scherlis, and Nora Belrose. Refusal in LLMs is an affine function, 2024. arXiv:2411.09003.arxiv.org/abs/2411.09003
  35. [35]Jim Lai. Projected abliteration. https://huggingface.co/blog/grimjim/projected-abliteration, 2025.
  36. [36]Takuya Akiba, Shotaro Shibata, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization framework. In KDD, pages 2623–2631, 2019. https://doi.org/10.1145/3292500.3330701.
  37. [37]jwest33. abliterator. https://github.com/jwest33/abliterator, 2025. GitHub repository (created 2025-12-12).
  38. [38]Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Jie Shi, Xiang Wang, Xiangnan He, and Tat-Seng Chua. AlphaEdit: Null-space constrained knowledge editing for language models. In ICLR, 2025. arXiv:2410.02355.arxiv.org/abs/2410.02355
  39. [39]Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. In NeurIPS, volume 35, pages 17359–17372, 2022. arXiv:2202.05262.arxiv.org/abs/2202.05262
  40. [40]Tony Cristofano. Surgical refusal ablation: Disentangling safety from intelligence via concept-guided spectral cleaning, 2026. arXiv:2601.08489.arxiv.org/abs/2601.08489
  41. [41]Valentin Petrov. On the failure of topic-matched contrast baselines in multi-directional refusal abliteration, 2026. arXiv:2603.22061.arxiv.org/abs/2603.22061
  42. [42]Gökdeniz Gülmez. Gabliteration: Adaptive multi-directional neural weight modification for selective behavioral alteration in large language models, 2025. arXiv:2512.18901.arxiv.org/abs/2512.18901
  43. [43]elder-plinius. OBLITERATUS. https://github.com/elder-plinius/OBLITERATUS, 2026. GitHub repository, release v0.1.4.
  44. [44]Thomas Winninger. Fast multi-dimensional refusal subspaces via RFM-AGOP, 2026. Mechanistic Interpretability Workshop at ICML 2026. arXiv:2607.02396.arxiv.org/abs/2607.02396
  45. [45]Grigoriy Evseev, Vladimir Solodkin, Dmitry Bylinkin, Daniil Medyakov, Vladimir Aletov, and Aleksandr Beznosikov. WRDO: Weighted refusal direction optimization in aligned language models. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 43929–43944, 2026. arXiv:2610.29555.arxiv.org/abs/2610.29555
  46. [46]Boyi Wei, Kaixuan Huang, Yangsibo Huang, Tinghao Xie, Xiangyu Qi, Mengzhou Xia, Prateek Mittal, Mengdi Wang, and Peter Henderson. Assessing the brittleness of safety alignment via pruning and low-rank modifications. In ICML, volume 235 of PMLR, pages 52588–52610, 2024. arXiv:2402.05162.arxiv.org/abs/2402.05162
  47. [47]Geraldin Nafack, Eugene Belilovsky, and Elvis Dohmatob. Efficient refusal ablation in LLM through optimal transport, 2026. arXiv:2603.04355.
  48. [48]Sekinal. jspace-refusal: Refusal in the J-space workspace. https://github.com/Sekinal/jspace-refusal, 2026. GitHub repository, commit 30a1b9c.
  49. [49]Wes Gurnee, Nicholas Sofroniew, Adam Pearce, Mateusz Piotrowski, Isaac Kauvar, Runjin Chen, Anna Soligo, Paul Bogdan, Euan Ong, Rowan Wang, Ben Thompson, David Abrahams, Subhash Kantamneni, Emmanuel Ameisen, Joshua Batson, and Jack Lindsey. Verbalizable representations form a global workspace in language models. Transformer Circuits Thread, 2026. https://transformer-circuits.pub/2026/workspace/index.html.
  50. [50]Neuronpedia. jacobian-lens: Pre-fitted Jacobian lenses. https://huggingface.co/neuronpedia/jacobian-lens, 2026. Hugging Face repository, revision qwen-n1000.
  51. [51]Wei Jie Yeo, Nirmalendu Prakash, Clement Neo, Ranjan Satapathy, Roy Ka-Wei Lee, and Erik Cambria. Understanding refusal in language models with sparse autoencoders. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 6377–6399, Suzhou, China, 2025. Association for Computational Linguistics. arXiv:2505.23556.arxiv.org/abs/2505.23556
  52. [52]Preethi Carmel Bosco and Gopalakrishnan Srinivasan. Locating and steering refusal beyond attention, 2026. arXiv:2609.04721.arxiv.org/abs/2609.04721
  53. [53]Andrew Lee, Xiaoyan Bai, Itamar Pres, Martin Wattenberg, Jonathan K. Kummerfeld, and Rada Mihalcea. A mechanistic understanding of alignment algorithms: A case study on DPO and toxicity, 2024. arXiv:2401.01967.arxiv.org/abs/2401.01967
  54. [54]Samyak Jain, Ekdeep Singh Lubana, Kemal Oksuz, Tom Joy, Philip H. S. Torr, Amartya Sanyal, and Puneet K. Dokania. What makes and breaks safety fine-tuning? A mechanistic study, 2024. arXiv:2407.10264.arxiv.org/abs/2407.10264
  55. [55]Harethah Abu Shairah, Hasan Abed Al Kader Hammoud, Bernard Ghanem, and George Turkiyyah. An embarrassingly simple defense against LLM abliteration attacks, 2025. arXiv:2505.19056.
  56. [56]Nathan Truong. Abliteration mitigation via refusal aliases, 2026. arXiv:2608.18093.arxiv.org/abs/2608.18093
  57. [57]Aashiq Muhamed, Mona T. Diab, and Virginia Smith. Decoy direction optimization: A post-hoc defense against LLM abliteration, 2026. arXiv:2609.16204.arxiv.org/abs/2609.16204
  58. [58]Mark Russinovich. Fool’s gold: Defensive deception against safety-removal attacks on open-weight models, 2026. arXiv:2608.17202.arxiv.org/abs/2608.17202
  59. [59]Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, Rowan Wang, Zico Kolter, Matt Fredrikson, and Dan Hendrycks. Improving alignment and robustness with circuit breakers. In NeurIPS, volume 37, pages 83345–83373, 2024. arXiv:2406.04313.arxiv.org/abs/2406.04313
  60. [60]Rishub Tamirisa, Bhrugu Bharathi, Long Phan, Andy Zhou, Alice Gatti, Tarun Suresh, Maxwell Lin, Justin Wang, Rowan Wang, Ron Arel, Andy Zou, Dawn Song, Bo Li, Dan Hendrycks, and Mantas Mazeika. Tamper-resistant safeguards for open-weight LLMs. In ICLR, pages 101802–101829, 2025. arXiv:2408.00761.arxiv.org/abs/2408.00761
  61. [61]Domenic Rosati, Jan Wehner, Kai Williams, Łukasz Bartoszcze, David Atanasov, Robie Gonzales, Subhabrata Majumdar, Carsten Maple, Hassan Sajjad, and Frank Rudnizc. Representation noising: A defence mechanism against harmful finetuning. In NeurIPS, volume 37, pages 12636–12676, 2024. arXiv:2405.14577.arxiv.org/abs/2405.14577
  62. [62]Kevin Kuo, Virginia Smith, and Chhavi Yadav. Open-weight LLM fine-tuning defenses are susceptible to simple attacks, 2026. arXiv:2605.26526.arxiv.org/abs/2605.26526
  63. [63]Justin Cui, Wei-Lin Chiang, Ion Stoica, and Cho-Jui Hsieh. OR-Bench: An over-refusal benchmark for large language models. In ICML, volume 267 of PMLR, pages 11515–11542, 2025. arXiv:2405.20947.arxiv.org/abs/2405.20947
  64. [64]Isaac David and Arthur Gervais. Ablating safety: Mechanisms for removing alignment in language models for security applications, 2026. arXiv:2605.17413.arxiv.org/abs/2605.17413
  65. [65]Cristina Carleo, Pietro Liguori, Naghmeh Ivaki, and Domenico Cotroneo. Willing but unable: Separating refusal from capability in code LLMs via abliteration, 2026. arXiv:2606.05396.arxiv.org/abs/2606.05396
  66. [66]Aleksander Fafula. Abliteration is not a scalpel: Off-target effects of refusal removal on decision disposition across model families, 2026. arXiv:2607.17427.
  67. [67]Ju-Chun Ko. Uncensored is not unbiased: Abliteration removes refusals but not script-dependent bias about Taiwan, 2026. arXiv:2610.29342.arxiv.org/abs/2610.29342
  68. [68]Richard J. Young. Evaluating the robustness of large language model safety guardrails against adversarial attacks, 2025. arXiv:2511.22047.arxiv.org/abs/2511.22047
  69. [69]Xiangyu Qi, Boyi Wei, Nicholas Carlini, Yangsibo Huang, Tinghao Xie, Luxi He, Matthew Jagielski, Milad Nasr, Prateek Mittal, and Peter Henderson. On evaluating the durability of safeguards for open-weight LLMs. In ICLR, pages 62651–62681, 2025. arXiv:2412.07097.arxiv.org/abs/2412.07097
  70. [70]Richard J. Young. The selection rule can reverse LLM abliteration tool comparisons: Heretic 1.x, Heretic 2.0, OBLITERATUS and community releases on Qwen2.5 and Qwen2.5-Coder, 2026. In preparation.
  71. [71]Jesse Dodge, Suchin Gururangan, Dallas Card, Roy Schwartz, and Noah A. Smith. Show your work: Improved reporting of experimental results. In EMNLP-IJCNLP, pages 2185–2194, 2019. arXiv:1909.03004.arxiv.org/abs/1909.03004
  72. [72]Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, et al. Gemma 2: Improving open language models at a practical size, 2024. arXiv:2408.00118.
  73. [73]Evan Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafjord, Taira Anderson, David Atkinson, Faeze Brahman, Christopher Clark, Pradeep Dasigi, Nouha Dziri, Allyson Ettinger, Michal Guerquin, David Heineman, Hamish Ivison, Pang Wei Koh, Jiacheng Liu, Saumya Malik, William Merrill, Lester James V. Miranda, Jacob Morrison, Tyler Murray, Crystal Nam, Jake Poznanski, Valentina Pyatkin, Aman Rangapur, Michael Schmitz, Sam Skjonsberg, David Wadden, Christopher Wilhelm, Michael Wilson, Luke Zettlemoyer, Ali Farhadi, Noah A. Smith, and Hannaneh Hajishirzi. 2 OLMo 2 furious (COLM’s version). In Second Conference on Language Modeling (COLM), 2025. arXiv:2501.00656.
  74. [74]Team Olmo, Allyson Ettinger, Amanda Bertsch, Bailey Kuehl, David Graham, David Heineman, Dirk Groeneveld, Faeze Brahman, Finbarr Timbers, Hamish Ivison, Jacob Morrison, Jake Poznanski, Kyle Lo, Luca Soldaini, Matt Jordan, Mayee Chen, Michael Noukhovitch, Nathan Lambert, Pete Walsh, Pradeep Dasigi, Robert Berry, Saumya Malik, Saurabh Shah, Scott Geng, Shane Arora, Shashank Gupta, Taira Anderson, Teng Xiao, Tyler Murray, Tyler Romero, Victoria Graf, Akari Asai, Akshita Bhagia, Alexander Wettig, Alisa Liu, Aman Rangapur, Chloe Anastasiades, Costa Huang, Dustin Schwenk, Harsh Trivedi, Ian Magnusson, Jaron Lochner, Jiacheng Liu, Lester James V. Miranda, Maarten Sap, Malia Morgan, Michael Schmitz, Michal Guerquin, Michael Wilson, Regan Huff, Ronan Le Bras, Rui Xin, Rulin Shao, Sam Skjonsberg, Shannon Zejiang Shen, Shuyue Stella Li, Tucker Wilde, Valentina Pyatkin, Will Merrill, Yapei Chang, Yuling Gu, Zhiyuan Zeng, Ashish Sabharwal, Luke Zettlemoyer, Pang Wei Koh, Ali Farhadi, Noah A. Smith, and Hannaneh Hajishirzi. Olmo 3, 2025. arXiv:2512.13961.
  75. [75]Biao Zhang and Rico Sennrich. Root mean square layer normalization. In NeurIPS, volume 32. Curran Associates, Inc., 2019. arXiv:1910.07467.arxiv.org/abs/1910.07467
  76. [76]Jeonghoon Kim, Byeongchan Lee, Cheonbok Park, Yeontaek Oh, Beomjun Kim, Taehwan Yoo, Seongjin Shin, Dongyoon Han, Jinwoo Shin, and Kang Min Yoo. Peri-LN: Revisiting normalization layer in the transformer architecture. In ICML, volume 267 of PMLR, pages 30400–30436, 2025. arXiv:2502.02732.arxiv.org/abs/2502.02732
  77. [77]Songlin Yang, Jan Kautz, and Ali Hatamizadeh. Gated delta networks: Improving Mamba2 with delta rule. In ICLR, pages 29687–29707, 2025. arXiv:2412.06464.arxiv.org/abs/2412.06464
  78. [78]Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models, 2023. arXiv:2307.15043.arxiv.org/abs/2307.15043
  79. [79]Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford Alpaca: An instruction-following LLaMA model. https://github.com/tatsu-lab/stanford_alpaca, 2023.
  80. [80]Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J. Pappas, Florian Tramer, Hamed Hassani, and Eric Wong. JailbreakBench: An open robustness benchmark for jailbreaking large language models. In NeurIPS, volume 37, pages 55005–55029, 2024. Datasets and Benchmarks Track; arXiv:2404.01318.arxiv.org/abs/2404.01318
  81. [81]Paul Röttger, Hannah Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. XSTest: A test suite for identifying exaggerated safety behaviours in large language models. In NAACL, pages 5377–5400, 2024. arXiv:2308.01263.arxiv.org/abs/2308.01263
  82. [82]Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, Pieter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, and Sam Toyer. A StrongREJECT for empty jailbreaks. In NeurIPS, volume 37, pages 125416–125440, 2024. Datasets and Benchmarks Track; arXiv:2402.10260.arxiv.org/abs/2402.10260
  83. [83]Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. HarmBench: A standardized evaluation framework for automated red teaming and robust refusal. In ICML, volume 235 of PMLR, pages 35181–35224, 2024. arXiv:2402.04249.arxiv.org/abs/2402.04249
  84. [84]Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In ICLR, 2022. arXiv:2106.09685.arxiv.org/abs/2106.09685
  85. [85]Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with PagedAttention. In SOSP, pages 611–626, 2023. arXiv:2309.06180.arxiv.org/abs/2309.06180
  86. [86]Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen. Catastrophic jailbreak of open-source LLMs via exploiting generation. In ICLR, pages 13707–13727, 2024. arXiv:2310.06987.arxiv.org/abs/2310.06987
  87. [87]Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, Bill Yuchen Lin, Nathan Lambert, Yejin Choi, and Nouha Dziri. WildGuard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of LLMs. In NeurIPS, volume 37, pages 8093–8131, 2024. Datasets and Benchmarks Track; arXiv:2406.18495.arxiv.org/abs/2406.18495
  88. [88]Francisco Eiras, Elliott Zemour, Eric Lin, and Vaikunth Mugunthan. Know thy judge: On the robustness meta-evaluation of LLM safety judges, 2025. ICBINB Workshop at ICLR 2025. arXiv:2503.04474.arxiv.org/abs/2503.04474
  89. [89]Haiquan Zhao, Chenhan Yuan, Fei Huang, Xiaomeng Hu, Yichang Zhang, An Yang, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin, Baosong Yang, Chen Cheng, Jialong Tang, Jiandong Jiang, Jianwei Zhang, Jijie Xu, Ming Yan, Minmin Sun, Pei Zhang, Pengjun Xie, Qiaoyu Tang, Qin Zhu, Rong Zhang, Shibin Wu, Shuo Zhang, Tao He, Tianyi Tang, Tingyu Xia, Wei Liao, Weizhou Shen, Wenbiao Yin, Wenmeng Zhou, Wenyuan Yu, Xiaobin Wang, Xiaodong Deng, Xiaodong Xu, Xinyu Zhang, Yang Liu, Yeqiu Li, Yi Zhang, Yong Jiang, Yu Wan, and Yuxin Zhou. Qwen3Guard technical report, 2025. arXiv:2510.14276.arxiv.org/abs/2510.14276
  90. [90]Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonnell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. The language model evaluation harness. https://zenodo.org/records/12608602, 2024.
  91. [91]Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Ahran Chandrasekar, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. MMLU-Pro: A more robust and challenging multi-task language understanding benchmark. In NeurIPS, volume 37, pages 95266–95290, 2024. Datasets and Benchmarks Track; arXiv:2406.01574.arxiv.org/abs/2406.01574
  92. [92]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reichichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021. arXiv:2110.14168.arxiv.org/abs/2110.14168
  93. [93]Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models, 2023. arXiv:2311.07911.arxiv.org/abs/2311.07911
  94. [94]Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating large language models trained on code, 2021. arXiv:2107.03374.arxiv.org/abs/2107.03374
  95. [95]Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. Program synthesis with large language models, 2021. arXiv:2108.07732.arxiv.org/abs/2108.07732
  96. [96]James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. Algorithms for hyper-parameter optimization. In NeurIPS, volume 24, 2011. https://proceedings.neurips.cc/paper_files/paper/2011/hash/86e8f7ab32cfd12577bc2619bc635690-Abstract.html.
  97. [97]Edwin B. Wilson. Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22(158):209–212, 1927. https://doi.org/10.1080/01621459.1927.10502953.
  98. [98]Lawrence D. Brown, T. Tony Cai, and Anirban DasGupta. Interval estimation for a binomial proportion. Statistical Science, 16(2):101–133, 2001. https://doi.org/10.1214/ss/1009213286.
  99. [99]Robert G. Newcombe. Confidence intervals for the difference between binomial proportions based on paired data. Statistics in Medicine, 17(22):2635–2650, 1998. https://doi.org/10.1002/(SICI)1097-0258(19981130)17:22<2635::AID-SIM954>3.0.CO;2-C.
  100. [100]Quinn McNemar. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2):153–157, 1947. https://doi.org/10.1007/BF02295996.
  101. [101]Evan Miller. Adding error bars to evals: A statistical approach to language model evaluations, 2024. arXiv:2411.00640.arxiv.org/abs/2411.00640
  102. [102]Kung-Yee Liang and Scott L. Zeger. Longitudinal data analysis using generalized linear models. Biometrika, 73(1):13–22, 1986. https://doi.org/10.1093/biomet/73.1.13.
  103. [103]Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1):1–48, 2015. https://doi.org/10.18637/jss.v067.i01.
  104. [104]R. H. Baayen, D. J. Davidson, and D. M. Bates. Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59(4):390–412, 2008. https://doi.org/10.1016/j.jml.2007.12.005.
  105. [105]Rebecca DerSimonian and Nan Laird. Meta-analysis in clinical trials. Controlled Clinical Trials, 7(3):177–188, 1986.pubmed.ncbi.nlm.nih.gov/3802833
  106. [106]Joachim Hartung and Guido Knapp. A refined method for the meta-analysis of controlled clinical trials with binary outcome. Statistics in Medicine, 20(24):3875–3889, 2001.pubmed.ncbi.nlm.nih.gov/11782040
  107. [107]Joanna IntHout, John P. A. Ioannidis, and George F. Borm. The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method. BMC Medical Research Methodology, 14:25, 2014.DOI
  108. [108]Julian P. T. Higgins and Simon G. Thompson. Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21(11):1539–1558, 2002.pubmed.ncbi.nlm.nih.gov/12111919
  109. [109]Julian P. T. Higgins, Simon G. Thompson, and David J. Spiegelhalter. A re-evaluation of random-effects meta-analysis. Journal of the Royal Statistical Society: Series A (Statistics in Society), 172(1):137–159, 2009.pubmed.ncbi.nlm.nih.gov/19381330
  110. [110]Jacob Cohen. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1):37–46, 1960. https://doi.org/10.1177/001316446002000104.
  111. [111]Joseph L. Fleiss. Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5):378–382, 1971. https://doi.org/10.1037/h0031619.
  112. [112]Alvan R. Feinstein and Domenic V. Cicchetti. High agreement but low kappa: I. the problems of two paradoxes. Journal of Clinical Epidemiology, 43(6):543–549, 1990. https://doi.org/10.1016/0895-4356(90)90158-L.
  113. [113]Sture Holm. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2):65–70, 1979. https://www.jstor.org/stable/4615733.DOI
  114. [114]Gavin C. Cawley and Nicola L. C. Talbot. On over-fitting in model selection and subsequent selection bias in performance evaluation. Journal of Machine Learning Research, 11(70):2079–2107, 2010. https://www.jmlr.org/papers/v11/cawley10a.html.DOI
  115. [115]Skipper Seabold and Josef Perktold. Statsmodels: Econometric and statistical modeling with Python. In Proceedings of the 9th Python in Science Conference, pages 92–96, 2010. https://doi.org/10.25080/Majora-92bf1922-011.
  116. [116]Andrey Labunets. Refusal geometry reflects refusal training: diverse refusal prefixes can raise stable rank and weaken refusal vector ablation attacks, 2026. arXiv:2608.25390.arxiv.org/abs/2608.25390
  117. [117]Subhadip Mitra. Cross-generational transfer of adversarial attacks reveals non-monotonic safety alignment in LLMs, 2026. arXiv:2606.00813.
  118. [118]Dongyub Jude Lee, Jungseob Lee, Seungyoon Lee, Seongtae Hong, Suhyune Son, Suhyeong Eo, Jaehyung Seo, and Heuiseok Lim. Skin-deep: A geometric diagnostic for alignment fragility in large language model representations, 2026. Accepted to Findings of AACL-IJCNLP 2026. arXiv:2606.22676.arxiv.org/abs/2606.22676
  119. [119]Vincent Siu, Nicholas Crispino, Zihao Yu, Sam Pan, Zhun Wang, Yang Liu, Dawn Song, and Chenguang Wang. COSMIC: Generalized refusal direction identification in LLM activations. In Findings of the Association for Computational Linguistics: ACL 2025, pages 25534–25553, 2025. arXiv:2506.00085.arxiv.org/abs/2506.00085
  120. [120]Yuki Nakamura. Measuring alignment-induced activation shifts correctly: A template-controlled difference-in-differences protocol, 2026. v1–v2 titled “An Effective-Rank Audit of Alignment-Induced Activation Shifts: Confound Control, Constructive Calibration, and Limits”. arXiv:2605.24583.arxiv.org/abs/2605.24583
  121. [121]Arthur Wuhrmann, Gaetan Stein, Daniel Brunner, and Andrei Kucharavy. Measuring & mitigating over-alignment for LLMs in multilingual criminal law courts, 2026. arXiv:2606.23375.arxiv.org/abs/2606.23375
  122. [122]Lukas Weidener, Marko Brkić, Mihailo Jovanović, Emre Ulgac, and Aakaash Meduri. RefusalBench: Why refusal rate misranks frontier LLMs on biological research prompts, 2026. arXiv:2605.21545.arxiv.org/abs/2605.21545
  123. [123]Elisabetta Rocchetti and Alfio Ferrara. Refusal beyond a single direction: A preliminary comparison of diff-in-means and INLP, 2026. arXiv:2606.13720.arxiv.org/abs/2606.13720
  124. [124]Leo Schwinn, Moritz Ladenburger, Tim Beyer, Mehrnaz Mofakhami, Gauthier Gidel, and Stephan Günnemann. A coin flip for safety: LLM judges fail to reliably measure adversarial robustness, 2026. arXiv:2603.06594.arxiv.org/abs/2603.06594
  125. [125]Elias Hossain, Niloofar Yousefi, and Ser-Nam Lim. Agreement overstates evidence: Error dependence in LLM judge consensus, 2026. arXiv:2609.22512.arxiv.org/abs/2609.22512
  126. [126]Zora Che, Stephen Casper, Robert Kirk, Anirudh Satheesh, Stewart Slocum, Lev E McKinney, Rohit Gandikota, Aidan Ewart, Domenic Rosati, Zichu Wu, Zikui Cai, Bilal Chughtai, Yarin Gal, Furong Huang, and Dylan Hadfield-Menell. Model tampering attacks enable more rigorous evaluations of LLM capabilities. Transactions on Machine Learning Research, 2025. arXiv:2502.05209.arxiv.org/abs/2502.05209
  127. [127]Saad Hossain, Tom Tseng, Punya Syon Pandey, Samanvay Vajpayee, Matthew Kowal, Nayeema Nonta, Samuel Simko, Stephen Casper, Zhijing Jin, Kellin Pelrine, and Sirisha Rambhatla. TamperBench: Systematically stress-testing LLM safety under fine-tuning and tampering. In KDD, pages 9036–9047, 2026. arXiv:2602.06911.DOI
  128. [128]Ann-Kathrin Dombrowski, Dillon Bowen, Adam Gleave, and Chris Cundy. The safety gap toolkit: Evaluating hidden dangers of open-source models, 2025. arXiv:2507.11544.arxiv.org/abs/2507.11544
  129. [129]Kia-Jüng Yang, Dominik Meier, Jiachen Zhao, Terry Ruas, and Bela Gipp. Beyond a single direction: Chain-of-thought disrupts simple steering of refusal, 2026. ICML 2026 Workshop on Mechanistic Interpretability. arXiv:2605.26772.arxiv.org/abs/2605.26772
  130. [130]Stephen Casper, Kyle O’Brien, Shayne Longpre, Elizabeth Seger, Kevin Klyman, Rishi Bommasani, Aniruddha Nrusimha, Ilia Shumailov, Sören Mindermann, Steven Basart, Frank Rudzicz, Kellin Pelrine, Avijit Ghosh, Andrew Strait, Robert Kirk, Ken Hendrycks, Peter Henderson, Zico Kolter, Geoffrey Irving, Yarin Gal, Yoshua Bengio, and Dylan Hadfield-Menell. Open technical problems in open-weight AI model risk management. Transactions on Machine Learning Research, 2026. arXiv:2608.07514.arxiv.org/abs/2608.07514
  131. [131]Abhishek Mishra. One coordinate breaks abliteration on Gemma-3. LessWrong, https://www.lesswrong.com/posts/qw4a97cQrgFLJY5jC/one-coordinate-breaks-abliteration-on-gemma-3–1, 2026. Blog post, 15 September 2026; not peer reviewed.
  132. [132]Mark Russinovich, Yanan Cai, Keegan Hines, Giorgio Severi, Blake Bullwinkel, and Ahmed Salem. GRP-Obliteration: Unaligning LLMs with a single unlabeled prompt, 2026. arXiv:2602.06258.arxiv.org/abs/2602.06258
  133. [133]Xavier Bouthillier, Pierre Delaunay, Mirko Bronzi, Assya Trofimov, Brennan Nichyporuk, Justin Szeto, Nazanin Mohammadi Sepahvand, Edward Raff, Kanika Madan, Vikram Voleti, Samira Ebrahimi Kahou, Vincent Michalski, Tal Arbel, Chris Pal, Gael Varoquaux, and Pascal Vincent. Accounting for variance in machine learning benchmarks. In Proceedings of Machine Learning and Systems (MLSys), volume 3, pages 747–769, 2021. arXiv:2103.03098.arxiv.org/abs/2103.03098
  134. [134]Oscar Sainz, Jon Campos, Iker García-Ferrero, Julen Etxaniz, Oier Lopez de Lacalle, and Eneko Agirre. NLP evaluation in trouble: On the need to measure LLM data contamination for each benchmark. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10776–10787, 2023. arXiv:2310.18018.arxiv.org/abs/2310.18018
  135. [135]Yang Gao. How reliable is your jailbreak judge? calibration and adversarial robustness of automated ASR scoring, 2026. arXiv:2606.25487.arxiv.org/abs/2606.25487

Paper details

vlnc:W_wa94qjqc
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents