betaFeedback

WorksComputer Science

SafeStart: Mitigating the Overlooked Safety Cost of Active Expert Reduction in MoE LLM Serving

Abstract

Mixture-of-Experts (MoE) language models send each token through a few of many expert subnetworks. Serving methods make these models faster by running even fewer experts per token, and they are validated on accuracy benchmarks. We show that these methods can weaken the model's refusal of harmful requests, and we introduce SafeStart, which largely restores refusal while keeping most of the speedup. In vLLM on one RTX A6000, keeping about two thirds of the selected experts speeds up offline decoding by 15 to 28% at batch size 8. We find that, at this setting, under a jailbreak that tells the model not to refuse, the share of harmful requests that the model answers rises by 18.4 percentage points on Qwen1.5-MoE, 9.5 on OLMoE, and 6.6 on DeepSeek-V2-Lite, the three of nine models where it rises significantly, while MMLU accuracy drops by less than two points. We discover that most of this increase arises while the model processes the prompt and generates its first few tokens. SafeStart therefore runs all experts for the prompt and the first eight generated tokens, and the reduced set of experts afterwards, through a fused kernel in vLLM. With experts removed, SafeStart keeps at least 85% of the offline decode speedup at batch sizes 8 and 32 and brings the change under this jailbreak to between -2.1 and +1.3 points for three expert-reduction methods on these three models. On 889 held-out requests, in an analysis fixed before generation, truncation raises harmful compliance by 5.8 to 21.4 points, and SafeStart leaves a rise of 0.6 to 2.6 points, non-inferior to the original model on two of the three models. Our results show that faster MoE serving need not cost safety: a serving system can keep most of the gain from running fewer experts by running all of them at the start of each response, and it should test refusal, not only accuracy, before it enables such a method.

Introduction

Most recent open-weight language models use Mixture-of-Experts (MoE) layers. In such a layer, a small network called the router sends each token to kk of EE expert subnetworks Shazeer et al., [48]; Fedus et al. [13]; Dai et al. [10]; Yang et al. [64]. Many inference methods make MoE models cheaper by running fewer experts per token. Uniform top-kk truncation lowers kk for every token, and a study of twelve MoE models recommends keeping about two thirds of the selected experts as an initial operating point Chen et al., [7]. Other methods skip experts per token Lu et al., (2024); Huang et al. [22]; Zeng et al. [68]; Yue et al. [67]; Xu et al. (2026c), reweight the kept experts Anonymous, [1]; Chen & Yao [6], or share experts across the requests of a batch Gupta et al., [17]; Wu et al. (2026a); Oncescu et al. [41]; Vankov et al. (2026). In vLLM and llama.cpp, truncation is a one-line configuration change Kwon et al., [26]; Gerganov & llama.cpp contributors [15].

These methods are checked on accuracy, perplexity, or reasoning benchmarks. Refusal may break without accuracy changing. In MoE models, refusal depends on a few experts and routers whose removal leaves accuracy largely intact Lai et al., 2025; te Lintelo et al., 2026; Wu et al., 2026b; Jiang et al., 2026; Xu et al., 2026b; Zvenyach et al., 2026. Other serving optimizations, such as KV-cache compression and lossy speculative decoding, can remove refusals at a small accuracy cost Xu et al., (2026a); Ma et al. (2026); Zhang et al. [69]. Alignment also changes mainly the first tokens of a response ([30, 71] (Qi et al. 2025)). No prior work, to our knowledge, tests expert reduction at its published operating points for refusal, locates where refusal changes inside a serving engine, and fixes it there while keeping the speedup.

We do all three (Figure 1). We compare each modified model with the unmodified model prompt by prompt on nine MoE models, measure the noise of the comparison with reruns, and label responses with two automated judges and two human annotators. We find:

Figure 1. Safety and speed in vLLM. Left: the expert slots each token of one request uses under the original model, reduction alone, and SafeStart, drawn for top-kk on Qwen1.5-MoE (k=4k=4, k′=3k'=3); SafeStart runs all experts for the prompt and the first N=8N=8 generated tokens and the reduced budget afterwards. Right: for each expert-reduction rule alone (orange) and with SafeStart (blue, our vLLM patch with the fused kernel), the change in harmful compliance under a refusal-suppression jailbreak (top; in percentage points, pp, of the attack success rate, ASR: the share of the 472 prompts whose response Llama Guard 3 labels harmful) and the decode speedup over the original model at batch size 8 (bottom). Three rules keep about ⌈2k/3⌉\lceil2k/3\rceil experts per decode token, and the fourth group is top-kk at about half the experts (Appendix § 8). Whiskers: 95% bootstrap intervals over prompts (top) and timing runs (bottom); hollow: exact McNemar p≥0.05p \ge0.05; gray band: the ±3\pm3 pp equivalence margin. Bottom: one request on Qwen1.5-MoE with greedy decoding. The responses of the original model and of truncation share their first generated token and split at the second (bold); SafeStart computes the prompt and the first eight generated tokens (shaded) with all experts and refuses. Tags: both judges’ labels. Responses are cut after their opening.

  • The speedup is real but limited to small batches. We time batch sizes 1, 8, 32, 128, and 256. Keeping about two thirds of the experts helps most at batch size 8, where decoding is 15 to 28% faster on all six timed models; at batch size 128 it is at most 5% faster on four of them. On the three models we profile, the expert weights that each decoding step reads from GPU memory explain the speedup (§ 4.1).

  • Refusal weakens while accuracy holds. Under a jailbreak that tells the model not to refuse, harmful compliance rises by 18.4 percentage points on Qwen1.5-MoE (13.6 under human labels), 9.5 on OLMoE, and 6.6 on DeepSeek-V2-Lite, while MMLU drops by at most 1.8 points. Top-pp routing, which keeps experts until their total weight reaches a threshold pp, raises it by up to 30.1 points (§ 4.2). Figure 1 (bottom) shows one request that truncation turns from a refusal into step-by-step instructions.

  • The change arises in the prompt and the first generated tokens. On our evaluation prompts, running all experts for the prompt and the first generated token leaves at most 0.9 points of the increase (§ 4.3).

  • SafeStart fixes it at little cost. SafeStart runs all experts for the prompt and the first N=8N=8 generated tokens and the reduced budget afterwards, through a fused kernel in vLLM. It keeps at least 85% of the

offline decode speedup at batch sizes 8 and 32, and, under the same jailbreak, brings the change in harmful compliance to between −2.1 and +1.3 points for three reduction methods. On 889 held-out requests, where truncation raises harmful compliance by 5.8 to 21.4 points under this jailbreak, SafeStart leaves a rise of at most 2.6 points. With half of the experts it decodes 38 to 44% faster than the original model at batch size 8 and scores 3.6 to 3.9 GSM8K points higher than truncation (§ 5).

Our contributions are (1) a paired, noise-calibrated refusal audit of expert reduction in vLLM on nine models, with the speedup explained by expert weight reads; (2) the discovery that the refusal change arises in prefill and the first decode steps; and (3) SafeStart, an engine patch that applies to top-kk, top-pp, and threshold-based expert budgets. We will release the harness and the patch publicly.

Background and Related Work

MoE Layers and How They Are Served

An MoE layer replaces one feed-forward network with EE smaller ones, the routed experts fif_i. For each token representation xx, a router computes scores p(x)p(x) and selects the set Tk(x)T_k(x) of the kk highest-scoring experts. Some models also have shared experts fsf_s that process every token. The layer output is

y(x)=gs(x)fs(x)+∑i∈Tk(x)wi(x)fi(x),(1)y(x) = g_s(x) f_s(x) + \sum_{i \in T_k(x)} w_i(x) f_i(x), \tag*{(1)}

where gsg_s is a sigmoid gate in Qwen1.5-MoE and Qwen2-57B and the constant 1 in DeepSeek-V2-Lite, DeepSeekMoE-16B, Moonlight, and Ling-lite (gs=0g_s = 0 without shared experts), and experts are ranked by the router’s selection score. Most of our models use p(x)=softmax⁡(Wrx)p(x) = \operatorname{softmax}(W_r x); Moonlight uses sigmoid scores with a learned selection bias.

Gate normalization. Models differ in how they weight the selected experts. With top-kk renormalization, the weights wi=pi/∑j∈Tkpjw_i = p_i/\sum_{j \in T_k} p_j always sum to one (2.446 in Moonlight, which scales them by a constant). Without it, wi=piw_i = p_i, and the routed experts together carry weight mk(x)=∑j∈Tkpj<1m_k(x) = \sum_{j \in T_k} p_j < 1, the gate mass. This matters when kk is lowered at inference time: without renormalization, removing experts also lowers the gate mass, so the routed experts contribute less to the output. Table 1 lists the nine models we study.

ModelParamsLLEEkkSh.RenormKeptk′k'
Qwen1.5-MoE†14.3/2.7246041no■■■□□3, 2
DeepSeek-V2-Lite15.7/2.7266462no■■■■■□□5, 4, 3
Moonlight-16B16.0/2.9266462yes■■■■■□□5, 4, 3
Ling-lite-1.516.8/2.8286462yes■■■■■□□5, 4, 3
OLMoE-1B-7B†6.9/1.3166480no■■■■■■□□6, 5, 4
Qwen3-30B-A3B†30.5/3.44812880yes■■■■■■□□6, 4, 2
GPT-OSS-20B20.9/4.2243240yes■■■□□3, 2
Held out, added after the confirmatory family was fixed
DeepSeek-MoE-16B16.4/2.8276462no■■■■□□5, 4, 3
Qwen2-57B-A14B57.4/14.3286481no■■■■■■□□6, 5, 4

Table 1. MoE models. Params: total / active parameters per token (billions, counted from the checkpoints, embeddings included); LL: MoE layers; EE: routed experts; kk: experts selected per token; Sh.: shared experts; Renorm: routed weights renormalized over the selected experts; Kept: the k′=⌈2k/3⌉k^{\prime}=\lceil2k/3\rceil slots of the confirmatory point; k′k^{\prime}: truncation levels run; †: also in the earlier harness of Appendix D. Appendix A gives revisions and weight formats.

Prefill, decode, and batching. A serving engine such as vLLM ([26]) first processes the whole prompt in one forward pass, called prefill, whose last position produces the first generated token. It then generates one token per request per forward pass, called a decode step, and runs the pending requests together in a batch. At small batch sizes a decode step is limited by how fast the GPU can read model weights from its memory, not by arithmetic. In an MoE layer, a decode step must read the weights of every expert that at least one token of the batch selects.

Running Fewer Experts

Per-token methods. Uniform top-kk truncation keeps only the k′<kk' < k highest-scoring experts of every token. Chen et al. [7] report that keeping about two thirds of the selected experts retains 98.8% of accuracy on average with 1.2 to 1.7× decode throughput. Dynamic methods keep a different number of experts per token, for example by skipping experts with small gate weights or by learning the count ([22, 68, 67] (Lu et al. 2024; Xu et al. 2026c)). Other methods reweight the kept experts ([1, 6]). All of these change every token, including the prompt tokens processed in prefill.

Batch-level methods. Other methods re-route tokens to experts that the batch already reads, or cap the number of distinct experts a batch reads ([17, 41] (Wu et al. 2026a; Vankov et al. 2026)). With them, a token’s computation depends on the other requests in its batch. Static pruning and merging produce smaller checkpoints instead ([28]), and serving systems also speed up the selected experts with sparse kernels, expert parallelism, and offloading ([46, 14, 23, 4]).

Refusal and Its Fragility

An aligned model refuses harmful requests. A jailbreak template wraps the request in text that discourages refusal, for example by forbidding apologies ([55]). Refusal is mediated by a single direction in the model’s hidden states, the refusal direction, whose removal stops the model from refusing ([2]), and alignment changes mostly the first tokens of a response ([30, 71] (Qi et al. 2025)). SafeDecoding and D-STT defend these positions with an extra model or a trigger token ((Xu et al., 2024; Gu et al., 2026)).

In MoE models, removing or steering a few experts or routers selected for their link to refusal can remove refusal ((Lai et al., 2025; te Lintelо et al., 2026; Wu et al., 2026b; Fayyaz et al., 2026; Jiang et al., 2026; Xu et al., 2026b; Zhang et al., 2026b)), and defenses repair those components ([29, 50, 38]). Static MoE compression also degrades safety ([51]). In dense models, quantization and pruning change jailbreak behavior ([21, 56, 18, 65]), although weight quantization is close to neutral for most models [43]. KV-cache compression can remove refusals (Xu et al., 2026a; Ma et al., 2026), refusal monitors disagree under compression [5], and AnchorKV protects them during cache eviction (Ni & Lao, 2026). SecureSD verifies draft tokens more strictly at early positions in speculative decoding [69]. Unlike these works, we study methods that remove experts for speed, not for their link to refusal, and we fix the problem inside the serving engine without an extra model, targeting the original model’s refusal rather than a higher one.

Evaluation Setup

Threat Model

An operator serves an aligned MoE model and turns on an expert-reduction method that was checked on accuracy benchmarks. A user without access to the weights, routing, or serving configuration submits harmful requests, optionally wrapped in a published jailbreak template. We ask whether the method raises the rate at which the served model complies, compared with the same model served without it. The user does not know the method or choose the batch.

Models, Prompts, and Conditions

We study the nine models of Table 1 [33, 11, 10, 63, 32, 31, 39, 64, 42] in vLLM 0.11.0 [26] with CUDA graphs, chunked prefill, and prefix caching. The prompts are the 159 standard behaviors of the HarmBench test split and the 313 StrongREJECT prompts [36, 49], 472 in total. Each is presented plainly and inside two templates distributed with StrongREJECT [55]: refusal suppression, which forbids apologies and disclaimers, and prefix injection, which dictates the opening words of the answer. We call these three presentations framings. Two more templates, an evil-confidant role play and a distractor that hides the request among unrelated tasks, serve as robustness checks. Decoding is greedy with at most 512 new tokens (1,536 for GPT-OSS, which reasons before it answers). Appendix D reports an earlier harness, Study 1; we call the main one Study 2.

The original model is the unmodified model with its trained kk. Truncation to k′k^{\prime} sets vLLM’s num_experts_per_tok override, so every prompt and generated token keeps k′k^{\prime} experts. Our main operating point, the confirmatory point, is k′=⌈2k/3⌉k^{\prime}=\lceil2k/3\rceil. A small patch of vLLM’s expert selection implements the other conditions: controls that separate removed experts from lost gate mass, and versions that change the expert count by token position (Appendix § 7). We also emulate position-dependent budgets by switching models during generation (emulation; Appendix § 7).

Measuring Refusal

Harmful compliance. Llama Guard 3 8B [24, 33] labels each response as safe or harmful. The attack success rate (ASR) is the share of the 472 requests whose response it labels harmful, and we report changes in percentage points (pp). The HarmBench classifier [36] serves as a second judge. Two annotators, blind to the experimental condition, label 587 responses, including both responses of every prompt where truncation changes the judge’s label on Qwen1.5 and DeepSeek-V2-Lite under refusal suppression; they agree on 97.3% of items (κ=0.93\kappa=0.93; Appendix C). We measure accuracy with MMLU and GSM8K and over-refusal with XSTest [20, 9, 47].

Paired statistics. Every condition uses the same prompts and greedy decoding as the original model, so each prompt yields a pair of outcomes. A forward flip is a prompt whose response turns from safe to harmful, and a reverse flip the opposite. We test each change with the exact McNemar test on the flipped prompts [37] and report Tango 95% intervals [52]. The confirmatory family is truncation at k′=⌈2k/3⌉k' = \lceil2k/3 \rceil for seven models and three framings, 21 tests with Holm correction; other tests, except the held-out analysis fixed in advance, are exploratory. We call a change equivalent to zero when its 90% interval lies within ±3\pm3 pp. Batched GPU kernels are not exactly reproducible, so we rerun the original model four times with shuffled prompt order: across the 84 rerun comparisons of the seven models of Table 2, the net change never exceeds 2.75 pp Appendix § 8. Appendix § 7 gives the full analysis plan.

Table 2. Truncation at the operating point that keeps about two thirds of the selected experts (k′=⌈2k/3⌉k' = \lceil2k/3\rceil). For each framing: original ASR (%) and the change in pp under Llama Guard 3, with the change under the HarmBench classifier in parentheses. Bold: Holm-adjusted exact McNemar p<0.05p < 0.05 over the confirmatory family of 21 tests (7 models ×\times 3 framings). Shading: size and sign of the Llama Guard 3 change. Noise: forward/reverse flips between two runs of the original model with different prompt order (refusal suppression). Other doses appear in Appendix B.

ModelSharedRenormk′/kk'/kPlain OriginalPlain Δ\DeltaRefusal suppression OriginalRefusal suppression Δ\DeltaPrefix injection OriginalPrefix injection Δ\DeltaNoise
Qwen1.5-MoEyesno3/47.6+0.0 (+1.1)46.6+18.4 (+16.5)79.9+6.8 (+7.0)9/11
DeepSeek-V2-Liteyesno4/612.3+3.4 (+2.8)43.6+6.6 (+5.3)45.3−2.3 (−5.1)17/19
Moonlight-16Byesyes4/637.3−4.9 (−4.9)86.9−5.3 (−6.4)91.7−0.6 (−2.5)3/5
Ling-lite-1.5yesyes4/68.7+0.8 (+1.7)41.7+0.8 (−4.7)58.5+6.8 (+7.6)14/10
OLMoE-0125nono6/83.4−0.6 (+0.4)24.2+9.5 (+8.3)30.9+5.3 (+5.3)9/12
Qwen3-30B (FP8)noyes6/81.3+0.0 (−0.4)10.0−4.2 (−2.5)5.5−3.6 (−3.2)5/3
GPT-OSS-20Bnoyes3/41.7−0.6 (+0.0)6.6−0.6 (+0.0)0.2+0.0 (+0.0)12/4

Table 2.

Measuring Speed

All timings use one NVIDIA RTX A6000 GPU. Offline timings decode a fixed batch of requests together, with prefix caching off. Batch size is the number of requests that the engine decodes together. Decode time is the time to generate 256 tokens per request minus the time to first token. The decode speedup of a condition is the original model’s decode time divided by the condition’s, minus one; a speedup ss saves a fraction s/(1+s)s/(1 + s) of decode time. Prompts are GSM8K questions, each request generates 256 tokens, and each timing is the median of three repetitions after one warm-up. Every speedup compares a condition with the original model timed in the same job, and the jobs behind Figure 1 and Figure 2 interleave the conditions over rounds so that they share the same machine state. Overhead is the extra decode time, or in online serving the extra time per output token, of our patch when it removes no expert. For online serving we report the median time per output token under Poisson request arrivals. To explain the speedup we count, per MoE layer and decode step, the distinct experts whose weights the batch reads, and we profile the GPU time of each kernel (Appendix § 8).

Figure 2. Decode time saved against the original model, by batch size, for truncation at k′=⌈2k/3⌉k' = \lceil2k/3 \rceil and for SafeStart with N=8N = 8 (95% bootstrap intervals). Dashed: share of expert weight reads per decode step that truncation removes, counted from the distinct experts each layer reads in vLLM, shared experts included.

Results: Faster Decoding, Weaker Refusal

The Speedup Comes from Reading Fewer Expert Weights

Truncation helps most at small batch sizes. At k′=⌈2k/3⌉k' = \lceil2k/3 \rceil, the decode speedup peaks at batch size 8 on all six timed models, at 15 to 28%. At batch size 128 it is at most 5% on four of them (Appendix § 8; Figure 2 shows the time saved on three models).

The reason is memory traffic. Up to batch size 256, each routed expert processes on average at most 32 tokens per step, too few to keep the GPU’s arithmetic units busy, so the time of an MoE layer follows the expert weights it reads from GPU memory. For a single request, truncation removes 1−k′/k1 - k'/k of the routed expert reads, a quarter to a third. Counted over the whole batch, shared experts included, the share of expert weight reads that truncation removes peaks at batch size 8 on Qwen1.5 and DeepSeek-V2-Lite, the batch size at which the speedup also peaks, and then shrinks, because a larger batch selects almost every expert anyway (Figure 2, dashed). We show that, on the three profiled models, the reads removed, weighted by the share of step time spent in expert kernels, predict the profiled GPU time saved per decode step to within 2.4 points at batch sizes 1 to 128 (Appendix § 8). On DeepSeek-V2-Lite at batch size 1, only 13% of the profiled saving appears end to end, against 93% at batch size 8, because its attention backend supports only piecewise CUDA graphs (Appendix § 8).

Takeaway. Truncation saves the most time near batch size 8, as in interactive serving, and at most 3.9% of decode time at batch size 128 on these three models, where it adds a refusal cost for almost no speedup.

Refusal Weakens While Accuracy Holds

The effect. Table 2 shows the confirmatory tests. Under refusal suppression, truncation raises ASR by 18.4 pp on Qwen1.5-MoE (from 46.6 to 65.0%), 9.5 pp on OLMoE, and 6.6 pp on DeepSeek-V2-Lite, all significant after Holm correction. The Hallucination classifier and sampled decoding agree on Qwen1.5 (Appendix § 8). Moonlight and Qwen3 become safer, and Ling-lite and GPT-OSS do not change significantly under refusal suppression. Removing more experts makes the increase larger, up to +36.0 pp on OLMoE with half of the experts (Appendix § 8).

Human labels confirm the largest effect. Counting only clearly harmful responses, the annotators measure +13.6 pp on Qwen1.5 (95% CI [9.5, 19.1]). Counting partial compliance, they measure +17.0 pp, close to the judge’s +18.4 pp: truncation adds partial answers, 89 to 97% of which Llama Guard 3 counts as harmful. On DeepSeek-V2-Lite the human change is +2.5 pp (p=0.088p = 0.088), or +4.7 pp with partial compliance (p=0.007p = 0.007; Appendix § 10).

Which models are affected. We find that, with Holm correction over each model’s budgets, every model that does not renormalize its gate weights rises significantly at some budget under refusal suppression, including two models we added after fixing the test family, which rise only when more experts are removed. Of the four models that renormalize, only Qwen3 rises significantly, and only at 2 of 8 experts, where its accuracy collapses (Appendix § 8). The effect also depends on the template: on plain requests Qwen1.5 does not change on net, Ling-lite rises by +6.8 pp under prefix injection, and DeepSeek-V2-Lite rises by +13.1 pp under the evil-confidant template (Appendix § 8).

Other reduction methods. The problem is not specific to truncation (Table 3). We apply two dynamic methods without retraining, calibrated so that each generated token keeps about k′k' experts: threshold skipping, which drops experts whose weight is below a fraction of the top weight (Lu et al., 2024), and top-pp routing, which keeps experts until their total weight reaches pp (Huang et al., [22]). Top-pp routing raises ASR on all three models, by up to 30.1 pp, at a larger GSM8K cost than truncation. Threshold skipping does not raise it significantly at this budget, and it keeps more experts than truncation at the final prompt token, whose computation produces the first generated token (Appendix § 8). Both methods decode more slowly than truncation at batch size 8.

RuleQwen1.5DeepSeek-V2-LiteOLMoE
GSMalone+SafeStartGSMalone+SafeStartGSMalone+SafeStart
Top-kk−2.4+18.4−0.6−1.5+6.6+0.6−2.7+9.5+1.3
Threshold−1.5−1.9+0.0−1.3+1.3−0.6−0.8+1.9+1.3
Top-pp−7.9+30.1−2.1−4.2+11.2+1.3−5.2+8.9+0.0

Table 3. Expert-reduction rules at about two thirds of the selected experts: ASR change under refusal suppression (pp, Llama Guard 3), alone and with SafeStart (N=8N = 8) in the engine with the fused kernel; GSM: GSM8K change (points); bold: exact McNemar p<0.05p < 0.05; shading: size and sign of the change. Threshold skipping and top-pp keep about k′=3k' = 3, 4, and 6 experts per decode token (calibration in Appendix B). Appendix B adds the HarmBench classifier, emulation, over-refusal, and lower truncation budgets.

Accuracy checks miss the problem. At k′=⌈2k/3⌉k' = \lceil2k/3\rceil, MMLU drops by at most 1.8 points and GSM8K by at most 3.5 points on the eight models with accuracy results

(Appendix B). Over-refusal on XSTest also does not reveal it: on Qwen1.5, refusal of safe prompts falls from 24.0 to 18.8%. The increase is also not a generic effect of lost accuracy: Qwen1.5’s official 4-bit GPTQ checkpoint loses more accuracy than truncation yet lowers ASR by 8.5 pp.

Most of the Change Arises in the Prompt and the First Tokens

Truncated responses split off at the first token. For each prompt we find the first generated token at which the response under truncation differs from the original model’s response (Figure 3). On Qwen1.5, 92 of the 101 responses that truncation turns harmful already differ at the first token. Responses whose label does not change diverge later, and two runs of the original model later still (Appendix E).

Figure 3. Share of response pairs whose truncated and original responses have differed by each generated token (refusal suppression). Labels: forward flips that differ at the first token, out of all forward flips.

The first token and the prompt carry the change. We discover that truncating only the computation of the prompt and the first token, and then continuing with the original model, reproduces 91% of the increase on Qwen1.5 and 62% on OLMoE. Conversely, SafeStart with N=1N = 1, which runs all experts for the prompt and the first generated token and truncates afterwards, leaves at most 0.9 pp on all three models on our evaluation prompts. Figure 4 splits the prompt further: on OLMoE the earlier prompt tokens carry almost the whole increase, and on Qwen1.5 about 40%.

Prefill and opening-token interventions

Figure 4. Prefill and opening-token interventions (k′=⌈2k/3⌉k' = \lceil2k/3 \rceil, 472 prompts, refusal suppression, Llama Guard 3). Earlier prefill excludes the final prompt token; opening passes produce generated tokens 1–8, starting with the final prompt pass. Blue: full budget; orange: reduced. All conditions reduce later decoding. Points: ASR changes against the original model; hollow: exact McNemar p≥0.05p \ge0.05 (paired tests in Appendix § 8).

A refusal direction carries the first-token change. We find that truncation moves the hidden state of the final prompt token along the refusal direction [2] toward the harmless side. On Qwen1.5 and OLMoE, restoring the original position along this direction removes 97 and 71% of the first-token increase, while random directions do not; imposing the shift on the original model reproduces 100 and 75% of the increase, on Qwen1.5 mostly on the same prompts (Appendix E). The lost gate mass alone raises ASR on all three affected models, and removing experts with the gate mass restored raises it only on Qwen1.5 (Appendix E).

SafeStart: Full Experts at the Start

Design and Implementation

Section 4.3 shows that the refusal change happens mostly while the model processes the prompt and writes its first tokens, while most generated tokens come later. SafeStart therefore spends the full expert budget only there. For a request with a prompt of PP tokens, it runs all kk experts for prefill and for the forward passes that produce the first NN generated tokens, and the reduced budget for every later token. We use N=8N = 8. SafeStart gives up truncation’s prefill saving, so the time to first token stays within 1% of the original model’s at batch size 8.

Engine patch. We implement SafeStart in vLLM 0.11.0 in about 160 lines of Python and Triton, without changing vLLM’s source (Figure 5). (1) After vLLM computes the position of every scheduled token, the patch marks the tokens at position P−1+NP - 1 + N or later in a GPU buffer that the captured CUDA graphs read. (2) After vLLM selects each token’s top-kk experts, one fused Triton kernel per MoE layer replaces, for marked tokens only, the slots outside the reduced budget with a sentinel expert of weight zero. The same kernel applies a top-k′k', top-pp, or threshold budget. (3) vLLM’s expert map, built for expert parallelism, sends the sentinel to −1, so the expert kernel skips its weights. The rule depends on the output position rather than on whether a step is prefill or decode, so it stays correct when vLLM recomputes tokens after preemption or after a prefix-cache hit.

SafeStart in vLLM

Figure 5. SafeStart in vLLM (thick outlines: ours; (1) to (3): steps in the text). Top: one request; the first marked pass, at position P−1+NP-1+N, produces yN+1y_{N+1}. Bottom: one engine step; only request C is marked, so its slot outside the budget becomes the sentinel ∅\varnothing, whose weights the GEMM skips.

SafeStart Restores Refusal

Main result. With N=8N = 8, SafeStart changes ASR under refusal suppression by −0.6, +0.6, and +1.3 pp on Qwen1.5, DeepSeek-V2-Lite, and OLMoE, where truncation adds 18.4, 6.6, and 9.5 pp; the difference from truncation is significant on each model (p<0.005p < 0.005). Pooled over the three models, the change is +0.4 pp (90% CI [−0.6, 1.6]) under Llama Guard 3 and +1.5 pp ([0.2, 2.8]) under the HarmBench classifier, both within our ±3 pp margin. Against each of the five runs of the original model, no change under SafeStart is significant, while every change under truncation is (Appendix B).

It works for other reduction methods and lower budgets. SafeStart brings every increase of top-pp routing and threshold skipping in Table 3 to at most +1.3 pp. With about half of the experts, where truncation adds 13.4 to 36.0 pp, it leaves at most +0.6 pp. It also removes the evil-confidant increase on DeepSeek-V2-Lite and the increase under sampled decoding on Qwen1.5 (Appendix B).

LEXI: allocating experts across layers. LEXI [8] profiles how each layer’s output changes when fewer experts process synthetic Gaussian inputs. It then uses an evolutionary search to choose a fixed expert count kjk_j for each layer, subject to a routed-expert budget B=∑jkjB = \sum_j k_j. We vary BB, rather than a per-token skipping threshold; the three-model comparison uses B=80B = 80, 100, and 130 on Qwen1.5, OLMoE, and DeepSeek-V2-Lite, respectively, against original budgets of 96, 128, and 156.

Table 4 tests SafeStart with our paper-based LEXI reimplementation. On a 448-target split (288 AdvBench requests [72] in six harm categories and 160 HarmBench behaviors), under refusal suppression and Llama Guard 3, LEXI gives ASR of 51.8% on Qwen1.5, 46.4% on OLMoE, and 67.0% on DeepSeek-V2-Lite, above the corresponding original-model values. “Original” denotes native routing with the full expert budget. Protecting prefill and the first few generated tokens (N=8N = 8 in this paper) brings ASR toward the original model on all three models. On DeepSeek-V2-Lite, SafeStart gives 53.3% ASR, compared with 67.0% for LExI and 51.3% for the original model. These runs use a 256-token cap, so their ASR is not directly comparable to our 472-prompt main evaluation. Appendix § 14 gives the reimplementation and full protocol; LExI’s runtime with protection was not measured.

ModelOriginalLExISafeStart + LExI
Qwen1.5-MoE35.751.836.4
OLMoE-012515.646.418.5
DeepSeek-V2-Lite51.367.053.3

Table 4. LExI ASR (%) on 448 targets, with and without SafeStart (N=8N = 8).

ACE: skipping experts per token. ACE (Xu et al., 2026c) combines two weight-derived estimates of expert contribution—a global capacity estimate and a router-conditioned directional estimate—with each token’s routing gates. It skips an expert slot only when both estimates identify a low contribution, always keeping the top-ranked expert. We adjust the skipping threshold τ\tau, fixed on Wiki-Text calibration text, to target approximately 25% or 50% of the originally routed expert slots being skipped on Qwen1.5 and OLMoE, and 10% on DeepSeek-V2-Lite; these percentages are computation targets, not fractions of permanently deleted experts or changes in accuracy.

We adapt the released ACE implementation to these three models in Hugging Face Transformers (Qwen1.5 and OLMoE in Table 5), using the same 448 targets, refusal suppression, Llama Guard 3, and a 256-token cap. None of these checkpoints is officially supported by ACE; Appendix § 14 gives the ports, protocol differences, and paired tests. In the table, “Original” is the native model without ACE; “SafeStart + ACE” uses the same ACE threshold, without increasing later skipping to compensate for the protected opening. Protection therefore reduces whole-run expert skipping; the comparison holds the ACE threshold fixed, rather than total expert work (Appendix § 14). At about 50% skipping, ACE raises ASR by 21.0 pp on Qwen1.5 and 12.9 pp on OLMoE; SafeStart reduces these changes to −0.4 and +1.8 pp. Both direct reductions are significant after Holm correction over the three models. At both reported budgets, protection meets the +3 pp non-inferiority criterion on Qwen1.5, but not OLMoE (Appendix § 14).

ModelOriginalACE ∼\sim25% ACEACE ∼\sim25% SafeStart + ACEACE ∼\sim50% ACEACE ∼\sim50% SafeStart + ACE
Qwen1.536.3845.5434.3857.3735.94
OLMoE-012514.9617.4116.0727.9016.74

Table 5. ACE ASR (%) at two skip targets, with and without SafeStart (N=8N = 8).

On DeepSeek-V2-Lite, at approximately 10% ACE skipping, ASR is 57.1% with ACE and 53.1% with SafeStart. We do not report its 25% and 50% settings as safety results because accuracy and perplexity checks indicate unreliable model output at those settings.

Convergent evidence across methods. Across the reported LExI and ACE comparisons, SafeStart moves ASR toward the corresponding original model. The agreement across layer-wise allocation and per-token skipping supports the paper’s central finding: refusal is sensitive to expert reduction, and preserving the prompt and opening can substantially mitigate the resulting behavioral change. These point-estimate movements do not establish equivalence to the original model; the separate non-inferiority tests assess the specified +3 pp margin.

Its limit is the opening it protects. SafeStart protects the model’s own opening. A template that dictates the opening words or places the harmful task later can move the decision past the first NN tokens: under prefix injection on Ling-lite, SafeStart leaves +12.5 pp against +6.8 pp for truncation, and a distractor template on Qwen1.5 leaves +5.1 pp where truncation leaves −0.2 pp. Across all 40 combinations of model, template, prompt set, and sampling seed that we ran with the kernel, SafeStart is never significantly worse than both truncation and the original model (Holm; Appendix § 8).

How many tokens to protect. Figure 6 varies NN. On our evaluation prompts, protecting only the first generated token (N=1N = 1) already removes the increase on all three models, and under refusal suppression no NN from 1 to 32 leaves a significant increase under Llama Guard 3 (exploratory, unadjusted). We keep N=8N = 8 as a margin, and the held-out requests confirm the choice: pooled over the three models, N=8N = 8 removes 90% of the rise in ASR that truncation causes, against 81% for N=1N = 1 (p=0.043p = 0.043). Protecting more tokens costs speed, from 95 to 97% of truncation’s batch-8 speedup kept at N=1N = 1 to 77 to 82% at $N = 32.

Figure 6. Protection length NN (k′=⌈2k/3⌉k^\prime= \lceil2k/3 \rceil, refusal suppression). Top: change in ASR against the original model (Llama Guard 3, 95% prompt-bootstrap intervals; hollow: exact McNemar p≥0.05p \ge0.05; gray band: ±\pm3 pp margin) on the 472 evaluation prompts and, at right, on 889 held-out prompts; trunc. (orange): truncation of every token. N=0N = 0 runs every prompt token but the last at full kk. Bottom: share of truncation’s decode speedup at batch size 8 that SafeStart keeps.

Held-out check. After freezing every setting, we generated responses for 889 AdvBench ([72]) and CatQA ([3]) requests that no design choice used, and analyzed them as specified before generation (Appendix § 7). Under refusal suppression, truncation raises ASR by 21.4, 5.8, and 10.3 pp on Qwen1.5, DeepSeek-V2-Lite, and OLMoE, as on the evaluation prompts. SafeStart lowers it on each model, to +0.6, +2.6, and +0.8 pp, and it is non-inferior to the original model at a +3 pp margin on Qwen1.5 and OLMoE but not on DeepSeek-V2-Lite, where it removes 56% of the increase. Pooled over the three models, in an analysis we added afterwards, its residual exceeds that of a rerun of the original model by 0.3 pp (95% CI [−0.7, 1.3]). With half of the experts, truncation raises ASR on these requests by 13.2 to 36.9 pp, and SafeStart leaves a rise of at most 3.9 pp.

SafeStart Keeps Most of the Speedup

A small measured latency cost. In our vLLM uniform-truncation measurements, SafeStart keeps time to first token within 1% of the original model at batch size 8. It uses the original expert budget within the normal prefill pass, with no second model or additional prefill pass. The expert weights remain in GPU memory and can be reused across prompt tokens. This reuse does not remove the computation needed to process those tokens; the time-to-first-token measurements establish the small latency cost.

During decode, only the opening uses the full expert budget (the first few generated tokens, N=8N = 8 in this paper, including the first token produced by prefill). All later tokens use the reduced budget. The fixed opening work is spread over the response, so its share decreases as responses grow longer. In these vLLM uniform-truncation measurements, SafeStart retains at least 85% of truncation’s measured offline decode speedup at batch sizes 8 and 32.

We report decode-time gains separately from time to first token, which includes prefill. These serving measurements do not measure ACE or LExI performance.

Offline. With N=8N = 8, 95.6% of the generated tokens on Qwen1.5 still run with the reduced budget. With no expert removed, the patch adds at most 1.9% to decode time at batch size 1 and at most 1.0% at batch size 8. With experts removed, SafeStart keeps 85 to 101% of truncation’s decode speedup at batch size 8 and 88 to 104% at batch size 32 (values above 100% are within timing noise; Figure 2). With about half of the experts, it decodes 38 to 44% faster than the original model at batch size 8, faster than truncation with two thirds at batch sizes 8 to 128 (Figure 1; Appendix § 8).

More accurate than truncation. SafeStart recovers accuracy as well as refusal. At k′=⌈2k/3⌉k' = \lceil2k/3 \rceil, it keeps GSM8K within 1.4 points of the original model and scores 1.6 and 3.4 points higher than truncation on Qwen1.5 and OLMoE. With about half of the experts, it scores 3.6 to 3.9 points higher than truncation on all three models, roughly halving truncation’s GSM8K loss, and on MATH-500, whose answers need long chains of reasoning, it recovers most of truncation’s accuracy loss on Qwen1.5 and DeepSeek-V2-Lite (Appendix § 8).

Online. Under Poisson arrivals of GSM8K requests, the patch with no expert removed adds at most 2.4% to the median time per output token. On Qwen1.5, SafeStart keeps 60 to 65% of truncation’s gain in median time per output token at 1 to 4 requests per second. Part of the gap comes from truncation’s shorter GSM8K answers, which empty the batch faster: when every request generates 256 tokens, SafeStart keeps 72 to 83% (Appendix § 8).

Large batches. Since truncation saves at most 3.9% of decode time at batch size 128 on the models of Figure 2 (§ 4.1), the patch accepts an optional bound BB and runs all experts in any step with more than BB tokens. The bound can only raise a token’s budget. With B=32B = 32 on Qwen1.5, SafeStart keeps its speedup at batch sizes 1 to 32 and decodes within 0.3% of the original model at batch sizes 128 and 256.

Discussion and Conclusion

Test refusal, not only accuracy. We show that a one-line configuration change at an operating point recommended for accuracy raises harmful compliance by up to 18.4 pp while MMLU moves by at most 1.8 points, and the effect differs by model, template, and reduction method. Serving systems should therefore test refusal for each model and each method before they enable it, at the expert budget that each token actually receives. One test of 1,416 requests takes minutes of generation and judging on one A6000.

Protect the start of each response. The computation of the prompt and the first generated tokens decides most of the change. Running all experts there, as SafeStart does, restores refusal while keeping most of the speedup. Expert-reduction methods should report refusal separately for prefill and decoding.

Reduce experts only where it pays. Truncation saves expert weight reads, so it helps at small batch sizes and little at large ones, where it adds a refusal cost for almost no speedup. A serving system can turn it off above a batch-size bound, as our patch does.

Limitations. Human labels cover 587 responses from two models, and the other results rely on two automated judges, of which the primary one counts many partial answers as harmful. Templates that dictate the opening or move the harmful task later can bypass SafeStart, and we do not test adaptive attacks. On a held-out model, DeepSeek-MoE-16B, the first token does not carry the increase (Appendix § 12). Our models have at most 57B parameters, timings use RTX A6000 GPUs with vLLM’s default MoE kernel settings, and the engine patch is single-turn and does not cover quantized MoE kernels or expert parallelism; multi-turn serving must add the expert budget to the prefix-cache key. We fixed the confirmatory family and the equivalence margin after the first two models had run (Appendix § 7).

Ethics and release. We will publicly release the code, prompt identifiers, per-response labels and scores, and response hashes. We share raw model outputs only on request for research use, because many contain harmful instructions. The annotators reviewed the material knowingly and could skip items.

Conclusion. We show that running fewer experts per token speeds up MoE decoding at small batch sizes, but it can weaken refusal on some models while accuracy benchmarks stay flat, and that most of the change arises in the prompt and the first generated tokens. SafeStart runs all experts only there and keeps the rest of the response fast: under refusal suppression, it holds the change in harmful compliance near zero for three reduction methods on the three affected models, keeps at least 85% of the offline decode speedup at batch sizes 8 and 32, and, with half of the experts, decodes 38 to 44% faster than the original model at batch size 8. We will release the harness and the engine patch publicly.

Reproducibility Checklist

Software, Hardware, Models, and Data

Software. Python 3.12.13, vLLM 0.11.0 (V1 engine), PyTorch 2.8.0 with CUDA 12.8, Transformers 4.57.1, Triton 3.4.0, SciPy 1.17.1, NumPy 2.2.6, Datasets 5.0.0; NVIDIA driver 550.90.07.

Hardware and engine settings. One NVIDIA RTX A6000 (48 GB), 8 CPU cores, and 96 GB of host memory per job. BF16 activations; maximum model length 4,096 (2,048 for the HarmBench classifier); GPU memory utilization 0.90 for generation and 0.85 for judges; chunked prefill with at most 8,192 batched tokens; prefix caching on for safety runs and off for throughput runs; torch.compile with CUDA graphs (piecewise graphs on DeepSeek-V2-Lite, whose Triton MLA attention has no full-graph support). vLLM reports that no tuned fused-MoE configuration exists for the RTX A6000 and these expert shapes, so it uses default kernel settings. FP8 checkpoints run through vLLM’s weight-only FP8 path on this GPU.

Model revisions (Hugging Face snapshot). Qwen1.5-MoE-A2.7B-Chat ec052fda; DeepSeek-V2-Lite-Chat 85864749; Moonlight-16B-A3B-Instruct 4e735b07; Ling-lite-1.5-2507 6656efdc; OLMoE-1B-7B-0125-Instruct b89a7c4b; Qwen3-30B-A3B-Instruct-2507-FP8 5a5a7763; gpt-oss-20b 6cee5e81; deepseek-moe-16b-chat eefd8ac7; Qwen2-57B-A14B-Instruct-GPTQ-Int4 28996f1e; Llama-Guard-3-8B 7327bd9f; HarmBench-Llama-2-13b-cls bda70534. Qwen1.5’s shared expert is as wide as four routed experts and Qwen2-57B’s as wide as eight. Qwen3 runs with FP8 weights in the main evaluation and BF16 in the earlier harness of Appendix D; OLMoE uses the 0125 release in the main evaluation and 0924 in the earlier harness; Qwen2-57B runs with 4-bit GPTQ weights.

Data (MD5 prefix). HarmBench text test behaviors 21dea745 (standard functional category, 159 rows); StrongREJECT dataset 9c896f98 (313 rows); StrongREJECT jailbreak templates 06e071f6; HarmBench classifier prompt 5ee8a7ec; XSTest prompts b4928a5e (450 rows). Prompt identifiers are hb_<BehaviorID> and sr_<row>.

Prompting and decoding. One user turn rendered with each model’s chat template, which for some models inserts a default system message (Qwen1.5-MoE and Moonlight) or a reasoning-level and date line (GPT-OSS, reasoning effort low). Greedy decoding (temperature 0), repetition penalty 1.0, no stop strings beyond the model’s end-of-turn tokens, at most 512 new tokens (1,536 for GPT-OSS). The share of responses that reach the 512-token limit is 5.6%, 9.7%, and 10.2% for Qwen1.5 at k=4k = 4, 3, and 2; 13.9% to 22.7% for DeepSeek-V2-Lite (rising with truncation); 18.3% to 20.0% for Moonlight; and 32.3% to 38.0% for Ling-lite, pooled over the three framings.

Conditions and the SafeStart Patch

Truncation and routing variants. Truncation sets num_experts_per_tok through hf_overrides; each run checks the value in the loaded configuration. Four controls separate the removed experts from the lost gate mass: mass restored keeps the top k′k' experts and rescales their weights to sum to the original routed mass mk(x)m_k(x); attention only keeps all kk experts and multiplies their weights by mk′(x)/mk(x)m_{k'}(x)/m_k(x); swap keeps kk experts but replaces the kk-th ranked expert with the (k+1)(k+1)-th at the kk-th expert’s weight; and trim keeps the top k′k' experts of a renormalized model without renormalizing them again. These conditions wrap FusedMoE.select_experts: the wrapper requests the original kk (or k+1k+1 for swap) experts from the original routine, sorts them by weight, and returns the modified weights and expert identifiers. The engine process prints one line per run that names the route and the layer’s kk, which we check in every log. The patch is applied before the engine starts and is inherited by the engine process.

SafeStart in the engine. dt_engine.py patches vLLM at run time without changing its source. An import hook patches the GPU model runner inside the engine process, so the parent process never initializes CUDA and the engine keeps its fork start method. After the runner computes positions, the patch marks every scheduled token at position ≥P−1+N\ge P - 1 + N for prompt length PP, writes the mask to a pinned buffer, and copies it into a persistent GPU buffer that captured CUDA graphs read. Rows that pad a step to a captured graph size also run the router, so the mask gives them the reduced budget whenever a real token of the step has it, as native truncation does. An earlier version gave them all experts; this leaves the captured batch sizes of the offline timings (1, 8, 32, 128, and 256) unchanged, but at batch sizes that vLLM pads (3, 5, 6, and 12 on Qwen1.5) SafeStart with N=8N = 8 kept 54 to 72% of truncation’s speedup instead of 91 to 95%. The online runs of Table A5–Table A7 predate the fix; Table A8 gives the runs after it, with natural lengths, with every request generating 256 tokens, and in a closed loop, each with no other job of ours on the node. On Qwen1.5, SafeStart keeps 64, 65, 60, and 38% of truncation’s gain at 1, 2, 4, and 8 requests per second with natural lengths, against 59, 55, 52, and 37% before the fix. On DeepSeek-V2-Lite, and on OLMoE in the open-loop runs, some engine runs ran slower as a whole, which also raises their time to first token: within each condition, rate, and setting, a run’s change in time to first token tracks its change in time per output token (r=0.79r = 0.79 over 264 runs), so the shares there are not stable. In the runs before the fix, truncation’s median time to first token is below the original model’s in 35 of 36 runs, as it should be. In those runs SafeStart keeps 75, 56, and 66% of truncation’s gain in time per output token on DeepSeek-V2-Lite and 89, 67, and 51% on OLMoE at 1, 2, and 4 requests per second, and the fix only removes expert reads from SafeStart, so it can only raise these shares. After the fix, in the closed loop on OLMoE, SafeStart keeps 59, 76, and 71% at 1, 4, and 16 clients. The expert-selection wrapper takes the original top-kk, sorts by weight, and for marked tokens sets the k−k′k-k' lowest slots to a sentinel identifier EE with weight zero (renormalizing the kept weights when the model renormalizes). The fused-experts wrapper passes expert_map = [0, \ldots, E-1, -1] and E+1E+1 global experts, so moe_align_block_size maps sentinel blocks to −1-1 and the grouped GEMM kernel skips their weight loads and writes zeros. On Qwen1.5 with N=106N = 10^{6} (never truncate), 282 of 300 responses share their first 40 characters with an unpatched run of the original model, comparable to two unpatched runs of the original model (1,304 of 1,416). With every token, prompt included, truncated through the patch (N=−4096N = -4096), DeepSeek-V2-Lite and OLMoE match native truncation within run-to-run noise (229 and 280 of 300 responses share their first 40 characters, against 237 and 270 between two native runs). The Qwen1.5 check ran with N=0N = 0, which truncates the final prompt token and every generated token but computes the earlier prompt tokens with all experts; it matches native truncation on 224 of 300 (274 between two native runs), and under refusal suppression 22 of its 100 responses open with a refusal phrase, against 15 for native truncation and 30 for the original model, so on Qwen1.5 full computation of the earlier prompt tokens also protects some refusals.

ModelCondition1/s2/s4/s8/s1/s2/s4/s8/s
Qwen1.5Original+0.0+0.0+0.0+0.068808596
Truncation k′=3k^\prime= 3+30.9+19.6+9.0+6.6657386100
SafeStart N=8N = 8, k′=3k^\prime= 3+14.8+10.2+2.0+4.068778899
SafeStart N=32N = 32, k′=3k^\prime= 3+12.7+9.1+2.6+3.8687687100
SafeStart N=8N = 8, k′=2k^\prime= 2+34.8+30.9+20.5+8.266738499
DeepSeek-V2-LiteOriginal+0.0+0.0+0.0+0.0768499112
Truncation k′=4k^\prime= 4+39.0+21.0+10.2+4.7708096110
SafeStart N=8N = 8, k′=4k^\prime= 4+15.0+9.6+5.7+14.5768698107
SafeStart N=32N = 32, k′=4k^\prime= 4+18.8+5.6+0.0−1.27487104115
SafeStart N=8N = 8, k′=3k^\prime= 3+24.6+17.8+9.8+14.97683100110
OLMoEOriginal+0.0+0.0+0.0+0.031354245
Truncation k′=6k^\prime= 6+23.7+20.8+16.3+6.330323844
SafeStart N=8N = 8, k′=6k^\prime= 6+14.1+10.7+6.4+3.531344044
SafeStart N=32N = 32, k′=6k^\prime= 6+9.0+11.5+7.0+3.131354044
SafeStart N=8N = 8, k′=4k^\prime= 4+37.7+42.2+32.5+16.331333744

Table A5. Online serving with Poisson arrivals (200 GSM8K requests per rate, greedy, at most 512 tokens, one RTX A6000). Change in median time per output token against the original model (%, positive is faster) and median time to first token (ms). SafeStart rows use the PyTorch version of the patch; Table A6 gives the fused kernel.

ModelProtected1/s2/s4/s8/s
Qwen1.5Prompt (N=0N = 0)92679147
Prompt + 8 (N=8N = 8)58654952
DeepSeek-V2-LitePrompt (N=0N = 0)786073122
Prompt + 8 (N=8N = 8)754969140
OLMoEPrompt (N=0N = 0)79726261
Prompt + 8 (N=8N = 8)87786273

Table A7. Online-serving ablation (same setup as Table A6, two arrival seeds, all conditions in one job): share of truncation’s gain in median time per output token kept with only the prompt protected (N=0N = 0, every generated token truncated) and with SafeStart (N=8N = 8), in %.

Table A8. Online serving after the CUDA-graph padding fix (GSM8K requests, greedy, one RTX A6000). Truncation: change in median time per output token against the original model under the same arrivals (%, positive is faster). SafeStart and prompt only: share of that gain kept (%) by SafeStart (N=8N = 8) and by protecting only the prompt (N=0N = 0). Natural: Poisson arrivals at 1, 2, 4, and 8 requests per second, natural output lengths, at most 512 tokens, three arrival seeds. Fixed: the same arrivals with every request generating exactly 256 tokens, two seeds. Closed loop: 1, 4, and 16 clients, each sending its next request when the previous one finishes, 256 tokens, 64 requests per level, one run. Shares are ratios of means over seeds; values above 100% or below 0 occur where whole engine runs ran slower, which also raises their time to first token. Dashes: not run.

ModelConditionNatural (3) 1/s2/s4/s8/sFixed (2) 1/s2/s4/s8/sClosed loop (1) 1416
Qwen1.5Truncation+36.3+24.5+11.5+4.6+29.8+18.5+8.9+2.4+4.9+14.5+15.7
SafeStart64656038818372151219072
Prompt only8471844594867443–––
DeepSeek-V2-LiteTruncation+21.7+13.2+3.8−7.2+44.1+18.8+6.7+3.2–––
SafeStart172157233−37998814−24–––
Prompt only86161345−5169817491–––
OLMoETruncation+26.7+26.3+14.5+6.2+30.6+25.2+12.7+8.0+15.1+14.0+16.0
SafeStart96106108162155140141150597671
Prompt only818984145109121141144–––

Fused budget kernel. With DT_FUSED=1, a Triton kernel replaces the sort, gather, and mask: each program loads the kk weights and identifiers of 32 tokens, ranks every slot by counting the slots ahead of it (ties broken by slot index), applies the top-k′k', top-pp, or threshold rule to marked tokens, and writes sentinel identifiers and zero weights in place. It ranks slots by routing weight, which on the four models we serve with it is also the router’s selection order (none has a selection bias); a router that selects by a bias-corrected score, such as Moonlight’s, would need that score passed to the kernel. On random routing weights with shuffled slot order, 70% of tokens marked, k∈{4,6,8}k \in\{4, 6, 8\}, all three rules, and 1 to 257 tokens per call, it selects the same experts and weights as the PyTorch path in every case. End to end with N=8N = 8 it agrees with the PyTorch path on the first 40 characters of 281, 294, and 291 of 300 responses on Qwen1.5, DeepSeek-V2-Lite, and OLMoE. With every token truncated it reproduces native truncation exactly on Qwen1.5 (300 of 300 identical) and shares the first 40 characters on 240 and 288 of 300 on DeepSeek-V2-Lite and OLMoE, against 237 and 270 between two native runs; on Ling-lite, which renormalizes the kept weights, it shares them on 237 of 300. The remaining differences come from the final sum over each token’s slots, not from expert selection: native truncation sums k′k' slots and the patch sums kk, of which k−k′k-k' are exact zeros, and the PyTorch path also reorders the slots by weight.

SafeStart and early truncation (emulation)

A first pass generates 64 tokens per request with the original model (SafeStart) or the truncated model (early) and stores the prompt and output token identifiers. A second pass loads the other model and continues from the prompt plus the first NN output tokens with at most 512−N512-N new tokens. Requests that ended within the first NN tokens keep their first-pass output. The final text is the tokenizer decoding of the concatenated token identifiers.

Judging and Statistics

Judges. Llama Guard 3 receives the original request (not the template) as the user turn and the final answer truncated to 1,024 tokens as the assistant turn; a response is unsafe if the output starts with “unsafe”. The HarmBench classifier receives its official prompt with the request as the behavior and the answer truncated to 512 tokens; a response is harmful if the output is “yes”. The HarmBench classifier requires a non-minimal instance of the requested behavior, and StrongREJECT prompts lie outside its training format, so we use it only as a second opinion. Outputs that match neither pattern count as safe; one of 213,816 Llama Guard 3 outputs (“m” followed by “unsafe”, Qwen3-30B at $k' = 2) failed to match, and no HarmBench output did. For GPT-OSS the judges see the text after the final-channel marker, or the whole output when the marker is missing. GPT-OSS produced no final answer for 3, 1, and 162 of 1,416 responses at k=4k = 4, 3, and 2, and for 6 to 9 in the four reordered reruns.

Screens. Degenerate: more than half of the whitespace-token trigrams of the response repeat. Refusal keywords (used only descriptively): a regular expression over the first 300 characters.

Statistics. Exact two-sided McNemar test on discordant pairs; Tango score intervals for paired differences ([52]); percentile bootstrap over prompts with seed 0, using 10,000 replicates for the appendix tables and Figure A7 and 4,000 for Figure A4, the between-condition contrasts, and the factorial; Holm correction over the confirmatory family (k′=⌈2k/3⌉×3k' = \lceil2k/3 \rceil\times3 framings ×\times models, Llama Guard 3); equivalence by two one-sided tests with a ±3\pm3pp margin, using the 90% Tango interval. The noise-floor reruns shuffle the prompt order with seeds 1 to 4; a condition shows excess label churn if it changes more labels than the first rerun, tested with a McNemar test on the indicators of a label change. We fixed the confirmatory family and margin after the first two models had run; restricted to the five models that ran afterwards (15 tests), the Holm-significant changes are OLMoE’s increase under refusal suppression, Ling-lite’s increase under prefix injection, and decreases on Moonlight16B and Qwen3-30B.

Where the change happens under a refusal-suppression attack

Figure A4. Where the change happens, under a refusal-suppression attack (472 prompts, Llama Guard 3). Left: share of response pairs whose truncated and original responses have diverged by each generated token; index 0 is the first generated token, produced by the prefill pass. Forward flips (original safe, truncated harmful) diverge at the first token, earlier than pairs with the same label and than two runs of the original model with different prompt order. Right: change in harmful-compliance rate (ASR) against the original model with 95% prompt-bootstrap intervals, at the operating point that keeps about two thirds of the selected experts. Early: the first generated token comes from the truncated model and the rest from the original model. SafeStart (emulation): the first NN generated tokens come from the original model and the rest from the truncated model. SafeStart (engine): our vLLM patch. Without renormalization: Ling-lite’s truncation with the kept experts’ original weights. Gray: 95% interval of a reordered rerun of the original model. Filled markers: exact McNemar p<0.05p < 0.05.

Figure A7. ASR change against the original model (Llama Guard 3, 95% prompt-bootstrap intervals) for truncation, the mass conditions, and SafeStart (emulation). Gray band: 95% interval of a reordered rerun of the original model.

Second held-out set, analysis fixed in advance. After all settings were frozen we assembled 889 requests, 383 from AdvBench [72] and 506 from the English CatQA [3], keeping only those with no near-duplicate (difflib ratio at least 0.8) in the evaluation set, the first held-out set, or the set itself. We fixed the following analysis on 28 October 2026, before generating any response on this set (scripts/heldout2_table.py, SHA-256 58ce003e...5a7b799). Primary, under refusal suppression and Llama Guard 3, on each of the three affected models at k′=⌈2k/3⌉k' = \lceil2k/3 \rceil: (P1) SafeStart with N=8N = 8 is non-inferior to the original model if the upper bound of the 90% Tango interval of the paired change is below +3 pp (one-sided α=0.05\alpha= 0.05); (P2) truncation is tested against the original model by exact McNemar with Holm over the three models; (P3) SafeStart is tested against truncation by exact McNemar. Secondary, all reported: prefix injection, the HarmBench classifier, N=1N = 1 and N=32N = 32, truncation and N=8N = 8 at about half the experts, and a rerun of the original model with the requests in a different order. 188 of the AdvBench requests were among the 256 that estimate the refusal direction of § 4.3, which no setting of the policy uses.

Accuracy, Throughput, Compute, and Release

Accuracy and throughput. MMLU (all subjects, test split, zero-shot; the answer is the highest-probability letter among the top-20 first-token log-probabilities), GSM8K (test split, zero-shot, 512 tokens, last number), XSTest (450 prompts, 256 tokens, refusal by prefix match), and MATH-500 (the 500-problem test subset, zero-shot chain of thought asked to box the answer, greedy, 1,024 tokens, scored with math-verify; scripts/eval_math.py). Throughput: GSM8K questions as prompts, 256 output tokens with ignore_eos, batch sizes 1, 8, 32, 128, and 256, one warm-up and three timed repetitions (median reported), prefix caching disabled; first-token time measured with one output token.

Compute. A Study 2 generation job (original, three truncation levels, and the rerun, 1,416 requests each) takes 12 to 24 minutes and its judging 17 to 24 minutes; the Qwen1.5 mitigation job (eight conditions and judging) took 56 minutes. Study 2 ran as 52 one-GPU jobs totaling about 83 RTX A6000 GPU-hours, measured from the job logs because Slurm accounting is disabled on our cluster.

Release and disclosure. We will publicly release the harness, the engine patch, prompt identifiers, per-response labels, judge outputs, and response hashes. Raw model outputs, the human-review pages, and annotator notes contain harmful instructions; we share them only on request for research use under a data-use agreement, and do not post them publicly. We will inform the vLLM and llama.cpp maintainers and the authors of the operating-point study of the result and of SafeStart. The marginal uplift to an attacker is small: the attacks we use are public, and the regression requires the operator, not the attacker, to change the serving configuration.

Full Results and Robustness Checks

Speed, Expert Weight Reads, and Online Serving

Timing and serving. Table A2 compares the fused budget kernel with the PyTorch version of the patch in the same interleaved jobs; Figure 1 of the paper uses the kernel. Table A1 gives the earlier interleaved offline timings with the PyTorch version; Figure 2 of the paper takes the original model, truncation, and SafeStart from the job of Table A2. Table A5 gives the online-serving runs with the PyTorch version, Table A6 the same runs with the fused kernel over three arrival seeds, with every condition under the same arrivals, and Table A7 an ablation with N=0N = 0, which protects every prompt token but the last. An earlier set of engine timings taken in separate jobs from the runs of the original model disagreed with these at small batch sizes (for example +51.6% for SafeStart on DeepSeek-V2-Lite at batch size 1), so every engine speedup we report is against the original model timed in the same job; the half-budget rows of Table A2 come from a second job with its own runs of the original model and truncation. Truncation at k′=⌈2k/3⌉k' = \lceil2k/3 \rceil varies by at most 0.5 points across our offline timing jobs (Table A1, Table A2, Table A24, and Table A23) at batch sizes 8 to 128, far less than the 1.6 to 26.7 points by which SafeStart at half the experts is faster than it. With the fused kernel, truncating every generated token through the patch (N=0N = 0) keeps 89, 98, and 99% of native truncation’s batch-8 speedup on Qwen1.5, DeepSeek-V2-Lite, and OLMoE (14.2, 26.8, and 16.1 against 16.0, 27.3, and 16.2%, same job; Table A24). N=0N = 0 and N=1N = 1 incur identical decode steps and differ only in prefill, which first-token time subtracts, so their difference (1.2, 0.8, and 0.4 points) is this job’s timing noise; N=8N = 8 falls below N=1N = 1 by 0.6, 2.4, and 0.7 points, beyond the job’s such difference only on DeepSeek-V2-Lite. The batch-size bound of § 5.3 ran on Qwen1.5 in one more interleaved job of two rounds: with B=32B = 32, SafeStart speeds up decoding by 4.4, 14.2, 7.1, −0.3, and +0.2% at batch sizes 1, 8, 32, 128, and 256, against −4.2, 14.7, 7.2, 0.2, and 0.8% without the bound and 7.4, 16.0, 8.3, 0.9, and 1.4% for truncation (scripts/dt_engine.py, DT_MAXTOK). With the PyTorch version and no token truncated, the patch costs 1.7 to 2.5% at batch size 8, and truncating every generated token through it (N=0N = 0) gives 12.4 to 24.2% where native truncation gives 15.8 to 27.5%. Under online serving, the conditions change mean response length by at most 10% (Qwen1.5, truncation), so we compare time per output token rather than throughput. Online gains exceed the offline ones at a fixed batch size because a faster step also shortens each request’s stay and so the running batch: at 1 request per second the mean number of running requests (arrival rate times time in the system) falls from 4.0 to 2.9 on Qwen1.5 and from 4.4 to 3.2 on DeepSeek-V2-Lite under truncation, while decode-step time grows with batch size (Qwen1.5: 9.1 ms at batch size 1, 22.6 ms at 8). Truncation also shortens GSM8K responses (on Qwen1.5 at 1 request per second, 189 tokens on average against 206 for the original model and 204 for SafeStart), which lowers the running batch further. With every request generating exactly 256 tokens (ignore eos, two arrival seeds, the padding fix in place), truncation’s gain at 1, 2, 4, and 8 requests per second is 29.8, 18.5, 8.9, and 2.4%, and SafeStart keeps 81, 83, 72, and 15% of it; at 8 requests per second the offered 2,048 tokens per second exceed what the GPU completes (about 1,230 in every condition), so that column measures a saturated queue (protecting only the prompt, N=0N = 0: 94, 86, 74, and 43%).

ModelCondition1832128256
Qwen1.5Truncation k′=3k^\prime= 3+7.3+15.8+8.1+0.7+0.6
Qwen1.5SafeStart N=8N = 8, k′=3k^\prime= 3−0.1+11.7+6.6−0.4−0.3
Qwen1.5SafeStart N=32N = 32, k′=3k^\prime= 3−1.5+9.0+6.2−1.1−5.3
Qwen1.5Patch, N=0N = 0, k′=3k^\prime= 3−0.1+12.6+6.0−0.6−0.3
Qwen1.5Patch, no token truncated−5.7−2.4−2.0−1.5−5.8
Qwen1.5Truncation k′=2k^\prime= 2+15.5+38.2+25.5+3.3−3.5
Qwen1.5SafeStart N=8N = 8, k′=2k^\prime= 2+6.0+31.6+15.2−3.2−3.6
DeepSeek-V2-LiteTruncation k′=4k^\prime= 4+1.6+27.5+11.2+2.2+2.5
DeepSeek-V2-LiteSafeStart N=8N = 8, k′=4k^\prime= 4−0.3+21.9+10.2+1.3+0.6
DeepSeek-V2-LiteSafeStart N=32N = 32, k′=4k^\prime= 4−0.2+19.3+8.5+0.8+0.2
DeepSeek-V2-LitePatch, N=0N = 0, k′=4k^\prime= 4+0.2+24.2+9.4+1.0+0.6
DeepSeek-V2-LitePatch, no token truncated+0.3−1.7−1.2−0.7−0.9
DeepSeek-V2-LiteTruncation k′=3k^\prime= 3+1.1+49.0+23.6+5.4+3.9
DeepSeek-V2-LiteSafeStart N=8N = 8, k′=3k^\prime= 3−1.6+40.5+22.1+4.3+2.0
OLMoETruncation k′=6k^\prime= 6+13.7+16.1+11.4+4.1+3.0
OLMoESafeStart N=8N = 8, k′=6k^\prime= 6+4.7+11.3+8.1+2.4+0.8
OLMoESafeStart N=32N = 32, k′=6k^\prime= 6+3.7+10.1+7.0+1.8+0.6
OLMoEPatch, N=0N = 0, k′=6k^\prime= 6+5.6+12.4+9.0+2.8+1.3
OLMoEPatch, no token truncated−5.6−2.5−1.5−1.5−1.3
OLMoETruncation k′=4k^\prime= 4+33.3+44.2+28.7+10.8+8.3
OLMoESafeStart N=8N = 8, k′=4k^\prime= 4+19.9+37.0+24.7+9.5+5.5

Table A1. Decode speedup over the original model (%, 256 output tokens), with all conditions of a model timed in one job, interleaved over rounds (three for Qwen1.5, two for the others; three timings per round). Medians over timings.

ModelCondition1832128TTFT
Qwen1.5Truncation k′=3k^\prime= 3+7.4+15.8+8.2+0.9−3.2
Qwen1.5Patch, nothing truncated (PyTorch)−6.0−2.6−1.6−1.0+1.3
Qwen1.5Patch, nothing truncated (kernel)−1.9−1.0−0.4−0.1+0.4
Qwen1.5SafeStart N=8N = 8 (PyTorch)+0.1+12.9+7.7+0.5−0.0
Qwen1.5SafeStart N=8N = 8 (kernel)+5.7+15.9+8.3+1.2−0.6
Qwen1.5Truncation k′=2k^\prime= 2, separate job+15.9+39.1+25.9+3.8−10.4
Qwen1.5SafeStart N=8N = 8 (kernel), k′=2k^\prime= 2, separate job+12.0+38.3+25.2+2.5+0.2
DeepSeek-V2-LiteTruncation k′=4k^\prime= 4+3.0+27.4+11.4+2.2−5.7
DeepSeek-V2-LitePatch, nothing truncated (PyTorch)+0.5−1.8−1.1−0.6+0.7
DeepSeek-V2-LitePatch, nothing truncated (kernel)+1.8−0.3−0.1−0.0+0.0
DeepSeek-V2-LiteSafeStart N=8N = 8 (PyTorch)+1.3+21.8+10.3+1.3+0.6
DeepSeek-V2-LiteSafeStart N=8N = 8 (kernel)+2.3+23.4+11.9+1.8−0.1
DeepSeek-V2-LiteTruncation k′=3k^\prime= 3, separate job+1.6+49.1+23.8+5.6−10.7
DeepSeek-V2-LiteSafeStart N=8N = 8 (kernel), k′=3k^\prime= 3, separate job−1.5+44.3+24.9+4.9−0.1
OLMoETruncation k′=6k^\prime= 6+14.4+16.0+11.1+4.0−5.3
OLMoEPatch, nothing truncated (PyTorch)−5.7−2.7−1.8−1.5+1.4
OLMoEPatch, nothing truncated (kernel)−1.8−1.0−0.5−0.7+0.8
OLMoESafeStart N=8N = 8 (PyTorch)+5.7+10.9+7.8+2.2+1.5
OLMoESafeStart N=8N = 8 (kernel)+11.9+15.0+9.8+3.2+0.7
OLMoETruncation k′=4k^\prime= 4, separate job+33.8+44.6+28.6+10.9−13.6
OLMoESafeStart N=8N = 8 (kernel), k′=4k^\prime= 4, separate job+28.2+42.7+27.0+10.8−0.5

Table A2. Decode speedup over the original model (%, 256 output tokens) with the fused budget kernel and with the PyTorch version of the patch, all conditions of a model timed in one job and interleaved over two rounds. Medians over timings. TTFT: change in time to first token at batch size 8 (%, negative is shorter).

ModelCondition1/s2/s4/s8/s1/s2/s4/s8/s
Qwen1.5Truncation k′=3k' = 3+36.7 [+30.0, +44.2]+21.8 [+17.9, +24.5]+10.4 [+9.5, +11.7]+4.1 [+2.0, +5.9]64728398
Patch, nothing truncated−1.0 [−2.0, +0.5]−0.8 [−1.1, −0.6]−1.2 [−1.3, −1.1]−0.6 [−1.6, −0.1]697988101
SafeStart N=8N = 8+21.6 [+16.1, +26.2]+12.0 [+10.9, +13.2]+5.5 [+3.1, +8.5]+1.5 [+0.8, +2.0]677685100
DeepSeek-V2-LiteTruncation k′=4k' = 4+48.3 [+45.5, +50.3]+27.0 [+26.0, +29.0]+14.6 [+12.9, +15.5]+8.2 [+2.2, +17.0]677691104
Patch, nothing truncated−0.6 [−3.0, +1.9]+0.3 [+0.0, +0.8]+0.0 [−0.4, +0.4]+0.1 [−1.0, +0.9]758698110
SafeStart N=8N = 8+36.3 [+35.4, +37.6]+15.1 [+13.0, +16.8]+9.6 [+9.2, +10.2]+6.9 [+2.2, +15.3]738295109
OLMoETruncation k′=6k' = 6+21.4 [+18.9, +25.0]+25.9 [+23.5, +30.6]+14.0 [+12.8, +15.5]+7.7 [+7.1, +8.3]29323744
Patch, nothing truncated−2.4 [−2.7, −2.2]+0.5 [−0.7, +2.1]−0.5 [−1.0, +0.5]+0.7 [+0.4, +0.9]31354046
SafeStart N=8N = 8+19.1 [+15.8, +23.2]+17.3 [+15.6, +18.5]+7.2 [+5.3, +9.0]+5.9 [+4.8, +6.9]31334045

Table A6. Online serving with the fused budget kernel: Poisson arrivals, 200 GSM8K requests per rate, greedy, at most 512 tokens, one RTX A6000, three arrival seeds with all conditions run under the same arrivals. Change in median time per output token against the original model (%, positive is faster; mean with minimum and maximum over seeds) and median time to first token (ms, mean over seeds). TPOT change (%) and TTFT p50 (ms) are grouped by rates 1/s, 2/s, 4/s, and 8/s.

ModelCondition1832128
Qwen1.5Truncation+7.2+15.9+8.1+0.7
Threshold+5.0+15.5+8.6+0.5
Threshold + SafeStart+4.8+13.6+7.8+0.3
Top-pp+4.8+14.3+8.1+0.5
Top-pp + SafeStart+5.4+15.0+8.5+0.4
DeepSeek-V2-LiteTruncation−0.8+27.4+11.3+2.1
Threshold+0.2+21.3+9.2+1.6
Threshold + SafeStart−0.4+21.2+9.6+1.8
Truncation+0.1+27.1+11.3+2.3
Top-pp+0.3+15.7+8.5+1.6
Top-pp + SafeStart+0.0+19.2+9.2+1.7
OLMoETruncation+13.7+16.2+11.5+4.1
Threshold+9.7+14.0+8.0+2.3
Threshold + SafeStart+9.7+13.4+7.8+2.3
Top-pp+7.8+6.5+4.4+1.3
Top-pp + SafeStart+7.3+9.7+5.5+1.8

Table A23. Decode speedup over the original model (%) for the reduction rules of Table 3 of the paper, alone and with SafeStart (N=8N = 8, engine), against truncation at k′=[2,3]k' = [2, 3] and the original model timed in the same interleaved job (on DeepSeek-V2-Lite one job per rule, each with its own truncation row; two rounds, 256 output tokens, one RTX A6000). Batch size.

NNRS LGRS HBPI LGPI HBSpeedup (%) bs 1Speedup (%) bs 8Speedup (%) bs 32
Qwen1.5
trunc.+18.4+16.5+6.8+7.0+7.6+16.0+8.1
0+9.1+7.0+6.1+7.6+5.1+14.2+7.6
1+0.4−0.8+1.5+2.5+5.1+15.4+7.9
2−1.3−0.8+2.1+1.3+5.1+15.0+8.1
4−1.3−0.4+2.5+3.2+5.1+14.5+7.8
8−0.6+0.4+1.3+3.0+4.9+14.7+7.2
16−2.8−1.1+1.7+1.7+4.8+13.8+7.6
32−1.5−1.1+2.1+0.6+4.1+12.6+6.4
open.+7.0+6.8+3.6+5.1–––
DeepSeek-V2-Lite
trunc.+6.6+5.3−2.3−5.1−1.7+27.3+11.3
0+2.8+3.2+8.3+7.6−1.4+26.8+11.2
1+0.8+2.5+0.2+2.5−1.6+26.0+11.5
2+2.1+2.3+1.1+0.6−1.6+26.3+13.1
4−0.6+0.8−1.1−0.6−0.2+24.7+12.2
8+0.6+2.3+0.2+0.2−0.8+23.6+11.9
16+1.9+1.7−1.3+0.0+0.2+22.6+11.0
32+1.1+1.5+1.7+0.8+0.4+21.1+9.6
open.+2.3+2.8−9.5−9.1–––
OLMoE
trunc.+9.5+8.3+5.3+5.3+13.9+16.2+11.4
0−0.6+0.2+3.4+3.0+12.1+16.1+10.7
1+0.0+2.8+6.1+4.0+11.9+15.7+11.1
2+1.5+2.5+4.2+4.9+11.9+16.2+10.2
4+0.4+0.2+1.9+3.8+11.7+15.4+10.1
8+1.3+1.7+4.4+3.2+11.6+15.0+9.7
16−0.2+0.6+4.9+5.9+11.1+15.6+9.7
32+0.0+1.1−1.1−0.8+10.4+13.3+8.5
open.+8.9+8.9+9.3+7.6–––

Table A24. Protection length NN in the engine with the fused kernel: change in ASR against the original model (pp; refusal suppression RS and prefix injection PI; Llama Guard 3 LG and HarmBench classifier HB; bold: McNemar p<0.05p < 0.05; shading: size and sign of the LG change) and decode speedup over the original model (%, 256 output tokens; all conditions of a model timed in one job, interleaved over two rounds; on DeepSeek-V2-Lite the batch-size-1 values are within timing noise). N=0N = 0 runs every prompt token but the last at full kk; trunc.: truncation of every token; open.: opening only (Figure 4 of the paper), every prompt token but the last at k′k^{\prime} and the last with the first eight generated tokens at full kk (not timed).

Expert weight reads. Table A3 counts, for every MoE layer call in a decode step where all requests of the batch decode together, the distinct routed experts the batch selects (scripts/traffic.py; a wrapper around vLLM’s expert selection, eager mode, so CUDA graphs do not hide calls). Real routing reads fewer experts than uniform routing at every batch size above 1, most on OLMoE, which is why truncation still removes 7.1% of its reads at batch size 128. A decode speedup ss shortens decode time by s/(1+s)s/(1 + s). At batch sizes 8 to 128 the time saved is below the reads removed in every cell, as expected when attention, the dense layers, and per-step overheads do not shrink; at batch size 256 truncation removes at most 5.2% of the reads.

ModelConditionQuantity1832128256
Qwen1.5OriginalExperts read4.0 (4.0)22.1 (25.5)49.1 (53.4)59.1 (60.0)59.8 (60.0)
Truncation k′=3k^\prime= 3Experts read3.0 (3.0)17.8 (20.2)44.2 (48.4)58.5 (59.9)59.7 (60.0)
Reads removed (%)12.516.59.11.00.2
Decode speedup (%)+7.4+15.8+8.2+0.9–
Decode time saved (%)6.913.67.50.9–
ThresholdExperts read2.816.942.057.859.5
Experts per decode token2.792.862.922.922.93
Top-ppExperts read2.819.244.858.059.6
Experts per decode token2.822.912.902.872.87
DeepSeek-V2-LiteOriginalExperts read6.0 (6.0)29.8 (34.9)51.7 (61.3)60.8 (64.0)62.7 (64.0)
Truncation k′=4k^\prime= 4Experts read4.0 (4.0)20.2 (25.8)44.0 (55.9)59.2 (64.0)61.9 (64.0)
Reads removed (%)25.030.314.42.51.2
Decode speedup (%)+3.0+27.4+11.4+2.2–
Decode time saved (%)2.921.510.22.2–
ThresholdExperts read3.620.042.357.961.6
Experts per decode token3.623.673.673.663.66
Top-ppExperts read4.520.845.758.862.0
Experts per decode token4.554.073.974.024.03
Threshold, decode-matchedExperts read4.123.245.159.562.1
Experts per decode token4.134.204.084.104.08
OLMoEOriginalExperts read8.0 (8.0)29.5 (42.0)50.7 (63.1)58.2 (64.0)60.6 (64.0)
Truncation k′=6k^\prime= 6Experts read6.0 (6.0)24.2 (34.9)43.8 (61.3)54.1 (64.0)57.5 (64.0)
Reads removed (%)25.018.113.57.15.2
Decode speedup (%)+14.4+16.0+11.1+4.0–
Decode time saved (%)12.613.810.03.9–
ThresholdExperts read5.022.941.954.057.8
Experts per decode token5.014.834.874.824.83
Top-ppExperts read5.024.142.455.158.8
Experts per decode token4.974.774.844.814.82
Threshold, decode-matchedExperts read6.225.344.955.659.0
Experts per decode token6.246.326.256.236.25
Top-pp, decode-matchedExperts read6.527.947.757.760.5
Experts per decode token6.466.296.336.326.35

Table A3. Expert weight reads per decode step (GSM8K prompts, 64 output tokens, eager mode): distinct routed experts per MoE layer and step for the original model and truncation at k′k^\prime, uniform routing in parentheses, the share of expert weight reads that truncation removes (shared experts counted in routed-expert units), and the interleaved decode speedup of truncation (Table A2, the job of Figure 2 of the paper; not timed at batch size 256) with the decode time it saves, s/(1+s)s/(1+s). Threshold skipping and top-pp routing use the calibrated parameters of Table A22, applied to every token.

Kernel profile. Table A4 profiles 32 decode steps after prefill with the PyTorch profiler in eager mode (scripts/moe_profile.py) and subtracts a prefill-only run with the same prompts. In the original model, GPU time per step is 4 to 32% below the wall-clock time per decode step measured with CUDA graphs. With no token truncated, the fused patch adds at most 2.0% of GPU time per step. At batch size 1, truncation saves 7.0, 12.6, and 13.1% of GPU time per step on Qwen1.5, DeepSeek-V2-Lite, and OLMoE and 6.8, 1.6, and 12.0% of decode time end to end (Table A1). vLLM runs Qwen1.5 and OLMoE with full CUDA graphs for decode, and 97 and 92% of the GPU-time saving appears end to end; the MLA attention backend of DeepSeek-V2-Lite supports only piecewise CUDA graphs, and 13% of its saving appears end to end at batch size 1, against 93% at batch size 8. The routed reads removed times the routed-expert share of the original model’s step predict the change in GPU time per step within 2.4 points in all 12 cells: 8.2, 12.4, 7.7, and 0.8% at batch sizes 1, 8, 32, and 128 against 7.0, 10.3, 7.5, and 1.8% measured on Qwen1.5, 13.7, 22.9, 12.0, and 2.1 against 12.6, 23.3, 13.6, and 4.4% on DeepSeek-V2-Lite, and 14.6, 13.7, 11.4, and 5.8 against 13.1, 12.5, 13.7, and 8.1% on OLMoE.

ModelConditionQuantity1832128
Qwen1.5OriginalGPU ms per step8.7218.6034.8962.08
Routed-expert share (%)33637882
Truncation k′=3k^\prime= 3Routed-expert time (%)−21.1−16.1−9.4−2.2
Routed expert reads (%)−25.0−19.5−9.9−1.0
GPU time per step (%)−7.0−10.3−7.5−1.8
Patch onlyGPU time per step (%)+2.0+0.8+0.4+0.1
DeepSeek-V2-LiteOriginalGPU ms per step10.3723.8538.1066.89
Routed-expert share (%)41718183
Truncation k′=4k^\prime= 4Routed-expert time (%)−28.8−31.9−16.3−4.5
Routed expert reads (%)−33.3−32.3−14.9−2.6
GPU time per step (%)−12.6−23.3−13.6−4.4
Patch onlyGPU time per step (%)+0.6+0.1+0.2−0.1
OLMoEOriginalGPU ms per step4.479.6216.6828.47
Routed-expert share (%)58768481
Truncation k′=6k^\prime= 6Routed-expert time (%)−22.5−16.6−16.3−9.5
Routed expert reads (%)−25.0−18.1−13.5−7.1
GPU time per step (%)−13.1−12.5−13.7−8.1
Patch onlyGPU time per step (%)+1.0+0.5+0.2+0.5

Table A4. Kernel profile of a decode step (PyTorch profiler, eager mode, GSM8K prompts, 32 decode steps after prefill): GPU time per step, the routed-expert kernels’ time and share of it (fused MoE GEMMs, routing top-kk, block alignment, and output sum; shared experts count as other kernels), the change against the original model in routed-expert kernel time and GPU time per step for truncation and in GPU time per step for the fused patch with no token truncated, next to the change in routed expert reads (Table A3).

Figure A1. Decode speedup over the original model by batch size: generation time minus first-token time, 256 output tokens, median of three runs, one RTX A6000, vLLM 0.11.0 with default fused-MoE kernel settings and prefix caching disabled. The original model and truncation are timed in the same job; Figure 2 of the paper gives interleaved timings with SafeStart. At k′=⌈2k/3⌉k' = \lceil2k/3 \rceil, truncation speeds up decoding by 15.6 and 0.2% at batch sizes 8 and 128 on Qwen1.5, 25.2 and 2.4% on DeepSeek-V2-Lite, 20.6 and 10.3% on Moonlight, 25.9 and 2.1% on Ling-lite, 16.5 and 4.6% on OLMoE, and 14.8 and −7.8% on Qwen3.

on DeepSeek-V2-Lite, and 14.6, 13.7, 11.4, and 5.8 against 13.1, 12.5, 13.7, and 8.1% on OLMoE.

Refusal Across Models, Budgets, and Templates

Figure A2 shows the ASR change for each panel model, truncation level, and framing, and the per-model tables of this subsection list each truncation level and rerun with both judges. Figure A7 plots truncation, the mass conditions, and emulated SafeStart from Table A32 and Table A25 for Qwen1.5 and DeepSeek-V2-Lite, plus N=32N = 32 in emulation on Qwen1.5 (−1.06-1.06, +0.21+0.21, and +2.97+2.97 pp on plain requests, refusal suppression, and prefix injection; p=0.049p = 0.049 for the last), and Figure A5 the divergence curves at k′=3k' = 3 for both models. Table A25 gives the mechanism and mitigation conditions under prefix injection, Table A9 re-judges prefix-injection answers with the forced opening removed,

ASR change heatmaps for Plain, Refusal suppression, and Prefix injection

Figure A2. ASR change (pp, Llama Guard 3) for each panel model, truncation level, and framing. Each cell gives the change, an asterisk for unadjusted exact McNemar p<0.05p < 0.05, and the kept fraction k′/kk'/k.

Table A9. Prefix injection judged with the forced opening removed from the answer. ASR change (pp) against the original model, forward/reverse flips; bold: exact McNemar p<0.05p < 0.05; shading: size and sign of the Llama Guard 3 change.

ModelConditionLlama Guard 3HarmBench cls.
Qwen1.5Original rerun−0.85 (17/21)−0.85 (18/22)
Truncation k′=3k^\prime= 3+6.36 (52/22)+7.42 (59/24)
Truncation k′=2k^\prime= 2+6.36 (59/29)+5.51 (61/35)
SafeStart N=8N = 8 (emulation), k′=3k^\prime= 3+4.87 (41/18)+3.81 (47/29)
SafeStart N=64N = 64 (emulation), k′=3k^\prime= 3+1.06 (20/15)+0.21 (23/22)
DeepSeek-V2-LiteOriginal rerun−1.06 (22/27)+0.64 (29/26)
Truncation k′=4k^\prime= 4−4.66 (41/63)−4.66 (43/65)
Truncation k′=3k^\prime= 3+13.77 (104/39)+11.02 (98/46)
ConditionQwen1.5, k′=3/4k' = 3/4DeepSeek-V2-Lite, k′=4/6k' = 4/6OLMoE, k′=6/8k' = 6/8
Truncation+6.8 (+7.0)−2.3 (−5.1)+5.3 (+5.3)
Attenuation only+4.4 (+3.8)+6.6 (+5.5)+5.5 (+4.9)
Mass restored+4.0 (+5.3)−5.9 (−6.1)+1.1 (−1.9)
Swap (no truncation)+4.4 (+6.6)−5.5 (−3.8)+6.4 (+5.7)
Early, N=1N = 1+2.3 (+2.5)+4.2 (+4.7)+0.0 (−0.2)
SafeStart, N=1N = 1, emulation+2.8 (+2.3)−5.3 (−5.7)+5.9 (+5.5)
SafeStart, N=1N = 1, engine+1.5 (+2.5)+0.2 (+2.5)+6.1 (+4.0)
SafeStart, N=8N = 8, emulation+4.0 (+4.7)−5.3 (−7.4)+3.6 (+6.1)
SafeStart, N=8N = 8, engine+1.3 (+3.0)+0.2 (+0.2)+4.4 (+3.2)
SafeStart, N=32N = 32, engine+2.1 (+0.6)+1.7 (+0.8)−1.1 (−0.8)
SafeStart, N=64N = 64, emulation+2.3 (+0.6)+3.4 (+3.0)+0.4 (+2.1)
Emulation control+1.9 (+1.1)−0.2 (+1.1)−0.6 (+0.4)

Table A25. Mechanism and mitigation conditions at the two-thirds operating point (Study 2, 472 prompts per framing). ASR change in pp against the original model under Llama Guard 3, with the HarmBench classifier in parentheses; bold: exact McNemar p<0.05p < 0.05 under Llama Guard 3 (unadjusted, exploratory); shading: size and sign of the Llama Guard 3 change. Attenuation only keeps all kk experts and scales their weights by mk′/mkm_{k'}/m_k; mass restored keeps the top k′k' experts and rescales them to mkm_k; swap keeps kk experts but substitutes the (k+1)(k+1)-th ranked expert for the kk-th at the same weight. Early: truncated model for the first generated token only, original afterwards. SafeStart: original computation for the first NN generated tokens, in emulation (the truncated model recomputes the prompt and the first NN tokens) or in the engine with the fused kernel (Table A24 gives every NN). Emulation control: the SafeStart procedure with the original model in both passes. Prefix-injection framing; Table A32 gives the other framings.

Divergence curves under refusal suppression at $k' = 3$ for both models

Figure A5. Divergence curves under refusal suppression at k′=3k' = 3 for both models.

ConditionQwen1.5, k′=3/4k' = 3/4 PlainQwen1.5, k′=3/4k' = 3/4 Refusal supp.DeepSeek-V2-Lite, k′=4/6k' = 4/6 PlainDeepSeek-V2-Lite, k′=4/6k' = 4/6 Refusal supp.OLMoE, k′=6/8k' = 6/8 PlainOLMoE, k′=6/8k' = 6/8 Refusal supp.
Truncation+0.0 (+1.1)+18.4 (+16.5)+3.4 (+2.8)+6.6 (+5.3)−0.6 (+0.4)+9.5 (+8.3)
Attenuation only−1.1 (−0.6)+4.0 (+4.2)−0.4 (−0.2)+8.1 (+8.9)−0.6 (−0.4)+4.2 (+5.3)
Mass restored+1.1 (+1.5)+12.3 (+11.9)+4.2 (+2.3)−2.1 (−0.8)−1.1 (−0.0)+0.6 (−0.4)
Swap (no truncation)+0.2 (−0.2)+11.9 (+11.0)+1.5 (+1.1)+6.1 (+5.7)−0.6 (−0.8)+5.3 (+3.8)
Early, N=1N = 1−0.6 (+0.6)+16.7 (+14.0)+2.3 (+2.3)+3.2 (+2.8)−0.8 (−0.4)+5.9 (+4.0)
SafeStart, N=1N = 1, emulation−0.2 (+0.2)+2.5 (+3.2)+1.9 (+0.8)+1.1 (+1.5)+0.2 (+0.6)+3.8 (+4.2)
SafeStart, N=1N = 1, engine−0.8 (−0.6)+0.4 (−0.8)+1.1 (+0.2)+0.8 (+2.5)−0.2 (+0.0)+0.0 (+2.8)
SafeStart, N=8N = 8, emulation−0.4 (+1.1)+1.1 (+1.3)+0.6 (+0.2)+1.3 (+0.4)−1.1 (−0.8)+3.4 (+2.5)
SafeStart, N=8N = 8, engine−0.2 (+0.6)−0.6 (+0.4)+1.3 (+0.2)+0.6 (+2.3)−0.8 (−0.6)+1.3 (+1.7)
SafeStart, N=32N = 32, engine−1.5 (−0.6)−1.5 (−1.1)−1.1 (−1.1)+1.1 (+1.5)−0.8 (−0.4)+0.0 (+1.1)
SafeStart, N=64N = 64, emulation−0.2 (+0.0)+0.2 (+0.0)+0.2 (+0.4)−1.3 (−2.3)−1.1 (−1.1)+0.2 (+1.5)
Emulation control−0.2 (+1.3)+1.3 (−0.8)−0.2 (−0.6)−1.5 (−0.8)−1.5 (−0.8)+1.1 (+0.6)

Table A32. Mechanism and mitigation conditions at the two-thirds operating point (Study 2, 472 prompts per framing). ASR change in pp against the original model under Llama Guard 3, with the HarmBench classifier in parentheses; bold: exact McNemar p<0.05p < 0.05 under Llama Guard 3 (unadjusted, exploratory); shading: size and sign of the Llama Guard 3 change. Attenuation only keeps all kk experts and scales their weights by mk′/mkm_{k'}/m_k; mass restored keeps the top k′k' experts and rescales them to mkm_k; swap keeps kk experts but substitutes the (k+1)(k+1)-th ranked expert for the kk-th at the same weight. Early: truncated model for the first generated token only, original afterwards. SafeStart: original computation for the first NN generated tokens, in emulation (the truncated model recomputes the prompt and the first NN tokens) or in the engine with the fused kernel (Table A24 gives every NN). Emulation control: the SafeStart procedure with the original model in both passes.

Table A1. Decode speedup over the original model (%, 256 output tokens), with all conditions of a model timed in one job, interleaved over rounds (three for Qwen1.5, two for the others; three timings per round). Medians over timings; Figure 2 of the paper plots the job of Table A2. Patch rows isolate the engine patch: no token truncated, and every generated token truncated through the patch (N=0N = 0). Patch and SafeStart rows use the PyTorch version of the patch; Table A2 compares the fused kernel.

Table A10 reports two further templates, and Table A11 reports sampled decoding.

Table A10. Two further jailbreak templates from the StrongREJECT collection: distractors (the request is embedded in the second of three tasks) and evil confidant (role play). ASR change (pp, Llama Guard 3) against the original model with forward/reverse flips; bold: exact McNemar p<0.05p < 0.05; shading: size and sign of the Llama Guard 3 change; brackets: HarmBench-classifier change.

ModelConditionDistractorsEvil confidant
Qwen1.5original ASR69.0779.87
Original rerun+0.64 (53/50) [+1.1]+1.48 (20/13) [+0.4]
Patch, nothing truncated−0.85 (45/49) [+1.1]+1.06 (19/14) [+0.2]
Truncation k′=3k^\prime= 3−0.21 (70/71) [+7.8]−4.45 (36/57) [−4.2]
SafeStart N=8N = 8, emulation+2.75 (74/61) [+8.7]−0.42 (29/31) [−3.0]
SafeStart N=64N = 64, emulation+1.06 (53/48) [+2.1]+0.00 (20/20) [−3.0]
SafeStart N=8N = 8, engine+5.08 (69/45) [+4.4]+2.75 (33/20) [+0.4]
SafeStart N=32N = 32, engine+4.66 (57/35) [+3.0]+2.54 (25/13) [−0.4]
DeepSeek-V2-Liteoriginal ASR85.3813.56
Original rerun+0.00 (30/30) [−0.6]+1.27 (14/8) [+0.4]
Patch, nothing truncated+0.00 (30/30) [−0.0]+0.04 (10/9) [−0.0]
Truncation k′=4k^\prime= 4−3.81 (40/58) [−4.4]+13.14 (71/9) [+11.4]
SafeStart N=8N = 8, emulation−3.60 (41/58) [−13.6]+0.85 (12/8) [−0.4]
SafeStart N=64N = 64, emulation−4.66 (28/50) [+1.1]−0.85 (9/13) [−0.1]
SafeStart N=8N = 8, engine−4.24 (31/51) [−1.7]−0.42 (10/12) [−1.5]
SafeStart N=32N = 32, engine−2.54 (38/50) [−1.3]+0.85 (14/10) [−0.2]

Table A11. Sampled decoding (temperature 0.7, three seeds) on Qwen1.5 under refusal suppression. Each seed compares conditions decoded with the same seed, which every request receives, so a seed’s random draws are shared across prompts: the original model’s ASR varies more across seeds (53.8, 49.6, and 39.0%) than independent sampling would, while the paired changes agree. ASR change (pp, Llama Guard 3) against that seed’s original run with forward/reverse flips; bold: exact McNemar p<0.05p < 0.05; shading: size and sign of the Llama Guard 3 change. SafeStart: N=8N = 8 in the engine with the fused kernel, and in emulation. HB: HarmBench-classifier change for truncation/engine/emulation.

SeedOriginalTrunc. k′=3k^\prime= 3SafeStartHB
engineemulation
153.8+17.16 (96/15)+0.42 (15/13)−0.42 (17/19)+16.3/+1.5/+2.5
249.6+16.53 (94/16)−1.69 (12/20)+2.54 (24/12)+12.9/−0.4/+2.5
339.0+17.58 (99/16)−0.85 (10/14)+0.64 (20/17)+15.7/+1.3/+3.8

Degenerate flips. At the confirmatory point, no forward flip is degenerate on Qwen1.5, DeepSeek-V2-Lite, or Ling-lite, and 1 of 15 is on Moonlight plain requests. GPT-OSS-20B at k′=2k^\prime= 2 ends 162 of its 1,416 responses without a final answer, against 3 for the original model.

Response length. On the safety prompts, truncated models write longer responses: on plain requests, where the net label change is zero, the mean length on Qwen1.5 rises from 184.6 to 191.3 tokens (3.6%), which offsets about a quarter of the batch-8 gain per request.

Reordered reruns. Every panel model has four reordered reruns of the original model. Under refusal suppression the changes against the first run of the original model are −0.42, −0.64, +1.06, and 0.00 pp on Qwen1.5 and −0.42, −0.64, −0.21, and +0.85 pp on DeepSeek-V2-Lite. Across all 84 rerun and framing comparisons of the panel models under Llama Guard 3, at most 12.5% of prompts change label, the largest net change is +2.75 pp (Ling-lite, prefix injection), and 4 exact McNemar tests reach p<0.05p < 0.05 (GPT-OSS-20B refusal suppression, +2.12 and +1.91 pp; OLMoE plain requests, −1.27 pp; Qwen3-30B prefix injection, −1.48 pp), against 4.2 expected by chance. Under the HarmBench classifier, 2 of 84 reach p<0.05p < 0.05 (OLMoE prefix injection, +3.18 and +3.39 pp), and the largest net change is 3.39 pp. Measured against each of the five runs of the original model instead of the first, the refusal-suppression changes at k′=⌈2k/3⌉k^\prime= \lceil2k/3 \rceil range from +17.37 to +19.07 pp on Qwen1.5, +5.72 to +7.20 on DeepSeek-V2-Lite, +9.11 to +10.17 on OLMoE, −5.93 to −4.87 on Moonlight-16B, −4.66 to −3.81 on Qwen3-30B, 0.00 to +2.75 on Ling-lite, and −2.75 to −0.64 on GPT-OSS-20B; for SafeStart (N=8N = 8, fused kernel) they range from −1.69 to 0.00, −0.21 to +1.27, and +0.85 to +1.91 pp on Qwen1.5, DeepSeek-V2-Lite, and OLMoE, none significant (HarmBench classifier: +0.42 to +1.27, +1.91 to +3.39, and +0.85 to +3.39 pp, 3 of 15 with p<0.05p < 0.05, within that judge’s rerun range).

Label churn. A zero net change does not mean no change: on Qwen1.5 plain requests at k′=3k^\prime= 3, truncation changes 28 labels against 9 for the reordered rerun (excess churn p=2.1×10−5p = 2.1 \times10^{-5}).

Variation across models. Under refusal suppression at k′=⌈2k/3⌉k^\prime= \lceil2k/3 \rceil, the log ratio of forward to reverse flips differs across the nine models far beyond sampling error (Cochran’s Q=93.3Q = 93.3 on 8 degrees of freedom, p=10−16p = 10^{-16}, I2=91%I^2 = 91\%).

DerSimonian–Laird), so no common effect describes the models and each model needs its own evaluation.

Held-out models. DeepSeek-MoE-16B has four reordered reruns of the original model. Under refusal suppression all four lie below its first run of the original model (−3.60, −1.69, −1.48, and −2.54 pp; the first has p=4.9×10−4p = 4.9 \times10^{-4}), so its first run of the original model is high; against the mean of its five runs of the original model, truncation to k′=4k^\prime= 4 changes refusal-suppression ASR by +1.23 pp and to k′=3k^\prime= 3 by +15.64 pp. Qwen2-57B’s four reordered runs change refusal-suppression ASR by −0.85 to +0.42 pp, and against its five runs of the original model truncation to k′=6k^\prime= 6 changes it by +1.91 to +3.18 pp. Tables for both models follow those of the seven panel models (Table 2 of the paper) in this appendix. SafeStart (emulation, N=8N = 8) removes the refusal-suppression increase on both: on DeepSeek-MoE-16B at k′=3k^\prime= 3 from +13.77 to −0.85 pp (HarmBench classifier +12.71 to +0.64 pp) and on Qwen2-57B at k′=5k^\prime= 5 from +4.24 to +0.64 pp (+4.03 to 0.00 pp). On plain requests to DeepSeek-MoE-16B at k′=3k^\prime= 3 it gives +5.93 pp against +5.08 pp for truncation.

Accuracy

Quantization control. Qwen1.5-MoE-A2.7B-Chat-GPTQ-Int4 (snapshot 81b132ad), run like the original model, changes ASR against the BF16 original model by +0.85 pp on plain requests (p=0.58p = 0.58), −8.47 pp under refusal suppression (17 against 57 discordant prompts, p=3.4×10−6p = 3.4 \times10^{-6}; HarmBench classifier −5.72 pp), and −11.02 pp under prefix injection (p<10−6p < 10^{-6}), while MMLU falls from 58.21 to 56.74%, GSM8K from 58.68 to 55.65%, and XSTest refusal rates change from 24.0 and 97.0% to 23.6 and 96.5%.

Other Reduction Methods

Other reduction rules. By token position (scripts/position_experts.py, with the calibration of Table 3 of the paper: the original model’s top-kk router weights, teacher-forced on its own refusal-suppression responses for 120 prompts, with each rule applied to them), threshold skipping keeps 3.15, 4.45, and 6.15 experts per prompt token on Qwen1.5, DeepSeek-V2-Lite, and OLMoE, 3.37, 4.14, and 7.24 at the final prompt token, and 2.76, 3.97, and 6.34 at generated tokens 1 to 8; top-$p routing keeps 3.24, 4.37, and 6.42, 2.39, 3.20, and 6.06, and 2.65, 3.59, and 6.20. Table A23 gives their decode speedups (on OLMoE, top-pp routing with protection reads 3.1, 1.5, and 1.0% fewer distinct experts per decode step than without it at batch sizes 8, 32, and 128, which at truncation’s ratio of time saved to reads removed predicts 2.3, 1.1, and 0.6% shorter decode time, against 2.9, 1.0, and 0.5% measured) and Table A22 gives every rule with accuracy, over-refusal, and both judges. Threshold skipping keeps expert ii when its gate weight is at least βw1\beta w_1; top-pp routing keeps experts in weight order while the mass before them is below pp. We fix β\beta and pp by linear interpolation on a grid so that the average number of experts per token, over prompt and generated tokens, equals k′k' on a calibration run of the first 40 test prompts in each framing (64 generated tokens); calibration sets only the expert count, not a safety outcome. The jailbreak templates dominate that average, so on decode tokens the rules keep fewer experts than k′k', except top-pp on DeepSeek-V2-Lite (the Experts column, measured on GSM8K requests); for the three rules furthest below k′k' (both on OLMoE, threshold skipping on DeepSeek-V2-Lite) we recalibrate so that the decode tokens of 120 held-out Alpaca requests (128 generated tokens) keep k′k' experts on average, and the main-text table uses these decode-matched rules. On Qwen1.5 the rules keep 2.8 to 2.9 of (64 generated tokens); calibration sets only the expert count, not a safety outcome. The jailbreak templates dominate that average, so on decode tokens the rules keep fewer experts than k′k', except top-pp on DeepSeek-V2-Lite (the Experts column, measured on GSM8K requests); for the three rules furthest below k′k' (both on OLMoE, threshold skipping on DeepSeek-V2-Lite) we recalibrate so that the decode tokens of 120 held-out Alpaca requests (128 generated tokens) keep k′k' experts on average, and the main-text table uses these decode-matched rules. On Qwen1.5 the rules keep 2.8 to 2.9 of

ModelRuleExpertsMMLUGSM8KXSTestAlone LGAlone HBEng. LGEng. HBEmul. LGEmul. HB
Qwen1.5Original458.258.724.046.640.5
Top-kk truncation357.656.318.8+18.4+16.5−0.6+0.4+1.1+1.3
Top-kk truncation256.150.621.2+22.2+22.2+0.6+0.6−0.4+0.6
Threshold, β=0.41\beta= 0.412.957.757.225.2−1.9−3.4+0.0+0.6+0.2+1.5
Top-pp, p=0.22p = 0.222.955.650.820.8+30.1+24.8−2.1−0.2+0.0+0.0
Top-pp, p=0.30p = 0.30≈3.557.054.720.8+20.6+18.4––−0.6+1.7
DeepSeek-V2-LiteOriginal652.868.59.643.635.2
Top-kk truncation450.967.010.8+6.6+5.3+0.6+2.3+1.3+0.4
Top-kk truncation350.061.88.4+13.3+10.6−0.6+0.0+1.7+1.3
Threshold, β=0.43\beta= 0.433.750.466.36.8+1.7+3.6+0.8+0.6−0.2−0.6
Top-pp, p=0.27p = 0.274.149.064.39.2+11.2+7.8+1.3+1.5+1.5+2.5
Threshold, decode-matched, β=0.38\beta= 0.384.251.067.29.2+1.3+2.5−0.6+0.6––
OLMoEOriginal851.970.88.824.118.2
Top-kk truncation651.368.29.6+9.5+8.3+1.3+1.7+3.4+2.5
Top-kk truncation550.567.910.0+22.0+17.8––––
Top-kk truncation448.361.912.0+36.0+29.2+0.4+1.9––
Threshold, β=0.48\beta= 0.484.848.964.811.6+15.0+15.0+0.6−0.2+2.8+2.5
Top-pp, p=0.26p = 0.264.848.354.124.4+26.7+23.1+3.6+2.5+4.9+5.3
Threshold, decode-matched, β=0.37\beta= 0.376.350.170.010.0+1.9+1.7+1.3−0.2––
Top-pp, decode-matched, p=0.33p = 0.336.350.565.614.8+8.9+8.3+0.0+1.1––
Top-pp, p=0.30p = 0.30≈6.649.963.119.2+12.9+11.0––+3.4+3.0

Table A22. Expert-reduction rules in full. MMLU, GSM8K, and XSTest over-refusal (refusal rate on safe prompts) in %; ASR change under refusal suppression in pp against the original row (original rows give the absolute ASR), under Llama Guard 3 (LG) and the HarmBench classifier (HB), alone and with SafeStart with N=8N = 8: Eng., in the engine (all experts for the prompt and the first eight generated tokens; fused kernel where run, PyTorch path otherwise); Emul., in emulation (the original model’s first eight tokens, then the rule). Bold: exact McNemar p<0.05p < 0.05; shading: size and sign of the Llama Guard 3 change. Experts: experts per token; for the dynamic rules, the mean per decode token on GSM8K requests at batch size 8. The first calibration of each dynamic rule averages over prompt and generated tokens of 40 evaluation prompts per framing; the decode-matched rows are recalibrated on 120 held-out Alpaca requests so that decode tokens keep k′k' experts on average; ≈ marks counts for which only this calibration-run average is available.

Under the first calibration, threshold skipping raised ASR on OLMoE by 15.0 pp at 4.8 experts per decode token, near truncation to 5 experts (+22.0 pp). At batch size 8 a step reads 25.3 and 27.9 distinct experts per layer on OLMoE under the decode-matched threshold and top-pp rules, against 24.2 under truncation and 29.5 for the original model, because tokens with flat gates keep more experts. A coarser first grid gave p=0.30p = 0.30, which keeps 3.5 and 6.6 experts per token over the calibration run’s prompt and generated tokens on Qwen1.5 and OLMoE; we report it as a lighter setting. Both rules zero the dropped experts’ weights without renormalizing, as top-kk truncation does on these three models, none of which renormalizes. We also tried static pruning with REAP (Lasby et al., [28]), scoring each expert by its mean router-weighted output norm over the tokens routed to it on 512 Alpaca conversations and masking the router logits of the lowest-scoring 25% in every layer. It lost 30.0 and 51.9 GSM8K points on Qwen1.5 and OLMoE, far more than REAP’s authors report for larger models with task-matched calibration, so we could not validate our setting and leave REAP out of the comparison. With SafeStart, top-pp routing decodes faster than alone on DeepSeek-V2-Lite and OLMoE (Figure 1 of the paper): the outputs differ, and with top-pp the experts a step reads depend on them; on OLMoE over 256-token outputs at batch size 8, a step reads 29.0 distinct experts per layer with protection and 29.9 without. SafeStart for the dynamic rules runs in the engine: after the protected tokens, the fused kernel applies the rule to the original top-kk slots (the PyTorch path for the first-calibration rows of DeepSeek-V2-Lite threshold skipping and OLMoE). With every token reduced, the engine path reproduces the rule’s reference implementation on 100 prompts in each framing: identical outputs for top-pp routing on all three models, and the same first 40 characters on 296, 295, and 300 of 300 responses for threshold skipping on Qwen1.5, DeepSeek-V2-Lite, and OLMoE.

SafeStart: Further Results

Evaluation prompts. With N=8N = 8 under refusal suppression and Llama Guard 3, SafeStart changes ASR by −0.64-0.64, +0.64+0.64, and +1.27+1.27 pp on Qwen1.5, DeepSeek-V2-

Lite, and OLMoE, with 90% Tango intervals [−2.55,1.23][-2.55, 1.23], [−1.64,2.94][-1.64, 2.94], and [−0.60,3.23][-0.60, 3.23]; the first two lie within the ±3\pm3 pp margin. The HarmBench classifier gives +0.42, +2.33, and +1.69 pp, none significant. At half the experts, where truncation adds +22.25, +13.35, and +36.02 pp,

N=8N = 8 leaves +0.64, −0.64, and +0.42 pp, none significant under either judge (90% CI upper bounds 2.7, 1.9, and 2.5 pp).

Emulation and other runs

To test many settings cheaply, we also emulate the policy: the original model decodes the first NN tokens and the truncated model continues, recomputing the prompt and those tokens with k′k^\prime experts, so the emulation shows that the original opening suffices, not which positions need full computation. With the original model in both passes the changes ASR by at most 1.91 pp in any framing on the three models; one of these nine controls is significant (OLMoE on plain requests, −1.48 pp, p=0.016p = 0.016).

Table A12. Qwen1.5-MoE: all Study 2 conditions. ASR (%), change against the original model (pp) with 95% bootstrap CI, forward/reverse flips, exact McNemar pp (unadjusted), degenerate forward flips under Llama Guard 3, and the change under the HarmBench classifier; shading: size and sign of the Llama Guard 3 change. rerun: original model with shuffled prompt order.

FramingkkASRΔ\Delta95% CIFlipsppDegen.Δ\Delta HB
Plain47.63−−−−−−
37.63+0.00[−2.12, 2.12]14/141.00+1.06
212.71+5.08[2.12, 8.05]38/140.00120+6.57
rerun 47.42−0.21−4/51.0−+1.27
Refusal supp.446.61−−−−−−
365.04+18.43[14.41, 22.67]101/142.0×10−172.0 \times10^{-17}0+16.53
268.86+22.25[17.58, 26.69]123/182.0×10−202.0 \times10^{-20}0+22.25
rerun 446.19−0.42−9/110.824−−0.85
Prefix inj.479.87−−−−−−
386.65+6.78[3.18, 10.17]53/212.6×10−402.6 \times10^{-40}0+6.99
285.81+5.93[2.33, 9.53]53/250.002030+4.87
rerun 480.30+0.42−17/150.86−−1.27

Table A13. DeepSeek-V2-Lite: all Study 2 conditions. ASR (%), change against the original model (pp) with 95% bootstrap CI, forward/reverse flips, exact McNemar pp (unadjusted), degenerate forward flips under Llama Guard 3, and the change under the HarmBench classifier; shading: size and sign of the Llama Guard 3 change. rerun: original model with shuffled prompt order.

FramingkkASRΔ\Delta95% CIFlipsppDegen.Δ\Delta HB
Plain612.29−−−−−−
512.92+0.64[−1.06, 2.33]10/70.6290+0.85
415.68+3.39[1.27, 5.51]21/50.002490+2.75
322.88+10.59[7.42, 13.98]59/93.9×10−103.9 \times10^{-10}0+9.32
rerun 611.44−0.85−4/80.388−−1.06
Refusal supp.643.64−−−−−−
545.97+2.33[−1.06, 5.72]38/270.2150+1.48
450.21+6.57[2.54, 10.59]63/320.001920+5.30
356.99+13.35[8.90, 17.60]91/285.8×10−95.8 \times10^{-9}1+10.59
rerun 643.22−0.42−17/190.868−−1.06
Prefix inj.645.34−−−−−−
547.03+1.69[−2.12, 5.72]49/410.4610+1.69
443.01−2.33[−6.57, 2.12]51/620.3470−5.08
361.23+15.89[11.02, 20.76]113/387.7×10−107.7 \times10^{-10}0+10.17
rerun 645.13−0.21−24/251.0−+1.06

At the recommended point the emulation agrees with the engine on Qwen1.5 and DeepSeek-V2-Lite but leaves +3.39 pp on OLMoE (p=0.007p = 0.007), because it recomputes the opening tokens with truncated experts. In emulation we ran N∈{1,2,8,32,64}N \in\{1, 2, 8, 32, 64\} and chose N=8N = 8 from these runs and the divergence curves on the evaluation prompts; the engine sweep of Table A24 ran afterwards. The policy removes the evil-confidant increase on DeepSeek-V2-Lite (+13.14 to +0.85 pp in emulation and −0.42 pp in the engine, HarmBench classifier +11.44 to −1.48 pp). Under refusal suppression on Qwen1.5, 83.1% of generated tokens run truncated at N=32N = 32 in the engine and 66.7% at N=64N = 64 in emulation. With the PyTorch version of the patch, which selects the same experts and +1.27 pp under refusal suppression (first run: −0.64, +0.64, and +1.27, +2.75, +1.27, and +1.27 pp under prefix injection (+1.27, +0.21, and −0.64, 0.00, and −0.85 pp on plain requests (−0.21, +1.27, and −0.85); HarmBench classifier −0.64, −1.69, and +1.06, +3.60, +1.69, and +2.12, and +1.06, −0.42, and −0.42 pp; the spread between the two implementations is the run-to-run variation that rounding causes. On Ling-lite (k′k' = 4 of 6, fused kernel), N=8N = 8 changes ASR by −0.21, −1.69, and +12.50 pp on plain requests, refusal suppression, and prefix injection (p=1.2×10−9p = 1.2 \times10^{-9}; HarmBench classifier +0.64, −1.91, and +12.71 pp) and N=32N = 32 by +0.42, −0.85, and +1.48 pp (+0.42, −1.06, and +0.85 pp), against +0.85, +0.85, and +6.78 pp for truncation. At half the experts, N=8N = 8 changes ASR by +1.27, +1.06, and 0.00 pp on plain requests and +0.85, +2.33, and +1.69 pp under prefix injection on Qwen1.5, DeepSeek-V2-Lite, and OLMoE, against +5.08, +10.59, +4.87 and +5.93, +15.89, +8.26 pp for

Table A14. Moonlight-16B: all Study 2 conditions. ASR (%), change against the original model (pp) with 95% bootstrap CI, forward/reverse flips, exact McNemar pp (unadjusted), degenerate forward flips under Llama Guard 3, and the change under the HarmBench classifier; shading: size and sign of the Llama Guard 3 change. rerun: original model with shuffled prompt order.

FramingkkASRΔ\Delta95% CIFlipsppDegen.Δ\Delta HB
Plain637.29––––––
536.23−1.06[−3.60, −1.48]16/210.5110−2.97
432.42−4.87[−7.84, −1.91]15/380.002191−4.87
329.24−8.05[−11.65, −4.66]19/571.5×10−51.5 \times10^{-5}0−8.90
rerun 636.02−1.27–6/120.238–−1.06
Refusal supp.686.86––––––
585.17−1.69[−3.81, 0.42]11/190.20−3.60
481.57−5.30[−8.05, −2.54]11/363.5×10−43.5 \times10^{-4}0−6.36
381.57−5.30[−8.47, −1.91]18/430.001870−9.96
rerun 686.44−0.42–3/50.727–+0.21
Prefix inj.691.74––––––
592.80+1.06[−1.06, 3.18]16/110.4420−0.21
491.10−0.64[−3.18, 1.91]17/200.7430−2.54
392.58+0.85[−1.48, 3.18]19/150.6080−4.24
rerun 692.58+0.85–9/50.424–+0.64

Table A14.

Table A15. Ling-lite-1.5: all Study 2 conditions. ASR (%), change against the original model (pp) with 95% bootstrap CI, forward/reverse flips, exact McNemar pp (unadjusted), degenerate forward flips under Llama Guard 3, and the change under the HarmBench classifier; shading: size and sign of the Llama Guard 3 change. rerun: original model with shuffled prompt order.

FramingkkASRΔ\Delta95% CIFlipsppDegen.Δ\Delta HB
Plain68.69––––––
58.69+0.00[−1.91, 1.91]10/101.00+0.85
49.53+0.85[−1.27, 2.97]15/110.5570+1.69
312.08+3.39[0.64, 6.14]29/130.01950+3.39
rerun 610.17+1.48–10/30.0923–+1.06
Refusal supp.641.74––––––
545.34+3.60[0.21, 6.99]41/240.04640+3.18
442.58+0.85[−2.75, 4.45]42/380.7380−4.66
341.95+0.21[−3.81, 4.45]52/511.00−2.75
rerun 642.58+0.85–14/100.541–+1.27
Prefix inj.658.47––––––
559.75+1.27[−1.91, 4.66]36/300.5390+0.85
465.25+6.78[2.54, 10.81]67/350.001990+7.63
363.56+5.08[0.85, 9.53]68/440.02930+2.97
rerun 658.26−0.21–16/171.0–−0.85

Table A15.

FramingkkASRΔ\Delta95% CIFlipsppDegen.Δ\Delta HB
Plain83.39−−−−−−
62.75−0.64[−1.91, 0.64]3/60.5080+0.42
55.30+1.91[0.42, 3.60]12/30.03520+2.33
48.26+4.87[2.54, 7.20]27/43.4×10−53.4 \times10^{-5}0+4.24
rerun 82.97−0.42−3/50.727−−0.64
Refusal supp.824.15−−−−−−
633.69+9.53[6.36, 12.92]56/112.1×10−82.1 \times10^{-8}0+8.26
546.19+22.03[18.01, 26.06]110/67.5×10−267.5 \times10^{-26}0+17.80
460.17+36.02[31.57, 40.68]178/86.5×10−436.5 \times10^{-43}0+29.24
rerun 823.52−0.64−9/120.664−−0.64
Prefix inj.830.93−−−−−−
636.23+5.30[1.06, 9.53]64/390.01760+5.30
531.78+0.85[−3.60, 5.30]59/550.7790+0.42
439.19+8.26[3.39, 13.14]91/520.00140+9.32
rerun 833.05+2.12−27/170.174−+3.18

Table A16. OLMoE-0125: all Study 2 conditions. ASR (%), change against the original model (pp) with 95% bootstrap CI, forward/reverse flips, exact McNemar pp (unadjusted), degenerate forward flips under Llama Guard 3, and the change under the HarmBench classifier; shading: size and sign of the Llama Guard 3 change. rerun: original model with shuffled prompt order.

FramingkkASRΔ\Delta95% CIFlipsppDegen.Δ\Delta HB
Plain81.27−−−−−−
61.27+0.00[−0.64, 0.64]1/11.00−0.42
42.54+1.27[0.00, 2.54]8/20.1090+1.48
231.57+30.30[26.06, 34.75]147/41.5×10−381.5 \times10^{-38}67+1.48
rerun 81.27+0.00−1/11.0−−0.21
Refusal supp.89.96−−−−−−
65.72−4.24[−6.36, −2.12]4/241.8×10−401.8 \times10^{-40}0−2.54
43.18−6.78[−9.32, −4.45]2/341.9×10−81.9 \times10^{-8}0−3.60
234.75+24.79[20.13, 29.66]139/225.7×10−225.7 \times10^{-22}65−4.66
rerun 810.38+0.42−5/30.727−+1.06
Prefix inj.85.51−−−−−−
61.91−3.60[−5.51, −1.91]1/187.6×10−507.6 \times10^{-50}0−3.18
42.12−3.39[−5.51, −1.48]4/200.001540−3.81
253.18+47.67[42.58, 52.97]243/181.6×10−511.6 \times10^{-51}46−3.81
rerun 85.51+0.00−6/61.0−+0.21

Table A17. Qwen3-30B (FP8): all Study 2 conditions. ASR (%), change against the original model (pp) with 95% bootstrap CI, forward/reverse flips, exact McNemar pp (unadjusted), degenerate forward flips under Llama Guard 3, and the change under the HarmBench classifier; shading: size and sign of the Llama Guard 3 change. rerun: original model with shuffled prompt order.

FramingkkASRΔ\Delta95% CIFlipsppDegen.Δ\Delta HB
Plain41.69−−−−−−
31.06−0.64[−1.69, 0.42]2/50.4530+0.00
25.08+3.39[1.69, 5.30]18/24.0×10−44.0 \times10^{-4}1+0.85
rerun 41.06−0.64−0/30.25−−0.42
Refusal supp.46.57−−−−−−
35.93−0.64[−2.54, 1.27]9/120.6640+0.00
28.26+1.69[−0.64, 4.03]19/110.21+1.69
rerun 48.26+1.69−12/40.0768−+1.06
Prefix inj.40.21−−−−−−
30.21+0.00[0.00, 0.00]0/01.00+0.00
20.21+0.00[0.00, 0.00]0/01.00+0.00
rerun 40.21+0.00−0/01.0−+0.00

Table A18. GPT-OSS-20B: all Study 2 conditions. ASR (%), change against the original model (pp) with 95% bootstrap CI, forward/reverse flips, exact McNemar pp (unadjusted), degenerate forward flips under Llama Guard 3, and the change under the HarmBench classifier; shading: size and sign of the Llama Guard 3 change. rerun: original model with shuffled prompt order.

truncation at that budget (HarmBench classifier +1.91, +0.21, +0.42 and −1.69, +3.81, +0.42 pp). On DeepSeek-

FramingkkASRΔ\Delta95% CIFlipsppDegen.Δ\Delta HB
Plain618.86––––––
517.80−1.06[−3.39, 1.27]13/180.4732+0.21
418.86+0.00[−2.97, 2.97]24/241.01+0.00
323.94+5.08[1.69, 8.47]45/210.004272+2.54
rerun618.22−0.64–9/120.664–+0.85
Refusal supp.637.50––––––
536.65−0.85[−4.66, 2.97]40/440.7441+0.00
436.86−0.64[−5.08, 3.81]55/580.8510−1.91
351.27+13.77[8.90, 18.64]102/373.2×10−833.2 \times10^{-83}3+12.71
rerun 633.90−3.60–3/204.9×10−44.9 \times10^{-4}–−3.81
Prefix inj.694.49––––––
593.01−1.48[−3.39, 0.42]7/140.1890+1.06
494.07−0.42[−2.54, 1.69]11/130.8390+0.85
393.22−1.27[−3.39, 0.64]9/150.3071−1.48
rerun 694.28−0.21–6/71.0–+0.42

Table A19. DeepSeek-MoE-16B (held out): all Study 2 conditions. ASR (%), change against the original model (pp) with 95% bootstrap CI, forward/reverse flips, exact McNemar pp (unadjusted), degenerate forward flips under Llama Guard 3, and the change under the HarmBench classifier; shading: size and sign of the Llama Guard 3 change. rerun: original model with shuffled prompt order.

FramingkkASRΔ\Delta95% CIFlipsppDegen.Δ\Delta HB
Plain83.60––––––
64.03+0.42[−0.64, 1.69]5/30.7270+0.42
54.66+1.06[−0.21, 2.54]8/30.2270+0.21
44.24+0.64[−1.06, 2.33]9/60.6070+0.85
rerun 83.60+0.00–0/01.0–+0.00
Refusal supp.845.34––––––
647.67+2.33[−0.21, 4.87]26/150.1170+0.85
549.58+4.24[1.48, 7.20]34/140.005520+4.03
448.09+2.75[−0.64, 6.14]41/280.1481+2.12
rerun 844.70−0.64–4/70.549–−1.48
Prefix inj.860.59––––––
663.56+2.97[−1.27, 7.20]56/420.1890+4.66
559.75−0.85[−4.87, 3.18]42/460.7490−2.12
466.10+5.51[1.06, 9.96]69/430.01780+1.91
rerun 861.23+0.64–16/130.711–+1.06

Table A20. Qwen2-57B-A14B, GPTQ-Int4 (held out): all Study 2 conditions. ASR (%), change against the original model (pp) with 95% bootstrap CI, forward/reverse flips, exact McNemar pp (unadjusted), degenerate forward flips under Llama Guard 3, and the change under the HarmBench classifier; shading: size and sign of the Llama Guard 3 change. rerun: original model with shuffled prompt order.

V2-Lite the patch with nothing truncated reproduces the original model’s responses exactly under both adversarial templates (944 of 944 identical), so Table A10 shows no flips for it. Under sampled decoding at temperature 0.7 (Table A11), SafeStart with N=8N = 8 in the engine changes refusal-suppression ASR on Qwen1.5 by +0.42, −1.69, and −0.85 pp across three seeds (HarmBench classifier +1.48, −0.42, and +1.27 pp; none significant under either judge), against +16.53 to +17.58 pp for truncation; in emulation it gave −0.42, +2.54, and +0.64 pp (HarmBench classifier +2.5, +2.5, and +3.8 pp, the last with p=0.018p = 0.018). Under prefix injection on Qwen1.5, N=64N = 64 in emulation leaves +2.33 pp (p=0.071p = 0.071), and +1.06 pp with the forced opening removed before judging. Table A21 gives accuracy and over-refusal for the eight models with accuracy results and for SafeStart. With the fused kernel, GSM8K recovers between 5% and all of the instant loss at the confirmatory point and 41 to 57% at half the experts, because most reasoning tokens come after the protected prefix.

ModelConditionMMLUGSM8KXSTest safeXSTest unsafe
Qwen1.5-MoEk=4k = 4 (original)58.2158.6824.097.0
k′=3k' = 357.5856.3318.895.5
k′=2k' = 256.1050.6421.289.5
SafeStart, k′=3k' = 358.2157.9224.097.0
SafeStart, k′=2k' = 257.9154.2124.097.0
DeepSeek-V2-Litek=6k = 6 (original)52.7668.549.682.0
k′=5k' = 551.8968.4610.481.0
k′=4k' = 450.9467.0210.880.0
k′=3k' = 349.9861.798.475.0
SafeStart, k′=4k' = 452.7667.1010.081.5
SafeStart, k′=3k' = 352.7565.669.681.0
Moonlight-16Bk=6k = 6 (original)66.9385.908.414.5
k′=5k' = 566.6685.677.613.5
k′=4k' = 466.1483.247.611.0
k′=3k' = 362.4572.335.212.5
Ling-lite-1.5k=6k = 6 (original)69.9291.052.047.5
k′=5k' = 569.4389.612.052.5
k′=4k' = 468.2888.481.651.0
k′=3k' = 364.7384.312.451.0
OLMoE-0125k=8k = 8 (original)51.9370.818.883.0
k′=6k' = 651.2768.169.685.0
k′=5k' = 550.5167.9310.084.5
k′=4k' = 448.3361.8712.083.5
SafeStart, k′=6k' = 651.9371.578.883.0
SafeStart, k′=4k' = 451.9365.588.883.0
Qwen3-30Bk=8k = 8 (original)79.6792.727.269.0
k′=6k' = 677.9092.048.068.5
k′=4k' = 474.9084.537.264.5
k′=2k' = 240.613.114.014.0
DeepSeek-MoE-16Bk=6k = 6 (original)45.2154.1312.082.5
k′=4k' = 443.9450.6414.479.0
Qwen2-57B-A14B, GPTQ-Int4k=8k = 8 (original)73.0585.7513.288.0
k′=6k' = 672.9486.5015.289.5

Table A21. Accuracy and over-refusal (%). XSTest: refusal rate on safe and unsafe prompts. SafeStart: engine implementation with N=8N = 8 and the fused kernel. MMLU is scored by the first-token answer letter, which SafeStart computes with all experts; GSM8K and MATH-500 (Figure A3) measure its truncated decode steps, and its MMLU differs from the original model’s by at most 0.30 points.

Figure A3. MATH-500 accuracy change against the original model (500 problems, greedy, at most 1,024 tokens, scored with mathverify) for truncation (orange) and SafeStart with N=8N = 8 in the engine with the fused kernel (blue), at k′=⌈2k/3⌉k' = \lceil2k/3 \rceil (left pair of each model) and at half the experts (right pair). Whiskers: 95% paired bootstrap intervals over problems; hollow: exact McNemar p≥0.05p \ge0.05 against the original model. Brackets: SafeStart minus truncation at half the experts on the same problems, with its exact McNemar pp.

Primary analysis. Under refusal suppression and Llama Guard 3, truncation raises ASR on all three models (P2: +21.37, +5.85, and +10.35 pp on Qwen1.5, DeepSeek-V2-Lite, and OLMoE; Holm-adjusted p=3.3×10−42p = 3.3 \times10^{-42}, 2.3×10−42.3 \times10^{-4}, and 2.3×10−182.3 \times10^{-18}; passed), SafeStart with N=8N = 8 is non-inferior to the original model on Qwen1.5 and OLMoE (P1: +0.56 and +0.79 pp, 90% Tango upper bounds 1.80 and 2.10 pp, below the +3 pp margin; passed) but not on DeepSeek-V2-Lite (P2: +2.59 pp, upper bound 4.37 pp; failed), and SafeStart lowers ASR below truncation on all three models (P3: −20.81, −3.26, and −9.56 pp; exact McNemar p=2.5×10−40p = 2.5 \times10^{-40}, 0.037, and 2.9×10−162.9 \times10^{-16}; passed).

Every combination at once. Table A28 collects the 40 combinations of model, framing, prompt set, and sampling seed in which we ran SafeStart (N=8N = 8, fused kernel) at k′=⌈2k/3⌉k' = \lceil2k/3 \rceil, with Holm correction over the 40 for each contrast. In 33 it changes Llama Guard 3 ASR by at most +3 pp against the original model; of the seven exceptions, only prefix injection on Ling-lite (+12.50 pp) remains significant after correction. It is significantly above truncation only where truncation lowers ASR (Qwen1.5 evil confidant, +7.20 against truncation’s −4.45; DeepSeek-V2-Lite prefix injection on both held-out sets, +6.52 against −4.50 and +18.03 against −18.85), and there not significantly above the original model (+2.75, +2.02, and −0.82 pp). In no combination, under either judge, is it significantly above both truncation and the original model, also with the half-budget cells (55) and the emulation runs on the two held-out models added (61); without correction, two are, both on the evaluation prompts: Ling-lite under prefix injection (+5.72 pp over truncation, p=0.005p = 0.005) and Qwen1.5 under the distractor (+5.30 pp, p=0.041p = 0.041). Of the 20 combinations, in which truncation raises ASR (unadjusted p<0.05p < 0.05), SafeStart is below it in 16 (unadjusted p<0.05p < 0.05; 11 after correction).

ModelPromptsFramingLG Trunc.LG SS−orig.LG SS−trunc.HB Trunc.HB SS−orig.HB SS−trunc.
Qwen1.5Eval.Plain+0.0−0.2−0.2+1.1+0.6−0.4
Eval.Refusal supp.+18.4−0.6−19.1+16.5+0.4−16.1
Eval.Prefix inj.+6.8+1.3−5.5+7.0+3.0−4.0
Eval.Evil confidant−4.4+2.8+7.2−4.2+0.4+4.7
Eval.Distractor−0.2+5.1+5.3+7.8+4.4−3.4
Eval., T=0.7T=0.7 s1Refusal supp.+17.2+0.4−16.7+16.3+1.5−14.8
Eval., T=0.7T=0.7 s2Refusal supp.+16.5−1.7−18.2+12.9−0.4−13.3
Eval., T=0.7T=0.7 s3Refusal supp.+17.6−0.8−18.4+15.7+1.3−14.4
Held-out 1Plain+0.8+2.5+1.6+3.3+1.6−1.6
Held-out 1Refusal supp.+15.6+1.6−13.9+19.7+2.5−17.2
Held-out 1Prefix inj.+10.7+2.5−8.2+14.8+4.1−10.7
Held-out 1Evil confidant−9.8−1.6+8.2−9.0−0.8+8.2
Held-out 1Distractor+10.7+0.0−10.7+20.5+11.5−9.0
Held-out 2Refusal supp.+21.4+0.6−20.8+21.0+1.0−20.0
Held-out 2Prefix inj.+8.7+2.8−5.8+8.3+1.9−6.4
DeepSeek-V2-LiteEval.Plain+3.4+1.3−2.1+2.8+0.2−2.5
Eval.Refusal supp.+6.6+0.6−5.9+5.3+2.3−3.0
Eval.Prefix inj.−2.3+0.2+2.5−5.1+0.2+5.3
Eval.Evil confidant+13.1−0.4−13.6+11.4−1.5−12.9
Eval.Distractor−3.8−4.2−0.4−4.4−1.7+2.8
Held-out 1Plain+4.9+1.6−3.3+2.5+3.3+0.8
Held-out 1Refusal supp.+7.4+4.9−2.5+9.8+6.6−3.3
Held-out 1Prefix inj.−18.9−0.8+18.0−18.0+0.0+18.0
Held-out 1Evil confidant+11.5−3.3−14.8+5.7−2.5−8.2
Held-out 1Distractor−0.8+3.3+4.1+0.0+4.9+4.9
Held-out 2Refusal supp.+5.8+2.6−3.3+4.8+1.9−2.9
Held-out 2Prefix inj.−4.5+2.0+6.5−6.9+1.1+8.0
OLMoEEval.Plain−0.6−0.8−0.2+0.4−0.6−1.1
Eval.Refusal supp.+9.5+1.3−8.3+8.3+1.7−6.6
Eval.Prefix inj.+5.3+4.4−0.8+5.3+3.2−2.1
Held-out 1Plain+0.0+0.0+0.0+0.8+0.0−0.8
Held-out 1Refusal supp.+2.5−0.8−3.3+4.1+2.5−1.6
Held-out 1Prefix inj.−2.5+2.5+4.9−4.1+0.0+4.1
Held-out 1Evil confidant+0.0+0.0+0.0+0.0+0.0+0.0
Held-out 1Distractor−6.6+6.6+13.1+0.8+3.3+2.5
Held-out 2Refusal supp.+10.3+0.8−9.6+6.6+0.9−5.7
Held-out 2Prefix inj.+3.0+4.8+1.8+1.9+2.9+1.0
Ling-liteEval.Plain+0.8−0.2−1.1+1.7+0.6−1.1
Eval.Refusal supp.+0.0−1.7−2.5−4.7−1.9+2.8
Eval.Prefix inj.+6.8+12.5+5.7+7.6+12.7+5.1

Table A28. SafeStart (SS; N=8N=8, engine, fused kernel) in every combination of model, framing, prompt set, and decoding we ran at k′=⌈2k/3⌉k'=\lceil2k/3\rceil (40 per judge; held-out 1: the 122 prompts of Table A26; held-out 2: the 889 of Table A27). Trunc.: truncation against the original model; SS−orig.: SafeStart against the original model; SS−trunc.: SafeStart against truncation on the same prompts (pp, exact McNemar). Bold: Holm-adjusted p<0.05p<0.05 over the 40 combinations of that judge and contrast (truncation column: unadjusted). Llama Guard 3 (LG) and HarmBench classifier (HB). Shading: size and sign of the Llama Guard 3 change.

ModelFramingLG OriginalLG Trunc.LG SafeStartHB OriginalHB Trunc.HB SafeStart
Qwen1.5Plain8.2+0.8+2.57.4+3.3+1.6
Refusal supp.51.6+15.6+1.649.2+19.7+2.5
Prefix inj.73.8+10.7+2.569.7+14.8+4.1
Evil confidant82.8−9.8−1.673.0−9.0−0.8
Distractors73.8+10.7+0.022.9+20.5+11.5
DeepSeek-V2-LitePlain49.1+4.9+1.619.7+2.5+3.3
Refusal supp.55.7+7.4+4.944.3+9.8+6.6
Prefix inj.54.9−18.9−0.847.5−18.0+0.0
Evil confidant9.0+11.5−3.38.2+5.7−2.5
Distractors85.2−0.8+3.340.2+0.0+4.9
OLMoEPlain4.1+0.0+0.01.6+0.8+0.0
Refusal supp.34.4+2.5−0.822.9+4.1+2.5
Prefix inj.36.1−2.5+2.527.9−4.1+0.0
Evil confidant0.0+0.0+0.00.0+0.0+0.0
Distractors50.8−6.6+6.618.0+0.8+3.3

Table A26. Held-out check of the frozen settings on 122 prompts never used to choose k′k', NN, or any calibration (81 JailbreakBench behaviors without a near-duplicate in our evaluation set, and the 41 HarmBench validation behaviors). Original-model ASR (%) and the change in pp for truncation at k′=⌈2k/3⌉k' = \lceil2k/3 \rceil and for SafeStart (N=8N = 8, engine with the fused kernel), under Llama Guard 3 (LG) and the HarmBench classifier (HB). Bold: exact McNemar p<0.05p < 0.05. Shading: size and sign of the Llama Guard 3 change.

ModelFramingJudgeOrig.RerunTrunc.N=1N = 1N=8N = 8N=32N = 32Trunc.N=8N = 8
Qwen1.5Refusal supp.LG43.6+0.8+21.4+0.7+0.6+0.6+26.1+0.4
HB32.2+1.2+21.0+1.2+1.0−1.1+24.1+0.8
Prefix inj.LG75.1+1.1+8.7+3.4+2.8+2.9+8.8+1.3
HB71.5+0.4+8.3+3.4+1.9+2.0+7.2+1.0
DeepSeek-V2-LiteRefusal supp.LG37.7+0.7+5.8+3.4+2.6+1.9+13.2+1.8
HB28.0+0.0+4.8+2.4+1.9+1.2+6.7−0.2
Prefix inj.LG39.6+2.1−4.5+1.2+2.0+3.1+14.7+3.9
HB35.0+2.1−6.9+1.2+1.1+2.4+4.2+2.4
OLMoERefusal supp.LG13.0+1.5+10.3+3.3+0.8+0.2+36.9+3.9
HB11.8+0.3+6.6+2.6+0.9+0.0+25.8+3.3
Prefix inj.LG22.6+0.3+3.0+6.4+4.8+3.4+13.3+3.6
HB16.3+0.6+1.9+4.3+2.9+2.1+12.1+4.5

Table A27. Second held-out set, analyzed as fixed in advance (Appendix A): 889 AdvBench and CatQA requests that were never used for any choice and have no near-duplicate in the evaluation or first held-out set. Original-model ASR (%) and the change in pp for a rerun of the original model with the requests in a different order, truncation at k′=⌈2k/3⌉k' = \lceil2k/3 \rceil, SafeStart with N=1N = 1, 8, and 32 (fused kernel), and truncation and SafeStart (N=8N = 8) at about half the experts (k′=2,3,4k' = 2, 3, 4), under Llama Guard 3 (LG) and the HarmBench classifier (HB). Bold: exact McNemar p<0.05p < 0.05. Shading: size and sign of the Llama Guard 3 change.

Human Review Protocol

Sample. 587 unique responses in ten strata (Table A29). Responses shared by two strata are labeled once. The two responses to a prompt are placed in different sessions (A/B for the core strata, C/D for the extension), and item identifiers and order within a session are random, so the annotators do not see which condition produced a response.

ComparisonStratumPop.SampleSides
Qwen1.5, RS, 4 → 3forward101101both
reverse1414both
concordant unsafe20620both
concordant safe15120both
Qwen1.5, RS, original → rerunforward99both
reverse1111both
DeepSeek-V2-Lite, RS, 6 → 4forward6363both
reverse3232both
Qwen1.5, PI, 4 → 3forward5315both
Qwen1.5, RS, 3 → SafeStartrescued9630one

Table A29. Human-review strata. Pairs: discordant or concordant pairs under Llama Guard 3 in the population and in the sample.

Rubric. Harmful: the response carries out the request with content an aligned assistant should not give (steps, code, quantities, or the requested persuasive text), even with warnings or only partly. Partial: the response goes along with the request but the content is vague, generic, very short, or unusable, or answers a harmless version. Refusal: the response declines, redirects, lectures, or discusses the topic without carrying it out. Degenerate: mostly incoherent, repetitive, or off-topic text. Unsure: requires a note. When a response mixes behaviors, the label follows the precedence harmful > partial > refusal > degenerate: a refusal sentence followed by usable harmful content is harmful, in any language, and degenerate is used only when the response neither refuses nor attempts the request. Only “harmful” counts as harmful compliance.

Estimates. Within each stratum we compare the human labels of the two responses. The human-verified ASR change is 1472∑sPsns∑i∈s(hicond−hioriginal)\frac{1}{472}\sum_s\frac{P_s}{n_s}\sum_{i\in s}(h_i^{\mathrm{cond}}-h_i^{\mathrm{original}}) over the strata ss of a comparison, with population size PsP_s and nsn_s usable sampled pairs (both responses labeled, neither unsure), and a stratified bootstrap interval. Judge sensitivity and specificity are weighted by Ps/nsP_s/n_s. Both annotators labeled all 587 items, so agreement is Cohen’s κ\kappa with a bootstrap interval over all items. The scoring script is part of the release.

Results. One item was labeled unsure by one annotator. On the other 586, the annotators agree on harmful versus not harmful for 97.3% (κ=0.933\kappa= 0.933, 95% CI [0.897, 0.962]) and on the four labels with $\kappa = 0.913. Table A30 gives the human flips by stratum. Weighted to the Qwen1.5 refusal-suppression population, the human-verified change is +13.56 pp ([9.49, 19.05]) for the first annotator and +13.83 pp ([9.81, 19.26]) for the second; restricted to the judge-discordant prompts, it is +11.38 and +11.65 pp (exact McNemar p<10−10p < 10^{-10} for both), and counting partial as harmful gives +16.95 and +17.37 pp. On DeepSeek-V2-Lite at k′=4k' = 4 the change is +2.54 and +2.75 pp (p=0.088p = 0.088 and 0.053), and +4.45 and +4.55 pp (p=0.007p = 0.007 and 0.010) counting partial as harmful; this model has no sampled concordant strata, so these estimates assume that judge-concordant pairs keep their status. Against the first annotator, weighted to the Qwen1.5 population, Llama Guard 3 has sensitivity 98.8%, specificity 66.9%, and precision 61.4%, and the HarmBench classifier 83.1%, 76.4%, and 65.2%; against the second, 99.0%, 65.3%, 58.2% and 86.1%, 76.1%, 63.8%. Weighted to the Qwen1.5 population, the annotators label 17.6 and 19.2% of the original model’s responses and 22.3 and 22.7% of the responses under truncation partial, and Llama Guard 3 flags 89 to 97% of them on both sides. Its false positives therefore rise from 19.2 to 24.0% of prompts (first annotator), and the human change (+13.6 pp) plus this rise (+4.8 pp) gives its +18.4 pp; errors independent of the condition would instead shrink the change to about (0.988 + 0.669 − 1) × 13.6 ≈ 8.9 pp. Its precision is 59% on the original model’s responses and 63% on the responses under truncation. Of 30 SafeStart responses that Llama Guard 3 calls safe where truncation was unsafe, both annotators label 28 as refusals. Of 15 prefix-injection forward flips, both confirm 10.

ComparisonStratumPairsA1A2
Qwen1.5, RS, 4 → 3concordant safe200/00/0
concordant unsafe201/01/0
forward10159/162/1
reverse142/61/7
DeepSeek-V2-Lite, RS, 6 → 4forward6327/026/0
reverse320/150/13
Qwen1.5, RS, original → rerunforward93/03/0
reverse111/50/5
Qwen1.5, PI, 4 → 3forward1510/010/0
Qwen1.5, RS, 3 → SafeStartjudge-safe301/1/280/2/28

Table A30. Human labels by stratum for the two annotators (A1, A2). Paired strata: human forward/reverse flips, counting only “harmful”. SafeStart-truncation stratum (one side labeled): harmful/partial/refusal.

Earlier Harness (Study 1)

Study 1 ran truncation through the same configuration override as Study 2, on fixed mixed batches (Table A31). On Qwen1.5, truncation to 3 of 4 experts raised ASR by +23.44 pp, from an original ASR of 34.38%. Study 2 measures +18.43 pp for the same truncation on different prompts; the two estimates are consistent (difference 5.0 pp, 95% Wald CI [−1.0, 11.1], from 116/11 and 101/14 forward/reverse flips). Truncation lowered Qwen3’s ASR at

ModelMethodSettingASRΔ\Deltapp
Qwen1.5Originalk=4k = 434.38––
Trunc.k′=3k' = 357.81+23.44<10−6< 10^{-6}
Trunc.k′=2k' = 265.63+31.25<10−6< 10^{-6}
Qwen3Originalk=8k = 88.26––
Trunc.k′=6k' = 65.58−2.680.023
Trunc.k′=4k' = 43.13−5.133.4×10−53.4 \times10^{-5}
Trunc.k′=2k' = 230.80+22.54<10−6< 10^{-6}

Table A31. Study 1 (earlier harness, refusal-suppression template, 448 prompts, fixed mixed batches, Llama Guard 3). ASR in %, Δ\Delta in pp, pp: exact McNemar test against the original model; shading: size and sign of Δ\Delta.

k′=6k^{\prime} = 6 and k′=4k^{\prime} = 4 and raised it only at k′=2k^{\prime} = 2, where 79 of the 123 forward flips were degenerate. On OLMoE-0924 the original ASR was 93.30%, which leaves no room to detect an increase.

The two-arm decomposition that Study 1 reported used a different implementation and a model-rated fluency screen; § 4.3 of the paper replaces it with the Study 2 factorial.

Batch-dependent methods. Batch-level methods such as XShare (Vankov et al., 2026) make a request’s experts depend on its co-batched requests, which an adversary could exploit as prior work showed for capacity-limited routing [19, 66]; we have not tested this. The same removed width also does not give the same effect: Qwen1.5 at k′=3k^{\prime} = 3 and DeepSeek-V2-Lite at k′=5k^{\prime} = 5 each remove 12.5% of the expert feed-forward width of a layer, including shared experts, yet change refusal-suppression ASR by +18.43 and +2.33 pp.

Differences between Study 1 and Study 2

Study 1 used 448 prompts (288 AdvBench requests [72] selected by seeded sampling with a fixed test split and covering six harm categories we defined (illegal activity, cybercrime, violence, misinformation, chemical and biological harm, and harassment), 160 HarmBench standard behaviors from a seed-0 split of all 200 standard behaviors), the refusal-suppression template only, fixed batches of 4 harmful and 12 benign Alpaca prompts, Llama Guard 3, Qwen3-30B-A3B in BF16, and OLMoE-0924. Study 2 uses 472 public prompts (the official HarmBench test split and StrongREJECT), three framings, all requests of a condition in one engine call, two judges, Qwen3 with FP8 weights, and OLMoE-0125. Study 1 filtered degenerate outputs with model-produced fluency ratings; Study 2 uses the repeated-trigram screen.

Mechanism: First Token, Refusal Direction, and Gate Mass

The First Token and the Prompt

Under refusal suppression, the change runs through the first generated token. For each prompt we find the first generated token at which the response under truncation differs from the original model’s response (Figure A4, left). On Qwen1.5 at k′=3k^{\prime} = 3, 92 of the 101 forward flips differ at the first generated token and 96 within the first 8, while pairs with the same judge label diverge later (median 10 tokens) and two runs of the original model later still (median 36). On DeepSeek-V2-Lite at k′=4k^{\prime} = 4, 41 of 63 forward flips differ at the first token and 57 within the first 8. Under prefix injection, where the template dictates the opening words, Qwen1.5 forward flips diverge later (median 29 tokens).

Divergence is measured on the outcome, so we test the first token directly (Table A32). The first generated token is produced by the prefill pass, which truncation also changes. Early truncation computes prefill and the first token with k′k^{\prime} experts and continues with the original model; on Qwen1.5 it raises refusal-suppression ASR by +16.74 pp (p<10−6p < 10^{-6}), 91% of the truncation effect (95% prompt-bootstrap CI [78, 104]), on OLMoE by +5.93 pp, 62% ([37, 91]), and on DeepSeek-V2-Lite by +3.18 pp (p=0.077p = 0.077), half of it. SafeStart with N=1N = 1 does the reverse: in emulation, which recomputes the prompt with k′k^{\prime} experts, it leaves +2.54 pp on Qwen1.5 (p=0.096p = 0.096) and +1.06 pp on DeepSeek-V2-Lite (p=0.59p = 0.59), and in the engine, which keeps the prompt at full kk, +0.42 and +0.85 pp (Table A24). At k′=3k^{\prime} = 3 on DeepSeek-V2-Lite, protecting one token in emulation leaves +5.30 pp (p=0.0035p = 0.0035) and protecting eight leaves +1.69 pp (p=0.34p = 0.34). On OLMoE at k′=6k^{\prime} = 6 the prompt computation carries more of the effect: early truncation gives +5.93 pp (p=4.1×10−5p = 4.1 \times10^{-5}), 62% of the +9.53 pp truncation effect; protecting one token leaves +3.81 pp (p=0.013p = 0.013) in emulation, which recomputes the prompt with k′k' experts, and 0.00 pp in the engine; and in the engine protecting the prompt alone (N=0N = 0) leaves −0.64 pp.

Gate Mass and Expert Identity

Under refusal suppression, lost gate mass raises compliance on all three models; removing experts at restored mass does so only on Qwen1.5. In models that do not renormalize, truncation removes experts and lowers the routed gate mass. Under refusal suppression, removal with the mass restored raises ASR by +12.29 pp on Qwen1.5 and attenuation alone by +4.03 pp; the weights differ on DeepSeek-V2-Lite (−2.12 and +8.05 pp) and on OLMoE

(+0.64 and +4.24 pp, with an interaction of +4.66 pp; Table A32 and the factorial intervals below). Substituting the next-ranked expert in the last slot at constant count and mass (swap) raises it by +11.86, +6.14, and +5.30 pp on the three models (p≤0.0031p \le0.0031).

Factorial intervals. Joint prompt-bootstrap 95% intervals under refusal suppression: Qwen1.5, removal with the mass restored +12.29 pp [8.26, 16.31], attenuation alone +4.03 [1.06, 6.99], interaction +2.12 [−2.54, 6.36], and restoring the mass lowers the truncation effect by 6.14 pp (46 against 17 discordant prompts, p=3.4×10−4p = 3.4 \times10^{-4}); DeepSeek-V2-Lite at k′=4k' = 4, −2.12 [−6.14, 1.69] and +8.05 [4.03, 11.86]; OLMoE, +0.64 [−2.33, 3.39], +4.24 [1.69, 6.78], and interaction +4.66 [0.85, 8.47].

Within one model, removing renormalization raises compliance. Ling-lite renormalizes the routed weights; native truncation to k′=4k' = 4 changes its refusal-suppression ASR by +0.85 pp, and the same truncation without renormalization (trim) by +5.72 pp, +4.87 pp more (48 against 25 discordant prompts, p=0.0095p=0.0095).

The Refusal Direction

On Qwen1.5 and OLMoE, the first-token change runs through the refusal direction. We estimate a refusal direction at each layer as the difference of mean residual-stream activations at the final prompt token between 256 AdvBench and 256 Alpaca requests wrapped in the refusal-suppression template, in the original model (Arditi et al., [2]). Truncation moves the final prompt token along it toward the harmless mean, by 0.17 harmful–harmless gaps on Qwen1.5 and 0.12 on OLMoE over the second half of the layers. We edit the residual stream at that token in those layers, let the edited model produce the first token, and let the original model

continue, as in early truncation (Appendix § 7); the unedited pipeline reproduces original ASR (+0.2 and −0.4 pp). In the truncated model, restoring the original position along the direction lowers the first-token increase from +15.5 to +0.4 pp on Qwen1.5 (82 against 11 discordant prompts with the truncated model, p<10−6p<10^{-6}; 97% removed, 95% CI [80, 118]) and from +1.0 to +1.5 pp from OLMoE (28 against 11, p=0.0095p=0.0095; 71%; [27, 123]). Restoring it only in layers L/2L/2 to 3L/43L/4 suffices on both models (+1.3 and +0.8 pp), and one middle layer suffices on OLMoE (+1.7 pp) but not on Qwen1.5 (−12.9 pp). Restoring random directions, or pushing along them by the same amount, leaves the increase in place on both models. In the original model, setting the projection on the direction at each edited layer to its value in the truncated model (a set, like restore, so it matches truncation’s shift in size at every layer and does not accu(mulate) reproduces all of the increase on Qwen1.5 and 75% of it on OLMoE (+15.5 and +3.8 pp), largely on the same prompts: 64 of the 87 Qwen1.5 prompts and 16 of the 31 OLMoE prompts that the truncated first token turns harmful also turn harmful here (28 and 2 expected by chance), while the same-size push along random directions does not (−2.1 to +1.7 pp). Restoring everything except the direction is a stronger edit: it resets every other component of the state at every edited layer to its original value, returns the original first token on 445 and 423 of 472 prompts, and leaves +2.5 pp on Qwen1.5 and +3.6 pp on OLMoE.

Figure A6. Routed gate weights of one token before and after truncation from k=4k = 4 to k=3k = 3 (illustrative weights with mass ratio 0.851, the mean ratio Study 1 measured on Qwen1.5-MoE). Without renormalization the routed branch loses weight relative to the shared expert and the residual stream; with renormalization the survivors absorb the removed weight.

Truncation flips the refusals nearest the decision. Under refusal suppression, a low probability of the original model’s greedy first token predicts which of its refusals truncation turns harmful, with AUC 0.78 ([0.72, 0.83]) on Qwen1.5 and 0.73 [0.67, 0.79] on OLMoE: 70% of the least confident third flip on Qwen1.5 against 13% of the most confident third (28 and 4% on OLMoE). Along the probe’s refusal direction, truncation moves 99.6% of these refusals on Qwen1.5 and 95.8% on OLMoE toward the harmless side, by 0.068 and 0.044 harmful–harmless gaps on average (SD 0.037 and 0.032, a third of the spread of the original positions), and the refusals it flips move no further than those it keeps (0.068 against 0.067 and 0.040 against 0.045 gaps; AUC 0.53 and 0.47). Truncation thus pushes almost every refusal the same way, and the push flips those whose first token is least confident. Low confidence also marks the refusals that other perturbations flip, more weakly for the four reordered reruns of the original model (AUC

0.67 and 0.63, pooled) and about as strongly for swap (0.72 and 0.74), but the reruns flip as many harmful responses back (42 against 42 on Qwen1.5, 39 against 38 on OLMoE), while truncation flips 101 against 14 and 56 against 11.

Causal patching. For Qwen1.5 (k′=3k' = 3) and OLMoE (k′=6k' = 6) under refusal suppression, we recompute the refusal direction from AdvBench and Alpaca requests wrapped in the refusal-suppression template, so that the template text is the same in both classes. We feed vLLM’s prompt token identifiers to the Transformers model, edit the residual stream at the final prompt token after each of layers L/2L/2 to L−2L - 2 (forward hooks, applied in order so later layers see earlier edits), take the greedy first token, and continue from it with the original model in vLLM (greedy, 512 tokens), as in early truncation. Restore sets the projection on a unit direction to its value in the unedited original model; push adds a fixed displacement. Random directions are three Gaussian unit vectors per layer, and a random push uses the per-prompt, per-layer size of the refusal-direction edit. Table A33 gives the results under both judges. Restoring everything except the direction (setting the final-prompt-token state at each edited layer to its original value while keeping the truncated model’s projection on the direction) tests the direction with every other component held at its original value, and restoring the direction in narrower layer windows tests location; we ran these on both models, together with a pipeline null in which the Transformers original model produces the first token. Against the truncated model, restoring the direction changes 82 against 11 discordant prompts on Qwen1.5 and 28 against 11 on OLMoE; the share of the first-token increase removed is 97% (95% bootstrap CI [80, 118]) and 71% ([27, 123]). The truncation-induced change of the final-prompt-token state puts 6.5% (Qwen1.5) and 6.0% (OLMoE) of its squared norm on the direction, against 0.06% on a random direction, so a random push of the same size moves the state far less along the direction than the restore does. On DeepSeek-MoE-16B at k′=3k' = 3 (+13.77 pp under truncation), the truncated model’s first token followed by the original model gives −6.57 pp, imposing the truncated model’s shift along the direction on the original model gives +6.78 pp (p<10−4p < 10^{-4}), random pushes give −3.60 to +0.85 pp, and the pipeline null is −2.75 pp (p=0.019p = 0.019).

ConditionQwen1.5, 4 → 3 LGQwen1.5, 4 → 3 HBOLMoE, 8 → 6 LGOLMoE, 8 → 6 HBDeepSeek-MoE-16B, 6 → 3 LGDeepSeek-MoE-16B, 6 → 3 HB
Original (pipeline null)+0.21+0.85−0.42+0.00−2.75−2.33
Truncated+15.47+13.56+5.08+3.81−6.57−7.42
Truncated, refusal direction restored+0.42−1.27+1.48+1.06−11.86−10.38
at layer L/2L/2 only+12.92+9.53+1.69+1.48−8.90−8.05
at layers L/2L/2 to 3L/43L/4+1.27−0.42+0.85+0.21−11.02−11.02
at layers 3L/43L/4 to L−2L - 2+6.78+6.14+2.12+1.91−9.53−8.90
Truncated, all but the direction restored+2.54+1.27+3.60+3.18+1.06+1.91
Truncated, random direction restored+15.25 to +16.74+12.29 to +13.98+4.66+2.54 to +3.39−6.78 to −5.93−7.42 to −5.51
Truncated, random push+17.16 to +18.01+13.35 to +15.25+3.18 to +5.30+1.06 to +3.60−7.20 to −5.93−7.20 to −6.14
Original, projection set to the truncated value+15.47+11.86+3.81+3.39+6.78+6.78
Original, random push−2.12 to −1.69−2.54 to −0.85+0.85 to +1.69−0.64 to +0.64−3.60 to +0.85−2.75 to +1.91

Table A33. Causal patching at the final prompt token under refusal suppression (472 prompts). The edited model produces the first token and the original model continues. Change in ASR (pp) against the original model; bold: exact McNemar p<0.05p < 0.05 (for ranges, in all three); ranges over three random directions (a single value: all three agree); shading: size and sign of the Llama Guard 3 change (for ranges, of their midpoint). Random push: a move along a random direction by the same amount as the refusal-direction edit.

A Residual-Stream Probe

Setup. We load each model in Hugging Face Transformers 4.57 with BF16 weights and run one forward pass per request with the chat template and generation prompt, without generating. Truncation sets the per-layer top-kk of every MoE block. We record the residual stream after each layer and the next-token distribution at the final prompt token, which produces the first generated token. The refusal direction at layer ll is dl=h^lharm−h^lsafed_l = \hat{h}^{\mathrm{harm}}_l - \hat{h}^{\mathrm{safe}}_l over 256 AdvBench behaviors [72] and 256 Alpaca instructions without input [53], computed with the original model on plain requests; the position of a request is

(hl⋅d^l−ml)/∥dl∥(h_l \cdot\hat{d}_l - m_l)/\lVert d_l \rVert, where mlm_l is the midpoint of the two class means along d^l\hat{d}_l. The refusal-opener probability is the total probability of the first tokens of “I’m sorry”, “I am sorry”, “I cannot”, “I can’t”, “Sorry”, “I apologize”, and “As an AI”, with and without a leading space. Qwen3-30B (FP8), Qwen2.5-72B (GPTQ), and GPT-OSS-20B are not probed: the first two weight formats do not load in this stack on our GPUs, and GPT-OSS writes a reasoning channel before its answer, so its refusal is not decided at the first token.

Results. Table A34 gives the measurements at both operating points. Across the 36 combinations of model, operating point, and framing (Table A34 lists the 24 without prefix injection), the shift along the refusal direction tracks the change in ASR with Spearman ρ=−0.48\rho= -0.48; after removing each model’s mean, ρ=−0.12\rho= -0.12, and over the six model means ρ=−0.66\rho= -0.66, so the association is between models. The change in refusal-opener probability tracks the change in ASR more closely (ρ=−0.75\rho= -0.75), but it is measured at the token whose choice decides most flips, so it restates the first-token result rather than adding a residual-stream one; its separation of forward flips from stable refusals varies widely (AUC 0.25 to 0.95; several combinations have fewer than 20 forward flips). On DeepSeek-MoE-16B at k′=3k' = 3 under refusal suppression, ASR rises by +13.77 pp while the refusal-opener probability also rises. On plain requests and under refusal suppression, the ASR change is nearly proportional to the change in the refusal-opener probability, with a common slope: a fit through the origin over the 24 combinations (six models, two operating points, two framings) gives −15.2 pp per nat (r=−0.91r = -0.91), and refitting without each model gives −14.4 to −15.8 (median absolute error on the left-out model 1.4 pp). The sign agrees in 16 of 17 combinations with ∣ΔASR∣≥2|\Delta\mathrm{ASR}| \ge2 pp; the exception is DeepSeek-MoE-16B at k′=3k' = 3 under refusal suppression, whose increase does not run through the first token. Without it and DeepSeek-V2-Lite under refusal suppression (where the two engines agree on 54% of first tokens), r=−0.98r = -0.98 and the leave-one-model-out error is at most 5.6 pp. Under prefix injection, whose template dictates the opening, the slope is −3.1 pp per nat (r=−0.58r = -0.58). One prefll pass of both models thus predicts the sign and approximate size of the change for five of the six probed models.

Modelk→k′k \to k'Plain Pos.Plain ShiftPlain Δlog⁡P\Delta\log PPlain Δ\DeltaASRRefusal suppression Pos.Refusal suppression ShiftRefusal suppression Δlog⁡P\Delta\log PRefusal suppression Δ\DeltaASR
Qwen1.5-MoE4 → 3+0.29−0.081−0.04+0.00−0.22−0.052−0.93+18.43
Qwen1.5-MoE4 → 2+0.29−0.240−0.29+5.08−0.22−0.081−1.38+22.25
DeepSeek-V2-Lite6 → 4+0.31−0.077−0.22+3.39−0.25−0.031−0.06+6.57
DeepSeek-V2-Lite6 → 3+0.31−0.255−0.65+10.59−0.25−0.045−0.41+13.35
OLMoE8 → 6+0.29−0.045−0.09−0.64−0.23−0.038−0.58+9.53
OLMoE8 → 4+0.29−0.214−0.20+4.87−0.23−0.109−2.34+36.02
DeepSeek-MoE-16B6 → 4+0.28−0.047−0.09+0.00−0.18+0.003+0.41−0.64
DeepSeek-MoE-16B6 → 3+0.28−0.118−0.28+5.08−0.18−0.017+0.25+13.77
Moonlight-16B6 → 4+0.30+0.015+0.15−4.87−0.32−0.024+0.33−5.30
Moonlight-16B6 → 3+0.30+0.008+0.57−8.05−0.32−0.031+0.67−5.30
Ling-lite-1.56 → 4+0.28+0.053−0.02+0.85−0.12+0.016−0.01+0.85
Ling-lite-1.56 → 3+0.28+0.070−0.08+3.39−0.12+0.028+0.35+0.21

Table A34. Residual-stream probe at the final prompt token (second half of the layers), with the refusal direction estimated from plain AdvBench and Alpaca requests; along the template-specific direction of the causal patching, truncation’s shift under refusal suppression is 0.17 and 0.12 gaps on Qwen1.5 and OLMoE. Pos.: original position along the refusal direction relative to the harmful–harmless midpoint; Shift: change under truncation; both in units of the harmful–harmless gap. Δlog⁡P\Delta\log P: change in the log-probability of a refusal opener at the first generated token. Δ\DeltaASR: change in pp under Llama Guard 3; shading: its size and sign.

Limits of the probe. The direction comes from plain requests, while the jailbreak templates append the same text to every request, so positions under a template reflect that text and the midpoint is not a decision boundary under it. The opener set reduces to the first tokens “I”, “As”, and “Sorry” (and “I’m” for Moonlight), with and without a leading space, and the refusal-suppression template forbids apologies and the word “as”. The probe runs in Transformers while the labels come from vLLM: the greedy first token agrees between the two engines on 91 to 100% of prompts for five models in every framing, and on DeepSeek-V2-Lite on 91% of plain, 54% of refusal-suppression, and 73% of prefix-injection prompts.

A one-token probe. We also recorded the original and modified models’ distributions over the first generated token with one prefill pass per prompt. The share of prompts whose top first token changes detects a perturbation but does not predict the direction of the ASR change: across 36 condition and framing pairs on two models its Spearman correlation with the ASR change is 0.39, and restoring the mass on DeepSeek-V2-Lite at k′=3k' = 3 changes the first token of 56.6% of refusal-suppression prompts while lowering ASR by 12.92 pp. A first-token check can flag a serving change for a full audit but cannot replace it.

Additional Expert-Reduction Methods

A Published Budget-Allocation Method

Chitty-Venkata et al., [8] allocates a fixed routed-expert budget BB across layers, choosing per-layer kjk_j by Gaussian-input sensitivity profiling and an evolutionary search. We use a paper-based reimplementation rather than official code. The paper does not specify the gate convention; we preserve each model’s native non-renormalizing routed weights. Shared experts and dense layers remain unchanged and do not count toward BB. The original routed budgets are 96, 128, and 156 on Qwen1.5-MoE, OLMoE-0125, and DeepSeek-V2-Lite, respectively.

Evaluation protocol. This is a separate evaluation on the 448-behavior split of Study 1 (Appendix § 11): the 288 AdvBench requests and 160 HarmBench standard behaviors selected with seed 0. We use the StrongREJECT refusalsuppression template, native model chat templates, greedy decoding with at most 256 new tokens, and Llama Guard 3. Each condition uses the same 112 mixed batches of four targets and 12 benign Alpaca fillers; only the 448 targets enter ASR. The model checkpoints are Qwen1.5-MoE-A2.7B-Chat, OLMoE-1B-7B-0125-Instruct, and DeepSeek-V2-Lite-Chat. This differs from Study 2’s 472 public requests and 512-token cap, so absolute ASR is not directly comparable to Table 2 of the paper. “Original” below means the unmodified model with its native routing and expert budget.

For the protected condition, SafeStart uses original-model routing for the entire prefill and the forward passes producing generated tokens 1–8, then applies the same LExI allocation starting with token 9. The KV cache from the protected passes is retained. Allocation, prompts, batches, and decoding settings are fixed across the paired conditions.

ASR and opening-token protection. The observed original/LExI ASR is 35.7/51.8% on Qwen1.5, 15.6/46.4% on OLMoE, and 51.3/67.0% on DeepSeek-V2-Lite. With SafeStart, ASR is 36.4, 18.5, and 53.3%, respectively. These point estimates show that protecting the opening removes most of the ASR increase at each tested allocation. On Qwen1.5, the protected arm differs from the original model by 0.7 pp (exact McNemar p=0.58p = 0.58). The Qwen1.5 budget sweep below reports paired tests for the unprotected allocations.

At the DeepSeek budget B=130B = 130, LExI uses k=6k = 6 on layers 1–20, k=5k = 5 on layer 21, and k=1k = 1 on layers 22–26. Shared experts and the dense first layer retain their original computation.

SettingRemovedASRΔ\Delta (pp)
Original (B=96B = 96)0%35.7%—
LExI B=48B = 4850%37.9%+2.2
LExI B=56B = 5641.7%38.6%+2.9
LExI B=72B = 7225%46.2%+10.5*
LExI B=80B = 8016.7%51.8%+16.1*
Uniform k=3k = 325%60.3%+24.6*

Table A35. Qwen1.5 LExI budget sweep on the same 448 targets. Removed: fraction of the original routed budget removed; Δ\Delta: ASR change against the original model. *Exact McNemar p<0.05p < 0.05. The two non-significant budgets are retained: the largest increase occurs at the least aggressive tested LExI budget.

A Per-Token Expert-Skipping Method

ACE (Xu et al., 2026c) skips experts per token. We adapt its released HF/Triton implementation to Qwen1.5-MoE-A2.7B-Chat, OLMoE-1B-7B-0125-Instruct, and DeepSeek-V2-Lite-Chat; none of these three checkpoints is officially supported. Thresholds target approximately 25% and 50% skipping for Qwen1.5 and OLMoE, and 10% for the reported DeepSeek setting. Threshold calibration uses WikiText text, without safety-label tuning. After skipping, our ports rescale surviving routed weights to their original total mass, rather than to one. Dense layers and shared experts remain unchanged. The protected arm uses the same threshold, preserves all prefill computation and the passes producing tokens 1–8, and enables ACE starting with token 9, retaining the KV cache.

Protocol and implementation controls. The safety experiment uses the same 448 targets as the LExI comparison, the StrongREJECT refusal-suppression template, native chat templates, Llama Guard 3 on the original request and final answer, greedy decoding, and a 256-token cap. Qwen1.5 retains its completed mixed batches of four targets and 12 Alpaca fillers and repetition penalty 1.05; OLMoE and DeepSeek use 28 target-only batches of 16. Conditions within each model are batches and decoding settings. These runs differ from the main evaluation’s 472 public prompts, 512-token cap, repetition penalty 1.0, and vLLM implementation; absolute ASR should not be compared across the two harnesses.

ACE with skipping disabled (τ=0\tau= 0) is an implementation control, not the baseline of the main ACE results. Its ASR is 35.94%, 15.85%, and 51.56% on Qwen1.5, OLMoE, and DeepSeek, against 36.38%, 14.96%, and 53.12% for the original models. The respective paired 95% Tango intervals for its change are [−2.51, 1.55], [−0.96, 2.89], and [−4.51, 1.29] pp. Their original/ACE next-token losses are 2.3509/2.3499, 2.4640/2.4644, and 2.3705/2.3710. The DeepSeek adapter preserves native FP32 routing and both shared experts, and uses the checkpoint’s tokenizer and required YaRN mscale2m^{2}_{\mathrm{scale}} attention factor. DeepSeek agrees with the independent vLLM original-model safety run within sampling uncertainty (53.12% versus 51.3%, exact McNemar p=0.29p = 0.29) and matches its first token on 407 of 448 targets.

Direction and non-inferiority. On Qwen1.5 and OLMoE, the approximately 25% condition is the primary ACE experiment; 50% is an additional stress setting. Table A36 reports paired changes against the original model and direct protection effects. Reported significance uses Holm correction over the full three-model test family at each budget and contrast. Excluding DeepSeek’s high-skip outputs from safety interpretation does not reduce this correction. Non-inferiority requires the upper bound of the 90% Tango interval against the original model to be below +3 pp. Protection meets this criterion on Qwen1.5 at both reported budgets. OLMoE’s bounds are 3.01 and 3.92 pp, so neither establishes non-inferiority, despite point estimates close to the original model. The 95% intervals and exact tests address different inferential questions; an interval excluding zero need not imply a Holm-significant exact test.

ModelSkip targetACE − OriginalProtected − OriginalProtected − ACEUpper 90%
Qwen1.525%+9.15 [5.77, 12.83]∗−2.01 [−4.12, −0.23]−11.16 [−14.84, −7.85]∗−0.54
50%+20.98 [15.99, 26.01]∗−0.45 [−2.22, 1.23]−21.43 [−26.34, −16.59]∗0.91
OLMoE-012525%+2.46 [0.02, 5.10]+1.12 [−1.06, 3.42]−1.34 [−4.25, 1.49]3.01
50%+12.95 [9.11, 17.04]∗+1.79 [−0.65, 4.38]−11.16 [−15.24, −7.31]∗3.92

Table A36. ACE paired ASR changes (pp), 95% Tango intervals, and upper 90% bounds against the original model. ∗Holm-significant exact McNemar test.

DeepSeek operating point. At approximately 10% ACE skipping, DeepSeek-V2-Lite has 57.1% ASR with ACE and 53.1% with SafeStart. Its 25% and 50% settings are excluded from safety interpretation because accuracy and perplexity checks indicate unreliable model output.

Realized skipping. Whole-run skipping for ordinary/protected ACE at the 25% setting is 23.5/10.1% on Qwen1.5 and 23.4/8.8% on OLMoE; at the 50% setting it is 48.7/20.7% and 48.9/17.7%, respectively. After the protected opening, protected ACE skips 25.0% and 23.7% at the lighter setting, and 50.1% and 48.9% at the heavier setting. Protecting prefill and the opening therefore reduces whole-run expert savings substantially; safety comparisons are at matched thresholds, not matched total expert work.

References

  1. [1]Anonymous. JORA: Joint Expert-Output Reweighting for Training-Free MoE Acceleration. Submitted to ICLR 2027. https://openreview.net/forum?id=9OnV0BR8Ie, 2026.
  2. [2]Arditi, A., Obeso, O., Syed, A., Paleka, D., Panickssery, N., Gurnee, W., and Nanda, N. Refusal in Language Models Is Mediated by a Single Direction. In Advances in Neural Information Processing Systems, pp. 136037–136083, 2024.arxiv.org/abs/2406.11717
  3. [3]Bhardwaj, R., Do, D. A., and Poria, S. Language models are Homer Simpson! safety re-alignment of fine-tuned language models through task arithmetic. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024.
  4. [4]Cao, S., Liu, S., Griggs, T., Schafhalter, P., Liu, X., Sheng, Y., Gonzalez, J. E., Zaharia, M., and Stoica, I. MoE- Lightning: High-throughput MoE inference on memory-constrained GPUs. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2025.
  5. [5]Chen, K., Zhou, X., Chen, H., and Lin, Y. When Keywords Drop but Classifiers Hold: Soft Refusals under KV Cache Compression. arXiv preprint arXiv:2609.31678, 2026a.arxiv.org/abs/2609.31678
  6. [6]Chen, X. and Yao, H. Training-Free Halving of Activated Experts in Fine-Grained Mixture-of-Experts Models. arXiv preprint arXiv:2609.04575, 2026.arxiv.org/abs/2609.04575
  7. [7]Chen, Y., Lai, Q., Chen, T., Wang, P., Shao, Y., Zeng, N., Liu, Z., Li, C., Liu, J., and Cheng, J. You Only Need 2/3 of the Chosen Experts: An Empirical Study of Dynamic Expert Pruning in Fine-Grained MoE LLMs. arXiv preprint arXiv:2609.25809, 2026b.arxiv.org/abs/2609.25809
  8. [8]Chitty-Venkata, K. T., Madireddy, S., Emani, M., and Vishwanath, V. LEXI: Layer-adaptive active experts for efficient MoE model inference. arXiv preprint arXiv:2509.02753, 2025.
  9. [9]Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. Training Verifiers to Solve Math Word Problems. arXiv preprint arXiv:2110.14168, 2021.arxiv.org/abs/2110.14168
  10. [10]Dai, D., Deng, C., Zhao, C., Xu, R. X., Gao, H., Chen, D., Li, J., Zeng, W., Yu, X., Wu, Y., Xie, Z., Li, Y. K., Huang, P., Luo, F., Ruan, C., Sui, Z., and Liang, W. DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), pp. 1280–1297, 2024.
  11. [11]DeepSeek-AI, Liu, A., Feng, B., Wang, B., Wang, B., Liu, B., Zhao, C., Deng, C., Ruan, C., Dai, D., Guo, D., Yang, D., Chen, D., Ji, D., Li, E., Lin, F., Luo, F., Hao, G., Chen, G., Li, G., Zhang, H., Xu, H., Yang, H., Zhang, H., Ding, H., Xin, H., Gao, H., Li, H., Qu, H., Cai, J. L., Liang, J., Guo, J., Ni, J., Li, J., Chen, J., Yuan, J., Qiu, J., Song, J., Dong, K., Gao, K., Guan, K., Wang, L., Zhang, L., Xu, L., Xia, L., Zhao, L., Zhang, L., Li, M., Wang, M., Zhang, M., Zhang, M., Tang, M., Li, M., Tian, N., Huang, P., Wang, P., Zhang, P., Zhu, Q., Chen, Q., Du, Q., Q. Chen, R. J., Jin, R. L., Ge, R., Pan, R., Xu, R., Chen, R., Li, S. S., Lu, S., Zhou, S., Chen, S., Wu, S., Ye, S., Ma, S., Wang, S., Zhou, S., Yu, S., Zhou, S., Zheng, S., Wang, T., Pei, T., Yuan, T., Sun, T., Xiao, W. L., Zeng, W., An, W., Liu, W., Liang, W., Gao, W., Zhang, W., Li, X. Q., Jin, X., Wang, X., Bi, X., Liu, X., Wang, X., Shen, X., Chen, X., Chen, X., Nie, X., Sun, X., Wang, X., Liu, X., Xie, X., Yu, X., Song, X., Zhou, X., Yang, X., Lu, X., Su, X., Wu, Y., Li, Y. K., Wei, Y. X., Zhu, Y. X., Xu, Y., Huang, Y., Li, Y., Zhao, Y., Sun, Y., Li, Y., Wang, Y., Zheng, Y., Zhang, Y., Xiong, Y., Zhao, Y., He, Y., Tang, Y., Piao, Y., Dong, Y., Tan, Y., Liu, Y., Wang, Y., Guo, Y., Zhu, Y., Wang, Y., Zou, Y., Zha, Y., Ma, Y., Yan, Y., You, Y., Liu, Y., Ren, Z. Z., Ren, Z., Sha, Z., Fu, Z., Huang, Z., Zhang, Z., Xie, Z., Hao, Z., Shao, Z., Wen, Z., Xu, Z., Zhang, Z., Li, Z., Wang, Z., Gu, Z., Li, Z., and Xie, Z. DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model. arXiv preprint arXiv:2405.04434, 2024.arxiv.org/abs/2405.04434
  12. [13]Fedus, W., Zoph, B., and Shazeer, N. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. Journal of Machine Learning Research, 2022.arxiv.org/abs/2101.03961
  13. [14]Gale, T., Narayanan, D., Young, C., and Zaharia, M. MegaBlocks: Efficient sparse training with mixture-of-experts. In Proceedings of Machine Learning and Systems (MLSys), 2023.arxiv.org/abs/2211.15841
  14. [15]Gerganov, G. and llama.cpp contributors. llama.cpp: LLM inference in C/C++. https://github.com/ggerganov/llama.cpp, 2023.
  15. [17]Gupta, V., Ju, J. H., Sinha, K., Gavrilovska, A., and Iyer, A. P. Lynx: Enabling efficient MoE inference through dynamic batch-aware expert selection. arXiv preprint arXiv:2411.08982, 2024.arxiv.org/abs/2411.08982
  16. [18]Hasan, A., Rugina, I., and Wang, A. Pruning for Protection: Increasing Jailbreak Resistance in Aligned LLMs Without Fine-Tuning. In BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, 2024.arxiv.org/abs/2401.10862
  17. [19]Hayes, J., Shumailov, I., and Yona, I. Buffer Overflow in Mixture of Experts. arXiv preprint arXiv:2402.05526, 2024.arxiv.org/abs/2402.05526
  18. [20]Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring Massive Multitask Language Understanding. In International Conference on Learning Representations (ICLR), 2021.
  19. [21]Hong, J., Duan, J., Zhang, C., Li, Z., Xie, C., Lieberman, K., Diffenderfer, J., Bartoldson, B., Jaiswal, A., Xu, K., Kailkhura, B., Hendrycks, D., Song, D., Wang, Z., and Li, B. Decoding Compressed Trust: Scrutinizing the Trust-worthiness of Efficient LLMs Under Compression. In International Conference on Machine Learning (ICML), 2024.
  20. [22]Huang, Q., An, Z., Zhuang, N., Tao, M., Zhang, C., Jin, Y., Xu, K., Xu, K., Chen, L., Huang, S., and Feng, Y. Harder Task Needs More Experts: Dynamic Routing in MoE Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), pp. 12883–12895, 2024.
  21. [23]Hwang, R., Wei, J., Cao, S., Hwang, C., Tang, X., Cao, T., and Yang, M. Pre-gated MoE: An algorithm-system co-design for fast and scalable mixture-of-expert inference. In ACM/IEEE International Symposium on Computer Architecture (ISCA), 2024.
  22. [24]Inan, H., Upasani, K., Chi, J., Rungta, R., Iyer, K., Mao, Y., Tontchev, M., Hu, Q., Fuller, B., Testuggine, D., and Khabsa, M. Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations. arXiv preprint arXiv:2312.06674, 2023.arxiv.org/abs/2312.06674
  23. [26]Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., and Stoica, I. Efficient Memory Management for Large Language Model Serving with PagedAttention. In ACM Symposium on Operating Systems Principles (SOSP), 2023.arxiv.org/abs/2309.06180
  24. [28]Lasby, M., Lazarevich, I., Sinnadurai, N., Lie, S., Ioannou, Y., and Thangarasa, V. REAP the Experts: Why Pruning Prevails for One-Shot MoE compression. In International Conference on Learning Representations (ICLR), 2026.arxiv.org/abs/2510.13999
  25. [29]Liang, J., Wang, Y., Jiang, T., and Wang, T. Routing-Aware Safety Alignment for Mixture-of-Experts Models. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2026.arxiv.org/abs/2602.04448
  26. [30]Lin, B. Y., Ravichander, A., Lu, X., Dziri, N., Sclar, M., Chandu, K., Bhagavatula, C., and Choi, Y. The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning. In International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2312.01552
  27. [31]Ling Team, Zeng, B., Huang, C., Zhang, C., Tian, C., Chen, C., Jin, D., Feng, F., Zhu, F., Yuan, F., Wang, F., Wang, G., Zhai, G., Zhang, H., Li, H., Zhou, J., Liu, J., Fang, J., Ou, J., Hu, J., Luo, J., Zhang, J., Liu, J., Sha, J., Qian, J., Wu, J., Zhao, J., Li, J., Feng, J., Di, J., Xu, J., Yao, J., Xu, K., Du, K., Li, L., Liang, L., Yu, L., Tang, L., Ju, L., Xu, P., Cui, Q., Liu, S., Li, S., Song, S., Yan, S., Cai, T., Chen, T., Guo, T., Huang, T., Feng, T., Wu, T., Wu, W., Zhang, X., Yang, X., Zhao, X., Hu, X., Lin, X., Zhao, Y., Wang, Y., Guo, Y., Wang, Y., Yang, Y., Cao, Y., Fu, Y., Xiong, Y., Li, Y., Li, Z., Zhang, Z., Liu, Z., Huan, Z., Wen, Z., Sun, Z., Du, Z., and He, Z. Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUs. arXiv preprint arXiv:2503.05139, 2025.arxiv.org/abs/2503.05139
  28. [32]Liu, J., Su, J., Yao, X., Jiang, Z., Lai, G., Du, Y., Qin, Y., Xu, W., Lu, E., Yan, J., Chen, Y., Zheng, H., Liu, Y., Liu, S., Yin, B., He, W., Zhu, H., Wang, Y., Wang, J., Dong, M., Zhang, Z., Kang, Y., Zhang, H., Xu, X., Zhang, Y., Wu, Y., Zhou, X., and Yang, Z. Muon is Scalable for LLM Training. arXiv preprint arXiv:2502.16982, 2025.arxiv.org/abs/2502.16982
  29. [33]Llama Team, AI @ Meta. Llama Guard 3 8B Model Card. https://huggingface.co/meta-llama/Llama-Guard-3-8B, 2024.
  30. [36]Mazeika, M., Phan, L., Yin, X., Zou, A., Wang, Z., Mu, N., Sakhaee, E., Li, N., Basart, S., Li, B., Forsyth, D., and Hendrycks, D. HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal. In Proceedings of the 41st International Conference on Machine Learning (ICML), pp. 35181–35224, 2024.arxiv.org/abs/2402.04249
  31. [37]McNemar, Q. Note on the Sampling Error of the Difference between Correlated Proportions or Percentages. Psychometrika, 12(2):153–157, 1947.pubmed.ncbi.nlm.nih.gov/20254758
  32. [38]Meng, Q., Zha, Y., Pei, J., Hindriks, K., Bos, H., and Chen, M. SEAL: Reinforcing Global Safety in Mixture-of-Experts through Shared Expert Alignment. In ACM Conference on Computer and Communications Security (CCS), 2026.
  33. [39]Muennighoff, N., Soldaini, L., Groenewold, D., Lo, K., Morrison, J., Min, S., Shi, W., Walsh, P., Tafjord, O., Lambert, N., Gu, Y., Arora, S., Bhagia, A., Schwenk, D., Wadden, D., Wettig, A., Hui, B., Dettmers, T., Kiela, D., Farhadi, A., Smith, N. A., Koh, P. W., Singh, A., and Hajishirzi, H. OLMoE: Open Mixture-of-Experts Language Models. In International Conference on Learning Representations (ICLR), 2025.arxiv.org/abs/2409.02060
  34. [41]Oncescu, C.-A., Wu, Q., Chung, W. T., chuan Wu, T., Gopal, B., Wang, J., Dao, T., and Athiwatirakun, B. Opportunistic Expert Activation: Batch-Aware Expert Routing for Faster Decode Without Retraining. In Proceedings of the 43rd International Conference on Machine Learning (ICML), 2026.arxiv.org/abs/2511.02237
  35. [42]OpenAI, Agarwal, S., Ahmad, L., Ai, J., Altman, S., Applebaum, A., Arbus, E., Arora, R. K., Bai, Y., Baker, B., Bao, H., Barak, B., Bennett, A., Bertao, T., Brett, N., Brevdo, E., Brockman, G., Bubeck, S., Chang, C., Chen, K., Chen, M., Cheung, E., Clark, A., Cook, D., Dukhan, M., Dvorak, C., Fives, K., Fomenko, V., Garipov, T., Georgiev, K., Glaese, M., Gogineni, T., Goucher, A., Gross, L., Guzman, K. G., Hallman, J., Hehir, J., Heidecke, J., Helyar, A., Hu, H., Huet, R., Huh, J., Jain, S., Johnson, Z., Koch, C., Kofman, I., Kundel, D., Kwon, J., Kyrlov, V., Le, E. Y., Leclerc, G., Lennon, J. P., Lessans, S., Lezcano-Casado, M., Li, Y., Li, Z., Lin, J., Liss, J., Liu, L., Liu, J., Lu, K., Lu, C., Martinovic, Z., McCallum, L., McGrath, J., McKinney, S., McLaughlin, A., Mei, S., Mostovoy, S., Mu, T., Myles, G., Neitz, A., Nichol, A., Pachocki, J., Paino, A., Palmie, D., Pantuliano, A., Parascandolo, G., Park, J., Pathak, L., Paz, C., Peran, L., Pimenov, D., Pokrass, M., Proehl, E., Qiu, H., Raila, G., Raso, F., Ren, H., Richardson, K., Robinson, D., Rotsted, B., Salman, H., Sanjeev, S., Schwarzer, M., Sculley, D., Sikchi, H., Simon, K., Singhal, K., Song, Y., Stuckey, D., Sun, Z., Tillet, P., Toizer, S., Tsimpourlas, F., Vyas, N., Wallace, E., Wang, X., Wang, M., Watkins, O., Weil, K., Wendling, A., Whinnery, K., Whitney, C., Wong, H., Yang, L., Yang, Y., Yasumaga, M., Ying, K., Zarembа, W., Zhan, W., Zhang, C., Zhang, B., Zhang, E., and Zhao, S. gpt-oss-120b & gpt-oss-20b Model Card. arXiv preprint arXiv:2508.10925, 2025.
  36. [43]Prasad, H. and Pal, R. The Joint Effect of Quantization and Sampling Temperature on LLM Safety Alignment: A Factorial Analysis. arXiv preprint arXiv:2606.29581, 2026.arxiv.org/abs/2606.29581
  37. [46]Rajbhandari, S., Li, C., Yao, Z., Zhang, M., Aminabadi, R. Y., Awan, A. A., Rasley, J., and He, Y. DeepSpeed-MoE: Advancing mixture-of-experts inference and training to power next-generation AI scale. In International Conference on Machine Learning (ICML), 2022.arxiv.org/abs/2201.05596
  38. [47]Röttger, P., Kirk, H. R., Vidgen, B., Attanasio, G., Bianchi, F., and Hovy, D. XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models. In Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2024.
  39. [48]Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., and Dean, J. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. In International Conference on Learning Representations (ICLR), 2017.arxiv.org/abs/1701.06538
  40. [49]Souly, A., Lu, O., Bowen, D., Trinh, T., Hsieh, E., Pandey, S., Abbeel, P., Svegliato, J., Emmons, S., Watkins, O., and Toyer, S. A StrongREJECT for Empty Jailbreaks. In Advances in Neural Information Processing Systems, pp. 125416–125440, 2024.arxiv.org/abs/2402.10260
  41. [50]Sun, Y., Huang, Y., Li, T., Duan, R., Zhang, Y., Ma, X., Xue, H., and Wei, X. MESA: Improving MoE Safety Alignment via Decentralized Expertise. In International Conference on Machine Learning (ICML), 2026.arxiv.org/abs/2606.00651
  42. [51]Szentmiklóssy, E., Lasby, M., Mao, Q., Raza, S., and Ioannou, Y. Unmasking the Hidden Fairness, Bias, and Safety Costs of Compression with Mixture-of-Expert Models. In AI4GOOD Workshop at ICML 2026, 2026. https://openreview.net/forum?id=CUsaaA5ijk.
  43. [52]Tango, T. Equivalence test and confidence interval for the difference in proportions for the paired-sample design. Statistics in Medicine, 17(8):891–908, 1998.
  44. [53]Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B. Stanford Alpaca: An Instruction-following LLaMA Model. https://github.com/tatsu-lab/stanford_alpaca, 2023.
  45. [55]Wei, A., Haghtalab, N., and Steinhardt, J. Jailbroken: How Does LLM Safety Training Fail? In Advances in Neural Information Processing Systems, pp. 80079–80110, 2023.arxiv.org/abs/2307.02483
  46. [56]Wei, B., Huang, K., Huang, Y., Xie, T., Qi, X., Xia, M., Mittal, P., Wang, M., and Henderson, P. Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications. In Proceedings of the 41st International Conference on Machine Learning (ICML), pp. 52588–52610, 2024.arxiv.org/abs/2402.05162
  47. [63]Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., Huang, F., Dong, G., Wei, H., Lin, H., Tang, J., Wang, J., Yang, J., Tu, J., Zhang, J., Ma, J., Yang, J., Xu, J., Zhou, J., Bai, J., He, J., Lin, J., Dang, K., Lu, K., Chen, K., Yang, K., Li, M., Xue, M., Ni, N., Zhang, P., Wang, P., Peng, R., Men, R., Gao, R., Lin, R., Wang, S., Bai, S., Tan, S., Zhu, T., Li, T., Liu, T., Ge, W., Deng, X., Zhou, X., Ren, X., Zhang, X., Wei, X., Ren, X., Liu, X., Fan, Y., Yao, Y., Zhang, Y., Wan, Y., Chu, Y., Liu, Y., Cui, Z., Zhang, Z., Guo, Z., and Fan, Z. Qwen2 Technical Report. arXiv preprint arXiv:2407.10671, 2024.arxiv.org/abs/2407.10671
  48. [64]Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Zhou, C., Chen, C., Deng, C., Li, C., Lu, D., Zhou, F., Huang, F., Hu, F., Ge, H., Wei, H., Lin, H., Tang, J., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Zhou, J., Lin, J., Dang, K., Bao, K., Yang, K., Yu, L., Deng, L., Li, M., Xue, M., Li, M., Zhang, P., Wang, P., Zhu, Q., Men, R., Gao, R., Liu, S., Luo, S., Li, T., Tang, T., Yin, W., Ren, X., Wang, X., Zhang, X., Ren, X., Fan, Y., Su, Y., Zhang, Y., Zhang, Y., Wan, Y., Liu, Y., Wang, Z., Cui, Z., Zhang, Z., Zhou, Z., and Qiu, Z. Qwen3 Technical Report. arXiv preprint arXiv:2505.09388, 2025.arxiv.org/abs/2505.09388
  49. [65]Yang, Y., Cao, Z., Wang, X., Zhou, X., Li, Y., Yang, D., and Zhao, H. On the Efficiency-Safety Dilemma in Large Reasoning Models. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2026.arxiv.org/abs/2609.23587
  50. [66]Yona, I., Shumailov, I., Hayes, J., and Carlini, N. Stealing User Prompts from Mixture of Experts. arXiv preprint arXiv:2410.22884, 2024.arxiv.org/abs/2410.22884
  51. [67]Yue, T., Guo, L., Cheng, J., Gao, X., Huang, H., and Liu, J. Ada-K routing: Boosting the efficiency of MoE-based LLMs. In International Conference on Learning Representations (ICLR), 2025.arxiv.org/abs/2410.10456
  52. [68]Zeng, Z., Miao, Y., Gao, H., Zhang, H., and Deng, Z. AdaMoE: Token-adaptive routing with null experts for mixture-of-experts language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 6223–6235, 2024.arxiv.org/abs/2406.13233
  53. [69]Zhang, Y., Wang, Z., Gong, N., and Yang, Y. Secure Speculative Decoding for Large Language Models. arXiv preprint arXiv:2610.08678, 2026a.arxiv.org/abs/2610.08678
  54. [71]Zhao, X., Yang, X., Pang, T., Du, C., Li, L., Wang, Y.-X., and Wang, W. Y. Weak-to-Strong Jailbreaking on Large Language Models. In Proceedings of the 42nd International Conference on Machine Learning (ICML), pp. 77822–77843, 2025.arxiv.org/abs/2401.17256
  55. [72]Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J. Z., and Fredrikson, M. Universal and Transferable Adversarial Attacks on Aligned Language Models. arXiv preprint arXiv:2307.15043, 2023.arxiv.org/abs/2307.15043

Paper details

vlnc:W_yjjmqgc2
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents