betaFeedback

WorksComputer Science

The Surprising Effectiveness of Deleting Weights in LLM Post-Training

Abstract

How little of a pretrained large language model (LLM) has to change for it to acquire new capabilities? We find that zeroing fewer than 0.05% of its weights, with no other modification, matches full fine-tuning (FFT) at scale across three LLM post-training regimes: reinforcement learning with verifiable rewards, supervised fine-tuning, and on-policy distillation. Our method, Deletion Tuning, learns a binary mask over the gate projection of transformer blocks. Two practical advantages follow: the trained mask is hundreds to thousands of times smaller than the corresponding FFT checkpoint, and Deletion Tuning forgets far less, which addresses a central challenge in continual learning. Under reinforcement learning, its prior-task accuracy stays near the base level while FFT, LoRA, and random sparse fine-tuning lose up to 5.4 points. To understand why masking alone suffices, we analyze the geometry of the learned weight changes under reinforcement learning using stable rank, spectrum drift, and principal-weight overlap. Compared with LoRA and random sparse fine-tuning, Deletion Tuning is closest to FFT across the three measures.

https://agenticlearning.ai/deleting-weights/

Introduction

To acquire new capabilities during LLM post-training, full fine-tuning can alter every parameter, LoRA (Hu et al., 2022) learns a low-rank update, and the Multiple Ticket Hypothesis (MTH) (Adewuyi et al., [1]) and SHiRA (Bhardwaj et al., [3]) optimize values at selected weight positions. Weight deletion is typically used differently: pruning removes weights while preserving an already-trained model’s capabilities for efficient inference ([43, 9]), and the Lottery Ticket Hypothesis finds sparse subnetworks that can be trained to match dense networks (Frankle and Carbin, [8]). Unlike full fine-tuning, LoRA, and sparse updates, deletion writes no new values and only chooses which pretrained weights to keep. Recent work shows that fine-tuned solutions lie close to the pretrained model. RL fine-tuning changes only a small subnetwork (Mukherjee et al., [32]), and task experts lie densely around pretrained weights (Gan and Isola, [10]). We push this observation to its limit across reinforcement learning with verifiable rewards (RLVR), supervised fine-tuning (SFT), and on-policy distillation (OPD), and ask:

Can a pretrained LLM acquire new capabilities by learning only which weights to set to zero?

The answer is yes across all three post-training regimes. We propose Deletion Tuning, which learns only a binary mask over the gate-projection weights of transformer blocks. At deployment, each target weight either retains its pretrained value or is set to zero, so the learned binary mask is the entire adapter. We test Deletion Tuning with Group Relative Policy Optimization (GRPO) [38] for RLVR, standard SFT, and Self-Distillation Fine-Tuning (SDFT) [39] for OPD. Our experiments span two model families, eight backbones from 0.5B to 30B, and both in-distribution and out-of-distribution evaluation. At the largest model scale tested in each setting, Deletion Tuning matches full fine-tuning (FFT) across reasoning, domain knowledge, and tool-use tasks.

The binary-mask form brings two practical advantages: (i) the adapter is compact, with the trained mask hundreds to thousands of times smaller than the corresponding FFT checkpoint and smaller than even LoRA r=1r=1 in our main GRPO comparison; and (ii) Deletion Tuning forgets less, with its forward KL remaining low and its prior-task accuracy staying near the base level across all three Llama scales. Under FFT, LoRA, and random sparse fine-tuning, forward KL grows and prior-task accuracy falls (Figure 2), consistent with forward KL as a predictor of forgetting [40]. Learning new tasks while keeping old ones makes Deletion Tuning a promising candidate for continual learning.

Figure 2. Deletion Tuning keeps prior-task accuracy at every Llama scale while the baselines drift. (a) Forward KL between the trained and base policies on GSM8K during GRPO training, the forgetting predictor of Shenfeld et al. [40]. (b) Change in mean accuracy on a six-task prior-task suite from the base model (dashed line).

Finally, we examine why Deletion Tuning can match FFT with deletion alone. We confirm that the gains come from learning the mask rather than the Wanda prior used in its initialization. We also find that the geometry of Deletion Tuning’s learned weight updates tracks FFT’s more closely than LoRA or random sparse fine-tuning across stable rank, spectrum drift, and principal-weight overlap.

Our contributions are as follows:

  • Deletion alone matches FFT. Deletion Tuning learns which gate-projection weights to delete (§ 4). It is on par with FFT at the largest scale we test in GRPO, SFT, and SDFT, up to a 30B mixture-of-experts model (§ 5.2 and § 5.7). Under GRPO, it also solves problems that the base model fails in all 128 sampled attempts (§ 5.2).

  • Deletion Tuning is compact and forgets less. Its masks are orders of magnitude smaller than FFT checkpoints in all three regimes (§ 5.2 and § 5.7), and it keeps prior-task accuracy throughout training (§ 5.5).

  • Deletion Tuning’s learned updates have FFT-like geometry. Its weight updates resemble FFT’s on three geometric measures (§ 5.6), which helps explain its accuracy.

Related Work

LLM post-training. We study three post-training regimes that differ in where the training signal comes from. Supervised fine-tuning (SFT) imitates fixed labeled targets [50, 33, 47]. Reinforcement learning optimizes a reward on the model’s own samples, from PPO-based RLHF [37, 33] to GRPO [38] with verifiable rewards, the recipe behind DeepSeek-R1 [12]. With verifiable rewards, rule-based checkers or LLM verifiers score each sample (Lu et al., 2026a). DPO [36] instead turns preferences into a closed-form loss. On-policy distillation trains a student on its own rollouts to match a teacher’s per-token distribution [2, 39, 54], and SDFT uses it to reduce catastrophic forgetting [19]. Self-distillation can also teach reasoning behaviors [42]. We use all three to test whether deletion works as a general post-training mechanism.

Sparsity and selective parameter updates. One-shot LLM pruning methods like Wanda [43] and SparseGPT [9] extend classical magnitude-based pruning [14, 58] to LLM scale, while continuous relaxations [24, 28, 17] make binary masks differentiable. Unstructured pruning can even retain the test-time scaling of reasoning models [30]. Earlier mask learning adapted vision backbones and encoder-only language models to downstream tasks such as image and sentence classification [29, 53, 35]. Recent mask methods target semi-structured LLM inference sparsity [6], refinement of a fine-tuned LLM [52], or vision-language backbones [56]. Deletion Tuning instead makes the mask the entire post-training update of a pretrained LLM across RLVR, SFT, and OPD, and it learns how many weights to delete. Sparse weight tuning writes float deltas at selected positions [11, 45, 1, 22, 3], additive adapters introduce trainable float parameters [31, 44, 15, 21] (Hu et al. 2022; Lu et al. 2026b), and robust fine-tuning methods restrict or regularize movement of pretrained weights under distribution shift [20, 48]. Deletion Tuning instead keeps every retained weight at its pretrained value and learns only which gate-projection weights to delete.

Figure 1. Figure 1: Deletion Tuning learns which pretrained weights to delete and writes no new values. The pretrained weights WW stay frozen, and each is paired with a trainable logit in GG, the only parameters that post-training updates. During training, each weight is multiplied by σ(αG)\sigma(\alpha G), a smooth mask between 0 and 1. The sharpness α\alpha rises over training until the sigmoid becomes a hard threshold at G=0G = 0. At deployment, weights with positive logits (blue) keep their pretrained values and weights with negative logits (red) are set to zero (crossed out), so the trained adapter is one bit per weight.

Understanding LLM fine-tuning. The classical Lottery Ticket Hypothesis [8] posits that randomly initialized networks contain sparse subnetworks that train to comparable accuracy in isolation. Recent work characterizes pretrained-model fine-tuning along three axes. Density: task-expert solutions densely populate the neighborhood of pretrained weights [10]. Geometry: Mukherjee et al. [32] report that RL fine-tuning naturally updates only 5 to 30% of parameters across PPO, GRPO, DPO, and several model families. Zhu et al. [57] show these updates land off the principal weights of W0W_0 while leaving the spectrum largely intact, where principal weights are the top-magnitude entries of the rank-kk SVD reconstruction of W0W_0 [22]. Forgetting: Shenfeld et al. [40] establish that the forward KL between fine-tuned and base policies on the new task accurately predicts forgetting. Deletion Tuning reproduces the off-principal update pattern and low spectrum drift. Deletion Tuning’s low forward KL also predicts the strong prior-task retention we measure (§ 5.5).

Background

Our experiments use three post-training objectives.

Supervised fine-tuning (SFT). Given a dataset D\mathcal{D} of prompts xx and target responses yy of length L=∣y∣L = |y|, SFT minimizes the negative log-likelihood of the target tokens, which are fixed in advance and never sampled from the model:

LSFT(θ)=−E(x,y)∼D∑j=1Llog⁡πθ(yj∣x,y<j).(1)\mathcal{L}_{\mathrm{SFT}}(\theta)=-\mathbb{E}_{(x,y)\sim\mathcal{D}}\sum_{j=1}^{L}\log\pi_{\theta}(y_j\mid x,y_{<j}). \tag*{(1)}

Group Relative Policy Optimization (GRPO). For RLVR, we use GRPO [38]. For each prompt xx, the policy before the update, πθold\pi_{\theta_{\mathrm{old}}}, samples a group of nn responses {yi}i=1n\{y^i\}_{i=1}^{n}, and a verifier assigns each one a reward ri=r(x,yi)r^i=r(x,y^i). Rather than learning a value function as in PPO [37], GRPO compares each response with the rest of its group, Ai=(ri−mean⁡(r))/std⁡(r)A^i=(r^i-\operatorname{mean}(r))/\operatorname{std}(r), and gives this advantage to every token of yiy^i in PPO’s clipped objective,

LGRPO(θ)=−Ex,{yi}1n∑i=1n1∣yi∣∑j=1∣yi∣min⁡(ρjiAi,clip⁡(ρji,1−ϵ,1+ϵ)Ai),(2)\mathcal{L}_{\mathrm{GRPO}}(\theta)=-\mathbb{E}_{x,\{y^i\}}\frac{1}{n}\sum_{i=1}^{n}\frac{1}{\lvert y^i\rvert}\sum_{j=1}^{\lvert y^i\rvert}\min\left(\rho^i_jA^i,\operatorname{clip}\left(\rho^i_j,1-\epsilon,1+\epsilon\right)A^i\right), \tag*{(2)}

where ρji=πθ(yji∣x,y<ji)/πθold(yji∣x,y<ji)\rho^i_j=\pi_{\theta}(y^i_j\mid x,y^i_{<j})/\pi_{\theta_{\mathrm{old}}}(y^i_j\mid x,y^i_{<j}) is the importance ratio for token jj. GRPO optionally adds a KL penalty to the reference model, weighted by β\beta.

Self-Distillation Fine-Tuning (SDFT). For on-policy distillation, we use SDFT [39], in which the model is its own teacher. The teacher’s weights θT\theta_T are an exponential moving average of the student’s weights θ\theta,

θT←(1−γ)θT+γθ,(3)\theta_T\leftarrow(1-\gamma)\theta_T+\gamma\theta, \tag*{(3)}

with γ<1\gamma<1, and the teacher’s prompt also includes dd, a reference response to the same question, which the student never sees. For a response y∼πθ(⋅∣x)y\sim\pi_{\theta}(\mathord{\cdot}\mid x) sampled by the student, the loss is the forward KL between the teacher’s and the student’s next-token distributions at each position:

LSDFT(θ)=Ex,y∼πθ(⋅∣x)∑j=1∣y∣KL⁡(πθT(⋅∣d,x,y<j)∥πθ(⋅∣x,y<j)).(4)\mathcal{L}_{\mathrm{SDFT}}(\theta)=\mathbb{E}_{x,y\sim\pi_{\theta}(\mathord{\cdot}\mid x)}\sum_{j=1}^{\lvert y\rvert}\operatorname{KL}\left(\pi_{\theta_T}(\mathord{\cdot}\mid d,x,y_{<j})\mathbin{\|}\pi_{\theta}(\mathord{\cdot}\mid x,y_{<j})\right). \tag*{(4)}

Deletion Tuning

Deletion Tuning parameterizes fine-tuning as one trainable scalar logit per pretrained weight at a chosen subset of linear layers, with the base model frozen. During training, a sigmoid of each logit is annealed toward a step function, and at deployment each logit becomes a single bit deciding whether the corresponding weight is kept or deleted (set to zero). In pruning terms, Deletion Tuning performs learned unstructured pruning to fine-tune the model.

Notation. Let W∈Rdout×dinW\in\mathbb{R}^{d_{\mathrm{out}}\times d_{\mathrm{in}}} denote a target weight matrix in the pretrained model and G∈Rdout×dinG\in\mathbb{R}^{d_{\mathrm{out}}\times d_{\mathrm{in}}} a trainable logit matrix of the same shape, where the sign of GijG_{ij} at deployment decides whether WijW_{ij} is kept (positive) or zeroed (negative). We mask one such WW per target layer, each with its own independent GG. In all our experiments, the target is the gate projection of transformer blocks (a single MLP weight per block). We follow GLU-specific pruning methods by treating the gate/up/down pathway as the main coupled sparsification structure [13] (Ma et al. 2023): in the Llama and Qwen2.5 backbones we evaluate, SwiGLU MLPs take the form down⁡(SiLU⁡(gate⁡(x))⊙up⁡(x))\operatorname{down}(\operatorname{SiLU}(\operatorname{gate}(x))\odot\operatorname{up}(x)), so the gate projection multiplicatively controls the intermediate channels routed through the down projection.

Forward pass. At training step t∈{0,1,…,T}t \in\{0,1,\ldots,T\} (with TT the total step count), the layer applies a sigmoid relaxation of the binary masking decision per weight:

W~(t)=W⊙σ(α(t)G),σ(x)=11+e−x.(5)\widetilde{W}(t) = W \odot\sigma(\alpha(t)G), \qquad\sigma(x) = \frac{1}{1+e^{-x}}. \tag*{(5)}

We call σ(α(t)G)\sigma(\alpha(t)G) the soft mask and 1[G>0]\mathbb{1}[G>0] the hard mask. The sharpness α(t)>0\alpha(t)>0 sets how steep the sigmoid is around G=0G=0. A smaller α\alpha widens the band of logits whose soft mask lies strictly between 0 and 1, and raising α\alpha pulls the soft mask onto the hard one by the end. We use a linear ramp from αstart=103\alpha_{\mathrm{start}}=10^3 to αend=104\alpha_{\mathrm{end}}=10^4, which we call mask sharpening:

α(t)=αstart+tT(αend−αstart).(6)\alpha(t) = \alpha_{\mathrm{start}} + \frac{t}{T}(\alpha_{\mathrm{end}}-\alpha_{\mathrm{start}}). \tag*{(6)}

At evaluation, the layer applies the hard mask:

W^=W⊙1[G>0].(7)\widehat{W} = W \odot\mathbb{1}[G>0]. \tag*{(7)}

Gradients are computed through the deterministic soft mask in Eq. (5), while the hard mask is used for evaluation and for generating rollouts in on-policy runs. By the end of training, the two agree: deploying the hard mask instead of the soft one changes accuracy by at most 0.08 points.

Wanda prior initialization. We initialize GG from the Wanda one-shot pruning score Sun et al., [43], an importance signal validated as a strong predictor of which base weights are safe to zero. The Wanda score of a weight is its magnitude times the norm of its input activation, sij=∣Wij∣⋅∥Xj∥2s_{ij}=|W_{ij}|\cdot\lVert X_j\rVert_2, where XjX_j collects the activations of input channel jj in one forward pass over calibration prompts from the task’s training distribution. Specifically, we sample each gate logit from one of two Gaussians depending on whether the weight is above or below the η\eta-quantile of ss:

Gij∼{N(0.03,0.012)if sij is in the top 1−η fraction (high importance),N(−0.03,0.012)otherwise (low importance).(8)G_{ij} \sim \begin{cases} \mathcal{N}(0.03,0.01^2) & \text{if }s_{ij}\text{ is in the top }1-\eta\text{ fraction (high importance)},\\ \mathcal{N}(-0.03,0.01^2) & \text{otherwise (low importance).} \end{cases} \tag*{(8)}

At our default η=1/2048\eta=1/2048, used in every main experiment, this zeros approximately ηΦ(3)+(1−η)Φ(−3)≈0.18%\eta\Phi(3)+(1-\eta)\Phi(-3)\approx0.18\% of gate-projection weights under the hard mask at initialization, where Φ\Phi is the standard normal CDF.

Training objective. We instantiate the policy πθ\pi_\theta of § 3 by replacing WW with W~(t)\widetilde{W}(t) and keeping pretrained weights elsewhere, so the gate logits GG are the only trainable parameters in the model. The training loss Ltask(W~(t))\mathcal{L}_{\mathrm{task}}(\widetilde{W}(t)) is the regime’s own objective from Eqs. (1) to (4), plus the MaskLLM sparse weight regularizer −λ∥W~(t)∥22-\lambda\lVert\widetilde{W}(t)\rVert_2^2, which mildly biases the mask toward retaining higher-magnitude weights. We do not constrain GG to any fixed sparsity level, leaving the training process to decide it.

  • SFT. The loss is LSFT\mathcal{L}_{\mathrm{SFT}}, with λ=10−5\lambda=10^{-5}.

  • GRPO. The loss is LGRPO\mathcal{L}_{\mathrm{GRPO}}, with λ=10−5\lambda=10^{-5}. Policy rollouts use the hard mask of Eq. (7), while gradients flow into GG through the soft mask of Eq. (5).

  • SDFT. The loss is LSDFT\mathcal{L}_{\mathrm{SDFT}}, with λ=0\lambda=0, since the EMA teacher already provides an anchor. Rollouts use the hard mask as in GRPO, and the EMA teacher of Eq. (3) specializes to a teacher gate GT←(1−γ)GT+γGG_T \leftarrow(1-\gamma)G_T+\gamma G, so the teacher’s frozen base weights WW stay the same as the student’s.

Compact storage and ease of deployment. The trained adapter is the hard mask 1[G>0]\mathbb{1}[G>0] at each masked layer, which takes one bit per masked pretrained weight before compression. We bit-pack the booleans with NumPy and compress the bit stream with zstd Collet and Kucherawy, [5]. Because fewer than 0.05% of all weights are deleted in every GRPO run of Table 1, the bit stream is highly redundant, and zstd compresses it to 1.0 MB at Llama1B and 8.4 MB at Llama8B. This gives Deletion Tuning a practical deployment advantage in compact storage and cheap adapter switching: changing tasks only loads a compact bitmask and scatters zeros into the base weights, rather than loading a separate full model checkpoint.

MethodAcc.SizeAcc.SizeAcc.SizeAcc.SizeAcc.SizeAcc.Size
Llama, GSM8KQwen, SimpleRL-Zoo
1B3B8B0.5B3B7B
Base1.21053.53060.27010.44037.73040.180
FFT60.50471482.491225683.243063319.01188544.351177251.8929051
LoRA r=159.292.782.395.883.3810.020.302.143.557.152.139.6
LoRA r=460.5810.882.4423.282.8740.020.308.443.5928.651.7038.5
MTH 99%60.7368.280.53177.483.31443.621.8427.344.16170.452.03420.7
MTH 99.95%58.614.182.1710.783.0726.620.161.743.8713.751.2425.3
Deletion Tuning54.441.081.493.183.558.417.640.541.982.752.326.9

Table 1. Deletion Tuning has the smallest adapter at every scale and matches FFT at the largest backbone of each family. Best-checkpoint validation accuracy (%) and adapter size (MB) under GRPO, on GSM8K for Llama and on the six-benchmark SimpleRL-Zoo mean for Qwen. The baselines are full fine-tuning (FFT), low-rank adapters (LoRA), and random sparse fine-tuning (MTH).

Connection to MaskLLM. MaskLLM [6] learns 2:42:4 masks for inference speedup with a Gumbel-Softmax selector over N:MN:M candidate patterns. Deletion Tuning has a different goal, expanding model capability, and a different form, with one deterministic logit per weight and no fixed sparsity pattern (Appendix § 12 relates the two formally). Deletion Tuning also adopts MaskLLM’s Wanda prior initialization (Eq. (8)) and its sparse weight regularizer.

Experiments

Experimental Setup

Tasks, benchmarks, and models. We study three post-training regimes on two model families, referring to a family as Llama or Qwen and to a size by its suffix (e.g., Llama1B, Qwen7B). Evaluation uses greedy decoding, and we report the best validation accuracy across saved checkpoints.

  • GRPO, in distribution. We train and evaluate on GSM8K [4] with Llama1B, Llama3B, and Llama8B (Llama3.2-1B/3B-Instruct and Llama3-8B-Instruct) [23], validating every 25 optimizer steps.

  • GRPO, out of distribution. Following SimpleRL-Zoo [51], we train on its math corpus and report the mean over six held-out benchmarks (GSM8K, MATH500, Minerva Math, OlympiadBench, AIME24, AMC23; per-benchmark results in Appendix § 9.1) with Qwen0.5B, Qwen3B, and Qwen7B (Qwen2.5 base) [34]. At larger scale, we add Qwen14B (Qwen2.5-14B) and the Qwen3-30B-A3B mixture-of-experts model (30B total and 3B active parameters). We validate every 10 optimizer steps.

  • SFT. We train Llama1B and Llama3B (Llama3.2-1B/3B-Instruct) separately on each of the 14 MMLU-Pro [49] subjects and 23 BBEH [18] subtasks, validate at the end of training, and average accuracy within each suite.

  • SDFT. We follow the ToolAlpaca [46] tool-use and Science settings of Shenfeld et al. [39] with Qwen7B (Qwen2.5-7B-Instruct), reporting exact-match accuracy under each task’s evaluator. Science is the Chemistry L-3 subset of SciKnowEval [7]. We validate every 100 optimizer steps on ToolAlpaca and every 25 on Science.

Training pipelines. All runs use AdamW with fp32 master weights, bf16 mixed precision, and gradient checkpointing.

  • GRPO. We use verl [41] with a constant learning rate. GSM8K uses prompt batch 64, n=4n=4 rollouts, response length 4096, 3 epochs, and no KL penalty, while SimpleRL-Zoo uses prompt batch 256, n=8n=8 rollouts, response length 4096, 150 steps, and KL penalty 10−410^{-4}.

  • SFT. We minimize cross-entropy with a constant learning rate for T=400T=400 steps per task at effective batch 4, with a 75/25 train/eval split (per-task counts in Appendix § 10).

  • SDFT. We follow Shenfeld et al. [39]: cosine schedule with 0.1 warmup ratio, prompt batch 32, 2 epochs, EMA mixup γ=0.01\gamma=0.01, and one demonstration prepended to the teacher’s input.

Baselines. All three baselines use the same data and number of steps as Deletion Tuning.

  • FFT. Full fine-tuning trains all weights and is the reference we aim to match. Its learning rate is 10−610^{-6} on GSM8K, 5×10−75 \times10^{-7} on SimpleRL-Zoo, 10−610^{-6} to 3×10−63 \times10^{-6} on SFT (Appendix § 14), and 5×10−55 \times10^{-5} on SDFT as in Shenfeld et al. [39].

  • LoRA. We train rank-1 and rank-4 adapters (Hu et al., 2022) on all seven projections in each block (q/k/v/o and gate/up/down) with learning rate 10−410^{-4}.

  • MTH. The closest sparse-update method (Adewuyi et al., [1]) trains a random 1% or 0.05% of all parameters (99% and 99.95% sparsity), with the learning rates recommended by Adewuyi et al. [1] (10−410^{-4} and 10−310^{-3}).

Deletion Tuning implementation. We compute the Wanda scores once per task, on 2048 training prompts for GRPO and SFT and on 32 for SDFT.

  • GRPO. Every main run uses η=1/2048\eta= 1/2048, learning rate 3×10−43 \times10^{-4}, (αstart,αend)=(103,104)(\alpha_{\mathrm{start}}, \alpha_{\mathrm{end}}) = (10^{3}, 10^{4}), and λ=10−5\lambda= 10^{-5}, with no tuning per benchmark or model size (§ 5.8 varies η\eta, learning rate, and λ\lambda).

  • SFT. We keep the same η\eta and λ\lambda, with learning rate 5×10−45 \times10^{-4} and (αstart,αend)=(102,103)(\alpha_{\mathrm{start}}, \alpha_{\mathrm{end}}) = (10^{2}, 10^{3}) for every subject and subtask.

  • SDFT. We set η=1/2048\eta= 1/2048 and λ=0\lambda= 0 and use a hard teacher mask, with learning rate 5×10−35 \times10^{-3} and (102,104)(10^{2}, 10^{4}) on ToolAlpaca and 3×10−33 \times10^{-3} and (102,103)(10^{2}, 10^{3}) on Science.

Hardware. We train on H200 GPUs, using one GPU for backbones up to 3B and multiple GPUs with FSDP (Zhao et al., [55]) for larger models. SFT and the post-hoc analyses use L40S GPUs.

Results on GRPO Post-Training

We train Deletion Tuning and three baselines (FFT, LoRA at ranks 1 and 4, MTH at 99% and 99.95%) on the two GRPO settings of § 5.1. Table 1 reports best-checkpoint validation accuracy and bytes-on-disk adapter size for six backbones from 0.5B to 8B across two model families. Every Deletion Tuning run deletes fewer than 0.05% of all weights (Appendix § 9). Appendix § 8 defines adapter size per method.

Deletion Tuning matches FFT at 7B and 8B. At the largest backbone in each family, Deletion Tuning scores 83.55% against FFT’s 83.24% on Llama8B GSM8K and 52.32% against 51.89% on Qwen7B SimpleRL-Zoo. Tuning FFT’s learning rate and KL coefficient does not change this (Appendix § 14). At these two

Learning the Mask Drives the Gains

backbones, Deletion Tuning is also competitive with both LoRA ranks and both MTH sparsity levels (Table 1). Since Deletion Tuning can only delete pretrained weights, this restriction may act as a mild regularizer, which could explain why Deletion Tuning matches FFT here. The same restriction pays off more clearly for retention, where Deletion Tuning keeps the prior-task accuracy that FFT loses (§ 5.5).

The gap to FFT closes with scale. At smaller backbones, Deletion Tuning recovers 90% to 99% of FFT’s accuracy. At larger scale, Deletion Tuning matches FFT on Qwen14B and on the Qwen3-30B-A3B mixture-of-experts model (Table 2). For the mixture-of-experts model, we mask the gate projection inside each of its 128 experts in every layer and leave the router unchanged. These results suggest that deletion becomes more effective as models grow.

MethodLlama8B Acc.Llama8B SizeQwen7B Acc.Qwen7B SizeQwen14B Acc.Qwen14B SizeQwen3-30B-A3B Acc.Qwen3-30B-A3B Size
Base60.27040.18044.37049.410
FFT83.243063351.892905154.875634358.43116471
Deletion Tuning83.558.452.326.955.4311.458.5429.6

Table 2. Deletion Tuning matches FFT on dense and mixture-of-experts models from 7B to 30B with an adapter thousands of times smaller. Best-checkpoint validation accuracy (%) and adapter size (MB) under GRPO. Qwen14B and Qwen3-30B-A3B use the SimpleRL-Zoo setup of Qwen7B.

Deletion Tuning acquires new capabilities. Prior sparsity methods aim to preserve base performance under aggressive pruning [6, 43, 9]. Deletion Tuning instead uses sparsity as the learning mechanism itself. On pass@kk (Table 3), Deletion Tuning leads the base model at every kk. Even at k=128k=128, the base model’s pass@kk stays 4.3 and 6.1 points below Deletion Tuning’s on Llama8B and Qwen7B.

MethodLlama8B, GSM8K k=1k=12832128Qwen7B, SimpleRL-Zoo k=1k=12832128
Base53.2066.7583.6492.4095.2927.3538.7556.1866.2674.01
Deletion Tuning81.7588.5195.7598.4499.6249.3953.2264.6473.2380.08

Table 3. Deletion Tuning acquires new capabilities. Pass@kk (%), the share of problems solved within kk attempts, estimated from 128 samples per problem at temperature 1.0. A lead at large kk means solving problems the base model misses even with many attempts.

Smallest adapter at every scale. Deletion Tuning’s adapter is 0.5 MB at Qwen0.5B and 8.4 MB at Llama8B, 3,600 to 4,700× smaller than the FFT checkpoints and the smallest of any adapter family at every scale. The closest baselines by size are LoRA r=1r=1 and MTH at 99.95% sparsity. They are still often several times larger than Deletion Tuning, and MTH at 99% is more than 50× larger in every run. After compression, the mask takes about 0.03 bits per target weight, while every other method stores floating-point values for the weights it changes.

Learning the Mask Drives the Gains

Do Deletion Tuning’s gains over the base model come from learning the mask itself, beyond the Wanda prior that initializes the gates? We test this directly. For each run in Table 1, we apply Wanda one-shot pruning without GRPO training at 8 sparsity levels from 1/2 to 1/8192. We evaluate each level on the same validation set and record the best accuracy (Appendix § 11 reports every level). Table 4 compares this best-Wanda score with the base model and the trained Deletion Tuning mask.

MethodLlama, GSM8K 1BLlama, GSM8K 3BLlama, GSM8K 8BQwen, SimpleRL-Zoo 0.5BQwen, SimpleRL-Zoo 3BQwen, SimpleRL-Zoo 7B
Base1.2153.5360.2710.4437.7340.18
Wanda (best η\eta)1.5952.4660.6511.4238.7042.49
Deletion Tuning54.4481.4983.5517.6441.9852.32

Table 4. Wanda alone stays near the base model. Validation accuracy (%) of the base model, of Wanda one-shot pruning at the best of eight sparsity levels, and of the best checkpoint of the Deletion Tuning mask trained with GRPO.

The best Wanda mask stays within 2.4 points of the base on all six backbones, while Deletion Tuning, starting from the same Wanda scores, gains up to 53.2 points.

Deletion Tuning Forgets Less

Forward KL on the new task predicts how much a fine-tuned model forgets ([40]). We hypothesize that deletion keeps the perturbation to the base policy small, because each edit can only set a weight to zero and training chooses which few weights to edit, yielding low forward KL and therefore little forgetting. Following Shenfeld et al. [40]’s evaluation, Figure 2 tracks forward KL and prior-task accuracy over training for FFT, LoRA, MTH, and Deletion Tuning at the three Llama scales. The prior-task suite has six benchmarks: HellaSwag, MMLU, TruthfulQA, WinoGrande, IFEval, and HumanEval.

By step 348, FFT’s forward KL reaches 1.0 on Llama1B and 1.5 on Llama8B, and its prior-task accuracy ends 1.3 to 3.1 points below base. Deletion Tuning’s forward KL stays at or below 0.05 at all three scales, and its prior-task accuracy stays within one point of base throughout training. LoRA and MTH train only a small fraction of parameters, yet their forward KL reaches 0.4 to 1.3 and their prior-task accuracy falls up to 5.4 points below base. MTH at 99.95% sparsity trains 0.05% of all parameters, comparable to the 0.035% to 0.046% that Deletion Tuning deletes at these scales (Table 7), suggesting that the small number of edited weights does not explain Deletion Tuning’s retention, and that deletion itself keeps forgetting low.

Llama, GSM8KQwen, SimpleRL-Zoo
1B3B8B0.5B3B7B
Earliest saved0.0342%0.0420%0.0456%0.0393%0.0379%0.0381%
Final saved0.0347%0.0422%0.0459%0.0405%0.0418%0.0415%

Table 7. Fraction of all weights deleted by Deletion Tuning in the six GRPO runs. Fraction of all model parameters set to zero by the trained mask, at the earliest and final saved checkpoint. Every value is below 0.05%.

Why Deletion Tuning Works

How can binary masking alone (§ 5.2) match FFT under GRPO at scale? We compare the weight updates of Deletion Tuning and the three baselines at a saved checkpoint on three geometric measures.

Update geometry. Let W0W_0 denote the pretrained value of a weight matrix, W+W_+ its effective deployed value at the saved checkpoint (Llama8B step 348, Qwen7B step 100), and ΔW=W+−W0\Delta W = W_+ - W_0 the weight delta. For FFT, W+W_+ is the trained matrix, and for the adapters it is the merged or masked matrix that runs at inference. We report three quantities, each averaged over all attention and MLP linear layers:

  • Stable rank of ΔW\Delta W is ∥ΔW∥F2/∥ΔW∥22=∑m=1qσm2/σ12\lVert\Delta W\rVert_{F}^{2}/\lVert\Delta W\rVert_{2}^{2}=\sum_{m=1}^{q}\sigma_{m}^{2}/\sigma_{1}^{2}, where q=min⁡(dout,din)q=\min(d_{\mathrm{out}},d_{\mathrm{in}}) and σ1≥⋯≥σq\sigma_{1}\ge\cdots\ge\sigma_{q} are the singular values of ΔW\Delta W. It counts how many directions the update uses, independent of its scale: it is close to 1 when one direction dominates and close to kk when kk directions carry similar energy.

  • Spectrum drift is the normalized singular-value shift ∥σ(W+)−σ(W0)∥2/∥σ(W0)∥2\lVert\sigma(W_{+})-\sigma(W_{0})\rVert_{2}/\lVert\sigma(W_{0})\rVert_{2} [57], where σ(W)\sigma(W) lists the singular values of WW. It is small when training leaves the singular values close to where they started.

  • Principal-weight overlap measures the fraction of changed weight positions that fall on what Zhu et al. [57] call the base model’s principal weights. The principal-weight mask

Mprinc=Top⁡50%(∣W0(128)∣)M_{\mathrm{princ}}=\operatorname{Top}_{50\%}\left(\left|W_{0}^{(128)}\right|\right)

marks the top 50% entries by magnitude of W0(128)W_{0}^{(128)}, the best rank-128 approximation of W0W_{0}. The update mask MupdateM_{\mathrm{update}} marks positions where the deployed matrix differs from the base by

∣W+(i,j)−W0(i,j)∣>10−3⋅max⁡(∣W0(i,j)∣,∣W+(i,j)∣),\left|W_{+}(i,j)-W_{0}(i,j)\right|>10^{-3}\cdot\max\left(\left|W_{0}(i,j)\right|,\left|W_{+}(i,j)\right|\right),

the relative tolerance of Zhu et al. [57], which ignores changes too small to register in bf16. The overlap is ∣Mprinc∩Mupdate∣/∣Mupdate∣\lvert M_{\mathrm{princ}}\cap M_{\mathrm{update}}\rvert/\lvert M_{\mathrm{update}}\rvert. Update positions spread uniformly at random would score 0.5, the density of MprincM_{\mathrm{princ}}.

Figure 3 shows the three metrics for Deletion Tuning and every baseline setting, and Deletion Tuning is the closest to FFT on the three together. Its stable rank of 66 to 93 sits far below MTH’s 410 to 650 and above LoRA’s 1 to 2. Its spectrum drift stays within 1.4× of FFT’s, while LoRA r=1r=1 drifts up to 84× more than FFT on Llama8B. Its principal-weight overlap of 0.43 to 0.46 matches FFT’s sub-random 0.41 to 0.46. LoRA stays near the random baseline at 0.48 to 0.50, and MTH sits at 0.50.

Figure 3. Deletion Tuning (blue) tracks FFT (red) most closely across three update-geometry metrics. Values at the saved GRPO checkpoint for Llama8B (filled) and Qwen7B (open): (a) stable rank of ΔW\Delta W, (b) spectrum drift, and (c) principal-weight overlap. Overlap below the random baseline of 0.5 is the RLVR signature reported by Zhu et al. [57].

Interpretation. Low spectrum drift and sub-random principal-weight overlap are the RLVR signature reported by Zhu et al. [57]: RLVR leaves the singular values of each weight matrix nearly unchanged and steers its updates away from the principal weights. LoRA caps the rank of its update at rr, and MTH trains weights chosen at random before training. Deletion Tuning has neither restriction, since it learns which weights to change and has no rank cap. These two freedoms help explain how Deletion Tuning reaches FFT-like geometry even though it never sets a weight to a new non-zero value.

Generalizing Beyond GRPO: Results on SFT and SDFT

Does deletion also work when the training signal is denser or comes from outside the model? GRPO samples from the model itself and rewards whole responses, and RL updates of this kind are known to be sparse (Mukherjee et al. [32]). SFT instead imitates fixed external targets token by token and produces dense updates (Mukherjee et al. [32]), which makes it a harder test for an adapter that can only delete. SDFT sits in between: the student samples its own responses, and a teacher supplies a target distribution at every token. We test Deletion Tuning with SFT on knowledge and reasoning benchmarks (MMLU-Pro and BBEH) and with SDFT on tool use (ToolAlpaca) and science questions (Science), using the setups of § 5.1.

Under SFT, Deletion Tuning matches FFT at 3B on both MMLU-Pro and BBEH (Table 5). Under SDFT, Deletion Tuning matches FFT on both Qwen7B tasks (Table 6). In both regimes, its adapters are 600 to 4,700× smaller than FFT checkpoints.

MethodMMLU-Pro 1B Acc.MMLU-Pro 1B SizeMMLU-Pro 3B Acc.MMLU-Pro 3B SizeBBEH 1B Acc.BBEH 1B SizeBBEH 3B Acc.BBEH 3B Size
Base10.37012.6302.1708.870
FFT16.97471421.19122568.35471411.3912256
Deletion Tuning18.621.022.283.15.651.011.393.1

Table 5. Under SFT, Deletion Tuning matches FFT at 3B. Best-checkpoint evaluation accuracy (%), averaged over the 14 MMLU-Pro subjects and the 23 BBEH subtasks, and adapter size (MB) for Llama1B and Llama3B.

MethodToolAlpaca Acc.ToolAlpaca SizeScience Acc.Science Size
Base41.24032.350
FFT68.042905171.2029051
Deletion Tuning69.0727.170.6147.1

Table 6. Under SDFT, Deletion Tuning matches FFT on Qwen7B. Best-checkpoint exact-match accuracy (%) and adapter size (MB) on ToolAlpaca tool use and Science.

Deletion Tuning Is Robust to Hyperparameter Changes

Deletion Tuning works across a range of values for its hyperparameters: initial sparsity η\eta, learning rate lrl_r, and sparse-weight regularization strength λ\lambda. We sweep η∈{1/2048,1/4096,1/8192}\eta\in\{1/2048, 1/4096, 1/8192\}, lr∈{10−4,3×10−4}l_r \in\{10^{-4}, 3 \times10^{-4}\}, and λ∈{0,10−5}\lambda\in\{0, 10^{-5}\} on Llama8B GSM8K and Qwen7B SimpleRL-Zoo (Table 13 and Table 14 in Appendix § 13). Every setting lands within 1.6 points of the best run on Llama8B and within 3.2 points on Qwen7B, and the single configuration we use for all main runs is at or near the best setting at both backbones. Varying the initial sparsity η\eta by 4× changes accuracy by at most 0.8 points on Llama8B and 1.7 points on Qwen7B at fixed lrlr and λ\lambda, so the default configuration is a reasonable starting point for new tasks.

η\lr\eta\backslash lrλ=0\lambda=0, 10−410^{-4}λ=0\lambda=0, 3×10−43\times10^{-4}λ=10−5\lambda=10^{-5}, 10−410^{-4}λ=10−5\lambda=10^{-5}, 3×10−43\times10^{-4}
1/20481/204883.4082.4982.3483.55
1/40961/409683.0282.0383.0983.09
1/81921/819283.4082.4983.0983.02

Table 13. Table 13: Robustness to hyperparameters on Llama8B GSM8K. Best-checkpoint accuracy (%) of Deletion Tuning for each (η,λ,lr)(\eta,\lambda,lr).

η\lr\eta\backslash lrλ=0\lambda=0, 10−410^{-4}λ=0\lambda=0, 3×10−43\times10^{-4}λ=10−5\lambda=10^{-5}, 10−410^{-4}λ=10−5\lambda=10^{-5}, 3×10−43\times10^{-4}
1/20481/204851.7951.9249.6052.32
1/40961/409652.2352.8050.2951.91
1/81921/819251.1452.1051.2952.34

Table 14. Table 14: Robustness to hyperparameters on Qwen7B SimpleRL-Zoo. Best-checkpoint accuracy (%) of Deletion Tuning for each (η,λ,lr)(\eta,\lambda,lr).

Conclusion

We introduced Deletion Tuning, which fine-tunes a model by learning only a binary mask over its gate projections. At the largest scale we test in each regime (Qwen14B and the Qwen3-30B-A3B mixture-of-experts model under GRPO, Qwen7B under SDFT, and Llama3B under SFT), it matches full fine-tuning. The mask is smaller than a rank-1 LoRA and hundreds to thousands of times smaller than an FFT checkpoint. In our GRPO retention runs, Deletion Tuning keeps prior-task accuracy near the base level throughout training, while FFT, LoRA, and MTH lose up to 5.4 points. Its weight updates also resemble FFT’s more closely than those of LoRA or MTH across stable rank, spectrum drift, and principal-weight overlap, which helps explain how deletion alone can match FFT. These results suggest that post-training can work by selection alone, keeping nearly every pretrained weight and choosing the few to delete.

Limitations and Future Work. Because Deletion Tuning forgets little, a natural next test is continual learning over a sequence of tasks. Training keeps one logit per masked weight, so Deletion Tuning uses less training memory than FFT but more than LoRA. Restricting the logits to a small set of candidate weights could close this gap. The masks are also small and discrete, which aids interpretability: comparing which weights different tasks delete could localize the changes that post-training makes to a model. Masks for different tasks might also be combined, for example by taking their union, to merge skills without further training. Finally, restricting the mask to N:MN:M, channel-wise, or block patterns could let one mask both adapt and compress the model.

Acknowledgments

We thank members of the NYU Agentic Learning AI Lab and the Calgary ML Lab for helpful discussions. JL is supported by an NSERC PGS-D Scholarship. This work was supported in part by Visko Platform, Pionex, a Google TPU Award, a Google ML and Systems Junior Faculty Award, and the Institute of Information & Communications Technology Planning & Evaluation (IITP) with a grant funded by the Ministry of Science and ICT (MSIT) of the Republic of Korea in connection with the Global AI Frontier Lab International Collaborative Research Agreement (No. RS-2024-00469482 & RS-2024-00509279). Computational support was provided by NYU High Performance Computing through its resources, services, and staff expertise. This research was enabled in part by support provided by the Digital Research Alliance of Canada (alliancecan.ca).

ML, TP, and YI gratefully acknowledge the support of Alberta Innovates (ALLRP 577350-22, ALLRP 600038-24), the Natural Sciences and Engineering Research Council of Canada (NSERC) (RGPIN-2022-03120, DGECR-2022-00358), Defence Research and Development Canada (DGDND-2022-03120), and NSERC/Agence Nationale de la Recherche (ANR) (ALLRP 602719-24). This project was undertaken thanks to funding from IVADO and the Canada First Research Excellence Fund. ML additionally acknowledges the support of RBC Borealis through the RBC Borealis AI Global Fellowship Award.

Adapter Size

The Size entries in Table 1, Table 2, Table 5, and Table 6 (in binary units, 1 GB = 1024 MB) are lower bounds on the bytes on disk needed to rebuild the trained model from the public base checkpoint, computed the same way for every method. We leave out training overhead that deployment discards (file headers, the Adam moments mm and vv, and learning-rate scheduler and RNG state), which would otherwise dominate the smallest adapters.

Full fine-tuning (FFT). A fine-tuned LLM checkpoint at fp32:

size⁡(FFT)=Ntotal⋅4 bytes,\operatorname{size}(\mathrm{FFT}) = N_{\mathrm{total}} \cdot4\ \text{bytes},

where NtotalN_{\mathrm{total}} is the total parameter count. We use fp32 to match the trained master weights, which we store in fp32 for every adapter family.

LoRA. The BB and AA low-rank factor matrices, no bias terms:

size⁡(LoRA-r)=∑ℓ∈TLoRA(r⋅din(ℓ)+dout(ℓ)⋅r)⋅4 bytes.\operatorname{size}(\mathrm{LoRA}\text{-}r) = \sum_{\ell\in\mathcal{T}_{\mathrm{LoRA}}} \left(r \cdot d_{\mathrm{in}}^{(\ell)} + d_{\mathrm{out}}^{(\ell)} \cdot r\right) \cdot4\ \text{bytes}.

MTH. The active fp32 values plus a compressed boolean mask of their positions:

size⁡(MTH)=Nactive⋅4 bytes+zstd⁡(packbits⁡(1[active])),\operatorname{size}(\mathrm{MTH}) = N_{\mathrm{active}} \cdot4\ \text{bytes} + \operatorname{zstd}(\operatorname{packbits}(\mathbf{1}[\mathrm{active}])) ,

where Nactive=(1−s)⋅NtotalN_{\mathrm{active}} = (1-s) \cdot N_{\mathrm{total}} for sparsity s∈{0.99,0.9995}s \in\{0.99, 0.9995\}.

Deletion Tuning. A compressed binary mask over the target weights:

size⁡(Deletion Tuning)=zstd⁡(packbits⁡(1[G>0])).\operatorname{size}(\mathrm{Deletion\ Tuning}) = \operatorname{zstd}(\operatorname{packbits}(\mathbf{1}[G > 0])).

No floating-point values are stored: kept weights are read from the public checkpoint, and zeroed weights are zero. At the learned sparsity below 0.05% across the six GRPO runs in Table 1, zstd compresses to roughly 0.03 bits per masked weight.

Sparsity Learned by Deletion Tuning per Run

Table 7 reports the fraction of all model weights that Deletion Tuning deletes in the GRPO runs of Table 1 (η=1/2048\eta= 1/2048, λ=10−5\lambda= 10^{-5}, lr=3×10−4lr = 3 \times10^{-4}), at the earliest and final saved checkpoints. Every value stays below 0.05%.

Per-Benchmark Qwen2.5 Generalization Results

Table 8 reports the full per-benchmark breakdown that underlies the SimpleRL-Zoo (Qwen) accuracy columns of Table 1.

MethodGSM8KMATH500MinervaOlympiadBenchAIME24AMC23
Qwen0.5B
FFT48.0736.004.787.730.0017.50
LoRA r=149.2033.807.358.920.0022.50
LoRA r=452.4636.608.8210.553.3310.00
MTH 99%50.3435.2010.299.363.3322.50
MTH 99.95%49.7335.206.257.280.0022.50
Deletion Tuning44.1233.405.516.983.3312.50
Qwen3B
FFT83.6266.2023.1626.4516.6750.00
LoRA r=186.8169.8021.6931.356.6745.00
LoRA r=485.6067.0022.7928.6810.0047.50
MTH 99%86.2867.4022.4329.686.6752.50
MTH 99.95%85.3763.8022.7927.9313.3350.00
Deletion Tuning83.7866.2022.7925.7813.3340.00
Qwen7B
FFT91.4378.2033.4639.9213.3355.00
LoRA r=192.6577.6027.2139.5213.3362.50
LoRA r=491.6677.0029.0439.9720.0052.50
MTH 99%91.2177.2029.7839.8216.6757.50
MTH 99.95%92.0475.0031.6238.7820.0050.00
Deletion Tuning91.2878.8032.7240.2923.3347.50

Table 8. Table 8: Qwen2.5 on SimpleRL-Zoo: per-benchmark validation accuracy (%) at the best checkpoint. Six benchmarks: GSM8K, MATH500, Minerva Math, OlympiadBench, AIME24, AMC23.

SFT Data Splits

For both MMLU-Pro and BBEH, we split each task’s released set 75%/25% into train and held-out evaluation sets. Table 9 and Table 10 list per-task counts.

CategoryTrainTestCategoryTrainTest
biology537180law825276
business591198math1013338
chemistry849283other693231
computer science307103philosophy374125
economics633211physics974325
engineering726243psychology598200
health613205history28596

Table 9. Per-task train/test split for MMLU-Pro under the 75/25 split. Total: 9,018 train / 3,014 test across 14 tasks.

Task groupTrainTest
22 tasks (200 examples each)150 each50 each
disambiguation_qa (120 examples)9030

Table 10. Per-task train/test split for BBEH under the 75/25 split. Twenty-two tasks share the 200-example pattern. The only outlier is disambiguation_qa with 120 examples. Total: 3,390 train / 1,130 test across 23 tasks.

Full Wanda Sparsity Sweep

Tables 11 and 12 report the per-sparsity breakdown underlying § 5.4’s best-Wanda picks. η\eta is the fraction of gate-projection weights deleted (same convention as Eq. (8)), and the sweep runs from η=1/2\eta= 1/2 (half deleted) to η=1/8192\eta= 1/8192 (≈0.012%\approx0.012\% deleted). Wanda uses noise-free gate init (Gij=±0.03G_{ij} = \pm0.03, hard-mask sparsity exactly η\eta), while the trained Deletion Tuning runs of Table 1 use the noisy init of Eq. (8). Qwen values are the six-benchmark mean across GSM8K, MATH500, Minerva Math, OlympiadBench, AIME24, and AMC23. “Base” is the model before fine-tuning. Bold marks the best Wanda accuracy per backbone.

η\etaLlama1BLlama3BLlama8B
1/20.0048.0760.65
1/81.3651.7159.14
1/321.5251.5558.91
1/1281.5252.0859.67
1/5121.4452.1660.05
1/20481.5952.1659.74
1/40961.5951.6360.05
1/81921.3652.4659.67
Base1.2153.5360.27

Table 11. Wanda sweep on Llama (%).

η\etaQwen0.5BQwen3BQwen7B
1/21.9221.0639.39
1/810.4536.0942.49
1/3211.2438.2741.81
1/12811.2137.7141.90
1/51210.5637.2740.56
1/204811.4238.2440.14
1/409611.0938.6040.82
1/819210.7438.7041.43
Base10.4437.7340.18

Table 12. Wanda sweep on Qwen (%).

Algebraic Connection to MaskLLM

Deletion Tuning and MaskLLM (Fang et al., [6]) arrive at their forward passes from different starting points: Deletion Tuning from a continuous relaxation of a per-weight binary masking decision (Eq. (5)), and MaskLLM from a Gumbel-Softmax over N:MN:M candidate patterns (Eq. (9)). After three structural modifications to Eq. (9) (binary candidates, no per-group cardinality, no Gumbel noise), the modified MaskLLM forward matches Eq. (5) algebraically.

MaskLLM’s forward. The base weight matrix is partitioned into groups of MM contiguous weights, and each group is handled independently. Within a single group, MaskLLM enumerates C(M,N)=(MN)C(M,N)=\binom{M}{N} candidate binary patterns M^c\hat{M}_c with c∈{0,…,C(M,N)−1}c \in\{0,\ldots,C(M,N)-1\}, each keeping exactly NN of MM entries. It places a learnable logit vector π∈RC(M,N)\pi\in\mathbb{R}^{C(M,N)} over the candidates and draws a Gumbel-Softmax sample (Jang et al., [17]; Maddison et al., [28]):

y~c=exp⁡((κπc+gc)/τ)∑c′exp⁡((κπc′+gc′)/τ),gc∼iidGumbel⁡(0,1).(9)\tilde{y}_c=\frac{\exp((\kappa\pi_c+g_c)/\tau)}{\sum_{c'}\exp((\kappa\pi_{c'}+g_{c'})/\tau)},\qquad g_c\mathrel{\overset{\mathrm{iid}}{\sim}}\operatorname{Gumbel}(0,1). \tag*{(9)}

The group’s soft mask is the candidate-pattern-weighted average M~=∑cy~cM^c\tilde{M}=\sum_c\tilde{y}_c\hat{M}_c, and the layer forward applies W⊙M~W\odot\tilde{M} within the group. At evaluation, the hard mask is M^c⋆\hat{M}_{c^\star} with c⋆=arg⁡max⁡cπcc^\star=\arg\max_c\pi_c. The temperature τ\tau and logit-scale κ\kappa both follow linear schedules over training.

Step 1: C(M,N)=2C(M,N)=2 collapses the two-candidate softmax to a sigmoid. Set M=2M=2, N=1N=1, so each group has exactly two candidates: M^0=(1,0)\hat{M}_0=(1,0) keeps position 0 and M^1=(0,1)\hat{M}_1=(0,1) keeps position 1. With only two candidates, the Gumbel-Softmax probability on candidate 0 simplifies to a sigmoid of the logit difference:

y~0=e(κπ0+g0)/τe(κπ0+g0)/τ+e(κπ1+g1)/τ=11+e−1τ[κ(π0−π1)+(g0−g1)]=σ(1τ[κ(π0−π1)+(g0−g1)]),(10)\tilde{y}_0=\frac{e^{(\kappa\pi_0+g_0)/\tau}}{e^{(\kappa\pi_0+g_0)/\tau}+e^{(\kappa\pi_1+g_1)/\tau}}=\frac{1}{1+e^{-\frac{1}{\tau}[\kappa(\pi_0-\pi_1)+(g_0-g_1)]}}=\sigma\left(\frac{1}{\tau}[\kappa(\pi_0-\pi_1)+(g_0-g_1)]\right), \tag*{(10)}

where the second equality divides numerator and denominator by the first exponential, and the third uses σ(x)=1/(1+e−x)\sigma(x)=1/(1+e^{-x}). The difference of two independent Gumbel(0,1)(0,1) variables is Logistic(0,1)(0,1), so

y~0=σ(1τ[κΔπ+ξ]),Δπ:=π0−π1,ξ∼Logistic⁡(0,1).(11)\tilde{y}_0=\sigma\left(\frac{1}{\tau}[\kappa\Delta\pi+\xi]\right),\qquad\Delta\pi:=\pi_0-\pi_1,\quad\xi\sim\operatorname{Logistic}(0,1). \tag*{(11)}

Eq. (11) is the Binary-Concrete distribution (Maddison et al., [28]). Step 2: drop the per-group cardinality constraint. MaskLLM forces each group of MM weights to keep exactly NN of them. Letting each weight choose its own binary mask value replaces this group categorical with an independent binary masking choice per weight, parameterized by a single logit GijG_{ij}. Each weight then has its own Binary-Concrete (Eq. (11) applied per-weight, with GijG_{ij} in place of Δπ\Delta\pi):

M~ij=σ(1τ(κGij+ξij)),ξij∼Logistic⁡(0,1).(12)\widetilde{M}_{ij}=\sigma\left(\frac{1}{\tau}(\kappa G_{ij}+\xi_{ij})\right),\qquad\xi_{ij}\sim\operatorname{Logistic}(0,1). \tag*{(12)}

Step 3: drop the noise. Setting the noise ξij\xi_{ij} to zero in Step 2’s per-weight Binary-Concrete gives a deterministic soft mask:

M~ij=σ(κτGij).(13)\widetilde{M}_{ij}=\sigma\left(\frac{\kappa}{\tau}G_{ij}\right). \tag*{(13)}

This matches Eq. (5) with the scalar α(t)≔κ(t)/τ(t)\alpha(t)\coloneqq\kappa(t)/\tau(t); in our implementation, we schedule this scalar directly and linearly.

Deletion Tuning Hyperparameter Sweeps

Tables 13 and 14 give every setting of the sweep in § 5.8.

FFT Hyperparameter Sweeps

To make sure the FFT baseline is not under-tuned, we sweep its learning rate under GRPO and SFT and its KL coefficient under GRPO. All other settings follow the main runs. None of these settings changes the comparison with Deletion Tuning (Tables 15 and 16).

MethodLearning rateKL coefficientLlama8B, GSM8KQwen7B, SimpleRL-Zoo
FFT5×10−75\times10^{-7}083.2350.55
FFT5×10−75\times10^{-7}10−410^{-4}82.5551.89
FFT10−610^{-6}083.2451.93
FFT10−610^{-6}10−410^{-4}81.8950.79
Deletion Tuning3×10−43\times10^{-4}as in Table 183.5552.32

Table 15. Table 15: FFT sweep under GRPO. Best-checkpoint validation accuracy (%), as in Table 1.

MethodLearning rateAccuracy
Base–12.63
FFT10−510^{-5}19.35
FFT3×10−63 \times10^{-6}21.19
FFT10−610^{-6}18.35
Deletion Tuning5×10−45 \times10^{-4}22.28

Table 16. FFT learning-rate sweep under SFT, Llama3B MMLU-Pro. Best-checkpoint accuracy (%), as in Table 5.

References

  1. [1]Adewuyi, I., Okibe, S., and Ivanov, V. (2026). The multiple ticket hypothesis: Random sparse subnetworks suffice for RLVR. In ICML Workshop on High-dimensional Learning Dynamics.arxiv.org/abs/2602.01599
  2. [2]Agarwal, R., Vieillard, N., Zhou, Y., Stanczyk, P., Ramos, S., Geist, M., and Bachem, O. (2024). On-policy distillation of language models: Learning from self-generated mistakes. In ICLR.arxiv.org/abs/2306.13649
  3. [3]Bhardwaj, K., Pandey, N. P., Priyadarshi, S., Ganapathy, V., Kadambi, S., Esteves, R., Borse, S., Whatmough, P., Garrepalli, R., Baalen, M. V., Teague, H., and Nagel, M. (2024). Sparse high rank adapters. In NeurIPS.arxiv.org/abs/2406.13175
  4. [4]Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. (2021). Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.arxiv.org/abs/2110.14168
  5. [5]Collet, Y. and Kucherawy, M. (2021). Zstandard compression and the application/zstd media type. IETF RFC 8878.
  6. [6]Fang, G., Yin, H., Muralidharan, S., Heinrich, G., Pool, J., Kautz, J., Molchanov, P., and Wang, X. (2024). MaskLLM: Learnable semi-structured sparsity for large language models. In NeurIPS.arxiv.org/abs/2409.17481
  7. [7]Feng, K., Ding, K., Wang, W., Zhuang, X., Wang, Z., Qin, M., Zhao, Y., Yao, J., Zhang, Q., and Chen, H. (2024). SciKnowEval: Evaluating multi-level scientific knowledge of large language models. arXiv preprint arXiv:2406.09098v1.arxiv.org/abs/2406.09098
  8. [8]Frankle, J. and Carbin, M. (2019). The lottery ticket hypothesis: Finding sparse, trainable neural networks. In ICLR.arxiv.org/abs/1803.03635
  9. [9]Frantar, E. and Alistarh, D. (2023). SparseGPT: Massive language models can be accurately pruned in one-shot. In ICML.arxiv.org/abs/2301.00774
  10. [10]Gan, Y. and Isola, P. (2026). Neural tickets: Diverse task experts are dense around pretrained weights. In ICML.arxiv.org/abs/2603.12228
  11. [11]Guo, D., Rush, A. M., and Kim, Y. (2021). Parameter-efficient transfer learning with diff pruning. In ACL.arxiv.org/abs/2012.07463
  12. [12]Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., Bi, X., et al. (2025a). DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature.DOI
  13. [13]Guo, Z., Kamigaito, H., and Watanabe, T. (2025b). Dependency-aware semi-structured sparsity of GLU variants in large language models. Transactions on Machine Learning Research.arxiv.org/abs/2405.01943
  14. [14]Han, S., Pool, J., Tran, J., and Dally, W. J. (2015). Learning both weights and connections for efficient neural networks. In NeurIPS.arxiv.org/abs/1506.02626
  15. [15]Houlsby, N., Giurgiu, A., Jastrzębski, S., Morrone, B., de Laroussilhe, Q., Gesmundo, A., Attariyan, M., and Gelly, S. (2019). Parameter-efficient transfer learning for NLP. In ICML.arxiv.org/abs/1902.00751
  16. [17]Jang, E., Gu, S., and Poole, B. (2017). Categorical reparameterization with Gumbel-Softmax. In ICLR.arxiv.org/abs/1611.01144
  17. [18]Kazemi, M., Fatemi, B., Bansal, H., Palowitch, J., Anastasiou, C., Mehta, S. V., Jain, L. K., Aglietti, V., Jindal, D., Chen, P., Dikkala, N., Tyen, G., Liu, X., Shalit, U., Chiappa, S., Olszewska, K., Tay, Y., Tran, V. Q., Le, Q. V., and Firat, O. (2025). BIG-bench extra hard. In ACL.arxiv.org/abs/2502.19187
  18. [19]Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., Hassabis, D., Clopath, C., Kumaran, D., and Hadsell, R. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences.DOI
  19. [20]Lee, Y., Chen, A. S., Tajwar, F., Kumar, A., Yao, H., Liang, P., and Finn, C. (2023). Surgical fine-tuning improves adaptation to distribution shifts. In ICLR.arxiv.org/abs/2210.11466
  20. [21]Lester, B., Al-Rfou, R., and Constant, N. (2021). The power of scale for parameter-efficient prompt tuning. In EMNLP.arxiv.org/abs/2104.08691
  21. [22]Liu, Z., Pang, T., Balabanov, O., Yang, C., Huang, T., Yin, L., Yang, Y., and Liu, S. (2025). LIFT the veil for the truth: Principal weights emerge after rank reduction for reasoning-focused supervised fine-tuning. In ICML.arxiv.org/abs/2506.00772
  22. [23]Llama Team, AI @ Meta (2024). The Llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  23. [24]Louizos, C., Welling, M., and Kingma, D. P. (2018). Learning sparse neural networks without L₀ regularization. In ICLR.
  24. [28]Maddison, C. J., Mnih, A., and Teh, Y. W. (2017). The concrete distribution: A continuous relaxation of discrete random variables. In ICLR.arxiv.org/abs/1611.00712
  25. [29]Mallya, A., Davis, D., and Lazebnik, S. (2018). Piggyback: Adapting a single network to multiple tasks by learning to mask weights. In ECCV.arxiv.org/abs/1801.06519
  26. [30]Monjur, O., Nahin, S. K., and Chhabra, A. (2026). Revisiting the effectiveness of LLM pruning for test-time scaling. In EMNLP Findings.arxiv.org/abs/2604.25098
  27. [31]Morris, J. X., Mireshghallah, N., Ibrahim, M., and Mahloujifar, S. (2026). Learning to reason in 13 parameters. arXiv preprint arXiv:2602.04118.arxiv.org/abs/2602.04118
  28. [32]Mukherjee, S., Yuan, L., Hakkani-Tür, D., and Peng, H. (2025). Reinforcement learning finetunes small subnetworks in large language models. In NeurIPS.arxiv.org/abs/2505.11711
  29. [33]Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., and Lowe, R. (2022). Training language models to follow instructions with human feedback. In NeurIPS.arxiv.org/abs/2203.02155
  30. [34]Qwen Team (2024). Qwen2.5 technical report. arXiv preprint arXiv:2412.15115.
  31. [35]Radiya-Dixit, E. and Wang, X. (2020). How fine can fine-tuning be? Learning efficient language models. In AISTATS.
  32. [36]Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C. D., and Finn, C. (2023). Direct preference optimization: Your language model is secretly a reward model. In NeurIPS.arxiv.org/abs/2305.18290
  33. [37]Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.arxiv.org/abs/1707.06347
  34. [38]Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y. K., Wu, Y., and Guo, D. (2024). DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.arxiv.org/abs/2402.03300
  35. [39]Shenfeld, I., Damani, M., Hübötter, J., and Agrawal, P. (2026a). Self-distillation enables continual learning. In ICML.arxiv.org/abs/2601.19897
  36. [40]Shenfeld, I., Pari, J., and Agrawal, P. (2026b). RL’s razor: Why online reinforcement learning forgets less. In ICLR.arxiv.org/abs/2509.04259
  37. [41]Sheng, G., Zhang, C., Ye, Z., Wu, X., Zhang, W., Zhang, R., Peng, Y., Lin, H., and Wu, C. (2025). HybridFlow: A flexible and efficient RLHF framework. In EuroSys.DOI
  38. [42]Sprague, Z., Lu, J., Wadhwa, M., Keh, S., Ren, M., and Durrett, G. (2026). SkillFactory: Self-distillation for learning cognitive behaviors. In ICLR.arxiv.org/abs/2512.04072
  39. [43]Sun, M., Liu, Z., Bair, A., and Kolter, J. Z. (2024). A simple and effective pruning approach for large language models. In ICLR.arxiv.org/abs/2306.11695
  40. [44]Sun, Q., Cetin, E., and Tang, Y. (2025). Transformer-Squared: Self-adaptive LLMs. In ICLR.arxiv.org/abs/2501.06252
  41. [45]Sung, Y.-L., Nair, V., and Raffel, C. (2021). Training neural networks with fixed sparse masks. In NeurIPS.arxiv.org/abs/2111.09839
  42. [46]Tang, Q., Deng, Z., Lin, H., Han, X., Liang, Q., Cao, B., and Sun, L. (2023). ToolAlpaca: Generalized tool learning for language models with 3000 simulated cases. arXiv preprint arXiv:2306.05301.arxiv.org/abs/2306.05301
  43. [47]Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B. (2023). Alpaca: A strong, replicable instruction-following model. Stanford CRFM tech report.
  44. [48]Tian, J., Huang, C., and Kira, Z. (2024). Rethinking weight decay for robust fine-tuning of foundation models. In NeurIPS.arxiv.org/abs/2411.01713
  45. [49]Wang, Y., Ma, X., Zhang, G., Ni, Y., Chandra, A., Guo, S., Ren, W., Arulraj, A., He, X., Jiang, Z., Li, T., Ku, M., Wang, K., Zhuang, A., Fan, R., Yue, X., and Chen, W. (2024). MMLU-Pro: A more robust and challenging multi-task language understanding benchmark. In NeurIPS Datasets and Benchmarks.arxiv.org/abs/2406.01574
  46. [50]Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., and Le, Q. V. (2022). Finetuned language models are zero-shot learners. In ICLR.arxiv.org/abs/2109.01652
  47. [51]Zeng, W., Huang, Y., Liu, Q., Liu, W., He, K., Ma, Z., and He, J. (2025). SimpleRL-Zoo: Investigating and taming zero reinforcement learning for open base models in the wild. In COLM.arxiv.org/abs/2503.18892
  48. [52]Zhang, M., Bai, Y., Wang, H., Wang, Y., Dong, Q., Zhang, Y., and Fu, Y. (2025). Boosting large language models with mask fine-tuning. arXiv preprint arXiv:2503.22764.arxiv.org/abs/2503.22764
  49. [53]Zhao, M., Lin, T., Mi, F., Jaggi, M., and Schütze, H. (2020). Masking as an efficient alternative to finetuning for pretrained language models. In EMNLP.arxiv.org/abs/2004.12406
  50. [54]Zhao, S., Xie, Z., Liu, M., Huang, J., Pang, G., Chen, F., and Grover, A. (2026). Self-distilled reasoner: On-policy self-distillation for large language models. In ICML.arxiv.org/abs/2601.18734
  51. [55]Zhao, Y., Gu, A., Varma, R., Luo, L., Huang, C.-C., Xu, M., Wright, L., Shojanazeri, H., Ott, M., Shleifer, S., Desmaison, A., Balioglu, C., Damania, P., Nguyen, B., Chauhan, G., Hao, Y., Mathews, A., and Li, S. (2023). PyTorch FSDP: Experiences on scaling fully sharded data parallel. Proceedings of the VLDB Endowment.arxiv.org/abs/2304.11277
  52. [56]Zheng, K., Wu, W., Feng, R., Zhu, K., Liu, J., Zhao, D., Zha, Z.-J., Chen, W., and Shen, Y. (2023). Regularized mask tuning: Uncovering hidden knowledge in pre-trained vision-language models. In ICCV.arxiv.org/abs/2307.15049
  53. [57]Zhu, H., Zhang, Z., Huang, H., Su, D., Liu, Z., Zhao, J., Fedorov, I., Pirsiavash, H., Lee, J., Pan, D. Z., Wang, Z., Tian, Y., and Tai, K. S. (2026). The path not taken: RLVR learns off the principals. In NeurIPS.arxiv.org/abs/2511.08567
  54. [58]Zhu, M. H. and Gupta, S. (2018). To prune, or not to prune: Exploring the efficacy of pruning for model compression. In ICLR Workshop.arxiv.org/abs/1710.01878

Paper details

vlnc:W_hgpegsjd
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents