WorksComputer Science
The Surprising Effectiveness of Deleting Weights in LLM Post-Training
Abstract
How little of a pretrained large language model (LLM) has to change for it to acquire new capabilities? We find that zeroing fewer than 0.05% of its weights, with no other modification, matches full fine-tuning (FFT) at scale across three LLM post-training regimes: reinforcement learning with verifiable rewards, supervised fine-tuning, and on-policy distillation. Our method, Deletion Tuning, learns a binary mask over the gate projection of transformer blocks. Two practical advantages follow: the trained mask is hundreds to thousands of times smaller than the corresponding FFT checkpoint, and Deletion Tuning forgets far less, which addresses a central challenge in continual learning. Under reinforcement learning, its prior-task accuracy stays near the base level while FFT, LoRA, and random sparse fine-tuning lose up to 5.4 points. To understand why masking alone suffices, we analyze the geometry of the learned weight changes under reinforcement learning using stable rank, spectrum drift, and principal-weight overlap. Compared with LoRA and random sparse fine-tuning, Deletion Tuning is closest to FFT across the three measures.
https://agenticlearning.ai/deleting-weights/
Introduction
To acquire new capabilities during LLM post-training, full fine-tuning can alter every parameter, LoRA (Hu et al., 2022) learns a low-rank update, and the Multiple Ticket Hypothesis (MTH) (Adewuyi et al., [1]) and SHiRA (Bhardwaj et al., [3]) optimize values at selected weight positions. Weight deletion is typically used differently: pruning removes weights while preserving an already-trained model’s capabilities for efficient inference ([43, 9]), and the Lottery Ticket Hypothesis finds sparse subnetworks that can be trained to match dense networks (Frankle and Carbin, [8]). Unlike full fine-tuning, LoRA, and sparse updates, deletion writes no new values and only chooses which pretrained weights to keep. Recent work shows that fine-tuned solutions lie close to the pretrained model. RL fine-tuning changes only a small subnetwork (Mukherjee et al., [32]), and task experts lie densely around pretrained weights (Gan and Isola, [10]). We push this observation to its limit across reinforcement learning with verifiable rewards (RLVR), supervised fine-tuning (SFT), and on-policy distillation (OPD), and ask:
Can a pretrained LLM acquire new capabilities by learning only which weights to set to zero?
The answer is yes across all three post-training regimes. We propose Deletion Tuning, which learns only a binary mask over the gate-projection weights of transformer blocks. At deployment, each target weight either retains its pretrained value or is set to zero, so the learned binary mask is the entire adapter. We test Deletion Tuning with Group Relative Policy Optimization (GRPO) [38] for RLVR, standard SFT, and Self-Distillation Fine-Tuning (SDFT) [39] for OPD. Our experiments span two model families, eight backbones from 0.5B to 30B, and both in-distribution and out-of-distribution evaluation. At the largest model scale tested in each setting, Deletion Tuning matches full fine-tuning (FFT) across reasoning, domain knowledge, and tool-use tasks.
The binary-mask form brings two practical advantages: (i) the adapter is compact, with the trained mask hundreds to thousands of times smaller than the corresponding FFT checkpoint and smaller than even LoRA in our main GRPO comparison; and (ii) Deletion Tuning forgets less, with its forward KL remaining low and its prior-task accuracy staying near the base level across all three Llama scales. Under FFT, LoRA, and random sparse fine-tuning, forward KL grows and prior-task accuracy falls (Figure 2), consistent with forward KL as a predictor of forgetting [40]. Learning new tasks while keeping old ones makes Deletion Tuning a promising candidate for continual learning.
Figure 2. Deletion Tuning keeps prior-task accuracy at every Llama scale while the baselines drift. (a) Forward KL between the trained and base policies on GSM8K during GRPO training, the forgetting predictor of Shenfeld et al. [40]. (b) Change in mean accuracy on a six-task prior-task suite from the base model (dashed line).
Finally, we examine why Deletion Tuning can match FFT with deletion alone. We confirm that the gains come from learning the mask rather than the Wanda prior used in its initialization. We also find that the geometry of Deletion Tuning’s learned weight updates tracks FFT’s more closely than LoRA or random sparse fine-tuning across stable rank, spectrum drift, and principal-weight overlap.
Our contributions are as follows:
Deletion alone matches FFT. Deletion Tuning learns which gate-projection weights to delete (§ 4). It is on par with FFT at the largest scale we test in GRPO, SFT, and SDFT, up to a 30B mixture-of-experts model (§ 5.2 and § 5.7). Under GRPO, it also solves problems that the base model fails in all 128 sampled attempts (§ 5.2).
Deletion Tuning is compact and forgets less. Its masks are orders of magnitude smaller than FFT checkpoints in all three regimes (§ 5.2 and § 5.7), and it keeps prior-task accuracy throughout training (§ 5.5).
Deletion Tuning’s learned updates have FFT-like geometry. Its weight updates resemble FFT’s on three geometric measures (§ 5.6), which helps explain its accuracy.
Related Work
LLM post-training. We study three post-training regimes that differ in where the training signal comes from. Supervised fine-tuning (SFT) imitates fixed labeled targets [50, 33, 47]. Reinforcement learning optimizes a reward on the model’s own samples, from PPO-based RLHF [37, 33] to GRPO [38] with verifiable rewards, the recipe behind DeepSeek-R1 [12]. With verifiable rewards, rule-based checkers or LLM verifiers score each sample (Lu et al., 2026a). DPO [36] instead turns preferences into a closed-form loss. On-policy distillation trains a student on its own rollouts to match a teacher’s per-token distribution [2, 39, 54], and SDFT uses it to reduce catastrophic forgetting [19]. Self-distillation can also teach reasoning behaviors [42]. We use all three to test whether deletion works as a general post-training mechanism.
Sparsity and selective parameter updates. One-shot LLM pruning methods like Wanda [43] and SparseGPT [9] extend classical magnitude-based pruning [14, 58] to LLM scale, while continuous relaxations [24, 28, 17] make binary masks differentiable. Unstructured pruning can even retain the test-time scaling of reasoning models [30]. Earlier mask learning adapted vision backbones and encoder-only language models to downstream tasks such as image and sentence classification [29, 53, 35]. Recent mask methods target semi-structured LLM inference sparsity [6], refinement of a fine-tuned LLM [52], or vision-language backbones [56]. Deletion Tuning instead makes the mask the entire post-training update of a pretrained LLM across RLVR, SFT, and OPD, and it learns how many weights to delete. Sparse weight tuning writes float deltas at selected positions [11, 45, 1, 22, 3], additive adapters introduce trainable float parameters [31, 44, 15, 21] (Hu et al. 2022; Lu et al. 2026b), and robust fine-tuning methods restrict or regularize movement of pretrained weights under distribution shift [20, 48]. Deletion Tuning instead keeps every retained weight at its pretrained value and learns only which gate-projection weights to delete.
Figure 1. Figure 1: Deletion Tuning learns which pretrained weights to delete and writes no new values. The pretrained weights stay frozen, and each is paired with a trainable logit in , the only parameters that post-training updates. During training, each weight is multiplied by , a smooth mask between 0 and 1. The sharpness rises over training until the sigmoid becomes a hard threshold at . At deployment, weights with positive logits (blue) keep their pretrained values and weights with negative logits (red) are set to zero (crossed out), so the trained adapter is one bit per weight.
Understanding LLM fine-tuning. The classical Lottery Ticket Hypothesis [8] posits that randomly initialized networks contain sparse subnetworks that train to comparable accuracy in isolation. Recent work characterizes pretrained-model fine-tuning along three axes. Density: task-expert solutions densely populate the neighborhood of pretrained weights [10]. Geometry: Mukherjee et al. [32] report that RL fine-tuning naturally updates only 5 to 30% of parameters across PPO, GRPO, DPO, and several model families. Zhu et al. [57] show these updates land off the principal weights of while leaving the spectrum largely intact, where principal weights are the top-magnitude entries of the rank- SVD reconstruction of [22]. Forgetting: Shenfeld et al. [40] establish that the forward KL between fine-tuned and base policies on the new task accurately predicts forgetting. Deletion Tuning reproduces the off-principal update pattern and low spectrum drift. Deletion Tuning’s low forward KL also predicts the strong prior-task retention we measure (§ 5.5).
Background
Our experiments use three post-training objectives.
Supervised fine-tuning (SFT). Given a dataset of prompts and target responses of length , SFT minimizes the negative log-likelihood of the target tokens, which are fixed in advance and never sampled from the model:
Group Relative Policy Optimization (GRPO). For RLVR, we use GRPO [38]. For each prompt , the policy before the update, , samples a group of responses , and a verifier assigns each one a reward . Rather than learning a value function as in PPO [37], GRPO compares each response with the rest of its group, , and gives this advantage to every token of in PPO’s clipped objective,
where is the importance ratio for token . GRPO optionally adds a KL penalty to the reference model, weighted by .
Self-Distillation Fine-Tuning (SDFT). For on-policy distillation, we use SDFT [39], in which the model is its own teacher. The teacher’s weights are an exponential moving average of the student’s weights ,
with , and the teacher’s prompt also includes , a reference response to the same question, which the student never sees. For a response sampled by the student, the loss is the forward KL between the teacher’s and the student’s next-token distributions at each position:
Deletion Tuning
Deletion Tuning parameterizes fine-tuning as one trainable scalar logit per pretrained weight at a chosen subset of linear layers, with the base model frozen. During training, a sigmoid of each logit is annealed toward a step function, and at deployment each logit becomes a single bit deciding whether the corresponding weight is kept or deleted (set to zero). In pruning terms, Deletion Tuning performs learned unstructured pruning to fine-tune the model.
Notation. Let denote a target weight matrix in the pretrained model and a trainable logit matrix of the same shape, where the sign of at deployment decides whether is kept (positive) or zeroed (negative). We mask one such per target layer, each with its own independent . In all our experiments, the target is the gate projection of transformer blocks (a single MLP weight per block). We follow GLU-specific pruning methods by treating the gate/up/down pathway as the main coupled sparsification structure [13] (Ma et al. 2023): in the Llama and Qwen2.5 backbones we evaluate, SwiGLU MLPs take the form , so the gate projection multiplicatively controls the intermediate channels routed through the down projection.
Forward pass. At training step (with the total step count), the layer applies a sigmoid relaxation of the binary masking decision per weight:
We call the soft mask and the hard mask. The sharpness sets how steep the sigmoid is around . A smaller widens the band of logits whose soft mask lies strictly between 0 and 1, and raising pulls the soft mask onto the hard one by the end. We use a linear ramp from to , which we call mask sharpening:
At evaluation, the layer applies the hard mask:
Gradients are computed through the deterministic soft mask in Eq. (5), while the hard mask is used for evaluation and for generating rollouts in on-policy runs. By the end of training, the two agree: deploying the hard mask instead of the soft one changes accuracy by at most 0.08 points.
Wanda prior initialization. We initialize from the Wanda one-shot pruning score Sun et al., [43], an importance signal validated as a strong predictor of which base weights are safe to zero. The Wanda score of a weight is its magnitude times the norm of its input activation, , where collects the activations of input channel in one forward pass over calibration prompts from the task’s training distribution. Specifically, we sample each gate logit from one of two Gaussians depending on whether the weight is above or below the -quantile of :
At our default , used in every main experiment, this zeros approximately of gate-projection weights under the hard mask at initialization, where is the standard normal CDF.
Training objective. We instantiate the policy of § 3 by replacing with and keeping pretrained weights elsewhere, so the gate logits are the only trainable parameters in the model. The training loss is the regime’s own objective from Eqs. (1) to (4), plus the MaskLLM sparse weight regularizer , which mildly biases the mask toward retaining higher-magnitude weights. We do not constrain to any fixed sparsity level, leaving the training process to decide it.
SFT. The loss is , with .
GRPO. The loss is , with . Policy rollouts use the hard mask of Eq. (7), while gradients flow into through the soft mask of Eq. (5).
SDFT. The loss is , with , since the EMA teacher already provides an anchor. Rollouts use the hard mask as in GRPO, and the EMA teacher of Eq. (3) specializes to a teacher gate , so the teacher’s frozen base weights stay the same as the student’s.
Compact storage and ease of deployment. The trained adapter is the hard mask at each masked layer, which takes one bit per masked pretrained weight before compression. We bit-pack the booleans with NumPy and compress the bit stream with zstd Collet and Kucherawy, [5]. Because fewer than 0.05% of all weights are deleted in every GRPO run of Table 1, the bit stream is highly redundant, and zstd compresses it to 1.0 MB at Llama1B and 8.4 MB at Llama8B. This gives Deletion Tuning a practical deployment advantage in compact storage and cheap adapter switching: changing tasks only loads a compact bitmask and scatters zeros into the base weights, rather than loading a separate full model checkpoint.
| Method | Acc. | Size | Acc. | Size | Acc. | Size | Acc. | Size | Acc. | Size | Acc. | Size |
| Llama, GSM8K | Qwen, SimpleRL-Zoo | |||||||||||
| 1B | 3B | 8B | 0.5B | 3B | 7B | |||||||
| Base | 1.21 | 0 | 53.53 | 0 | 60.27 | 0 | 10.44 | 0 | 37.73 | 0 | 40.18 | 0 |
| FFT | 60.50 | 4714 | 82.49 | 12256 | 83.24 | 30633 | 19.01 | 1885 | 44.35 | 11772 | 51.89 | 29051 |
| LoRA r=1 | 59.29 | 2.7 | 82.39 | 5.8 | 83.38 | 10.0 | 20.30 | 2.1 | 43.55 | 7.1 | 52.13 | 9.6 |
| LoRA r=4 | 60.58 | 10.8 | 82.44 | 23.2 | 82.87 | 40.0 | 20.30 | 8.4 | 43.59 | 28.6 | 51.70 | 38.5 |
| MTH 99% | 60.73 | 68.2 | 80.53 | 177.4 | 83.31 | 443.6 | 21.84 | 27.3 | 44.16 | 170.4 | 52.03 | 420.7 |
| MTH 99.95% | 58.61 | 4.1 | 82.17 | 10.7 | 83.07 | 26.6 | 20.16 | 1.7 | 43.87 | 13.7 | 51.24 | 25.3 |
| Deletion Tuning | 54.44 | 1.0 | 81.49 | 3.1 | 83.55 | 8.4 | 17.64 | 0.5 | 41.98 | 2.7 | 52.32 | 6.9 |
Table 1. Deletion Tuning has the smallest adapter at every scale and matches FFT at the largest backbone of each family. Best-checkpoint validation accuracy (%) and adapter size (MB) under GRPO, on GSM8K for Llama and on the six-benchmark SimpleRL-Zoo mean for Qwen. The baselines are full fine-tuning (FFT), low-rank adapters (LoRA), and random sparse fine-tuning (MTH).
Connection to MaskLLM. MaskLLM [6] learns masks for inference speedup with a Gumbel-Softmax selector over candidate patterns. Deletion Tuning has a different goal, expanding model capability, and a different form, with one deterministic logit per weight and no fixed sparsity pattern (Appendix § 12 relates the two formally). Deletion Tuning also adopts MaskLLM’s Wanda prior initialization (Eq. (8)) and its sparse weight regularizer.
Experiments
Experimental Setup
Tasks, benchmarks, and models. We study three post-training regimes on two model families, referring to a family as Llama or Qwen and to a size by its suffix (e.g., Llama1B, Qwen7B). Evaluation uses greedy decoding, and we report the best validation accuracy across saved checkpoints.
GRPO, in distribution. We train and evaluate on GSM8K [4] with Llama1B, Llama3B, and Llama8B (Llama3.2-1B/3B-Instruct and Llama3-8B-Instruct) [23], validating every 25 optimizer steps.
GRPO, out of distribution. Following SimpleRL-Zoo [51], we train on its math corpus and report the mean over six held-out benchmarks (GSM8K, MATH500, Minerva Math, OlympiadBench, AIME24, AMC23; per-benchmark results in Appendix § 9.1) with Qwen0.5B, Qwen3B, and Qwen7B (Qwen2.5 base) [34]. At larger scale, we add Qwen14B (Qwen2.5-14B) and the Qwen3-30B-A3B mixture-of-experts model (30B total and 3B active parameters). We validate every 10 optimizer steps.
SFT. We train Llama1B and Llama3B (Llama3.2-1B/3B-Instruct) separately on each of the 14 MMLU-Pro [49] subjects and 23 BBEH [18] subtasks, validate at the end of training, and average accuracy within each suite.
SDFT. We follow the ToolAlpaca [46] tool-use and Science settings of Shenfeld et al. [39] with Qwen7B (Qwen2.5-7B-Instruct), reporting exact-match accuracy under each task’s evaluator. Science is the Chemistry L-3 subset of SciKnowEval [7]. We validate every 100 optimizer steps on ToolAlpaca and every 25 on Science.
Training pipelines. All runs use AdamW with fp32 master weights, bf16 mixed precision, and gradient checkpointing.
GRPO. We use verl [41] with a constant learning rate. GSM8K uses prompt batch 64, rollouts, response length 4096, 3 epochs, and no KL penalty, while SimpleRL-Zoo uses prompt batch 256, rollouts, response length 4096, 150 steps, and KL penalty .
SFT. We minimize cross-entropy with a constant learning rate for steps per task at effective batch 4, with a 75/25 train/eval split (per-task counts in Appendix § 10).
SDFT. We follow Shenfeld et al. [39]: cosine schedule with 0.1 warmup ratio, prompt batch 32, 2 epochs, EMA mixup , and one demonstration prepended to the teacher’s input.
Baselines. All three baselines use the same data and number of steps as Deletion Tuning.
FFT. Full fine-tuning trains all weights and is the reference we aim to match. Its learning rate is on GSM8K, on SimpleRL-Zoo, to on SFT (Appendix § 14), and on SDFT as in Shenfeld et al. [39].
LoRA. We train rank-1 and rank-4 adapters (Hu et al., 2022) on all seven projections in each block (q/k/v/o and gate/up/down) with learning rate .
MTH. The closest sparse-update method (Adewuyi et al., [1]) trains a random 1% or 0.05% of all parameters (99% and 99.95% sparsity), with the learning rates recommended by Adewuyi et al. [1] ( and ).
Deletion Tuning implementation. We compute the Wanda scores once per task, on 2048 training prompts for GRPO and SFT and on 32 for SDFT.
GRPO. Every main run uses , learning rate , , and , with no tuning per benchmark or model size (§ 5.8 varies , learning rate, and ).
SFT. We keep the same and , with learning rate and for every subject and subtask.
SDFT. We set and and use a hard teacher mask, with learning rate and on ToolAlpaca and and on Science.
Hardware. We train on H200 GPUs, using one GPU for backbones up to 3B and multiple GPUs with FSDP (Zhao et al., [55]) for larger models. SFT and the post-hoc analyses use L40S GPUs.
Results on GRPO Post-Training
We train Deletion Tuning and three baselines (FFT, LoRA at ranks 1 and 4, MTH at 99% and 99.95%) on the two GRPO settings of § 5.1. Table 1 reports best-checkpoint validation accuracy and bytes-on-disk adapter size for six backbones from 0.5B to 8B across two model families. Every Deletion Tuning run deletes fewer than 0.05% of all weights (Appendix § 9). Appendix § 8 defines adapter size per method.
Deletion Tuning matches FFT at 7B and 8B. At the largest backbone in each family, Deletion Tuning scores 83.55% against FFT’s 83.24% on Llama8B GSM8K and 52.32% against 51.89% on Qwen7B SimpleRL-Zoo. Tuning FFT’s learning rate and KL coefficient does not change this (Appendix § 14). At these two
Learning the Mask Drives the Gains
backbones, Deletion Tuning is also competitive with both LoRA ranks and both MTH sparsity levels (Table 1). Since Deletion Tuning can only delete pretrained weights, this restriction may act as a mild regularizer, which could explain why Deletion Tuning matches FFT here. The same restriction pays off more clearly for retention, where Deletion Tuning keeps the prior-task accuracy that FFT loses (§ 5.5).
The gap to FFT closes with scale. At smaller backbones, Deletion Tuning recovers 90% to 99% of FFT’s accuracy. At larger scale, Deletion Tuning matches FFT on Qwen14B and on the Qwen3-30B-A3B mixture-of-experts model (Table 2). For the mixture-of-experts model, we mask the gate projection inside each of its 128 experts in every layer and leave the router unchanged. These results suggest that deletion becomes more effective as models grow.
| Method | Llama8B Acc. | Llama8B Size | Qwen7B Acc. | Qwen7B Size | Qwen14B Acc. | Qwen14B Size | Qwen3-30B-A3B Acc. | Qwen3-30B-A3B Size |
| Base | 60.27 | 0 | 40.18 | 0 | 44.37 | 0 | 49.41 | 0 |
| FFT | 83.24 | 30633 | 51.89 | 29051 | 54.87 | 56343 | 58.43 | 116471 |
| Deletion Tuning | 83.55 | 8.4 | 52.32 | 6.9 | 55.43 | 11.4 | 58.54 | 29.6 |
Table 2. Deletion Tuning matches FFT on dense and mixture-of-experts models from 7B to 30B with an adapter thousands of times smaller. Best-checkpoint validation accuracy (%) and adapter size (MB) under GRPO. Qwen14B and Qwen3-30B-A3B use the SimpleRL-Zoo setup of Qwen7B.
Deletion Tuning acquires new capabilities. Prior sparsity methods aim to preserve base performance under aggressive pruning [6, 43, 9]. Deletion Tuning instead uses sparsity as the learning mechanism itself. On pass@ (Table 3), Deletion Tuning leads the base model at every . Even at , the base model’s pass@ stays 4.3 and 6.1 points below Deletion Tuning’s on Llama8B and Qwen7B.
| Method | Llama8B, GSM8K | 2 | 8 | 32 | 128 | Qwen7B, SimpleRL-Zoo | 2 | 8 | 32 | 128 |
| Base | 53.20 | 66.75 | 83.64 | 92.40 | 95.29 | 27.35 | 38.75 | 56.18 | 66.26 | 74.01 |
| Deletion Tuning | 81.75 | 88.51 | 95.75 | 98.44 | 99.62 | 49.39 | 53.22 | 64.64 | 73.23 | 80.08 |
Table 3. Deletion Tuning acquires new capabilities. Pass@ (%), the share of problems solved within attempts, estimated from 128 samples per problem at temperature 1.0. A lead at large means solving problems the base model misses even with many attempts.
Smallest adapter at every scale. Deletion Tuning’s adapter is 0.5 MB at Qwen0.5B and 8.4 MB at Llama8B, 3,600 to 4,700× smaller than the FFT checkpoints and the smallest of any adapter family at every scale. The closest baselines by size are LoRA and MTH at 99.95% sparsity. They are still often several times larger than Deletion Tuning, and MTH at 99% is more than 50× larger in every run. After compression, the mask takes about 0.03 bits per target weight, while every other method stores floating-point values for the weights it changes.
Learning the Mask Drives the Gains
Do Deletion Tuning’s gains over the base model come from learning the mask itself, beyond the Wanda prior that initializes the gates? We test this directly. For each run in Table 1, we apply Wanda one-shot pruning without GRPO training at 8 sparsity levels from 1/2 to 1/8192. We evaluate each level on the same validation set and record the best accuracy (Appendix § 11 reports every level). Table 4 compares this best-Wanda score with the base model and the trained Deletion Tuning mask.
| Method | Llama, GSM8K 1B | Llama, GSM8K 3B | Llama, GSM8K 8B | Qwen, SimpleRL-Zoo 0.5B | Qwen, SimpleRL-Zoo 3B | Qwen, SimpleRL-Zoo 7B |
| Base | 1.21 | 53.53 | 60.27 | 10.44 | 37.73 | 40.18 |
| Wanda (best ) | 1.59 | 52.46 | 60.65 | 11.42 | 38.70 | 42.49 |
| Deletion Tuning | 54.44 | 81.49 | 83.55 | 17.64 | 41.98 | 52.32 |
Table 4. Wanda alone stays near the base model. Validation accuracy (%) of the base model, of Wanda one-shot pruning at the best of eight sparsity levels, and of the best checkpoint of the Deletion Tuning mask trained with GRPO.
The best Wanda mask stays within 2.4 points of the base on all six backbones, while Deletion Tuning, starting from the same Wanda scores, gains up to 53.2 points.
Deletion Tuning Forgets Less
Forward KL on the new task predicts how much a fine-tuned model forgets ([40]). We hypothesize that deletion keeps the perturbation to the base policy small, because each edit can only set a weight to zero and training chooses which few weights to edit, yielding low forward KL and therefore little forgetting. Following Shenfeld et al. [40]’s evaluation, Figure 2 tracks forward KL and prior-task accuracy over training for FFT, LoRA, MTH, and Deletion Tuning at the three Llama scales. The prior-task suite has six benchmarks: HellaSwag, MMLU, TruthfulQA, WinoGrande, IFEval, and HumanEval.
By step 348, FFT’s forward KL reaches 1.0 on Llama1B and 1.5 on Llama8B, and its prior-task accuracy ends 1.3 to 3.1 points below base. Deletion Tuning’s forward KL stays at or below 0.05 at all three scales, and its prior-task accuracy stays within one point of base throughout training. LoRA and MTH train only a small fraction of parameters, yet their forward KL reaches 0.4 to 1.3 and their prior-task accuracy falls up to 5.4 points below base. MTH at 99.95% sparsity trains 0.05% of all parameters, comparable to the 0.035% to 0.046% that Deletion Tuning deletes at these scales (Table 7), suggesting that the small number of edited weights does not explain Deletion Tuning’s retention, and that deletion itself keeps forgetting low.
| Llama, GSM8K | Qwen, SimpleRL-Zoo | |||||
| 1B | 3B | 8B | 0.5B | 3B | 7B | |
| Earliest saved | 0.0342% | 0.0420% | 0.0456% | 0.0393% | 0.0379% | 0.0381% |
| Final saved | 0.0347% | 0.0422% | 0.0459% | 0.0405% | 0.0418% | 0.0415% |
Table 7. Fraction of all weights deleted by Deletion Tuning in the six GRPO runs. Fraction of all model parameters set to zero by the trained mask, at the earliest and final saved checkpoint. Every value is below 0.05%.
Why Deletion Tuning Works
How can binary masking alone (§ 5.2) match FFT under GRPO at scale? We compare the weight updates of Deletion Tuning and the three baselines at a saved checkpoint on three geometric measures.
Update geometry. Let denote the pretrained value of a weight matrix, its effective deployed value at the saved checkpoint (Llama8B step 348, Qwen7B step 100), and the weight delta. For FFT, is the trained matrix, and for the adapters it is the merged or masked matrix that runs at inference. We report three quantities, each averaged over all attention and MLP linear layers:
Stable rank of is , where and are the singular values of . It counts how many directions the update uses, independent of its scale: it is close to 1 when one direction dominates and close to when directions carry similar energy.
Spectrum drift is the normalized singular-value shift [57], where lists the singular values of . It is small when training leaves the singular values close to where they started.
Principal-weight overlap measures the fraction of changed weight positions that fall on what Zhu et al. [57] call the base model’s principal weights. The principal-weight mask
marks the top 50% entries by magnitude of , the best rank-128 approximation of . The update mask marks positions where the deployed matrix differs from the base by
the relative tolerance of Zhu et al. [57], which ignores changes too small to register in bf16. The overlap is . Update positions spread uniformly at random would score 0.5, the density of .
Figure 3 shows the three metrics for Deletion Tuning and every baseline setting, and Deletion Tuning is the closest to FFT on the three together. Its stable rank of 66 to 93 sits far below MTH’s 410 to 650 and above LoRA’s 1 to 2. Its spectrum drift stays within 1.4× of FFT’s, while LoRA drifts up to 84× more than FFT on Llama8B. Its principal-weight overlap of 0.43 to 0.46 matches FFT’s sub-random 0.41 to 0.46. LoRA stays near the random baseline at 0.48 to 0.50, and MTH sits at 0.50.
Figure 3. Deletion Tuning (blue) tracks FFT (red) most closely across three update-geometry metrics. Values at the saved GRPO checkpoint for Llama8B (filled) and Qwen7B (open): (a) stable rank of , (b) spectrum drift, and (c) principal-weight overlap. Overlap below the random baseline of 0.5 is the RLVR signature reported by Zhu et al. [57].
Interpretation. Low spectrum drift and sub-random principal-weight overlap are the RLVR signature reported by Zhu et al. [57]: RLVR leaves the singular values of each weight matrix nearly unchanged and steers its updates away from the principal weights. LoRA caps the rank of its update at , and MTH trains weights chosen at random before training. Deletion Tuning has neither restriction, since it learns which weights to change and has no rank cap. These two freedoms help explain how Deletion Tuning reaches FFT-like geometry even though it never sets a weight to a new non-zero value.
Generalizing Beyond GRPO: Results on SFT and SDFT
Does deletion also work when the training signal is denser or comes from outside the model? GRPO samples from the model itself and rewards whole responses, and RL updates of this kind are known to be sparse (Mukherjee et al. [32]). SFT instead imitates fixed external targets token by token and produces dense updates (Mukherjee et al. [32]), which makes it a harder test for an adapter that can only delete. SDFT sits in between: the student samples its own responses, and a teacher supplies a target distribution at every token. We test Deletion Tuning with SFT on knowledge and reasoning benchmarks (MMLU-Pro and BBEH) and with SDFT on tool use (ToolAlpaca) and science questions (Science), using the setups of § 5.1.
Under SFT, Deletion Tuning matches FFT at 3B on both MMLU-Pro and BBEH (Table 5). Under SDFT, Deletion Tuning matches FFT on both Qwen7B tasks (Table 6). In both regimes, its adapters are 600 to 4,700× smaller than FFT checkpoints.
| Method | MMLU-Pro 1B Acc. | MMLU-Pro 1B Size | MMLU-Pro 3B Acc. | MMLU-Pro 3B Size | BBEH 1B Acc. | BBEH 1B Size | BBEH 3B Acc. | BBEH 3B Size |
| Base | 10.37 | 0 | 12.63 | 0 | 2.17 | 0 | 8.87 | 0 |
| FFT | 16.97 | 4714 | 21.19 | 12256 | 8.35 | 4714 | 11.39 | 12256 |
| Deletion Tuning | 18.62 | 1.0 | 22.28 | 3.1 | 5.65 | 1.0 | 11.39 | 3.1 |
Table 5. Under SFT, Deletion Tuning matches FFT at 3B. Best-checkpoint evaluation accuracy (%), averaged over the 14 MMLU-Pro subjects and the 23 BBEH subtasks, and adapter size (MB) for Llama1B and Llama3B.
| Method | ToolAlpaca Acc. | ToolAlpaca Size | Science Acc. | Science Size |
| Base | 41.24 | 0 | 32.35 | 0 |
| FFT | 68.04 | 29051 | 71.20 | 29051 |
| Deletion Tuning | 69.07 | 27.1 | 70.61 | 47.1 |
Table 6. Under SDFT, Deletion Tuning matches FFT on Qwen7B. Best-checkpoint exact-match accuracy (%) and adapter size (MB) on ToolAlpaca tool use and Science.
Deletion Tuning Is Robust to Hyperparameter Changes
Deletion Tuning works across a range of values for its hyperparameters: initial sparsity , learning rate , and sparse-weight regularization strength . We sweep , , and on Llama8B GSM8K and Qwen7B SimpleRL-Zoo (Table 13 and Table 14 in Appendix § 13). Every setting lands within 1.6 points of the best run on Llama8B and within 3.2 points on Qwen7B, and the single configuration we use for all main runs is at or near the best setting at both backbones. Varying the initial sparsity by 4× changes accuracy by at most 0.8 points on Llama8B and 1.7 points on Qwen7B at fixed and , so the default configuration is a reasonable starting point for new tasks.
| , | , | , | , | |
| 83.40 | 82.49 | 82.34 | 83.55 | |
| 83.02 | 82.03 | 83.09 | 83.09 | |
| 83.40 | 82.49 | 83.09 | 83.02 |
Table 13. Table 13: Robustness to hyperparameters on Llama8B GSM8K. Best-checkpoint accuracy (%) of Deletion Tuning for each .
| , | , | , | , | |
| 51.79 | 51.92 | 49.60 | 52.32 | |
| 52.23 | 52.80 | 50.29 | 51.91 | |
| 51.14 | 52.10 | 51.29 | 52.34 |
Table 14. Table 14: Robustness to hyperparameters on Qwen7B SimpleRL-Zoo. Best-checkpoint accuracy (%) of Deletion Tuning for each .
Conclusion
We introduced Deletion Tuning, which fine-tunes a model by learning only a binary mask over its gate projections. At the largest scale we test in each regime (Qwen14B and the Qwen3-30B-A3B mixture-of-experts model under GRPO, Qwen7B under SDFT, and Llama3B under SFT), it matches full fine-tuning. The mask is smaller than a rank-1 LoRA and hundreds to thousands of times smaller than an FFT checkpoint. In our GRPO retention runs, Deletion Tuning keeps prior-task accuracy near the base level throughout training, while FFT, LoRA, and MTH lose up to 5.4 points. Its weight updates also resemble FFT’s more closely than those of LoRA or MTH across stable rank, spectrum drift, and principal-weight overlap, which helps explain how deletion alone can match FFT. These results suggest that post-training can work by selection alone, keeping nearly every pretrained weight and choosing the few to delete.
Limitations and Future Work. Because Deletion Tuning forgets little, a natural next test is continual learning over a sequence of tasks. Training keeps one logit per masked weight, so Deletion Tuning uses less training memory than FFT but more than LoRA. Restricting the logits to a small set of candidate weights could close this gap. The masks are also small and discrete, which aids interpretability: comparing which weights different tasks delete could localize the changes that post-training makes to a model. Masks for different tasks might also be combined, for example by taking their union, to merge skills without further training. Finally, restricting the mask to , channel-wise, or block patterns could let one mask both adapt and compress the model.
Acknowledgments
We thank members of the NYU Agentic Learning AI Lab and the Calgary ML Lab for helpful discussions. JL is supported by an NSERC PGS-D Scholarship. This work was supported in part by Visko Platform, Pionex, a Google TPU Award, a Google ML and Systems Junior Faculty Award, and the Institute of Information & Communications Technology Planning & Evaluation (IITP) with a grant funded by the Ministry of Science and ICT (MSIT) of the Republic of Korea in connection with the Global AI Frontier Lab International Collaborative Research Agreement (No. RS-2024-00469482 & RS-2024-00509279). Computational support was provided by NYU High Performance Computing through its resources, services, and staff expertise. This research was enabled in part by support provided by the Digital Research Alliance of Canada (alliancecan.ca).
ML, TP, and YI gratefully acknowledge the support of Alberta Innovates (ALLRP 577350-22, ALLRP 600038-24), the Natural Sciences and Engineering Research Council of Canada (NSERC) (RGPIN-2022-03120, DGECR-2022-00358), Defence Research and Development Canada (DGDND-2022-03120), and NSERC/Agence Nationale de la Recherche (ANR) (ALLRP 602719-24). This project was undertaken thanks to funding from IVADO and the Canada First Research Excellence Fund. ML additionally acknowledges the support of RBC Borealis through the RBC Borealis AI Global Fellowship Award.
Adapter Size
The Size entries in Table 1, Table 2, Table 5, and Table 6 (in binary units, 1 GB = 1024 MB) are lower bounds on the bytes on disk needed to rebuild the trained model from the public base checkpoint, computed the same way for every method. We leave out training overhead that deployment discards (file headers, the Adam moments and , and learning-rate scheduler and RNG state), which would otherwise dominate the smallest adapters.
Full fine-tuning (FFT). A fine-tuned LLM checkpoint at fp32:
where is the total parameter count. We use fp32 to match the trained master weights, which we store in fp32 for every adapter family.
LoRA. The and low-rank factor matrices, no bias terms:
MTH. The active fp32 values plus a compressed boolean mask of their positions:
where for sparsity .
Deletion Tuning. A compressed binary mask over the target weights:
No floating-point values are stored: kept weights are read from the public checkpoint, and zeroed weights are zero. At the learned sparsity below 0.05% across the six GRPO runs in Table 1, zstd compresses to roughly 0.03 bits per masked weight.
Sparsity Learned by Deletion Tuning per Run
Table 7 reports the fraction of all model weights that Deletion Tuning deletes in the GRPO runs of Table 1 (, , ), at the earliest and final saved checkpoints. Every value stays below 0.05%.
Per-Benchmark Qwen2.5 Generalization Results
Table 8 reports the full per-benchmark breakdown that underlies the SimpleRL-Zoo (Qwen) accuracy columns of Table 1.
| Method | GSM8K | MATH500 | Minerva | OlympiadBench | AIME24 | AMC23 |
| Qwen0.5B | ||||||
| FFT | 48.07 | 36.00 | 4.78 | 7.73 | 0.00 | 17.50 |
| LoRA r=1 | 49.20 | 33.80 | 7.35 | 8.92 | 0.00 | 22.50 |
| LoRA r=4 | 52.46 | 36.60 | 8.82 | 10.55 | 3.33 | 10.00 |
| MTH 99% | 50.34 | 35.20 | 10.29 | 9.36 | 3.33 | 22.50 |
| MTH 99.95% | 49.73 | 35.20 | 6.25 | 7.28 | 0.00 | 22.50 |
| Deletion Tuning | 44.12 | 33.40 | 5.51 | 6.98 | 3.33 | 12.50 |
| Qwen3B | ||||||
| FFT | 83.62 | 66.20 | 23.16 | 26.45 | 16.67 | 50.00 |
| LoRA r=1 | 86.81 | 69.80 | 21.69 | 31.35 | 6.67 | 45.00 |
| LoRA r=4 | 85.60 | 67.00 | 22.79 | 28.68 | 10.00 | 47.50 |
| MTH 99% | 86.28 | 67.40 | 22.43 | 29.68 | 6.67 | 52.50 |
| MTH 99.95% | 85.37 | 63.80 | 22.79 | 27.93 | 13.33 | 50.00 |
| Deletion Tuning | 83.78 | 66.20 | 22.79 | 25.78 | 13.33 | 40.00 |
| Qwen7B | ||||||
| FFT | 91.43 | 78.20 | 33.46 | 39.92 | 13.33 | 55.00 |
| LoRA r=1 | 92.65 | 77.60 | 27.21 | 39.52 | 13.33 | 62.50 |
| LoRA r=4 | 91.66 | 77.00 | 29.04 | 39.97 | 20.00 | 52.50 |
| MTH 99% | 91.21 | 77.20 | 29.78 | 39.82 | 16.67 | 57.50 |
| MTH 99.95% | 92.04 | 75.00 | 31.62 | 38.78 | 20.00 | 50.00 |
| Deletion Tuning | 91.28 | 78.80 | 32.72 | 40.29 | 23.33 | 47.50 |
Table 8. Table 8: Qwen2.5 on SimpleRL-Zoo: per-benchmark validation accuracy (%) at the best checkpoint. Six benchmarks: GSM8K, MATH500, Minerva Math, OlympiadBench, AIME24, AMC23.
SFT Data Splits
For both MMLU-Pro and BBEH, we split each task’s released set 75%/25% into train and held-out evaluation sets. Table 9 and Table 10 list per-task counts.
| Category | Train | Test | Category | Train | Test |
| biology | 537 | 180 | law | 825 | 276 |
| business | 591 | 198 | math | 1013 | 338 |
| chemistry | 849 | 283 | other | 693 | 231 |
| computer science | 307 | 103 | philosophy | 374 | 125 |
| economics | 633 | 211 | physics | 974 | 325 |
| engineering | 726 | 243 | psychology | 598 | 200 |
| health | 613 | 205 | history | 285 | 96 |
Table 9. Per-task train/test split for MMLU-Pro under the 75/25 split. Total: 9,018 train / 3,014 test across 14 tasks.
| Task group | Train | Test |
| 22 tasks (200 examples each) | 150 each | 50 each |
disambiguation_qa (120 examples) | 90 | 30 |
Table 10. Per-task train/test split for BBEH under the 75/25 split. Twenty-two tasks share the 200-example pattern. The only outlier is disambiguation_qa with 120 examples. Total: 3,390 train / 1,130 test across 23 tasks.
Full Wanda Sparsity Sweep
Tables 11 and 12 report the per-sparsity breakdown underlying § 5.4’s best-Wanda picks. is the fraction of gate-projection weights deleted (same convention as Eq. (8)), and the sweep runs from (half deleted) to ( deleted). Wanda uses noise-free gate init (, hard-mask sparsity exactly ), while the trained Deletion Tuning runs of Table 1 use the noisy init of Eq. (8). Qwen values are the six-benchmark mean across GSM8K, MATH500, Minerva Math, OlympiadBench, AIME24, and AMC23. “Base” is the model before fine-tuning. Bold marks the best Wanda accuracy per backbone.
| Llama1B | Llama3B | Llama8B | |
| 1/2 | 0.00 | 48.07 | 60.65 |
| 1/8 | 1.36 | 51.71 | 59.14 |
| 1/32 | 1.52 | 51.55 | 58.91 |
| 1/128 | 1.52 | 52.08 | 59.67 |
| 1/512 | 1.44 | 52.16 | 60.05 |
| 1/2048 | 1.59 | 52.16 | 59.74 |
| 1/4096 | 1.59 | 51.63 | 60.05 |
| 1/8192 | 1.36 | 52.46 | 59.67 |
| Base | 1.21 | 53.53 | 60.27 |
Table 11. Wanda sweep on Llama (%).
| Qwen0.5B | Qwen3B | Qwen7B | |
| 1/2 | 1.92 | 21.06 | 39.39 |
| 1/8 | 10.45 | 36.09 | 42.49 |
| 1/32 | 11.24 | 38.27 | 41.81 |
| 1/128 | 11.21 | 37.71 | 41.90 |
| 1/512 | 10.56 | 37.27 | 40.56 |
| 1/2048 | 11.42 | 38.24 | 40.14 |
| 1/4096 | 11.09 | 38.60 | 40.82 |
| 1/8192 | 10.74 | 38.70 | 41.43 |
| Base | 10.44 | 37.73 | 40.18 |
Table 12. Wanda sweep on Qwen (%).
Algebraic Connection to MaskLLM
Deletion Tuning and MaskLLM (Fang et al., [6]) arrive at their forward passes from different starting points: Deletion Tuning from a continuous relaxation of a per-weight binary masking decision (Eq. (5)), and MaskLLM from a Gumbel-Softmax over candidate patterns (Eq. (9)). After three structural modifications to Eq. (9) (binary candidates, no per-group cardinality, no Gumbel noise), the modified MaskLLM forward matches Eq. (5) algebraically.
MaskLLM’s forward. The base weight matrix is partitioned into groups of contiguous weights, and each group is handled independently. Within a single group, MaskLLM enumerates candidate binary patterns with , each keeping exactly of entries. It places a learnable logit vector over the candidates and draws a Gumbel-Softmax sample (Jang et al., [17]; Maddison et al., [28]):
The group’s soft mask is the candidate-pattern-weighted average , and the layer forward applies within the group. At evaluation, the hard mask is with . The temperature and logit-scale both follow linear schedules over training.
Step 1: collapses the two-candidate softmax to a sigmoid. Set , , so each group has exactly two candidates: keeps position 0 and keeps position 1. With only two candidates, the Gumbel-Softmax probability on candidate 0 simplifies to a sigmoid of the logit difference:
where the second equality divides numerator and denominator by the first exponential, and the third uses . The difference of two independent Gumbel variables is Logistic, so
Eq. (11) is the Binary-Concrete distribution (Maddison et al., [28]). Step 2: drop the per-group cardinality constraint. MaskLLM forces each group of weights to keep exactly of them. Letting each weight choose its own binary mask value replaces this group categorical with an independent binary masking choice per weight, parameterized by a single logit . Each weight then has its own Binary-Concrete (Eq. (11) applied per-weight, with in place of ):
Step 3: drop the noise. Setting the noise to zero in Step 2’s per-weight Binary-Concrete gives a deterministic soft mask:
This matches Eq. (5) with the scalar ; in our implementation, we schedule this scalar directly and linearly.
Deletion Tuning Hyperparameter Sweeps
Tables 13 and 14 give every setting of the sweep in § 5.8.
FFT Hyperparameter Sweeps
To make sure the FFT baseline is not under-tuned, we sweep its learning rate under GRPO and SFT and its KL coefficient under GRPO. All other settings follow the main runs. None of these settings changes the comparison with Deletion Tuning (Tables 15 and 16).
| Method | Learning rate | KL coefficient | Llama8B, GSM8K | Qwen7B, SimpleRL-Zoo |
| FFT | 0 | 83.23 | 50.55 | |
| FFT | 82.55 | 51.89 | ||
| FFT | 0 | 83.24 | 51.93 | |
| FFT | 81.89 | 50.79 | ||
| Deletion Tuning | as in Table 1 | 83.55 | 52.32 |
Table 15. Table 15: FFT sweep under GRPO. Best-checkpoint validation accuracy (%), as in Table 1.
| Method | Learning rate | Accuracy |
| Base | – | 12.63 |
| FFT | 19.35 | |
| FFT | 21.19 | |
| FFT | 18.35 | |
| Deletion Tuning | 22.28 |
Table 16. FFT learning-rate sweep under SFT, Llama3B MMLU-Pro. Best-checkpoint accuracy (%), as in Table 5.
References
- [1]Adewuyi, I., Okibe, S., and Ivanov, V. (2026). The multiple ticket hypothesis: Random sparse subnetworks suffice for RLVR. In ICML Workshop on High-dimensional Learning Dynamics.arxiv.org/abs/2602.01599
- [2]Agarwal, R., Vieillard, N., Zhou, Y., Stanczyk, P., Ramos, S., Geist, M., and Bachem, O. (2024). On-policy distillation of language models: Learning from self-generated mistakes. In ICLR.arxiv.org/abs/2306.13649
- [3]Bhardwaj, K., Pandey, N. P., Priyadarshi, S., Ganapathy, V., Kadambi, S., Esteves, R., Borse, S., Whatmough, P., Garrepalli, R., Baalen, M. V., Teague, H., and Nagel, M. (2024). Sparse high rank adapters. In NeurIPS.arxiv.org/abs/2406.13175
- [4]Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. (2021). Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.arxiv.org/abs/2110.14168
- [5]Collet, Y. and Kucherawy, M. (2021). Zstandard compression and the application/zstd media type. IETF RFC 8878.
- [6]Fang, G., Yin, H., Muralidharan, S., Heinrich, G., Pool, J., Kautz, J., Molchanov, P., and Wang, X. (2024). MaskLLM: Learnable semi-structured sparsity for large language models. In NeurIPS.arxiv.org/abs/2409.17481
- [7]Feng, K., Ding, K., Wang, W., Zhuang, X., Wang, Z., Qin, M., Zhao, Y., Yao, J., Zhang, Q., and Chen, H. (2024). SciKnowEval: Evaluating multi-level scientific knowledge of large language models. arXiv preprint arXiv:2406.09098v1.arxiv.org/abs/2406.09098
- [8]Frankle, J. and Carbin, M. (2019). The lottery ticket hypothesis: Finding sparse, trainable neural networks. In ICLR.arxiv.org/abs/1803.03635
- [9]Frantar, E. and Alistarh, D. (2023). SparseGPT: Massive language models can be accurately pruned in one-shot. In ICML.arxiv.org/abs/2301.00774
- [10]Gan, Y. and Isola, P. (2026). Neural tickets: Diverse task experts are dense around pretrained weights. In ICML.arxiv.org/abs/2603.12228
- [11]Guo, D., Rush, A. M., and Kim, Y. (2021). Parameter-efficient transfer learning with diff pruning. In ACL.arxiv.org/abs/2012.07463
- [12]Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., Bi, X., et al. (2025a). DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature.DOI
- [13]Guo, Z., Kamigaito, H., and Watanabe, T. (2025b). Dependency-aware semi-structured sparsity of GLU variants in large language models. Transactions on Machine Learning Research.arxiv.org/abs/2405.01943
- [14]Han, S., Pool, J., Tran, J., and Dally, W. J. (2015). Learning both weights and connections for efficient neural networks. In NeurIPS.arxiv.org/abs/1506.02626
- [15]Houlsby, N., Giurgiu, A., Jastrzębski, S., Morrone, B., de Laroussilhe, Q., Gesmundo, A., Attariyan, M., and Gelly, S. (2019). Parameter-efficient transfer learning for NLP. In ICML.arxiv.org/abs/1902.00751
- [17]Jang, E., Gu, S., and Poole, B. (2017). Categorical reparameterization with Gumbel-Softmax. In ICLR.arxiv.org/abs/1611.01144
- [18]Kazemi, M., Fatemi, B., Bansal, H., Palowitch, J., Anastasiou, C., Mehta, S. V., Jain, L. K., Aglietti, V., Jindal, D., Chen, P., Dikkala, N., Tyen, G., Liu, X., Shalit, U., Chiappa, S., Olszewska, K., Tay, Y., Tran, V. Q., Le, Q. V., and Firat, O. (2025). BIG-bench extra hard. In ACL.arxiv.org/abs/2502.19187
- [19]Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., Hassabis, D., Clopath, C., Kumaran, D., and Hadsell, R. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences.DOI
- [20]Lee, Y., Chen, A. S., Tajwar, F., Kumar, A., Yao, H., Liang, P., and Finn, C. (2023). Surgical fine-tuning improves adaptation to distribution shifts. In ICLR.arxiv.org/abs/2210.11466
- [21]Lester, B., Al-Rfou, R., and Constant, N. (2021). The power of scale for parameter-efficient prompt tuning. In EMNLP.arxiv.org/abs/2104.08691
- [22]Liu, Z., Pang, T., Balabanov, O., Yang, C., Huang, T., Yin, L., Yang, Y., and Liu, S. (2025). LIFT the veil for the truth: Principal weights emerge after rank reduction for reasoning-focused supervised fine-tuning. In ICML.arxiv.org/abs/2506.00772
- [23]Llama Team, AI @ Meta (2024). The Llama 3 herd of models. arXiv preprint arXiv:2407.21783.
- [24]Louizos, C., Welling, M., and Kingma, D. P. (2018). Learning sparse neural networks without L₀ regularization. In ICLR.
- [28]Maddison, C. J., Mnih, A., and Teh, Y. W. (2017). The concrete distribution: A continuous relaxation of discrete random variables. In ICLR.arxiv.org/abs/1611.00712
- [29]Mallya, A., Davis, D., and Lazebnik, S. (2018). Piggyback: Adapting a single network to multiple tasks by learning to mask weights. In ECCV.arxiv.org/abs/1801.06519
- [30]Monjur, O., Nahin, S. K., and Chhabra, A. (2026). Revisiting the effectiveness of LLM pruning for test-time scaling. In EMNLP Findings.arxiv.org/abs/2604.25098
- [31]Morris, J. X., Mireshghallah, N., Ibrahim, M., and Mahloujifar, S. (2026). Learning to reason in 13 parameters. arXiv preprint arXiv:2602.04118.arxiv.org/abs/2602.04118
- [32]Mukherjee, S., Yuan, L., Hakkani-Tür, D., and Peng, H. (2025). Reinforcement learning finetunes small subnetworks in large language models. In NeurIPS.arxiv.org/abs/2505.11711
- [33]Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., and Lowe, R. (2022). Training language models to follow instructions with human feedback. In NeurIPS.arxiv.org/abs/2203.02155
- [34]Qwen Team (2024). Qwen2.5 technical report. arXiv preprint arXiv:2412.15115.
- [35]Radiya-Dixit, E. and Wang, X. (2020). How fine can fine-tuning be? Learning efficient language models. In AISTATS.
- [36]Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C. D., and Finn, C. (2023). Direct preference optimization: Your language model is secretly a reward model. In NeurIPS.arxiv.org/abs/2305.18290
- [37]Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.arxiv.org/abs/1707.06347
- [38]Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y. K., Wu, Y., and Guo, D. (2024). DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.arxiv.org/abs/2402.03300
- [39]Shenfeld, I., Damani, M., Hübötter, J., and Agrawal, P. (2026a). Self-distillation enables continual learning. In ICML.arxiv.org/abs/2601.19897
- [40]Shenfeld, I., Pari, J., and Agrawal, P. (2026b). RL’s razor: Why online reinforcement learning forgets less. In ICLR.arxiv.org/abs/2509.04259
- [41]Sheng, G., Zhang, C., Ye, Z., Wu, X., Zhang, W., Zhang, R., Peng, Y., Lin, H., and Wu, C. (2025). HybridFlow: A flexible and efficient RLHF framework. In EuroSys.DOI
- [42]Sprague, Z., Lu, J., Wadhwa, M., Keh, S., Ren, M., and Durrett, G. (2026). SkillFactory: Self-distillation for learning cognitive behaviors. In ICLR.arxiv.org/abs/2512.04072
- [43]Sun, M., Liu, Z., Bair, A., and Kolter, J. Z. (2024). A simple and effective pruning approach for large language models. In ICLR.arxiv.org/abs/2306.11695
- [44]Sun, Q., Cetin, E., and Tang, Y. (2025). Transformer-Squared: Self-adaptive LLMs. In ICLR.arxiv.org/abs/2501.06252
- [45]Sung, Y.-L., Nair, V., and Raffel, C. (2021). Training neural networks with fixed sparse masks. In NeurIPS.arxiv.org/abs/2111.09839
- [46]Tang, Q., Deng, Z., Lin, H., Han, X., Liang, Q., Cao, B., and Sun, L. (2023). ToolAlpaca: Generalized tool learning for language models with 3000 simulated cases. arXiv preprint arXiv:2306.05301.arxiv.org/abs/2306.05301
- [47]Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B. (2023). Alpaca: A strong, replicable instruction-following model. Stanford CRFM tech report.
- [48]Tian, J., Huang, C., and Kira, Z. (2024). Rethinking weight decay for robust fine-tuning of foundation models. In NeurIPS.arxiv.org/abs/2411.01713
- [49]Wang, Y., Ma, X., Zhang, G., Ni, Y., Chandra, A., Guo, S., Ren, W., Arulraj, A., He, X., Jiang, Z., Li, T., Ku, M., Wang, K., Zhuang, A., Fan, R., Yue, X., and Chen, W. (2024). MMLU-Pro: A more robust and challenging multi-task language understanding benchmark. In NeurIPS Datasets and Benchmarks.arxiv.org/abs/2406.01574
- [50]Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., and Le, Q. V. (2022). Finetuned language models are zero-shot learners. In ICLR.arxiv.org/abs/2109.01652
- [51]Zeng, W., Huang, Y., Liu, Q., Liu, W., He, K., Ma, Z., and He, J. (2025). SimpleRL-Zoo: Investigating and taming zero reinforcement learning for open base models in the wild. In COLM.arxiv.org/abs/2503.18892
- [52]Zhang, M., Bai, Y., Wang, H., Wang, Y., Dong, Q., Zhang, Y., and Fu, Y. (2025). Boosting large language models with mask fine-tuning. arXiv preprint arXiv:2503.22764.arxiv.org/abs/2503.22764
- [53]Zhao, M., Lin, T., Mi, F., Jaggi, M., and Schütze, H. (2020). Masking as an efficient alternative to finetuning for pretrained language models. In EMNLP.arxiv.org/abs/2004.12406
- [54]Zhao, S., Xie, Z., Liu, M., Huang, J., Pang, G., Chen, F., and Grover, A. (2026). Self-distilled reasoner: On-policy self-distillation for large language models. In ICML.arxiv.org/abs/2601.18734
- [55]Zhao, Y., Gu, A., Varma, R., Luo, L., Huang, C.-C., Xu, M., Wright, L., Shojanazeri, H., Ott, M., Shleifer, S., Desmaison, A., Balioglu, C., Damania, P., Nguyen, B., Chauhan, G., Hao, Y., Mathews, A., and Li, S. (2023). PyTorch FSDP: Experiences on scaling fully sharded data parallel. Proceedings of the VLDB Endowment.arxiv.org/abs/2304.11277
- [56]Zheng, K., Wu, W., Feng, R., Zhu, K., Liu, J., Zhao, D., Zha, Z.-J., Chen, W., and Shen, Y. (2023). Regularized mask tuning: Uncovering hidden knowledge in pre-trained vision-language models. In ICCV.arxiv.org/abs/2307.15049
- [57]Zhu, H., Zhang, Z., Huang, H., Su, D., Liu, Z., Zhao, J., Fedorov, I., Pirsiavash, H., Lee, J., Pan, D. Z., Wang, Z., Tian, Y., and Tai, K. S. (2026). The path not taken: RLVR learns off the principals. In NeurIPS.arxiv.org/abs/2511.08567
- [58]Zhu, M. H. and Gupta, S. (2018). To prune, or not to prune: Exploring the efficacy of pruning for model compression. In ICLR Workshop.arxiv.org/abs/1710.01878


