betaFeedback

WorksComputer Science

LEAN: Loss-Engineered Modulation of the Model Weight-Sensitivity Distribution for Efficient, Error-Tolerant Neural Networks

Abstract

The weights of a trained network do not have uniform sensitivity to model accuracy: the 1% most sensitive weights carry about a fifth of the damage a bit flip can do. This distribution decides the bit-flip tolerance of a model, the fraction of weights a code must cover and the bits each weight needs; existing schemes work with the distribution training leaves. We show that we can systematically modulate or alter the weight sensitivity distribution of a trained model, thereby shifting weights from one sensitivity bin to another. The sensitivity of a stored weight is the product of its gradient and the step a flip takes in the storage format, and LEAN adds a composite regularizer on both factors, a range term that bounds the flip step and a tail term that lowers the gradient of the most sensitive weights, with one coefficient λ setting how far the distribution moves. Ten epochs with the term, as a fine-tune or as the last epochs of training, shrink the sensitive tail 5 to 13 times on nine vision models (ResNet-18, ResNet-50, VGG-16 on CIFAR-10, CIFAR-100, ImageNet) within 0.5 points of accuracy; real flips confirm that the reshaped weights do less damage, a plain gradient-norm penalty does not reproduce the effect, and a preliminary extension to three small language models shows the same tail shaping. With a BCH code over 1% of the weights the model then survives 300 progressive bit-search flips in every training seed and 500 in two of three, where three of four published defenses with the same code fail by 300 and WCAT falls to 75% at 500. The reshaped distribution also cuts the parity for a fixed guarantee 3.7 times and makes a sensitivity-ordered 8/6/4/3-bit format affordable, which stores ImageNet ResNet-50 at a 3.29-bit weight payload, 2.4 times smaller than INT8, above the accuracy of LSQ at 3 bits.

Effect of LEAN on ResNet-18, CIFAR-10

Figure 1. Effect of LEAN on ResNet-18, CIFAR-10. (a) The raw weight-sensitivity distribution: number of weights per sensitivity bin (40 log-spaced bins of the sign-flip damage did_i, as a percentage of the baseline maximum); the bulk of the weights per sensitivity between 0.01 and 1% and the HIGH band is a thin tail, which LEAN moves to the left. (b) The cumulative distribution, counted from the tail: the number of weights with damage at least a threshold ss; one ten-epoch fine-tune leaves 9 times fewer weights above the same HIGH threshold at the same accuracy, while Hessian-aware training leaves the tail where normal training left it (0.1% against 1.00% above the threshold). (c) Accuracy under progressive bit search when the same 1% BCH code protects each model: the normally trained and the Hessian-aware model fall to chance by 300 flips, the LEAN model (λ=40\lambda= 40) keeps 94% after 500 in two of three training seeds and after 300 in all (Table V).

Introduction

Modern neural networks have tens of millions to billions of weights and they do not impact model functional behavior uniformly. If a single stored weight is corrupted, by a memory fault, a bit-flip attack or a coarse quantization step, the loss change depends on which weight it is, and it differs by a factor of 10,000 or more between the median weight and the most sensitive one. On ResNet-18 trained on CIFAR-10, the 1% of weights with the largest first-order sensitivity account for 19.5% of the total loss change under sign-bit flips, the next 8% for most of the rest, and the remaining 91% contribute little under this metric (Figure 1a,b). Hence, weight sensitivity is

strongly non-uniform, and its distribution has a large spread between the highest and the lowest sensitivity.

The shape of that distribution influences the cost of the stored model in terms of error-tolerance and computation efficiency. It sets bit-flip tolerance, because Rowhammer [1], [2], progressive bit search [3], its targeted variant [4] and low-voltage SRAM errors [5] all do their damage through the tail (the attack surface of AI hardware is surveyed in [6]), and an adaptive attacker only needs to hit it. It sets the price of protection, because a selective code [7] has to cover the tail and its parity grows with the tail’s size, while a code over everything costs 12 to 20% of memory. And it sets the price of storage, because a variable-precision format must keep the tail at full width and can be cheap only on the rest. A heavier tail means more weights to protect, more parity, more bits and a shorter survival; a lighter one means the opposite. Robust training [8, 9, 10, 11] changes the distribution as a side effect, and selective protection and mixed precision live with whatever distribution they are given; none of them optimizes the size of the tail, measured in the format the weights are stored in.

This paper treats the sensitivity distribution as something to be chosen. LEAN is a training-time procedure that reshapes it. The damage of a flip is the product of two factors, the gradient of the weight and the step the flip takes in the storage format, and LEAN regularizes both: a range term bounds the step of every layer, and a tail term, the mean sensitivity of the top 1% of weights differentiated through the gradient, lowers the gradient where it is largest. A coefficient λ\lambda sets how far the tail is pushed (Figure 2). The change is to the whole distribution, not to a handful of weights, and it costs essentially no accuracy. The paper includes two checks: real flips confirm that the reshaped weights do less damage, not only that their gradients read smaller, and an ablation separates the two terms. At INT8 the range term does most of the work; the tail term adds to it but cannot replace it, and alone it lowers the gradient without lowering the damage. Figure 1 previews the effect on one model: the tail shrinks by an order of magnitude, and under the same 1% code the model survives an attack that reduces the normally trained one to chance.

Overview of LEAN training and its three deployment payoffs

Figure 2. Overview. Step 1: a normally trained network has a heavy-tailed weight-sensitivity distribution; the top 1% of weights (HIGH) carry about 20% of the first-order damage of sign-bit flips. Step 2: ten epochs with the extra term λPLEAN\lambda P_{\mathrm{LEAN}}, as a fine-tune or as the last epochs of training, move weights out of HIGH and MID into LOW; λ\lambda sets how far, and accuracy changes by at most 0.5 points. Step 3: the deployed model needs 3.7 times less parity for the same guarantee, survives 300 to 500 progressive bit-search flips with a 1% BCH code, and stores at 3.3 bits per weight in a sensitivity-ordered format.

The contributions of this paper are the following.

  • A systematic way to modulate the weight-sensitivity distribution of a network during training. One composite regularizer on the two factors of the damage, with a coefficient λ\lambda that sets the amount, an ablation that attributes the effect to each term, a real-flip check that the effect is not gradient masking (Table IV), and a plain gradient-norm baseline that does not reproduce it. Measured on nine vision model-dataset configurations (ResNet-18, ResNet-50 and VGG-16 on CIFAR-10, CIFAR-100 and ImageNet) and three language models (Qwen2-0.5B, Llama-3.2-1B and Qwen2-1.5B): the sensitive tail shrinks 5 to 13 times on the vision models within 0.5 points of accuracy, and 1.7 to 5.3 times on the language models, after one fine-tune and monotonically in λ\lambda. It is applied as a ten-epoch fine-tune of a trained network or switched on in the last epochs of training; both routes reach the same floor.

modelaccHIGH top-1%worst 100base topΔ\Delta ratio
baseline95.11.00%20.4%0.0310.0069 1.00
PtailP_{\mathrm{tail}} only94.50.87%19.3%0.0300.0061 0.99
PtailP_{\mathrm{tail}} only, λ=400\lambda=40094.10.35%16.3%0.0310.0062 1.07
PclipP_{\mathrm{clip}} only95.10.13%12.8%0.0120.0019 0.51
gradient norm, λ=0.01\lambda=0.0194.50.64%19.5%0.0260.0058 1.02
LEAN (both)94.90.11%11.7%0.0090.0014 0.51

Table IV. Ablation and real-flip check, ResNet-18 CIFAR-10, λ=40\lambda=40 unless noted. Columns are defined in the text.

  • Consequences of the reshaped distribution for protection, precision and storage. Bit-flip protection: a 1% code that fails on the normal model by 200 flips holds through 300 in every run and 94% through 500 in two of three seeds, and the parity for a fixed guarantee drops 3.7 times. Precision and storage: a sensitivity-ordered 8/6/4/3-bit format is 1.2 points above LSQ on ImageNet ResNet-50 at an exactly matched 4 bits per weight, and at 3.29 bits reaches 77.2% against 74.4% for LSQ at 3 bits, 2.4 times less weight payload than INT8 and an estimated 59% less weight-fetch energy; most of that gain comes from ordering the bits by sensitivity, which the small tail makes affordable, and λ\lambda adds the rest. With a small interleaved code the CIFAR low-bit models survive 500 flips as well. The evaluation uses adaptive bit-flip attacks as the hardest case of a stored-weight fault.

The rest of the paper is organized as follows. Section II explains why the sensitivity distribution matters, through the attacks, the codes and the formats that depend on it. Section III presents the objective that reshapes it. Section IV measures the reshaping, checks what each term does and that the damage really falls, and measures the effects on protection, precision and storage; § 5 discusses what that means for the systems that would use it and extends the measurement to language models; Section VI reviews related work and Section VII concludes.

Background

This section explains why the sensitivity distribution matters, through the three things that depend on it: the attacks that exploit its tail, the codes that must cover it, and the formats that must store it. It also introduces the baselines.

a) Bit-flip attacks on stored weights: Stored weights are reached by Rowhammer-style faults, by low-voltage operation and by hardware or infrastructure Trojans [6, 12]; whatever the vector, the damage depends on which weight is hit. Progressive bit search (PBS) [3] is the standard adaptive attacker against a quantized network: at each step it ranks stored bits by the gradient of the loss, evaluates the loss after flipping each of the top candidates, commits the worst, and repeats; a handful of flips take ImageNet ResNets to chance. T-BFA [4] uses the same search to send one source class to a chosen target class and is reported by its attack success rate. Rowhammer makes such flips practical in DRAM [1, 2], and low-voltage SRAM produces random ones [5]. Both attacks succeed by finding the tail of the sensitivity distribution; in all our experiments the attacker is white-box and sees the weights, the gradients and, where a code is present, the code.

b) The sensitivity distribution, tolerance and cost: For a weight stored as an 8-bit two’s complement integer with a per-tensor scale, flipping the sign bit changes the integer by 128, which is half of the representable range. Every weight in a layer therefore moves by the same large amount when its sign bit flips, and the damage is decided by the gradient alone. Gradients are heavy-tailed: in a trained ResNet-18 the 1% of weights with the largest first-order damage account for about 20% of the total, and the next 8% for most of the rest; the curves in Figure 6 fall by a factor of 10,000 or more between the median weight and the most sensitive one. This non-uniformity is the fact every selective scheme builds on, and it is also what PBS exploits, since the gradient ranking that PBS uses is the same ranking that defines the sensitive set. It is worth being explicit about the two roles the distribution plays. The first is error tolerance: a random flip usually lands in the LOW mass and does little, while an adaptive one is aimed at the tail and does much, so the tolerance of a model to a given number of flips is set by how many weights sit above the damage level at which a flip matters, not by the size of the model. The second is efficiency: a selective code must cover the tail, a mixed-precision format must store it at full width, and a budget of bits or parity buys protection in proportion to how small the tail is. A model whose sensitivity distribution has a short tail is therefore both more tolerant and cheaper to protect, for the same code and the same format. Robust

Cumulative weight-sensitivity distributions for nine models

Figure 6. Cumulative weight-sensitivity distribution of the nine models, counted from the tail (number of weights whose sensitivity is at least ss), on log-log axes; baseline solid, LEAN dashed. The shaded bands are the baseline LOW, MID and HIGH bins. The LEAN curve separates from the baseline at the MID boundary and stays about one decade lower across the HIGH band. The box in each panel gives the count above the HIGH threshold before and after.

training methods change the distribution as a side effect; the method in Section III changes it on purpose and by a chosen amount.

c) Selective error correction: A code over every weight bit stops flips outright but costs 12 to 20% of model memory in parity, which embedded codes such as E3-CODE [13] reduce by folding parity into the weight storage. A sensitivity-aware selective code protects selectively: it sorts weights into HIGH, MID and LOW bins by first-order damage, encodes the four most significant bit planes of the HIGH weights with a BCH(127,43) code that corrects t=12t = 12 errors per 127-bit codeword, and stores the parity in the low-order bit planes of LOW weights, where it costs nothing visible in accuracy. Only four of the eight planes are coded, so the parity is at most 4/8×84/43≈0.98%4/8 \times84/43 \approx0.98\% of the weight bits for every 1% of weights covered, and 0.91% as packed in Table VII, and it occupies low-order bits of LOW weights rather than extra memory; its cost is a parity budget inside the model, not bits added to it. Defeating one codeword takes more than tt errors in that word. In our attack evaluation a flip on a covered bit is modelled as corrected, with no persistent effect; this models an interleaved deployment under the assumption that the attacker cannot accumulate more than tt errors in one codeword before it is corrected. The code itself is not simulated, and the guarantee rests on that assumption. In-place zero-space protection [7] and similar schemes make the same trade. For all of them the cost is set by one number, the size of the HIGH bin. The method in the next section is built to make that number small.

d) Quantization baselines: For storage we compare against LSQ [14], which learns the quantization step size by gradient descent and is the standard strong baseline for low-bit uniform quantization, and against KURE [15], which regularizes the kurtosis of the weight distribution so that one model quantizes well at several bit widths. Both are retrained in our pipeline with the same schedule as our own models.

Method

We first write down what a flip costs, then turn that cost into a training term, show what one gradient step does to it, and finally describe the storage format that a small sensitive set makes affordable.

a) Sensitivity of a stored weight: We store weights as INT8 with one scale per tensor: layer ℓ\ell has sℓ=max⁡∣Wℓ∣/127s_{\ell} = \max|W_{\ell}|/127, and a weight ww is kept as the integer q=round⁡(w/sℓ)q = \operatorname{round}(w/s_{\ell}). In two’s complement, flipping the sign bit of an 8-bit code changes qq by 128, so every weight in the layer moves by the same amount Δℓ=128sℓ\Delta_{\ell} = 128s_{\ell}, whatever its own magnitude. The first-order change in the loss produced by that flip is

di=∣∂L∂wi∣Δℓ(i)(1)d_i = \left|\frac{\partial L}{\partial w_i}\right| \Delta_{\ell(i)} \tag*{(1)}

which we call the damage, or sensitivity, of weight ii. We fix two thresholds once, from the baseline model: the HIGH threshold is the damage of the weight at the top 1% of the baseline distribution and the MID threshold that at the top 9% (so HIGH holds 1% of the weights and MID the next 8%), so the baseline has 1% HIGH, 8% MID and 91% LOW. A trained model is always measured against these fixed thresholds, so “fewer HIGH weights” means fewer weights above the baseline’s own cut, and the three bins of any model sum to the same total.

b) Objective: Let TT be the set of the k=⌈0.01N⌉k = \lceil0.01N\rceil weights with the largest damage. LEAN minimizes

Lλ=LCE+λPLEAN,PLEAN=Ptail+βPclip,(2)L_{\lambda} = L_{\mathrm{CE}} + \lambda P_{\mathrm{LEAN}}, \qquad P_{\mathrm{LEAN}} = P_{\mathrm{tail}} + \beta P_{\mathrm{clip}}, \tag*{(2)}
Ptail=1k∑i∈Tdi,P_{\mathrm{tail}} = \frac{1}{k}\sum_{i\in T} d_i,

where Pclip=∑ℓmean⁡(relu⁡(∣Wℓ∣−q99(∣Wℓ∣))2)P_{\mathrm{clip}} = \sum_{\ell} \operatorname{mean}\left(\operatorname{relu}(|W_{\ell}|-q_{99}(|W_{\ell}|))^2\right) bounds the INT8 range of each layer, and with it the flip step Δℓ\Delta_{\ell}; β=100\beta= 100 in every experiment and is never tuned, so the effective weight of the range term is 100λ100\lambda and grows with the knob. The two terms act on the two factors of (1): PclipP_{\mathrm{clip}} shrinks Δℓ\Delta_{\ell} where a layer has outlying weights, which is what sets the sign-flip step, and PtailP_{\mathrm{tail}} lowers ∣gi∣|g_i| on the current tail. Section IV separates the two; at INT8 the range term does most of the reshaping and the tail term adds to it. PtailP_{\mathrm{tail}} is a function of the gradient, so its gradient is a Hessian-vector product. We compute it by double backpropagation on every fourth step, with the tail term scaled by four on that step so that its average weight over training is λ\lambda; the fine-tune then costs about 1.5 times a plain one. In the double backward pass the step Δℓ\Delta_{\ell} (and with it the scale sℓs_{\ell}), the quantile that selects TT and the quantile q99q_{99} are detached, so the second-order term differentiates only through gg, as in the proposition below. All experiments use ten epochs from the baseline checkpoint with SGD, learning rate 0.002 on CIFAR and 0.0005 on ImageNet, batch size 128 and 32.

Algorithm 1 lists the steps, and Figure 3 shows the same procedure as a flow, from a trained network to a deployed one. The INT8 pipeline has one LEAN stage of ten epochs, a fine-tune or the last epochs of training: the baseline is measured, trained with the LEAN term, re-measured against the thresholds fixed from the baseline, and packed, either with a code over a fixed budget (the top 1% by damage, as in all attack tables) or over everything above τH\tau_H (the threshold mode of the parity experiment). The low-bit format of the last paragraph of this section is a second deployment and uses a second, quantization-aware fine-tune.

The LEAN procedure

Figure 3. The LEAN procedure. The trained network is quantized and its sensitivity distribution is measured once, which fixes the HIGH and MID thresholds; ten epochs with the LEAN term follow, as a fine-tune or as the last epochs of training, with the tail TT re-selected and the tail term applied, through double backpropagation, every fourth step; the model is then re-measured against the fixed thresholds and packed. λ\lambda is the only knob; the low-bit format uses a separate quantization-aware fine-tune (last paragraph of this section).

Proposition 1 (Observation). Let g=∇wLCEg = \nabla_w L_{\mathrm{CE}}, H=∇w2LCEH = \nabla_w^2 L_{\mathrm{CE}}, D=diag⁡(Δℓ(i))D = \operatorname{diag}(\Delta_{\ell(i)}) and σi=sign⁡(gi) 1[i∈T]/k\sigma_i = \operatorname{sign}(g_i)\,1[i \in T]/k. Hold TT, the signs of gg on TT and DD fixed. A gradient step of size η\eta on λPtail\lambda P_{\mathrm{tail}} alone changes the tail sensitivity by ΔPtail=−ηλ∥HDσ∥2+O(η2)\Delta P_{\mathrm{tail}} = -\eta\lambda\lVert HD\sigma\rVert^2 + O(\eta^2).

With TT and the signs held fixed over the step, Ptail=σTDgP_{\mathrm{tail}} = \sigma^\mathsf{T}Dg, so ∇wPtail=HDσ\nabla_w P_{\mathrm{tail}} = HD\sigma; the step is w←w−ηλHDσw \leftarrow w-\eta\lambda HD\sigma, and the first-order change of PtailP_{\mathrm{tail}} is −ηλ(HDσ)T(HDσ)-\eta\lambda(HD\sigma)^\mathsf{T}(HD\sigma). This is the descent lemma applied to PtailP_{\mathrm{tail}} and we claim nothing more for it; it is local to the tail-penalty step, and TT can change between steps. Its use is to say what the tail term can and cannot do: it moves ww along the Hessian rows of the tail weights and lowers their gradients, which is the quantity a gradient-guided attacker reads, and a detached proxy such as ∣wi∣|w_i| or a frozen ∣gi∣|g_i| has no HH term and cannot do that. It does not act on Δℓ\Delta_{\ell}, which is the other factor of the damage and, as Section IV shows, the larger lever at INT8. Whether lowering the gradient also lowers the real damage of a flip is an empirical question, and we answer it with real flips rather than with the proxy.

Algorithm 1 LEAN: shaping the sensitivity distribution

Figure 1 (PDF p. 4).

Algorithm 1 LEAN: shaping the sensitivity distribution

Require: trained ww; sign-flip step Δℓ\Delta_{\ell} of the storage format; λ\lambda; probe batch BB

  1. di←∣∂LCE/∂wi∣ Δℓ(i)d_i \gets|\partial L_{\mathrm{CE}}/\partial w_i|\,\Delta_{\ell(i)} on BB ▷ baseline

  2. τH,τM←\tau_H,\tau_M \gets top-1% and top-9% quantiles of dd ▷ fixed

  3. for each fine-tuning step with batch (x,y)(x,y) do

  4. g←∇wLCE(x,y)g \gets\nabla_w L_{\mathrm{CE}}(x,y)

  5. u←0u \gets0

  6. if step mod 4 = 0 then ▷ graph retained

  7. di←∣gi∣ Δℓ(i)d_i \gets|g_i|\,\Delta_{\ell(i)}; T←T \gets top k=⌈0.01N⌉k = \lceil0.01N\rceil of dd

  8. Ptail←1k∑i∈TdiP_{\mathrm{tail}} \gets\frac{1}{k}\sum_{i\in T}d_i ▷ Δℓ\Delta_{\ell}, TT detached

  9. u←4∇wPtailu \gets4\nabla_w P_{\mathrm{tail}} by double backprop ▷ =4HDσ=4HD\sigma

  10. end if

  11. w←w−η[g+λ(u+β∇wPclip)]w \gets w-\eta\left[g+\lambda\left(u+\beta\nabla_wP_{\mathrm{clip}}\right)\right] ▷ (2)

  12. end for

  13. re-measure dd on BB; shares above τH,τM\tau_H,\tau_M ▷ HIGH, MID

  14. pack: BCH over the top 1% by dd (fixed budget) or over {i:di>τH}\{i:d_i>\tau_H\} (threshold)

c) Sensitivity-ordered low-bit storage: The proposition covers the training side. On the deployment side, a cheaper code and a longer survival need no change to the storage format, but the third consequence, a smaller model, does. Once HIGH and MID are small, a mixed format can afford to store them well. We keep the HIGH tier at 8 bits with a per-tensor scale, the MID tier at 6 bits per channel, fill a 4-bit tier with the next most sensitive weights until an average-bit budget is met, and store the remaining weights at 3 bits per channel. This is a second, ten-epoch quantization-aware fine-tune, started from the LEAN INT8 checkpoint, that carries the same λ\lambda term, now evaluated with each weight’s own deployed flip step 2b−1s2^{b-1}s; the tiers are refreshed every 50 of its steps. The bits-per-weight figures we report count the weight payload only; the per-channel scales and the tier map are not included, and code parity is added only where stated. The map is the larger of the two: tiers are assigned per weight, so an uncompressed map costs 2 bits per weight, an entropy-coded one about 1 bit at the tier shares of Table VIII, and a block-wise assignment would cost a fraction of a bit; we leave its layout to the deployment and compare payloads. Two details decide whether this low-bit model can be attacked cheaply. The scale of the 6/4/3-bit tiers is set by the 0.99 quantile of the tier range at its maximum, so that a sign flip in the 3-bit tier moves a weight 2 to 4 times less than it otherwise would; and tiers are ranked by deployed-format damage, so that the 8-bit tier is the most damaging 1% of the model that is actually stored, not of the floating-point

formatbits/wtop-1
INT88.0080.7
LSQ uniform [14]4.0078.4
KURE uniform [15]4.0075.9
tiered, λ=0\lambda=04.0078.9
tiered, LEAN λ=40\lambda=404.0079.6
LSQ uniform3.0074.4
KURE uniform3.0073.2
ours, uniform QAT3.0074.3
tiered, λ=0\lambda=03.2976.8
tiered, LEAN λ=40\lambda=403.2977.2

Table VIII. ImageNet ResNet-50 top-1 (%) at matched (4.00) and comparable (3.00 vs 3.29) bits per weight; tiered rows are mean of three seeds.

[curtail] model it came from.

Results

The experiments are in two parts. The first part establishes that the procedure reshapes the distribution on every model, by an amount that λ\lambda controls, without hurting accuracy. The second part measures the consequences of the reshaped distribution for bit-flip protection, for precision and storage, and for the low-bit model’s own resilience. Language models are in § 5.

Experimental setup

We use ResNet-18, ResNet-50 [16] and VGG-16 with batch normalization [17] on CIFAR-10, CIFAR-100 [18] and ImageNet [19]; ResNet-34 is added in the storage and low-bit experiments (Table XI, Table IX and Table X) as a fourth deployment target. CIFAR baselines are trained from scratch. ImageNet baselines are the torchvision pretrained models, and every ImageNet fine-tune and evaluation in this paper uses one fixed random split of the 50k validation set (seed 0), 40k images for fine-tuning and 10k held out for evaluation; ImageNet accuracies are therefore on this 10k held-out set and are not directly comparable to published

modelBCH only 25BCH only 50BCH only 100+ sign-plane code 100+ sign-plane code 200+ sign-plane code 300+ sign-plane code 500
C10 ResNet-1891681494949490
C10 ResNet-3493897394949493
C10 ResNet-5082632894939290
C10 VGG-1683652793929186
C100 ResNet-1867561175727167
C100 ResNet-3472511676767570
C100 ResNet-504012473665438
C100 VGG-1650311170645949

Table IX. CIFAR models at 3.29 bits per weight under progressive bit search, accuracy (%): BCH on the 1% tier only (3.45 B/W) and with the interleaved sign-plane code (3.63 B/W).

modelclean1% tier, t=19t=19 1001% tier, t=19t=19 2001% tier, t=19t=19 5001% tier, t=79t=79 1001% tier, t=79t=79 2001% tier, t=79t=79 5003% tier, t=79t=79 1003% tier, t=79t=79 2003% tier, t=79t=79 500
ResNet-1867.754271635625676454
ResNet-3471.1665822706964717170
ResNet-5071.881048242696859
VGG-1672.1655939716962737373

Table X. ImageNet models at 3.29 bits per weight under progressive bit search, accuracy (%) after 100 / 200 / 500 flips, with the sign-plane code at t=19t = 19 (3.63 b/w) or t=79t = 79 (3.68), and with the 3% tier and t=79t = 79 (4.09). No retraining.

modelMwM_w†INT8 (+code) MBINT8 (+code) mJ3.63 b/w MB3.63 b/w mJ4.09 b/w MB4.09 b/w mJ
CIFAR ResNet-1811.211.21.795.10.815.70.92
CIFAR ResNet-3421.321.33.419.71.5510.91.74
CIFAR ResNet-5023.623.63.7810.71.7112.11.93
CIFAR VGG-1614.714.72.356.71.077.51.20
ImageNet ResNet-1811.711.71.875.30.856.00.96
ImageNet ResNet-3421.821.83.499.91.5811.11.78
ImageNet ResNet-5025.625.64.1011.61.8613.12.09
ImageNet VGG-16138.4138.422.1462.810.0570.811.32

Table XI. Weight memory (MB) and estimated DRAM weight-fetch energy per inference (mJ) for the deployed formats; formats and assumptions in the text. †Millions of weights.

50k-set numbers. Every LEAN INT8 model is a single ten-epoch fine-tune from its baseline with the objective of Eq. (2) (Figure 4 shows one such run); the published defenses and the quantization baselines are fine-tuned with the same schedule. Unless a table says otherwise, each entry is one training run and one attack run with a fixed seed; Table VIII uses three seeds, and the ResNet-18 CIFAR-10 λ=40\lambda= 40 fine-tune was repeated with three seeds: clean accuracy 94.84 ±\pm 0.06, HIGH 0.113 ±\pm 0.003%, MID 1.84 ±\pm 0.17%, and with the 1% code every seed holds 94% through 300 PBS flips while two of three hold 94% at 500 (the third falls to chance there). The tail fraction is 1% throughout; 0.5% and 2% give HIGH 0.115% and 0.118% at 94.7% and 94.9% accuracy, so the result is not sensitive to it. The protection experiments instantiate the selective code with the layout of Section III, but LEAN does not depend on it: any scheme whose cost grows with the number of high-sensitivity weights benefits from the same tail reduction. Sensitivity is measured by Eq. (1) on 20 batches of 128 images, against thresholds fixed from the baseline: the thresholds used during training come from training batches, and the reported distributions, counts and shares (Table I, Figure 5 and Figure 6) from held-out batches. Attacks are real PBS with 50 candidate bits per step and two attack batches of 128 images, reported as top-1 accuracy on the held-out set after nn committed flips; T-BFA is reported as accuracy and attack success rate after 30 flips. Clean accuracy is top-1 on the held-out set. All experiments were run in PyTorch on NVIDIA RTX A6000 GPUs (48 GB of memory each), one GPU per run; a ten-epoch LEAN fine-tune takes about 9 minutes for ResNet-18 on CIFAR-10 and 1 to 3 hours on ImageNet, and a 500-flip PBS attack takes 25 minutes on CIFAR and 1 to 3 hours on ImageNet.

Training losses and accuracy for one LEAN fine-tune

Figure 4. One LEAN fine-tune, ResNet-18 CIFAR-10, λ=40\lambda= 40: (a) the regularizer falls 28 times while the cross-entropy stays flat; (b) test accuracy dips after the first epoch and recovers to within 0.2 points of the baseline.

Raw weight-sensitivity distribution of the nine models

Figure 5. Raw weight-sensitivity distribution of the nine models: number of weights per sensitivity bin (40 log-spaced bins of did_i as a percentage of the baseline maximum; the first bin collects every weight below 10−310^{-3}), baseline solid, LEAN dashed, with the baseline LOW, MID and HIGH bands. Figure 6 shows the same data as a cumulative distribution. The bulk of the distribution is unchanged; the HIGH tail moves left into the MID and LOW bands.

b) The distribution moves on every model: Table I counts the weights above the baseline HIGH threshold before and after LEAN on nine models; Figure 5 shows the raw distribution and Figure 6 the cumulative distribution, counted from the tail. The count falls 5 to 13 times. The share of the total first-order damage carried by the top 1% of weights falls 1.4 to 2.1 times on all nine models (Table I, from 9 to 28% down to 5 to 18%); a uniform rescaling of the flip steps would leave this share unchanged, so the distribution is reshaped, not only scaled. The same share computed from the gradients alone, with no flip step, is unchanged by LEAN (8.2 against 8.7% on ResNet-18 CIFAR-10, and alike elsewhere), so the reshaping acts through the per-layer flip steps, which the range term lowers where the tail lives, while the tail term’s contribution shows in the real-flip damage of Table IV rather than in this share. Both figures use logarithmic axes, which is why the change looks modest at first sight: the bulk of the distribution spans six decades of count and is untouched, while the tail that moves is one decade lower, a factor of 10 in the number of weights that the code has to cover. The clean-accuracy change is at most 0.5 points (ResNet-50 on CIFAR-100) and is usually 0.1 or less; on ImageNet ResNet-18 it is zero. The curves make clear that the change is not confined to the very top of the distribution: the LEAN curve leaves the baseline curve at the MID boundary and runs about one decade below it through the whole HIGH band, so the entire tail has moved to lower sensitivity. Where the weights go is also visible: since every weight is counted somewhere, a lower curve at the MID line is the same statement as a larger LOW bin, and on ResNet-18 CIFAR-10 LOW grows from 91% to about 92% while HIGH drops from 1% to 0.11%.

datamodelbaseLEAN×top-1%1\% shareacc.
C10ResNet-18112k13k (0.11%)919.5 → 10.895.1 → 94.9
C10ResNet-50235k18k (0.08%)1316.0 → 10.995.0 → 94.8
C10VGG-16147k16k (0.11%)928.3 → 17.992.9 → 92.8
C100ResNet-18112k11k (0.10%)109.1 → 5.177.2 → 77.1
C100ResNet-50237k31k (0.13%)810.2 → 7.175.9 → 75.4
C100VGG-16148k13k (0.09%)1118.3 → 8.672.3 → 72.3
ImNetResNet-18 (λ30\lambda30)117k19k (0.16%)611.6 → 6.269.9 → 69.9
ImNetResNet-50255k51k (0.20%)514.9 → 9.280.7 → 80.5
ImNetVGG-16 (λ100\lambda100)1383k199k (0.14%)726.0 → 18.373.4 → 73.3

Table 1. HIGH-BIN WEIGHTS BEFORE AND AFTER LEAN, THE SHARE OF TOTAL FIRST-ORDER DAMAGE CARRIED BY THE TOP 1% OF WEIGHTS (%, SCALE-FREE, MEASURED ON 20 HELD-OUT BATCHES; THE ABLATION OF TABLE IV MEASURES THE SAME QUANTITY ON ITS OWN PROBE BATCHES, 20.4 → 11.7%) AND CLEAN ACCURACY (%), EACH AS BASE → LEAN; λ=40\lambda= 40 UNLESS NOTED.

c) λ\lambda is a knob: Table I uses one λ\lambda per model; Figure 7 plots the HIGH share and the clean accuracy against λ\lambda for every value we trained on six models. The HIGH share falls monotonically with λ\lambda and saturates near 0.1% of the weights; accuracy stays within 0.5 points of the baseline up to λ=40\lambda= 40 and begins to fall at λ=100\lambda= 100 (ResNet-50 ImageNet goes from 80.7 to 79.7). We therefore use λ=40\lambda= 40 on CIFAR and λ=30\lambda= 30 to 40 on ImageNet unless stated otherwise. The two ImageNet models are the ones that shrink least (5 to 6 times); their sensitive sets are the largest to begin with and need the strongest penalty.

Six pairs of bar and line plots showing HIGH-bin share and clean accuracy against $\lambda$ for six models

Figure 7. The knob. HIGH-bin share (bars, log scale) and clean accuracy (line) against λ\lambda for six models, at the values trained for the sweep. The HIGH bin falls monotonically and saturates near 0.1%; accuracy stays within 0.5 points up to λ=40\lambda= 40 and drops at λ=100\lambda= 100.

d) What sets the floor: The knob does not go to zero, and Figure 8 shows why. The damage of a weight is ∣giΔℓ∣|g_i \Delta_\ell|, so the tail can be lowered only as far as its two factors allow. The range term shrinks Δℓ\Delta_\ell down to the level of a layer’s ordinary weights and no further without clipping them, so the flip step of a layer falls by at most its outlier ratio,

Three-panel plots showing the floor, where the floor lives in ResNet-18, and HIGH reduction across models

Figure 8. The limit of the reshaping. (a) HIGH share against λ\lambda (solid, left axis) and change in clean accuracy (dotted, right axis) for ResNet-18 CIFAR-10 and ResNet-50 ImageNet; the share flattens near 0.1% while accuracy begins to fall. (b) HIGH weights per layer before and after LEAN, ResNet-18 CIFAR-10, λ=40\lambda= 40: the deep layers empty, the first convolution and first stage keep most of what remains. (c) HIGH reduction factor for the twelve models, largest to smallest: CIFAR, then ImageNet, then the language models.

Δℓ≥128127q99(∣Wℓ∣),ΔℓbeforeΔℓafter≤ρℓ=max⁡∣Wℓ∣q99(∣Wℓ∣)(3)\Delta_\ell\ge\frac{128}{127}q_{99}(|W_\ell|), \qquad \frac{\Delta_\ell^{\mathrm{before}}}{\Delta_\ell^{\mathrm{after}}} \le\rho_\ell= \frac{\max|W_\ell|}{q_{99}(|W_\ell|)} \tag*{(3)}

which is a large gain in a layer with outlying weights and none in a layer without them. The tail term lowers the top-kk gradients at a speed set by the curvature term ∥HDσ∥\lVert H D \sigma\rVert of the Observation, and because the set TT is re-selected every step, what it does in practice is flatten the top of the distribution into a plateau rather than remove it (Figure 6). Where the two stop is the joint optimum λ∇wLLEAN=−∇wLCE\lambda\nabla_w \mathcal{L}_{\mathrm{LEAN}} = -\nabla_w \mathcal{L}_{\mathrm{CE}}, beyond which every further reduction of the tail is paid for in task loss. On ResNet-18 CIFAR-10 the HIGH bin falls monotonically with λ\lambda and flattens near 0.1% from λ=30\lambda= 30 on, while accuracy begins to drop (Figure 8a); the penalty itself plateaus by epoch 6 to 8 of the fine-tune (Figure 4). Where the remaining weights sit is instructive (Figure 8b): the deep layers empty completely, and four fifths of the 12.6k weights still above the threshold are in the first convolution and first stage, small layers that every input passes through, whose weights have no outliers to clip (the first convolution has max⁡∣W∣/q99∣W∣=1.7\max|W|/q_{99}|W| = 1.7, the lowest in the network, and keeps 92% of its HIGH weights) and whose gradients cannot be lowered without changing what the network computes. The floor is therefore a property of the model, set by its per-layer outlier ratios ρℓ\rho_\ell (Eq. (3)) and by the curvature of the loss around its early weights, not by λ\lambda or the number of epochs; λ\lambda chooses how far toward it to go. Across models the reduction is largest where there are few weights and large outliers, 9 to 13 times on CIFAR, and smallest where many weights sit near the threshold, 5 to 7 times on ImageNet and 1.7 to 3.4 times on the language models (Figure 8c).

e) Fine-tune or last epochs of training: LEAN is a loss term, so it can be switched on inside normal training instead of run as a separate fine-tune. Figure 9 and Tables II and III test this on six model/dataset pairs: one 200-epoch run from scratch per pair, forked at epoch 190 into a plain control and branches with the term on for the last epochs. Switched on as is, in the last ten epochs of the cosine schedule, the term works but reaches less far than the fine-tune, HIGH 0.25 to 0.50% against 0.11%, and the penalty is still falling at epoch 200 (Figure 9b). The cause is the learning rate, which in those epochs is 0.0005 falling to zero, a quarter of the fine-tune’s. Holding it at the fine-tune’s 0.002 for the same ten epochs reaches the floor on every pair, HIGH 0.04 to 0.14% and MID 0.7 to 2.9%, at the fine-tune’s accuracy; twenty epochs or λ=100\lambda= 100 at the dying rate get halfway, HIGH 0.05 to 0.20%. Reaching the floor costs 0.1 to 0.7 points except on VGG-16 CIFAR-100 (1.3 points, where a smaller λ\lambda would be used), stopping halfway costs 0.1 or nothing. Under attack the routes line up in the same order (Table V, bottom block, ResNet-18 CIFAR-10 with the 1% code): the control breaks at 200 flips, the last-epochs model at 300, the rate-held model holds 94% at 300 and breaks at 500; the targeted attack with the code stays at 3 to 6% success for all of them, as for the fine-tune. Giving the last ten epochs the fine-tune’s full schedule, 0.002 decaying to zero (rate held, cos.), closes the remaining gap: 93% at 500 flips at 95.0% clean accuracy, the fine-tune’s protection at a slightly higher accuracy, and on VGG-16 the same schedule reaches HIGH 0.04% at no accuracy cost. The decay matters more than the last few hundredths of HIGH: twenty epochs at a constant 0.002 (rate held, 20 ep.) leave a smaller tail by count (0.13%, MID 0.6%) yet break at 400 flips, because the undecayed rate leaves the surviving weights less settled. Both routes are the same technique; we use the fine-tune because it gives the term that schedule by construction and because the ImageNet and language-model baselines are pretrained checkpoints with no training of ours to switch it on.

LEAN inside training results for ResNet-18 and VGG-16 on CIFAR-10, with four additional model/dataset pairs

Figure 9. LEAN inside training, λ=40\lambda= 40; ResNet-18 CIFAR-10 in the top row, VGG-16 CIFAR-10 below. (a, d) Test accuracy for epochs 180 to 200 of a 200-epoch run; at epoch 190 the run is forked into a plain control and a branch with the LEAN term on. (b, e) The penalty over the ten LEAN epochs inside training (learning rate 0.0005 to 0) and in the fine-tune (0.002, cosine). (c, f) HIGH share and accuracy of the control, the last-epochs model, the last-epochs model with the rate held at 0.002, and the fine-tune. (g) HIGH share per recipe on four more pairs, accuracy inside the bars.

methodHIGHno code 10no code 25no code 50no code 100with 1% BCH code 100with 1% BCH code 200with 1% BCH code 300with 1% BCH code 500
normal1.004111101093261710
PC [8]0.894411101095332214
Hessian [11]1.014411101095312214
FaR [10]1.051610101095954512
WCAT [9]0.178048181195959475
LEAN λ=10\lambda=100.217730101095949316
LEAN λ=30\lambda=300.126725121195949490
LEAN λ=40\lambda=400.118136101095959594
ablations and gradient-norm baseline (Table IV)
PclipP_{\mathrm{clip}} only0.1379251010957749
PtailP_{\mathrm{tail}} only0.874511101094433410
PtailP_{\mathrm{tail}}, λ=400\lambda=4000.352610101094939110
gradient norm0.644710101094141511
LEAN inside training (Table II):
control1.008611101095937810
last epochs0.409490151095953110
rate held0.149477261095959411
rate held, cos.0.169465181095959593
rate held, 20 ep.0.139591531395949316

Table V. ResNet-18 CIFAR-10, INT8, accuracy (%) after nn progressive bit-search flips, without and with the same 1% BCH code; HIGH in %. Clean accuracy 95. Bottom block: the models trained with LEAN in the last epochs.

Which term does the work, and is the damage real

Two questions have to be settled before the consequences are measured. The first is whether the proxy di=∣gi∣Δℓ(i)d_i = \lvert g_i\rvert\Delta\ell(i) can be gamed: an attacker ranks weights by the gradient, and a term that lowers the gradient could hide weights without making them safer. The second is which of the two terms of Eq. (2) moves the distribution. Table IV answers both on ResNet-18 CIFAR-10. HIGH and the top-1% damage share are from the proxy. For the first question we flip real sign bits one at a time, in the INT8 model, for the 1000 weights the proxy ranks highest and for 1000 random weights, and record the true loss increase; the table gives the mean over the 100 worst weights found (worst 100) and over the baseline’s own top 1000 reflipped inside each fine-tuned model (base top), and random weights cause no measurable change in any model. Δ\Delta ratio is the mean per-layer flip step relative to the baseline. In the LEAN model the worst 100 weights cost 0.009 per flip against 0.031 in the baseline, and the baseline’s top 1000 cost 0.0014 against 0.0069, so the damage really falls; the proxy’s share of total damage held by the top 1%, which does not depend on the scale of Δℓ\Delta_{\ell}, falls from 20% to 12%. For the second, the range term alone reaches almost the same point (HIGH 0.13%, worst-100 damage 0.012, Δℓ\Delta_{\ell} halved on average) at no accuracy cost, and the tail term on top of it lowers the damage a further 25% (0.012 to 0.009, and 0.0019 to 0.0014 on the fixed set). The tail term alone does what the first question feared: at λ=400\lambda= 400 it reports a 3 times smaller HIGH bin while the real damage of the worst weights is unchanged and accuracy drops a point. At INT8, then, the lever is the flip step, which the range term controls, and the tail term is a useful addition that must not be used on its own. The last row is the plain gradient-norm penalty, the squared norm of the

weight gradient, under the same schedule, the nearest generic regularizer: at λ=0.01\lambda= 0.01, the largest value that keeps accuracy within a point, it leaves HIGH at 0.64% and the real damage of the worst weights at 0.026 against 0.031, and at λ=0.03\lambda= 0.03 accuracy falls to 88% with the tail unchanged; without the storage-format weighting and the tail selection the gradient is smoothed everywhere and the tail is not where the pressure goes.

g) Effect on bit-flip protection: The first consequence of a smaller tail is that a small code becomes enough. We attack the INT8 model with real progressive bit search [3] over all layers and protect it, where stated, with the selective BCH(127,43)\mathrm{BCH}(127,43) code (t=12t = 12) on the top four bit planes of the HIGH weights. Table V places LEAN next to four published robust-training defenses retrained in the same pipeline: piecewise clustering [8], WCAT [9], Forget-and-Rewire [10] and Hessian-aware training [11]. Without a code every method, LEAN included, is at chance within 50 to 100 flips: an attacker that may flip anything always finds the next most damaging weight. With the same 1% code the methods separate, and the HIGH column gives the first reason: the defenses leave 0.89 to 1.05% of the weights above the threshold (WCAT 0.17%), so a 1% code is exactly full and PBS finds the next weight. A small tail is necessary but not sufficient; what also counts is how damaging the weights just below the code are, the worst-100 column of Table IV: the range-only model, WCAT and the rate-held 20-epoch model all leave 0.13 to 0.17% HIGH yet end at 49, 75 and 16% at 500 flips. In what follows a model breaks at the first flip count at which its accuracy falls below 90%. The normal model breaks at

recipeep.rateλ\lambdaResNet-18 accResNet-18 HIGHResNet-18 MIDVGG-16 accVGG-16 HIGHVGG-16 MID
control100.0005→\to0095.31.008.093.61.008.0
last epochs100.0005→\to04095.20.406.493.70.256.9
longer200.0006→\to04095.30.203.993.50.066.2
stronger100.0005→\to010095.00.193.993.20.055.3
rate held100.0024094.90.140.993.20.042.9
rate held, cos.100.002→\to04095.00.162.293.60.044.2
rate held, 20 ep.200.0024095.10.130.692.70.042.0
fine-tune100.002 cos.4094.90.112.092.80.114.8

Table 2. LEAN in the last epochs of a 200-epoch run, CIFAR-10; HIGH and MID against the thresholds of the matching control of the same run. The fine-tune rows are the paper’s models, measured against their own baseline.

modelrecipeaccHIGHMIDmodelrecipeaccHIGHMID
ResNet-18control78.51.008.0ResNet-50control78.91.008.0
C-100last 1078.40.304.9C-100last 1078.60.505.6
20 ep.78.40.122.420 ep.78.60.173.1
λ\lambda=10078.30.132.6λ\lambda=10078.60.203.3
rate held78.10.090.7rate held78.20.041.2
VGG-16control74.81.008.0ResNet-50control95.31.008.0
C-100last 1074.20.336.6C-10last 1095.00.344.7
20 ep.74.80.104.220 ep.94.90.132.8
λ\lambda=10074.00.093.8λ\lambda=10094.90.163.1
rate held73.50.050.9rate held94.80.070.9

Table 3. The same recipes on four more pairs (C-10, C-100: CIFAR), λ=40\lambda= 40 unless stated, ten epochs unless stated. HIGH and MID in % of weights.

200 flips, piecewise clustering and Hessian-aware training at 200, Forget-and-Rewire at 300, WCAT at 500 (75%); LEAN at λ=40\lambda= 40 holds 94% at 500 (in two of three training seeds; the third holds 94% at 300 and breaks at 500, the experimental setup of Section IV), and the λ\lambda sweep (λ=10\lambda= 10 failing at 500, λ=30\lambda= 30 holding 90%) shows the dependence directly. The ablation rows settle what each term buys under attack: the range term alone (HIGH 0.13%) holds at 200 and breaks at 300, short of WCAT, and the tail term alone breaks at 200 like the normal model, or at 500 when pushed to λ=400\lambda= 400;

only the two together pass WCAT. The plain gradient-norm fine-tune breaks at 200. The code removes the weights the attacker wants, and λ\lambda makes sure that nothing of comparable sensitivity is left behind them. On ImageNet ResNet-18 with the 1% code the LEAN model holds 83% at 100 flips and 72% at 200 against 69% and 41% for the normal model. Table VI repeats the exercise with the targeted attack [4] on all nine models, 30 flips, three class pairs each. CIFAR uses the held-out test set. ImageNet uses all 50 validation images of each class (25 for the attack, 25 for the success rate), 40 of which were in the LEAN fine-tuning split while the normal model is the torchvision checkpoint that never saw them, so the overlap favours the LEAN side and the ImageNet rows should be read with that in mind; the CIFAR rows have no such overlap. Accuracy on the non-source classes stays within one point of clean on CIFAR; on ImageNet VGG-16 without a code it falls to 22 to 27%. The same rule holds, and it also shows where the rule stops. Where the targeted attack goes through weights that the mixed-data sensitivity ranks highly (the CIFAR-10 models and ImageNet VGG-16, whose classifier holds most of its tail), LEAN with the code ends it: 1 to 5% success against 30 to 93% for the normal model with the same code. Where the attack lives in the last layer’s row for one of 100 or 1000 classes, weights whose mixed-data sensitivity is modest, LEAN lowers the success rate on CIFAR-100 and changes nothing on the ImageNet ResNets. The reason is that LEAN reshapes the sensitivity it is trained on, the damage under the mixed-class training objective, and robustness does not transfer by itself to a class-conditional sensitivity that this objective does not rank. A tier chosen by the maximum damage over target classes closes that route for the normal model too; it is a packing-time choice that does not separate LEAN from the baseline, so we report the plain ranking and count the ImageNet ResNets as a limit of the mixed-data objective rather than of the code.

modelno code normalno code LEANwith 1% BCH code normalwith 1% BCH code LEAN
C10 ResNet-1895.444.692.73.9
C10 ResNet-5095.825.386.15.3
C10 VGG-1656.27.330.25.4
C100 ResNet-1896.075.396.770.7
C100 ResNet-5096.782.785.355.3
C100 VGG-1692.732.093.328.7
ImageNet ResNet-1896.096.090.792.0
ImageNet ResNet-5098.796.097.396.0
ImageNet VGG-1641.321.373.31.3
modelcov.parityBER prot.BER unprot.PBS-25 drop
normal20.0%18.095.088.00.00
normal19.2%17.694.9–0.48
normal16.8%15.495.0–0.48
normal15.2%13.995.0–0.52
normal12.5%11.595.088.30.52
LEAN λ=10\lambda=1011.9%10.995.090.60.00
LEAN λ=10\lambda=105.3%4.895.094.10.00
LEAN λ=40\lambda=405.3%4.894.993.90.16

Table VI. T-BFA, 30-flip budget, mean attack success rate (%) over three class pairs, without and with the 1% code; LEAN at the λ\lambda of Table I. Data and caveats in the text.

h) Effect on parity, precision, storage and energy: The second consequence of a smaller tail is cost: less parity, fewer bits and less energy for the same model. Figure 10 collects the three cost numbers. Protection first. The guarantee is fixed in advance and has a random and an adaptive part: 95% accuracy at a 6% random bit error rate on the protected weights, and no accuracy drop at 25 progressive flips (Table VII; the code is BCH t=12t=12 over four planes with the parity embedded in LOW weights, parity in % of weight bits; BER prot. and BER unprot. are the accuracy when the 6% error rate hits the covered weights with the code on, and the uncovered weights with no code). The 6% rate is a stress level, far above the 10−310^{-3} to 10−210^{-2} of low-voltage SRAM, chosen so that the guarantee is conservative; the 25-flip point sizes the parity, and the 500-flip survival of Table V is the separate, stronger result for the λ=40\lambda=40 model with the 1% code. The top 1% by damage is not enough for the normal ResNet-18, because PBS moves on to the weights just below the cut, which are nearly as damaging; it needs the code on 20% of the weights, which is 18% of the weight bits as parity; coverages of 12.5, 15.2, 16.8 and 19.2% all lose about 0.5 points at 25 flips, so the minimum lies between 19.2 and 20%. The LEAN model with λ=10\lambda=10 meets the same guarantee with the code on 5.3% of the weights, 4.8% parity, because the weights below its cut are far less damaging; the parity is 3.7 times smaller. We use λ=10\lambda=10 in this table because it is the smallest λ\lambda that meets the zero-drop guarantee; the paper’s λ=40\lambda=40 model at the same 5.3% coverage misses it by 0.16 points at 25 flips, and loses one point when the 6% error rate hits its uncovered weights. The same table gives the random-fault side: with no code on the uncovered weights, a 6% bit error rate there costs the normal model seven points at 20% coverage and the LEAN model four points at 11.9% and one point at 5.3%. Since the HIGH bin shrinks with negligible change in accuracy (Table I), the parity budget of any selective scheme shrinks with it.

Cost of protection, storage and weight fetch before and after LEAN

Figure 10. Cost of protection, storage and weight fetch before and after LEAN. (a) Parity for the same protection guarantee (Table VII), ResNet-18 CIFAR-10, measured; coverage in brackets. (b) Weight memory of ImageNet ResNet-50 at INT8, where the 1% code’s parity is embedded in LOW weights, and in the tiered 3.29-bit format without and with the 1% BCH and t=19t = 19 sign-plane codes (3.63 bits per weight); the 4.09-bit ImageNet hardening mode of Table X is not shown. (c) DRAM weight-fetch energy per inference with the 45 nm access energies of Han et al. [20]; on-chip SRAM scales identically (32 to 13 µJ).

Storage next. Table VIII compares formats for ImageNet ResNet-50 at exactly matched (4.00) and comparable (3.00 against 3.29) bits per weight under identical ten-epoch training. At a uniform 3 bits our quantization-aware training

reaches 74.3%, the same as LSQ [14] (74.4%), so λ\lambda by itself is not a better quantizer. The gain decomposes as follows: sensitivity-ordered tiering at 3.29 bits per weight adds 2.4 points over the uniform 3-bit model, and λ\lambda adds 0.4 more, consistently over three seeds. Most of the improvement is therefore from the ordering; what LEAN contributes is a smaller 8-bit and 6-bit tier, which is what makes the ordering affordable at this budget, plus the last 0.4 points. At an exactly matched 4.00 bits the tiered LEAN model is 1.2 points above LSQ and 3.7 above KURE [15]. In bytes, the 25.6 MB INT8 model becomes 10.5 MB of weight payload, or 11.6 MB with the codes of the next paragraph (at 3 bits per weight there are no LOW low-order bits to hide parity in, so there it is counted), not counting scales and the tier map.

Energy last. This is an analytical estimate, not a measurement: fewer bits per weight means fewer bits cross the memory interface, and we scale the weight traffic by published access energies. Taking the 45 nm access energies of Han et al. [20], 640 pJ for a 32-bit DRAM access and 5 pJ for a 32-bit SRAM access, the weight fetch for one ResNet-50 inference falls from 4.1 mJ to 1.7 mJ from DRAM and from 32 to 13 μ\muJ from on-chip SRAM, an estimated 59% less weight-fetch energy in either case; tier lookup, scale and metadata fetch and decoder activity are outside this estimate. The same reduction applies to the SRAM weight buffer an accelerator must provision, which shrinks 2.2 to 2.4 times. Table XI gives the same two numbers for every model in the paper: INT8 with the 1% code stays at 8 bits per weight because the parity is embedded in LOW weights; the low-bit format with both codes is 3.63 bits per weight (Table IX, CIFAR, or 4.09 with the 3% tier and the t=79t=79 code (Table X, given for the CIFAR models for comparison); the CIFAR and ImageNet ResNets of the same depth differ only in the classifier, so one row covers both; decoder and tier-lookup energy are not included. The ratio is fixed by the format (2.2 times at 3.63 bits per weight, 2.0 times at 4.09, with the INT8 parity embedded), so what changes from model to model is the absolute saving, from 1 mJ per inference on a CIFAR ResNet-18 to 12.5 mJ on ImageNet VGG-16.

i) Effect on the low-bit model’s own resilience: The INT8 model and the 3.29-bit model are different deployments; the natural question is whether the low-bit one can be protected by the 1% code as well. It cannot, and the reason is itself a statement about the distribution: at 3 bits per channel almost every weight is one sign flip away from a change of its full magnitude, so the tail the code must cover is the whole model (Table IX, left block, chance by 100 flips). Three changes, none of them retraining, close the gap, at a price in clean accuracy of about five points on ImageNet (Table X, 71.8% against 77.2% for the storage-only format of Table VIII), which is the cost of the smaller scales and the deployed-format ranking. The 6/4/3-bit scales are set by the 0.99 quantile instead of the maximum, which shrinks every low-bit flip 2 to 4 times; the tiers are ranked by deployed-format damage; and a long interleaved BCH code over the sign-bit plane (1023 sign bits per codeword, t=19t=19, 0.19 bits per weight) spreads each codeword over random addresses, so an adaptive attacker must spend t+1t+1 flips to gain one. With these, the CIFAR models hold 100 to 500 flips within 2 to 7 points of clean at 3.63 bits per weight (Table IX, right block). ImageNet needs more: t=79t=79 and a 3% tier obtained by promoting the 2% most damaging uncovered weights at packing time, at 4.09 bits per weight, after which VGG-16 (72.5% clean after promotion) is unchanged by 500 flips and ResNet-34 loses one point, while ResNet-18 and ResNet-50 lose 14 and 13 (Table X). The general lesson for the code is that correction over neighbouring words does not help against an adaptive flipper, since neighbouring weights are useful to it as one; interleaving is what makes the attacker’s extra flips worthless.

Applications

This section says what a reshaped distribution means for the systems that would use it, and extends the measurement to language models.

a) Accelerators and deployment: A selective code hides its parity in the low-order bits of LOW weights, so its cost is the number of HIGH weights times the code rate and nothing else. LEAN reduces that number 5 to 13 times at training time without touching the memory system: the decoder, the bin table and the parity embedding are reused unchanged, and the only difference the hardware sees is a shorter list of protected addresses. Two deployments follow from the two fine-tunes. The LEAN INT8 checkpoint with the 1% code is a security mode, surviving 300 flips in every seed and 500 in two of three, with its parity embedded in LOW weights at no added memory; the quantization-aware checkpoint in the 8/6/4/3-bit format is a storage mode, 2.4 times smaller in weight payload and, by the access-energy model of Section IV, an estimated 59% cheaper to fetch, and with the sign-plane code it is a security mode too at 3.63 to 4.09 bits per weight. The two checkpoints share the baseline, the thresholds and the λ\lambda term; a device picks the mode that fits its memory.

b) Effect on language models: This part is a preliminary extension, smaller in scale than the vision results: three models of 0.5 to 1.5 billion weights, one epoch on one million WikiText-2 tokens, perplexity as the metric. Weight sensitivity in transformers is skewed in the same way, and Attention-Breaker [21] and SilentStriker [22] exploit it with a few dozen flips. Table XII and Figure 11 apply the same objective to the linear layers of Qwen2-0.5B, Llama-3.2-1B and Qwen2-1.5B, fine-tuned for one epoch on one million WikiText-2 tokens against a control fine-tuned identically with λ=0\lambda= 0; HIGH and MID are measured against the control’s top-1% and top-9% thresholds, as for the CNNs. The HIGH set shrinks 1.7 to 5.3 times (5.3 is Llama at λ=10\lambda= 10, at a 2.5 perplexity cost; 1.7 to 3.4 at the λ\lambda we would deploy, Figure 8c) and the MID set by about the same factor, and the cost is 0.6 perplexity at the λ\lambda we would deploy (λ=10\lambda= 10 on Qwen2, λ=3\lambda= 3 on Llama; the penalty is summed over weights, so a larger model needs a smaller coefficient for the same pressure). Post-training INT4 improves on the two smaller models, from 21.1 to 19.3 on Qwen2-0.5B and from 21.4 to 18.9 on Llama, for the same reason as in Section IV: the weights the penalty removes from the tail are the ones that set the quantization grid. Qwen2-1.5B shows the other side of the knob: the penalty is summed over 1.3 billion weights, so λ=3\lambda= 3 there is as strong as λ=10\lambda= 10 on Llama, the tail shrinks 3.4 times but perplexity rises by 2.1 and INT4 does not improve; a smaller λ\lambda is the right setting for a model of that size. Table XIII puts the three models under the same progressive sign-bit attack as the CNNs, with and without a 1% code chosen by each model’s own damage ranking. Without a code all three are destroyed within 10 flips. With the code the picture depends on size. On Llama-3.2-1B the LEAN model is worse than the control for the first 20

modelλ\lambdaHIGHMIDppl (fp32)ppl (INT4)
Qwen2-0.5B0 (control)1.00%8.0%13.8621.1
Qwen2-0.5B100.58%4.9%14.5119.3
Llama-3.2-1B0 (control)1.00%8.0%11.4721.4
Llama-3.2-1B30.43%4.1%12.0918.9
Llama-3.2-1B100.19%2.1%13.9619.3
Qwen2-1.5B0 (control)1.00%8.0%10.7813.3
Qwen2-1.5B30.29%2.6%12.9014.4

Table XII. LEAN on language models: high and mid shares against the control’s thresholds, and WikiText-2 perplexity at fp32 and after per-channel INT4 (lower is better).

Three bar charts showing HIGH bin, MID bin, and WikiText-2 perplexity for Qwen2-0.5B, Llama-1B, and Qwen2-1.5B

Figure 11. LEAN on three language models (Table XII in bars): (a) HIGH and (b) MID shares against the control’s thresholds, with the reduction factor; (c) WikiText-2 perplexity of the control and the LEAN model, with the cost. λ=10\lambda= 10 for Qwen2-0.5B, 3 for the other two.

modelcodeclean102050100
Llama-3.2-1B controlnone12.2deaddeaddeaddead
Llama-3.2-1B LEAN λ=3\lambda=3none12.6deaddeaddeaddead
Llama-3.2-1B control1%12.212.514.4122210
Llama-3.2-1B LEAN λ=3\lambda=31%12.620.828.541.961.4
Qwen2-0.5B controlnone14.3deaddeaddeaddead
Qwen2-0.5B LEAN λ=10\lambda=10none14.7deaddeaddeaddead
Qwen2-0.5B control1%14.329.5deaddeaddead
Qwen2-0.5B LEAN λ=10\lambda=101%14.715.0115deaddead
Qwen2-1.5B controlnone11.0deaddeaddeaddead
Qwen2-1.5B LEAN λ=3\lambda=3none13.0deaddeaddeaddead
Qwen2-1.5B control1%11.011.011.2deaddead
Qwen2-1.5B LEAN λ=3\lambda=31%13.013.116.3deaddead

Table XIII. PRELIMINARY: PROGRESSIVE SIGN-BIT ATTACK ON INT8 LANGUAGE MODELS, PERPLEXITY AFTER nn FLIPS, WITHOUT AND WITH A 1% CODE BY EACH MODEL’S OWN RANKING; “DEAD” IS ABOVE 10610^{6}.

flips (21 against 12.5 perplexity at 10 flips) and 3 times better from 50 flips on (42 against 122 at 50, 61 against 210 at 100), which is the regime a coded attacker has to work in. On the smaller Qwen2-0.5B a 1% code is not enough for either model beyond 20 flips; LEAN is at its clean perplexity after 10 flips where the control has already doubled, and still usable at 20 where the control is gone. On Qwen2-1.5B, where λ=3\lambda= 3 was already too strong, the attack shows the same thing: both models hold 20 flips with the code and both are gone at 50, and the LEAN model is slightly worse at 20 (16.3 against 11.2), so an over-strong λ\lambda buys nothing. Where λ\lambda is set so that perplexity moves by less than one point, the gain is the same kind as on the CNNs, the code removes the weights the attacker wants and λ\lambda thins what is left.

c) Limitations: LEAN needs training data and about 1.5 times the cost of a plain fine-tune, and the low-bit format needs a second one. Our adaptive attacker is white-box and knows the code, but it is greedy; a search that plans across codewords could do better, and the protection rests on the interleaving assumption of Section II. Most entries are single runs; the storage comparison and the ResNet-18 CIFAR-10 reference model have three seeds (the experimental setup of Section IV), and the accuracy differences of 0.1 to 0.5 points in Table I should be read with that in mind; the 500-flip outcome of the coded ResNet-18 sits at a threshold and varies with the training seed, while the 300-flip outcome does not. The evaluation is of adaptive flips; random faults are measured at one bit error rate only (Table VII). The robust-training baselines of Table V were fine-tuned with our ten-epoch schedule rather than their original recipes. The tail term is not a defense on its own: Table IV shows that alone it lowers the gradient without lowering the damage, which is why LEAN always pairs it with the range term, and why the real-flip check should accompany any proxy of this kind. The ImageNet models cost more bits in the low-bit security mode: ResNet-50 needs the 3% tier and the t=79t = 79 code, 4.09 bits per weight, to hold 500 flips, and ResNet-18 at that cost is still 14 points below clean. A λ\lambda fine-tune that targets the deployed format more directly, or a wider tier chosen during training rather than at packing, are the natural next steps. In this paper λ\lambda is the only knob, and it moves the whole tail at once. The objective has other parameters that could shape the distribution more selectively and that we have not explored: the tail fraction that defines the set TT (0.5 to 2% gives the same HIGH bin, but a per-bin target could act on MID and HIGH separately), the relative weight β\beta of the range and tail terms, the clipping quantile q99q_{99}, and the learning rate and number of epochs, which Table II already shows to set how far the tail moves for a fixed λ\lambda. A family of such controls, each tied to one part of the distribution, is the natural extension of the one-knob procedure presented here.

LEAN sits between three lines of work, and takes a different position from each.

a) Robust training against bit flips: Piecewise clustering [8] and quantization-range learning (WCAT, [9]) make the stored values of a network tolerant to flips by constraining weight magnitudes; Forget-and-Rewire [10] reroutes the most important connections so that single flips matter less; Hessian-aware training [11] regularizes curvature. All of these change the network so that a flip hurts less, and all of them, like LEAN, are at chance within 50 flips of an adaptive attacker when no code is present (Table V). WCAT is the closest to the range term of LEAN, since both bound the per-layer quantization range; Table V separates them under the same code (WCAT, one run, 75% at 500 flips; LEAN 94% in two of three seeds), and the real-flip rows of Table IV show what the tail term adds on top of range control. LEAN differs from all four in what it optimizes: not robustness to flips in general but the size of the set that a selective code has to cover, measured in the storage format.

b) Gradient-norm regularization: PtailP_{\mathrm{tail}} belongs to an old family: double backpropagation [23] penalizes the input gradient, gradient-norm regularization and sharpness-aware minimization [24] penalize the weight gradient or the loss in a neighbourhood. LEAN differs in two details that the ablation shows are the ones that matter for stored weights: the penalty is top-kk, so only the tail is pushed, and it is weighted by the storage-format step Δℓ\Delta_{\ell}, which ties it to the range term. A plain gradient-norm fine-tune under the same budget is the natural further baseline, and Table IV and Table V include it: at the largest coefficient the accuracy tolerates it leaves the tail at 0.64% and the coded model breaks at 200 flips like the normal one, so the tail-selective, storage-weighted form is not a generic gradient smoother.

c) Selective and in-place protection: Selective and embedded codes such as in-place zero-space protection [7] and E3-CODE [13] reduce the parity cost of error correction by protecting only the weights that matter or by hiding the parity in the weights themselves. Our contribution is upstream of them: it makes the set they protect smaller, and is compatible with any of them. Detection schemes such as RADAR [25] and Aegis [26] catch flips at run time instead of correcting them; a correcting code was chosen here because it needs no response path, but the smaller sensitive set would lower the cost of a checksum in the same way.

d) Low-bit quantization: LSQ [14] learns the quantizer step; KURE [15] shapes the weight distribution for robustness across bit widths; HAWQ [27] assigns bit widths per layer from Hessian information; SpQR [28] and SqueezeLLM [29] keep a small set of outlier or sensitive weights at high precision inside a low-bit model, which is per-weight precision of the same kind as our tiers. Our tiered format assigns bit widths per weight by stored-format sensitivity, and its difference from those methods is upstream: λ\lambda makes the high-precision set smaller before the format is chosen, and the per-channel INT4 we use as the LLM baseline is a deliberately simple one (GPTQ [30] and AWQ [31] would lower both rows of Table XII). At a uniform bit width LEAN matches LSQ rather than beating it (Table VIII); the gain comes from the ordering, which the shrunken sensitive set makes affordable. Generating weights from a compact seed [32] is an orthogonal route to storage efficiency that a reshaped sensitivity distribution does not preclude.

e) Attacks on large language models: AttentionBreaker [21] and SilentStriker [22] carry gradient-guided bit flips to transformers, where a few dozen flips suffice. The sensitivity structure they exploit is the one LEAN shapes, and § 5 reports a first adaptation.

Conclusion

The weights of a trained neural network model are not equally sensitive with respect to its functional behavior. The shape of their sensitivity distribution sets what a bit-flip attacker needs, what a code must cover, and how many bits each weight has to keep. Existing methods do not optimize the shape of the weight sensitivity distribution. We have presented LEAN that allows us to modulate the weight sensitivity distribution, leading to varying numbers of weights in different weight sensitivity bins. For example, it helps greatly reduce the number of weights in the highest sensitivity bins by moving them to lower sensitivity bins. Such a shift in weight sensitivity distribution may result in lower cost for protection against bit-flip attacks as well as increased efficiency in model weight storage (e.g., sensitivity-guided variable precision quantization of weights) and energy/performance for computing activation at each node.

One composite regularizer on the two factors of the damage, the flip step and the gradient, controlled by a single coefficient, moves the sensitive tail 5 to 13 times lower on nine vision models for an accuracy change under 0.5 points and 1.7 to 5.3 times on three language models, and the amount follows λ\lambda monotonically. The term works as a ten-epoch fine-tune or in the last epochs of training, and it stops at a floor set by the model itself, its per-layer outlier ratios and the curvature around its early layers. The reshaped distribution then does the rest. A 1% code holds ResNet-18 through 300 to 500 adaptive flips where three of four published defenses fail by 300 and WCAT falls to 75% at 500, the parity for a fixed guarantee drops 3.7 times, a 3.29-bit sensitivity-ordered format, which the small tail makes affordable, stores the ImageNet ResNet-50 weight payload 2.4 times smaller than INT8 above the accuracy of LSQ, and a small interleaved code lets the low-bit models survive too. The targeted attack on the ImageNet ResNets, which moves through weights the mixed-data sensitivity does not rank, is the case the current objective does not reach, and shaping the distribution under a class-aware sensitivity is the next step, as is control through parameters other than λ\lambda, such as a per-bin target or the balance of the two terms, so that different parts of the distribution can be moved independently.

References

References

  1. [1]Y. Kim, R. Daly, J. Kim, C. Fallin, J. H. Lee, D. Lee, C. Wilkerson, K. Lai, and O. Mutlu, “Flipping bits in memory without accessing them: An experimental study of DRAM disturbance errors,” in Proceedings of the 41st International Symposium on Computer Architecture (ISCA), 2014, pp. 361–372.
  2. [2]F. Yao, A. S. Rakin, and D. Fan, “DeepHammer: Depleting the intelligence of deep neural networks through targeted chain of bit flips,” in Proceedings of the 29th USENIX Security Symposium, 2020, pp. 1463–1480.arxiv.org/abs/2003.13746
  3. [3]A. S. Rakin, Z. He, and D. Fan, “Bit-flip attack: Crushing neural network with progressive bit search,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 1211–1220.arxiv.org/abs/1903.12269
  4. [4]A. S. Rakin, Z. He, J. Li, F. Yao, C. Chakrabarti, and D. Fan, “T-BFA: Targeted bit-flip adversarial weight attack,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 11, pp. 7928–7939, 2022.
  5. [5]D. Stutz, N. Chandroamorthy, M. Hein, and B. Schiele, “Bit error robustness for energy-efficient DNN accelerators,” in Proceedings of Machine Learning and Systems (MLSys), vol. 3, 2021, pp. 569–598.arxiv.org/abs/2006.13977
  6. [6]H. Rahaman, S. Paria, A. Chatterjee, and S. Bhunia, “Evolving landscape of attacks on AI hardware and robust defenses,” in 2026 27th International Symposium on Quality Electronic Design (ISQED). IEEE, 2026, pp. 1–6.DOI
  7. [7]H. Guan, L. Ning, Z. Lin, X. Shen, H. Zhou, and S.-H. Lim, “In-place zero-space memory protection for CNN,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 32, 2019.arxiv.org/abs/1910.14479
  8. [8]Z. He, A. S. Rakin, J. Li, C. Chakrabarti, and D. Fan, “Defending and harnessing the bit-flip based adversarial weight attack,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 14 095–14 103.
  9. [9]K. Chitsaz, G. Mordido, J.-P. David, and F. Leduc-Primeau, “Training DNNs resilient to adversarial and random bit-flips by learning quantization ranges,” Transactions on Machine Learning Research, 2023.
  10. [10]N. Nazari, H. M. Makrani, C. Fang, H. Sayadi, S. Rafatirad, K. N. Khasavneh, and H. Homayoun, “Forget and rewire: Enhancing the resilience of transformer-based models against bit-flip attacks,” in Proceedings of the 33rd USENIX Security Symposium, 2024.
  11. [11]T. H. Pranto, S. Kim, L. Chen, and S. Hong, “Hessian-aware training for enhancing DNNs resilience to parameter corruptions,” arXiv preprint arXiv:2504.01933, 2025.
  12. [12]H. Rahaman, Q. Xu, Z. Haider, P. Chakraborty, S. Bhunia, and F. Suya, “(A)iSym: Retrieval for machine learning infrastructure,” arXiv preprint arXiv:2607.17550, 2026.
  13. [13]V. N. Kamineni, H. Rahaman, O. Sen, B. Chatterjee, S. Bhunia, and R. Ewetz, “E3-CODE: Embedded and efficient error-correcting code for error-resilient neural networks,” 2026.
  14. [14]S. K. Esser, J. L. McKinstry, D. Bablani, R. Appuswamy, and D. S. Modha, “Learned step size quantization,” in International Conference on Learning Representations (ICLR), 2020.
  15. [15]M. Shkolnik, C. Chmiel, R. Banner, G. Shomron, Y. Nahshan, A. Bronstein, and U. Weiser, “Robust quantization: One model to rule them all,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 33, 2020, pp. 5308–5317.arxiv.org/abs/2002.07686
  16. [16]K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778.arxiv.org/abs/1512.03385
  17. [17]K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in International Conference on Learning Representations (ICLR), 2015.
  18. [18]A. Krizhevsky, “Learning multiple layers of features from tiny images,” University of Toronto, Tech. Rep., 2009.
  19. [19]J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009, pp. 248–255.
  20. [20]S. Han, X. Liu, H. Mao, J. Pu, A. Pedram, M. A. Horowitz, and W. J. Dally, “EIE: Efficient inference engine on compressed deep neural network,” in Proceedings of the 43nd International Symposium on Computer Architecture (ISCA), 2016, pp. 243–254.arxiv.org/abs/1602.01528
  21. [21]S. Das, S. Bhattacharya, S. Kundu, S. Kundu, A. Menon, A. Raha, and K. Basu, “AttentionBreaker: Adaptive evolutionary optimization for unmasking vulnerabilities in LLMs through bit-flip attacks,” arXiv preprint arXiv:2411.13757, 2024.
  22. [22]H. Xu, Q. Peng, J. Shi, H. Zheng, Y. Li, and C. Zhuo, “SilentStriker: Toward stealthy bit-flip attacks on large language models,” in Advances in Neural Information Processing Systems (NeurIPS), 2025.arxiv.org/abs/2509.17371
  23. [23]H. Drucker and Y. Le Cun, “Improving generalization performance using double backpropagation,” IEEE Transactions on Neural Networks, vol. 3, no. 6, pp. 991–997, 1992.pubmed.ncbi.nlm.nih.gov/18276495
  24. [24]P. Foret, A. Kleiner, B. Mobahi, and B. Neyshabur, “Sharpness-aware minimization for efficiently improving generalization,” in International Conference on Learning Representations (ICLR), 2021.arxiv.org/abs/2010.01412
  25. [25]J. Li, A. S. Rakin, Z. He, D. Fan, and C. Chakrabarti, “RADAR: Run-time adversarial weight attack detection and accuracy recovery,” in Design, Automation and Test in Europe Conference (DATE), 2021.DOI
  26. [26]J. Wang, Z. Zhang, M. Wang, H. Qiu, T. Zhang, Q. Li, Z. Li, T. Wei, and C. Zhang, “AEgis: Mitigating targeted bit-flip attacks against deep neural networks,” in Proceedings of the 32nd USENIX Security Symposium, 2023.arxiv.org/abs/2302.13520
  27. [27]Z. Dong, Z. Yao, A. Gholami, M. W. Mahoney, and K. Keutzer, “HAWQ: Hessian Aware quantization of neural networks with mixed-precision,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 293–302.arxiv.org/abs/1905.03696
  28. [28]T. Dettmers, R. Shvets, S. V. Eziagarian, D. Kuznedelev, E. Frantar, S. Ashkboos, A. Borzunov, T. Hoefler, and D. Alistarh, “SpQR: A sparse-quantized representation for near-lossless LLM weight compression,” arXiv preprint arXiv:2306.03078, 2023.arxiv.org/abs/2306.03078
  29. [29]S. Kim, C. Hooper, A. Gholami, Z. Dong, X. Li, S. Shen, M. W. Mahoney, and K. Keutzer, “SqueezeLLM: Dense-and-sparse quantization,” arXiv preprint arXiv:2306.07629, 2023.arxiv.org/abs/2306.07629
  30. [30]E. Frantar, S. Ashkboos, T. Hoefler, and D. Alistarh, “GPTQ: Accurate post-training quantization for generative pre-trained transformers,” in International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2210.17323
  31. [31]J. Lin, J. Tang, H. Tang, S. Yang, W.-M. Chen, W.-C. Wang, G. Xiao, X. Dang, C. Gan, and S. Han, “AWQ: Activation-aware weight quantization for on-device LLM compression and acceleration,” in Proceedings of Machine Learning and Systems (MLSys), 2024.DOI
  32. [32]H. Rahaman, A. Chatterjee, and S. Bhunia, “Secure and storage-efficient deep learning models for edge AI using automatic weight generation,” arXiv preprint arXiv:2507.06380, 2025.arxiv.org/abs/2507.06380

Paper details

vlnc:W_dz6gcqmx
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents