betaFeedback

WorksComputer Science

SHq: Learnable Grids for LLM Weight Quantization

Abstract

Block-scaled lookup-table (LUT) formats, which represent weights as indices into a small value set (the grid), have become standard for low-bit LLM quantization, ranging from NF4 and BOF4 to the 3-bit LUT instruction in NVIDIA Rubin. While these formats leave the grid definition flexible, existing implementations rely on fixed grids derived either from numerical data types (INT, FP) or assumed weight distributions (NF, BOF). We demonstrate that no single fixed grid is universally optimal; the best fixed grid varies with bit-width: BOFS at 2 bits, NF at 4 and 5 bits, and INT at 6 bits. We introduce SHq, a closed-form family of grids parameterized by two critical degrees of freedom: a companding strength α that interpolates level spacing from uniform to floating-point-like, and an offset c that introduces asymmetry. This offset allows the sign bit of the FP8 block scale, otherwise unused, to select between the grid and its mirror image, up to doubling the number of distinct values available per block. SHq modifies only the lookup table contents and serves as a drop-in replacement for the grid in any such format. Across five models ranging from 3B to 35B parameters, including a mixture-of-experts architecture, evaluated at 2-6 bits using RTN and GPTQ, SHq achieves the lowest KL divergence in every setting at 3 and 4 bits and in most settings at 2, 5 and 6 bits. It outperforms the best fixed grid by up to 34% and remains superior under mixed-precision configurations. At 4.5 bits per weight, the SHq format reduces the KL divergence of NVFP4 by 41-67% while improving downstream accuracy. Grid families incorporating both spacing and offset parameters perform comparably, whereas those lacking an offset exhibit clearly higher error, confirming that these two degrees of freedom drive the observed gains. We therefore propose SHq as the standard grid for block-scaled lookup-table formats.

Introduction

Weight quantization has become a standard way to reduce the memory footprint and serving cost of large language models (LLMs) [12, 21, 34, 13]. Specifically, microscaled quantized formats [11, 43] (NVIDIA et al. 2025) have become very popular: a small block of weights, typically 16 or 32, shares a scale, and each normalized weight is stored as an index into a grid of 2q2^q values. When that index is decoded through a small lookup table [31], the grid need not be an integer or floating-point grid at all, which raises a basic question: which 2q2^q values should the table hold? Lookup-table formats now range from NF4 [13] and BOF4 [4] to the 3-bit lookup-table instruction of NVIDIA Rubin [42]; while these tables can hold arbitrary values, each format fixes them in advance.

Optimal grids. Finding good grids for a given distribution is an old problem. The Lloyd–Max conditions are necessary conditions for the MSE-optimal scalar quantizer of a known source [36, 38]. In machine learning, ZipML was among the first to compute variance-optimal quantization levels directly from the data distribution [54]. For LLM weights, this line of work has produced multiple families of grids: NF4 places its levels at quantiles of a normal distribution [13], Student Float derives its levels from a heavier-tailed Student-tt distribution [14], BOF4 derives Lloyd-style codebooks that minimize the blockwise reconstruction error under absmax scaling [4], and SqueezeLLM runs sensitivity-weighted kk-means for each output channel [32]. Other grids are inherited from number formats, such as INT and the E2M1 grid of NVFP4 (NVIDIA et al., 2025). A related line of work compares integer and floating-point formats per layer, or selects between them [50, 55], and recent formats go further, choosing between two grids for every block [9, 19]. In most of these formats, the grids are fixed before the model is seen.

Limits of current approaches. Fixing the grid in advance has three weaknesses. First, the optimal grid depends on the bit-width: in our experiments, the best fixed grid is BOFS at 2 bits, NF at 4 and 5 bits, and, in half of the settings, INT at 6 bits (§ 4), so no single fixed grid is a safe choice. Second, the optimal grid depends on the surrounding format as much as on the weights: the encoding of the block scale in E4M3 and the search over its values shift the error-minimizing grid (§ 2), whereas NF and BOF are derived from an assumed weight distribution under plain absmax scaling. Third, and more fundamentally, grids are fit to mean-squared error, possibly weighted by activation or curvature statistics; however, a lower layerwise error, weighted or not, does not necessarily yield a lower divergence of the model’s output ([52, 4]). Learning the grid for every layer or channel ([32]) removes the distributional assumption, but a learned codebook has 2q2^q free levels per layer and is in practice fit to a layerwise proxy.

This paper. We take a different approach: rather than fixing the grid in advance or fitting it to a layerwise proxy, we select it from a two-parameter family against the end-to-end objective. We use a parametrized grid family, SHq(α,c)\mathrm{SH}_q(\alpha,c), whose 2q2^q levels are obtained by sampling sinh⁡(αx)\sinh(\alpha x) on a uniform lattice and shifting by an offset cc. Each grid is described by two continuous parameters rather than 2q2^q free levels, so the space of grids is small enough to be searched by its effect on the model’s output (Figure 1). For each model and bit-width, a scan selects (α,c)(\alpha,c) by the KL divergence between the next-token distributions of the quantized and original models on calibration text, and the model is then quantized in that format with RTN or GPTQ. A fixed 4-bit member of the family, with α=1.5\alpha=1.5 and c=−0.02c=-0.02, serves as one of the read-only codebooks of a methodology for per-layer lookup-table hardware ([31]); here we analyze the family as a whole, select its parameters per model and width, and evaluate it end to end against fixed grids. Since only the contents of the lookup table change, SHq\mathrm{SH}_q requires no change to storage or decoding, and we propose it as the default grid of block-scaled lookup-table formats.

Figure 1. Fixed grid: the lookup table holds a grid derived once for an assumed weight distribution (NF, BOF). Tunable grid (this work): it holds SHq(α,c)\mathrm{SH}_q(\alpha,c), with (α,c)(\alpha,c) selected for each model by the calibration KL divergence. Levels at 3 bits before E4M3 rounding; SH parameters as selected for Qwen3-8B.

The SH grid. SH is a companding quantizer in the classical sense ([23]): uniform quantization applied after a smooth compressive map (here sinh⁡\sinh), so that its levels are dense near zero, where most normalized weights lie, and sparse in the tails (Figure 2). For c=0c=0, the parameter α\alpha sets how strongly it compands, interpolating between an almost-uniform, INT-like grid as α→0\alpha\to0 and increasingly geometric, FP-like spacing as α\alpha grows and sinh⁡\sinh becomes exponential. The offset cc breaks the odd symmetry of sinh⁡\sinh, making the grid asymmetric. Like the grid pairs of recent multi-grid formats ([9, 19]), the otherwise idle sign bit of the E4M3 block scale then selects between two grids. For each grid and its mirror image. Their levels interleave: for the 3-bit grid selected for Qwen3-8B, the two provide 16 distinct values, against 8 for SH without an offset, 9 for INT3 and 13 for NF3, at no storage cost. BOF4-S uses the same mirroring, with the sign fixed by the largest weight of the block ([4]). All levels are snapped to FP8 (E4M3), so SH is decoded through a 2q2^q-entry lookup table exactly like NF or BOF, and needs no new arithmetic in hardware.

Figure 2. Construction of the 3-bit SH grid selected for Qwen3-8B (α=1.75\alpha= 1.75, c=−0.1875c = -0.1875; levels before E4M3 rounding). A negative block scale selects the mirror image −G-G, whose levels interleave with those of GG: 16 values per block, against 8, 9 and 13 for SH with c=0c = 0, INT3 and NF3.

Contributions. In summary:

  • We introduce SHq(α,c)\mathrm{SH}_{q}(\alpha, c), a two-parameter family of asymmetric companding grids that is FP8-representable and lookup-table decodable, and contains near-uniform and near-geometric grids as limiting cases. We show that its error-minimizing parameters depend on the encoding and search of the block scale as much as on the weights, and that its offset lets the sign of the block scale double the number of values available to each block, up to 2q+12^{q+1} (§ 2).

  • We show that these two degrees of freedom, level spacing and offset, account for the gains: two other families with both, NormalFloat with a free width and offset and an arctan compander, come within 3% of SH after GPTQ, whereas NormalFloat without an offset, with or without a free width, remains 8.9–16.7% above it, and removing the sign bit, on which the offset relies, raises the KL of SH by 21–38% at 3 and 4 bits (Appendices § 12 and § 11). A Lloyd codebook learned for every layer matches SH in calibration KL and is 6.5% lower after GPTQ on Llama-3.1-8B-Instruct, so SH is not optimal in an absolute sense; it exposes the two relevant degrees of freedom in closed form.

  • Across five models from 3B to 35B parameters, including a mixture-of-experts model, at 2 to 6 bits with RTN and GPTQ, SH attains the lowest held-out KL divergence in 45 of 50 settings, including all 20 at 3 and 4 bits, where it is 4.0–34.4% below the best fixed grid (Table 1); differences within the measured run-to-run variation of GPTQ are not interpreted (Appendix § 8). The expected acceptance rate follows the KL ordering in 47 of the 50 settings, and downstream accuracy agrees wherever differences exceed noise, except on the mixture-of-experts model at 2 bits after GPTQ (§ 4).

Llama-3.2-3B-Inst.Qwen3-8BLlama-3.1-8B-Inst.Qwen3.8-27BQwen3.6-35B-A3B
SHBOFSNFINTSHBOFSNFINTSHBOFSNFINTSHBOFSNFINTSHBOFSNFINT
2 bits (2.5 bpw incl. scale)
uniform RTN744.1939.81228.21209.7520.6561.8601.8618.2643.2819.01001.7955.4212.2242.2286.0273.8250.6289.7336.7328.7
uniform GPTQ577.0782.7813.6985.9379.8378.2432.5405.3509.0628.8727.8685.7194.5212.5249.6230.1208.2224.4263.6240.5
3 bits (3.5 bpw incl. scale)
uniform RTN94.45122.7112.2168.178.3187.2388.60109.690.82107.0105.2138.044.4448.2849.8358.0655.4063.7462.4577.77
uniform GPTQ82.66116.899.31159.361.9066.9467.6081.5181.4698.2595.94118.838.4540.0741.4046.7246.9249.4549.6357.83
4 bits (4.5 bpw incl. scale)
uniform RTN20.3627.8625.6035.5618.3522.8221.6128.8921.2126.0925.4734.2710.9912.4212.0315.2416.6819.6718.4824.26
uniform GPTQ16.9830.2925.9037.4814.4616.1415.7819.9419.8125.1325.6332.939.339.8110.1111.7414.4415.3415.1318.15
5 bits (5.5 bpw incl. scale)
uniform RTN5.687.166.679.915.366.556.088.355.926.876.708.963.193.513.434.277.287.897.489.17
uniform GPTQ4.627.856.5711.494.224.734.525.695.267.656.749.812.712.782.883.316.486.586.427.32
6 bits (6.5 bpw incl. scale)
uniform RTN2.172.742.462.592.172.752.462.542.272.802.542.581.361.711.471.324.484.824.714.61
uniform GPTQ1.762.812.282.931.732.061.911.791.992.622.343.331.161.331.181.094.114.364.194.06

Table 1. KL divergence (mn, lower is better) to the BF16 model on 500K held-out tokens, one format for all layers. Each width adds 0.5 bpw for the E4M3 block scale. Bold: best per model and row.

  • At 4.5 bits per weight, the SH4 format lowers the KL divergence of N VFP4 by 40.7–67.4% and attains higher average downstream accuracy in all six model-quantizer combinations evaluated, by 0.1–0.7 points. Much of the KL gap comes from the signed, searched block scale; under the same scale, SH4 lowers the KL of the best fixed grid by a further 4.6–34.4% (Table 3). When RCO (Helcig & Alistarh, [26]) allocates per-layer bit-widths, SH remains the lowest-KL grid after GPTQ (Table 2); tuning the grid for each layer adds little, so a single grid per model and width suffices (§ 4).

Table 2. Held-out KL (mn) of the 8B models with one code width for all layers (uniform) and with per-layer widths chosen by RCO at the same average width (mixed, one seed), on RTX A6000. Bold: best per model and row.

Qwen3-8BLlama-3.1-8B-Inst.
SHBOFSNFINTSHBOFSNFINT
3 bits
RTN uniform78.3187.2388.60109.690.82107.0105.2138.0
RTN mixed77.7980.2683.84101.185.1291.0996.23107.1
GPTQ uniform61.9066.9467.6081.5183.6698.1894.57119.4
GPTQ mixed60.4962.7764.2574.4678.1878.3985.6289.53
4 bits
RTN uniform18.3522.8221.6128.8921.2126.0925.4734.27
RTN mixed20.3619.2928.8324.6119.4020.4921.1024.98
GPTQ uniform14.4616.1415.7819.9419.8825.3925.6233.30
GPTQ mixed13.5514.5614.3717.6516.6618.3318.3122.96
5 bits
RTN uniform5.366.556.088.355.926.876.708.96
RTN mixed4.865.625.376.625.245.915.877.07
GPTQ uniform4.224.734.525.695.257.776.769.78
GPTQ mixed3.864.154.004.744.695.415.147.08

Table 2.

Table 3. 4-bit formats vs NVFP4, all at 4.5 bpw. NVFP4: E2M1 codes, unsigned E4M3 block scale without scale search, FP32 tensor scale (not counted). Bold: best per row.

ModelMethodNVFP4SHBOFSNFINT
Llama-3.2-3B-Inst.RTN52.1820.3627.8625.6035.56
Llama-3.2-3B-Inst.GPTQ52.0416.9830.2925.9037.48
Qwen3-8BRTN48.7018.3522.8221.6128.89
Qwen3-8BGPTQ32.6514.4616.1415.7819.94
Llama-3.1-8B-Inst.RTN52.2621.2126.0925.4734.27
Llama-3.1-8B-Inst.GPTQ52.9119.8125.1325.6332.93
Qwen3.8-27BRTN26.8810.9912.4212.0315.24
Qwen3.8-27BGPTQ18.249.339.8110.1111.74
Qwen3.6-35B-A3BRTN36.6416.6819.6718.4824.26
Qwen3.6-35B-A3BGPTQ24.3514.4415.3415.1318.15

Table 3.

The SH Grid

SHq(α,c)\mathrm{SH}_q(\alpha,c) (Figure 2) samples sinh on the uniform qq-bit lattice uk=−1+2k/(2q−1)u_k=-1+2k/(2^q-1), adds the offset cc, and normalizes the largest-magnitude level to 1:

yk=sinh⁡(αuk)+cmax⁡j∣sinh⁡(αuj)+c∣,uk=−1+2k2q−1.(1)y_k=\frac{\sinh(\alpha u_k)+c}{\max_j\left|\sinh(\alpha u_j)+c\right|},\qquad u_k=-1+\frac{2k}{2^q-1}. \tag*{(1)}

Each block of 16 weights stores indices into a lookup table holding these levels in E4M3, together with a signed E4M3 scale whose code and sign a per-block search selects (§ 3.1); a negative scale quantizes the block against the mirror image of the grid. The mirror of the grid is obtained by negating cc and is already available through the sign of the block scale, so c≤0c\leq0.

The grid SHq(α,c)\mathrm{SH}_q(\alpha,c) carries two parameters that act on different aspects of the quantization: the companding strength α\alpha sets the density of levels within the grid, and the offset cc breaks the symmetry between the grid and its reflection. We analyze each degree of freedom in turn, in the language of classical companding quantization, and show by measurement that these two degrees of freedom, which SHq\mathrm{SH}_q exposes in closed form, account for its performance: other families with a width and an offset perform comparably, whereas families without an offset remain clearly behind (Appendix § 12).

Companding. Without the offset, the 2q2^q levels of SHq(α,0)\mathrm{SH}_q(\alpha,0) are

yk=sinh⁡(αuk)sinh⁡α,uk=−1+2k2q−1,(2)y_k=\frac{\sinh(\alpha u_k)}{\sinh\alpha},\qquad u_k=-1+\frac{2k}{2^q-1}, \tag*{(2)}

for k=0,…,2q−1k=0,\ldots,2^q-1, where the points uku_k are uniformly spaced in [−1,1][-1,1]. Equivalently, the levels are uniformly spaced after the compressor y↦asinh⁡(ysinh⁡α)y\mapsto\operatorname{asinh}(y\sinh\alpha), as in a classical companding quantizer. The resulting level density,

λα(y)∝(1+y2sinh⁡2α)−1/2,(3)\lambda_\alpha(y)\propto\left(1+y^2\sinh^2\alpha\right)^{-1/2}, \tag*{(3)}

is nearly constant around the origin, decays as 1/∣y∣1/|y| towards the ends of the range, and is lower at the edge than at the center by a factor of cosh⁡α\cosh\alpha. The parameter α\alpha thus interpolates between uniform quantization (α→0\alpha\to0) and geometric, FP-like spacing (large α\alpha; Figure 2).

Under the high-resolution approximation, the mean-squared error of a scalar quantizer with level density λ\lambda on a source of density pp is proportional to ∫p/λ2 dy\int p/\lambda^2\,dy and is minimized by λ∝p1/3\lambda\propto p^{1/3} (Gersho & Gray, [23]). The SH density has exactly this form for a Student-tt source with two degrees of freedom, so the grid is asymptotically MSE-optimal for that source. More generally, because 1/λα21/\lambda_\alpha^2 is quadratic in yy, the high-resolution error of the family depends on the source only through its second moment σ2=E[y2]\sigma^2=\mathbb{E}[y^2]. Normalized on [−1,1][-1,1], the density is λα(y)=sinh⁡α/(2α1+y2sinh⁡2α)\lambda_\alpha(y)=\sinh\alpha/(2\alpha\sqrt{1+y^2\sinh^2\alpha}), so that ∫p/λα2 dy=(2α/sinh⁡α)2∫p(y)(1+y2sinh⁡2α) dy\int p/\lambda_\alpha^2\,dy=(2\alpha/\sinh\alpha)^2\int p(y)(1+y^2\sinh^2\alpha)\,dy, and the last integral equals 1+σ2sinh⁡2α1+\sigma^2\sinh^2\alpha:

MSE⁡(α)∝(αsinh⁡α)2(1+σ2sinh⁡2α).(4)\operatorname{MSE}(\alpha)\propto\left(\frac{\alpha}{\sinh\alpha}\right)^2\left(1+\sigma^2\sinh^2\alpha\right). \tag*{(4)}

After division by the block absmax, the weights of all layers share a single, approximately Gaussian shape; the heavy tails of the raw weights are block-to-block variation of the absmax, which the per-block scale absorbs. A single choice of α\alpha is therefore appropriate for the entire model by Eq. (4).

Dependence on the format

The minimizing value of α\alpha is determined by the discretizing format as much as by the weights. Minimizing the MSE of Eq. (4) on the empirical block histograms of both 8B models, with exact scales and unrounded levels, yields a smaller α\alpha at 3 bits than the format scan selects. Under a signed E4M3 block scale pinned to the block absmax, the per-layer error of Qwen3-8B is instead minimized at larger values of α\alpha. When the scale code and sign of every block are free to be chosen, as in the per-block scale search of § 3.1 but without the channel-norm weights, the minimum lies at α=1.75\alpha=1.75 in 249 of 252 layers (Figure 4), the value selected by the calibration-KL scan of § 3.4 for this model. Fixed codebooks such as NF and BOF are designed for other operating points and cannot shift their optimum in this way.

Figure 4. RTN weight error of the 252 linear layers of Qwen3-8B at 3 bits against α\alpha (c=−0.1875c=-0.1875), in % above each layer’s lowest error over the ten values. Dashed: the α\alpha chosen by the format scan.

Role of the offset

The E4M3 block scale carries a sign bit, so each block is quantized against either the grid GG or against its reflection −G-G. For a symmetric grid the two coincide and the sign bit is idle; the offset cc breaks this symmetry. A block can place its largest-magnitude weight at the longer end of the grid, and at the selected offset the levels of −G-G interleave with those of GG (Figure 2). For the 3-bit grid selected for Qwen3-8B, G∪(−G)G\cup(-G) contains 16 distinct values, against 8 for c=0c=0, 9 for INT3 and 13 for NF3. The per-block scale search selects the orientation minimizing the error weighted by the squared input-channel norms, so the additional resolution acts primarily on the weights of high-norm channels, which dominate that weighted error. NF and BOF are asymmetric as well, but by a fixed amount: NF with a free width remains 13.0–25.9% above SH in calibration KL, and a free offset closes 64–97% of this gap (Table 8). Without the sign, the held-out KL rises in every configuration of Appendix § 11, by at least 10%.

Qwen3-8BLlama-3.1-8B-Inst.
3 bits4 bits3 bits4 bits
Free width and offset
NF, free width and offset (NFOC)+0.4+5.0+4.8+4.9
Arctan compander (GC, β\beta=1)+1.6+5.5+2.3+6.8
Heavier-tail compander (GC, β\beta=0.25)+2.6+2.4+6.4
No offset
NF, free width (NFO)+13.0+19.3+25.9+25.1
NF+15.0+23.2+27.2+32.2
Learned codebooks
Lloyd per layer (DDW)−1.2+1.1−0.1+0.8
Lloyd per model (DDWG)+0.8+0.3+5.9+0.6
Lloyd per layer, unweighted (DDWU)+2.6+2.2+5.2+2.2

Table 8. Calibration KL of the best grid of each family with RTN, in % relative to SH (negative: lower than SH; −: not scanned).

Relation to fixed grids

In terms of Eq. (3), a grid is characterized by its level density λ\lambda and asymmetry. The baseline grids (§ 3.2) fix both parameters in advance, following one of four approaches:

  • Uniform spacing (INT). The level density is constant. This configuration is optimal for values distributed uni-

formly across the range {(citep}(Gersho & Gray, 1992)}; however, for bell-shaped weight distributions, it allocates an equal number of levels to sparse tails and dense centers.

  • Geometric spacing (floating point). Floating-point grids distribute levels geometrically {(citep}(Micikevicius et al., 2022; van Baalen et al., 2023)}, with λ∝1/∣y∣\lambda\propto1/\lvert y\rvert away from zero, ensuring that the relative rounding error is approximately constant across binades. E2M1 provides fine resolution near zero and coarser resolution in the tails; its symmetry implies that the sign of a scale factor carries no information.

  • Quantiles of a normal distribution (NF). Each level covers approximately equal probability mass under an assumed normal distribution, yielding λ∝p\lambda\propto p. Consequently, levels are densest near zero and more concentrated there than under the MSE-optimal density p1/3p^{1/3}. Since the grid includes 0 and ±1\pm1, one side contains one more level than the other.

  • Minimum MSE for Gaussian blocks (BOF, BOFS). The Lloyd–Max conditions yield the MSE-optimal grid for a specific source, Gaussian blocks normalized by their absmax; its level density approaches p1/3p^{1/3} at high resolution. In BOFS, negative levels terminate before −1-1, and the sign of the scale determines the orientation of each block, analogous to SH.

SH replaces these fixed choices with two parameters: companding strength α\alpha {(citep}(Gersho & Gray, 1992)} and offset cc. The parameter α\alpha continuously adjusts spacing from uniform (INT, α→0\alpha\to0) toward geometric (floating-point grids). Its density, given by Eq. (3), is constant as α→0\alpha\to0, approaches 1/∣y∣1/\lvert y\rvert for large α\alpha, and corresponds to the MSE-optimal density p1/3p^{1/3} of a Student-tt source for each α\alpha. The offset cc controls the asymmetry that NF, BOF, and BOFS fix a priori. Both parameters are optimized per model and width using calibration KL divergence (§ 3.4), allowing the density to be determined by the model’s output distribution rather than an assumed weight distribution. This enables SH to adapt to each width, where different fixed grids may be optimal (§ 4).

Role of the sinh form. Both adaptations analyzed above are available to any smooth two-parameter family; the measurements below attribute the performance to exactly these two degrees of freedom, which SHq_q exposes in closed form. At 3 bits, SH, the arctan compander, a compander with heavier tails, and NF with a free width and an offset all lie close to the per-layer Lloyd–Max grid in weight MSE, whereas NF with a free width but no offset lies clearly above it. In calibration KL, NF with width and offset is within 0.4–5.0% of SH and the two other companders within 1.6–6.8% (Table 8); after GPTQ (single runs), the former and the arctan compander remain within 3.0%, whereas NF without an offset remains 8.9–16.7% above SH (Table 9). A Lloyd codebook learned per layer matches SH in calibration KL and attains a lower KL after GPTQ on Llama-3.1-8B-Instr., so SH is not optimal in an absolute sense. Its value is practical: the levels admit a closed form; its two parameters correspond exactly to the two degrees of freedom measured above; the uniform and geometric grids arise as limiting cases; the levels are representable in an E4M3 lookup table; and the two-dimensional parameter space is small enough to be searched directly on the KL divergence of the full model.

Qwen3-8BLlama-3.1-8B-Inst.
SH grid of the other model−0.7−3.7
NF, free width and offset (NFOC)+2.3+0.2
Arctan compander (GC, β=1\beta=1)+3.0+1.6
Heavier-tail compander (GC, β=0.25\beta=0.25)+7.7+1.9
NF, free width (NFO)+10.4+16.7
NF (placement 23/42^{3/4})+8.9+14.0
Lloyd per layer (DDW)−1.2−6.5
Lloyd per model (DDWG)+2.8+5.2
Lloyd per layer, unweighted (DDWU)+1.5+1.5

Table 9. Held-out KL of the best 3-bit grid of each family after GPTQ, in % relative to SH on the same GPU (single runs).

Method

§ 4 reports configurations: a model, a grid family, an average code width, a quantizer and an allocation. A format scan fixes the free parameters of each family once per model and width, on calibration KL (§ 3.4); the model is then quantized in that format with RTN and with GPTQ, which yields the uniform-width models, or a mixed model takes each layer from the uniform-width model of its width (§ 3.5). Each configuration is scored by the expected acceptance rate and the KL divergence of the quantized model to the BF16 model, both measured on held-out text (§ 3.6). No choice behind Table 4, Table 1, Table 2 and Table 3 uses that text; earlier design decisions, such as the adoption of the per-block scale search, rest on experiments evaluated on its first 50K tokens.

Llama-3.2-3B-Inst.Qwen3-8BLlama-3.1-8B-Inst.Qwen3.8-27BQwen3.6-35B-A3B
SHBOFSNFINTSHBOFSNFINTSHBOFSNFINTSHBOFSNFINTSHBOFSNFINT
2 bits (2.5 bpw incl. scale)
uniform RTN70.4468.0063.9264.0375.4874.7073.9273.1973.3970.5667.4767.8884.1883.0881.6281.6882.7881.5180.0580.19
uniform GPTQ73.9671.2770.4767.7679.2279.1677.7078.3076.5674.3872.5473.2885.0984.2983.0183.6284.4683.7682.4483.12
3 bits (3.5 bpw incl. scale)
uniform RTN88.8587.4187.9285.2490.6990.0090.0688.7089.9288.9589.1287.3593.0592.7392.6291.9392.0191.3591.5190.43
uniform GPTQ89.9488.0088.9886.0291.9791.7191.6190.7990.9490.0090.1588.8593.8293.6493.5593.1092.8492.6192.6191.99
4 bits (4.5 bpw incl. scale)
uniform RTN94.8494.0194.2693.2195.5294.9595.1194.2995.2194.6594.7393.8696.6496.4196.4695.9895.6895.3195.4494.75
uniform GPTQ95.4693.9394.4493.2496.1995.9796.0095.4995.6595.0595.0094.3297.0496.9696.9296.6596.0995.9696.0095.59
5 bits (5.5 bpw incl. scale)
uniform RTN97.2896.9697.0796.4697.6097.3197.4196.9397.4797.2897.3196.8998.2198.1098.1297.8997.2097.0897.1696.84
uniform GPTQ97.6396.9497.1996.3197.9597.8397.8897.6097.7797.3197.4696.9498.4298.3898.3798.2497.4297.4097.4397.24
6 bits (6.5 bpw incl. scale)
uniform RTN98.3298.1298.2198.1798.4898.2898.3798.3398.4398.2698.3498.3298.8398.7198.7898.8497.8597.7597.7897.80
uniform GPTQ98.5398.1598.3498.1298.6898.5798.6198.6598.6098.4098.4898.2698.9698.8798.9399.0097.9797.9197.9497.98

Table 4. Expected acceptance rate (EAR, %, higher is better) to the BF16 model on 500K held-out tokens, one format for all layers. Each width adds 0.5 bpw for the E4M3 block scale. Bold: best per model and row.

Block-scaled lookup-table formats

Storage. Each row of a weight matrix is partitioned into blocks of g=16g = 16 consecutive weights. A qq-bit format stores, per weight wiw_i, an index kik_i into a lookup table LL of 2q2^q E4M3 values and, per block, a signed E4M3 scale ss, so that w^i=s[L[ki]]\widehat{w}_i = s[L[k_i]]; the code costs q+0.5q + 0.5 bits per weight. The lookup table (one per format for the whole model) and a per-layer exponent shift of the scales are not counted in the bit rate. A grid G={y0<⋯<y2q−1}G = \{y_0 < \cdots< y_{2^q-1}\} with max⁡k∣yk∣=1\max_k \lvert y_k\rvert= 1 is a unit codebook: its table stores the levels after a common scaling, rounded to E4M3. The LUT placement factor f∈{1,21/4,21/2,23/4}f \in\{1, 2^{1/4}, 2^{1/2}, 2^{3/4}\} sets that scaling; it shifts the levels against the binades of E4M3 and thereby changes the set of values that survives the rounding, so different placements realize different effective grids. In a separate experiment, storing the levels of SH in E4M4 to E4M7 instead of E4M3 changed the KL of both 8B models by −1.1% to +3.6%, without a consistent improvement.

Block scale, sign and the lever. The naive scale of a block maps its absmax to the largest-magnitude level of the table. A negative scale quantizes the block against the reflected grid −G-G (§ 2). The lever extends this choice to a search: it considers 25 multipliers of the naive scale, geometrically spaced from 0.6 to 1.4, in both signs, rounds each candidate to E4M3, and keeps the candidate minimizing ∑icni2(wi−w^i)2\sum_i c_{n_i}^2(w_i-\hat{w}_i)^2, where cnic_{n_i} is the RMS of the input channel of wiw_i on calibration text. ScaleSweep [35] searches NVFP4 block scales in a related way.

Baselines

We evaluate against baselines defined by existing quantization formats. Each baseline employs a fixed grid determined either by a specific number format or an assumption about weight distribution. With the exception of NVFP4, all baselines utilize the same lookup table, block scale, and lever as SH, ensuring that the grid structure is the sole variable under comparison.

INTqq [50] employs a two’s-complement integer grid {−2q−1,…,2q−1−1}\{-2^{q-1},\ldots,2^{q-1}-1\} with uniform spacing. We evaluate two variants: one using per-block mirroring under absmax scaling and another under rail scaling. For each configuration, we select the variant yielding lower calibration KL divergence.

E2M1, the element format for NVFP4 and OCP microscaling formats [43, 11], represents values {0,±0.5,±1,±1.5,±2,±3,±4,±6}\{0,\pm0.5,\pm1,\pm1.5,\pm2,\pm3,\pm4,\pm6\}. NVFP4 (NVIDIA et al., 2025) (4.5 bits per weight) combines E2M1 elements with an unsigned E4M3 scale per 16 weights and an uncounted FP32 tensor scale, consistent with NVIDIA’s ModelOpt export; it does not employ a lever.

NF [13] positions grid levels at quantiles of a normal distribution, ensuring that each level captures approximately equal mass from normally distributed weights.

BOF [4] utilizes the Lloyd–Max codebook [36, 38], which minimizes mean squared error (MSE). It is computed as the fixed point of Lloyd’s iteration on the exact density of normalized weights, where blocks of 16 Gaussian weights are divided by their absmax. BOFS, a signed-absmax variant [4], divides each block by its largest-magnitude weight (including sign), mapping this weight to +1+1.

Quantizers

All linear layers inside the decoder blocks are quantized, including the projections of the linear-attention blocks and of the routed experts (one matrix per expert); the embeddings, the output head and the MoE router remain in BF16, and activations are not quantized. RTN quantizes each layer independently: every block is rounded to its nearest level under the lever, with channel norms from one BF16 pass over 128 calibration windows. GPTQ [21] quantizes the decoder blocks in order, each with Hessians and channel norms from the outputs of the already quantized preceding blocks; the diagonal of the Hessian is damped by 1% of its mean, columns are processed in natural order in lazy batches of 128, and at the first column of each group of 16 the lever refits the scales and signs of the group’s blocks on the error-compensated weights.

Format selection on calibration KL

For each model and width, the format scan ranks the candidate formats by the KL divergence of the RTN model to the BF16 model on 64 calibration windows, both distributions coarsened to the 128 tokens most likely under the BF16 model plus one bucket for the rest. The scan picks SH’s (α,c)(\alpha,c) by a two-stage box search: a coarse lattice, grown while the best point lies on its edge, followed by a finer grid around the best point. SHq_q at its scanned (α,c)(\alpha,c), BOF, BOFS and NF are then scored at the four placement factors and INT as its two variants, and each family keeps its best format on these windows; the RTN and GPTQ configurations of a model use exactly these formats (Appendix § 9 lists the scanned SH parameters). To guard against drift between scan and configuration, every RTN configuration re-scores its calibration KL and is refused if the value differs from the scan’s by more than 1% or 2⋅10−42\cdot10^{-4} nats, whichever is larger.

Per-layer bit-width allocation

The uniform-width models of the two 8B models, at q=2,…,6q=2,\ldots,6, store the quantized weights of every layer, for RTN and GPTQ alike. The GPTQ weights come from runs on the RTX A6000 with the formats and calibration windows of Table 1: for Qwen3-8B the very runs of that table, and for Llama-3.1-8B-Instruct a second run. A mixed model reuses these stored layers: each layer takes the width of its uniform-width model, and no quantization is rerun, so a uniform-width model is exactly the assignment ql=qq_l=q for all ll. The target BB fixes the average code width ∑lnlql/∑lnl\sum_l n_lq_l/\sum_l n_l, with nln_l the number of weights of layer ll. RCO (Hellegouarch & Alistarh, 2026) optimizes a categorical distribution over the five bit-widths of each layer, constrained to assignments whose expected average width is BB: for each Gumbel sample it solves the multiple-choice knapsack that projects the sample onto this constraint, and back-propagates the KL divergence of the resulting assignment to the unquantized model on calibration text (600 steps, 4 Gumbel samples, one seed). Table 2 reports B=3B=3, 4 and 5.

Evaluation

Data. All text is FineWeb-Edu (sample-10BT), split by document: evaluation [0, 5000), validation [50000, 55000) (used only to tune RCO) and calibration from document 100000 on. Each split is tokenized as one stream and cut into consecutive 2048-token windows.

Held-out KL. Every KL in Table 1, Table 2, Table 3 and Table 5 is

GPTQ runH200L40SRTX A6000differ
H20062.6562.6962.92–
H200, lazy batch 6461.8961.8761.9741%
L40S66.0366.1266.1642%
RTX A600061.8461.8861.9042%

Table 5. One GPTQ configuration (Qwen3-8B, SH at 3 bits) from four runs that differ only in GPU type or lazy-batch width (rows), scored on three GPU types: KL (mn). Last column: weights that differ from the H200 run.

KL=1000N∑t=1N∑vpt(v)ln⁡pt(v)p^t(v)(mn),(5)\mathrm{KL} = \frac{1000}{N} \sum_{t=1}^{N} \sum_v p_t(v) \ln\frac{p_t(v)}{\hat{p}_t(v)}\quad(\mathrm{mn}), \tag*{(5)}

the full-vocabulary divergence of the next-token distribution p^\hat{p} of the quantized model from that pp of the BF16 model, in millinats, over the NN next-token positions of the first 500K evaluation tokens (245 windows). Both models compute the output head and the softmax in FP32. The Llama models get a BOS token prepended to every window; the Qwen tokenizers have no BOS.

Expected acceptance rate. Table 4 reports, on the same positions and FP32 logits, the top-10 expected acceptance rate [27]

EAR=100N∑t=1N∑v∈Ttmin⁡(pˉt(v),p^ˉt(v))(%),(6)\mathrm{EAR} = \frac{100}{N} \sum_{t=1}^{N} \sum_{v \in T_t} \min\left(\bar{p}_t(v), \bar{\hat{p}}_t(v)\right)\quad(\%), \tag*{(6)}

where TtT_t holds the ten tokens most likely under the BF16 model and pˉt\bar{p}_t and p^ˉt\bar{\hat{p}}_t are the two distributions renormalized over TtT_t. In speculative decoding it is the acceptance probability of a token proposed by the BF16 model, restricted to these ten tokens; higher is better, and 100% corresponds to identical distributions.

Statistics and hardware. Every configuration is a single run. Two configurations are compared by their relative difference in KL, or, for EAR, by their difference in percentage points; the 95% interval of a difference comes from 10,000 bootstrap resamples of the 245 windows, and the interval of a difference between two such differences from the same resamples. A difference whose 95% interval includes zero is not interpreted. Reruns with the same code, inputs and GPU type are bit-identical, and scoring the same weights on a different GPU type changes the KL only little. GPTQ is an exception: it is sensitive to small numerical differences, and on a different GPU type, or on the same GPU with a different lazy-batch width (which only reorders floating-point sums), it returns a different quantization (Appendix § 8). Differences in KL below about 3% between single configurations are therefore not interpreted, and neither are differences in EAR within the spread of two such GPTQ runs: 0.19 percentage points at 2 and 3 bits and 0.02 at 4 to 6 bits on Llama3.1-8B-Instruct. Except for Llama-3.2-3B-Instruct, which ran entirely on RTX 3090, the format scans ran on H200 (Qwen3-8B, Qwen3.8-27B), L40S (Llama-3.1-8B-Instruct) and RTX A6000 (Qwen3.6-35B-A3B, except at 2, 5 and 6 bits, which ran on RTX 3090); the GPTQ configurations on RTX A6000 (Qwen3-8B, Qwen3.6-35B-A3B) and L40S (Llama-3.1-8B-Instruct, Qwen3.8-27B); and the RTN configurations, the layer databases of the mixed models and RCO on RTX A6000.

Results

We evaluate SH against the fixed grids NF, BOFS, and INT on Llama-3.2-3B-Instruct, Qwen3-8B, Llama-3.1-8B-Instruct, Qwen3.8-27B, and the mixture-of-experts model Qwen3.6-35B-A3B. Experiments are conducted at code widths of 2 to 6 bits, applying a single format to all linear layers. Each model is quantized using both RTN and GPTQ (§ 3.3). All formats incorporate an additional 0.5 bpw for the E4M3 block scale, ensuring that grids of identical width have equal size. Table 1 reports held-out KL divergence relative to the BF16 model; Figure 3 presents accuracy on six downstream benchmarks; Table 2 details KL divergence when RCO optimizes per-layer widths; and Table 3 compares performance with NFP4 at 4.5 bpw. All results are from single runs. We do not interpret KL differences below 3%, a magnitude attributable to GPTQ run-to-run variation (Appendix § 8), or differences whose 95% bootstrap confidence interval includes zero (§ 3.6).

Figure 3. Average accuracy (%) over six downstream benchmarks after RTN with one format for all layers; dashed line: BF16 model. The tasks, per-task accuracies and the GPTQ results are in Appendix G.

Held-out KL. Table 1 reveals three primary patterns. First, SH achieves the lowest KL in 45 of the 50 settings where all four grids are available, including all 20 settings at 3 and 4 bits, where it outperforms the best fixed grid by 4.0–34.4%. The average margin over the best fixed grid is 14%, 11%, and 14% at 2, 3, and 4 bits, respectively, decreasing to 11% and 7% at 5 and 6 bits. Three of the five exceptions fall within GPTQ run-to-run variation: BOFS on Qwen3-8B at 2 bits (0.4% below SH), and NF and INT on Qwen3.6-35B-A3B at 5 and 6 bits (0.9% and 1.2%, respectively). The other two are INT at 6 bits on Qwen3.8-27B, 3.3% below SH under RTN and 5.6% below SH under GPTQ. Second, no single fixed grid dominates across all widths: BOFS is the best fixed grid in all ten settings at 2 bits; NF leads in 17 of 20 settings at 4 and 5 bits; and INT leads in five of ten settings at 6 bits. The parameters selected for SH vary across models and widths (Appendix § 9). Third, the advantage of SH persists under both quantizers. GPTQ reduces the KL of SH by 7–27% relative to RTN, maintaining SH as the lowest-KL method in 21 of 25 GPTQ settings, compared to 24 of 25 under RTN. GPTQ narrows the margin for Qwen3-8B, Qwen3.8-27B, and Qwen3.6-35B-A3B (to at most 8.3%, 8.5%, and 7.2%, respectively) but widens it for both Llama models, from 11.8–20.8% under RTN to 16.8–34.4% on Llama-3.2-3B-Instruct and from 10.5–21.5% to 15.0–21.9% on Llama-3.1-8B-Instruct. The expected acceptance rate (EAR; Appendix § 7) matches the KL ordering in 47 of the 50 settings, and SH exhibits the highest EAR in 46 of them.

Downstream accuracy. Figure 3 displays average accuracy for RTN models across six benchmarks; Appendix § 13 lists individual tasks and GPTQ results. Under both quantizers, all grids remain within 2.1 points of BF16 at 3 bits and within 0.9 points at 4 bits. At 4.5 bpw, SH4 outperforms NVFP4 on both 8B models by 0.4 and 0.5 points after GPTQ, and by 0.2 and 0.7 points under RTN, while remaining within 0.3 points of BF16; the difference is significant for Llama-3.1-8B-Instruct. On Qwen3.6-35B-A3B, SH4 exceeds NVFP4 by 0.1 points under RTN and 0.4 points after GPTQ, which, again, and stays within 0.1 points of BF16. At 3 bits, SH achieves the highest average accuracy on both 8B models under both quantizers, exceeding the next-best grid by 0.1–0.4 points, though no such gap is statistically significant. On Qwen3.6-35B-A3B, BOFS exceeds SH by 0.4 points (0.5 under RTN), a difference within noise. At 2 bits, the grids diverge. On Llama-3.1-8B-Instruct, SH leads BOFS by 1.5 points after GPTQ and 1.6 under RTN, and NF/INT by 2.8–4.3 points; all differences are significant. On Qwen3-8B, BOFS exceeds SH by 0.6 points (0.5 under RTN), within noise. On Qwen3.6-35B-A3B, BOFS and INT exceed SH by 1.3 and 1.2 points after GPTQ, respectively, a significant difference despite SH having lower KL (208.2 vs. 224.4 and 240.5 mn); under RTN, SH and BOFS are comparable (64.2 vs. 64.1). Where differences exceed noise, SH’s lower KL generally translates to higher accuracy, with the exception of 2-bit GPTQ models for the mixture-of-experts model.

Per-layer bit widths. In Table 2, RCO optimizes per-layer widths for the two 8B models at average widths of 3, 4, and 5 bits (§ 3.5). After GPTQ, every mixed model exhibits lower KL than its uniform-width counterpart by 2.3–31.1%. Fixed grids benefit more from this allocation than SH; in each of the six model-width combinations, the reduction is smallest for SH (2.3–16.2%). Consequently, the gap between grids narrows from 6.6–22.4% to 0.3–9.0%. SH remains the lowest-KL mixed model in all six combinations, with its smallest margin, 0.3% over BOFS on Llama-3.1-8B-Instruct at 3 bits, within noise. This suggests that superior grids leave less residual error for allocation to mitigate. Under RTN, this pattern holds in five of six combinations, where SH is 3.1–10.8% below the best fixed grid. On Qwen3-8B at 4 bits, mixed SH and NF models perform worse than their uniform-width counterparts by 10.9% (driven by a single evaluation window) and 33.4%, respectively, with BOFS achieving the lowest KL.

Comparison with NVFP4. At 4.5 bpw, SH4 achieves 40.7–67.4% lower KL than NVFP4 across all ten settings in Table 3. A substantial portion of this gap stems from block scaling: NVFP4 rounds the absolute maximum scale to an unsigned E4M3 value, whereas lookup-table formats search for signed scales using the lever (§ 3.1). With the lever, even INT4 outperforms NVFP4 by 25.4–43.3%. Among formats sharing the lever, the best fixed grid is 37.9–55.6% below NVFP4, and SH4 further reduces KL by 4.6–34.4% relative to this best fixed grid.

Sign of the block scale. Without the sign bit of the block scale, the held-out KL of every grid rises on both 8B models (Appendix § 11). At 3 and 4 bits, SH, whose offset relies on the sign, loses more than NF, BOF and INT in seven of the eight settings (21–38% against 10–31%), but it remains below the best of them in all eight settings.

Per-layer grids. Tuning per-layer grids around global choices rarely reduces KL after GPTQ, so using a single grid per model and width incurs minimal loss (Appendix § 14).

Related Work

Grid design

Choosing the reconstruction levels of a scalar quantizer is a classical problem. The Lloyd–Max conditions are necessary for the MSE-optimal scalar quantizer of a known source ([36, 38]), and Bennett’s observation is that a nonuniform quantizer is equivalent to uniform quantization between a compressor and an expander, with the high-resolution distortion determined by the point density of the levels ([1, 24]). For neural networks, learnable piecewise-linear companding grids were developed for convolutional networks ([51]). For LLM weights, NF4 places its levels at normal quantiles ([13]), Student Float fits a Student-tt distribution ([14]), and BOF4 applies Lloyd iterations to block-normalized weights, with the error measured on the original weights ([4]). Other methods learn a codebook for every output channel by sensitivity-weighted kk-means ([32]). Most of these grids are fit to weight-space or layerwise errors, and a lower reconstruction error does not necessarily yield a better model ([52, 4]).

Parametric grid families

The work closest to SHq consists of grid families whose levels are generated by a few continuous parameters. PowerQuant maps a uniform lattice through a power function whose exponent is chosen by the weight error ([52]); M-ANT selects, for every group of 128 weights, a member of a one-parameter family spanning power-of-two, floating-point, and integer grids (Hu et al., 2025); and the concurrent CubicQuant fits a two-parameter cubic level function to every group of 128 to 512 weights by weight MSE, evaluated on synthetic data ([22]). Orr et al. construct grids whose point density is proportional to p1/3p^{1/3} for normal, Laplace, and Student-tt weights ([44]); in their parametrization, SHq corresponds to the limiting Student-tt source with two degrees of freedom, admissible only on the bounded range produced by block scaling. These families are symmetric, or their parameters are derived from weight statistics and layerwise errors. To our knowledge, SHq is the first parametric grid whose parameters are selected by the KL divergence of the full model.

Block-scaled formats and per-block grid selection. NVFP4 pairs E2M1 elements with an E4M3 scale for every 16 elements (NVIDIA et al., 2025). Several methods improve the scales of a fixed E2M1 grid, by mapping the block maximum to a smaller level [8], or by searching E4M3 scale codes under a plain or activation-weighted error [18, 35]; our per-block scale search belongs to this group. Other formats let each block or group choose among several grids: LO-BCQ stores an explicit selector among learned codebooks [20]. Because NVFP4 leaves the sign bit of its E4M3 scale unused, recent formats spend this bit on a selector: IF4 and MixFP4 choose between FP4 and INT4 [9, 57], RaZeR remaps the redundant zero to a special value [5], AAAC chooses between two learned tables [30], and Grid Games analyzes such grid pairs and shows that their advantage vanishes as the block size grows [19]. Signed scales have also been used directly: BOF4-S fixes the sign from the largest weight of the block [4], sign-max scaling stores the signed block maximum [44], and signed symmetric quantization selects the sign for integer grids [7]. In SHq_q the sign bit retains its meaning: with c≠0c \ne0, a single lookup table and its reflection provide up to 2q+12^{q+1} distinct values, and the sign is selected jointly with the scale code.

Bit allocation. Mixed-precision quantization assigns each layer a bit-width, chosen by a linear model of the loss in the layerwise errors [37] or by Shapley-value estimates of layer interactions [56, 27]. Such proxy models are accurate near full precision but degrade at low bit-widths, where interactions between layers become significant [47]. Methods that optimize the end-to-end divergence directly use evolutionary search [47] or gradient-based search under an exact budget constraint [26]. A few allocators also choose the format: FLIQS between integer and floating-point formats during quantization-aware training [15], and Q-Palette among quantizer types with fractional bit-widths [33]. We use RCO to allocate per-layer bit-widths for each grid § 3.5.

Rounding and vector quantization. Once the grid is fixed, the rounding of the weights can itself be optimized. GPTQ [21] compensates the error of each quantized weight with second-order updates, while YAQA targets the end-to-end loss [49]. These methods are largely orthogonal to the grid: GPTQ requires only a nearest-level rounding step and therefore applies to any lookup-table grid, which is how we use it. Vector and trellis quantizers [17, 48] attain the highest accuracy at 2 to 3 bits per weight, at a higher decoding cost than one table lookup per weight; scalar lookup-table formats target hardware that decodes every weight independently. Learned discrete codes [10] keep one table lookup per weight and optimize which level each weight takes, together with the scales Appendix § 10.

Lookup-table decoding and evaluation. On the decoding side, kernels that materialize the codebook as a table of weight values and apply a standard matrix multiplication [25] execute arbitrary grids at a speed close to that of optimized integer kernels for small batch sizes. The tensor-core instruction added by NVIDIA’s Rubin architecture decodes 3-bit weight indices through an eight-entry E4M3 table [42], the table type of the 3-bit formats studied here; its block scale is absent or an unsigned UE8M0 value per 32 weights, whereas our formats use a signed E4M3 scale per 16 weights. On the evaluation side, benchmark accuracy can remain unchanged while the answer a model produces change, and the KL divergence to the unquantized model tracks such changes [16]. We therefore report the KL divergence throughout; close to the lossless regime it is, however, a coarse criterion for ranking models that are near-identical [40].

Conclusion

Block-scaled lookup-table formats typically leave the quantization grid unconstrained; however, existing implementations fix this grid in advance, and the optimal fixed grid varies with bit width. SHq_q replaces this ad hoc selection with a closed-form family parameterized by two variables: a companding strength α\alpha, which interpolates level spacing between uniform and floating-point-like distributions, and an offset cc, which allows the sign bit of the E4M3 block scale to select between the grid and its mirror image. By modifying only the lookup table, SHq_q achieves the lowest KL divergence across all settings at 3 and 4 bits, and in most settings at 2, 5, and 6 bits. This performance holds across five models ranging from 3B to 35B parameters, and SHq_q persists as the lowest-KL grid under mixed precision following GPTQ quantization. At a precision of 4.5 bits per weight, the SHq_q format reduces the KL divergence of NVFP4 by 41–67% and improves average downstream accuracy on both 8B models. Empirical results indicate that families incorporating both spacing and offset parameters perform comparably, whereas those lacking an offset remain clearly inferior; thus, these two degrees of freedom account for the observed gains. Given that no single fixed grid is optimal across all bit widths and SHq_q requires no changes to storage or decoding logic, we propose it as the default grid for block-scaled lookup-table formats.

Expected Acceptance Rate

Table 4 gives the top-10 expected acceptance rate to the BF16 model (§ 3.6) of the models of Table 1, on the same 500K held-out tokens. Each EAR was computed on the weights of the corresponding cell of Table 1, on L40S, except for RTN on Qwen3.8-27B and RTN and GPTQ on Qwen3.6-35B-A3B at 4, 5 and 6 bits (RTX A6000), and Llama-3.2-3B-Inst. (RTX 3090).

Run-to-Run Variation

We isolate the effects of evaluation and quantization on the KL divergence for a single configuration: Qwen3-8B with SH at 3 bits and GPTQ, corresponding to the cell in Table 1, which reports its RTX A6000 run. Table 5 reports four GPTQ runs that differ only in GPU type or lazy-batch width; each run is evaluated on H200, L40S, and RTX A6000 GPUs.

Evaluation. For this configuration, the choice of evaluation GPU has a negligible effect: when scored across the three GPU types, identical weights exhibit KL differences of less than 0.5%.

Quantization. GPTQ is deterministic for a fixed GPU type but sensitive to the ordering of floating-point operations. A rerun with identical code, inputs, and GPU type reproduces the result bit-for-bit; an L40S run repeated after one week matches in every weight. Conversely, a rerun on a different GPU type, or on the H200 with half the lazy-batch width (which only reorders floating-point sums), alters 41–42% of the weights. Relative to the H200 run on the same evaluation GPU, these runs exhibit KL values that are 1.2–1.6% lower, with one exception: the L40S run is 5.1–5.5% higher, though its 95% confidence interval includes zero. A single evaluation window accounts for this entire increase; excluding it, the L40S run is 1.4% lower. The median per-window ratio of every run to the H200 run is 0.98–0.99. None of these differences satisfy both criteria defined in § 3.6: a magnitude of at least 3% and a 95% confidence interval that excludes zero. This variation motivates the two criteria applied throughout § 4.

SH Parameters

Table 6 lists the SH parameters (α\alpha, cc) used in Table 4 and Table 1, with one pair per model and code width. Each pair was selected via a format scan on the calibration KL of the RTN model (§ 3.4). The selected values vary across models and widths. For every model, α\alpha is largest at 6 bits (2.25–2.56), and the magnitude of the offset is smaller at 4 bits (0.047–0.078) than at 2 and 3 bits (0.11–0.47).

2 bits3 bits4 bits5 bits6 bits
Llama-3.2-3B-Inst. α\alpha1.93751.8751.93751.5∗1.5^{*}2.375
cc−0.46875−0.1875−0.078125−0.125−0.09375
Qwen3-8B α\alpha0.6251.751.625†1.625^{\dagger}1.5†1.5^{\dagger}2.25
cc−0.15625−0.1875−0.078125−0.03125−0.125
Llama-3.1-8B-Inst. α\alpha1.3751.8751.625‡1.625^{\ddagger}1.75‡1.75^{\ddagger}2.5625
cc−0.359375−0.171875−0.0625−0.09375−0.109375
Qwen3.8-27B α\alpha1.56251.43751.51.52.5625
cc−0.390625−0.109375−0.046875−0.03125−0.015625
Qwen3.6-35B-A3B α\alpha1.56251.8751.93752.252.375
cc−0.390625−0.171875−0.078125−0.1875−0.09375

Table 6. SH parameters (α,c)(\alpha, c) per model and code width, chosen by the format scan on calibration KL. †^{\dagger}: LUT placement factor 21/42^{1/4}; ∗^{*}: 21/22^{1/2}; ‡^{\ddagger}: 23/42^{3/4} (otherwise 1).

Learned Codes

We test whether the advantage of SH persists when codes are learned. GSQ (Dadgarni et al., [10]) extends GPTQ by learning, via Gumbel-softmax, on the output error of each decoder block, a shift of up to two levels for each weight’s code, along with continuous per-group scales. We added a lookup-table path to its quantizer and ran a modified version of its recipe (1024 instead of 4096 sequences, 10 instead of 20 epochs, groups of 16 instead of 128, FineWeb-Edu only) on the MLP layers of Qwen3-8B at 3 bits using the INT3, NF3, and SH3⟨1.875,−0.171875⟩\langle1.875, -0.171875\rangle codebooks. KL was scored with our protocol on 262K held-out FineWeb-Edu tokens (single runs). This pipeline differs from ours in several respects: it quantizes only the MLP layers, uses SH3⟨1.875,−0.171875⟩\langle1.875, -0.171875\rangle instead of the scanned ⟨1.75,−0.1875⟩\langle1.75, -0.1875\rangle, searches for BF16 scales at 1 bit per weight rather than E4M3 scales at 0.5 bits, uses unrounded tables and a different integer grid, and employs different calibration and evaluation text.

Code learning. GSQ reduces the KL of every grid by 13.4% (INT3), 18.2% (NF3), and 21.3% (SH3). After training, SH3 achieves the lowest KL, which is 8.2% below INT3 and 5.4% below NF3; both differences have 95% window intervals that exclude zero.

Size of the grid advantage. In this pipeline, the advantage of the grid is smaller than in our format, and learned codes contribute more to performance than the choice of grid. At GSQ’s GPTQ initialization with continuous scales, the three grids lie within 2.7% of each other, below the 3% threshold. The margins after code learning are smaller than the 8.4% and 24.1% advantages by which SH leads NF and INT in the GPTQ models of Table 1 on the same model and width. Furthermore, the gain from code learning on INT3 exceeds the final lead of SH3 over INT3. This is consistent with part of SH’s advantage compensating for the coarse E4M3 block scale; continuous scales provide this compensation by other means. We did not isolate which specific differences between the two pipelines narrow the gap between grids.

Sign of the Block Scale

Table 7 removes the sign of the E4M3 block scale, so that every block uses the grid GG and never its reflection −G-G (§ 3.1), and repeats the format scan without it, on the two 8B models at 2 to 4 bits. BOFS has no unsigned form. RTN ran on RTX A6000 and GPTQ on L40S; the signed references are the configurations of Table 1; BOF uses the same protocol, and Qwen3-8B GPTQ was rerun on L40S. Without the sign, the held-out KL rises in every configuration. At 2 bits under RTN, it rises by 890–3175% for NF, BOF and INT and by 82–123% for SH, whose scan moves to a larger α\alpha. At 3 and 4 bits, the KL of SH, whose offset relies on the sign, rises by 21–38%, and that of NF, BOF and INT by 10–31%. Without the sign, SH remains below the best other grid in all 8 settings at 3 and 4 bits, by more than 3% in 5.

RTNGPTQ
SHNFBOFINTSHNFBOFINT
Qwen3-8B2 bits+81.7+1947+1815+3175+48.8+37.3+40.7+52.6
3 bits+32.0+17.7+25.8+23.1+20.7+22.3+23.6+27.0
4 bits+29.3+16.6+12.5+10.1+23.2+17.3+11.7+13.4
Llama-3.1-8B-Inst.2 bits+123+890+898+1195+102+42.9+69.6+94.2
3 bits+34.0+20.3+19.3+31.0+37.9+17.3+23.2+24.4
4 bits+26.9+19.3+16.5+11.6+30.9+16.0+16.4+12.3

Table 7. Increase of the held-out KL (%) when the block scale has no sign bit, with each grid’s format re-scanned without it.

Grid Families

We investigate whether the performance of SH depends on the sinh⁡\sinh form of its level density or on its two degrees of freedom, namely the level spacing and the offset § 2. On Qwen3-8B and Llama-3.1-8B-Instruct, we compare SH with four additional families. NFO is NormalFloat with a free outer quantile zz, which determines the grid width; NFOC adds an SH-style offset cc to NFO. GC is a compander with level density (1+y2sinh⁡2α)−β(1+y^2\sinh^2\alpha)^{-\beta}, which reduces to SH for β=0.5\beta=0.5 and to the arctan compander for β=1\beta=1; we evaluate β=1\beta=1 and the heavier-tailed case β=0.25\beta=0.25. DDW is a Lloyd codebook learned per layer with the lever in the loop; DDWG learns a single codebook for the entire model, while DDWU learns per-layer codebooks without channel-norm weights. Each parametric family is optimized via coordinate descent initialized at the SH parameters (for

NFO, this involves a line scan over zz). The optimization criterion is the calibration KL from an earlier scan campaign: RTN with the lever and LUT placement factor 1, where the KL is coarsened to the 128 most likely reference tokens across 64 calibration windows of 2048 tokens each, without a BOS token per window. Consequently, these values are not comparable to the held-out KL reported in Table 1. Table 8 reports the calibration KL of the best grid for each family at 3 and 4 bits, while Table 9 reports the held-out KL after GPTQ at 3 bits, both relative to SH. The scans were executed on L40S and RTX 3090 GPUs and terminated before all planned stages completed; GPTQ results are available only for 3-bit quantization.

Calibration KL. Table 8 reveals two primary patterns. First, the offset accounts for most of the performance gap between NF and SH. Allowing the width of NF to vary reduces its excess KL over SH only from 15.0–32.2% to 13.0–25.9% (NFO), whereas adding the offset closes 74–97% of the remaining gap, leaving NFOC within 0.4–5.0% of SH. Second, among families incorporating an offset, the specific shape of the level density has a comparatively minor impact: the arctan and heavier-tailed companders exhibit KL values 1.6–2.6% higher than SH at 3 bits and 5.5–6.8% higher at 4 bits. Only the per-layer Lloyd codebook matches SH across all four settings (−1.2% to +1.1%); a single global codebook (DDWG) or a per-layer codebook without channel-norm weights (DDWU) yields KL values 0.3–5.9% higher than SH.

After GPTQ. Table 9 repeats the comparison at 3 bits following GPTQ, using single runs on one RTX 3090 node, each compared against an SH control run on the same hardware. The patterns observed in calibration KL persist. NF without an offset remains 8.9–16.7% above the control, whereas NFOC and the arctan compander remain within 3.0%, a margin consistent with GPTQ run-to-run variation (Appendix § 8); the heavier-tailed compander lies 7.7% above the control on Qwen3-8B and 1.9% on Llama-3.1-8B-Instruct. Among other families, only the per-layer Lloyd codebook falls more than 3% below the control, achieving a 6.5% reduction on Llama-3.1-8B-Instruct with lower KL in 186 of 245 windows; for this model, the SH grid selected for Qwen3-8B also yields a 3.7% improvement over the control. Thus, SH is not optimal in an absolute sense. Its advantage over fixed grids stems from its two degrees of freedom, which NFOC and the arctan compander replicate within run-to-run variation.

Downstream Benchmarks

We evaluate quantized models on the benchmark suite defined by BOF4 (Blumenberg et al., [4]): MMLU (Hendrycks et al., [28]) with five shots, and ARC-Challenge (Clark et al., [6]), HellaSwag (Zellers et al., [53]), PIQA (Bisk et al., [3]), SIQA (Sap et al., [46]), and WinoGrande (Sakaguchi et al., [45]) zero-shot. All tasks follow the definitions in lm-evaluation-harness 0.4.11 (Biderman et al., [2]), executed on its Hugging Face backend for the 8B models and on vLLM 0.32 for Qwen3.6-35B-A3B. Following BOF4, we report plain accuracy rather than length-normalized accuracy.

The RTN and GPTQ rows correspond to runs detailed in Table 1 and Table 3; BOF, which is not listed in these tables, derives from runs under the same protocol. For the 8B models, a difference is deemed significant if its 95% confidence interval, derived from the normal approximation of per-question differences, excludes zero. For Qwen3.6-35B-A3B, we apply a paired per-question sign-flip test to the difference in average accuracy, using the normal approximation and Holm correction across the 18 comparisons for this model.

Comparison with NVFP4. At 4.5 bpw (Table 10), SH4 achieves higher average accuracy than NVFP4 in all six model-quantizer combinations, by 0.1–0.7 points. On the two 8B models, SH4 also achieves higher MMLU accuracy; all four MMLU differences are statistically significant, while the differences in average accuracy are significant only for Llama-3.1-8B-Instruct. On Qwen3.6-35B-A3B, SH4 exceeds NVFP4 in average accuracy by 0.1 points under RTN and 0.4 points after GPTQ, neither of which is significant.

MMLUARC-CHellaSwagPIQASIQAWinoGrandeAvgKL (mn)EAR (%)
Qwen3-8BBF1674.955.557.276.651.668.063.9––
RTN SH474.755.556.676.651.168.163.818.3595.5
NVFP473.655.556.376.449.969.863.648.7092.5
GPTQ SH474.654.856.776.451.568.063.714.4696.2
NVFP474.154.956.476.450.867.063.332.6594.3
Llama-3.1-8B-Inst.BF1668.353.959.880.149.873.764.3––
RTN SH467.753.459.480.150.074.164.121.2195.2
NVFP466.252.259.179.450.973.063.552.2692.3
GPTQ SH467.752.059.480.649.974.064.019.8195.7
NVFP466.952.459.379.050.572.663.452.9192.8
Qwen3.6-35B-A3BBF1682.554.963.382.354.273.868.5––
RTN SH482.455.263.282.654.673.768.616.6895.7
NVFP482.554.163.282.354.174.968.536.6493.5
GPTQ SH482.754.963.482.454.073.468.514.4496.1
NVFP482.254.063.182.352.674.468.124.3594.9

Table 10. Benchmark accuracy (%) of SH4 and NVFP4 at 4.5 bpw, with the KL (mn) and EAR (%) of the same weights. Avg: mean of the six tasks. Bold: higher Avg of each pair.

Grids on the 8B models. At 3 bits, SH achieves the highest average accuracy among the five grids on both 8B models under both quantizers (Table 11 and Table 12); in each of these four cases, the gap to the next-best grid is not statistically significant. At 2 bits, performance diverges. On Llama-3.1-8B-Instruct, SH leads all other grids under both quantizers, and all such differences are significant. On Qwen3-8B, BOFS exceeds SH by 0.5 points under RTN and 0.6 points after GPTQ, and INT exceeds SH by 0.2 points under RTN; none of these advantages are statistically significant.

MMLUARC-CHellaSwagPIQASIQAWinoGrandeAvgKL (mn)EAR (%)
BF1674.955.557.276.651.668.063.9––
3 bits (3.5 bpw incl. scale)
RTN SH73.752.955.575.948.869.162.678.3190.7
BOFS72.753.855.475.748.468.262.487.2390.0
BOF72.453.554.976.748.768.462.486.46–
NF73.453.555.375.549.767.762.588.6090.1
INT71.853.254.976.248.866.361.9109.688.7
GPTQ SH72.956.755.275.849.068.763.061.9092.0
BOFS73.154.555.376.149.767.962.866.9491.7
BOF72.953.955.376.649.368.062.767.36–
NF72.453.155.276.049.668.062.467.6091.6
INT72.453.954.876.250.367.862.681.5190.8
2 bits (2.5 bpw incl. scale)
RTN SH62.043.145.571.945.461.855.0520.675.5
BOFS63.342.945.772.445.263.155.4561.874.7
BOF57.039.444.871.343.062.052.9642.1–
NF57.539.645.072.042.661.653.1601.873.9
INT60.744.245.272.645.562.855.2618.273.2
GPTQ SH64.344.647.173.146.365.556.8379.879.2
BOFS64.047.447.472.647.465.757.4378.279.2
BOF59.743.246.571.344.367.455.4425.3–
NF60.641.946.371.344.167.755.3432.577.7
INT63.745.546.771.846.365.756.6405.378.3

Table 11. Benchmark accuracy (%) at 2 and 3 bits on Qwen3-8B, with the KL (mn) and EAR (%) of the same weights. Bold: best per column within each block (lowest KL, highest accuracy and EAR).

MMLUARC-CHellaSwagPIQASIQAWinoGrandeAvgKL (mn)EAR (%)
BF1668.353.959.880.149.873.764.3––
3 bits (3.5 bpw incl. scale)
RTNSH65.052.458.179.450.273.263.190.8289.9
BOFS64.751.857.878.749.673.162.6107.089.0
BOF64.451.358.078.950.173.262.7109.4–
NF64.751.558.078.749.772.162.5105.289.1
INT64.349.757.578.949.373.562.2138.087.4
GPTQSH66.550.457.879.450.172.962.981.4690.9
BOFS65.450.958.279.247.172.562.298.2590.0
BOF65.652.858.078.948.972.862.895.25–
NF65.550.257.878.848.573.162.395.9490.2
INT65.051.557.679.348.773.062.5118.888.9
2 bits (2.5 bpw incl. scale)
RTNSH47.040.548.675.245.068.154.1643.273.4
BOFS41.940.646.574.046.465.552.5819.070.6
BOF39.833.743.171.843.765.549.61102.2–
NF40.832.844.571.642.965.849.71001.767.5
INT40.939.244.972.745.165.051.3955.467.9
GPTQSH53.540.448.875.546.769.655.8509.076.6
BOFS50.641.546.773.444.269.554.3628.874.4
BOF45.237.645.372.244.366.851.9699.8–
NF46.137.645.071.945.067.352.2727.872.5
INT48.539.544.772.042.368.852.6685.773.3

Table 12. Benchmark accuracy (%) at 2 and 3 bits on Llama-3.1-8B-Inst., with the KL (mn) and EAR (%) of the same weights. Bold: best per column within each block (lowest KL, highest accuracy and EAR).

Mixture-of-experts model. On Qwen3.6-35B-A3B (Table 13), SH achieves the lowest KL in all four settings. At 3 bits, BOFS and BOF exceed SH in average accuracy under both quantizers, though none of these 3-bit differences are significant. At 2 bits, SH achieves the highest average accuracy under RTN. After GPTQ, BOFS and INT exceed SH by 1.3 and 1.2 points, respectively, despite having higher KL values. The 2-bit differences between SH and BOF, NF, and INT under RTN, and between SH and BOFS and INT after GPTQ, are statistically significant (adjusted p<0.05p < 0.05). Across the three models, this is the only setting in which a grid with higher KL than SH achieves significantly higher accuracy.

MMLUARC-CHellaSwagPIQASIQAWinoGrandeAvgKL (mn)EAR (%)
BF1682.554.963.382.354.273.868.5––
3 bits (3.5 bpw incl. scale)
RTNSH83.356.462.681.953.272.868.355.4092.0
BOFS83.657.662.482.353.374.268.963.7491.3
BOF84.156.962.781.754.473.468.963.35–
NF83.756.062.881.552.473.268.262.4591.5
INT83.756.262.482.253.073.268.577.7790.4
GPTQSH84.254.963.081.252.473.368.146.9292.8
BOFS84.255.062.781.954.672.868.549.4592.6
BOF84.055.462.782.053.873.468.649.27–
NF83.756.362.782.251.672.868.249.6392.6
INT84.055.062.782.551.172.167.957.8392.0
2 bits (2.5 bpw incl. scale)
RTNSH78.850.958.379.546.671.364.2250.682.8
BOFS77.750.058.280.248.270.664.1289.781.5
BOF76.349.656.379.147.569.963.1342.6–
NF74.849.656.379.145.068.062.1336.780.1
INT75.749.756.478.649.669.363.2328.780.2
GPTQSH79.152.058.979.745.870.564.3208.284.5
BOFS77.955.557.480.750.170.765.6224.483.8
BOF77.653.157.378.547.270.964.1258.2–
NF76.853.357.479.447.370.064.1263.682.4
INT78.255.758.180.348.672.265.5240.583.1

Table 13. Benchmark accuracy (%) at 2 and 3 bits on Qwen3.6-35B-A3B, with the KL (mn) and EAR (%) of the same weights. Bold: best per column within each block (lowest KL, highest accuracy and EAR).

Per-Layer Grids

Tuning per-layer grids around global choices rarely reduces KL after GPTQ. For SH, NF, and the superior of BOF/BOFS at 2–6 bits on both 8B models, RCO selected the LUT placement factor for each layer; for SH, it also optimized α\alpha and cc within α±0.25\alpha\pm0.25 and c±1/64c \pm1/64, based on calibration KL with

RTN weights. Under RTN, this reduces KL on the first 50K evaluation tokens by 11–28% at 2 bits and 4–15% at 3 bits. After GPTQ, 26 of 30 configurations remain within ±2.2%\pm2.2\% of untuned grids. The only gains exceeding 3% that persist in at least two-thirds of the 245 windows occur on Llama-3.1-8B-Instruct (4.0–6.3%). End-to-end selection among 30 GPTQ-quantized candidates per layer for Qwen3-8B at 3 bits reduces KL by 2.5% and 3.2% from two starting points, a lower bound given incomplete search convergence. Under unweighted weight error, layers prefer nearly identical grids: normalized by absolute maximum, blocks of 16 weights exhibit similar mean sorted profiles across layers. With scale search excluding channel norms, the scanned α\alpha is the best of ten values for 249 of 252 layers in Qwen3-8B at 3 bits (Figure 4). When weighted by squared input channel norms, as in the lever, 7 of 21 sampled layers prefer a different α\alpha. The reason RTN gains from per-layer search do not persist after GPTQ was not isolated. Thus, using a single grid per model and width incurs minimal loss. Appendix D compares grids when GSQ ((Dag dargnia et al., 2026)) jointly learns codes and scales.

References

  1. [1]Bennett, W. R. Spectra of quantized signals. Bell System Technical Journal, 27(3):446–472, 1948. doi: 10.1002/j.1538-7305.1948.tb01340.x.DOI
  2. [2]Biderman, S., Schoelkopf, H., Sutawika, L., Abbasi, B., Forde, J. Z., Gao, L., Tow, J., Aji, A. F., Ammanamanchi, P. S., Black, S., Clive, J., DiPofi, A., Extaniz, J., Fattori, B., Foster, C., Hsu, J., Jaiswal, M., Lee, W. Y., Li, H., Lovering, C., Muennighoff, N., Pavlick, E., Phang, J., Skowron, A., Tan, S., Tang, X., Wang, K. A., Winata, G. I., Yvon, F., and Zou, A. Lessons from the trenches on reproducible evaluation of language models. arXiv preprint arXiv:2405.14782, 2024.arxiv.org/abs/2405.14782
  3. [3]Bisk, Y., Zellers, R., Le Bras, R., Gao, J., and Choi, Y. PIQA: Reasoning about physical commonsense in natural language. In Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI), volume 34, pp. 7432–7439, 2020. doi: 10.1609/aaai.v34i05.6239.DOI
  4. [4]Blumenberg, P., Graave, T., and Fingscheidt, T. Improving block-wise LLM quantization by 4-bit block-wise optimal float (BOF4): Analysis and variations. In International Conference on Learning Representations (ICLR), 2026.arxiv.org/abs/2505.06653
  5. [5]Chen, Y., Dai, X., Hyun, J., Chang, C.-C., Jang, W., Wu, Y., Tambe, T., sun Seo, J., and Abdelfattah, M. S. RaZeR: Pushing the limits of NVFP4 quantization with redundant zero remapping. In Advances in Neural Information Processing Systems (NeurIPS), 2026.arxiv.org/abs/2501.04052
  6. [6]Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O. Think you have solved question answering? Try ARC, the AI2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018.
  7. [7]Colbert, I., Dash, E., Monteagudo-Lago, P., Amboage, J., N, S., Franco, G., Fraser, N. J., and Ramachandran, A. Signed symmetric quantization for few-bit integers. arXiv preprint arXiv:2607.08779, 2026.arxiv.org/abs/2607.08779
  8. [8]Cook, J., Guo, J., Xiao, G., Lin, Y., Wyss, K., Nazemi, M., Mishra, A., del Mundo, C., Blankevoort, T., and Han, S. Four over six: More accurate NVFP4 quantization with adaptive block scaling. arXiv preprint arXiv:2512.02010, 2025.arxiv.org/abs/2512.02010
  9. [9]Cook, J., Lee, H. S., Le, K., Guo, J., Traverso, G., Chandrakasan, A. P., and Han, S. Adaptive block-scaled data types. arXiv preprint arXiv:2603.28765, 2026.arxiv.org/abs/2603.28765
  10. [10]Dadgarnia, A., Tabesh, S., Nikdan, M., Helcig, M., Kurtić, E., Kleinegger, M., and Alistarh, D. GSQ: Highly-accurate low-precision scalar quantization for LLMs via Gumbel-Softmax sampling. arXiv preprint arXiv:2604.18556, 2026.arxiv.org/abs/2604.18556
  11. [11]Darvish Rouhani, B., Zhao, R., More, A., Hall, M., Khodamoradi, A., Deng, S., Choudhary, D., Cornea, M., Dellinger, E., Denolf, K., Stosic, D., Elango, V., Golub, M., Heinecke, A., James-Roxby, P., Jani, D., Kolhe, G., Langhammer, M., Li, A., Melnick, L., Mesmakhosroshahi, M., Rodriguez, A., Schulte, M., Shafipour, R., Shao, L., Siu, M., Dubey, P., Micikevicius, P., Naumov, M., Verrilli, C., Wittig, R., Burger, D., and Chung, E. Microscaling data formats for deep learning. arXiv preprint arXiv:2310.10537, 2023.
  12. [12]Dettmers, T., Lewis, M., Belkada, Y., and Zettlemoyer, L. LLM.int8(): 8-bit matrix multiplication for transformers at scale. In Advances in Neural Information Processing Systems (NeurIPS), 2022.arxiv.org/abs/2208.07339
  13. [13]Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L. QLoRA: Efficient finetuning of quantized LLMs. In Advances in Neural Information Processing Systems (NeurIPS), 2023.arxiv.org/abs/2305.14314
  14. [14]Dotzel, J., Chen, Y., Kotb, B., Prasad, S., Wu, G., Li, S., Abdelfattah, M. S., and Zhang, Z. Learning from students: Applying t-distributions to explore accurate and efficient formats for LLMs. In International Conference on Machine Learning (ICML), 2024a.arxiv.org/abs/2405.03103
  15. [15]Dotzel, J., Wu, G., Li, A., Umar, M., Ni, Y., Abdelfattah, M. S., Zhang, Z., Cheng, L., Dixon, M. G., Jouppi, N. P., Le, Q. V., and Li, S. FLIQS: One-shot mixed-precision floating-point and integer quantization search. In International Conference on Automated Machine Learning (AutoML), 2024b.arxiv.org/abs/2308.03290
  16. [16]Dutta, A., Krishnan, S., Kwatra, N., and Ramjee, R. Accuracy is not all you need. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, pp. 124347–124390, 2024. doi: 10.52202/079017-3950.DOI
  17. [17]Egiazarian, V., Panferov, A., Kuznedelev, D., Frantar, E., Babenko, A., and Alistarh, D. Extreme compression of large language models via additive quantization. In International Conference on Machine Learning (ICML), 2024.arxiv.org/abs/2401.06118
  18. [18]Egiazarian, V., Castro, R. L., Kuznedelev, D., Panferov, A., Kurtic, E., Pandit, S., Marques, A., Kurtz, M., Ashkboos, S., Hoefler, T., and Alistarh, D. Bridging the gap between promise and performance for microscaling FP4 quantization. In International Conference on Learning Representations (ICLR), 2026a.arxiv.org/abs/2509.23202
  19. [19]Egiazarian, V., Schultheis, E., Panferov, A., Killian, E., Hoefler, T., and Alistarh, D. Grid games: The power of multiple grids for quantizing large language models. In Advances in Neural Information Processing Systems (NeurIPS), 2026b.arxiv.org/abs/2605.12327
  20. [20]Elangovan, R., Sakr, C., Raghunathan, A., and Khailany, B. LO-BQC: Locally optimal block clustered quantization for 4-bit (W4A4) LLM inference. Transactions on Machine Learning Research, 2025. OpenReview id loWISTqGwW; arXiv:2502.05376.arxiv.org/abs/2502.05376
  21. [21]Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D. GPTQ: Accurate post-training quantization for generative pre-trained transformers. In International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2210.17323
  22. [22]Gao, X. CubicQuant: Parametric non-uniform codebooks for high-throughput LLM inference with 1–8-bit weights. arXiv preprint arXiv:2608.06763, 2026.
  23. [23]Gersho, A. and Gray, R. M. Vector Quantization and Signal Compression, volume 159 of Kluwer International Series in Engineering and Computer Science. Kluwer Academic Publishers, Boston, MA, 1992. ISBN 978-0-7923-9181-4.
  24. [24]Gray, R. M. and Neuhoff, D. L. Quantization. IEEE Transactions on Information Theory, 44(6):2325–2383, 1998. doi: 10.1109/18.720541.DOI
  25. [25]Guo, H., Brandon, W., Cholakov, R., Ragan-Kelley, J., Xing, E. P., and Kim, Y. Fast matrix multiplications for lookup table-quantized LLMs. In Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 12419–12433. Association for Computational Linguistics, 2024. doi: 10.18653/v1/2024.findings-emnlp.724.
  26. [26]Helcig, M. and Alistarh, D. Model compression with exact budget constraints via Riemannian manifolds. arXiv preprint arXiv:2605.00649, 2026.arxiv.org/abs/2605.00649
  27. [27]Helcig, M., Kurtić, E., and Alistarh, D. Statistically-lossless quantization of large language models. In Conference on Language Modeling (COLM), 2026.arxiv.org/abs/2605.02404
  28. [28]Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring massive multitask language understanding. In International Conference on Learning Representations (ICLR), 2021.
  29. [30]IslamBouli, B. and Jin, D. AAAC: Activation-aware adaptive codebooks for 4-bit LLM weight quantization, 2026. Presented at the ICML 2026 Workshop on Resource-Adaptive Foundation Model Inference (AdaptFM).
  30. [31]Killian, E. A hardware-aware, per-layer methodology for post-training quantization of large language models. arXiv preprint arXiv:2605.14929, 2026. Provisional patent application filed May 2026; earliest formal reference for the HIF format family (HIF7, HIF8).
  31. [32]Kim, S., Hooper, C., Gholami, A., Dong, Z., Li, X., Shen, S., Mahoney, M. W., and Keutzer, K. SqueezeLLM: Dense-and-sparse quantization. In International Conference on Machine Learning (ICML), 2024.arxiv.org/abs/2306.07629
  32. [33]Lee, D. and Song, H. O. Q-Palette: Fractional-bit quantizers toward optimal bit allocation for efficient LLM deployment. In Advances in Neural Information Processing Systems (NeurIPS), 2025.
  33. [34]Lin, J., Tang, J., Tang, H., Yang, S., Chen, W.-M., Wang, W.-C., Xiao, G., Dang, X., Gan, C., and Han, S. AWQ: Activation-aware weight quantization for on-device LLM compression and acceleration. In Proceedings of Machine Learning and Systems (MLSys), 2024.
  34. [35]Lin, L. and Wan, X. ScaleSweep: Accurate NVFP4 post-training quantization of LLMs via block scale initialization. arXiv preprint arXiv:2606.07618, 2026.
  35. [36]Lloyd, S. P. Least squares quantization in PCM. IEEE Transactions on Information Theory, 28(2):129–137, 1982. doi: 10.1109/TIT.1982.1056489.DOI
  36. [37]Malinovskii, V., Panferov, A., Ilin, I., Guo, H., Richtárik, P., and Alistarh, D. HIGGS: Pushing the limits of large language model quantization via the linearity theorem. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL): Human Language Technologies, pp. 10857–10886, Albuquerque, New Mexico, 2025. Association for Computational Linguistics.
  37. [38]Max, J. Quantizing for minimum distortion. IRE Transactions on Information Theory, 6(1):7–12, 1960. doi: 10.1109/TIT.1960.1057548.DOI
  38. [40]Nikolić, M., Zadeh, A. H., Sanchez, E. T., and Moshovos, A. Displacement is not direction: Evaluating fidelity metrics for quantized LLM deployment. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track, pp. 3025–3039. Association for Computational Linguistics, 2026.
  39. [42]NVIDIA Corporation. Parallel thread execution ISA version 9.4: Decompression of input matrices (tcgen05.mma .decompress::lut::b). https://docs.nvidia.com/cuda/parallel-thread-execution/index.html, 2026. Section 9.7.18.10.8; target sm_107f. Accessed 2026-10-21.
  40. [43]Open Compute Project. OCP microscaling formats (MX) specification, version 1.0. Open Compute Project specification, 2023. Sep. 7, 2023.
  41. [44]Orr, D., Ribar, L., and Luschi, C. Optimal formats for weight quantisation, 2025. ES-FoMo III Workshop at ICML 2025.arxiv.org/abs/2505.12988
  42. [45]Sakaguchi, K., Le Bras, R., Bhagavatula, C., and Choi, Y. WinoGrande: An adversarial Winograd schema challenge at scale. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), volume 34, pp. 8732–8740, 2020. doi: 10.1609/aaai.v34i05.6399.DOI
  43. [46]Sap, M., Rashkin, H., Chen, D., Le Bras, R., and Choi, Y. Social IQa: Commonsense reasoning about social interactions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 4463–4473, 2019. doi: 10.18653/v1/D19-1454.DOI
  44. [47]Sieberling, O., Kuznedelev, D., Kurtic, E., and Alistarh, D. EvoPress: Accurate dynamic model compression via evolutionary search. In International Conference on Machine Learning (ICML), 2025.arxiv.org/abs/2410.14649
  45. [48]Tseng, A., Sun, Q., Hou, D., and De Sa, C. QTIP: Quantization with trellises and incoherence processing. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, pp. 59597–59620. Curran Associates, Inc., 2024.arxiv.org/abs/2406.11235
  46. [49]Tseng, A., Sun, Z., and De Sa, C. Model-preserving adaptive rounding. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pp. 122351–122370. PMLR, 2026.arxiv.org/abs/2505.22988
  47. [50]van Baalen, M., Kuzmin, A., Nair, S. S., Ren, Y., Mahurin, E., Patel, C., Subramanian, S., Lee, S., Nagel, M., Soriaga, J., and Blankevoort, T. FP8 versus INT8 for efficient deep learning inference. arXiv preprint arXiv:2303.17951, 2023.arxiv.org/abs/2303.17951
  48. [51]Yamamoto, K. Learnable companding quantization for accurate low-bit neural networks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021. doi: 10.1109/CVPR46437.2021.00499.DOI
  49. [52]Yvinec, E., Dapogny, A., Cord, M., and Bailly, K. Pow­erQuant: Automorphism search for non-uniform quantization. In International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2301.09858
  50. [53]Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., and Choi, Y. HellaSwag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 4791–4800, 2019. doi: 10.18653/v1/P19-1472.DOI
  51. [54]Zhang, H., Li, J., Kara, K., Alistarh, D., Liu, J., and Zhang, C. ZipML: Training linear models with end-to-end low precision, and a little bit of deep learning. In International Conference on Machine Learning (ICML), 2017.
  52. [55]Zhang, Y., Zhao, L., Cao, S., Zhang, S., Wang, W., Cao, T., Yang, F., Yang, M., Zhang, S., and Xu, N. Integer or floating point? New outlooks for low-bit quantization on large language models. In IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6, 2024. doi: 10.1109/ICME57554.2024.10688089.DOI
  53. [56]Zhao, J., Derakhshan, A., Hyman, J., Dong, J., Jyothi, S. A., and Harris, I. CoopQ: Cooperative game inspired layer-wise mixed precision quantization for LLMs. In Findings of the Association for Computational Linguistics: ACL 2026, pp. 7566–7578, 2026. doi: 10.18653/v1/2026.findings-acl.373. Earlier arXiv version titled IMPQ.DOI
  54. [57]Zou, J., Chen, Y., Wu, R., and Chen, X. MixFP4: Enhancing NVFP4 with adaptive FP4/INT4 block representations. In Proceedings of the 43rd International Conference on Machine Learning (ICML), volume 306 of Proceedings of Machine Learning Research, 2026.arxiv.org/abs/2605.31035

Paper details

vlnc:W_fj58xmf5
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents