A Unified Hessian-Guided Framework for Post-Training Quantization

Introduction

Large language models achieve strong performance across a wide range of language tasks, but their memory and computational requirements make deployment expensive. Quantization reduces these costs by representing weights and activations at lower precision, improving memory efficiency and enabling faster integer computation. The central challenge is to obtain these benefits while preserving the predictions of the full-precision model.

We focus on integer post-training quantization (PTQ) of both weights and activations. A small calibration set informs the quantization parameters. At a fixed bit width, accuracy depends on the quantization ranges, rounding choices, and representation basis. Quantization ranges set the scales: narrower ranges give finer grids but can increase clipping error. Rounding then selects weights on those grids. Both choices operate on a given representation of the layer. An invertible, function-preserving transform ([28]), such as a rotation or Hadamard transform ([3]), changes that representation by redistributing weight and activation magnitudes while preserving the full-precision layer output. Range selection and transforms can address both weights and activations, whereas adaptive rounding primarily addresses weights.

Local reconstruction evaluates these choices by the error they introduce at a layer’s output. But, as shown in Figure 1, the magnitude of local perturbations is not necessarily a good proxy for the

Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc. changes in the model output. Existing methods [27] capture these downstream effects through an approximation to the Hessian of the full-model output KL divergence with respect to a layer’s weights. However, methods based on the Weight Hessian fail to capture the error introduced by quantizing the activations (Figure 1, second column).

In this work, we use a quadratic expansion of the KL divergence between the original and quantized model predictions around the layer output with a tractable approximation (Figure 1, third and last column). This lets us derive a unified algorithm that optimizes transformations, range setting, and weight rounding under the same curvature metric, accounting for the downstream effects of both weight and activation quantization.

Our contributions are:

• We introduce a layer Output Kronecker-factorized approximation to the layer-output Fisher matrix that defines a quadratic proxy for the KL divergence between original and quantized outputs, accounting for both weight and activation quantization.

• Using the output Kronecker factorization, we derive $H_z$-PTQ, a unified procedure for transformation design, weight range selection, adaptive rounding, and activation clipping under a common downstream-sensitivity objective.

• We demonstrate improvements over existing PTQ baselines across multiple model families and quantization granularities.

Background

We first introduce the quantization choices considered in this work. Let $X \in\mathbb{R}^{s \times i}$ represent the activation matrix for $s$ tokens and $i$ input features, and let $W \in\mathbb{R}^{i \times o}$ be the weights of a linear layer with $o$ output features. Its output is $Z = XW$.

Quantization scales and clipping. For signed integer quantization with $b \ge2$ bits, $k_{\min} = -2^{b-1}$ and $k_{\max} = 2^{b-1} - 1$. Given a grid step $s > 0$ and shift $\mu$, round-to-nearest quantization is

The grid step $s$ determines the tradeoff between clipping and rounding resolution. Increasing $s$ expands the representable range and reduces clipping, but produces a coarser quantization grid. Decreasing $s$ improves resolution within the representable range, but may clip large-magnitude values.

Unlike weights, whose grid steps $s_w$ are fixed after calibration, activation scales depend on the current input. We therefore optimize a shared clipping factor $\gamma_x \in(0,1]$ [24] compute the corresponding input-dependent activation steps $s_x(X,\gamma_x)$ at runtime. The activations $x_i$ are clipped to $[-\gamma_x\max_i|x_i|,\gamma_x\max_i|x_i|]$ under symmetric quantization and to $[\gamma_x(\min_i x_i-\mu),\gamma_x(\max_i x_i-\mu)]$ under asymmetric quantization. We then apply Eq. (1) using a grid fitted to these bounds, with a zero-centered grid in the symmetric case and a shifted grid in the asymmetric case.

Weight rounding. Once the representation and quantization grid are fixed, rounding assigns each weight to a representable value. Round-to-nearest treats weights independently, whereas adaptive rounding methods coordinate these decisions so that later choices can compensate for errors introduced earlier ([20, 11]). Existing methods generally optimize either a layer-reconstruction objective or a weight-space approximation to the downstream model loss.

Transformations. For any invertible transformation $T \in\mathbb{R}^{i\times i}$, transforming the activations and weights preserves the full-precision output, $(XT)(T^{-1}W)=Z$, but generally changes the quantized output, $\widetilde{Z}=Q(XT)Q(T^{-1}W)$. We write $\Delta Z=Z-\widetilde{Z}$ for the resulting layer-output perturbation. If $T$ reduces a weight group’s range under the same quantizer, $Q(T^{-1}W)$ can use a finer grid and have lower weight error than $Q(W)$ in their respective representations ([24]). Diagonal transforms scale channels ([19, 29, 15]). Orthogonal transforms mix channels through randomized Hadamard transforms ([26, 2]), learned rotations ([16]), or blockwise rotations with permutations ([14]). More general transforms combine rotations and scaling ([13]). FlatQuant jointly learns invertible Kronecker factors, channel scaling, and clipping through block reconstruction ([24]). CAT ([9]) and WUSH ([6]) design transforms to minimize a local error proxy.

Method

Our derivation proceeds in four steps. We first express PTQ as the minimization of the prediction divergence induced by replacing one full-precision layer with its quantized counterpart. We then approximate this divergence by a quadratic form in the layer-output perturbation. Because the resulting output Hessian is intractably large, we introduce a tractable approximation. Finally, we use this common quadratic proxy to derive practical optimization procedures for transformations, weight scales and rounding, and activation clipping.

Joint objective

Ideally, post-training quantization would jointly select the transforms, weight and activation grids, and weight rounding decisions to minimize the downstream discrepancy between the full-precision and quantized models. For a linear layer, this idealized objective can be written as:

in which $\mathcal{L}()$ refers to a measure of change of the model output.

To model the downstream effect of weight and activation quantization, we use a second-order Taylor expansion around the full-precision layer output $Z$.

in which $\operatorname{vec}()$ refers to a vectorization operation that stacks the rows of a matrix into a single column vector, $G_Z$ is the loss gradient with respect to $Z$, and $H_Z$ is the (flattened) Hessian matrix with shape $[s\times o,s\times o]$.

Common choices of loss $\mathcal{L}()$ include measures of divergence of the model output distribution with respect to the original (unperturbed model). For these choices of loss, the zero and first order terms vanish since $Z=\widetilde{Z}$ is a global minimum for which the divergence is zero:

where $p_Z$ and $p_{\widetilde{Z}}$ refer to the model output distribution induced by the original and quantized layer, respectively. We use the shorter expression $\lVert\Delta Z\rVert_{H_Z}^2$ to indicate that the norm of the layer output perturbation is weighted by a quadratic proxy of the output loss sensitivity $H_Z$.

Modeling the Hessian matrices

Empirically estimating a full rank Hessian matrix is intractable for modern-sized deep learning architectures due to the computation and memory requirements. However, when selecting the Kullback-Leibler divergence for Equation (4), the Hessian becomes equivalent to the Fisher Information Matrix (FIM). The (empirical) Fisher matrix can be estimated by considering the outer product of the flattened Output gradients with respect to sampled labels from the output distribution:

Estimating a full Fisher matrix, which is quadratic in the product of sequence length and output feature dimension, is clearly still not practical for full-scale deep neural networks. For this reason, following a similar procedure as Tseng et al. [27], we simplify the computation by factorizing the effect of the FIM on the sequence dimension $F_s(Z)$ and feature output dimension $F_f(Z)$:

in which $F_s(Z)$ and $F_f(Z)$ represent proxies for the sequence and feature curvatures, respectively.

We refer to this approximation as the Output Kronecker Fisher factorization, which exactly reproduces each sampled gradient outer product when $G_Z$ is rank 1. A summary and comparison with the literature is included in Table 1, while the correlation between the Output Kronecker Fisher weighted error and the KL divergence is visualized in Figure 1.

Estimate	Output Fisher: $F_{\otimes Z}$ $F_s(Z) \otimes F_f(Z)$	Weight Fisher: $F_{\otimes W}$ $F_i(W) \otimes F_o(W)$	Input Fisher: $F_{\otimes X}$ $F_s(X) \otimes F_f(X)$	Assumptions
Isotropic	$I_s \otimes I_o$	$X^{T}X \otimes I_o$	$I_s \otimes WW^{T}$	Isotropic $F_Z$
K-FAC	$I_s \otimes G_Z^{T}G_Z$	$X^{T}X \otimes G_Z^{T}G_Z$	$I_s \otimes G_X^{T}G_X$	Isotropic $F_s(Z)$
YAQA	$G_ZG_Z^{T} \otimes G_WG_W$	$G_WG_W^{T} \otimes G_WG_W$	$G_ZG_Z^{T} \otimes G_WG_WW^{T}$	Rank-1 $G_W$
Ours	$G_ZG_Z^{T} \otimes G_Z^{T}G_Z$	$G_WG_W^{T} \otimes G_Z^{T}G_Z$	$G_ZG_Z^{T} \otimes G_X^{T}G_X$	Rank-1 $G_Z$

In order to perform the optimization, we estimate and store the following quantities on a small calibration set: the expected output feature curvature $\mathbb{E}\left[G_Z^{T}G_Z\right]$, the weight input curvature $\mathbb{E}\left[G_WG_W^{T}\right]$ and the token gradient magnitude vector $\mathbb{E}\left[\operatorname{Diag}\left(G_ZG_Z^{T}\right)\right]$. The input feature curvature can be obtained at runtime by left and right multiplying the output feature curvature by the weight matrices $\mathbb{E}\left[G_X^{T}G_X\right] = W\mathbb{E}\left[G_Z^{T}G_Z\right]W^{T}$.

HZ-PTQ: Optimization Procedure

We start optimizing the output-Fisher-weighted proxy by considering the linear transformation $T$. This is because the transformation $T$ affects model weights, activations, and intermediate gradients, making a joint optimization very challenging:

in which $\widehat{T}$ represents the optimal transformation $\arg\min_T\mathbb{E}\left[\lVert\Delta Z\rVert_{F_{\otimes Z}}^2\right]$, and $\Delta Z_{\widehat{T}}$ is the output discrepancy with the transformation $\widehat{T}$ applied. Once $\widehat{T}$ is fixed, we denote the transformed activations and weights by $X_{\widehat{T}} = X\widehat{T}$ and $W_{\widehat{T}} = \widehat{T}^{-1}W$, respectively. We express the corresponding activation and weight Fisher estimates in these coordinates and use them to determine the remaining quantization parameters.

Once the transformations are fixed, using mild independence assumptions, we can factorize the effect of activation and weight quantization:

The approximate split in Eq. (8) permits disentangling weight- and activation-specific parameters. First, we find the optimal weight scale and rounding:

Lastly, we can determine the optimal clipping based on the updated weights $\widehat{W}_{\widehat{T}}^{*}$ by determining:

We refer to the resulting framework as $H_{\mathit{Z}}$-PTQ because all optimization stages are derived from the same layer-output curvature proxy $F_{\otimes Z}$. Sections 3.4–3.6 describe each stage.

Optimal Transformations

In order to find the optimal transformation $\widehat{T}$, we first use standard rounding-noise assumptions to explicitly express our quadratic loss proxy as a function of the transformed quantization ranges for activation and weights, which determine the stretching of the quantization grid, and the norm of the transformed gradients:

The activation and weight bit widths are $b_x$, $b_w$, and $r(v)$ denotes the range of the quantization grid. The weights $\alpha_t=\left\|(G_Z)_{t,:}\right\|_2^2/\left\|G_Z\right\|_F^2$ and $\beta_j=\left\|(G_Z)_{:,j}\right\|_2^2/\left\|G_Z\right\|_F^2$ measure the relative gradient energy of token $t$ and output feature $j$. They remain unchanged by the transforms because $X_TW_T=Z$. A full derivation is reported in Appendix A.1. Overall, as expected, the expression in Eq. (11) is invariant to scaling of $T$. Furthermore, the norm of the transformed gradients $\left\|G_XT^{-T}\right\|_F^2$ and $\left\|T^TG_W\right\|_F^2$ are invariant to orthogonal transformations.

Following Federici et al. [9], we approximately minimize Eq. (11) with a two-step procedure. First, we choose the symmetric positive-definite component $M$ to minimize the sum of squared transformed gradient norms:

Here, $\#$ denotes the matrix geometric mean, see Appendix A.2 for the derivation. In practice, we use a block-wise approximation $\widehat{M}_{\mathrm{block}}=\operatorname{BlockDiag}(\widehat{M},\mathrm{block})$ to reduce the memory and computation cost ([9, 6]).

Secondly, we apply a Hadamard rotation to reduce the quantization ranges ([9]). The final transformation is obtained by combining the two components $\widehat{T}=\widehat{M}_{\mathrm{block}}H$.

Note that by making the additional assumption that the output gradients are isotropic ($G_Z\propto I$), the expression in Eq. (11) simplifies to the one reported in Federici et al. [9], and the solution in Eq. (12) is equivalent to the transforms proposed in Federici et al. [9]; Chen et al. [7], for which $\widehat{M}$ aligns the directions of variations of activations and weights instead of their gradients.

Weight scales and rounding

The optimal quantization scales and rounding can be determined by considering:

In which $\hat{T}^{T}\mathbb{E}\left[G_WG_W^{T}\right]\hat{T}$ determines the (transformed) weight input-side curvature while $\mathbb{E}\left[G_Z^{T}G_Z\right]$ the output-side curvature. In order to disentangle the optimization of scales and rounding, we follow a procedure similar to the one introduced in Tseng et al. [27]: first we perform a block-LDL decomposition of the input and output curvature based on the size of the quantization tile $[k_i,k_o]$. Here $k_i$ represents the number of input channels that share the same scale, while $k_o$ represents the number of output channels that share the same scale (usually 1):

Then, we use an iterative procedure in which we alternately set a quantization scale and adjust the weight rounding, starting from the original (transformed) weights $W^{(0)}=W_{\hat{T}}$.

The curvature for the quantization tile in coordinate $(j,k)$ is fully determined by the block diagonal components of $D_i[j]$ and $D_o[k]$. Since $D_o[k]$ is a scalar, the optimal scale $s_{jk}$ can be determined by selecting:

in which $B(j,k)$ is used to indicate the entries of the block $(j,k)$. The expression in Eq. (15) can be easily optimized using a grid search over a few candidates to pick the best scale. Fixing the scale $\hat{s}_{jk}$, we can determine the quantization error given by the quantization function in the block $(j,k)$:

and zero everywhere else. Lastly, we can use the $L_i$ and $L_o$ matrices to determine the correction and update the rounded weight for the remaining blocks

Proceeding in anti-diagonal order starting from the bottom-right, the quantization scales can be set one at a time, and the final $W^{(i/k_i\times o/k_o)}$ represents the adjusted weights $\hat{W}_{\hat{T}}^*$. The key difference from Tseng et al. [27] lies in using $D_i$ to set the scale of each block and in the different Fisher proxy. Similar to Frantar et al. [10], before running the algorithm, we sort the block rows and columns so that blocks corresponding to larger Fisher traces are quantized first.

Activation clipping.

For the activation clipping it is non-trivial to determine a good $\gamma_x\in(0,1]$: a heuristic constant value may not be optimal, but training it end-to-end is expensive and may yield sub-optimal solutions. We propose to use the Hessian proxy as a good middle ground that takes into account the downstream sensitivity.

The optimal $\gamma_x$ can then be determined on a small calibration set by picking the minimum error over a small set of candidates. Note that instead of using the dense sequence curvature $\mathbb{E}\left[G_ZG_Z^{T}\right]$, we consider only a diagonal component $\mathbb{E}\left[\operatorname{Diag}\left(G_ZG_Z^{T}\right)\right]$. This is to drastically reduce the memory requirements since only one vector needs to be stored. This approximation results in little error since we empirically validated that the sequence curvature component is mostly diagonal.

Experimental Results

We evaluate our downstream-output proxy, compare quantization performance across models and quantization granularities, and ablate the contributions of individual components. Appendix B experimental details and Appendix C includes additional results.

Quantization and Calibration We consider two quantization granularities and evaluate three configurations: (i) W4A4KV4, a channel-wise setting in which weights, activations, and the KV cache are quantized to 4-bit integers [24, 3, 9]; weights use one scale per output channel, while activations use one scale per token; (ii) W4-g32, a block-wise weight-only setting in which groups of 32 weight values share a floating-point scale, similar to the one used in Tseng et al. [27]; and (iii) W4A4-g32, a block-wise weight-and-activation setting in which groups of 32 values share a floating-point scale Chen et al. [7]. For all methods, quantization parameters and Hessian estimates are computed using 2048 randomly sampled sequences of 2048 tokens from the DCLM-Edu dataset Allal et al. [1].

Evaluation Protocol Following standard evaluation practice for LLM quantization [24, 9], we report perplexity on WikiText Merity et al. [17] using a context length of 2048 tokens and the mean normalized accuracy across six common-sense reasoning tasks (csr): PIQA Bisk et al. [4], WinoGrande Sakaguchi et al. [23], HellaSwag Zellers et al. [31], ARC-e and ARC-c Clark et al. [8], and LAMBADA Paperno et al. [21]. For methods involving calibration-dependent optimization, we report the mean and standard deviation over three random calibration seeds.

Channel-wise W4A4KV4 Quantization

We first evaluate the W4A4KV4 setting, where only a single scale is available for each weight output channel and activation token. Results are reported in Table 2. $H_Z$-PTQ consistently achieves the lowest perplexity across all evaluated architectures while matching or improving reasoning accuracy. Importantly, the gains are observed for every model family considered, ranging from the 1B-parameter Llama model to larger 8B-parameter architectures. This consistency indicates that the benefits of the proposed objective are not tied to a specific model architecture or size. Compared to FlatQuant, which learns quantization-aware transformations through layer-wise optimization, $H_Z$-PTQ achieves lower perplexity on all four models. Likewise, $H_Z$-PTQ consistently outperforms CAT, despite both methods relying on the same transformation with different parameters.

Method	Llama v3.2 1B inst. wiki ($\downarrow$)	Llama v3.2 1B inst. csr ($\uparrow$)	Llama v3 8B wiki ($\downarrow$)	Llama v3 8B csr ($\uparrow$)	Ministral 8B inst. wiki ($\downarrow$)	Ministral 8B inst. csr ($\uparrow$)	Qwen v3 8B wiki ($\downarrow$)	Qwen v3 8B csr ($\uparrow$)
FP	13.16	57.61	6.14	70.12	6.97	71.29	9.73	67.46
RTN	878.89	30.02	317.60	32.40	39.73	40.62	10441.69	29.23
QuaRot+GPTQ	$20.03 \pm0.33$	$48.93 \pm0.56$	$8.34 \pm0.04$	$63.39 \pm0.15$	$8.18 \pm0.00$	$67.77 \pm0.09$	$34.43 \pm9.39$	$51.64 \pm3.05$
CAT+GPTQ	$17.68 \pm0.09$	$52.19 \pm0.46$	$7.53 \pm0.02$	$65.38 \pm0.13$	$7.90 \pm0.02$	$68.06 \pm0.21$	$10.67 \pm0.02$	$64.60 \pm0.34$
FlatQuant+GPTQ	$17.23 \pm0.11$	$52.95 \pm0.40$	$7.19 \pm0.01$	$67.32 \pm0.55$	$7.82 \pm0.01$	$68.88 \pm0.33$	$10.60 \pm0.03$	$65.26 \pm0.11$
$H_Z$-PTQ	$16.05 \pm0.02$	$54.21 \pm0.11$	$7.06 \pm0.01$	$67.25 \pm0.23$	$7.69 \pm0.01$	$69.32 \pm0.37$	$10.43 \pm0.03$	$65.56 \pm0.46$

Block-wise Quantization

We next consider block-wise quantization using groups of size 32. This setting is particularly challenging for transformation-based methods since smaller quantization groups already reduce range variability, leaving less room for improvement through range equalization alone. Following Chen et al. [7], we apply independent block-wise transforms before quantization and fuse the corresponding inverses into the weights. We evaluate both weight-only (W4-g32) and weight-and-activation (W4A4-g32) quantization scenarios. Results are reported in Table 3.

Method	Llama v3.2 1B inst. wiki (↓)	Llama v3.2 1B inst. csr (↑)	Llama v3 8B wiki (↓)	Llama v3 8B csr (↑)	Ministral 8B inst. wiki (↓)	Ministral 8B inst. csr (↑)	Qwen v3 8B wiki (↓)	Qwen v3 8B csr (↑)
FP	13.16	57.61	6.14	70.12	6.97	71.29	9.73	67.46
W4-g32 RTN	15.47	55.27	6.65	68.67	7.28	70.88	9.99	65.95
Had.+GPTQ	13.73±0.02	56.83±0.18	6.43±0.01	68.97±0.19	7.13±0.01	71.01±0.12	9.99±0.05	66.80±0.15
CAT+GPTQ	13.77±0.08	56.77±0.17	6.44±0.01	69.64±0.18	7.14±0.00	70.99±0.03	9.92±0.04	67.14±0.28
WUSH+GPTQ	13.85±0.08	56.81±0.14	6.44±0.01	69.43±0.33	7.13±0.01	70.83±0.05	9.88±0.08	67.06±0.32
Had.+YAQA	13.74±0.02	57.02±0.18	6.39±0.00	69.61±0.03	7.22±0.00	70.96±0.12	9.93±0.06	67.29±0.22
HZ-PTQ	13.53±0.04	57.04±0.19	6.32±0.00	69.93±0.17	7.09±0.01	70.97±0.04	9.84±0.01	67.42±0.25
W4A4-g32 RTN	17.99	51.75	7.25	66.68	7.72	69.21	10.39	63.89
Had.+GPTQ	14.57±0.02	55.59±0.29	6.72±0.01	67.66±0.37	7.35±0.02	70.19±0.23	10.19±0.06	66.20±0.27
CAT+GPTQ	14.54±0.09	55.64±0.45	6.71±0.01	68.66±0.20	7.40±0.01	70.02±0.14	10.11±0.04	66.38±0.11
WUSH+GPTQ	14.65±0.04	55.28±0.45	6.74±0.02	67.98±0.54	7.42±0.01	69.83±0.21	10.11±0.09	66.30±0.31
Had.+YAQA	14.56±0.02	55.53±0.17	6.68±0.01	68.51±0.15	7.46±0.01	70.10±0.11	10.13±0.07	66.22±0.33
HZ-PTQ	14.22±0.05	55.75±0.16	6.56±0.00	68.82±0.15	7.29±0.00	70.38±0.13	9.99±0.01	66.77±0.38

Despite the reduced optimization headroom available in the block-wise setting, $H_Z$-PTQ consistently achieves the lowest perplexity and highest accuracy across all evaluated architectures and quantization formats. The improvement persists in both W4 and W4A4 settings, indicating that the proposed objective benefits not only transformation design, but also downstream range selection and rounding decisions. Notably, the relative gains remain stable across model families, further supporting the robustness of the proposed framework.

Table 3: For block-wise quantization, $H_Z$-PTQ consistently achieves the lowest perplexity across both W4-g32 and W4A4-g32 settings, demonstrating that the proposed Hessian-guided objective remains effective even at finer quantization granularities.

Table 4: Ablation for the components for $H_Z$-PTQ on W4A4KV4. −Round refers to a version with round-to-nearest and a standard $L_p$ weight range-setting. −Trans. refers to the use of Hadamard transform instead of our gradient alignment transforms. −Hess. indicates the use of a GPTQ-style Hessian approximation instead of our output Kronecker approximation.

Method	Llama v3.2 1B inst. wiki (↓)	Llama v3.2 1B inst. csr (↑)	Llama v3 8B wiki (↓)	Llama v3 8B csr (↑)	Ministral 8B inst. wiki (↓)	Ministral 8B inst. csr (↑)	Qwen v3 8B wiki (↓)	Qwen v3 8B csr (↑)
$H_Z$-PTQ	16.05±0.02	54.11±0.19	7.06±0.01	67.25±0.23	7.69±0.01	69.30±0.26	10.43±0.03	65.56±0.46
−Round	16.52±0.07	53.13±0.57	7.13±0.00	66.54±0.04	7.74±0.02	69.10±0.15	10.56±0.04	65.14±0.42
−A.clip	17.05±0.03	52.35±0.36	7.41±0.01	65.72±0.14	7.89±0.01	68.35±0.62	10.63±0.05	64.08±0.05
−Tran.	18.64±0.13	51.68±0.86	7.61±0.01	66.06±0.39	8.13±0.01	67.98±0.30	25.19±0.90	55.80±0.48
−Hess.	16.83±0.10	52.80±0.64	7.37±0.02	65.75±0.11	7.83±0.03	68.90±0.34	10.42±0.04	65.77±0.03

Ablation Studies

Contribution of Individual Components Table 4 evaluates the contribution of each component within $H_Z$-PTQ in the W4A4KV4 setting. Removing any component results in a measurable degradation in performance, demonstrating that the final gains arise from the interaction of all optimization stages rather than from a single dominant factor. Among the examined components, the transformation stage provides the largest individual contribution, followed by activation clipping and Hessian modeling. Replacing the proposed output Kronecker Hessian approximation with a simpler gradient-free Hessian estimate also degrades performance, providing evidence that the improved results are not solely due to the optimization procedure but also depend on the quality of the underlying curvature model.

Hessian-Guided Weight Range Selection Existing range-setting methods typically minimize an $L_p$ reconstruction objective, requiring the practitioner to select an appropriate value of $p$. Figure 2 evaluates the sensitivity of this choice. The optimal value of $p$ varies substantially across quantization settings and transformation choices, making fixed reconstruction objectives difficult to tune. In contrast, the proposed Hessian-weighted objective proposed in eq. (13) directly incorporates downstream sensitivity information and consistently matches or outperforms the best manually selected $L_p$ objective without requiring any hyperparameter tuning. These results suggest that incorporating curvature information into scale selection provides a more reliable objective than purely reconstruction-based metrics.

Hessian estimation and Rounding Strategies Table 5 compares different rounding algorithms under several Hessian approximations. This experiment disentangles the effect of the curvature estimate from that of the rounding procedure itself. First, for a fixed rounding algorithm, our proposed output Kronecker Hessian proxy performs on par with or outperforms the YAQA-style estimate, outperforming the gradient-free GPTQ-style hessian sketches.

Tran.	Hessian Estimate	W4 Per-channel			W4 Per-block (32)
		GPTQ	YAQA	ours	GPTQ	YAQA	ours
Id.	$\mathbf{X}^{T}\mathbf{X}\otimes\mathbf{I}$	$30.96^{\pm4.58}$	$30.56^{\pm0.00}$	$17.65^{\pm0.09}$	$14.35^{\pm0.04}$	$69.59^{\pm10.14}$	$14.11^{\pm0.07}$
Id.	$\mathbf{G}_{W}\mathbf{G}_{W}^{T}\otimes\mathbf{G}_{W}\mathbf{G}_{W}^{T}$	$18.42^{\pm0.48}$	$18.81^{\pm0.10}$	$15.34^{\pm0.03}$	$14.28^{\pm0.11}$	$13.75^{\pm0.05}$	$13.64^{\pm0.05}$
Id.	$\mathbf{G}_{W}\mathbf{G}_{W}^{T}\otimes\mathbf{G}_{Z}\mathbf{G}_{Z}^{T}$	$19.66^{\pm3.89}$	$19.49^{\pm0.42}$	$15.23^{\pm0.06}$	$14.26^{\pm0.25}$	$13.92^{\pm0.08}$	$13.64^{\pm0.02}$
Had.	$\mathbf{X}^{T}\mathbf{X}\otimes\mathbf{I}$	$14.96^{\pm0.08}$	$20.68^{\pm0.00}$	$14.50^{\pm0.01}$	$13.68^{\pm0.03}$	$13.90^{\pm0.07}$	$13.69^{\pm0.05}$
Had.	$\mathbf{G}_{W}\mathbf{G}_{W}^{T}\otimes\mathbf{G}_{W}\mathbf{G}_{W}^{T}$	$14.50^{\pm0.06}$	$19.12^{\pm0.29}$	$14.26^{\pm0.05}$	$13.61^{\pm0.01}$	$13.70^{\pm0.05}$	$13.56^{\pm0.02}$
Had.	$\mathbf{G}_{W}\mathbf{G}_{W}^{T}\otimes\mathbf{G}_{Z}\mathbf{G}_{Z}^{T}$	$14.49^{\pm0.04}$	$19.29^{\pm0.24}$	$14.25^{\pm0.08}$	$13.60^{\pm0.02}$	$13.66^{\pm0.01}$	$13.56^{\pm0.01}$

Second, for a fixed Hessian estimate, incorporating our Hessian-aware rounding and range-setting procedure consistently improves performance over conventional GPTQ/LDLQ and YAQA optimization. This observation suggests that using hessian information to set the weight ranges while performing the rounding correction can better incorporate curvature information into the local decisions. This is especially relevant for coarse grained quantization granularities. Note that when using GPTQ rounding, the output Kronecker and YAQA-style hessian are equivalent since only the first component of the Kronecker product determines the weight corrections.

Discussion and Conclusion

We introduced a unified Hessian-guided framework for post-training quantization. Unlike existing approaches that optimize transformations, range selection, activation clipping, and rounding independently and using separate reconstruction objectives, our framework derives all quantization decisions from a common approximation of downstream prediction loss. Central to this framework is the proposed Output Kronecker approximation, a tractable output-space Fisher factorization that better correlates with downstream KL divergence than conventional local reconstruction metrics. Across diverse architectures and quantization granularities, jointly optimizing these components consistently improved performance over existing state-of-the-art methods. These results suggest that much of the PTQ design space can be viewed as a single downstream optimization problem rather than a collection of independent heuristics.

The method, nevertheless, remains an approximation to the ideal downstream objective. Its derivation relies on a second-order expansion, a Kronecker-rank-one Fisher approximation, a decomposition of activation and weight errors, and rounding-noise and covariance assumptions. Moreover, estimating output gradients adds calibration time and memory relative to gradient-free reconstruction methods. Future work could investigate richer output-curvature models, adaptive approximations that trade accuracy for calibration cost, and extensions to non-uniform, mixed-precision, and hardware-aware quantization.

Quantization loss decomposition

Derivation of the transformation objective

We derive Eq. (11) from the output KL divergence, first separating activation and weight errors to make the derivation of the optimal transform tractable.

From output KL to a quadratic proxy. Using the second-order KL approximation in Eq. (4) and the Output Kronecker approximation in Eq. (6), we obtain

For a label $y \sim p_Z$, define $G_Z = \nabla_Z[-\log p_Z(y)]$. Here and below, expectations average calibration inputs, sampled labels, and the rounding-noise model defined next.

For the trace calculations, write $A = G_ZG_Z^{T}$, $B = G_Z^{T}G_Z$, and $c = \left\|G_Z\right\|_{\mathrm{F}}^{2}$. For $c > 0$ and any output perturbation $U$, row vectorization gives $\left\|U\right\|_{\mathrm{F}\otimes Z}^{2} = \operatorname{Tr}\left(AUBU^{T}\right)/c$.

Rounding model and error decomposition. For this derivation, activation row $t$ uses grid step $d_{x,t}$ and weight column $j$ uses grid step $d_{w,j}$. We model scalar rounding errors as conditionally independent and uniform on their quantization bins, with negligible clipping. Conditioning on the full-precision activations, weights, sampled gradients, and grid steps, the errors have zero mean and variances $d_{x,t}^{2}/12$ and $d_{w,j}^{2}/12$.

The output perturbation satisfies $\Delta Z = X\Delta W + \Delta XW - \Delta X\Delta W$. The product $\Delta X\Delta W$ contributes terms of third and fourth order in the quadratic proxy, so we omit it in the second-order approximation. Conditional independence and zero means eliminate the mixed quadratic term between $X\Delta W$ and $\Delta XW$. This gives the activation–weight decomposition in Eq. (8), following Federici et al. [9].

Activation contribution. The chain rule gives $G_X = G_ZW^{T}$. Using the trace form of the proxy,

Only matching scalar error indices survive the conditional expectation. Thus

using $\alpha_t = A_{tt}/c$.

Write $x_t = X_{t,:}$ and $r(x_t) = (2^{b_x}-1)d_{x,t}$. For our asymmetric per-token activation grid, $r(x_t) = \max_a X_{ta} - \min_a X_{ta}$.

Weight contribution. For the weights, $G_W = X^{T}G_Z$. The same calculation contracts $\Delta W B\Delta W^{T}$, whose conditional expectation is $\frac{1}{12}\left(\sum_j B_{jj}d_{w,j}^{2}\right)I_i$. Thus

using $\beta_j = B_{jj}/c$. For column $w_j = W_{:,j}$, define $r(w_j) = (2^{b_w}-1)d_{w,j}$ using the step of the configured symmetric weight grid. This range is the grid span, with the signed endpoints defined in § 2.

Separating ranges and gradient norms. Let $R_X = \sum_t \alpha_t r(x_t)^2$ and $R_w = \sum_j \beta_j r(w_j)^2$. Averaging the conditional results gives $\mathbb{E}\left[R_X\lVert G_X\rVert_F^2\right]/[12(2^{b_x}-1)^2]$ and $\mathbb{E}\left[R_w\lVert G_W\rVert_F^2\right]/[12(2^{b_w}-1)^2]$ for the activation and weight contributions, respectively. To split the range and gradient terms we assume $\operatorname{Cov}(R_X,\lVert G_X\rVert_F^2) \approx0$ and $\operatorname{Cov}(R_w,\lVert G_W\rVert_F^2) \approx0$ over calibration inputs and sampled labels. Applying $\mathbb{E}[UV] = \mathbb{E}[U]\mathbb{E}[V] + \operatorname{Cov}(U,V)$ to the two contributions yields

Combining these with Eq. (8) gives Eq. (11).

Geometric-mean solution for gradient balancing

We derive the solution of the gradient-norm objective in Eq. (12). Define the gradient second-moment matrices

and assume $A,B \succ0$. For any invertible $M$, set $P = MM^T \succ0$. Using $\lVert U\rVert_F^2 = \operatorname{Tr}(U^TU)$ and cyclicity of the trace, the two terms become

The objective therefore depends on $M$ only through $P$. Every positive-definite $P$ has a unique symmetric positive-definite square root $M = P^{1/2}$, so Eq. (12) is equivalent to

This is exactly the trace minimization in Zadeh et al. [30]. Their solution following Eq. (12) gives the unique minimizer

where $\#$ is the Pusz–Woronowicz geometric mean [22]. Taking the symmetric positive-definite square root yields

which is the solution stated in Eq. (12). Over all invertible matrices, the minimizers are $M = (P^*)^{1/2}R$ with $RR^T = I_i$. Thus the gradient-norm objective fixes the positive-definite component while leaving the rotation free for the subsequent range-reduction step.

Range clipping for weights derivation

Our algorithm applies the selected transformation before setting quantization ranges and rounding the weights. These steps require gradients and Fisher estimates in the transformed coordinates. The chain rule lets us compute these quantities from the original gradient estimates, avoiding an additional backward pass through the model. For the transformed coordinates defined in § 2, the chain rule gives

This shows that, instead of estimating the Fisher approximation directly from $G_{W_T}$, we can transform the existing approximation computed from the untransformed gradient $G_W$, using the identity $G_{W_T} = T^TG_W$. This relation is used in Eq. (13) and in the weight contribution to Eq. (11).

For fixed $T$ and $\Delta W_T$, write $P = G_{W_T}G_{W_T}^{T}$ and $Q = G_Z^{T}G_Z/\lVert G_Z\rVert_F^2$. We assume that $\lVert G_Z\rVert_F^2$ is constant, and that the correlations between the input- and output-side curvature factors contribute negligibly to the expected weight-error proxy:

Using $G_{W_T}=X_T^{T}G_Z$ in the trace form of the proxy gives

Under the constant-norm assumption, $\mathbb{E}[Q]=\mathbb{E}[G_Z^{T}G_Z]/\mathbb{E}[\lVert G_Z\rVert_F^2]$. Substituting the definitions of $P$ and $Q$ and applying row vectorization therefore yields

Experimental details

Models and Baselines

We evaluate across multiple model families and scales, including Llama v3 8B and Llama v3.2 1B instruct [12], Ministral 8B instruct (Mistral.ai, 2025), and Qwen v3 8B [25]. We compare against representative state-of-the-art transformation methods, including QuaRot [3], FlatQuant [24], CAT [9] and WUSH [7]. For rounding algorithms, we consider GPTQ/LDLQ [10, 5], and YAQA Sketch B [27].

Detailed Setup

The experimental configuration for the results reported in Table 2 and Table 3 is reported in Table 6 and Table 7, respectively. In the tables $\Sigma_X$ refers to $\mathbb{E}[X^{T}X]$, $\Sigma_W$ to $WW^{T}$, and $\Sigma_{G_X}=\mathbb{E}[G_X^{T}G_X]$, $\Sigma_{G_W}=\mathbb{E}[G_WG_W^{T}]$. The matrix $U$ refers to the eigen-decomposition of $\Sigma_X^{-1}\#\Sigma_W=U\Lambda U^{T}$. $B(\cdot)$ refers to the extraction of a block-diagonal component. Note that for block-wise quantization experiments, we omit the activation clipping strategy because of the limited impact on the small block size.

Method	W. Scale	W. Round	Act. Clip	Transforms
RTN	$2\max_i \lvert w_i\rvert$	None	$\gamma= 1$	None
QuaRot$^{+\mathrm{GPTQ}}$	$\min_{s_w}\lVert w-Q(w;s_w)\rVert_2^{2.4}$	GPTQ	$\gamma= 1$	(Fused) $H$
CAT$^{+\mathrm{GPTQ}}$	$\min_{s_w}\lVert w-Q(w;s_w)\rVert_2^{2.4}$	GPTQ	$\gamma= 1$	$B(\Sigma_X^{-1}\#\Sigma_W)^{1/2}H$
FlatQuant$^{+\mathrm{GPTQ}}$	Learned $s_w$	GPTQ	Learned $\gamma$	Learned $T_i \otimes T_o$
$H_Z$-PTQ	Algorithm in Sec. 3.5	Sec. 3.6		$B(\Sigma_{G_W}^{-1}\#\Sigma_{G_X})^{1/2}H$

Method	Weight Scale	Weight Round	Act. Clip	Block Transforms
RTN	$2\max_i \lvert w_i\rvert$	None	$\gamma=1$	None
Had.$+GPTQ$	$2\max_i \lvert w_i\rvert$	GPTQ	$\gamma=1$	Hadamard
Had.$+YAQA$	$2\max_i \lvert w_i\rvert$	YAQA	$\gamma=1$	Hadamard
CAT$+GPTQ$	$2\max_i \lvert w_i\rvert$	GPTQ	$\gamma=1$	$(\Sigma_X^{-1}\#\Sigma_W)^{1/2}H$
WUSH$+GPTQ$	$2\max_i \lvert w_i\rvert$	GPTQ	$\gamma=1$	$(\Sigma_X^{-1}\#\Sigma_W)^{1/2}UH$
$H_Z$-PTQ	Algorithm in Sec. 3.5		$\gamma=1$	$(\Sigma_{G_W}^{-1}\#\Sigma_{G_X})^{1/2}H$

For all the experiments, we use 500 candidates sampled uniformly in $[\max_i\lvert w_i\rvert,2\max_i\lvert w_i\rvert]$ to determine the minimizer for Hessian-weighted and $L_p$ norms. For the minimization of the clipping parameter, we use only 100 candidates uniformly sampled in $[0.5,1]$, determining the best by summing the error over 32 different sequences. In block-wise quantization settings, both YAQA and our algorithm process 32 output channels at a time during rounding. We select the quantization scale independently for each block, but batch the scale search across the 32 blocks. This reduces runtime without significantly affecting quantization performance.

Additional Results

Activation Clipping Figure 3 shows the effect of different activation clipping strategies. Overall, using the proposed Hessian proxy $\mathbb{E}\left[\operatorname{Diag}\left[\lVert g_{Z,1}\rVert^2,\ldots,\lVert g_{Z,s}\rVert^2\right]\otimes\mathbb{E}\left[G_X^T G_X\right]\right]$, performs strongly against manual selection of the clipping percentage or clipping using simpler hessian proxies. However, the sub-optimality is likely due to the fact that the same ”average” offline estimated curvature is used for all tokens instead of using a curvature estimation for each token independently. For $H_Z$-PTQ we assume the covariances have been transformed by $T$ according to Eq. (29).

Results with simple Round to Nearest Table 8 reports the results for the methods and quantization setting in Table 2 without any weight rounding adjustment. Our $H_Z$-PTQ still outperforms the baselines even without the improved weight-rounding and scale setting algorithm, demonstrating the effectiveness of our transformations and activation clipping method.

Method	Llama v3.2 1B inst. wiki (↓)	Llama v3.2 1B inst. csr (↑)	Llama v3 8B wiki (↓)	Llama v3 8B csr (↑)	Ministral 8B inst. wiki (↓)	Ministral 8B inst. csr (↑)	Qwen v3 8B wiki (↓)	Qwen v3 8B csr (↑)
FP	13.16	57.61	6.14	70.12	6.97	71.29	9.73	67.46
QuaRot	29.77	46.23	10.99	57.07	9.14	64.55	13.68	57.33
CAT	$18.20\pm0.10$	$51.41\pm0.28$	$7.67\pm0.01$	$63.90\pm0.24$	$7.99\pm0.01$	$67.72\pm0.20$	$11.01\pm0.04$	$63.66\pm0.05$
FlatQuant	$17.09\pm0.04$	$52.60\pm0.42$	$7.50\pm0.52$	$66.28\pm2.04$	$7.84\pm0.05$	$68.71\pm0.14$	$10.55\pm0.13$	$65.34\pm0.32$
$H_Z$-PTQ$-a.clip$	$17.78\pm0.12$	$51.42\pm0.16$	$7.51\pm0.01$	$64.55\pm0.56$	$7.95\pm0.02$	$68.23\pm0.25$	$10.78\pm0.04$	$63.21\pm0.00$
$H_Z$-PTQ	$16.52\pm0.07$	$53.13\pm0.57$	$7.13\pm0.00$	$66.54\pm0.04$	$7.74\pm0.02$	$69.10\pm0.15$	$10.56\pm0.04$	$65.14\pm0.42$