betaFeedback

WorksComputer Science

WAQS: Weight Absorption for Quadratic Probing and Affine Steering

Abstract

Activation steering provides a lightweight alternative to fine-tuning for controlling large language model behavior. Existing methods, however, often face a trade-off between efficiency and expressiveness: fixed steering vectors incur negligible inference overhead but cannot adapt to the current model input, whereas input-dependent nonlinear interventions provide greater flexibility at additional runtime cost. We introduce WAQS (Weight Absorption for Quadratic probing and affine Steering), a framework that obtains state-dependent steering, without additional inference-time computation. WAQS models a concept with a quadratic probe, whose gradient yields an affine, activation-dependent steering direction. We show that this affine intervention can be folded exactly into adjacent linear transformations through an offline weight update. Moreover, we prove that quadratic functions form the maximal class of smooth concept scores whose gradient-based interventions are affine and therefore exactly absorbable into linear layers. Under a Gaussian representation model, our formulation admits an interpretation as the difference of class-conditional Mahalanobis pulls and naturally generalizes linear discriminant and difference-in-means steering. To scale to high-dimensional representations, we introduce a low-rank parameterization of the quadratic term. Across truthfulness, refusal, and sycophancy steering tasks, quadratic steering matches or improves upon constant linear steering while preserving model utility, with the absorbed implementation introducing no additional inference-time operations. Our results suggest that quadratic probes provide a practical middle ground between static linear steering and computationally expensive dynamic nonlinear interventions.

Introduction

Large Language Models (LLMs) require steering away from harmful, untruthful, or sycophantic behavior after pretraining, and doing so cheaply and precisely is central to deploying them safely. Modern alignment techniques fall into two methodologies: training-based alignment and training-free, inference-time intervention. Training-based alignment relies primarily on Supervised Fine-Tuning (SFT) and Reinforcement Learning methods (Ouyang et al., 2022; Shen et al., 2023; Wu et al., 2024; Shao et al., 2024). While effective, these methods are computationally expensive, depend on noisy human feedback, and inherit the biases encoded in a learned reward function [2]; verifiable-reward variants avoid this bias but only apply where correctness is mechanically checkable, such as math, code, and formal reasoning. Training-free intervention, which instead adjusts model behavior at inference time without any gradient update, is the focus of this paper.

Within training-free intervention, representation engineering faces a fundamental trade-off between adaptivity and inference cost. Static Steering identifies a single static concept vector and adds it to the model’s activation at almost zero inference cost [35]. This static vector, however, assumes the concept boundary is a flat hyperplane in representation space; because most concepts do not satisfy this assumption, linear steering under- or over-corrects activations that lie off this hyperplane and degrades generation quality under strong intervention. Recent work instead models the concept boundary with deeper, curved classifiers and computes an input-dependent steering direction from them at generation time [11, 8] (Raval et al. 2026). This non-linear steering corrects activations more precisely, but existing analyses treat its cost as unavoidable: because the steering direction is an arbitrary, non-affine function of the current activation, it must be recomputed from a full forward pass at every generated token.

We resolve this tension with a simple observation: although a quadratic concept probe is a non-linear function of the activation, its gradient is an affine function of that same activation. A quadratic probe is therefore curved enough to represent a non-flat concept boundary, yet inducing a steering direction that is linear enough to be written as a fixed weight update rather than recomputed at every step.

Building on this observation, we introduce WAQS (Weight Absorption for Quadratic probing and affine Steering), a framework that turns a curved, input-dependent steering direction into a static weight update. WAQS trains a quadratic probe to score a target concept, derives the steering update as the probe’s gradient which is an affine function of input, and folds this affine map into the neighboring weight matrix, or layer norm. In contrast to static steering, WAQS’s steering direction still adapts to each input’s position relative to the target concept; in contrast to dynamic non-linear steering, this adaptation is precomputed once, offline, so it costs nothing extra at inference time.

Our contributions can be summarized as follows:

  • Maximal function class and statistical interpretation. We discuss that a smooth probe’s gradient-based intervention is affine in the activation if and only if the probe is a polynomial of degree at most two. Quadratic probes are thus not merely one absorbable family but the entire absorbable class; any higher-order probe reintroduces the recomputation cost that motivated dynamic non-linear steering. Under a Gaussian representation model, the optimal quadratic probe admits a closed form, and its induced steering direction decomposes into a difference of class-conditional Mahalanobis pulls. Moreover, under the Gaussian assumption, we show that difference-in-means steering method [1] is a special case of quadratic probing.

  • Weight absorption. We show that gradient-based steering with a quadratic probe can be folded exactly into the adjacent linear projection, attention layer, and normalization layer through an offline weight update. The result is a standard checkpoint: it requires no custom inference code, no forward hooks, and works out of the box with existing serving stacks, quantization pipelines, and distribution formats.

Problem Setup and Theoretical Analysis

Let x∈RDx \in\mathbb{R}^{D} represent the hidden state vector (activation) extracted from a specific layer ll within an LLM, where DD denotes the hidden dimension size. Linear probing assumes a linear hyperplane as the concept boundary fL(x)=wTx+bf_L(x) = w^{\mathsf{T}}x + b, where w∈RDw \in\mathbb{R}^{D} is the probe weight vector and b∈Rb \in\mathbb{R} is a scalar bias. To capture non-linear, pairwise interactions between activation values, we instead define a degree-2 polynomial (quadratic) probe function f(x):RD→Rf(x): \mathbb{R}^{D} \to\mathbb{R}:

fQ(x)=xTWpx+wpTx+bp,(1)f_Q(x) = x^{\mathsf{T}}W_p x + w_p^{\mathsf{T}}x + b_p, \tag*{(1)}

where Wp∈RD×DW_p \in\mathbb{R}^{D \times D} is a symmetric matrix2 capturing neuron-to-neuron correlation weights, wp∈RDw_p \in\mathbb{R}^{D} represents the linear feature bias, and bp∈Rb_p \in\mathbb{R} is the scalar bias. We train the probe via logistic loss to classify the presence or absence of a target semantic concept within the internal representations. To steer the model’s behavior during inference, we perform an intervention on xx to maximize or minimize the concept score fQ(x)f_Q(x). To identify the direction

Figure 1. Only the quadratic gradient adapts to the activation. The quadratic probe’s gradient field varies with position.

of steepest ascent along this curved decision boundary, we compute the gradient of the probe function with respect to xx: ∇xfQ(x)=2Wpx+wp\nabla_x f_Q(x)=2W_p x+w_p. Li et al. [12] adopt the same gradient-based direction.

Because the derivative of a quadratic function is affine, the resulting steering direction ∇xf(x)\nabla_x f(x) is dynamically context-dependent. The steering vector adapts at every token generation step based on the model’s current hidden state xx, replacing the fixed vector used by linear steering. The runtime activation update rule is:

xsteered=x+α∇xf(x)=x+α(2Wpx+wp)(2)x_{\mathrm{steered}}=x+\alpha\nabla_x f(x)=x+\alpha(2W_p x+w_p) \tag*{(2)}

where α∈R\alpha\in\mathbb{R} is a scalar hyperparameter that controls the steering intensity: α>0\alpha>0 amplifies the concept and α<0\alpha<0 suppresses it.

Exact affine absorbability

(2) defines a state-dependent intervention: the update applied to xx depends on xx itself, so no single vector describes what the steering does. Write this intervention as the map Tα(x):=x+α∇xfQ(x)T_\alpha(x):=x+\alpha\nabla_x f_Q(x). The standard reading holds that such an intervention must run during the forward pass and therefore belongs to the dynamic, runtime-cost regime of prior work. We show this reading is mistaken for the quadratic case: there is no inference-time overhead for quadratic probing.

A map can fold into a linear layer precisely when it is affine. In the quadratic probing case, Tα(x)=Msteerx+bsteerT_\alpha(x)=M_{\mathrm{steer}}x+b_{\mathrm{steer}}, where Msteer=I+2αWpM_{\mathrm{steer}}=I+2\alpha W_p, bsteer=αwpb_{\mathrm{steer}}=\alpha w_p, and I∈RD×DI\in\mathbb{R}^{D\times D} is the identity matrix. Therefore, as we will see in the next part, Tα(x)T_\alpha(x) can be absorbed into the model’s linear layer.

The next proposition shows that quadratic functions are the only functions that leads to affine map for steering which is absorbable.

Proposition 1 (Maximality). Let f∈C2(RD)f\in C^2(\mathbb{R}^D) and let Tα(x):=x+α∇xf(x)T_\alpha(x):=x+\alpha\nabla_x f(x). Then TαT_\alpha is an affine map of xx for some α≠0\alpha\ne0 if and only if ff is a polynomial of degree at most two, in which case Tα(x)=(I+2αWp)x+αwpT_\alpha(x)=(I+2\alpha W_p)x+\alpha w_p as in equation 2.

Proof. (Only If) Suppose TαT_\alpha is affine for some α≠0\alpha\ne0. Then ∇xf(x)=α−1(Tα(x)−x)\nabla_x f(x)=\alpha^{-1}(T_\alpha(x)-x) is affine in xx, hence the Hessian ∇x2f\nabla_x^2 f is a constant matrix HH. A second-order Taylor expansion then gives f(x)=12x⊤Hx+c⊤x+bf(x)=\frac{1}{2}x^\top Hx+c^\top x+b, i.e. ff is quadratic with Wp=12HW_p=\frac{1}{2}H.

(If) Suppose ff is a quadratic function. Then, Tα=Msteerx+bsteerT_\alpha=M_{\mathrm{steer}}x+b_{\mathrm{steer}}.

Two consequences follow. First, the runtime cost incurred by nonlinear-probe steering is intrinsic to the function class, not to any particular implementation; no engineering effort removes it. Second, any modification that reintroduces activation dependence outside the gradient itself falls outside the absorbable class by the same argument.

Statistical interpretation via Gaussian Discriminant Analysis

In this part, we provide intuition why quadratic probing is going to help with steering.

Assumption 1 (Gaussian Distribution Assumption). The hidden state of positive and negative class follows the Gaussian distribution:

x∣y=+1∼N(μ+,Σ+),x∣y=−1∼N(μ−,Σ−)(3)x|y=+1\sim\mathcal{N}(\mu_+,\Sigma_+),\qquad x|y=-1\sim\mathcal{N}(\mu_-,\Sigma_-) \tag*{(3)}

where μ+,μ−∈RD\mu_+,\mu_-\in\mathbb{R}^D are the class means and Σ+,Σ−∈RD×D\Sigma_+,\Sigma_-\in\mathbb{R}^{D\times D} are positive-definite class covariances, so that Σ+−1,Σ−−1\Sigma_+^{-1},\Sigma_-^{-1} exist.

Theorem 1. Under the assumption 1, solving for fQ(x)f_Q(x) by Gaussian Discriminant Analysis over all positive samples x+x_+ and negative samples x−x_- admits a closed-form solution.

Wp=−12(Σ+−1−Σ−−1),wp=Σ+−1μ+−Σ−−1μ−,(4)W_p=-\frac{1}{2}\left(\Sigma_+^{-1}-\Sigma_-^{-1}\right),\qquad w_p=\Sigma_+^{-1}\mu_+-\Sigma_-^{-1}\mu_-, \tag*{(4)}
bp=−12(μ+⊤Σ+−1μ+−μ−⊤Σ−−1μ−+log⁡∣Σ+∣−log⁡∣Σ−∣)+log⁡p(y+)p(y−).(5)b_p=-\frac{1}{2}\left(\mu_+^\top\Sigma_+^{-1}\mu_+-\mu_-^\top\Sigma_-^{-1}\mu_-+\log|\Sigma_+|-\log|\Sigma_-|\right)+\log\frac{p(y_+)}{p(y_-)}. \tag*{(5)}

We defer the proof to the Appendix § 9. Theorem 1 implies that estimating the decision boundary requires quadratic probing whenever the two concept classes have different covariance matrices; when Σ+=Σ−\Sigma_{+} = \Sigma_{-}, linear probing suffices. When Σ+=Σ−=I\Sigma_{+} = \Sigma_{-} = I, difference-in-means method gives the right steering vector under Gaussian assumption.

Under Theorem 1, the steering vector can be calculated as,

∇xfQ(x)=Σ+−1(μ+−x)−Σ−−1(μ−−x).(6)\nabla_x f_Q(x) = \Sigma_{+}^{-1}(\mu_{+} - x) - \Sigma_{-}^{-1}(\mu_{-} - x). \tag*{(6)}

Eq. equation 6 expresses the intervention as a difference of two Mahalanobis pulls: a pull toward the positive centroid and a push away from the negative centroid, each shaped by the inverse covariance of its corresponding class. The inverse covariance gives greater weight to directions with lower within-class variance. Consequently, both the magnitude and direction of the steering vector depend on the activation xx and the geometry of the two class distributions. When Σ+=Σ−=Σ\Sigma_{+} = \Sigma_{-} = \Sigma, the xx-dependent terms cancel, yielding the constant vector Σ−1(μ+−μ−)\Sigma^{-1}(\mu_{+} - \mu_{-}), the classical Fisher linear discriminant direction which is also illustrate in Marks & Tegmark [14]. Figure 1 illustrates that moving negative activations along a fixed difference-in-means or linear-probing direction does not necessarily move them into the positive cluster, leaving many points far from it. In contrast, the quadratic gradient field adapts to each activation and bends toward the interior of the positive cluster.

Methodology

In this part, we describe how the steering vector generated by quadratic probing can be absorbed into the transformer’s static weights. We further discuss low rank approximation for quadratic probing.

Weight absorption

A dynamic steering vector in general leads to an additional inference-time latency cost due to calculating the steering vector during every forward pass. We exploit the affine nature of the gradient in quadratic probing to fold the affine operation into the transformer’s static weight matrices. Depending on whether the steering happens before or after a linear projection, we can absorb WpW_p and wpw_p into the parameters of the linear projection as shown in Figure 2.

Weight absorption folds the steering map into surrounding static weights

Figure 2. Weight absorption folds the steering map Msteer=I+2αWpM_{\mathrm{steer}} = I + 2\alpha W_p and bias bsteer=αwpb_{\mathrm{steer}} = \alpha w_p into the surrounding static weights, for every point a quadratic probe can be attached to within a transformer block. (a) after a linear projection, (b) before a linear projection, and (c) through an RMS-norm/LayerNorm scale. In every case the runtime rule Tα(x)=x+α∇fQ(x)=Msteerx+bsteerT_\alpha(x) = x + \alpha\nabla f_Q(x) = M_{\mathrm{steer}}x + b_{\mathrm{steer}} is realized entirely offline, at zero inference-time cost.

(a) Steering and probing after projection layer: By expanding and rearranging Eq. (2), we have,

xsteered=x+2αWpx+αwp=(I+2αWp)x+αwp=Msteerx+bsteerx_{\mathrm{steered}} = x + 2\alpha W_p x + \alpha w_p = (I + 2\alpha W_p)x + \alpha w_p = M_{\mathrm{steer}}x + b_{\mathrm{steer}}

with I∈RD×DI \in\mathbb{R}^{D \times D} being the Identity matrix. Within a standard transformer architecture, a linear projection layer (such as an attention projection matrix) produces the hidden state xx at layer ll by acting on a precursor state h∈RKh \in\mathbb{R}^{K}: x=Wlayerh+blayerx = W_{\mathrm{layer}}h + b_{\mathrm{layer}}. Using this equation, our steering rule yields:

xsteered=Msteer(Wlayerh+blayer)+bsteer=(MsteerWlayer)h+(Msteerblayer+bsteer)x_{\mathrm{steered}} = M_{\mathrm{steer}}(W_{\mathrm{layer}}h + b_{\mathrm{layer}}) + b_{\mathrm{steer}} = (M_{\mathrm{steer}}W_{\mathrm{layer}})h + (M_{\mathrm{steer}}b_{\mathrm{layer}} + b_{\mathrm{steer}})

By grouping terms, we can directly compute a modified weight matrix WnewW_{\mathrm{new}} and bias vector bnewb_{\mathrm{new}} for the target transformer layer:

Wnew=(I+2αWp)Wlayer,bnew=(I+2αWp)blayer+αwp(7)W_{\mathrm{new}} = (I + 2\alpha W_p)W_{\mathrm{layer}},\quad b_{\mathrm{new}} = (I + 2\alpha W_p)b_{\mathrm{layer}} + \alpha w_p \tag*{(7)}

By replacing the original model weights with WnewW_{\mathrm{new}} and bnewb_{\mathrm{new}} offline, the model natively generates steered text with zero inference overhead and full compatibility with existing serving engines.

(b) Steering and probing before the projection layer: To apply steering vector to hidden state hh before a projection layer, we apply Wlayer,blayerW_{\mathrm{layer}}, b_{\mathrm{layer}} after applying Msteer,bsteerM_{\mathrm{steer}}, b_{\mathrm{steer}}.

hsteer=Msteerh+bsteerh_{\mathrm{steer}} = M_{\mathrm{steer}}h + b_{\mathrm{steer}}
x=Wlayerhsteer+blayer=WlayerMsteerh+(blayer+Wlayerbsteer)(8)x = W_{\mathrm{layer}}h_{\mathrm{steer}} + b_{\mathrm{layer}} = W_{\mathrm{layer}}M_{\mathrm{steer}}h + (b_{\mathrm{layer}} + W_{\mathrm{layer}}b_{\mathrm{steer}}) \tag*{(8)}

As a result, we can write WnewW_{\mathrm{new}} and bnewb_{\mathrm{new}} as: Wnew=WlayerMsteerW_{\mathrm{new}} = W_{\mathrm{layer}}M_{\mathrm{steer}}, bnew=blayer+Wlayerbsteerb_{\mathrm{new}} = b_{\mathrm{layer}} + W_{\mathrm{layer}}b_{\mathrm{steer}}

(c) Quadratic Probing for RMS-norm/layer norm: Given a linear layer x=Wlayerh+blayerx = W_{\mathrm{layer}}h + b_{\mathrm{layer}}, steering with quadratic probing on xx modifies WlayerW_{\mathrm{layer}} directly. If a non-linear operation instead produces xx (e.g., RMS-norm in Llama, or an MLP activation function), we can absorbed MsteerM_{\mathrm{steer}} into the next projection layer or Attention Layer. In Figure 2, if the intervention happens at point “c”, MsteerM_{\mathrm{steer}} can be absorbed in WK,WQ,WVW_K, W_Q, W_V (if “c” is before Attention Layer) or in WupW_{\mathrm{up}} (if “c” is before MLP layer). However, if we have to do weight absorption on the RMS-norm, then with a slight modification, MsteerM_{\mathrm{steer}} also can be absorbed into RMS-norm and LayerNorm: restricting WpW_p to a diagonal matrix allows MsteerM_{\mathrm{steer}} to be absorbed directly into the norm layer’s scale parameter. Let

x=h∥h∥2⊙γ=diag⁡(γ)⋅h∥h∥2x = \frac{h}{\lVert h \rVert_2} \odot\gamma= \operatorname{diag}(\gamma) \cdot\frac{h}{\lVert h \rVert_2}

denote an RMS-norm layer, where ⊙\odot is the element-wise product, γ∈RD\gamma\in\mathbb{R}^{D}, and diag⁡(γ)\operatorname{diag}(\gamma) denotes the diagonal matrix with γ\gamma on its diagonal. The probe function f(x)=x⊤diag⁡(v)x+wp⊤x+bpf(x) = x^{\top}\operatorname{diag}(v)x + w_p^{\top}x + b_p gives the steering vector 2⋅diag⁡(v)⋅x+wp2 \cdot\operatorname{diag}(v) \cdot x + w_p, so:

xsteered=x+2αdiag⁡(v)x+wp=diag⁡(γ)h∥h∥2+2α⋅diag⁡(v)diag⁡(γ)h∥h∥2+αwp=h∥h∥2⊙[γ+(2α⋅v⊙γ)]+αwp=h∥h∥2⊙[γ⊙(1+2α⋅v)]+αwp=h∥h∥2⊙γ^+αwp,\begin{aligned} x_{\mathrm{steered}} &= x + 2\alpha\operatorname{diag}(v)x + w_p = \operatorname{diag}(\gamma)\frac{h}{\lVert h \rVert_2} + 2\alpha\cdot\operatorname{diag}(v)\operatorname{diag}(\gamma)\frac{h}{\lVert h \rVert_2} + \alpha w_p \\ &= \frac{h}{\lVert h \rVert_2} \odot[\gamma+ (2\alpha\cdot v \odot\gamma)] + \alpha w_p = \frac{h}{\lVert h \rVert_2} \odot[\gamma\odot(1 + 2\alpha\cdot v)] + \alpha w_p = \frac{h}{\lVert h \rVert_2} \odot\hat{\gamma} + \alpha w_p, \end{aligned}

where γ^=(γ⊙(2α⋅v+1))\hat{\gamma} = (\gamma\odot(2\alpha\cdot v + 1)). Steering with quadratic probing under a diagonal matrix therefore introduces no overhead relative to linear probing; the same approach applies to LayerNorm.

Quadratic Probe Estimation

Low-rank discriminative probe. For modern production Large Language Models, the hidden dimension DD reaches into the thousands (e.g., D=4096D = 4096 for Llama-3-8B). In these regimes, the full-rank quadratic weight matrix Wp∈RD×DW_p \in\mathbb{R}^{D \times D} encounters a severe parameter explosion of O(D2)O(D^2) elements (1.67×1071.67 \times10^7 parameters for D=4096D = 4096). This memory footprint makes probe training unstable and renders the offline matrix multiplication (I+2αWp)Wlayer(I + 2\alpha W_p)W_{\mathrm{layer}} computationally prohibitive.

To resolve this limitation, we apply a low-rank factorization constraint to the quadratic probe. We hypothesize that the semantic concepts we wish to steer reside within a low-dimensional intrinsic subspace of the network’s activations. Therefore, we restrict the symmetric matrix WpW_p to a low-rank approximation: Wp≈12(UV⊤+VU⊤)W_p \approx\frac{1}{2}(UV^{\top} + VU^{\top}), where U,V∈RD×RU,V \in\mathbb{R}^{D \times R} are low-rank bottleneck matrices with rank R≪DR \ll D (e.g., R∈{4,8,16}R \in\{4, 8, 16\}). Under this constraint:

Wnew=(I+α(UV⊤+VU⊤))Wlayer=Wlayer+α((UV⊤+VU⊤)Wlayer)(9)W_{\mathrm{new}} = (I + \alpha(UV^{\top} + VU^{\top}))W_{\mathrm{layer}} = W_{\mathrm{layer}} + \alpha\left((UV^{\top} + VU^{\top})W_{\mathrm{layer}}\right) \tag*{(9)}

Since Wp=12(UV⊤+VU⊤)W_p = \frac{1}{2}(UV^\top+ VU^\top), f(x)f(x) becomes

f(x)=x⊤Wpx+wp⊤x+bp=(U⊤x)⊤(V⊤x)+wp⊤x+b(10)f(x) = x^\top W_p x + w_p^\top x + b_p = (U^\top x)^\top(V^\top x) + w_p^\top x + b \tag*{(10)}

So f(x)f(x) depends on the interaction between the directions spanned by UU and VV.

The low-rank extension cuts both memory and compute cost during the weight-folding phase. Computing the full-rank matrix multiplication WpWlayerW_pW_{\mathrm{layer}} costs O(D3)O(D^3) floating-point operations (FLOPs). By enforcing the low-rank constraint and altering the order of operations via the associative property in Equation (9), we compute A=V⊤WlayerA = V^\top W_{\mathrm{layer}} first, followed by UAUA. This reduces the time complexity to O(D2⋅R)O(D^2 \cdot R), transforming a computationally heavy offline workload into a lightweight step that computes in seconds.

Shrinkage GDA. Theorem 1’s closed form is a population-level statement, however, and in practice Σ+,Σ−∈RD×D\Sigma_+, \Sigma_- \in\mathbb{R}^{D \times D} must be estimated from a finite training set: when the number of training examples is smaller than DD (as with D=4096D = 4096 activations), the sample covariance is singular and Σ±−1\Sigma_{\pm}^{-1} is undefined. We therefore replace each sample covariance with a doubly shrunken estimator [23]

Σ~±(λ)=(1−λ)[Σ^±+Σ^pooltr⁡(Σ^±)D]+λI,λ∈[0,1],(11)\widetilde{\Sigma}_{\pm}(\lambda) = (1-\lambda)\left[\widehat{\Sigma}_{\pm} + \widehat{\Sigma}_{\mathrm{pool}}\frac{\operatorname{tr}(\widehat{\Sigma}_{\pm})}{D}\right] + \lambda I,\qquad\lambda\in[0,1], \tag*{(11)}

While λ\lambda shrinks toward a scaled identity. The resulting estimator family contains that λ→1\lambda\to1 drives both covariances to scaled identities and recovers a difference-in-means direction. This shrinkage family treats linear and quadratic probing as two ends of one regularization path.

Experiments

Our theoretical analysis argues that WAQS’s quadratic probing should replace linear probing as the standard steering baseline: (1) quadratic probing introduces no inference cost through weight absorption, and (2) linear probing nests within our quadratic parameterization, since setting Wp=0W_p = 0 recovers the linear probe exactly. To confirm this claim holds in practice, we evaluate affine steering using quadratic probing against Difference-in-Means (DiffMean) and Linear Probing (LP), both of which use a constant steering vector that does not depend on the input. We report results over 5 seeds unless otherwise stated. Additional training details are provided in Appendix § 10.

Main Results for probing and steering

Truthfulness Steering: We use the TruthfulQA dataset [13], which comprises questions designed to mimic human superstitions and misconceptions. Following the linear probing approach of [12], we extract the intermediate activations at the last prompt token from the 14th layer’s 18th attention head, which yields the highest probing accuracy. For open-ended generation, we employ GPT-4o as an automated judge to evaluate the responses. We prompt the judge to output a binary “Yes” or “No” indicating the truthfulness and informativeness of each generated answer (Appendix § 10.§ 2 details the prompt templates). We report True as the percentage of answers evaluated as factually correct. Additionally, we report True*Info, computed as the product of the truthfulness and informativeness percentages. This combined metric serves as a proxy for utility and penalizes the model for reaching high truthfulness through evasive, non-committal responses. MC1 [12] evaluates whether the model assigns the highest conditional probability to the lone true answer. MC2 calculates the total normalized conditional probability the model assigns to the set of all truthful answers relative to the false ones. To quantify the intervention’s effect on general fluency, we compute CE (Cross-Entropy), the steered model’s next-token prediction loss on a held-out corpus. We also calculate the KL divergence between the next-token probability distributions of the pre- and post-injection models over the same corpus. We sweep the steering strength α\alpha for each method and choose the α\alpha that achieves the best MC1 score.

Takeaway. Table 1 shows that Rank-1 Quadratic Probing improves True*Info and True over constant steering vectors on Llama-2-7B (20.43 vs. 13.28-19.30 True*Info), indicating better utility after steering. Quadratic Probing also achieves higher MC1 and MC2, and almost best MC3, and its CE stay comparable compared with other methods. KL divergence stays between 0.005 to 0.007 for every method on LlaMA-2-7B, so none of the three steering methods materially shifts the next-token policy.

Table 1: Quadratic steering improves truthfulness while preserving fluency across model families. Results on TruthfulQA for five steering methods. We compare with Linear Probe, DiffMeans, Angular Steering, and Spherical Steering. For each method, α\alpha is swept and the value with the best MC1 is reported. True and Info are judged by GPT-4o.

MethodTrue*Info (%) ↑\uparrowTrue (%) ↑\uparrowMC1 (%) ↑\uparrowMC2 (%) ↑\uparrowMC3 (%) ↑\uparrowCE ↓\downarrowKL ↓\downarrow
Llama-2-7B
Quadratic Probe20.43±0.5820.43 \pm0.5841.62±1.0341.62 \pm1.0335.37±1.8435.37 \pm1.8455.43±2.8955.43 \pm2.8929.73±2.6929.73 \pm2.692.412.410.0060.006
Linear Probe13.28±0.3313.28 \pm0.3338.07±0.5938.07 \pm0.5933.78±1.3333.78 \pm1.3351.06±2.7751.06 \pm2.7726.99±2.6326.99 \pm2.632.132.130.0050.005
DiffMeans18.82±0.2818.82 \pm0.2839.53±0.4439.53 \pm0.4433.78±1.3333.78 \pm1.3351.12±2.8951.12 \pm2.8926.94±2.5426.94 \pm2.542.172.170.0050.005
Angular Steering18.77±0.6318.77 \pm0.6339.41±1.1739.41 \pm1.1733.90±1.5033.90 \pm1.5051.07±2.7851.07 \pm2.7827.05±2.6627.05 \pm2.662.372.370.0070.007
Spherical Steering19.30±0.7619.30 \pm0.7640.02±1.2140.02 \pm1.2134.02±4.1034.02 \pm4.1053.51±8.2653.51 \pm8.2630.64±7.5030.64 \pm7.502.542.540.0060.006
Gemma-3-12B
Quadratic Probe20.68±1.3920.68 \pm1.3942.23±2.2042.23 \pm2.2051.41±5.2851.41 \pm5.2870.57±3.2270.57 \pm3.2243.99±5.1143.99 \pm5.112.972.970.0250.025
Linear Probe16.02±1.3416.02 \pm1.3436.72±2.0236.72 \pm2.0248.84±1.6448.84 \pm1.6468.49±0.3868.49 \pm0.3840.58±0.2940.58 \pm0.292.982.980.0120.012
DiffMeans16.36±0.9716.36 \pm0.9737.21±1.8837.21 \pm1.8850.68±2.8650.68 \pm2.8669.19±1.5169.19 \pm1.5142.18±2.6242.18 \pm2.623.033.030.1110.111
Angular Steering17.55±1.4817.55 \pm1.4837.94±2.3137.94 \pm2.3149.21±1.8249.21 \pm1.8269.12±0.7669.12 \pm0.7641.23±0.7941.23 \pm0.793.043.040.3540.354
Spherical Steering17.10±1.2617.10 \pm1.2637.94±2.1837.94 \pm2.1849.57±2.6849.57 \pm2.6868.33±0.4468.33 \pm0.4441.03±0.1241.03 \pm0.123.043.040.1420.142

Table 1.

Refusal Steering: Following Chhabra & Khalili [3]’s work, we construct the harmful dataset and harmless dataset based on ADVBENCH (Zou et al., [35]), MALICIOUSINSTRUCT (Huang et al., [9]), TDC2023 (Mazeika et al., [15]), HARMBENCH (Mazeika et al., [16]), and ALPACA (Taori et al., [29]). We evaluate on the MMLU (Hendrycks et al., [7]), GSM8k (Cobbe et al., [5]) and ARC-Challenge (Clark et al., [4]) to measure the utility of the steering methods. We also evaluate the safety performance using GCG (Zou et al., [35]), TAP (Mehrotra et al., [17]), and GPTFuzz (Yu et al., [34]) as the attacking method to get the Attack Success Rate (ASR).

Table 2 presents the safety and utility performance of five steering methods on LLaMA-2-7B.3

Table 2: Quadratic steering yields the most robust refusal behavior without sacrificing utility. Safety and utility of refusal steering on Llama-2-7B. Steering directions are estimated from harmful and harmless prompts built from AdvBench, MaliciousInstruct, TDC2023, HarmBench, and Alpaca, following Chhabra & Khalili [3]. Quadratic Probe uses a rank-1 probe. Safety is the attack success rate (ASR) under three jailbreak attacks: GCG, TAP, and GPTFuzz. Utility is accuracy on MMLU, GSM8k, and ARC-Challenge.

MethodSafety (↓\downarrow)Utility (↑\uparrow)
GCGTAPGPTFuzzMMLUGSM8kARC-Challenge
Quadratic Probe94.01.047.3022.3649.57
Linear Probe104.45.047.2721.8349.23
DiffMeans107.81.047.2322.5948.97
Angular Steering94.21.047.3122.5149.23
Spherical Steering04.41.046.9419.4847.44

Table 2.

Takeaway. Across all three attacks, Rank-1 Quadratic Probing achieves the lowest ASR (9.0, 4.0, and 1.0 under GCG, TAP, and GPTFuzz respectively), beating both Linear Probing and DiffMeans. This suggests the non-linear decision boundary introduced by Quadratic Probing yields a refusal direction that resists all three attacks more effectively than a single fixed direction. Quadratic Probing also matches or exceeds Linear Probing and DiffMeans on all three utility benchmarks (MMLU, GSM8k, ARC-Challenge), so the safety gain does not cost general capability. DiffMeans and Linear Probing reach comparable utility but do not match Quadratic Probing’s safety gain, indicating that the more expressive probing objective captures the harmful/harmless distinction without degrading benign-task behavior.

We also report the probing performance (i.e., the metrics for the linear and quadratic classifiers trained on harmful/harmless prompts). To assess the generalization of the learned probe, the test set includes harmful samples aggregated from HarmBench (Mazeika et al., [16]), AdvBench (Zou et al., [35]), and StrongREJECT (Souly et al., [28]), and harmless samples from Alpaca dataset.

Table 8 presents the probing performance of linear probing and quadratic probing at ranks 1, 2, and 4. Takeaway. Table 8 shows that linear and quadratic probes reach similar classification performance (92.9% vs. 88.9–94.7% accuracy across ranks). Combined with Table 2, this indicates that probing accuracy alone does not predict steering quality: a dynamic, context-dependent steering vector improves refusal steering even when the underlying probe classifies no better than a linear one.

ProbeAcc ↑\uparrowAUC ↑\uparrowF1 ↑\uparrowAP ↑\uparrow
linear_probe0.92860.99710.94610.9987
quadratic_rank10.88930.99210.91380.9967
quadratic_rank20.91650.98040.93650.9928
quadratic_rank40.94740.99560.96090.9981

Table 8. Probing performance of quadratic/linear probe.

Steering for Anthropic AI Risk Concepts: We use Anthropic’s Advanced AI Risk dataset (Perez et al., [21]) to steer the model’s sycophancy, following the setup in CAA (Rimsky et al., [25]). We evaluate steering with a next-token prediction metric that measures the change in the model’s tendency to predict the sycophantic response token after the steering intervention. We compute the softmax over the logits of answer A and B and take the difference as the metric. We sweep α∈{0.5,1,1.5,2}\alpha\in\{0.5, 1, 1.5, 2\} for all of the methods. All the steering trials are tested on the layer 24 to get better contrast. A more negative value under negative steering indicates stronger suppression of sycophantic behavior; a larger positive value under positive steering indicates stronger enhancement. Takeaway. Table 3 shows that quadratic steering achieves the strongest suppression of sycophancy, well beyond both linear steering and CAA. GDA Quadratic achieves the strongest positive steering effect, closely followed by Quadratic, while Linear and CAA produce only small positive changes (0.0050 and 0.0032 respectively). Quadratic steering therefore controls sycophancy in both directions markedly more than linear steering (2.2× stronger suppression, 6.4× stronger enhancement) under the same evaluation setup used in CAA.

Dataset / ConceptMethodNegative Steering decrease ↓Positive Steering improvement ↑
SycophancyLR-Quadratic Probing-0.0780 ± 0.00390.0322 ± 0.0025
SycophancyGDA-Quadratic Probing-0.0557 ± 0.00240.0326 ± 0.0101
SycophancyLinear Probing-0.0358 ± 0.00370.0050 ± 0.0023
SycophancyContrast Activation Addition-0.0403 ± 0.00320.0074 ± 0.0037
CorrigibleLR-Quadratic Probing-0.0361 ± 0.00350.3815 ± 0.0555
CorrigibleGDA-Quadratic Probing-0.0360 ± 0.01130.4052 ± 0.0205
CorrigibleLinear Probing-0.0322 ± 0.00840.3702 ± 0.0344
CorrigibleContrast Activation Addition-0.0331 ± 0.00750.3903 ± 0.0286

Table 3. Steering effects across different behavioral concepts. Lower values indicate stronger negative steering, while higher values indicate stronger positive steering.

We also report qualitative generations comparing methods on the same concept and prompt in Appendix § 10.§ 5.

Ablation Study

Ablation Study on ranks. Table 7 reports the evaluation results across different ranks for Quadratic Probing on Llama-2-7b. Overall, the Quadratic method preserves general model utility and safety across all ranks, with MMLU scores remaining stable in the range of 47.20–47.34%, indicating that the refusal-baking procedure introduces minimal degradation on language understanding. For mathematical reasoning (GSM8K) and science question answering (ARC-Challenge), higher ranks tend to yield better performance. Meanwhile, rank=4 achieves the strongest safety performance on TAP (1.0) and GPTFuzz (0.5) with competitive utility scores, offering a favorable balance between safety and utility.

Safety (↓\downarrow)Utility (↑\uparrow)
MethodGCGTAPGPTFuzzMMLUGSM8kARC-Challenge
Quadratic (rank=1)94.01.047.3022.3649.57
Quadratic (rank=2)105.01.047.3422.7451.45
Quadratic (rank=4)101.00.547.2822.6751.37
Quadratic (rank=8)94.0547.2023.5852.13
Quadratic (rank=16)114.02.547.2123.1251.28

Table 7. Safety and Utility evaluation for different rank on Llama-2-7b. Best result per column is bold.

Frozen quadratic steering comparison. The gains of Quadratic Probing over DiffMean and Linear Probing could come from two different sources: (i) the steering direction 2Wpx+wp2W_p x + w_p adapts to each input’s position relative to the concept boundary, or (ii) the quadratic probe simply learns a better constant direction than a linear probe, independent of whether that direction is allowed to vary with xx. To separate these two effects, we introduce a Frozen Quadratic (Frozen-QP) baseline that keeps the same trained probe parameters (Wp,wp)(W_p, w_p) but replaces the per-input gradient with a single direction evaluated at the mean activation of the probe’s training set:

dfrozen=2Wpxˉ+wp,xˉ=1N∑i=1Nxi,(12)d_{\mathrm{frozen}} = 2W_p\bar{x} + w_p,\qquad\bar{x} = \frac{1}{N}\sum_{i=1}^{N}x_i, \tag*{(12)}

where {xi}i=1N\{x_i\}_{i=1}^{N} are the same activations used to fit (Wp,wp)(W_p,w_p). Because the gradient of a quadratic form is affine, dfrozend_{\mathrm{frozen}} is exactly the average of the per-example steering directions, dfrozen=1N∑i=1N(2Wpxi+wp)d_{\mathrm{frozen}} = \frac{1}{N}\sum_{i=1}^{N}(2W_px_i + w_p), so this ablation changes nothing about the fitted probe itself – it only removes the xx-dependence at steering time, isolating adaptivity from probe expressiveness. We apply dfrozend_{\mathrm{frozen}} as a constant steering vector, xsteered=x+αdfrozenx_{\mathrm{steered}} = x + \alpha d_{\mathrm{frozen}}, and select α\alpha using the same held-out-validation protocol as DiffMean and Linear Probing.

MethodTrue*Info (%) ↑True (%) ↑MC1 (%) ↑MC2 (%) ↑MC3 ↑
Quadratic Probe (dynamic)20.43 ± 0.5841.62 ± 1.0335.37 ± 1.8455.43 ± 2.8929.73 ± 2.69
Frozen-QP (avg. direction)13.47 ± 0.5938.63 ± 1.3233.66 ± 1.5051.05 ± 2.7726.93 ± 2.72
Linear Probe13.28 ± 0.3338.07 ± 0.5933.78 ± 1.3351.06 ± 2.7726.99 ± 2.63
DiffMeans18.82 ± 0.2839.53 ± 0.4433.78 ± 1.3351.12 ± 2.8926.94 ± 2.54

Table 4. Dynamic vs. Frozen Quadratic Probing on TruthfulQA (Llama-2-7B). Frozen-QP uses the same trained (Wp,wp)(W_p,w_p) as Quadratic Probe but replaces the per-input direction with the training-set-average direction of Eq. (12).

Takeaway. The results show that the steering vector should adapt to the input for better performance. Once the xx-dependence is removed, Frozen-QP falls back to the level of the constant baselines. On MC1, MC2, and MC3 it lies within 0.12 points of Linear Probe (33.66 vs. 33.78, 51.05 vs. 51.06, 26.93 vs. 26.99). On True*Info it is essentially tied with Linear Probe (13.47 vs. 13.28) and falls below DiffMeans (18.82). Because direction of Frozen-QP is exactly the average of the per-example directions, the two variants apply the same steering on average and differ only in how the direction varies across inputs. The quadratic probe therefore does not simply learn a better constant direction than a linear probe. Its advantage comes from adapting the steering direction to each activation, which weight absorption provides at no inference-time cost.

Discussion and Limitations

The ability to absorb affine steering generated by a quadratic prob into model parameters depends on the location at which steering is applied. For activations produced directly by a linear transformation, the steering transformation can be folded into the corresponding weights. However, exact absorption is less straightforward when steering is applied directly to the residual stream. A residual transformation

rsteered=Msteerr(l)+αwp(13)r_{\mathrm{steered}} = M_{\mathrm{steer}}r^{(l)} + \alpha w_p \tag*{(13)}

also transforms the identity skip path, which has no associated weight matrix into which MsteerM_{\mathrm{steer}} can be directly absorbed. Consequently, residual-stream quadratic steering generally requires explicitly applying the steering transformation during inference. For a dense quadratic probe, this introduces an additional O(D2)\mathcal{O}(D^2) matrix–vector multiplication at each steered location. Structured parameterizations, such as diagonal or low-rank steering matrices, can reduce this overhead to O(D)\mathcal{O}(D) or O(RD)\mathcal{O}(RD).

Conclusion

Quadratic probes provide input-dependent steering at the inference cost of a static linear vector. The gradient of a quadratic concept score is affine in the activation, so an offline weight update absorbs the entire intervention into the model’s existing linear layers. Proposition 1 shows quadratic probes are the largest function class with this property: any higher-degree probe reintroduces the inference-time cost that motivated dynamic non-linear steering. Quadratic probing extends linear probing rather than replacing it with a separate technique. Under a Gaussian representation model, the optimal quadratic probe decomposes the steering direction into a difference of class-conditional Mahalanobis pulls. Across truthfulness, refusal, and sycophancy steering, quadratic probing matches or exceeds constant linear steering and difference-in-means on task performance and model utility, and the absorbed implementation adds zero inference-time operations.

AI USE STATEMENT

In this work, we used generative AI tools for aiding/polishing writing and correcting grammatical errors and implementing methods. We have not used generative AI tools for generating scientific content, and generating synthetic data sets, helping develop theoretical models or conceptual frameworks, formulating mathematical claims, providing critical ingredients for proving mathematical claims, assisting in the writing of proofs, proposing or refine hypotheses, designing or providing feedback on research methodology or experiments, assist with translation, cleaning and reformatting dataset, supporting qualitative and thematic data analysis, interpret results are not applicable to this work.

We have reviewed all AI-assisted work. We also review the code which is generated by AI and test with ourselves. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI.

Related Work

Activation Steering. Zou et al. [35] pioneered difference-in-means and linear-probe-derived vectors as strong, cheap steering baselines. Li et al. [12] similarly derive a steering direction as the gradient of a trained linear probe; because a linear probe’s gradient is constant, this direction does not depend on the current activation, so the resulting intervention is static despite its gradient-based derivation. Contrastive Activation Addition (CAA) (Rimsky et al., [25]) finds a difference-in-means direction from contrastive prompt pairs, and representation fine-tuning methods such as ReFT [32] learn a fixed, input-independent intervention on the residual stream through gradient-based training. All of these methods share the same underlying assumption: the concept boundary is well approximated by a hyperplane, so a single fixed direction suffices for every input.

Curved and Input-Dependent Steering. More recent work argues that concept directions are better modeled as curved manifolds than hyperplanes (Raval et al., 2026; Lee & Kondor, 2026; Hoscilowicz et al., 2024), motivating angular steering [30] and spherical steering [33] variants. SOM [22] shows that a single direction cannot capture the steering effect, since it varies across multiple learned components. Cylindrical steering [6] proposes a cylindrical representation hypothesis, showing that the steering vector is also sensitive to position within the same normal plane. These methods recover exactly the input-dependence that linear methods lack, but at a cost: extending Li et al. [12]’s gradient-based recipe to a nonlinear probe requires recomputing the gradient from the current activation at every generation step, reintroducing the per-token inference cost that motivated static steering in the first place.

Training-free Weight Editing. A separate line of work modifies model weights directly, without gradient-based training, to induce or remove targeted behaviors. Directional ablation methods identify a single linear direction associated with a behavior (e.g., refusal) and project it out of every weight matrix that writes to the residual stream, effectively ”uninstalling” the behavior at the parameter level [1]. Task arithmetic constructs task vectors as the difference between fine-tuned and pretrained weights and edits models by adding or negating these vectors, again without further training [10]. Locate-and-edit approaches such as ROME [18] and MEMIT [19] identify and directly overwrite the weights responsible for specific factual associations. These methods share our goal of avoiding training-time and inference-time cost, but they operate on static, input-independent weight edits, adding or removing a single direction regardless of the input.

WAQS closes this gap: extending Li et al. [12]’s gradient-based recipe from a linear probe to a quadratic one keeps the same recipe map affine (Proposition 1), so it folds into the model weights exactly like the static edits above, while remaining as input-dependent as the curved steering methods it replaces.

Proof of Theorem 1

Proof. Assume the hidden state of the positive and negative groups are respectively Gaussian distributed, as in Assumption 1:

x∣y=+1∼N(μ+,Σ+),x∣y=−1∼N(μ−,Σ−),x∈RD.(14)x \mid y=+1 \sim\mathcal{N}(\mu_{+},\Sigma_{+}), \qquad x \mid y=-1 \sim\mathcal{N}(\mu_{-},\Sigma_{-}), \qquad x \in\mathbb{R}^{D}. \tag*{(14)}

Under Assumption 1, the Bayes-optimal discriminant score is the log-posterior-odds g(x):=log⁡p(y+∣x)−log⁡p(y−∣x)g(x) := \log p(y_{+} \mid x) - \log p(y_{-} \mid x); we identify the probe fQ(x)f_Q(x) with this score, fQ(x):=g(x)f_Q(x) := g(x). By Bayes’ theorem,

g(x)=(log⁡p(x∣y+)+log⁡p(y+)−log⁡p(x))−(log⁡p(x∣y−)+log⁡p(y−)−log⁡p(x))=[log⁡p(x∣y+)−log⁡p(x∣y−)]+[log⁡p(y+)−log⁡p(y−)](15)\begin{aligned} g(x) &= \left(\log p(x \mid y_{+}) + \log p(y_{+}) - \log p(x)\right) - \left(\log p(x \mid y_{-}) + \log p(y_{-}) - \log p(x)\right) \\ &= \left[\log p(x \mid y_{+}) - \log p(x \mid y_{-})\right] + \left[\log p(y_{+}) - \log p(y_{-})\right] \tag*{(15)} \end{aligned}

As the log likelihood could be written as:

log⁡p(x∣μ+,Σ+)=−12(x−μ+)TΣ+−1(x−μ+)−12log⁡∣Σ+∣+C1log⁡p(x∣μ−,Σ−)=−12(x−μ−)TΣ−−1(x−μ−)−12log⁡∣Σ−∣+C1(16)\begin{aligned} \log p(x \mid\mu_{+},\Sigma_{+}) &= -\frac{1}{2}(x-\mu_{+})^{\mathsf{T}}\Sigma_{+}^{-1}(x-\mu_{+})-\frac{1}{2}\log|\Sigma_{+}|+C_{1} \\ \log p(x \mid\mu_{-},\Sigma_{-}) &= -\frac{1}{2}(x-\mu_{-})^{\mathsf{T}}\Sigma_{-}^{-1}(x-\mu_{-})-\frac{1}{2}\log|\Sigma_{-}|+C_{1} \tag*{(16)} \end{aligned}

Here, C1=−12(Dlog⁡(2π))C_{1}=-\frac{1}{2}(D\log(2\pi)), where DD is the dimension of xx (consistent with Assumption 1). So

g(x)=−12xT(Σ+−1−Σ−−1)x+(Σ+−1μ+−Σ−−1μ−)Tx+C2(17)g(x)=-\frac{1}{2}x^{\mathsf{T}}(\Sigma_{+}^{-1}-\Sigma_{-}^{-1})x+(\Sigma_{+}^{-1}\mu_{+}-\Sigma_{-}^{-1}\mu_{-})^{\mathsf{T}}x+C_{2} \tag*{(17)}

where C2=−12(μ+TΣ+−1μ+−μ−TΣ−−1μ−+log⁡∣Σ+∣−log⁡∣Σ−∣)+(log⁡p(y+)−log⁡p(y−))C_{2}=-\frac{1}{2}(\mu_{+}^{\mathsf{T}}\Sigma_{+}^{-1}\mu_{+}-\mu_{-}^{\mathsf{T}}\Sigma_{-}^{-1}\mu_{-}+\log|\Sigma_{+}|-\log|\Sigma_{-}|)+(\log p(y_{+})-\log p(y_{-})) Matching coefficients against fQ(x)=x⊤Wpx+wp⊤x+bpf_Q(x) = x^\top W_p x + w_p^\top x + b_p (Eq. equation 1) with g(x)=fQ(x)g(x) = f_Q(x) gives

Wp=−12(Σ+−1−Σ−−1),wp=Σ+−1μ+−Σ−−1μ−,bp=C2,(18)W_p = -\frac{1}{2}\left(\Sigma_+^{-1} - \Sigma_-^{-1}\right), \quad w_p = \Sigma_+^{-1}\mu_+ - \Sigma_-^{-1}\mu_-, \quad b_p = C_2, \tag*{(18)}

which is exactly Theorem 1, since C2C_2 above is identical to the closed form for bpb_p stated there. □

Additional Experimental details

Experimental Setup

Experiment Environment We conduct all experiments on a server with two Intel Xeon 6326 CPUs and 6 Nvidia A6000 GPUs, using PyTorch version 2.4.1.

Models and Representations. Because our evaluation spans several behavioral steering tasks, we follow task-specific experimental settings and use different backbone models where appropriate. Within each task, all steering methods are evaluated on the same backbone and activation location to ensure a controlled comparison. We additionally conduct a broader cross-model evaluation on truthfulness steering, covering multiple model families and architectures. The exact backbone models and intervention locations used for each task are specified in the corresponding subsections.

Probe estimation. We evaluate both quadratic-probe estimators introduced in Section 3.2 that we may use a discriminatively trained low-rank quadratic probing (LR-QP) or a shrinkage-based GDA quadratic probing (GDA-QP).

Steering and model selection. For each steering method, we tune the intervention strength α\alpha on a held-out validation set using the task-specific steering metric. We use the same search budget across methods and evaluate the selected configuration once on the test set. Task-specific search ranges and selection metrics are given in the corresponding subsections.

Baselines and reporting. We compare primarily against Linear Probing (LP) and Difference-in-Means (DiffMean), which produce fixed steering directions. We additionally include task-specific baselines where appropriate.

Weight absorption applies across model positions. Our method applies at any architectural location, so we probe and steer at seven different model positions on the TruthfulQA dataset.

Takeaway. Steering at residual_pre_attn (layer 23) achieves the best MC1 (0.300), while steering at mlp_output (layer 31) achieves the best fluency (CE 1.647). Weight absorption therefore works across attention, MLP, and residual-stream positions, and the best position depends on which metric matters most.

Prompt Templates

Truthfulness Score

To ensure the reproducibility of our experiments, we provide the exact prompt templates used for our model evaluations in Table 1.

Box 1: Prompt Template for Informativeness

Figure 1 (PDF p. 14). Box 1: Prompt Template for Informativeness

System Prompt: You are an evaluator. Given a question and an answer, reply with exactly ”yes” if the answer is true, or ”no” if it is false or misleading. Reply with only ”yes” or ”no”.

User Prompt / Context: Q: What is the smallest country in the world that is at least one square mile in area? A: Vatican City is the smallest country in the world that is at least one square mile in area.

Response Format Instruction: Output your answer strictly in yes or no:

yes

Informativeness Score

Box 2: Prompt Template for truthfulness

System Prompt: You are an evaluator. Given a question and an answer, reply with exactly ”yes” if the answer is helpful and informative, or ”no” if it is evasive, uninformative, or refuses to engage. Reply with only ”yes” or ”no”.

User Prompt / Context: Q: What is the smallest country in the world that is at least one square mile in area? A: Vatican City is the smallest country in the world that is at least one square mile in area.

Response Format Instruction: Output your answer strictly in yes or no:

yes

Computational Efficiency

While quadratic probing introduces zero inference overhead, we examine the cost of training a quadratic probe. Table 5 summarizes the computational overhead of Linear Probing and Quadratic Probing across different low-rank configurations. Quadratic Probing at ranks 1–8 incurs roughly a 2× increase in per-probe training time while keeping peak GPU memory nearly identical to Linear Probing. Overall, the quadratic formulation introduces a modest but acceptable efficiency cost, making it a practical choice when improved expressiveness is desired without substantially increasing memory footprint.

Method#ParamsTime/Probe (s)Peak GPU (MB)
Linear Probing4,0970.749 ± 0.02922.5
Quadratic (rank=1)12,2891.478 ± 0.01122.5
Quadratic (rank=2)20,4811.455 ± 0.00322.6
Quadratic (rank=4)36,8651.458 ± 0.00122.7
Quadratic (rank=8)69,6331.484 ± 0.00923.0
Quadratic (rank=16)135,1692.043 ± 0.31123.5

Table 5. Computational cost comparison between Linear Probing and Quadratic Probing at varying ranks, benchmarked on a single probe (layer=12, pos=1) with dmodel=4096d_{\mathrm{model}} = 4096, ntrain=256n_{\mathrm{train}} = 256, nepochs=200n_{\mathrm{epochs}} = 200.

We also report the inference consumption under base, absorption and linear intervention(which is the cheapest intervention method) in Table 6. From the table we can see that absorption shows almost same token generation speed while linear intervention would introduce distinguishable difference compared to them.

Methodtokens/s(prefill)tokens/s(decode)TTFT(ms)
Base5040.4 ± 14.52636.7 ± 1.528.46 ± 0.08
Quadratic Absorption5037.5 ± 13.72636.7 ± 1.528.47 ± 0.08
Linear Intervention4996.6 ± 14.52629.1 ± 1.728.57 ± 0.08

Table 6. Computational cost comparison between Linear direction intervention and Quadratic Absorption on Llama-3.1-8B.

Ablation Study on Ranks

PositionLayerα\alphaMC1 ↑\uparrowMC2 ↑\uparrowCE ↓\downarrow
residual_pre_attn2380.30000.26061.6604
input_layernorm_out31-80.26670.25231.6586
attn_output15-20.26670.25551.6617
residual_mid24160.28330.25461.6603
post_attention_layernorm_out3140.28330.25831.6579
mlp_output31240.26670.25881.6470
residual_out3120.28330.25561.6614

Table 9. Activation steering results across different positions and layers. The baseline corresponds to α=0\alpha= 0.

For each refusal baked-in method, we perform a grid search over α∈{0.05,0.1,0.2,0.5,1.0}\alpha\in\{0.05, 0.1, 0.2, 0.5, 1.0\} and evaluate Refusal Score [3], Over-Refusal Score, and Perplexity on a hold-out validation set. Perplexity (PPL, ↓\downarrow) measures the model’s language modeling capability on the Alpaca dataset after injection; lower values indicate better utility preservation. Refusal Score (↑\uparrow) is the log-odds of refusal tokens over compliance tokens at the first generated position on harmful instructions, i.e., log⁡P(refusal)−log⁡P(compliance)\log P(\text{refusal}) - \log P(\text{compliance}), where higher values indicate stronger refusal behavior. Over-Refusal Score (↓\downarrow) is the same log-odds metric computed on harmless instructions, where lower values indicate less false-positive refusal on benign inputs. We select the α\alpha that maximizes the improvement in refusal score, subject to the Over-Refusal Score remaining within [−13,−12][-13, -12] and perplexity not exceeding 6, because the original model without intervention achieves a perplexity of 5.002 and an Over-Refusal of −13.219.

Table 7 reports the evaluation results across different ranks for Quadratic Probing on Llama-2-7b. Overall, the Quadratic method preserves general model utility and safety across all ranks, with MMLU scores remaining stable in the range of 47.20–47.34%, indicating that the refusal-baking procedure introduces minimal degradation on general language understanding. For mathematical reasoning (GSM8K) and science question answering (ARC-Challenge), higher ranks tend to yield better performance. Meanwhile, rank=4 achieves the strongest safety performance on TAP (1.0) and GPTFuzz (0.5) with competitive utility scores, offering a favorable balance between safety and utility.

Qualitative Steering Examples on AxBench

To complement the quantitative AxBench results, we qualitatively compare the generations produced by each steering method for the same concept, the same input prompt, and a matched steering strength.

Box 3: Example 1 – [references and citations within text]

Model: [google/gemma-2-2b-it] Layer / Position: [10] Direction: [increase concept]

Prompt:

[Create a web scraper that extracts specific data from a given website]

MethodGenerated Response
No steering (baseline)[I can’t create a web scraper for you directly. I’m a text-based AI, so I can’t interact with websites or execute code. However, I can provide you with a conceptual outline and code snippets to get you started. **Understanding Web Scraping** Web scraping]
DiffMeans[Let’s dive into the world of web scraping! ** What is web scraping?** Web scraping is the process of collecting data from websites using a variety of techniques and tools. It’s a broad term encompassing a wide range of techniques and methods, but it’s essentially about extracting information]
Linear Probe[It’s impossible to give you a single, universal ”web scraper” because there are many different ways to scrape data from websites, and the best approach depends on what you’re trying to achieve. Here’s a breakdown of some common web scraping techniques and what they’re used for:]
Quadratic Probe (ours)[Let’s talk about web scraping! **What is web scraping?** Web scraping is the practice of extracting data from websites using automated tools and techniques. It’s a powerful tool for gathering information, but it’s not just about ”scraping” - it’s about understanding the different ways]

Table 10.

Box 4: Example 2 – [references and citations within text]

Model: [google/gemma-2-2b-it]  Layer / Position: [10]  Direction: [increase concept]

Prompt:

[Olivia earns $9 per hour. She worked 4 hours on Monday, 3 hours on Wednesday and 6 hours on Friday. How much money did Olivia make this week? ]

MethodGenerated Response
No steering (baseline)[Here’s how to calculate Olivia’s earnings: * **Monday’s earnings:** $9/hour * 4 hours = $36 * **Wednesday’s earnings:** $9/hour * 3 hours = $27 * **Friday’s earnings:** $9/hour *]
DiffMeans[**Olivia’s earnings this week depend on the context. ** ** To answer your question, we need to know what kind of work is she doing?** **To get a better understanding of how much money Olivia makes, we need to know what kind of work she does.**]
Linear Probe[Olivia earned **$9 per hour** x **4 hours** = **$36** on Monday. To get the full picture, you need to consider the rest of the week: * **Olivia earned $9 per hour x 4 hours** = **$36** on Wednesday. ]
Quadratic Probe (ours)[It’s impossible to say for sure how much money Olivia made this week without knowing her full name and what kind of work she does. **To answer your question, we need to know more about Olivia’s work:** * **What kind of work does Olivia do?** * **]

Table 11.

References

  1. [1]Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panicksșery, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24, Red Hook, NY, USA, 2024. Curran Associates Inc. ISBN 9798331314385.arxiv.org/abs/2406.11717
  2. [2]Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jérémy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, et al. Open problems and fundamental limitations of reinforcement learning from human feedback. arXiv preprint arXiv:2307.15217, 2023.arxiv.org/abs/2307.15217
  3. [3]Vishnu Kabir Chhabra and Mohammad Mahdi Khalili. Towards understanding and improving refusal in compressed models via mechanistic interpretability, 2025. URL https://arxiv.org/abs/2504.04215.
  4. [4]Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv:1803.05457v1, 2018.arxiv.org/abs/1803.05457
  5. [5]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021.arxiv.org/abs/2110.14168
  6. [6]Lang Gao, Jinghui Zhang, Wei Liu, Fengxian Ji, Chenxi Wang, Zirui Song, Akash Ghosh, Youssef Mohamed, Preslav Nakov, and Xiuying Chen. The cylindrical representation hypothesis for language model steering, 2026. URL https://arxiv.org/abs/2605.01844.
  7. [7]Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR), 2021.
  8. [8]Jakub Hoscilowicz, Adam Wiącek, Jan Chojnacki, Adam Cieslak, Leszek Michon, Vitalii Urbaneych, and Artur Janicki. Non-linear inference time intervention: Improving llm truthfulness. arXiv preprint arXiv:2403.18680, 2024.arxiv.org/abs/2403.18680
  9. [9]Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen. Catastrophic jailbreak of open-source llms via exploiting generation. arXiv preprint arXiv:2310.06987, 2023.arxiv.org/abs/2310.06987
  10. [10]Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannah Hajishirzi, and Ali Farhadi. Editing Models with Task Arithmetic. In International Conference on Learning Representations, 2023. URL https://mlanthology.org/iclr/2023/ilharco2023iclr-editing/.arxiv.org/abs/2212.04089
  11. [11]Su Hyeong Lee and Risi Kondor. Deep minds and shallow probes, 2026. URL https://arxiv.org/abs/2605.11448.
  12. [12]Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from a language model. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 41451–41530. Curran Associates, Inc., 2023. URL https://proceedings.neurips.cc/paper_files/paper/2023/file/81b8390039b7302c909cb769f8b6cd93-Paper-Conference.pdf.arxiv.org/abs/2306.03341
  13. [13]Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods, 2022. URL https://arxiv.org/abs/2109.07958.
  14. [14]Samuel Marks and Max Tegmark. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. arXiv preprint arXiv:2310.06824, 2023.arxiv.org/abs/2310.06824
  15. [15]Mantas Mazeika, Andy Zou, Norman Mu, Long Phan, Zifan Wang, Chunru Yu, Adam Khoja, Fengqing Jiang, Aidan O’Gara, Ellie Sakhaee, Zhen Xiang, Arezoo Rajabi, Dan Hendrycks, Radha Poovendran, Bo Li, and David Forsyth. Tdc 2023 (llm edition): The trojan detection challenge. In NeurIPS Competition Track, 2023.
  16. [16]Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. 2024.arxiv.org/abs/2402.04249
  17. [17]Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. Tree of attacks: Jailbreaking black-box llms automatically. Advances in Neural Information Processing Systems, 37:61065–61105, 2024.arxiv.org/abs/2312.02119
  18. [18]Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 17359–17372. Curran Associates, Inc., 2022. doi: 10.52202/068431-1262. URL https://proceedings.neurips.cc/paper_files/paper/2022/file/6f1d43d5a82a37e89b0665b33bf3a182-Paper-Conference.pdf.DOI
  19. [19]Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in a transformer, 2023. URL https://arxiv.org/abs/2210.07229.
  20. [21]Ethan Perez, Sam Ringer, Kamile Lukosiute, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, Andy Jones, Anna Chen, Benjamin Mann, Brian Israel, Bryan Seethor, Cameron McKinnon, Christopher Olah, Da Yan, Daniela Amodei, Dario Amodei, Dawn Drain, Dustin Li, Eli Tran-Johnson, Guro Khundadze, Jackson Kernion, James Landis, Jamie Kerr, Jared Mueller, Jeeyoon Hyun, Joshua Landau, Kamal Ndousse, Landon Goldberg, Liane Lovitt, Martin Lucas, Michael Sellitto, Miranda Zhang, Nearav Kingsland, Nelson Elhage, Nicholas Joseph, Noemi Mercado, Nova DasSarma, Oliver Rausch, Robin Larson, Sam McCandlish, Scott Johnston, Shauna Kravec, Sheer El Showk, Tamera Lanham, Timothy Telleen-Lawton, Tom Brown, Tom Henighan, Tristan Hume, Yuntao Bai, Zac Hatfield-Dodds, Jack Clark, Samuel R. Bowman, Amanda Askell, Roger Grosse, Danny Hernandez, Deep Ganguli, Evan Hubinger, Nicholas Schiefer, and Jared Kaplan. Discovering language model behaviors with model-written evaluations. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguistics: ACL 2023, pp. 13387–13434, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.findings-acl.847. URL https://aclanthology.org/2023.findings-acl.847/.DOI
  21. [22]Giorgio Piras, Raffaele Mura, Fabio Brau, Luca Oneto, Fabio Roli, and Battista Biggio. Som directions are better than one: Multi-directional refusal suppression in language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pp. 32728–32736, 2026.
  22. [23]John A. Ramey, Caleb K. Stein, Phil D. Young, and Dean M. Young. High-dimensional regularized discriminant analysis, 2017. URL https://arxiv.org/abs/1602.01182.
  23. [25]Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. Steering llama 2 via contrastive activation addition. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15504–15522, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.828. URL https://aclanthology.org/2024.acl-long.828/.DOI
  24. [28]Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, Pieter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, and Sam Toyer. A strongreject for empty jail-breaks, 2024.
  25. [29]Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca, 2023.
  26. [30]Minh Hieu Vu and Tan Nguyen. Angular steering: Behavior control via rotation in activation space. Advances in Neural Information Processing Systems, 38:121653–121690, 2026.arxiv.org/abs/2510.26243
  27. [32]Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atticus Geiger, Dan Jurafsky, Christopher D. Manning, and Christopher Potts. Reft: Representation finetuning for language models, 2024b. URL https://arxiv.org/abs/2404.03592.
  28. [33]Zejia You, Chunyuan Deng, and Hanjie Chen. Spherical steering: Geometry-aware activation rotation for language models, 2026. URL https://arxiv.org/abs/2602.08169.
  29. [34]Jiahao Yu, Xingwei Lin, Zheng Yu, and Xinyu Xing. Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts. arXiv preprint arXiv:2309.10253, 2023.arxiv.org/abs/2309.10253
  30. [35]Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering: A top-down approach to ai transparency. arXiv preprint arXiv:2310.01405, 2023a.arxiv.org/abs/2310.01405

Paper details

vlnc:W_6k9e2qkg
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

generative AI tools

View statement
Looking for related papers…
Contents