WorksComputer Science
Creating Steering Vectors via Input Optimisation
Abstract
Activation steering influences a language model's behaviour by adding a steering vector to its residual stream during generation, but it requires white-box access to modify the hidden states at every generation step. We introduce ActGCG, a modification of Greedy Coordinate Gradient which optimises a suffix to match the target activations obtained by adding a steering vector to the residual stream. We further introduce universal suffixes trained across many prompts at once, a model-translation step that rewrites the suffix into human-readable text midway through optimisation, and a soft-suffix variant to remove the discrete token space constraint. We study two behaviours: steering a model to answer in German, and anti-refusal steering for jailbreaking. For German, a discrete suffix reproduces the steering vector's behaviour almost perfectly and transfers well across prompts and models. For anti-refusal, the equivalence is weaker, succeeding under a phrase-avoidance metric but only partially under the stricter HarmBench classifier. Our results show that a steering vector's behaviour is reachable from the input, with no access to model internals at generation time, up to a loss that depends on the concept being steered.
Bachelor’s Thesis
Supervisors:
Lukas Fluri, Prof. Dr. Florian Tramèr
To Dennis
Acknowledgements
I’m grateful to Lukas Fluri, my supervisor, for guiding me throughout this thesis. Without his support, this project would not have been possible. I’m also grateful to the whole SPY Lab team, as their amazing environment has helped me stay strongly motivated throughout the semester. Last but not least, I am thankful to my friends and family, especially my brother Dennis, and my parents Tolga and Banu, for always being there for me.
Thank you.
Introduction
The behaviour of large language models can be influenced by directly editing their internal activations. A common method of achieving this is to use activation steering, where one computes a steering vector to induce a target behaviour and adds the vector to the residual stream during generation to push the model towards it. One can build such a vector by averaging the difference in activations between two datasets with prompts that do and do not exhibit the behaviour [21]. Steering vectors are useful as they’re cheap to produce and interpretable. However, since they are an intervention in the activation space, one requires full access to the model’s weights as well as the ability to modify hidden states at every generation step. This is not feasible when a model can only be reached through a chat interface.
At the same time, model behaviour can be changed automatically from the input alone. Greedy Coordinate Gradient (GCG) [30] is a suffix finding method to cause a model to yield an affirmative string such as “Sure, here is how to. . . ”, also called the target string. The adversarial suffix is optimised by only changing the suffix tokens themselves to maximise the chance of obtaining the target as output, without directly modifying internal activations.
We look for an equivalence relationship between suffixes and steering vectors by combining these two methods, and investigating whether the effect of a steering vector can be reproduced by an input suffix. To this end, we modify the loss function of GCG to reach a new algorithm, which we call ActGCG. Instead of targeting an output string, ActGCG targets the activations that a steering vector would have induced. The vector is first given on a set of filler tokens to obtain the target activations. The algorithm then searches for a discrete suffix whose activations are on those same tokens to match the target, minimising the squared difference summed over positions and over several layers. If such a suffix exists, it means that the steering vector’s behaviour was reachable from the input, with no access to the model’s internals at generation time.
We study two behaviours. The first task is to steer a model to answer in German, using a contrastive German/English steering vector built from the Dolly instruction dataset [4]. The second is anti-refusal steering, where the vector is negated so that matching the target pushes the model away from refusing a harmful request, and we measure success with the HarmBench jailbreak classifier [15] as well as checking if a model avoids a set of refusal phrases. For German, the discrete suffix reproduces the steering vector’s behaviour almost perfectly. For the anti-refusal direction, we achieve a high success rate for phrase avoidance, but lower success rate for jailbreaking.
Beyond the core method, we make three further contributions. First, we introduce a soft-suffix variant that replaces the discrete tokens with continuous embeddings optimised directly by gradient descent. We use this as an upper bound on how well the activation-matching objective can be met, as it is unconstrained by the discrete token space. Second, we move from single prompts to universal suffixes: a single string trained across multiple prompts at once, which we then use to study transfer across new prompts of the same model, and across different models. Third, we add a model-translation step to the universal setting, where in the middle of the optimisation steps the unreadable suffix is rewritten into fluent, human-readable text using another model. We then continue the optimisation as before.
The rest of the thesis is organised as follows. Chapter 2 reviews the linear representation hypothesis, steering vectors, and the GCG algorithm. Chapter 3 introduces the ActGCG method and the experimental setup. Chapter 4 presents our experiments on the German and anti-refusal directions, covering per-prompt, universal, model-translation, and soft suffixes, as well as transferring across models. Chapters 5 and 6 discuss the findings and conclude.
Background
We first discuss the internal mechanisms of a Large Language Model, explain the role of steering vectors in this context, and elaborate on the GCG algorithm.
Layers and Tokens
Large Language Models are built up of stacks of layers between an input and an output. These layers allow the model to represent increasingly complex functions. Each layer consists of a Multi-Layer Perceptron (MLP) and multiple attention heads. The MLP processes information in each token position, whereas attention moves information between token positions.
A token is a unit of text that the model can process, represented by a vector of numbers. We have two notions of a token. A discrete token corresponds to a word of the model’s fixed vocabulary. In contrast, a soft token is an arbitrary vector in embedding space that does not necessarily correspond to a word in a vocabulary. Hence, allowing soft tokens gives an optimiser access to the entire continuous embedding space rather than the discrete vocabulary.
Activations
For each layer and each token position, the transformer [24] of a Large Language Model maintains a vector that is updated by the attention and MLP sublayers. Attention combines vectors from earlier positions, and the MLP applies a non-linear transformation per position. The transformer adds each sublayer’s output back onto its input through a residual connection [24]. More formally, following the residual-stream view of Elhage et al. [5] and ignoring layer normalisation for clarity, we let be the vector at token position after layer . We first have attention write into the stream:
Then, we let the MLP read that post-attention value and write into it:
Therefore, each sublayer writes into a running sum. This vector is variously called the hidden state, the activation vector, or the residual stream, and it lives in a high-dimensional activation space. The residual form of Equation (2) is what makes activation steering possible, as an intervention added to is carried forward to every subsequent layer.
The linear representation hypothesis claims that human-interpretable concepts are represented along linear subspaces[16, 19, 2, 6, 18, 17], i.e. as directions in the activation space. We adopt this assumption in our work. For example, if a concept corresponds to a direction , we assume that moving an activation along strengthens the concept. The difference between the mean activations of examples that do and do not exhibit the concept is an estimate of , which we explore as steering vectors.
Prompt structures with chat templates
Instruction-tuned models wrap user text in a chat template. A template uses additional tokens to determine where the user begins and ends, and where the assistant turn starts. The number of chat template tokens differs from model to model, which can change the optimal token position to extract steering vectors. For our method, since we read the activations at fixed positions between the user prompt and the response, the chat template determines the token positions to be extracted. We refer to this as filler positions as given in Chapter 3.
Steering Vectors
Under the linear representation hypothesis, we can influence a model’s output by adding a vector to the residual stream at a chosen layer and token position. Concretely, given a steering vector for a target concept and a strength , we replace the activation during generation by
which pushes the model towards the concept for and away from it for . Steering vectors can nudge models along behaviours of varying complexity: the output language, refusal, sycophancy (telling the user what they want to hear), and so on. They are computationally easy to build and interpretable by humans.
The underlying intuition predates language models. In classical word embeddings, semantic relationships appear as consistent directions, so that vector arithmetic such as king man woman queen recovers analogies (Figure 2.1). Steering vectors carry this intuition into the residual stream of a transformer.

Figure 2.1. Semantic relationships as directions in an embedding space.1
Steering Vector Algorithms
Several methods exist for constructing the direction ; they differ mainly in how many contrast examples they use, at which layer and position they read activations, and whether they apply a further linear transformation. We summarise the most common ones.
Activation Addition (ActAdd) [23] takes a single contrast pair, runs both prompts through the model, records the activations at one layer, and subtracts them to obtain the steering vector, which is then added at one token position during generation.
Contrastive Activation Addition (CAA) [21] generalises to many contrast pairs. For each pair it computes the positive and negative activations at a single layer and last token position and takes the mean of their differences across all pairs, adding the resulting vector at every token position during generation. CAA is evaluated on both multiple-choice and open-ended prompts. Our own steering vectors follow this difference-of-means recipe (Chapter 3).
Inference-Time Intervention (ITI) [13] considers the attention-head space rather than the full residual stream. Given a dataset of true/false question–answer pairs, it records the activation of each attention head, trains a linear probe per head to separate true from false, and keeps the top- most predictive heads. For each such head it takes the mean-difference direction between true and false examples and shifts activations along the direction, targeting truthfulness specifically.
Representation Engineering (RepE) [29] collects contrast pairs where the contrast is only in the target concept, and reads activations (by default at the last token) for both. For each layer it forms the difference vectors and runs principal component analysis over them; the first principal component is taken as the reading direction for that concept at that layer, which can then be used either as a classifier (by projection) or for control (by addition or subtraction).
Steering based on sparse autoencoders (SAEs), which impose a sparsity constraint so that only a few hidden units activate at once, is also possible but is harder to evaluate and has not shown a clear advantage in comparable settings.2
We adopt the simpler and well-supported difference-of-means (CAA-style) construction.
GCG mechanism
Greedy Coordinate Gradient (GCG) [30] optimises a suffix appended to a prompt such that the model begins its answer with a chosen affirmative target string. For example, this can be a result starting with “Sure, here is how to. . . ”. To obtain such a suffix, the algorithm works in the following manner:
For a given prompt, our loss function is the cross entropy (i.e. the “surprise”) of the target tokens given the prompt and the suffix, which we minimise. For a current suffix, we first take the embeddings of the tokens and find the gradients of the loss with respect to these embeddings. We compute the top-k promising tokens per position for a replacement. Then, for each batch, we choose a random token position in the suffix, and replace it with a uniformly sampled token from the top-k options. We receive a number of candidate suffixes, and choose the suffix of the batch that minimises the loss (and hence maximises the chance of reaching our target string). We repeat this T times.
Crucially, GCG searches over discrete tokens. Figure 2.2 states the algorithm formally.

Figure 2.2. The Greedy Coordinate Gradient (GCG) algorithm, reproduced from Zou et al. [30].
GCG modifications and alternatives
There are numerous ways of modifying GCG, as well as alternatives to the GCG algorithm. We explore three such methods.
Steering-SSR [26] is the most similar prior method. It optimises a suffix so that the model’s activations match a target where a steering vector is added, but using a cosine-similarity loss rather than a squared distance.3 Writing for the suffix’s readout activation and for the unit steering direction at layer , its loss sums a per-layer squared cosine term over the chosen layers,
where the per-layer target sets the desired cosine. We were unable to reproduce the reported jailbreak success rates, since re-judging the paper’s outputs with our own classifier (see Chapter 3) gave substantially lower rates than reported, which we attribute to the choice of jailbreak judge.
PEZ (Hard Prompts Made Easy) [25] performs gradient-based discrete optimisation by combining soft and discrete updates. In each iteration it projects the current continuous embeddings onto their nearest discrete vocabulary tokens, computes the gradient at the projected point, and applies that gradient to update the continuous iterate. A final projection yields the discrete output. Figure 2.3 states the algorithm formally.

Figure 2.3. The PEZ gradient-based discrete optimisation scheme, reproduced from Wen et al. [25]
Algorithm 1 Hard Prompts made EaZy: PEZ Algorithm
Input: Model $\theta$, vocabulary embedding $\mathbf{E}^{|V|}$, projection function $\operatorname{Proj}$, broadcast function $\mathcal{B}$, optimization steps $T$, learning rate $\gamma$, Dataset $D$
Sampled from real embeddings:
$\mathbf{P} = [\mathbf{e}_1, \ldots, \mathbf{e}_M] \sim \mathbf{E}^{|V|}$
for $1, \ldots, T$ do
Retrieve current mini-batch $(X,Y) \subseteq D$.
Forward Projection:
$\mathbf{P}' = \operatorname{Proj}_{\mathbf{E}}(\mathbf{P})$
Calculate the gradient w.r.t. the projected embedding:
$\mathbf{g} = \nabla_{\mathbf{P}'}\mathcal{L}_{\mathrm{task}}(\mathcal{B}(\mathbf{P}', X_i),Y_i,\theta)$
Apply the gradient on the continuous embedding:
$\mathbf{P} = \mathbf{P} - \gamma\mathbf{g}$
end for
Final Projection:
$\mathbf{P} = \operatorname{Proj}_{\mathbf{E}}[\mathbf{P}]$
return $\mathbf{P}$
Finally, we can remove the discrete token constraint altogether by considering soft suffixes. We use this as a comparison method in § 5.15.
Methodology
We now introduce our method. We first describe how we construct the steering vectors that define each target behaviour, and the ActGCG objective that searches for a suffix reproducing the activations when a steering vector was added. Then, we follow with the universal and model-translation versions of the objective, our tasks and datasets, as well as the models, judges that were used together with the comparisons to other methods.
Constructing Steering Vectors
We build a steering vector for each behaviour with a difference-of-means construction inspired by Contrastive Activation Addition [21]. For a set of contrast pairs (such as German against English), we run both prompts through the model, and read the residual-stream activation at a chosen layer and token position. We average the difference across all pairs. Since our loss matches activations at several layers at once (§ 4.2), we build one such vector per target layer and stack them together. For every model, we use four layers, with a preference towards the middle and later layers where steering vectors are commonly applied, as listed in Table 3.1.
| Model | Total layers | Chosen layers |
| Gemma-2-2B-it | 26 | 10, 14, 18, 22 |
| Qwen3-1.7B | 28 | 10, 13, 16, 20 |
| Llama-3.2-3B | 28 | 12, 16, 20, 24 |
| Qwen3-4B | 36 | 14, 18, 22, 28 |
Table 3.1. The four layers at which each model’s steering vectors are built and at which the ActGCG objective matches activations.
We choose a CAA style method because it is both simple to implement and effective relative to its alternatives. The best method to obtain steering vectors overall is not yet fully clear; for instance, Im and Li [11] reports that mean-difference vectors outperform other steering constructions, whereas Pres et al. [20] argues that CAA is less optimal than what earlier work suggested. We proceed with it regardless, as we found no alternative that gives a clear improvement for our purposes.
For the language task, we first tried Turkish. However, we noticed that the level of knowledge for the language was quite limited. For example, Figure 3.1 illustrates that Llama-2-7B has German as the second largest recognised language [22]. As a result, we believed German might be the more optimal language choice. For the German vector, we use the Dolly instruction dataset [4]. We pair each German (both question and response) prompt (positive) with its English counterpart (negative). We read the activations near the end of the response (position , or second to last token). The difference of means gives our “answer in German” direction.

| Language | Percent | Language | Percent |
| en | 89.70% | uk | 0.07% |
| unknown | 8.38% | ko | 0.06% |
| de | 0.17% | ca | 0.04% |
| fr | 0.16% | sr | 0.04% |
| sv | 0.15% | id | 0.03% |
| zh | 0.13% | cs | 0.03% |
| es | 0.13% | fi | 0.03% |
| ru | 0.13% | hu | 0.03% |
| nl | 0.12% | no | 0.03% |
| it | 0.11% | ro | 0.03% |
| ja | 0.10% | bg | 0.02% |
| pl | 0.09% | da | 0.02% |
| pt | 0.09% | sl | 0.01% |
| vi | 0.08% | hr | 0.01% |
Figure 3.1. Language distribution in pretraining data with percentage . Most data is in English, meaning that Llama 2 will perform best for English-language use cases. The large unknown category is partially made up of programming code data. Reproduced from Touvron et al. [22].
For the refusal vector we use the harmful and harmless instruction datasets of Arditi et al. [1], taking the harmful set as positive and the harmless set as negative. Here, unlike the German vector, there is no answer appended after the instruction, so we consider the last tokens before the response. The exact position changes depending on the model. We read at position (last token of the prompt) for Gemma, Llama, and Qwen2.5 [27]; but for Qwen3, since its chat template also inserts an empty reasoning scaffold (<think></think>). Since the resulting vector points towards refusal, to steer against refusal (the jailbreak direction) we negate it, which corresponds to using a negative strength .
Our vector extraction was implemented with the NNSight library [7], which made our efforts simpler in tracing activations at specific layers and token positions. We do not claim our chosen layers and token extraction positions to be optimal (due to the amount of computational resources it would require). However, we chose our token and layer extraction positions after checking the success rates of multiple different combinations.
The ActGCG Objective
Our method, which we call ActGCG, modifies the loss of a GCG implementation (from the NanoGCG library [10]) while keeping its discrete search unchanged. Instead of a cross-entropy towards a target string, we match the activations that a steering vector would have produced.
Concretely, let be a prompt and let the suffix be the tokens that follow it. We call the chat-template tokens that sit after the user prompt and before the generated response the filler tokens, and read activations at those positions. We first build our target. For each target layer we run the plain prompt through the model, add to the residual stream at every filler position, and read the resulting activations at filler position . We then search for a suffix whose own forward pass reproduces those activations. Writing for the activation at layer , filler position , of the prompt with suffix appended, the loss is the squared error summed over filler positions and over four layers:
where is the set of four layers and the filler positions. Minimising Equation (5) with the ordinary GCG search lets us reproduce the steering vector’s effect using a discrete suffix, without activation edits during generation.
We match at the filler positions rather than at the generated response since the response tokens change from run to run and differ across prompts, so the variance is quite high. The filler tokens, by contrast, are a fixed set of chat-template tokens that is present for every prompt and directly before generation. Reading the filler tokens provides a prompt-independent set of positions, and since there are several of them the steering vector signal is stronger from multiple positions at once rather than from a single token.
The variable scales the strength of the steering vector. We sweep across different for both the German and the anti-refusal directions. Unless noted otherwise, we optimise a suffix of 20 tokens initialised to “x x ... x”, for 250 steps with search width 512, top-, replacing a single token per candidate, and a fixed seed for reproducibility.
Universal Suffixes and Model Translation
Our original objective is optimised on a single suffix for a single prompt. We extend our method in two ways to create suffixes that generalise across different prompts and models.
First, we consider a universal suffix. To obtain a single suffix that steers multiple prompts at once, we optimise one shared token string whose loss and gradient are summed over a training set of prompts. At each step, a token swap depends on the sum of the per-prompt gradients. We train on prompts, and evaluate on the training set as well as a set of new prompts to measure generalisation. To check model transfer, we evaluate the universal suffix on the same training prompt set across different models.
Second, a GCG suffix is typically unreadable to humans. To make it interpretable without losing its effect, we adopt the prompt translation idea from Li et al. [14] and insert a model-translation step into the middle of the optimisation steps. We run the universal search for the first half of the steps, and then make a single call to a larger proposer model asking it to rewrite the current suffix into a coherent version of the same length. Afterwards, we resume the search from the rewritten suffix for the remaining steps. We also compare the mid-optimisation step against a variant that translates only once at the very end.
Experimental Setup
We test ActGCG across two concepts, German and refusal.1 We first optimise one suffix for each prompt to match the steering vector target activations at the filler tokens (Equation (5)). We then test our method on universal suffixes that work across multiple prompts, as well as across multiple models. Finally, we apply model translation to the universal suffix, having a larger model rewrite the optimised suffix into readable text midway through the optimisation (§ 4.3).
Models. Our primary models are Gemma-2-2B-it [8], Qwen3-1.7B [28], Llama3.2-3B [9] and Qwen3-4B [28].
Test data. For German the per-prompt experiments use 30 test English prompts from the Dolly dataset [4], which are disjoint from the contrast pairs used to build the steering vector (§ 4.1). The universal experiments train a single suffix on 20 prompts and test the suffix on 30 further prompts. For refusal, we utilise the harmful test set of Arditi et al. [1] in two ways. The breadth split takes one prompt from each of the 22 categories in the dataset, resulting in 22 total prompts. The depth split instead takes 5 prompts from each of 4 training categories, with a disjoint set of 5 other prompts from the same 4 categories as test, as well as an additional set of 5 prompts from 4 categories that are distinct from the training ones (60 prompts in total).
Judging. We judge German success rates with the langdetect library [3], additionally checking whether the response consists of degenerate repetitive phrases. We note that a short response that is genuinely German can still be misclassified when it contains English names or titles. A result is German if and only if the text is non-repetitive and the library gives “de” as output. To judge refusal success rates, following Arditi et al. [1], we first check the model responses across a set of phrases, i.e. whether a part of the output and a phrase match together (e.g. “I cannot help with that”). We then also check for safety, where we use the HarmBench-Llama-2-13b classifier [15] as the model judge. We acknowledge the difficulty in objective scoring for safety, and settled on this judge after comparing it against alternatives such as gpt-5.5, claude-opus-4.8, and HarmBench-Mistral-7b-val-cls. For Claude and GPT, we found issues in obtaining a score from the models due to safety classifiers, and we preferred the Llama classifier over Mistral as it is more widely used in the literature. We treat the HarmBench judge as our main refusal score, as otherwise the phrase based metric scores roughly twice as high as Harmbench (page 23).
Comparisons. We compare ActGCG against three alternatives. Soft suffixes optimise directly over continuous embeddings that do not necessarily correspond to words, where we use gradient descent with Adam [12] to bring the activations as close as possible to the same activation matching loss of Equation (5). Since the optimiser explores a continuous space rather than the discrete vocabulary, a soft suffix can reach the target more easily than a discrete one. The original GCG method [30] minimises the cross entropy towards a fixed string (an affirmative string for refusal, a German string for the language task), determining what a discrete suffix can do without any steering vector. SSR steering (Equation (4)) is the closest prior method, matching the steering direction with a cosine loss rather than the full target activation.
Default settings. Unless a subsection states otherwise, every suffix is 20 tokens initialised to x x ...x and optimised for 250 steps, with search width 512, top- = 256, and a single token replaced per step (as in Section 3). The target activations are built at the four layers of Table 3.1 with steering strength for German and for refusal. The per-prompt, universal and model translation ActGCG runs use a fixed seed of 0 (except for the seed sweep in § 5.2). Each subsection below reports only its deviations from these defaults.
Experiments
Different values of for German
We test different values of to change the strength of steering vectors added in the four chosen layers for a given model. We use ActGCG to optimise a suffix towards the target activations for each of the 30 German test prompts. We sweep on Gemma-2-2B and Qwen3-1.7B, and run Llama-3.2-3B and Qwen3-4B at only.
We find that is a good value for the German steering vector across the four chosen layers, as it gives the highest success rate. The suffix optimised on the German steering vector reliably induces the same behaviour.
Different seeds for different values of alpha
We found some variance in results when running an experiment more than once since the original GCG method is stochastic. To minimise the effect of randomness in our results, we repeat our sweep at for five different seeds (0–4) on Gemma-2-2B. We also fix the default seed to 0 in our ActGCG experiments.
Figure 4.2. Per-prompt German success rate on 30 prompts across , with five different seeds on Gemma-2-2B.
Despite the variance from GCG’s randomness, remains the strongest choice overall.
Suffix lengths
We now vary the number of optimised tokens in the suffix, which we otherwise fix at 20. On Gemma-2-2B we sweep the suffix length over tokens, repeating each length for five seeds (0–4). We test on German prompts.
We find that changing the suffix length can increase the success rate. At the same time, variance is low at suffix token length 20, while increasing suffix length costs more optimisation time but minimal gain. Hence, we kept our experiments to have the same suffix length of 20.
Figure 4.3. Per-prompt German success rate on 30 prompts for different suffix lengths, with one point per seed and the mean.
Checking target on German
We check whether the activation target itself yields German answers. We do this by adding the German steering vector () directly to the residual stream at the filler positions of the four target layers (Table 3.1) before generation (with no suffix involved) using Gemma-2-2B, and measuring the German rate on the same 30 prompts.
| Method | Success (%) |
| Original SV on filler tokens | 90.00 |
Table 4.1. Steering vector added at the filler positions, the target which ActGCG optimises towards (Gemma-2-2B, , 30 prompts).
Our target is worth striving towards; however, the result itself is not 100%. Therefore, the success of our method is capped by the success of the target.
Comparing results to the original GCG method and SSR for German
We compare our method to the original GCG algorithm using the same per-prompt setup. Instead of matching the target activations with a steering vector, the original GCG minimises the cross-entropy loss towards a fixed target string. We determine the target string to be “Hier ist meine Antwort auf Deutsch:” (“Here is my answer in German:”). Otherwise, we use the same German Dolly prompts, same initialisation, and the same 250 steps.
| Model | Original GCG | ActGCG |
| Gemma-2-2B | 36.7 | 86.7 |
| Qwen3-1.7B | 83.3 | 96.7 |
Table 4.2. Comparing ActGCG to the original GCG method with the same per-prompt setup apart from the loss.
We notice that ActGCG performs better for the German concept on both models. We note that this experiment could be further improved by sweeping a set of German phrases for the target string.
Besides GCG, we also compare our method to SSR [26] due to the similarity in goals. Instead of minimising the squared distance to the target activations, SSR drives the cosine similarity between the steering vector and the activations towards a target value (Equation 2.4). The original implementation reads the last prompt token, which differs from our method. Therefore, we run SSR twice; first by faithfully following its single token position, and second by reading the filler positions the same way our method does. We run it on Gemma-2-2B, and the rest of the setup is the same.
| Method | German (%) |
| SSR cosine, faithful | 40.0 |
| SSR cosine, filler | 53.3 |
| ActGCG | 86.7 |
Table 4.3. Comparing ActGCG to the SSR method with the same per-prompt setup and
For Gemma-2-2B, our method outperforms both variants. This indicates that the cosine objective is weaker than our loss. We also see that reading at the filler positions performs better than the single token.
Universal suffixes on German
We now optimise a single universal suffix that steers many prompts at once, rather than having one suffix per prompt. Following § 4.3, a suffix is trained jointly on 20 German prompts, and then evaluated on the same 20 training prompts as well as a set of 30 test prompts. We also transfer the finished suffix from the source model to a different model, evaluated on the 20 training prompts. Where available, we transfer to a similarly sized (or smaller) and a slightly larger model of the same family.
We find that the test set generally performs similarly to or worse than the training set. For Llama-3.2-3B, both sets only had one failed prompt, but the test set showed a slight increase, likely due to larger number of prompts tested. Transferring the suffix to a similarly sized or smaller model performed better than transferring it to a larger model.
| Source model | Evaluation | German (%) |
| Gemma-2-2B | training (20) | 90.0 |
| Gemma-2-2B | test (30) | 73.3 |
| Gemma-2-2B | Gemma-1.1-2B | 65.0 |
| Gemma-2-2B | Gemma-2-9B | 5.0 |
| Qwen3-1.7B | training (20) | 95.0 |
| Qwen3-1.7B | test (30) | 80.0 |
| Qwen3-1.7B | Qwen2.5-1.5B | 95.0 |
| Qwen3-1.7B | Qwen3-4B | 60.0 |
| Llama-3.2-3B | training (20) | 95.0 |
| Llama-3.2-3B | test (30) | 96.7 |
| Llama-3.2-3B | Llama-3.2-1B | 95.0 |
Table 4.4. Universal German suffix, with and 20 train prompts. Training and test are for the source model; transfer applies the same suffix to another model, evaluated on the training prompts.
Model translation for universal suffixes on German
The universal suffix is mostly not interpretable, so we check if a model translation preserves the meaning of the concept better. We run the universal steps as usual for the first half, and then insert a model-translation step in the middle of the steps (at step 125) where we make one call to a proposer model that rewrites the suffix into fluent text. We try two proposers, GPT-5.5 and Llama-3.1-8B. For the remaining steps, we continue as usual. We use the same 20 training and 30 test prompts. We also check a variant end_ask where we ask at the end instead of the middle. Suffixes transferred to other models are evaluated on the 30 test prompts (in comparison, cross-model transfers in § 5.6 were checked on the easier training prompts).
| Source model | Proposer | Training | Test | Gemma-1-2B | Gemma-2-9B | end_ask |
| Gemma-2-2B | GPT-5.5 | 95.0 | 96.7 | 83.3 | 100.0 | 70.0 |
| Gemma-2-2B | Llama-3.1-8B | 85.0 | 83.3 | 83.3 | 100.0 | 0.0 |
| Llama-3.2-3B | GPT-5.5 | 100.0 | 100.0 | – | – | 93.3 |
Table 4.5. Universal modified GCG with model translation, German (, 20 train, 30 test prompts). The proposer rewrites the suffix in the middle of the steps (step 125); end_ask rewrites only after the final step instead. Transferred suffixes are evaluated on the 30 test prompts.
We find that translating the suffix with a model in the middle of the steps performs better in inducing German behaviour than using no translation step, as well as in generalising to new prompts and transferring to other models.
Tracking the change of loss for German
We log the cosine similarity (between the suffix’s filler token activations and the target activations) and the distance to the target every 25 steps at each of the four target layers, for both the universal suffix and the model-translation run.
Universal suffix. We check how the cosine similarity and the distance loss changes throughout the ActGCG steps. We log both every 25 steps at each of the four target layers, for universal suffix experiments (Figure 4.4). Figure 4.4 sums both losses over the layers and Appendix § 8.1 considers the cosine loss for each of the chosen layers (with one target per layer). Figure 4.5 shows the contribution of each chosen layer to the loss. Figure 4.6 considers the change in German rate for the given suffix every 25 steps. We notice overall that the loss decreases and the results get closer to the target. However, we also see that we get a roughly logarithmic curve, and increasing the number of steps for longer than 100 steps doesn’t significantly change the loss. The same descent holds for the individual per-prompt runs (Appendix § 8.2).
Figure 4.4. distance to the target activations (normalised to its step-0 value) and cosine similarity between the filler-token activations and the target, every 25 steps and averaged across the four target layers, for the universal German run with and 20 training prompts.
Figure 4.5. Per-layer activation matching for the universal German suffix (). The four target layers’ distances to the target, stacked.
Figure 4.6. Fraction of train prompts answered in German every 25 steps, for the universal German suffix.
Model translation. We repeat the same procedure for the model-translation run. Figure 4.7 sums the distance and averages the cosine over the layers, and Figure 4.8 breaks the down per layer. Figure 4.9 tracks the German rate over the run. We observe a sharp change in loss when the suffix is translated by the proposer model. Despite the fact that model translation experiments produce higher success results, we do not observe a significant change in loss at the end of the steps. Nevertheless, we believe the re-initialisation in the middle of the steps might drive the model to explore new suffixes better.
Figure 4.7. Model-translation German run: (normalised to step 0) and cosine to the target over steps, averaged across layers. The dashed line marks the mid-run proposer rewrite.
Figure 4.8. Per-layer activation matching for the model-translation German run for . Stacked per-layer distances; the dashed line is the step-125 proposer rewrite.
Figure 4.9. Fraction of train prompts answered in German every 25 steps for the model-translation run.
Refusal
We now apply ActGCG to the anti-refusal direction rather than German, on 22 harmful prompts (using the breadth split, one prompt from each category). For every prompt, we optimise a single suffix. We sweep for , and compare against a “None” baseline where we don’t optimise for any suffix. The jailbreak rates are judged with HarmBench-Llama-2-13b. We note that values are negated in the code for the anti direction.
The suffix induces jailbreaking more often than the unsteered baseline, which indicates that our optimisation induces the anti-refusal behaviour. At the same time, the steering strength is much weaker than for German (Figure 4.1).
Figure 4.1. Per-prompt German success rate (langdetect) versus the steering strength , for each of the four main models on 30 prompts. Each scales the steering vector that defines the activation target.
On the suffix itself, Gemma-2-2B and Qwen3-1.7B together do slightly better at than at , and the rate then falls as grows further. We nonetheless adopt as the default, since the reachable target itself peaks at while is too weak (§ 5.10). The suffix still matches the target well at but degrades beyond it.
Figure 4.10. Anti-refusal direction, breadth split, 22 prompts, judged with HarmBench. The grey band marks the unsteered baseline (None) range across the four models (4.5–13.6%).
Checking our target for refusal
We check whether the activation target itself yields jailbreaking answers, as we did for the German direction. We add the refusal steering vector for directly into the residual stream at the filler positions of the four target layers (Table 3.1) before generation on Gemma-2-2B, with no suffix involved, and measure both metrics on the same 22 breadth-split prompts (one per category).
| HarmBench (%) | Phrase-based (%) | |
| 0.5 | 22.73 | 63.64 |
| 1.0 | 50.00 | 81.82 |
| 1.5 | 50.00 | 90.91 |
| 2.0 | 40.91 | 77.27 |
Table 4.6. Refusal steering vector applied at the filler positions, the target ActGCG optimises towards (Gemma-2-2B, breadth split, ). HarmBench jailbreak rate and phrase-based anti-refusal rate (%).
The HarmBench target peaks at , while the phrase-based rate is roughly twice as high. The target never reaches 100%, hence the success of our ActGCG method is capped by it. It is also far weaker than its German counterpart (90.00%, § 5.4), which may be the reason for the lower refusal success rates throughout our experiments.
Comparing results to the original GCG method and SSR for refusal
Similar to § 5.5, we now compare our method to the original GCG algorithm with the same per-prompt setup for refusal. We use the same prompts with the same suffix initialisation, as well as the same number of 250 steps. We compare for both the phrase based metric and the safety metric for jailbreaking.
| Model | Original GCG | ActGCG | ||
| HarmBench | Phrase | HarmBench | Phrase | |
| Gemma-2-2B | 13.6 | 36.4 | 27.3 | 90.9 |
| Qwen3-1.7B | 50.0 | 100.0 | 40.9 | 90.9 |
Table 4.7. Per-prompt refusal, breadth split (22 prompts), comparing the original GCG against ActGCG with . HarmBench jailbreak rate and the phrase-based anti-refusal rate, both in %.
Considering both the safety and the phrase metric, our ActGCG performs stronger than the original GCG for Gemma-2-2B, but worse for Qwen3-1.7B. Overall, our activation target matches the fixed string target of GCG, but does not show a major improvement.
We compare our method to SSR [26] again, using the same last token and filler token variants as before. Apart from the change in loss, we use the same per-prompt setup as before.
| Method | HarmBench | Phrase |
| SSR cosine, faithful | 4.5 | 100.0 |
| SSR cosine, filler | 22.7 | 100.0 |
| ActGCG | 27.3 | 90.9 |
Table 4.8. Per-prompt anti-refusal results on Gemma-2-2B, breadth split with 22 prompts. Comparing both SSR variants against ActGCG. HarmBench jailbreak rate and the phrase-based anti-refusal rate, both in %.
We observe that ActGCG performs better than both SSR variants on the safety metric. At the same time, for the phrase based metric, ActGCG nevertheless performs slightly worse.
Universal suffixes on refusal
We now optimise a single universal suffix that steers many prompts at once, rather than having one suffix per prompt. We evaluate in two ways. First, we do a “breadth split”, where we train on 22 prompts each from a different category, and then test on 22 new prompts from all different categories again. Then, we do a “depth split” (five prompts from each of four training categories, then five prompts from the same categories for test, and finally five prompts from four new categories). We also transfer to a similarly sized (or smaller) and a slightly larger model of the same family, evaluated on training prompts for both split variants.
| Source model | Evaluation | HarmBench | Phrase |
| Gemma-2-2B | training | 45.5 | 95.5 |
| Gemma-2-2B | test | 31.8 | 63.6 |
| Gemma-2-2B | Gemma-1.1-2B | 9.1 | 22.7 |
| Gemma-2-2B | Gemma-2-9B | 0.0 | 4.5 |
| Qwen3-1.7B | training | 45.5 | 100 |
| Qwen3-1.7B | test | 59.1 | 100 |
| Qwen3-1.7B | Qwen2.5-1.5B | 31.8 | 50.0 |
| Qwen3-1.7B | Qwen3-4B | 9.1 | 22.7 |
| Llama-3.2-3B | training | 36.4 | 86.4 |
| Llama-3.2-3B | test | 36.4 | 81.8 |
| Llama-3.2-3B | Llama-3.2-1B | 4.5 | 18.2 |
Table 4.9. Universal suffix on refusal, breadth split (22 prompts, one prompt per category). HarmBench jailbreak rate and phrase-based anti-refusal rate, both in %.
| Source model | Evaluation | HarmBench | Phrase |
| Gemma-2-2B | training | 45.0 | 90.0 |
| Gemma-2-2B | test | 30.0 | 85.0 |
| Gemma-2-2B | new categories | 30.0 | 60.0 |
| Gemma-2-2B | Gemma-1.1-2B | 5.0 | 10.0 |
| Gemma-2-2B | Gemma-2-9B | 10.0 | 10.0 |
| Qwen3-1.7B | training | 35.0 | 100 |
| Qwen3-1.7B | test | 45.0 | 95.0 |
| Qwen3-1.7B | new categories | 35.0 | 100 |
| Qwen3-1.7B | Qwen2.5-1.5B | 40.0 | 100 |
| Qwen3-1.7B | Qwen3-4B | 25.0 | 50.0 |
| Llama-3.2-3B | training | 65.0 | 85.0 |
| Llama-3.2-3B | test | 60.0 | 90.0 |
| Llama-3.2-3B | new categories | 50.0 | 80.0 |
| Llama-3.2-3B | Llama-3.2-1B | 10.0 | 10.0 |
Table 4.10. Universal suffix on refusal, depth split. HarmBench jailbreak rate and phrase-based anti-refusal rate, both in %.
We notice that the phrase based metric scores significantly higher than the safety metric. The training prompts generally score higher than the test prompts (except for slight increase in Qwen3-1.7B), and transfers to smaller or similarly sized models perform better than larger models, similar to the German transfers (§ 5.6). We also observe that the suffix does generalise to unseen categories.
Model translation for universal suffixes on refusal
We now evaluate a variant of the universal suffix again, where we insert a model-translation in the middle of the steps. We namely call a proposer model that rewrites the suffix into coherent text as in § 5.7. As proposers, we utilise GPT-5.5 and Llama-3.1-8B again. We compare for both the depth and breadth split variants. The transfer models are evaluated on the test prompts (same as in § 5.7).
We find that in the case of refusal, a model translation in the middle of the steps generally performs worse than the universal ActGCG with no translation. This is in contrast to our results for German, and shows that the German language may be easier to translate to a coherent meaning than the more complex refusal concept.
| Source model | Proposer | Evaluation | HarmBench | Phrase |
| Gemma-2-2B | GPT-5.5 | training | 45.5 | 86.4 |
| Gemma-2-2B | GPT-5.5 | test | 31.8 | 68.2 |
| Gemma-2-2B | GPT-5.5 | Gemma-1.1-2B | 4.5 | 4.5 |
| Gemma-2-2B | GPT-5.5 | Gemma-2-9B | 4.5 | 13.6 |
| Gemma-2-2B | Llama-3.1-8B | training | 40.9 | 90.9 |
| Gemma-2-2B | Llama-3.1-8B | test | 22.7 | 72.7 |
| Gemma-2-2B | Llama-3.1-8B | Gemma-1.1-2B | 0.0 | 9.1 |
| Gemma-2-2B | Llama-3.1-8B | Gemma-2-9B | 0.0 | 0.0 |
| Llama-3.2-3B | GPT-5.5 | training | 36.4 | 81.8 |
| Llama-3.2-3B | GPT-5.5 | test | 31.8 | 72.7 |
| Llama-3.2-3B | GPT-5.5 | Llama-3.2-1B | 4.5 | 4.5 |
Table 4.11. Breadth split (, one prompt per category). Model-translation universal suffix on refusal with HarmBench jailbreak rate and phrase-based anti-refusal rate, both in %.
| Source model | Proposer | Evaluation | HarmBench | Phrase |
| Gemma-2-2B | GPT-5.5 | training | 10.0 | 85.0 |
| Gemma-2-2B | GPT-5.5 | test | 15.0 | 90.0 |
| Gemma-2-2B | GPT-5.5 | new categories | 5.0 | 60.0 |
| Gemma-2-2B | GPT-5.5 | Gemma-1.1-2B | 10.0 | 15.0 |
| Gemma-2-2B | GPT-5.5 | Gemma-2-9B | 5.0 | 5.0 |
| Gemma-2-2B | Llama-3.1-8B | training | 55.0 | 95.0 |
| Gemma-2-2B | Llama-3.1-8B | test | 20.0 | 85.0 |
| Gemma-2-2B | Llama-3.1-8B | new categories | 15.0 | 85.0 |
| Gemma-2-2B | Llama-3.1-8B | Gemma-1.1-2B | 10.0 | 10.0 |
| Gemma-2-2B | Llama-3.1-8B | Gemma-2-9B | 10.0 | 35.0 |
| Llama-3.2-3B | GPT-5.5 | training | 45.0 | 80.0 |
| Llama-3.2-3B | GPT-5.5 | test | 45.0 | 90.0 |
| Llama-3.2-3B | GPT-5.5 | new categories | 40.0 | 80.0 |
| Llama-3.2-3B | GPT-5.5 | Llama-3.2-1B | 0.0 | 0.0 |
Table 4.12. Depth split (20 prompts each for training, test, new categories), model-translation universal suffix on refusal. HarmBench jailbreak rate and phrase-based anti-refusal rate, both in %.
Tracking the change of loss for refusal
We repeat the loss-tracking analysis of § 5.8 for refusal, on the breadth split.
Universal suffix. We first check the change of loss in our universal experiments every 25 steps. Figure 4.11 sums the distance and averages the cosine over the layers, Figure 4.12 shows the per-layer contribution to the loss (per-layer cosine in Appendix § 8.1), and Figure 4.13 tracks the change in refusal rate for the phrase metric. The per-prompt runs show the same descent (Appendix § 8.2).
Figure 4.11. distance to the target activations (normalised to its step-0 value) and cosine similarity between the filler-token activations and the target, over optimisation steps and averaged across the four target layers, for the universal refusal run (, ).
Figure 4.12. Stacked per-layer distance to the target for the universal anti-refusal suffix ().
Figure 4.13. Fraction of train prompts whose output is not a phrase-based refusal every 25 steps, for the universal anti-refusal suffix.
Model translation. We then track the change of loss for our model-translation experiments. Figure 4.14 sums the distance and averages the cosine over the layers, Figure 4.15 breaks the down per layer, and Figure 4.16 tracks the change in refusal rate. The change of suffix in the middle of the steps by the proposer sharply increases the loss. Despite the fact that the safety metric performs significantly worse than its German counterpart, we do not notice a strong difference between the losses for German and for refusal. This may be an indication that the steering vector for refusal is more related to avoiding the phrase substrings than pure jailbreaking.
Figure 4.14. Model-translation refusal run. (normalised to step 0) and cosine to the target every 25 steps, averaged across layers. The dashed line marks the mid-run proposer rewrite.
Figure 4.15. Stacked per-layer distance to the target for the model-translation anti-refusal run.
Figure 4.16. Fraction of train prompts whose output is not a phrase-based refusal every 25 steps for the model-translation run.
Soft suffixes
To check how removing the discrete token constraint performs, we replace the 20 discrete suffix tokens with 20 soft tokens, and optimise them directly with Adam (with learning rate 0.05 and 150 steps) against the same activation target as ActGCG (Equation (5)). We see this as an indication on how well the objective can be met, since the optimiser explores the full continuous embedding space. We run this on Gemma-2-2B and Qwen3-1.7B, and the same 30 German prompts, at both and the .
| Model | (%) | (%) |
| Gemma-2-2B | 96.67 | 93.33 |
| Qwen3-1.7B | 93.33 | 70.00 |
Table 4.13. Soft suffix German success rate at two steering strengths (30 prompts).
At , the soft suffix reaches near perfect German rates compared to its discrete version with ActGCG. This shows that the discrete search over tokens is a limitation. For , we note a peculiarity where the success rate of Qwen3-1.7B decreases strongly due to empty outputs, likely due to the vector being too strong, driving the activations to extreme values which the model cannot decode.
Decoding soft tokens
We tried to convert the optimised soft embeddings back into discrete tokens by projecting each soft token to its nearest vocabulary embedding (using cosine similarity) and evaluating the resulting discrete string as a suffix. This attempt failed completely, as the suffix produced zero German responses. Hence, a soft suffix cannot be easily converted into a discrete one, which justifies why a discrete search is useful to obtain a hard suffix.
Different dataset extraction for German
We change how the German steering vector is extracted. The default construction (the German only column) contrasts a German prompt with a German answer against an English prompt with an English answer. We change the way we extract the dataset such that we hold the prompt fixed in English and contrast only the answer language (the English + German column), with the positive example being an English prompt followed by its German answer, and the negative example being the same English prompt followed by its original English answer (en_prompt + de_answer against en_prompt + en_answer). The vector is built from the same Dolly dataset and at the same four layers as before, and we then run per-prompt ActGCG on Gemma-2-2B and Qwen3-1.7B for 30 prompts. We use rather than the German default of 2.0.
| Model | German only (%) | English + German (%) |
| Qwen3-1.7B | 80.00 | 83.33 |
| Gemma-2-2B | 53.33 | 53.33 |
Table 4.14. Per-prompt German success rate with and 30 prompts, comparing two different Dolly dataset extraction methods.
The new dataset variant shows no consistent or clear improvement. Since we started with the German only method, we simply stay with the original way we extract the Dolly dataset.
More steps
We test whether the default 250 optimisation steps are enough for refusal, or whether the discrete search continues to improve, by re-running the per-prompt anti-refusal experiment at 500 steps with , and breadth split using 22 prompts.
| Model | 250 steps | 500 steps |
| Gemma-2-2B | 27.3 | 31.8 |
| Qwen3-1.7B | 40.9 | 54.5 |
Table 4.15. Per-prompt anti-refusal, breadth split. HarmBench jailbreak rate (%) at 250 against 500 optimisation steps.
More steps do increase the success rate. Nevertheless, computation time increases significantly as well. Hence, we continue to use 250 steps throughout our experiments.
Weight precision
We test whether the model’s weight precision, the number of bits per weight, affects our method’s search. We run the per-prompt German experiment on Gemma-2-2B in full fp32 precision instead of the default bf16, keeping everything else fixed while using 30 prompts. Note that this run uses rather than the of the main German results, but we mainly consider the difference between the two precisions at the same .
| Precision | Success (%) |
| bf16 (default) | 53.33 |
| fp32 | 70.00 |
Table 4.16. Effect of weight precision at (Gemma-2-2B).
We notice that fp32 precision does perform better. However, since it takes a significantly longer amount of computation time, and the gap is within the seed-to-seed variance seen earlier, we continue to use bf16 throughout our experiments.
Suffix to steering vectors
To complete the equivalence relationship in the reverse direction, we ask whether a suffix’s effect can itself be distilled back into a steering vector. For each of the 30 Gemma-2-2B German prompts, we take its optimised per-prompt ActGCG suffix and measure the activation difference it induces at the filler positions of the four target layers, where we take the activations with the suffix appended minus those without. We obtain a recovered steering vector, and add it at the filler positions during generation exactly as we would a normal steering vector, and measure the success rate. Here, we compare two variants: a per-prompt recovered vector (each prompt steered by the vector recovered from its own suffix) and the mean recovered vector (a single vector averaged over all 30 prompts, applied to every prompt), each for and .
| Mean SV (%) | Per-prompt SV (%) | |
| 1.0 | 66.67 | 50.00 |
| 2.0 | 80.00 | 63.33 |
Table 4.17. Success rate of steering vectors recovered from a suffix (Gemma-2-2B, German, 30 prompts).
A steering vector recovered from the suffix does reproduce the behaviour, and the mean vector recovers it better than the per-prompt vector. Although we do observe a slight decrease in effectiveness, this supports the equivalence between the suffix and the steering vector.
Prefix instead of suffix
We check whether optimising a prefix placed before a prompt performs better than a suffix placed after the prompt. Unlike a suffix, a prefix can, in principle, match an activation target that also covers the prompt positions. Therefore, we compare two targets for the prefix: the steering vector added only at the filler positions, and the steering vector added at both the prompt and filler positions. We evaluate on German with Gemma-2-2B for 30 prompts with .
| Target | (%) | (%) | (%) |
| Filler only | 16.67 | 60.00 | 66.67 |
| Prompt and filler | 23.33 | 33.33 | 30.00 |
Table 4.18. Prefix optimisation on German with Gemma-2-2B for a filler only target against a prompt and filler target.
Even though a prefix can reach the prompt positions, matching the prompt and filler target is generally worse than matching the filler positions alone. Moreover, the best prefix result is still behind its suffix counterpart. Therefore, the suffix remains the stronger choice.
Starting from a readable German instruction
Until now, we have initialised the optimised suffix from the string x x ...x. We check whether starting from a readable instruction that already asks for the target behaviour gives ActGCG an easier starting point and a higher final success rate. We run the per-prompt German experiment for the four main models with only the initial suffix changed, keeping everything else fixed (30 Dolly prompts, , seed 0). The initial suffix is “Please answer in German”.
| Model | x x ...x | Please answer in German |
| Gemma-2-2B | 86.67 | 86.67 |
| Llama-3.2-3B | 100.00 | 100.00 |
| Qwen3-1.7B | 96.67 | 90.00 |
| Qwen3-4B | 96.67 | 93.33 |
Table 4.19. Per-prompt German success rate (%) when the suffix is initialised from x x ...x against a readable instruction.
A target-oriented initialisation makes essentially no difference. This shows that GCG, as well as our ActGCG method, does not preserve coherence between suffix tokens, as it only optimises them independently.
Discussion
Limitations
Our results come with several limitations. The most fundamental is that the success rate depends heavily on the concept being steered, and we test only two of them here: the German language and refusal. These two already behave very differently: German is reproduced almost perfectly by a suffix and transfers well across prompts and models, whereas refusal is weaker. From here, it is not possible to infer how other concepts would perform.
The evaluations are also run at a relatively small scale. The per-prompt and universal experiments were tested on 30 prompts for German, and between 22 and 60 for refusal. Most runs use a single fixed seed. As a result, our scores may vary with different prompts or seeds.
By construction, ActGCG searches over discrete tokens, which limits how closely it can match the target activations. Our soft suffix experiments, which optimise continuous embeddings directly, reach lower losses than any discrete suffix and act as an upper bound on what the objective can achieve. The gap between them reflects the reduction in success rate due to the discrete token space.
The method is also bounded by the steering vector it imitates. The suffix can only reproduce a behaviour that the steering vector itself induces. If the vector is weak, the suffix inherits that ceiling as well. Moreover, reaching the activation target is not sufficient to guarantee the behaviour. For instance, a suffix can match the target activations closely and still fail to jailbreak. Matching the activations is thus a useful proxy for the behaviour, but not a guarantee of it. Furthermore, the best way to construct a steering vector in the first place is still an open question. A more optimal steering vector could further improve our method.
Finally, judging the responses is not perfect. Especially for refusal, the measured safety rate depends heavily on the judge. We standardise on a single classifier throughout our experiments to keep the runs comparable. The phrase based metric is not a good proxy for safety either, as a response can often avoid refusal phrases without delivering harmful content. For German, the langdetect judge occasionally mislabels short responses that are German overall but contain English titles or names. In addition, the definition of the concept itself matters, and results can vary drastically across the literature due to the difficulty of rigorously defining behaviours.
Ethical Considerations
Our work considers the anti-refusal direction, which can sometimes result in bypassing safety guardrails. We conduct all of our jailbreak experiments on small, open-weight models that are already publicly available, and we do not present any harmful model outputs in this thesis. The practical offensive value of the resulting suffixes is limited. They jailbreak only at modest rates and, crucially, transfer poorly across models. Therefore, a suffix optimised on one model is of little use against another.
Our aim in understanding an equivalence relationship between suffixes and steering vectors is defensive. Showing that a suffix can imitate a steering vector is a step towards building measures against inputs that reproduce vectors of malicious concepts.
Future Work
Our findings suggest a number of directions worth pursuing. The most direct option would be to test more concepts, and to check how well our approach generalises. German reaches a high success rate while refusal shows highly mixed results; hence, it would be informative to see how other concepts fall on this scale.
The matching objective could also be further improved. We use a squared- loss, but there may be other losses that can approximate the target activation more closely. Since squaring amplifies large deviations, a more robust objective could reduce their influence.
Furthermore, the steering vector itself could be strengthened by searching for better construction methods than the difference-of-means vectors. More broadly, steering vectors themselves may not be the best proxy for a given behaviour. Our method draws its target activation from a steering vector, but for instance, we could also choose activations from another model that already exhibits the desired behaviour. Investigating such alternative targets with input suffixes would test whether there are equivalence relationships for behaviours that a steering vector does not fully capture.
Conclusion
In this thesis, we investigate whether there is an equivalence between input suffixes and steering vectors. We introduce ActGCG, a modification of GCG that, rather than optimising a suffix towards a fixed output string, optimises it such that its activations match the target obtained by adding a steering vector to the residual stream. We test this hypothesis on two concepts, namely the German language and refusal. For German, we largely retain the behaviour of the steering vector with a discrete suffix using our method. For refusal, the equivalence is weaker. Our method largely suppresses the model’s refusal, such that it succeeds under a substring-based metric that only checks whether the response avoids a set of refusal phrases. However, when measured with the stricter HarmBench classifier, the suffix that reproduces the steering vector does not directly produce a jailbreak. The substring metric roughly doubles the true HarmBench rate.
We find the same asymmetry between the two concepts throughout the rest of our results. We extend the method to universal suffixes that steer many prompts at once, and to a single suffix shared across models, and find that our behaviour is still retained. The German suffix transfers well both to test prompts and to other models, whereas the anti-refusal suffix generalises across prompts but transfers poorly across models. The same pattern appears when we ask a larger model to rewrite the unreadable suffix into coherent text midway through optimisation. The model translation improves success for German, most significantly for transferring to different models, but gives only mixed results for refusal. All in all, our results show that a steering vector’s behaviour is reachable from the input, with no access to the model’s internals at generation time, but with a certain loss depending on the concept. Establishing how far this equivalence extends to further concepts beyond the two studied here, to better steering vector constructions, as well as to alternative losses, remains a promising direction for the future.
Additional Figures
Per-layer cosine trajectories
Figure A.1. Per-layer cosine to the target activations over optimisation steps for the universal German run.
Figure A.2. Per-layer cosine to the target activations over optimisation steps for the universal anti-refusal run.
Per-prompt loss trajectories
Figure A.3. Per-prompt multilayer loss for German runs (Gemma-2-2B, , 30 prompts), with normalised and cosine to the target over optimisation steps. Panel titles are green where the answer is German, red otherwise.
Figure A.4. Per-prompt multilayer loss for the anti-refusal runs (Gemma-2-2B, , 22 prompts), with normalised and cosine to the target over optimisation steps. Panel titles are green where the prompt is a jailbreak judged by HarmBench, red otherwise.
References
- [1]Andy Arditi, Oscar Obeso, Aa quib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction. In Advances in Neural Information Processing Systems 37 (NeurIPS), 2024.arxiv.org/abs/2406.11717
- [2]Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in Neural Information Processing Systems 29 (NeurIPS), pages 4349–4357. Curran Associates, Inc., 2016.arxiv.org/abs/1607.06520
- [3]Michal Danilak. langdetect: Port of google’s language-detection library to Python. https://github.com/Mimino666/langdetect, 2014. Accessed: 2026-08-25.
- [4]Databricks. Free dolly: Introducing the world’s first truly open instruction-tuned llm. https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm, 2023.
- [5]Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova Das-Sarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. A mathematical framework for transformer circuits. Transformer Circuits Thread, 2021.
- [6]Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, et al. Toy models of superposition. Transformer Circuits Thread, 2022.arxiv.org/abs/2209.10652
- [7]Jaden Fiotto-Kaufman, Alexander R. Loftus, Eric Todd, Jannik Brinkmann, Koyena Pal, Dmitrii Troitskii, Michael Ripa, et al. Nnsight and ndif: Democratizing access to open-weight foundation model internals. arXiv preprint arXiv:2407.14561, 2024.
- [8]Gemma Team. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118, 2024.
- [9]Aaron Graffafiori et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024.
- [10]Gray Swan AI. nanoGCG: A fast and lightweight implementation of the GCG algorithm in PyTorch. https://github.com/GraySwanAI/nanoGCG, 2024. Accessed: 2026-08-25.
- [11]Shawn Im and Sharon Li. A unified understanding and evaluation of steering methods. arXiv preprint arXiv:2502.02716, 2025.arxiv.org/abs/2502.02716
- [12]Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), 2015.arxiv.org/abs/1412.6980
- [13]Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from a language model. In Advances in Neural Information Processing Systems 36 (NeurIPS), 2023.arxiv.org/abs/2306.03341
- [14]Qizhang Li, Xiaochen Yang, Wangmeng Zuo, and Yiwen Guo. Deciphering the chaos: Enhancing jailbreak attacks via adversarial prompt translation. arXiv preprint arXiv:2410.11317, 2024.arxiv.org/abs/2410.11317
- [15]Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. Proceedings of the 41st International Conference on Machine Learning (ICML), 2024.arxiv.org/abs/2402.04249
- [16]Tomás Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeffrey Dean. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems 26 (NeurIPS), pages 3111–3119. Curran Associates, Inc., 2013.arxiv.org/abs/1310.4546
- [17]Neel Nanda, Andrew Lee, and Martin Wattenberg. Emergent linear representations in world models of self-supervised sequence models. In Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 16–30, Singapore, 2023. Association for Computational Linguistics.
- [18]Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models. In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024.arxiv.org/abs/2311.03658
- [19]Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar, 2014. Association for Computational Linguistics.DOI
- [20]Itamar Pres, Laura Ruis, Ekdeep Singh Lubana, and David Krueger. Towards reliable evaluation of behavior steering interventions in llms. arXiv preprint arXiv:2410.17245, 2024.arxiv.org/abs/2410.17245
- [21]Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering llama 2 via contrastive activation addition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024.arxiv.org/abs/2312.06681
- [22]Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023.arxiv.org/abs/2307.09288
- [23]Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering language models with activation engineering. arXiv preprint arXiv:2308.10248, 2023.arxiv.org/abs/2308.10248
- [24]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems 30 (NeurIPS), pages 5998–6008. Curran Associates, Inc., 2017.arxiv.org/abs/1706.03762
- [25]Yuxin Wen, Neel Jain, John Kirchenbauer, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery. In Advances in Neural Information Processing Systems 36 (NeurIPS), 2023.arxiv.org/abs/2302.03668
- [26]Thomas Winninger, Boussad Addad, and Katarzyna Kapusta. Using mechanistic interpretability to craft adversarial attacks against large language models. arXiv preprint arXiv:2503.06269, 2025.
- [27]An Yang et al. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115, 2024.arxiv.org/abs/2412.15115
- [28]An Yang et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.arxiv.org/abs/2505.09388
- [29]Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, et al. Representation engineering: A top-down approach to ai transparency. arXiv preprint arXiv:2310.01405, 2023.arxiv.org/abs/2310.01405
- [30]Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043, 2023.arxiv.org/abs/2307.15043



















