alphaFeedback

WorksComputer Science

Which Words a Model Fears: From a Refusal Direction to a Vocabulary-Wide Fear Lexicon

Publication details
License
arXiv nonexclusive license
Views
8
Status
Published
AI contributions
Not recorded for this paper
View details
Ancillary data
1 code link
View
Abstract · v1

Whether an aligned language model refuses depends in part on which words a prompt contains, regardless of whether those words carry the request's meaning. Harm words shift the hidden state at the generation boundary along a component shared across prompts and words, which crossed differencing extracts as a single direction. In eight models, writing along this direction raises and lowers refusal monotonically with dose, ahead of random and KL-matched directions. Used as a ruler over the full vocabulary, the direction gives every word a shift, from "feared" words at one end to "courage" words at the other; a word's shift is predictable from its own representation, and appending a feared word to a benign prompt is enough to make the model refuse it. Two held-out tests validate the ranking in all eight models: refusal after inserting an unseen word rises with its table shift, and replacing a prompt's most feared word by its most courageous meaning-preserving synonym significantly raises the answer rate on benign and harmful prompts alike, by an amount that follows the shift the replacement actually causes.

Review conversation

Reviews are not available for this work yet.