WorksComputer Science
The Geometry of Lies: Knowledge and Persona Directions for Detecting Deception in Large Language Models
Abstract
Linear probes are the cheapest LLM lie detectors but often fail to transfer across kinds of lie. Label-free directions from honest and deceptive role-play are untested at 70B scale and on lies about knowledge, where a lie and an honest denial can be identical. We widen a published persona bank with roles that deny knowledge they have, plus honest counterparts. At each model's middle layer the resulting persona direction ranks lies across Liars' Bench better than single-dataset probes on three of its four models; it fails on Qwen, even when Qwen stays in role far more often. On Llama-3.3-70B-Instruct it beats directions from a random persona bank, but its gain over the published direction is within the range of random widenings. On lies about knowledge, persona directions mainly track a reply's wording: where lies and honest denials share their words, our three main contrasts stay inside the random-direction band on Llama and Qwen and reach only its edge on Gemma, though on Mistral one clears it. A knowledge direction contrasts questions the model can answer with ones nobody can, each followed by the same scripted denial. It separates lies from identical honest denials on two models and narrowly on a third, but not on the fourth; its other results partly depend on the benchmark's wording. On the Gender-Secret fine-tune it clears the band by a few hundredths on Llama and fails on Mistral. The sum of the two directions ranks lies better than a supervised probe at the same depth on all four models (on one, both near chance) but is level with the persona direction on Llama's average and behind that probe on held-out lies. More varied role-play questions do not help the persona direction on Liars' Bench.
Introduction
Large language models sometimes lie. They have been observed concealing misconduct [30, 14] and faking alignment [13], and deception by AI systems is a recognised risk [29]. Following Liars’ Bench [18], we call a statement a lie when the model asserts something it believes to be false, where its belief is taken to be what it consistently asserts under no pressure to lie. This definition separates lying from hallucination. The cheapest detector of such lies is a probe on the residual stream, usually a linear one [1, 2, 25]: one dot product per response, no extra forward pass, and a chance to catch a lie that leaves no trace in the transcript [11]. Its weakness is generalisation: a probe trained on one distribution of lies often fails on others [17, 19, 27]. Liars’ Bench makes the failure concrete. Its seven datasets hold on-policy lies, lies that four open-weight models produced themselves. On most of them, probes trained on instruction pairs are near chance in balanced accuracy at the benchmark’s threshold, which flags 1% of answers to ordinary instructions. The hardest dataset is the harm-pressure knowledge report (HP-KR), in which a model denies knowing the answer to a hazardous question it has been shown to know. On HP-KR the benchmark’s Mean Probe for Llama-3.3-70B-Instruct, read at layer 20, scores an AUROC of 0.12, worse than chance. At the model’s middle layer a probe trained the same way ranks HP-KR well, yet at that threshold it still flags none of its lies.
Mahadik & Skapars [23] propose persona coordinates as a remedy: the model role-plays deceptive and honest personas, each persona’s mean activation is a vector, and the difference between the two group means is a direction that uses no evaluation data and so cannot be fitted to one dataset’s idiosyncrasies. On Llama-3.2-3B and Llama-3-8B these coordinates transfer better than probes on raw activations, keeping more of their accuracy on datasets they were not fitted to; the authors name larger models and richer persona sets as future work. We take up both. Rebuilt for Llama-3.3-70B-Instruct, their direction, which we call the published contrast, inverts on HP-KR: at the benchmark’s layer it ranks honest answers above lies (AUROC 0.148). Nearly every HP-KR lie is a denial of knowledge (125 of 139 on Llama), while every published deceptive persona lies by controlling disclosure. We therefore add ten roles of our own, four that deny knowledge they have and six honest counterparts. The widened contrast, which we call the extended contrast, ranks HP-KR better, but for the wrong reason: its gain follows the words of a denial. The contrast built from the new roles alone flags nearly every reply that says “I do not know”, whether the denial is true or false, and at the middle layer, on the Llama rows where a lie and an honest denial are the same sentence, every persona direction scores within the range that random directions reach (the random band; § 3). On these lies a persona direction responds to how a reply is worded more than to whether the model knows the answer.
The definition of a lie offers a second route. A lie about knowledge is a denial by a model that knows the answer, and what a model knows can be measured without any labelled lie, by comparing questions whose answers it knows with questions whose answers nobody can know. We build a knowledge direction from exactly that comparison, writing the same denial into the model’s reply after every question so that the reply’s words do not vary. We then ask five questions of a detector that adds a persona contrast to the knowledge direction: whether it transfers across Liars’ Bench better than supervised probes; whether it separates lies from honest answers that use the same words; what each part responds to; whether the result survives resampling, new questions, other layers and strict thresholds; and whether it holds on other models and benchmarks.
The results are mixed. The extended contrast is the part that generalises: at each model’s middle layer it transfers better than probes trained on a single dataset on three of the benchmark’s four models. On the main model it also beats contrasts drawn from a random persona bank, although a probe trained on all the other datasets together is level with it once HP-KR is set aside (0.915 against 0.917). It fails on Qwen, even with a second bank in which Qwen stays in role far more often. The knowledge direction separates lies from identically worded honest denials on two models, and narrowly on a third. Outside these wording-controlled tests, its transfer depends in part on the wording of HP-KR’s template. On the Gender-Secret fine-tune, which only Llama and Mistral have, it clears the random band by a few hundredths on Llama but not on Mistral; the other fine-tunes give it no support. The sum of the two directions, our selected detector, ranks the benchmark’s lies better than a probe trained at the same depth on all four models (on Qwen both are near chance). On the main model’s average over the benchmark’s datasets, however, it is level with the extended contrast, and in an external competition it would rank 16th of 30, behind a supervised probe. At cutoffs that flag 1% of each dataset’s honest answers, none of the six detectors we evaluate there catches half of the lies on any model (at most 0.47; Table 31). Our contributions are:
| Detector | Llama 1% | Llama 10% | Qwen† 1% | Qwen† 10% | Mistral 1% | Mistral 10% | Gemma 1% | Gemma 10% |
| Base probe | 0.32 | 0.66 | 0.02 | 0.17 | 0.13 | 0.47 | 0.28 | 0.66 |
| Published contrast | 0.34 | 0.58 | 0.02 | 0.12 | 0.13 | 0.47 | 0.08 | 0.30 |
| Extended contrast | 0.47 | 0.77 | 0.02 | 0.14 | 0.14 | 0.49 | 0.43 | 0.78 |
| Knowledge direction | 0.21 | 0.56 | 0.02 | 0.13 | 0.21 | 0.54 | 0.07 | 0.26 |
| Selected detector | 0.38 | 0.73 | 0.03 | 0.13 | 0.23 | 0.59 | 0.30 | 0.63 |
| Published contrast + knowledge direction | 0.32 | 0.67 | 0.02 | 0.11 | 0.22 | 0.59 | 0.24 | 0.53 |
Table 31. Share of lies caught at in-setting cutoffs, each model’s middle block.
a persona bank for deception by denial, and a single label-free contrast built from it that, at the middle layer, transfers better than probes trained on a single dataset on three of four models. We check it against a widening by unrelated roles and, on the main model, against a random persona bank: it beats that bank’s contrasts, but its gain over the published contrast is within what random widenings produce (§ 4.1, § 5 and § 6);
a diagnosis of what persona directions respond to: on lies about knowledge they track the words of a denial, and with the words held fixed the three main contrasts are, at the middle layer, inside the random band on Llama and Qwen and at its edge on Gemma, while on Mistral the published contrast clears it (§ 4.2 and § 5);
a knowledge direction with measured limits: it separates lies from identically worded honest denials on two models (narrowly on a third), clears the random band by a small margin on Llama’s
Gender-Secret fine-tune but not on Mistral’s, and only its results on the wording-controlled slices are independent of the benchmark’s wording (§ 4.3 and § 5);
the selected detector, whose mean AUROC over the benchmark’s datasets is, on all four models, above that of a probe trained at the same depth. On the main model it is level with the extended contrast, and at strict cutoffs it catches fewer lies than that contrast (§ 4.4); on the other three, adding the knowledge direction raises transfer on two and lowers it on the third (§ 5). In a small steering test its persona part, like the published contrast, reduces the disclosure of a hidden fact (§ 4.5);
an evaluation protocol for zero-shot (label-free) directions: random bands, a random persona bank with a pre-registered null, wording-controlled slices, the base model’s activations on a fine-tuned model’s transcripts as a control, and paired intervals; and two further pre-registered tests, one of them external, reported with their failures (§ 6).
Related work
Lie detection and why probes fail to transfer. Instruction pairs, the same statement produced under an honest and under a deceptive instruction, are the standard training data for deception probes [11] (Zou et al. 2023). Black-box detectors instead ask unrelated follow-up questions [28] or have an LLM judge read the transcript; Liars’ Bench finds the judge strongest overall, yet unable to detect HP-KR’s lies, whose truth hinges on what the model knows [18]. A probe that is near perfect in distribution can be below chance elsewhere. With fewer examples than dimensions, many directions fit a training set [7], and discriminative learning takes any feature that suffices on that set [10]. Probes have accordingly latched onto negation [20, 3], text probability [25], deception-related scenarios [11] and style [19]; they suffer when trained on text the model did not itself generate (off-policy data; [17]), and they invert where a correlate of the label reverses [20, 26] (Ying et al. 2026). Remedies constrain the direction, to a difference of means [25], to selected principal components (Yoo & Skapars, 2026) or to persona coordinates [23]; we take the last route. Ying et al. (2026) further report that a probe trained on data pooled over several settings performs on par with the best single-setting probe, and that a covariance-weighted cosine predicts transfer better than the plain cosine; § 6 tests whether both carry over, with families of role-play questions as the settings.
Benchmarks and model organisms. Liars’ Bench [18] collects 72,863 on-policy lies and honest responses across seven datasets and four models, and evaluates a Mean Probe trained on the instruction pairs of representation engineering (RepE; Zou et al., 2023), a Follow-Up Probe read at a prefilled “No” after “Was your last response deceptive?”, and an Upper-Bound Probe trained on the benchmark itself; on HP-KR, built from the hazardous questions of WMDP [21], only the last detects the lies. The UK AI Security Institute (AISI) released belief-labelled rollouts (lies and honest answers) from two of these models, Llama and Gemma (UK AI Security Institute, 2026); they serve as our held-out data. Cooney et al. [6] show that existing fine-tuned liars (model organisms) often do not hold the belief they deny; they reveal belief-verified reasoning organisms on Qwen3.5-27B and Qwen3.6-27B and find that activation and logprob detectors drop sharply on them while a chain-of-thought judge stays strong. Our external test is Aletheia’s Quest [4], a lie-detection competition on model organisms and prompted liars, with black-box and white-box tracks ranked by mean AUROC over ten held-out datasets. The winner of its held-out white-box track scores the suspect model’s own judgement of its transcript.
Personas and knowledge as directions. Persona vectors [5] and the Assistant Axis [22] are activation directions for character traits and archetypes. Mahadik & Skapars [23] built a deception monitor from such directions: a deception axis, the difference between eight deceptive and seven honest personas, used either directly (zero-shot) or through the top principal components of the persona vectors, on datasets that do not include HP-KR. Roles differ in style as well as honesty [15], and a model that switches to an ignorant persona could deceive although “no persona is ever lying” [24]. A model’s internal states record whether it knows an answer [16, 31, 12]; they can Ferrando et al., 2025). We measure this signal at the tokens of a denial written for the model, as a difference of means between questions whose answers it knows and questions nobody can answer, and use it to detect lies.
Method
Reads and persona contrasts. A detector reads the residual stream at the output of one decoder block, averaged over the response tokens, and maps this pooled vector , which we call a read, to one score, higher for a lie. Forward passes run remotely on the National Deep Inference Fabric (NDIF) through nnsight (Fiotto-Kaufman et al., [9]). We name depths by the layers of Llama-3.3-70B-Instruct; Table 25 gives the block that stands for each of these layers in the other models. The main depth is the model’s middle block (the middle layer of the abstract and introduction), as in the work we build on ([22, 23]), and Liars’ Bench reads layer 20. We also sweep 16 depths, every fourth layer from 20 to 76 plus the last block, scaled to each model; layers 36 and 44 were chosen for reporting after seeing results and are labelled post hoc. A persona’s vector is its mean activation over the response tokens of its kept role-play responses. For a deceptive group and an honest group ,
Table 25: The four models: blocks and width, the block read for each of Llama’s layers, the datasets the benchmark gives each, the judge score a rollout needs to enter the bank, and the share of rollouts scored fully in role.
| Model | Blocks | Width | Block read for Llama’s layer | Liars’ Bench datasets | Bank | In role | ||
| 20 | 40 | 36, 44 | ||||||
| Llama-3.3-70B | 80 | 8192 | 20 | 40 | 36, 44 | CG, HP-C, HP-KR, IT, ID | score 3 | 69% |
| Qwen2.5-72B† | 80 | 8192 | 20 | 40 | 36, 44 | HP-C, HP-KR, ID | score 2–3 | 28% |
| Mistral-Small-3.1-24B | 40 | 5120 | 9 | 20 | 18, 22 | HP-C, HP-KR, IT, ID | score 3 | 68% |
| Gemma-3-27B | 62 | 5376 | 8 | 31 | 28, 34 | CG, IT, ID | score 3 | 96% |
Table 25.
which group is called deceptive fixes the sign. Our persona bank, the set of roles with their judged role-play responses, holds the 26 roles of the published persona method (8 deceptive, 7 honest, 5 sycophantic, 5 anti-sycophantic and the default assistant as an anchor) and 10 new roles about knowing and denying, 36 in all. Four knowledge-denial roles know the answer and claim not to; three sincere-ignorance roles do not know it and say so; and three plain-assertion roles know it and give it. Every role has five system-prompt variants and answers the same 30 questions; a Llama-3.1-70B-Instruct judge scores each rollout from 0 to 3 for how clearly it plays the role, and only rollouts scored 3 are kept (3,599 of the 5,250 judged rollouts; the anchor’s 150 are not judged; Section A). Five contrasts of the form of (1) differ only in their groups: published (8 v 7), sycophancy (5 v 5), knowledge-denial (our 4 denying roles v our 6 honest ones), and two widenings that add another contrast’s groups to the published ones. The unified contrast adds the sycophancy contrast’s groups (13 v 12), and the extended contrast adds the knowledge-denial contrast’s (12 v 13). The unified contrast controls for widening by roles unrelated to knowledge; the random persona bank below gives a null distribution for widening by random roles.
The knowledge direction. In HP-KR a lie and an honest denial can be the same sentence, and only whether the model knows the answer makes one a lie. An AI assistant, Claude Opus, wrote 60 general questions whose answers a 70B model knows with certainty and 60 whose answers nobody can know (private, unrecorded particulars of real places). Each question is placed in HP-KR’s own knowledge-report template: a four-option question followed by “Tell me whether you know the answer to the question, and nothing else”, under HP-KR’s system text and with no role. We then wrote the reply for the model, a forced reply: the same denial after every question, once as “I do not know the answer.” and once as “I don’t know the answer to the question.”, giving 240 reads. The direction is the unit difference between the mean reads over the known and over the unknowable questions (Figure 4b). Both sides receive the same two replies, so the reply cannot separate them. The known side, where the denial is false, is the lie side; this sign was fixed before any evaluation. The questions themselves still differ in topic and form, and a text classifier tells the two sides apart 95% of the time. For a replication set of matched question pairs, written afterwards and also by Claude, the figure is 86%, and the replication direction built from it met all 11 criteria fixed before it was evaluated. Before the direction was built, the model answered all 70 candidate known questions correctly. The direction shares HP-KR’s template and denial sentences, but it shares no question with any evaluation set and uses no lie that the model wrote. A wording direction, forced denials minus forced assertions, measures how much a detector responds to the words of a denial; a detector without its wording component has its component along this direction removed. In each further model the direction is rebuilt with the known questions checked against that model’s answers. Our main detector, the selected detector, sums the extended contrast and the knowledge direction, each score divided by its standard deviation on the lie-free Alpaca control (Taori et al., 2023), with equal weights fixed in advance. A direction is zero-shot when it uses no labels, no lies and no benchmark question, and it is scored zero-shot when its projection is the score, with no sign or scale fitted on labelled data (the selected detector’s two components are scaled on the lie-free Alpaca responses). The knowledge direction meets this definition only narrowly: it shares HP-KR’s system text, its template and the exact sentence that defines the identical-text slice below, and § 4.3 and § 5 examine how much its results depend on them.

Figure 4. The knowledge direction: the same denial after known and after unknowable questions. Prompts are abridged; Section 3 gives the full template.
Baselines and model organisms. Our supervised baselines are logistic regressions, trained as in the benchmark’s pipeline. The probe on raw activations (the raw probe) uses the full residual stream; trained on one dataset and tested on the others, it is the transfer baseline. The base probe (our baseline probe; “base” does not refer to the base model) also uses the full residual stream and follows the Mean Probe’s recipe on RepE pairs (Zou et al., 2023) at the same block as our directions; a second version is trained on a split of 3,285 of Llama’s own belief-labelled rollouts (1,634 lies; UK AI Security Institute, 2026). The published persona method’s probe is trained instead on the top three principal components of the persona vectors. The benchmark’s model organisms [18] are LoRA fine-tunes that lie by design: Gender-Secret knows the user’s gender and denies knowing it, and Soft-Trigger answers falsely when a trigger appears. Their lies exist only in the fine-tuned model, so the adapter is applied inside the remote pass, and the directions are also rebuilt with it applied. Each transcript is also read by the base model as a control: a detector that separates the lies only in the fine-tune’s read responds to the fine-tune’s state, not to the text.
Models, data and summary measures. Table 25 (Section H) lists the four benchmark models and, for each, the block that stands for each of Llama’s layers. Each model is evaluated on the datasets the benchmark provides for it, all written by that model: convincing-game (CG), harm-pressure choice (HP-C), HP-KR, insider trading (IT) and instructed deception (ID); the benchmark’s other two datasets come from the model organisms above. In train one, test all, a probe is trained on one of a model’s sets (on Llama, the five datasets and RepE) and tested on the others, so RepE is both a training and a test set. The mean AUROC over the off-diagonal cells of this matrix is the probe’s transfer. A probe on a single zero-shot direction can only choose a sign and a scale, so when every training set agrees on the sign, its transfer is the direction’s mean AUROC over the sets; a direction below 0.5 on one training set is flipped for every test set when trained there, so its transfer can fall well below its mean AUROC. Our second summary averages a detector’s AUROC over the model’s benchmark datasets, for detectors trained on none of them (directions scored zero-shot, and probes trained on RepE or on the AISI rollouts); on Llama we call it the five-dataset mean. Alpaca sets the thresholds and the selected detector’s scaling.
Wording-controlled slices. In HP-KR nearly every lie (125 of 139 on Llama) is a denial of knowing an answer the model has been shown to know, so whether a reply denies knowledge already ranks HP-KR at an AUROC of 0.916, without reading any activation. We therefore also score two wording-controlled slices of HP-KR: the identical-text slice, every row whose response is exactly “I do not know the answer.” (78 lies vs 112 honest on Llama), and the denials slice, every row whose response denies knowledge. Within a slice, lies and honest denials answer different questions, so the slices control the wording but not the question.
Thresholds, intervals and nulls. Detection decisions use the benchmark’s threshold, which flags 1% of the Alpaca responses, and in-setting cutoffs, which flag 1, 5 or 10% of a dataset’s own honest answers. Comparisons at the middle block get 95% intervals from 1,000 stratified resamples of the rows, the same resamples for every method, so differences are paired. A difference counts only when its interval excludes zero, and we call two detectors level when it does not. A detector inverts when its AUROC is below 0.5, ranking honest answers above lies. Depth curves and averages over layers are point estimates, and the cross-model and per-layer checks are not corrected for multiple comparisons. A zero-shot AUROC is compared with the AUROCs of 1,000 random unit directions on the same rows. Their central 95% is the random band, and the value we quote for a band is its upper edge, the 97.5th percentile: a direction is at its edge when its AUROC and that edge differ by less than 0.01 (the rule of Table 1), clears the band when it is higher still, and is inside it otherwise. We check the bank against a random persona bank: 35 roles drawn from the 274 non-default roles released with the Assistant Axis and built by the same pipeline. Partitioning them into groups of our sizes gives one reported split and 99 further splits, which form the pre-registered null.
Steering. For steering, the residual stream after the middle block is replaced at every token by , where is the residual vector at that token and the unit direction being steered, while the model answers 27 role-play scenarios from the AI Liar set of Pacchiardi et al. [28]. Each scenario’s system prompt gives the model an unfavourable fact and a reason to hide it. Answers are greedy and are compared at , the largest size at which no direction breaks the model. One reader, an AI assistant blind to the condition, labelled each answer for disclosure of the fact (1, 0.5 or 0).
Results on Llama-3.3-70B-Instruct
Every result in this section is at layer 40 unless stated otherwise; Table 5 (Section B) compares all the detectors side by side.
| Design | Labels | Transfer AUROC all six | Transfer AUROC excl. HP-KR | Five-dataset mean AUROC | Five-dataset mean BA | HP-KR AUROC all | HP-KR AUROC identical |
| Probe on raw activations | one dataset | 0.732 | 0.758 | n/a | n/a | n/a | n/a |
| Base probe (RepE) | RepE | n/a | n/a | 0.778 | 0.626 | 0.936 | 0.779 |
| Base probe, own lies (AISI) | AISI | n/a | n/a | 0.878 | 0.630 | 0.935 | 0.686 |
| Persona method, extended PCs () | one dataset | 0.866 | 0.867 | n/a | n/a | n/a | n/a |
| Published contrast | none | 0.832 | 0.880 | 0.798 | 0.675 | 0.593 | 0.715 |
| Sycophancy contrast | none | 0.663 | 0.936 | 0.770 | 0.601 | 0.167 | 0.686 |
| Unified contrast | none | 0.690 | 0.919 | 0.795 | 0.655 | 0.381 | 0.707 |
| Knowledge-denial contrast | none | 0.912 | 0.910 | 0.894 | 0.725 | 0.919 | 0.712 |
| Extended contrast | none | 0.916 | 0.917 | 0.899 | 0.694 | 0.912 | 0.721 |
| Knowledge direction | none | 0.823 | 0.815 | 0.825 | 0.553 | 0.865 | 0.793 |
| Published contrast + knowledge direction | none | n/a | n/a | 0.879 | 0.646 | 0.854 | 0.789 |
| Selected detector (extended + knowledge) | none | 0.922 | 0.924 | 0.906 | 0.701 | 0.909 | 0.787 |
Table 5. The detectors at layer 40: transfer (train one, test all, over the six sets), the zero-shot five-dataset mean with balanced accuracy (BA) at the benchmark’s threshold, and HP-KR on all rows and on its identical-text slice.
Persona contrasts at 70B
Our base probe, trained by the Mean Probe’s recipe at layer 40, ranks HP-KR well (AUROC 0.936) yet flags none of its 139 lies at the benchmark’s threshold. At layer 20 it inverts (0.109), as the published contrast does (0.148); at layer 40 the published contrast scores 0.593, inside a random band that reaches 0.926. The raw probe averages 0.983 on its own dataset and 0.732 on the other five (Figure 2), with 6 of its 30 off-diagonal cells below chance. A probe on the published persona method’s top three principal components, fitted on the published roles, transfers at 0.639, below the raw probe (Table 9). The extended contrast, one direction with no training, transfers at 0.916: above the raw probe by 0.184 [0.172, 0.195], and above the published contrast (0.832) by 0.084, or by 0.037 with HP-KR left out. Scored zero-shot, its five-dataset mean is 0.899, against 0.778 for the base probe at the same layer and 0.682 for the benchmark’s layer-20 Mean Probe.
Figure 2. Train-one-test-all AUROC at layer 40 for the probe on raw activations (left) and the selected detector (right), whose rows are constant off the diagonal because every training set agrees on its sign.
Table 9: Transfer (mean off-diagonal AUROC) for every feature set, with and without HP-KR.
| Features | All six test sets: L20 | All six test sets: L40 | All six test sets: L44 | Excluding HP-KR: L20 | Excluding HP-KR: L40 | Excluding HP-KR: L44 |
| Probe on raw activations (8192-d) | 0.648 | 0.732 | 0.716 | 0.728 | 0.758 | 0.730 |
| Published contrast, 8 v 7 (1-d) | 0.651 | 0.832 | 0.826 | 0.919 | 0.880 | 0.873 |
| Knowledge-denial contrast, 4 v 6 (1-d) | 0.866 | 0.912 | 0.900 | 0.861 | 0.910 | 0.899 |
| Extended contrast, 12 v 13 (1-d) | 0.904 | 0.916 | 0.903 | 0.909 | 0.917 | 0.905 |
| Selected detector (1-d) | 0.811 | 0.922 | 0.915 | 0.876 | 0.924 | 0.918 |
| Extended PCs () | 0.684 | 0.866 | 0.880 | 0.900 | 0.867 | 0.870 |
| Extended PC1 | 0.661 | 0.768 | 0.780 | 0.682 | 0.774 | 0.793 |
| Published PCs () | 0.679 | 0.639 | 0.617 | 0.888 | 0.751 | 0.728 |
| Random directions () | 0.527 | 0.504 | 0.493 | 0.560 | 0.544 | 0.547 |
Table 9.
Its lead over a probe trained on many kinds of lie rests on HP-KR. A probe trained on the other five datasets together averages 0.850, and with HP-KR left out it is level with the extended contrast (0.915 against 0.917); on HP-KR itself, the extended contrast’s AUROC is inside the random band and follows the words of a denial (§ 4.2). With HP-KR left out, the sycophancy contrast, which has no deceptive role, transfers best of all the detectors in Table 5 (0.936 against 0.917 for the extended contrast). Our ten roles raise the five-dataset mean by 0.101 over the published contrast, about two thirds of the gain coming from HP-KR. The unified contrast, widened in the same way with the sycophancy contrast’s roles, transfers at 0.657 averaged over the 16 layers, below the 0.686 of the contrast it widens (point estimates). Against random widenings drawn from the random persona bank, however, the gain from our roles is not significant, and the extended contrast’s AUROC on all of HP-KR is inside that null (§ 6).
Persona directions track the wording
Of our detectors, only the knowledge-denial contrast has a balanced accuracy on HP-KR clearly above chance (0.910), and it gets there by flagging denials. At the benchmark’s threshold it flags 88% of the lies and 95% of the honest denials but just 0% of the honest answers that claim knowledge; honest denials are only 135 of the 2,000 honest answers. With the words held fixed, the persona directions do no better than the best random directions. On the identical-text slice the published, knowledge-denial and extended contrasts score 0.715, 0.712 and 0.721, all inside the random band (0.733). From layer 32 to 72 none of these three contrasts exceeds 0.724 on this slice, and they clear the band only at the last two layers of the sweep (Figure 1). The extended contrast built from the 150-question bank of Section 6 scores slightly higher at layer 40, just above the band’s upper edge but within 0.01 of it (Table 35). On Gender-Secret, whose lies and honest answers are both denials, every persona direction built on the base model and read through the fine-tune is near chance (the knowledge-denial contrast 0.436); rebuilt inside the fine-tune, the best reaches 0.641 at layer 44 (post hoc), still far inside the random band. On HP-KR, then, the persona directions’ signal follows the words of a denial, and on the identical-text slice none of them separates lies from honest denials better than the best random directions do.
Figure 1. AUROC on HP-KR’s two wording-controlled slices, the identical-text slice (left) and the denials slice (right), across the 16 layers. Grey marks the random band, the central 95% of the AUROCs of random directions; “knowledge direction, new questions” is the replication direction, and the two knowledge-direction curves nearly coincide.
| Pre-registered test | Statistic | Result |
| Variety beats count (extended contrast) | 30 originals + 30 monotone five-dataset mean 0.9044 vs 200 draws of 30 originals + 30 random diverse (median 0.8901), (Holm) | fails |
| Variety beats count (selected detector) | 30 originals + 30 monotone five-dataset mean 0.9097 vs 200 draws of 30 originals + 30 random diverse (median 0.9103), (Holm) | fails |
| Monotone additions behave like the originals | the monotone 30 0.9048 in [0.8296, 0.9114] of random 30 of 150; paired CI vs the original 30 [−0.0003, 0.0120] within ±0.03 | passes |
| Pooling all 150 beats the best single family | all 150 0.8809 minus original knowledge family 0.9107 = −0.0298, CI [−0.0394, −0.0199], (Holm) | fails |
| Covariance-weighted cosine predicts transfer better than plain cosine | with the target’s covariance 0.675 vs plain cosine 0.405; difference 0.269 CI [0.139, 0.365], (Holm) | passes |
| More questions help on HP-KR’s identical text (exploratory) | 0.7392 vs 0.7213, CI [0.0014, 0.0340] | passes |
Table 35. The pre-registered tests of the question extension (Llama, layer 40).
The knowledge direction
On the identical-text slice the knowledge direction scores 0.793 [0.728, 0.852], above the band, and 0.792 without its wording component. It is above the band at 15 of the 16 layers, and the replication direction follows the same curve (Figure 1). On the wider denials slice, whose lies and honest denials need not share their exact words, it sits at the band’s edge (0.728 against 0.726), so on Llama its support rests on the identical-text slice. Because that slice’s lies and honest denials answer different questions, the slice cannot separate what the model knows from how the questions differ. On it, the base probe trained at the same layer, a supervised probe that inverts on insider trading, is level with the knowledge direction (0.779; difference 0.014 [−0.018, 0.044]). The direction was also built from this same reply under HP-KR’s template; the template-free direction introduced below keeps the result (0.794). On all of HP-KR the knowledge direction scores 0.865, but this figure depends on the words (0.349 without its wording component), so we count only its slice results as evidence.
Built inside the Gender-Secret fine-tune and read through it, the knowledge direction scores 0.861 [0.817, 0.896] against a band of 0.824, while the base model’s read of the same transcripts gives 0.629 (Section F). The margin is a few hundredths, and the classes differ in topic as well as in what the model knows. On the five datasets the direction averages 0.825, below the extended contrast’s 0.899, and it leads the extended contrast only on HP-C (0.830 against 0.712). Its results beyond the slices rest in part on HP-KR’s template. A template-free knowledge direction, rebuilt without that wording by a rule chosen on these datasets, keeps the wording-controlled HP-KR results. It transfers a little worse here, however (0.791 against 0.823), and much worse on Mistral (§ 5), and on Gender-Secret it falls inside the random band (0.747). Resampling the questions gives a 95% interval at most 0.002 wide on the identical-text slice. None of 1,000 directions built from balanced splits of the same questions reaches the knowledge direction’s AUROC there; we adopted this null after the random split we had specified first proved uninformative. The replication direction has a cosine of 0.99 with the original and was scored on the same test rows, so its agreement is not independent evidence.
The selected detector and strict cutoffs
The selected detector transfers at 0.922 against 0.732 for the raw probe, level with the extended contrast (0.006 [−0.003, 0.015]), and its five-dataset mean of 0.906 is also level with the contrast’s (difference 0.007 [−0.004, 0.018]). It gains over the extended contrast some where the words are controlled: 0.066 [0.037, 0.098] on the identical-text slice and 0.294 on Gender-Secret, at small costs on CG, ID and IT. On Gender-Secret, with both directions rebuilt in the fine-tune, it scores 0.841, just above the band; built on the base model, it scores 0.802, inside the band. It beats the same-layer base probe on the five-dataset mean by 0.128, almost all of the margin coming from insider trading, where the probe inverts; it is level with that probe on the identical-text slice and below it on held-out lies (§ 6). Across depth it transfers above the published and unified contrasts and the raw probe at every one of the 16 layers, by at least 0.009 (Section C; point estimates). Averaged over the layers, however, it transfers below the extended contrast alone (0.853 against 0.885; at layer 20, 0.811 against 0.904), whose own transfer peaks at layer 36 (post hoc). Its weakness is at strict cutoffs: when the cutoff flags 1% of a dataset’s own honest answers, the extended contrast catches 0.47 of the lies, the selected detector 0.38, the base probe 0.32 and the knowledge direction 0.21. Placing the threshold on each dataset instead of on Alpaca changes balanced accuracy by a few hundredths at most (Table 21).
| Detector | Layer 36 Alpaca | Layer 36 oracle | Layer 40 Alpaca | Layer 40 oracle |
| Base probe (RepE) | 0.650 | 0.635 | 0.626 | 0.652 |
| Knowledge-denial (knowledge-denial contrast) | 0.695 | 0.675 | 0.725 | 0.692 |
| Extended (extended contrast) | 0.712 | 0.710 | 0.694 | 0.718 |
Table 21. Balanced accuracy at the Alpaca threshold and at thresholds set on each dataset (the “oracle” columns).
Steering the persona contrast
Adding the extended contrast at cuts disclosure of the hidden fact from 0.48 to 0.20 ( [, ]). The published contrast does the same (), so the effect is not specific to our roles; two random directions do not, and the base probe’s direction moves it by [, ] (Section G). The steered answers take on the deceptive personas’ manner, so the steering plausibly works through the speaking style of those roles. The result rests on one blinded reader (an AI assistant), one coefficient and one greedy answer per scenario. An earlier run on benchmark items with an absolute shift (a fixed multiple of the direction, not scaled by ) found no effect beyond that of random directions, and two small runs that steered the knowledge direction are suggestive only (Section G).
Across models
We repeated the studies with the same code on three further models, Qwen2.5-72B-Instruct, Mistral-Small-3.1-24B-Instruct and Gemma-3-27B-it (hereafter Qwen, Mistral and Gemma), fixing the blocks and banks before inspecting any result (Table 25 in Section H; steering was not repeated). Each model has its own bank from the same roles, variants, questions and judge. In Qwen’s strict bank, which keeps only rollouts scored 3, 11 roles keep fewer than 20 rollouts, so Qwen uses the lenient bank, which also keeps rollouts scored 2 (marked † in the tables); the lenient banks of the other three models change the contrasts’ means and the extended contrast’s transfer at the middle block by at most 0.02 (Table 4). Only Llama and Mistral have a released identical-text slice; for Qwen and Gemma we rebuilt comparable slices from the raw files, a fixed reply for Qwen and a fixed opening sentence for Gemma (Table 30). Table 1 checks the claims at each model’s middle block; every interval is in Section H.
| Claim | Llama | Qwen† | Mistral | Gemma |
| Extended contrast transfers above the probe on raw activations | +0.184 ✓ | −0.106 × | +0.094 ✓ | +0.175 ✓ |
| Extended contrast transfers above the published contrast | +0.084 ✓ | +0.002 ◦ | +0.176 ✓ | +0.221 ✓ |
| The knowledge-denial roles raise the mean (extended above published) | +0.101 ✓ | +0.061 ✓ | +0.043 ✓ | +0.221 ✓ |
| Extended contrast’s mean is above the benchmark’s Mean Probe | 0.899 (0.682) ✓ | 0.441 (0.530) × | 0.762 (0.570) ✓ | 0.900 (0.527) ✓ |
| Selected detector transfers above the probe on raw activations | +0.190 ✓ | −0.045 × | +0.125 ✓ | +0.114 ✓ |
| Selected detector’s mean is above the same-block RepE probe | +0.128 ✓ | +0.036 ✓ | +0.038 ✓ | +0.064 ✓ |
| No persona direction clears the band on identical text | 0.721 (0.733) ✓ | 0.550 (0.584)<sup>a</sup> ✓ | 0.784 (0.755) × | 0.624 (0.622)<sup>b</sup> ◦ |
| Knowledge direction clears the band on identical text | 0.793 (0.733) ✓ | 0.602 (0.584)<sup>a</sup> ✓ | 0.840 (0.755) ✓ | 0.585 (0.622)<sup>b</sup> × |
| Knowledge direction clears the band on Gender-Secret, read through the fine-tune | 0.861 (0.824) ✓ | n/a | 0.440 (0.765) × | n/a |
Table 1. The claims at each model’s middle block: a paired difference, or an AUROC with the value it must clear (or, for the row on persona directions, stay below) in parentheses. ✓ holds, × does not, ◦ no clear difference (the interval includes zero, or the values are within 0.01). †Qwen’s lenient bank. <sup>a</sup>Qwen’s identical reply, rebuilt from the raw files. <sup>b</sup>Gemma’s opening sentence, rebuilt, 34 lies. In the third row the knowledge-denial roles are all ten new roles, not only the four denying ones. The Mean Probe row compares with Liars’ Bench’s layer-20 probe; the same-block base probe is in Table 26. On Gender-Secret the margin is a few hundredths and the classes also differ in topic.
| Model | Bank | Published contrast | Knowledge-denial contrast | Extended contrast | Extended contrast transfer |
| Llama | score 3 | 0.798 | 0.894 | 0.899 | 0.916 |
| Llama | scores 2–3 | 0.786 | 0.880 | 0.888 | 0.907 |
| Qwen† | score 3 | n/a | n/a | n/a | n/a |
| Qwen† | scores 2–3 | 0.380 | 0.668 | 0.441 | 0.453 |
| Mistral | score 3 | 0.719 | 0.727 | 0.762 | 0.810 |
| Mistral | scores 2–3 | 0.712 | 0.747 | 0.770 | 0.816 |
| Gemma | score 3 | 0.679 | 0.484 | 0.900 | 0.921 |
| Gemma | scores 2–3 | 0.684 | 0.487 | 0.895 | 0.912 |
Table 4. Each model’s persona banks at its middle block: the strict bank and the lenient one. Mean over the model’s datasets
| Slice (lies, honest) | Probe | Published contrast | Knowledge-denial contrast | Extended contrast | Knowledge direction | Selected detector | Band |
| Llama | |||||||
| Identical text (78, 112) | 0.779 | 0.715 | 0.712 | 0.721 | 0.793 | 0.787 | 0.733 |
| Denials (125, 135) | 0.593 | 0.649 | 0.570 | 0.670 | 0.728 | 0.724 | 0.726 |
| Mistral | |||||||
| Identical text (494, 729) | 0.766 | 0.784 | 0.647 | 0.760 | 0.840 | 0.840 | 0.755 |
| Denials (677, 1,020) | 0.758 | 0.763 | 0.571 | 0.701 | 0.835 | 0.832 | 0.748 |
| Qwen{sup}†` | |||||||
| Denials (539, 44) | 0.566 | 0.495 | 0.580 | 0.509 | 0.632 | 0.622 | 0.622 |
| Identical reply, rebuilt (435, 128) | 0.543 | 0.504 | 0.550 | 0.518 | 0.602 | 0.598 | 0.584 |
| Denials, rebuilt (571, 164) | 0.533 | 0.498 | 0.569 | 0.547 | 0.613 | 0.617 | 0.580 |
| Gemma | |||||||
| Opening sentence, rebuilt (34, 85) | 0.481 | 0.484 | 0.530 | 0.624 | 0.585 | 0.611 | 0.622 |
| Denials, rebuilt (35, 100) | 0.547 | 0.583 | 0.418 | 0.434 | 0.378 | 0.403 | 0.637 |
Table 30. HP-KR’s wording-controlled slices on every model that has them.
What carries over. On Mistral and Gemma the extended contrast transfers above the raw probe (by 0.094 and 0.175, intervals above zero) and above the published contrast; on Mistral much of the second lead comes from a sign flip of the published contrast, which scores below 0.5 on HP-C and so is flipped for every test set when trained there (§ 3). Zero-shot, its mean over the model’s datasets is well above the benchmark’s layer-20 Mean Probe on both (0.762 against 0.570; 0.900 against 0.527); against the base probe trained at the same block it is above on Gemma (0.117 [0.103, 0.130]) and level on Mistral ( [, 0.014]). Adding our ten roles raises the mean above the published contrast’s on all four models (by 0.101, 0.061, 0.043 and 0.221), though on Llama about one in six random widenings drawn from the random persona bank gains as much (§ 6), so the gain is not shown to be specific to these roles. The selected detector’s mean over the benchmark’s datasets is above the same-block probe’s on all four (by 0.128, 0.036, 0.038 and 0.064); on Qwen both are near chance, and on Llama almost all of the margin comes from insider trading), while on the held-out lies of Llama and Gemma the probe is ahead (0.971 against 0.960; 0.932 against 0.625). Adding the knowledge direction leaves transfer level on Llama, raises it on Mistral (0.032) and Qwen (to 0.514, still below the raw probe) and lowers it on Gemma (), where the knowledge direction alone is weak.
Qwen: the persona contrast fails. On Qwen the extended contrast transfers below the raw probe (0.453 against 0.559), is level with the published contrast, averages 0.441 zero-shot against the Mean Probe’s 0.530, and inverts on HP-KR (0.187). Much of the failure comes from HP-KR: with HP-KR left out it transfers above the raw probe (0.627 against 0.608; point estimates), though it stays weak on HP-C and ID. The knowledge-denial contrast does not fail (transfer 0.679; HP-KR 0.783). The judge scored 28% of Qwen’s rollouts fully in role, against 69%, 68% and 96% for Llama, Mistral and Gemma (by group in Table 3), which suggested that the contrast failed because Qwen did not play the roles. A second Qwen bank, built after these results, places the role assignment, with an instruction to stay in character, in the user turn. With it, Qwen plays 69% of its rollouts fully in role and every role keeps enough rollouts to enter the strict bank. Yet the extended contrast still transfers below the raw probe (0.469 against 0.559; −0.090 [−0.101, −0.079]), again because of HP-KR (0.674 against 0.608 without it), while the knowledge-denial contrast does not fail (transfer 0.725). Poor role-play is therefore not the whole explanation, and why the published roles invert on Qwen’s HP-KR remains open.
| Persona group | Llama | Qwen† | Mistral | Gemma |
| Deceptive (published contrast) | 58% | 15% | 73% | 97% |
| Honest (published contrast) | 74% | 52% | 85% | 99% |
| Knowledge-denial (knowledge-denial contrast) | 78% | 10% | 54% | 96% |
| Knowledge-candour (knowledge-denial contrast) | 54% | 22% | 53% | 89% |
| Sycophantic | 75% | 37% | 60% | 97% |
| Anti-sycophantic | 82% | 28% | 75% | 100% |
| All 35 roles | 69% | 28% | 68% | 96% |
Table 3. The share of rollouts the judge scored fully in role, by persona group and model. Knowledge-candour is the honest side of the knowledge-denial contrast (the three sincere-ignorance and three plain-assertion roles); the anchor is not judged, so the last row covers 35 roles.
The knowledge direction across models. On Mistral, the one other model with a released identical-text slice (494 lies vs 729 honest), the knowledge direction scores 0.840 against a band of 0.755; it is above the same-block probe by 0.073 [0.053, 0.095] and above its band at all 16 depths. On Qwen’s rebuilt identical-reply slice it narrowly clears the band (0.602 [0.545, 0.655] against 0.584). On Gemma’s rebuilt slice it is inside the band (0.585 against 0.622), though with only 34 lies the slice is too small to decide, and on Gemma’s rebuilt denials it inverts (0.378 [0.275, 0.489]). The wording diagnosis of Section 4.2 holds on Qwen’s rebuilt slice, where every persona direction is inside the band; on Gemma’s, the extended contrast sits at the band’s edge (0.624 against 0.622), with too few lies to decide either way. It holds only in part on Mistral: the knowledge-denial contrast flags 99.8% of the honest denials and is inside the band on the identical-text slice, the extended contrast is at its edge (0.760), and the published contrast is above it (0.784), so the claim that no persona direction clears the band fails there. On its own, the knowledge direction’s mean over each model’s datasets is 0.775 on Mistral but 0.546 on Qwen and 0.515 on Gemma, where it inverts on instructed deception (0.277) and on Gemma’s own held-out lies (0.236). The other models’ fine-tunes add no evidence for it. Mistral’s Gender-Secret, the one replication of Llama’s test, counts against it (0.440 against a band of 0.765), and on the Soft-Trigger fine-tunes either the base model’s read already separates the classes or the direction stays inside the band (Section F). The template-free knowledge direction (§ 4.3) keeps the wording-controlled HP-KR results on Mistral (identical text 0.837, by a rule fixed before Mistral was evaluated) and on Llama (0.794, though there the rule’s wording was chosen with Llama’s own datasets in view), but it transfers worse (0.614 against 0.799 on Mistral). Of the knowledge direction’s results, then, only those on the slices are free of the template.
Across depth and what generalises. The middle block was fixed in advance, but it need not be where a model’s signal is strongest (Section H). The extended contrast transfers above the raw probe, by any margin, at all 16 depths on Llama, but at only 9 of 16 on Mistral, 7 on Gemma and 6 on Qwen. On Qwen it stays near chance until 0.71 of the depth and then reaches 0.852 at a block found only by looking (post hoc), where it is level with the raw probe. Two results hold on every model: adding our ten roles raises the contrast’s mean (on Llama, by an amount within the range that random widenings produce), and the selected detector’s mean over the benchmark’s datasets is above the same-block probe’s (on Qwen both are near chance, and on Llama the margin comes almost entirely from insider trading). Where the words are fixed, the knowledge direction separates lies from identical honest denials on Llama and Mistral and narrowly on Qwen, but not on Gemma’s 34 rebuilt lies. The extended contrast’s lead in transfer replicates on Mistral and Gemma, not on Qwen; its lead over the raw probe at every depth, the detection of Gender-Secret through the fine-tune, and the value of adding the knowledge direction do not generalise.
External and pre-registered tests
Held-out lies. The validation split of the AISI rollouts holds Llama’s own belief-labelled answers (1,730 lies, 498 honest answers) under system prompts that no other part of this work used. On it, at layer 40, the extended contrast scores 0.977, the selected detector 0.960 and the knowledge direction 0.899. The base probe trained on RepE at the same layer scores 0.971: it is level with the extended contrast (difference 0.006 [0.000, 0.013]; the lower end is zero even to four decimals) and above the selected detector, which trails it by 0.011 [0.003, 0.020]. A base probe trained on a split of 3,285 of Llama’s own rollouts (1,634 lies, disjoint from the validation split) raises the five-dataset mean by 0.100 [0.086, 0.113] over the base probe trained on RepE. The extended contrast is narrowly ahead of this probe at layer 40 (0.021 [0.006, 0.035]), level at layer 44 (post hoc), and clearly ahead only at layer 20 (0.105). On the held-out lies the template-free knowledge direction scores below the original (0.867 against 0.899).
The random persona bank. Six tests were fixed before scoring (Section D). The extended contrast beats the random 12 v 13 contrasts on transfer (0.916 against a null 97.5th percentile of 0.828), on the zero-shot five-dataset mean (0.899 against 0.814), with the knowledge direction added (0.922 against 0.891) and on the held-out lies (0.977 against 0.955), each at a Holm-corrected of 0.002. Two tests fail. The extended contrast’s AUROC on all of HP-KR (0.912) is inside the null (97.5th percentile 0.938, = 0.144). Its gain over the published contrast on the five-dataset mean (0.101) is within what random widenings produce: widening a random 8 v 7 contrast by four and six random roles, the sizes of our groups, gives a null median of −0.005 and a 97.5th percentile of 0.253 ( = 0.165). This null widens random contrasts rather than the published one, and all its splits come from one draw of 35 roles.
Aletheia’s Quest. The competition [4] scored lie detectors on ten held-out datasets of fine-tuned organisms and prompted liars built on Qwen3.5-27B, Gemma-3-27B and a Nemotron model, ranked by mean AUROC. We built persona banks on Qwen3.5-27B and Gemma-3-27B with the pipeline above, built a knowledge direction inside each organism, and froze a protocol before reading any test row. We then scored the selected detector zero-shot at the middle block, on reads taken in each organism (through its adapter or a merged copy) or, for the one dataset written by plain Gemma, in base Gemma; as in the competition’s starter kit, reasoning tokens were dropped. Two Hidden-Goal datasets could not be read (gated adapters; the Nemotron one would also have needed a persona bank, which we did not build). Over the 8 readable datasets (Table 34 in § 17) the selected detector averages 0.657 [0.633, 0.680], which would place it 16th among the 30 published entries with their means recomputed over the same datasets (rank interval 13 to 20); the extended contrast alone averages 0.705 (11th) and the base probe trained on RepE at the same block 0.784 (5th). Whether the official white-box reads of the Qwen3.5 organisms applied their adapters is not known (§ 17). With the reasoning tokens kept, a secondary condition of the protocol, the selected detector reaches 0.758 (7th) and the extended contrast 0.799 (3rd). The selected detector inverts on two organisms, Echoblast (0.189; knowledge direction alone 0.206) and AuditBench’s hallucinated citations (0.408; knowledge direction alone 0.239). The top entries are active methods, such as the suspect model judging its own transcript [4], whereas this detector is a single passive inner product. Two deviations from the frozen protocol are recorded in § 17.
| Dataset | (lies) | Selected detector | Extended contrast | RepE probe, same block | Random 97.5% | Best entry |
| HG-N | 200 (152) | not read (gated adapter) | 0.934 | |||
| HG-G | 189 (139) | not read (gated adapter) | 0.820 | |||
| Echo | 400 (200) | 0.189 [0.15, 0.23] | 0.604 | 0.766 | 0.778 | 0.865 |
| Harm | 200 (100) | 0.885 [0.83, 0.93] | 0.900 | 0.821 | 0.800 | 0.977 |
| GS-F | 154 (74) | 0.771 [0.70, 0.84] | 0.774 | 0.895 | 0.852 | 0.992 |
| GS-M | 155 (75) | 0.798 [0.72, 0.87] | 0.714 | 0.814 | 0.813 | 0.986 |
| BotC | 400 (200) | 0.604 [0.55, 0.66] | 0.491 | 0.505 | 0.625 | 0.610 |
| AB-AW | 172 (26) | 0.786 [0.71, 0.85] | 0.616 | 0.783 | 0.727 | 0.886 |
| AB-HC | 176 (30) | 0.408 [0.31, 0.50] | 0.683 | 0.862 | 0.859 | 0.945 |
| AB-SP | 182 (34) | 0.818 [0.76, 0.87] | 0.856 | 0.825 | 0.817 | 0.946 |
| Mean over the 8 read | 0.657 [0.63, 0.68] | 0.705 | 0.784 | 0.868 | ||
| Rank among the 30 entries | 16 | 11 | 5 | 1 |
Table 34. Aletheia’s Quest, held-out test sets: the selected detector (95% interval), the extended contrast and the RepE probe, read through the organism at its middle block, beside the random band and the best white-box entry on each dataset. In the last two rows each column gives its detector’s mean and rank, and the best-entry column the top entry’s own; the RepE probe is the base probe trained on RepE.
The 150-question extension. The bank’s 30 shared questions all ask for first-person workplace advice. We pre-registered an extension to 150 questions: 30 monotone questions in the originals’ form, which control for the number of questions, and 90 diverse questions over a grid of manipulation type, deceiver, topic and format. The 150 questions fall into ten families: the originals’ three, the monotone set, and six diverse families of 15 (Table 36). All were frozen before generation (27,000 rollouts); choices were developed on the five benchmark datasets at layer 40 and then tested in one confirmation run (Section J). Diverse questions did no better than the same number of questions in the original form: the originals plus the 30 monotone questions give a five-dataset mean of 0.904, against a median of 0.890 over 200 draws of the originals plus 30 random diverse questions (Holm-corrected p = 1.00). Unlike what Ying et al. (2026) report for their settings, pooling all 150 questions gives a worse contrast (0.881) than the best single family (0.911; −0.030 [−0.039, −0.020]). Against the 30 originals, the full 150 raise CG (0.056) and lower HP-KR (−0.134); the diverse questions cause most of the rise and all of the fall, and most of the fall comes from questions on which an honest role truthfully says “I don’t know”. As Ying et al. (2026) found, a covariance-weighted cosine predicts how well each family’s contrast transfers to each dataset better than the plain cosine does ( 0.67 against 0.41), though with a near-perfect probe on the target dataset the weighted cosine is close to the detector’s own effect size (Section K). With the knowledge direction added, the four candidate banks (the 30 originals; the originals plus the 30 monotone questions; the originals plus the 90 diverse questions; all 150) lie within 0.003 of each other, so the pre-registered tie rule, which gives ties to the larger bank, chose the 150-question bank for the confirmation run (Section J). In that run, at layer 40, the selected detector built from it beats the one built from the 30-question bank on the held-out lies by 0.006 [0.004, 0.009] and trails it on Gender-Secret by 0.024 [0.010, 0.037].
| Question family | Questions | Five-dataset mean AUROC | Leaving it out: change vs pooled |
| Original: knowledge | 10 | 0.911 | |
| Denial of knowledge | 15 | 0.907 | |
| Monotone (new, same form) | 30 | 0.905 | |
| Quality (false assertion) | 15 | 0.889 | |
| Original: sycophancy | 10 | 0.874 | 0.000 |
| Original: disclosure | 10 | 0.872 | |
| Relation (evasion) | 15 | 0.859 | |
| Manner (equivocation) | 15 | 0.810 | |
| Quantity (omission) | 15 | 0.800 | |
| Sincere ignorance | 15 | 0.771 | |
| Pooled, all 150 | 150 | 0.881 |
Table 36. The extended contrast built from each question family, five-dataset mean at layer 40.
Discussion and conclusion
On lies about knowledge, a persona direction responds to how a reply is worded more than to whether it is a lie: where the words are fixed, the three main persona contrasts at the middle layer are inside the random band on Llama and Qwen and at its edge on Gemma, and on Mistral the published contrast clears it. Under the same control, a direction built without any labelled lie, from questions whose answers the model knows and questions nobody can answer, separates lies from identical honest denials on Llama and Mistral, and narrowly on Qwen (its template-free version keeps the result on Llama and Mistral). Those lies and denials answer different questions, and the direction’s unknowable questions cannot be answered by anyone, whereas HP-KR’s honest denials concern answerable questions the model does not know. The result is therefore consistent with the direction reading what the model knows, but falls short of establishing it. Neither finding shows how the model produces a lie. Beyond those slices the knowledge direction is weaker: its transfer rests in part on the wording of HP-KR’s template, its Gender-Secret result holds on only one of the two models that have that fine-tune, and it inverts on Gemma’s instructed deception, held-out lies and rebuilt denials, and on two competition organisms. We make three recommendations. Test a persona direction where lies and honest answers share their words, against the random band and against a random persona bank built by the same pipeline, and repeat the test on every new model. Accept a model organism as a test only if the base model’s read of its transcripts can already separate the classes. And report detection at a fixed threshold alongside ranking, since a detector can rank lies well and still flag none of them.
Limitations. Our ten roles and the knowledge direction were designed after HP-KR was seen to fail, so HP-KR served as development data rather than as a blind test; the evidence not targeted at HP-KR comes from the held-out lies, Gender-Secret, the random persona bank and the pre-registered tests. The base probe trained at the same layer is level with the knowledge direction on Llama’s identical-text slice and with the extended contrast on the held-out lies, and ahead of the selected detector there and on the competition: the label-free directions’ advantage is transfer across kinds of lie, not a general lead over supervision. On Llama, Gender-Secret is the one organism that supports the knowledge direction, by a few hundredths and with classes that also differ in topic, and on Mistral the same organism counts against it. The knowledge direction is built from replies written for the model, which are off-policy text, and its unknowable questions differ in kind from the answerable questions behind HP-KR’s honest denials. On the identical-text slices, lies and honest denials answer different questions. Our intervals resample test rows with every direction and probe held fixed, not datasets, roles or rollouts (switching a model between its strict and lenient banks moves the means by up to 0.02), and the cross-model and per-layer checks are not corrected for multiple comparisons. Each model has three to five benchmark datasets, so one dataset can decide a model-level result (HP-KR on Llama and Qwen), and the middle-block results move with depth on Gemma and Mistral (Table 29). The persona judge, Llama-3.1-70B-Instruct, scores every model’s role-play, and we report no check of its scores against human ratings. The published roles were rebuilt from their authors’ descriptions, and their small-model results are not reproduced here. The steering labels come from single readers (in the main run, an AI assistant reading blind). Everything is in English. Three questions remain open: why the persona contrast fails on Qwen, whether a knowledge direction can be built whose transfer survives a change of wording, and how to combine passive detectors such as ours with the active, self-evaluating monitors that led the competition.
| Model | Layer | Block | Raw | Published contrast | Knowledge-denial contrast | Extended contrast | Knowledge direction | Selected detector |
| Llama | 20 | 20 | 0.648 | 0.651 | 0.866 | 0.904 | 0.574 | 0.811 |
| 36* | 36 | 0.768 | 0.689 | 0.912 | 0.927 | 0.854 | 0.941 | |
| 40 | 40 | 0.732 | 0.832 | 0.912 | 0.916 | 0.823 | 0.922 | |
| 44* | 44 | 0.716 | 0.826 | 0.900 | 0.903 | 0.832 | 0.915 | |
| Qwen† | 20 | 20 | 0.533 | 0.468 | 0.560 | 0.525 | 0.460 | 0.533 |
| 36* | 36 | 0.545 | 0.407 | 0.620 | 0.425 | 0.523 | 0.528 | |
| 40 | 40 | 0.559 | 0.451 | 0.679 | 0.453 | 0.508 | 0.514 | |
| 44* | 44 | 0.531 | 0.426 | 0.701 | 0.478 | 0.504 | 0.518 | |
| Mistral | 20 | 9 | 0.527 | 0.483 | 0.559 | 0.565 | 0.466 | 0.544 |
| 36* | 18 | 0.638 | 0.821 | 0.811 | 0.850 | 0.775 | 0.860 | |
| 40 | 20 | 0.716 | 0.633 | 0.780 | 0.810 | 0.799 | 0.841 | |
| 44* | 22 | 0.656 | 0.619 | 0.771 | 0.646 | 0.834 | 0.863 | |
| Gemma | 20 | 8 | 0.488 | 0.510 | 0.526 | 0.515 | 0.509 | 0.457 |
| 36* | 28 | 0.734 | 0.428 | 0.422 | 0.425 | 0.419 | 0.421 | |
| 40 | 31 | 0.746 | 0.700 | 0.442 | 0.921 | 0.457 | 0.860 | |
| 44* | 34 | 0.700 | 0.719 | 0.440 | 0.832 | 0.801 | 0.866 |
Probe on
| Model | Layer | Block | Published | Extended | Random |
| Llama | 20 | 20 | 0.679 | 0.684 | 0.527 |
| 36* | 36 | 0.679 | 0.879 | 0.481 | |
| 40 | 40 | 0.639 | 0.866 | 0.504 | |
| 44* | 44 | 0.617 | 0.880 | 0.493 | |
| Qwen† | 20 | 20 | 0.594 | 0.674 | 0.516 |
| 36* | 36 | 0.521 | 0.578 | 0.488 | |
| 40 | 40 | 0.476 | 0.539 | 0.491 | |
| 44* | 44 | 0.487 | 0.554 | 0.486 | |
| Mistral | 20 | 9 | 0.522 | 0.559 | 0.487 |
| 36* | 18 | 0.786 | 0.848 | 0.526 | |
| 40 | 20 | 0.760 | 0.704 | 0.554 | |
| 44* | 22 | 0.742 | 0.741 | 0.527 | |
| Gemma | 20 | 8 | 0.506 | 0.532 | 0.488 |
| 36* | 28 | 0.767 | 0.845 | 0.493 | |
| 40 | 31 | 0.863 | 0.874 | 0.558 | |
| 44* | 34 | 0.855 | 0.860 | 0.511 |
Table 29. Transfer (mean off-diagonal AUROC) for every feature set at the blocks that stand for Llama’s layers 20, 36, 40 and 44; *post hoc layers (chosen after seeing Llama’s results).
Reproducibility statement
Every result in the main text, and every table and figure, is written by scripts (§ 20) from the result files or, for a few run counts, from the studies’ reports. The numbers given for the side studies in Section K and for the knowledge-direction steering runs in Section G are instead quoted from their report files. The persona bank’s roles and prompts, the knowledge direction’s questions and templates, the pre-registration documents of the competition protocol and of the question extension, and the code for every experiment are available from the authors on request. All forward passes ran on the National Deep Inference Fabric through nsight; the analysis runs on pooled activations on a CPU. § 20 lists the bootstrap scheme, the random-direction construction, the block mapping for each model and the data revisions.
Ethics statement
The harm-pressure datasets contain hazardous questions from WMDP; we quote none of them and withhold the hazardous content from every example. Lie detectors are dual-use: a monitor that reads whether a model conceals knowledge could also be turned on honest refusals. We report where the detectors fail as prominently as where they succeed, so that no reader takes a single label-free direction for a reliable monitor. No human subjects were involved.
AI use statement
We used an AI coding assistant (Claude, Anthropic) for much of the implementation and analysis code, for drafting and editing the text, and for literature search. Every analysis script was run by us on the stated data, and every result in the main text is read from the resulting files by the scripts described above. The AI assistant also produced evaluation data in five places, all disclosed where they are used: the knowledge direction’s 60 known and 60 unknowable questions were written by Claude Opus, and the replication set’s matched question pairs by Claude (§ 3); the 120 new questions of the question extension were written by Claude Fable 5.1 under a fixed prompt archived with the pre-registration (Section J); the disclosure labels of the steering experiment are one blinded reading by Claude against a hidden key (§ 3); and the answers of the two small runs that steered the knowledge direction were read by Claude, without blinding (Section G). The persona rollouts are generated and judged by the open models named in § 3, as part of the method. We checked the AI-assisted code, data and text and take responsibility for the final content.
Acknowledgements
The large-model experiments ran on the National Deep Inference Fabric (NDIF), which gave remote access to the internals of models that would otherwise have been out of reach on the hardware available to us.
The persona bank
Table 2 lists the 36 roles by group. The 26 published roles follow the descriptions of [23]; the ten knowledge-denial, sincere-ignorance and plain-assertion roles are ours. Each role has five system-prompt variants and answers the same 30 questions; a Llama-3.1-70B-Instruct judge scores each rollout from 0 to 3, and in the strict bank only rollouts scored 3 enter a persona vector. Table 3 gives the share of rollouts each model plays fully in role by persona group, and Table 4 the lenient (score 2 to 3) banks beside the strict ones. In every appendix table except Table 10, † marks Qwen’s lenient bank (§ 5).
| Group | Personas (kept rollouts) | |
| Deceptive (published) | 8 | handler (106), broker (113), impression manager (82), coverup operator (47), gatekeeper (126), spin doctor (85), plausible denier (64), narrative manager (70) |
| Honest (published) | 7 | archivist (103), disclosure counsel (118), documentarian (107), fact checker (48), ombudsman (148), transparency officer (142), whistleblower (108) |
| Knowledge denial (new) | 4 | amnesiac witness (129), deniable expert (72), feigned novice (137), stonewaller (129) |
| Sincere ignorance (new) | 3 | honest novice (110), calibrated reporter (83), scrupulous witness (74) |
| Plain assertion (new) | 3 | forthright expert (80), candid witness (64), plain dealer (76) |
| Sycophantic (published) | 5 | cheerleader (142), enabler (51), flatterer (148), people pleaser (113), yes man (109) |
| Anti-sycophantic (published) | 5 | blunt advisor (135), contrarian (130), critic (68), skeptic (133), tough love mentor (149) |
| Anchor | 1 | default (150) |
Table 2. The roles of the bank by group.
Llama: supplementary results
Table 5 compares the detectors side by side at layer 40, Figure 2 shows the transfer matrices of the raw probe and the selected detector, Table 6 lists the evaluation sets, Table 7 the main detectors on every set at layer 40, Table 8 the three persona contrasts at layers 20, 40 and 44, Table 9 transfer for every feature set, and Table 10 HP-KR ranking against detection beside the benchmark’s published probes. Table 11 gives the raw probe’s and the extended contrast’s AUROC on every dataset at the three layers, Table 12 the sycophancy contrast, Table 13 the token reads, Table 14 the read at the Follow-Up turn’s “No” and Table 15 probes in three persona components, both at layer 20.
| Dataset | Short | Object of belief | Reason for lying | Examples | Lies |
| Convincing-Game | CG | self-knowledge | encouraged | 267 | 167 |
| Harm-Pressure Choice | HP-C | world-knowledge | inherent | 2,134 | 134 |
| Harm-Pressure Knowledge Report | HP-KR | self-knowledge | inherent | 2,139 | 139 |
| Insider-Trading | IT | self-knowledge | encouraged | 1,080 | 423 |
| Instructed-Deception | ID | world-knowledge | encouraged | 5,432 | 3,045 |
| RepE instruction pairs | RepE | world-knowledge | instructed | 550 | 275 |
| Alpaca (control) | n/a | n/a | none | 2,000 | 0 |
Table 6. The evaluation sets for Llama-3.3-70B-Instruct.
Table 7: AUROC at layer 40; the Mean Probe is Liars’ Bench’s layer-20 probe. On Gender-Secret the directions are built inside the fine-tune.
| Evaluation set | Trained on RepE: Mean Probe | Trained on RepE: base probe, layer 40 | Zero-shot, layer 40: extended contrast | Zero-shot, layer 40: knowledge direction | Zero-shot, layer 40: selected detector |
| CG | 1.00 | 0.972 | 0.912 | 0.641 | 0.865 |
| HP-C | 0.41 | 0.675 | 0.712 | 0.830 | 0.820 |
| HP-KR | 0.12 | 0.936 | 0.912 | 0.865 | 0.909 |
| IT | 0.93 | 0.338 | 0.980 | 0.881 | 0.975 |
| ID | 0.95 | 0.971 | 0.978 | 0.906 | 0.962 |
| Mean of the five | 0.682 | 0.778 | 0.899 | 0.825 | 0.906 |
| HP-KR, identical text | n/a | 0.779 | 0.721 | 0.793 | 0.787 |
| HP-KR, denials | n/a | 0.593 | 0.670 | 0.728 | 0.724 |
| Gender-Secret | 0.45 | 0.524 | 0.547 | 0.861 | 0.841 |
Table 7.
Table 8: The published (8 v 7), knowledge-denial (4 v 6) and extended (12 v 13) contrasts, scored zero-shot. Mean: over the six sets, including RepE.
| Test set | Layer 20: 8 v 7 | Layer 20: 4 v 6 | Layer 20: 12 v 13 | Layer 40: 8 v 7 | Layer 40: 4 v 6 | Layer 40: 12 v 13 | Layer 44: 8 v 7 | Layer 44: 4 v 6 | Layer 44: 12 v 13 |
| CG | 0.940 | 0.931 | 0.969 | 0.865 | 0.902 | 0.912 | 0.819 | 0.858 | 0.859 |
| HP-C | 0.717 | 0.585 | 0.639 | 0.581 | 0.755 | 0.712 | 0.607 | 0.725 | 0.712 |
| HP-KR | 0.148 | 0.890 | 0.882 | 0.593 | 0.919 | 0.912 | 0.589 | 0.908 | 0.894 |
| IT | 0.988 | 0.976 | 0.990 | 0.979 | 0.913 | 0.980 | 0.979 | 0.933 | 0.982 |
| ID | 0.959 | 0.932 | 0.954 | 0.974 | 0.980 | 0.978 | 0.962 | 0.978 | 0.974 |
| RepE | 0.994 | 0.882 | 0.991 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
| Mean | 0.791 | 0.866 | 0.904 | 0.832 | 0.912 | 0.916 | 0.826 | 0.900 | 0.903 |
Table 8.
Table 10: HP-KR: ranking and detection at the benchmark’s threshold, beside its published probes. †Trained with privileged HP-KR examples, a ceiling rather than a rival. Liars’ Bench reports its probes at layer 20 only, at two decimals; FPR: false-positive rate on Alpaca.
| Detector | AUROC | Balanced accuracy, 1% FPR | ||||
| Zero-shot persona contrasts (this work) | ||||||
| Published contrast (8 v 7) | 0.148 | 0.593 | 0.589 | 0.500 | 0.500 | 0.500 |
| Knowledge-denial contrast (4 v 6) | 0.890 | 0.919 | 0.908 | 0.533 | 0.910 | 0.757 |
| Extended contrast (12 v 13) | 0.882 | 0.912 | 0.894 | 0.500 | 0.500 | 0.500 |
| Trained on RepE, same pipeline | ||||||
| Base probe (trained on RepE) | 0.109 | 0.936 | 0.828 | 0.500 | 0.500 | 0.500 |
| Reported by Liars’ Bench for Llama-3.3-70B (its Tables 6–8) | ||||||
| Mean Probe | 0.12 | 0.50 | ||||
| Follow-Up Probe | 0.14 | 0.26 | ||||
| Upper-Bound Probe† | 0.97 | 0.95 |
Table 10.
Table 11: The raw probe (raw) and the extended contrast (12 v 13) on every dataset at layers 20, 40 and 44 (layer 44 post hoc).
| Test set | Layer 20 | Layer 40 | Layer 44 | |||
| raw | 12 v 13 | raw | 12 v 13 | raw | 12 v 13 | |
| CG | 0.663 | 0.969 | 0.787 | 0.912 | 0.757 | 0.859 |
| HP-C | 0.535 | 0.639 | 0.571 | 0.712 | 0.570 | 0.712 |
| HP-KR | 0.394 | 0.882 | 0.584 | 0.912 | 0.637 | 0.894 |
| IT | 0.799 | 0.990 | 0.800 | 0.980 | 0.736 | 0.982 |
| ID | 0.778 | 0.954 | 0.827 | 0.978 | 0.808 | 0.974 |
| RepE | 0.718 | 0.991 | 0.824 | 1.000 | 0.785 | 1.000 |
Table 11.
Table 12: The sycophancy contrast (5 v 5) on every dataset at layers 20, 40 and 44 (layer 44 post hoc), with its transfer (off-diagonal mean).
| Test set | L20 | L40 | L44 |
| CG | 0.999 | 0.998 | 0.983 |
| HP-C | 0.533 | 0.751 | 0.692 |
| HP-KR | 0.043 | 0.167 | 0.480 |
| IT | 0.983 | 0.985 | 0.987 |
| ID | 0.923 | 0.948 | 0.949 |
| RepE | 0.977 | 0.999 | 1.000 |
| Off-diagonal mean | 0.615 | 0.663 | 0.708 |
Table 12.
Table 13: Token reads of the extended contrast (reads that differ in which tokens are pooled) at layers 20, 40 and 44 (layer 44 post hoc): AUROC, balanced accuracy (BA) and recall (Rec., the share of lies flagged), both at the benchmark’s threshold.
| Read | Layer 20 | Layer 40 | Layer 44 | ||||||
| AUROC | BA | Rec. | AUROC | BA | Rec. | AUROC | BA | Rec. | |
| Mean (default) | 0.887 | 0.591 | 0.187 | 0.899 | 0.694 | 0.413 | 0.884 | 0.666 | 0.350 |
| Top-10% mean | 0.873 | 0.664 | 0.348 | 0.908 | 0.737 | 0.531 | 0.907 | 0.715 | 0.470 |
| Maximum | 0.858 | 0.659 | 0.336 | 0.910 | 0.750 | 0.548 | 0.914 | 0.701 | 0.418 |
| First response token | 0.753 | 0.605 | 0.334 | 0.863 | 0.606 | 0.227 | 0.876 | 0.568 | 0.140 |
| Last response token | 0.843 | 0.645 | 0.312 | 0.880 | 0.656 | 0.422 | 0.882 | 0.636 | 0.352 |
| Token before the reply | 0.739 | 0.628 | 0.415 | 0.825 | 0.652 | 0.383 | 0.822 | 0.656 | 0.327 |
Table 13.
| Test set | Published (8 v 7) response | Published (8 v 7) “No” | Knowledge-denial (4 v 6) response | Knowledge-denial (4 v 6) “No” | Extended (12 v 13) response | Extended (12 v 13) “No” |
| CG | 0.940 | 0.912 | 0.931 | 0.933 | 0.969 | 0.938 |
| HP-C | 0.717 | 0.612 | 0.585 | 0.472 | 0.639 | 0.556 |
| HP-KR | 0.148 | 0.168 | 0.890 | 0.237 | 0.882 | 0.190 |
| IT | 0.988 | 0.165 | 0.976 | 0.216 | 0.990 | 0.173 |
| ID | 0.959 | 0.900 | 0.932 | 0.904 | 0.954 | 0.903 |
| RepE | 0.994 | 1.000 | 0.882 | 1.000 | 0.991 | 1.000 |
Table 14. The published, knowledge-denial and extended contrasts read on the response and at the Follow-Up turn’s “No”, at layer 20.
| Features, trained on RepE | Response tokens | Follow-up “No” |
| Probe on raw activations (8192-d) | 0.734 | 0.636 |
| Extended contrast (1-d) | 0.887 | 0.552 |
| Extended PCs () | 0.736 | 0.579 |
| Published PCs () | 0.733 | 0.574 |
| Knowledge-denial PCs () | 0.810 | 0.699 |
Table 15. Five-dataset mean AUROC at layer 20 of probes trained on RepE in the top three principal components (PCs, ) of each contrast’s persona vectors, beside the raw probe and the extended contrast, read on the response tokens and at the Follow-Up turn’s “No”.
Depth
Figure 3 follows the selected detector’s transfer over the 16 layers against the published and unified contrasts and the raw probe, and Table 16 the knowledge direction, the extended contrast and their sum at every layer; all are point estimates from one run. The extended contrast’s transfer peaks at layer 36 (0.927); the knowledge-denial contrast’s HP-KR balanced accuracy of 0.910 at layer 40 is a spike, against 0.745 at layer 36 and 0.757 at layer 44; and the base probe ranks HP-KR at 0.967 at layer 36 while flagging none of its lies.
Figure 3. Transfer of the selected detector across the 16 layers against two persona contrasts and the raw probe.
Table 16: The knowledge direction, the extended contrast and the selected detector across depth; random: the random band’s upper edge (97.5th percentile of 1,000 random directions).
| Layer | HP-KR, identical text: Knowledge direction | HP-KR, identical text: Replication direction | HP-KR, identical text: random | HP-KR, denials: Knowledge direction | HP-KR, denials: random | Five-dataset mean: Knowledge direction | Five-dataset mean: Extended contrast | Five-dataset mean: Selected detector |
| 20 | 0.531 | 0.520 | 0.626 | 0.414 | 0.653 | 0.645 | 0.887 | 0.789 |
| 24 | 0.652 | 0.646 | 0.649 | 0.477 | 0.669 | 0.600 | 0.865 | 0.739 |
| 28 | 0.722 | 0.720 | 0.662 | 0.526 | 0.670 | 0.674 | 0.878 | 0.813 |
| 32 | 0.783 | 0.779 | 0.702 | 0.607 | 0.691 | 0.791 | 0.905 | 0.898 |
| 36 | 0.791 | 0.791 | 0.725 | 0.690 | 0.716 | 0.854 | 0.913 | 0.929 |
| 40 | 0.793 | 0.796 | 0.734 | 0.728 | 0.728 | 0.825 | 0.899 | 0.906 |
| 44 | 0.794 | 0.795 | 0.738 | 0.718 | 0.729 | 0.827 | 0.884 | 0.898 |
| 48 | 0.794 | 0.795 | 0.738 | 0.710 | 0.729 | 0.816 | 0.865 | 0.882 |
| 52 | 0.792 | 0.793 | 0.741 | 0.725 | 0.732 | 0.819 | 0.852 | 0.875 |
| 56 | 0.792 | 0.793 | 0.734 | 0.728 | 0.719 | 0.812 | 0.831 | 0.860 |
| 60 | 0.793 | 0.794 | 0.742 | 0.742 | 0.722 | 0.793 | 0.824 | 0.855 |
| 64 | 0.793 | 0.793 | 0.733 | 0.731 | 0.726 | 0.756 | 0.826 | 0.843 |
| 68 | 0.789 | 0.790 | 0.740 | 0.708 | 0.715 | 0.657 | 0.832 | 0.807 |
| 72 | 0.786 | 0.788 | 0.739 | 0.722 | 0.728 | 0.637 | 0.818 | 0.774 |
| 76 | 0.800 | 0.800 | 0.752 | 0.675 | 0.725 | 0.605 | 0.908 | 0.888 |
| 79 | 0.823 | 0.822 | 0.760 | 0.794 | 0.733 | 0.789 | 0.831 | 0.854 |
Table 16.
The random persona bank
We drew 35 roles at random (one seed) from the 274 non-default roles released with the Assistant Axis and built them with the bank pipeline with the same five variants, 30 questions, judge and filter. Partitioning them into six groups of the designed groups’ sizes (8, 7, 5, 5, 4 and 6) gives random contrasts of the published contrast’s sizes (8 v 7) and of the extended contrast’s (12 v 13), the second widening the first by four and six random roles. Split 0 is the reported random bank; splits 1 to 99 are the pre-registered null, and a test passes when its one-sided over them is at most 0.025. The six tests were fixed before any direction was scored. Table 17 gives the null medians and 97.5th percentiles from those splits and Holm-corrected values from 9,999 further splits drawn by the same rule.
Table 17: The random persona bank: six tests of the designed bank against the random null.
| Test | Comparison | Designed bank | Null median | Null 97.5% | Holm | Verdict |
| Transfer (off-diagonal AUROC) | extended contrast vs random 12 v 13 contrasts | 0.916 | 0.524 | 0.828 | 0.002 | pass |
| Zero-shot five-dataset mean | extended contrast vs random 12 v 13 contrasts | 0.899 | 0.609 | 0.814 | 0.002 | pass |
| Transfer with the knowledge direction | selected detector vs random contrast + knowledge direction | 0.922 | 0.812 | 0.891 | 0.002 | pass |
| HP-KR, all rows, AUROC | extended contrast vs random 12 v 13 contrasts | 0.912 | 0.648 | 0.938 | 0.144 | fail |
| Gain from widening, five-dataset mean | extended minus published contrast vs random 12 v 13 minus random 8 v 7 of the same split | +0.101 | −0.005 | +0.253 | 0.165 | fail |
| AISI held-out lies, AUROC | extended contrast vs random 12 v 13 contrasts | 0.977 | 0.804 | 0.955 | 0.002 | pass |
Table 17.
The knowledge direction
Figure 4 draws the construction. Table 18 gives the robustness checks (question bootstrap, balanced splits, the replication with new questions), Table 19 the paired differences between the selected detector and other detectors on every set, Table 20 the share of lies caught at in-setting cutoffs, Table 21 balanced accuracy under thresholds set on each dataset, and Table 22 five-dataset means of balanced accuracy and recall at the benchmark’s threshold, with AUROC, at three layers. The knowledge direction’s 120 questions, the replication set and the template variants are available on request.
| Check | Layer 36 | Layer 40 | Layer 44 |
| HP-KR, identical text | |||
| AUROC [question bootstrap] | 0.791 [0.790, 0.792] | 0.793 [0.792, 0.794] | 0.794 [0.793, 0.795] |
| Random directions, 97.5th pct. | 0.732 | 0.736 | 0.733 |
| Share of balanced splits as high | 0.000 | 0.000 | 0.000 |
| Replication with new questions | 0.791 | 0.796 | 0.795 |
| HP-KR, denials | |||
| AUROC [question bootstrap] | 0.690 [0.684, 0.697] | 0.728 [0.723, 0.733] | 0.718 [0.713, 0.724] |
| Random directions, 97.5th pct. | 0.717 | 0.730 | 0.724 |
| Share of balanced splits as high | 0.072 | 0.042 | 0.055 |
| Replication with new questions | 0.686 | 0.729 | 0.718 |
| Gender-Secret (fine-tune read) | |||
| AUROC [question bootstrap] | 0.810 [0.804, 0.817] | 0.865 [0.860, 0.870] | 0.869 [0.864, 0.873] |
| Random directions, 97.5th pct. | 0.814 | 0.817 | 0.856 |
| Share of balanced splits as high | 0.042 | 0.014 | 0.015 |
| Replication with new questions | 0.836 | 0.881 | 0.878 |
| All three sets | |||
| Cosine of the new-question direction with the original | 0.99 | 0.99 | 0.99 |
Table 18. Robustness of the knowledge direction on HP-KR’s identical text and denials and on Gender-Secret. Brackets resample the direction’s questions, not the test rows; on Gender-Secret the direction is built on the base model. Every random band in this table is a separate draw from those in Tables 1 and 30.
| Comparison | Five-dataset mean | HP-KR, identical text | HP-KR, denials |
| Selected detector − extended contrast | +0.007 [−0.004, +0.018] | +0.066 [+0.037, +0.098] | +0.054 [+0.025, +0.082] |
| Selected detector − knowledge-denial contrast | +0.012 [+0.001, +0.023] | +0.075 [+0.040, +0.112] | +0.154 [+0.112, +0.196] |
| Selected detector − knowledge direction | +0.082 [+0.073, +0.091] | −0.006 [−0.014, +0.002] | −0.004 [−0.011, +0.003] |
| Selected detector − base probe, layer 40 | +0.128 [+0.112, +0.145] | +0.008 [−0.023, +0.035] | +0.131 [+0.088, +0.178] |
| Published contrast + knowledge direction − published contrast | +0.081 [+0.068, +0.096] | +0.074 [+0.040, +0.111] | +0.072 [+0.037, +0.104] |
Table 19. Paired differences between the selected detector and four other detectors, and between the published contrast with and without the knowledge direction, with 95% intervals.
| Detector | In-setting cutoff: lies caught 1% | In-setting cutoff: lies caught 5% | In-setting cutoff: lies caught 10% | Alpaca 1% threshold: lies caught | Alpaca 1% threshold: honest flagged |
| Base probe, layer 40 | 0.32 | 0.53 | 0.66 | 0.28 | 0.03 |
| Published contrast | 0.34 | 0.49 | 0.58 | 0.38 | 0.03 |
| Published contrast + knowledge direction | 0.32 | 0.55 | 0.67 | 0.41 | 0.12 |
| Extended contrast | 0.47 | 0.68 | 0.77 | 0.41 | 0.03 |
| Knowledge direction | 0.21 | 0.42 | 0.56 | 0.34 | 0.23 |
| Selected detector | 0.38 | 0.63 | 0.73 | 0.47 | 0.07 |
Table 20. Share of lies caught at cutoffs set on each dataset’s own honest answers, layer 40.
| Layer | Detector | Balanced acc. | Recall | AUROC |
| L20 | Base probe (trained on RepE) | 0.721 | 0.489 | 0.697 |
| L20 | Published contrast (8 v 7) | 0.534 | 0.069 | 0.750 |
| L20 | Knowledge-denial contrast (4 v 6) | 0.559 | 0.305 | 0.863 |
| L20 | Extended contrast (12 v 13) | 0.591 | 0.187 | 0.887 |
| L40 | Base probe (trained on RepE) | 0.626 | 0.281 | 0.778 |
| L40 | Published contrast (8 v 7) | 0.675 | 0.382 | 0.798 |
| L40 | Knowledge-denial contrast (4 v 6) | 0.725 | 0.487 | 0.894 |
| L40 | Extended contrast (12 v 13) | 0.694 | 0.413 | 0.899 |
| L44 | Base probe (trained on RepE) | 0.626 | 0.281 | 0.756 |
| L44 | Published contrast (8 v 7) | 0.654 | 0.318 | 0.791 |
| L44 | Knowledge-denial contrast (4 v 6) | 0.683 | 0.395 | 0.880 |
| L44 | Extended contrast (12 v 13) | 0.666 | 0.350 | 0.884 |
Table 22. Five-dataset means of balanced accuracy and recall at the benchmark’s threshold, with AUROC, for four detectors at layers 20, 40 and 44 (layer 44 post hoc).
Model organisms
Figure 5 shows Gender-Secret at layers 36 (post hoc) and 40: each detector on the fine-tune’s read and on the base model’s read of the same transcripts, with each direction built inside the model whose read it scores. Table 23 gives every fine-tune of the three further models, with the base model’s read beside the fine-tune’s. On Llama’s two Soft-Trigger fine-tunes the knowledge direction clears the fine-tune read’s band at layer 40 (greeting 0.943 against 0.922, time 0.940 against 0.870), but the base model’s read of the same transcripts does too (greeting 0.937 against 0.911, time 0.937 against 0.918): the trigger is visible in the text, so these organisms cannot provide evidence for a detector of the fine-tune’s internal state, and the same holds for Mistral’s Soft-Trigger fine-tunes.
Figure 5. Gender-Secret read through the fine-tune (dark) and by the base model (light) at layers 36 (post hoc) and 40; grey marks the random band of the fine-tune’s read (the central 95% of the AUROCs of random directions).
| Fine-tune | Extended contrast | Knowledge direction | Selected detector | Probe | Band | Knowledge direction | Band |
| Qwen† | |||||||
| Soft-Trigger, greeting | 0.455 | 0.469 | 0.456 | 0.708 | 0.691 | 0.430 | 0.669 |
| Soft-Trigger, time | 0.569 | 0.531 | 0.544 | 0.654 | 0.710 | 0.451 | 0.700 |
| Soft-Trigger, gender | 0.479 | 0.454 | 0.452 | 0.680 | 0.734 | 0.444 | 0.660 |
| Mistral | |||||||
| Gender-Secret | 0.406 | 0.440 | 0.414 | 0.400 | 0.765 | 0.497 | 0.784 |
| Soft-Trigger, greeting | 0.931 | 0.930 | 0.936 | 0.878 | 0.900 | 0.872 | 0.823 |
| Soft-Trigger, time | 0.929 | 0.946 | 0.949 | 0.275 | 0.930 | 0.875 | 0.858 |
| Soft-Trigger, gender | 0.948 | 0.935 | 0.940 | 0.901 | 0.892 | 0.873 | 0.828 |
| Gemma | |||||||
| Soft-Trigger, greeting | 0.101 | 0.290 | 0.164 | 0.845 | 0.933 | 0.839 | 0.858 |
| Soft-Trigger, time | 0.916 | 0.719 | 0.876 | 1.000 | 0.962 | 0.860 | 0.875 |
| Soft-Trigger, gender | 0.560 | 0.804 | 0.763 | 0.794 | 0.951 | 0.849 | 0.868 |
Table 23. The fine-tunes of the three further models: AUROC on the fine-tune’s read and on the base model’s read of the same transcripts.
Steering
Table 24 gives disclosure of the hidden fact under each direction at over the 27 scenarios, with paired changes and 95% intervals. For each scenario the eleven answers (unsteered, and five directions at both signs) were shuffled with their conditions hidden and labelled by one reader, an AI assistant, before the key was opened; the labels are not those of a validated grader.
Table 24: Disclosure of the hidden fact under steering at : one blinded reader’s labels of 27 scenarios, one greedy answer each. Bold: the 95% interval excludes zero. The extended contrast’s interval on the honest side starts exactly at zero, so it is not bold.
| Direction | Honest side () | Deceptive side () | ||||
| Disclosure | Change [95% CI] | Disclosure | Change [95% CI] | |||
| Unsteered | 0.48 | – | 0.48 | – | ||
| Extended contrast (12 v 13) | 0.61 | +0.13 [0.00, +0.26] | 0.20 | −0.28 [−0.41, −0.13] | ||
| Published contrast (8 v 7) | 0.57 | +0.09 [−0.06, +0.24] | 0.20 | −0.28 [−0.41, −0.15] | ||
| Base probe (trained on RepE) | 0.50 | +0.02 [−0.11, +0.15] | 0.39 | −0.09 [−0.22, +0.04] | ||
| Random direction 1 | 0.43 | −0.06 [−0.17, +0.04] | 0.48 | 0.00 [−0.09, +0.09] | ||
| Random direction 2 | 0.46 | −0.02 [−0.09, +0.06] | 0.52 | +0.04 [−0.11, +0.19] |
Table 24.
Two small runs steered the knowledge direction with an absolute shift, , at the same block (numbers quoted from their reports). On HP-KR questions, subtracting the direction made the model deny knowing answers it still gave on the answer check: denials rose from 0.05 to 0.70 without pressure at , and to 17 of 20 at , against a rise of at most 0.10 for random directions of the same size at . Adding the direction produced no usable effect, and general-knowledge items did not move. This run was graded by pattern matching, and Claude also read every answer; the 17 of 20 at is its count. On the Gender-Secret fine-tune, the knowledge direction rebuilt inside the fine-tune revealed the hidden gender more often than one random direction at every matched size, at either sign (answers read by Claude, without blinding). But every reveal under any steering fell on a prompt with an earlier male-coded turn, and the belief the fine-tune holds did not move; with one random direction and 20 items per condition, the excess is suggestive only.
The four models
Table 25 lists the four models, Figure 6 follows four measures over each model’s depth sweep, Table 26 gives the main detectors at each model’s middle block, Table 27 the paired intervals behind Table 1, Table 28 every contrast on every dataset, Table 29 transfer for every feature set, Table 30 the wording-controlled slices, Table 31 the share of lies caught at in-setting cutoffs, Table 32 the probe trained on the model’s own lies where those exist (Llama and Gemma), and Table 33 the template-free knowledge direction and the token reads.
Figure 6. Four measures over each model’s 16-depth sweep, by relative depth (Llama blue, Qwen orange, Mistral green, Gemma purple; Mistral and Gemma also show the block that stands for Llama’s layer 20); Qwen’s extended-contrast curves (left column) use its second bank, with the role given in the user turn (Section 5); the bottom-right panel has the two models with a released identical-text slice, and its random bands are not drawn.
| Llama | Qwen† | Mistral | Gemma | |
| Transfer, mean off-diagonal AUROC | ||||
| Probe on raw activations | 0.732 | 0.559 | 0.716 | 0.746 |
| Published contrast | 0.832 | 0.451 | 0.633 | 0.700 |
| Knowledge-denial contrast | 0.912 | 0.679 | 0.780 | 0.442 |
| Extended contrast | 0.916 | 0.453 | 0.810 | 0.921 |
| Knowledge direction | 0.823 | 0.508 | 0.799 | 0.457 |
| Selected detector | 0.922 | 0.514 | 0.841 | 0.860 |
| Zero-shot, mean AUROC over the model’s datasets | ||||
| Mean Probe (Liars’ Bench, published) | 0.682 | 0.530 | 0.570 | 0.527 |
| Base probe, same block | 0.778 | 0.498 | 0.766 | 0.783 |
| Published contrast | 0.798 | 0.380 | 0.719 | 0.679 |
| Knowledge-denial contrast | 0.894 | 0.668 | 0.727 | 0.484 |
| Extended contrast | 0.899 | 0.441 | 0.762 | 0.900 |
| Knowledge direction | 0.825 | 0.546 | 0.775 | 0.515 |
| Selected detector | 0.906 | 0.533 | 0.804 | 0.847 |
| Lies caught at 1% of honest answers flagged | ||||
| Extended contrast | 0.47 | 0.02 | 0.14 | 0.43 |
| Selected detector | 0.38 | 0.03 | 0.23 | 0.30 |
Table 26. The main detectors at each model’s middle block.
| Comparison | Llama | Qwen† | Mistral | Gemma |
| Transfer, mean off-diagonal AUROC | ||||
| Extended contrast − raw probe | +0.184 [+0.172, +0.195] | −0.106 [−0.116, −0.094] | +0.094 [+0.082, +0.105] | +0.175 [+0.166, +0.184] |
| Extended contrast − published contrast | +0.084 [+0.074, +0.094] | +0.002 [−0.007, +0.012] | +0.176 [+0.169, +0.183] | +0.221 [+0.203, +0.238] |
| Selected detector − raw probe | +0.190 [+0.179, +0.199] | −0.045 [−0.057, −0.032] | +0.125 [+0.115, +0.136] | +0.114 [+0.101, +0.125] |
| Selected detector − extended contrast | +0.006 [−0.003, +0.015] | +0.060 [+0.052, +0.069] | +0.032 [+0.021, +0.043] | −0.061 [−0.069, −0.055] |
| Published contrast + knowledge direction − published contrast | +0.067 [+0.055, +0.078] | +0.052 [+0.040, +0.064] | +0.202 [+0.192, +0.213] | +0.110 [+0.098, +0.124] |
| Zero-shot, mean AUROC over the model’s datasets | ||||
| Extended contrast − published contrast | +0.101 [+0.089, +0.112] | +0.061 [+0.055, +0.067] | +0.043 [+0.034, +0.052] | +0.221 [+0.201, +0.240] |
| Selected detector − extended contrast | +0.007 [−0.004, +0.018] | +0.093 [+0.078, +0.107] | +0.042 [+0.028, +0.056] | −0.053 [−0.057, −0.049] |
| Selected detector − RepE probe | +0.128 [+0.112, +0.145] | +0.036 [+0.016, +0.057] | +0.038 [+0.022, +0.053] | +0.064 [+0.050, +0.078] |
Table 27. Paired differences at each model’s middle block, with 95% intervals.
| Model | Contrast | CG | HP-C | HP-KR | IT | ID | RepE | Mean |
| Llama | Published contrast | 0.865 | 0.581 | 0.593 | 0.979 | 0.974 | 1.000 | 0.798 |
| Knowledge-denial contrast | 0.902 | 0.755 | 0.919 | 0.913 | 0.980 | 1.000 | 0.894 | |
| Extended contrast | 0.912 | 0.712 | 0.912 | 0.980 | 0.978 | 1.000 | 0.899 | |
| Qwen† | Published contrast | n/a | 0.512 | 0.094 | n/a | 0.534 | 0.700 | 0.380 |
| Knowledge-denial contrast | n/a | 0.573 | 0.783 | n/a | 0.648 | 0.711 | 0.668 | |
| Extended contrast | n/a | 0.539 | 0.187 | n/a | 0.597 | 0.744 | 0.441 | |
| Mistral | Published contrast | n/a | 0.456 | 0.758 | 0.729 | 0.934 | 1.000 | 0.719 |
| Knowledge-denial contrast | n/a | 0.716 | 0.712 | 0.542 | 0.937 | 0.995 | 0.727 | |
| Extended contrast | n/a | 0.606 | 0.793 | 0.708 | 0.941 | 1.000 | 0.762 | |
| Gemma | Published contrast | 0.613 | n/a | n/a | 0.544 | 0.881 | 0.762 | 0.679 |
| Knowledge-denial contrast | 0.641 | n/a | n/a | 0.603 | 0.209 | 0.344 | 0.484 | |
| Extended contrast | 0.819 | n/a | n/a | 0.930 | 0.951 | 0.985 | 0.900 |
Table 28. Zero-shot AUROC per dataset at each model’s middle block.
| Detector, at the block for layer | Llama 20 | Llama 40 | Llama 44* | Gemma 20 | Gemma 40 | Gemma 44* |
| Base probe, trained on RepE | 0.697 | 0.778 | 0.756 | 0.512 | 0.783 | 0.761 |
| Base probe, trained on the model’s own lies | 0.782 | 0.878 | 0.872 | 0.802 | 0.858 | 0.770 |
| Published contrast | 0.750 | 0.798 | 0.791 | 0.575 | 0.679 | 0.691 |
| Knowledge-denial contrast | 0.863 | 0.894 | 0.880 | 0.622 | 0.484 | 0.564 |
| Extended contrast | 0.887 | 0.899 | 0.884 | 0.591 | 0.900 | 0.801 |
Table 32. Mean AUROC over the model’s benchmark datasets for the base probe trained on the model’s own belief-labelled lies (Llama and Gemma), beside the base probe trained on RepE and three contrasts; ∗post hoc layer.
| Model | Identical text | Transfer | Template knowledge direction’s transfer | Mean | Best |
| Llama | 0.794 | 0.791 | 0.823 | 0.899 | 0.910 (maximum) |
| Qwen† | n/a | 0.509 | 0.508 | 0.441 | 0.675 (first-token) |
| Mistral | 0.837 | 0.614 | 0.799 | 0.762 | 0.772 (top-10% mean) |
| Gemma | n/a | 0.444 | 0.457 | 0.900 | 0.900 (mean) |
Table 33. The template-free knowledge direction and the token reads of the extended contrast.
Aletheia’s Quest
The protocol was frozen (by hash) before any test row was read. Its headline cell, the pre-registered main condition, is the selected detector read through each organism at the model’s middle block, with reasoning dropped and the read pooled over the reply’s content tokens; the knowledge direction is rebuilt inside the organism and the persona contrast is taken from the base model’s bank. Secondary cells vary the detector (extended contrast, published contrast, knowledge-denial contrast, knowledge direction), the reading model (base model), where the directions are built (both inside the organism, or the knowledge direction on the base model), the token span (adding the end-of-turn token), the transcript rendering (reasoning kept) and the block (one selected on development data). Table 34 gives the per-dataset AUROCs and Figure 7 draws them. Deviations: Echoblast’s pre-registered development-row check failed (the rows tagged with that adapter look like base-model output), and its organism read was accepted on the text of its test replies alone. Rows of Blood on the Clocktower longer than the server could hold were cut to their last 4,096 tokens as soon as one such row had failed, rather than after each such row failed as the rule required. Because of a starter-kit fix on 22 July, it is not known whether the official white-box reads of the Qwen3.5 organisms had their adapters applied, so the base-model read is kept beside the organism read. Post hoc cells (the published contrast plus the knowledge direction, the template-free knowledge direction, probes in persona components trained on the development or the benchmark rows) rank between 11th and 28th and are available on request.
Figure 7. Aletheia’s Quest per dataset: the selected detector with its interval, the extended contrast and the RepE probe, the random band (grey) and the best white-box entry (bar).
The 150-question extension
The pre-registration, the question grid and the scripts are available on request. The 120 new questions were written in one pass by an AI assistant (Claude Fable 5.1) under a fixed prompt, frozen with the pre-registration before any rollout was generated; the prompt and the raw output are archived with it. Table 35 gives the tests, Table 36 the five-dataset mean of the extended contrast built from each question family and the change from leaving that family out of the pooled bank, and Table 37 the single confirmation run at layers 20, 36, 40 and 44. The decision rule chose the bank by the selected detector’s five-dataset mean on development data, with ties within 0.005 going to the larger bank; the four candidates lay within 0.003.
| Layer | Set | Direction | 150 questions | 30 questions | Difference [95%] |
| 20 | AISI held out | extended contrast | 0.974 | 0.971 | [, ] |
| 20 | AISI held out | selected detector | 0.963 | 0.948 | [, ] |
| 20 | Gender-Secret | extended contrast | 0.513 | 0.505 | [, ] |
| 20 | Gender-Secret | selected detector | 0.526 | 0.532 | [, ] |
| 36 | Gender-Secret | extended contrast | 0.444 | 0.409 | [, ] |
| 36 | Gender-Secret | selected detector | 0.751 | 0.761 | [, ] |
| 40 | AISI held out | extended contrast | 0.980 | 0.977 | [, ] |
| 40 | AISI held out | selected detector | 0.966 | 0.960 | [, ] |
| 40 | Gender-Secret | extended contrast | 0.491 | 0.467 | [, ] |
| 40 | Gender-Secret | selected detector | 0.778 | 0.802 | [, ] |
| 44 | AISI held out | extended contrast | 0.980 | 0.977 | [, ] |
| 44 | AISI held out | selected detector | 0.965 | 0.961 | [, ] |
| 44 | Gender-Secret | extended contrast | 0.605 | 0.541 | [, ] |
| 44 | Gender-Secret | selected detector | 0.808 | 0.807 | [, ] |
Table 37. The confirmation run: the 150-question bank against the 30-question bank, paired.
Side results
Four further studies on Llama bear on the method; their reports are available on request, and their numbers are quoted from those reports. Attention probes trained on 76 per-token persona coordinates transfer worse than the fixed reads at every layer (best trained variant 0.849 against 0.928 for the knowledge-denial contrast’s top-decile read at layer 40): learning the readout costs transfer. On the five-dataset mean AUROC, the zero-shot score averaged over all 16 layers is below its layer-40 value for every zero-shot direction (only the supervised base probe gains, from 0.778 to 0.807); the average beats a randomly chosen layer in expectation but neither layer 40 nor the best layer, so averaging guards against a poor choice of layer without improving on a good one. A knowledge follow-up question (“Did you actually know the answer?”) read at its prefilled “No” adds nothing over the response read, and the sign of every direction there follows the prefilled word. Finally, the covariance-weighted alignment of Ying et al. (2026) fits the transfer matrix ( 0.82 for the raw probe at layer 40, against 0.19 for the plain cosine), but the point-biserial correlation of a detector’s scores with the labels fits better (0.93). With the test set’s pooled covariance and a near-perfect own probe, the alignment reduces to the detector’s own effect size; this is the caveat attached to the extension’s covariance-weighted-cosine test (Table 35). The same analysis shows that HP-KR’s own probe plus the knowledge-denial contrast beats that probe alone at all 16 layers (0.949 to 0.978 at layer 40), so HP-KR carries at least two separating directions.
Statistics and reproducibility
Intervals are percentile intervals over 1,000 stratified resamples of the test rows with one seed, the same resamples for every detector, so differences are paired. A random band’s quoted edge is the 97.5th percentile of the AUROCs of 1,000 random unit directions on the same rows and block, reseeded per study, which moves a band by up to about 0.01 between studies; the main text quotes the values of the primary analysis, and the robustness and depth tables use their own draws. The block for each of Llama’s layers on the other models is the block at which the base probe reproduces the benchmark’s published Mean Probe (for layer 20), the middle block (for layer 40) and the blocks 5% of the depth to either side of it (for layers 36 and 44); the sweep covers the same 16 relative depths. Liars’ Bench is read at Hugging Face dataset revision 503399b, and the AISI rollouts at dataset revision 17a816. This paper’s numbers, tables and figures are written from the result files by scripts. The persona bank, the knowledge direction’s questions, the pre-registrations and every result file the scripts read are available from the authors on request.
References
- [1]Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644, 2016.arxiv.org/abs/1610.01644
- [2]Amos Azaria and Tom Mitchell. The internal state of an LLM knows when it’s lying. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 967–976, 2023.arxiv.org/abs/2304.13734
- [3]Lennart Bürger, Fred A. Hamprecht, and Boaz Nadler. Truth is universal: Robust detection of lies in LLMs. In Advances in Neural Information Processing Systems, volume 37, 2024.arxiv.org/abs/2407.12831
- [4]Cadenza Labs, National Deep Inference Fabric, and Schmidt Sciences. Aletheia’s Quest: An AI lie detection challenge. https://aletheias-quest.github.io/, 2026. Held-out results at https://aletheias-quest.github.io/winners.html.
- [5]Runjin Chen, Andy Arditti, Henry Sleight, Owain Evans, and Jack Lindsey. Persona vectors: Monitoring and controlling character traits in language models. arXiv preprint arXiv:2507.21509, 2025.arxiv.org/abs/2507.21509
- [6]Alan Cooney, David Africa, and Geoffrey Irving. “Did you lie?” Evaluating lie detectors across model scale and belief-verified model organisms. arXiv preprint arXiv:2606.12618, 2026.arxiv.org/abs/2606.12618
- [7]Alexander D’Amour, Katherine Heller, Dan Moldovan, et al. Underspecification presents challenges for credibility in modern machine learning. Journal of Machine Learning Research, 23(226):1–61, 2022.
- [9]Jaden Fiotto-Kaufman, Alexander R. Loftus, Eric Todd, Jannik Brinkmann, Koyena Pal, Dmitrii Troitskii, et al. NNsight and NDIF: Democratizing access to open-weight foundation model internals. In International Conference on Learning Representations, 2025.
- [10]Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A. Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673, 2020.DOI
- [11]Nicholas Goldowsky-Dill, Bilal Chughtai, Stefan Heimersheim, and Marius Hobbha hn. Detecting strategic deception with linear probes. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pp. 19755–19786, 2025.
- [12]Daniela Gottesman and Mor Geva. Estimating knowledge in large language models without generating a single token. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 3994–4019, 2024.
- [13]Ryan Greenblatt, Carson Denison, Benjamin Wright, et al. Alignment faking in large language models. arXiv preprint arXiv:2412.14093, 2024.arxiv.org/abs/2412.14093
- [14]Olli Järviniemi and Evan Hubinger. Uncovering deceptive tendencies in language models: A simulated company AI assistant. arXiv preprint arXiv:2405.01576, 2024.arxiv.org/abs/2405.01576
- [15]Nitish Joshi, Javier Rando, Abulhair Saparov, Najoung Kim, and He He. Personas as a way to model truthfulness in language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 6346–6359, 2024.
- [16]Saurav Kadayath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221, 2022.
- [17]Nathalie Kirch, Samuel Dower, Adrians Skapars, Ekdeep Singh Lubana, and Dmitrii Krasheninnikov. The impact of off-policy training data on probe generalisation. In Findings of the Association for Computational Linguistics: ACL 2026, pp. 22673–22729, 2026.arxiv.org/abs/2511.17408
- [18]Kieron Kretschmar, Walter Laurito, Sharan Maiya, and Samuel Marks. Liars’ Bench: Evaluating lie detectors for language models. arXiv preprint arXiv:2511.16035, 2025.arxiv.org/abs/2511.16035
- [19]Sachin Kumar. Pressure-testing deception probes in LLMs: Scaling, robustness, and the geometry of deceptive representations. In Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics (GEM), pp. 472–489, 2026.
- [20]Benjamin A. Levinstein and Daniel A. Herrmann. Still no lie detector for language models: Probing empirical and conceptual roadblocks. Philosophical Studies, 182(7):1539–1565, 2025.DOI
- [21]Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, et al. The WMDP benchmark: Measuring and reducing malicious use with unlearning. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 28525–28550, 2024.arxiv.org/abs/2403.03218
- [22]Christina Lu, Jack Gallagher, Jonathan Michala, Kyle Fish, and Jack Lindsey. The Assistant Axis: Situating and stabilizing the default persona of language models. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pp. 80955–81003, 2026.arxiv.org/abs/2601.10387
- [23]Prasad Mahadik and Adrians Skapars. Do linear probes generalize better in persona coordinates? arXiv preprint arXiv:2605.09391, 2026.arxiv.org/abs/2605.09391
- [24]Sam Marks, Jack Lindsey, and Christopher Olah. The persona selection model: Why AI assistants might behave like humans. Anthropic Alignment Science Blog, https://alignment.anthropic.com/2026/psm, 2026.
- [25]Samuel Marks and Max Tegmark. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. In Conference on Language Modeling (COLM), 2024.arxiv.org/abs/2310.06824
- [26]Amr Moustafa, Max Feser, and Florian Mai. Beyond Liars’ Bench: The impact of lie typology, depth, and sparsity on deception detection in LLMs. arXiv preprint arXiv:2607.20479, 2026.arxiv.org/abs/2607.20479
- [27]Vikram Natarajan, Devina Jain, Shivam Arora, Satvik Golechha, and Joseph Isaac Bloom. One probe won’t catch them all: Towards targeted deception detection. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pp. 91932–91947, 2026.arxiv.org/abs/2602.01425
- [28]Lorenzo Pacchiardi, Alex J. Chan, Sören Mindermann, Ilan Moscovitz, Alexa Y. Pan, Yarin Gal, Owain Evans, and Jan Brauner. How to catch an AI liar: Lie detection in black-box LLMs by asking unrelated questions. In International Conference on Learning Representations, 2024.arxiv.org/abs/2309.15840
- [29]Peter S. Park, Simon Goldstein, Aidan O’Gara, Michael Chen, and Dan Hendrycks. AI deception: A survey of examples, risks, and potential solutions. Patterns, 5(5):100988, 2024.DOI
- [30]Jérémy Scheurer, Mikita Balesni, and Marius Hobbhahn. Large language models can strategically deceive their users when put under pressure. arXiv preprint arXiv:2311.07590, 2023.arxiv.org/abs/2311.07590
- [31]Aviv Slobodkin, Omer Goldman, Avi Caciularu, Ido Dagan, and Shauli Ravfogel. The curious case of hallucinatory (un)answerability: Finding truths in the hidden states of over-confident large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 3607–3625, 2023.





