WorksComputer Science
Reva: Real-Time Vulnerability Detection through Contextual Probing of Large Language Models
Abstract
A vulnerability costs more to fix the later it is discovered. As coding agents generate code faster than developers can review it, software needs a first line of defense screening each new function in real time. LLM agents have recently become capable of finding real-world vulnerabilities by working as a security expert would: they run analysis tools, explore the surrounding code, test hypotheses and weigh the evidence before forming their own judgement. This reasoning, however, spans many model calls, so that each verdict takes minutes of compute, far too long for real-time screening. Faster detectors, which fine-tune deep learning models or prompt an LLM for a direct verdict, are less accurate: they judge a function without the context an agent gathers, and an LLM's generated verdict is unreliable even when given some of it. We present REVA (REal-time Vulnerability Analysis), an AI vulnerability detector that brings the security context and code understanding components behind agentic detectors into a single forward pass of an LLM. For each new function, REVA gives the LLM a pre-computed threat model of its module, lightweight static analysis findings and call-graph information. REVA leverages a trained probe to read the LLM's internal representations of the new function in this context. On PrimeVul's held-out test set, REVA on GLM-5.3-Flash reaches 60.0% accuracy in about one second per function. This matches VulTriage (59.2%), the state-of-the-art agentic detector, run on the same LLM, while being over 250 times faster and less than a tenth as costly. On the same prompt, the probe is 14% to 17% more accurate than the LLM's own direct verdict. Across three open LLMs, REVA is at least as accurate as VulTriage, and about 10% more accurate on two of them. An LLM's internal representations thus encode much of what agentic detectors obtain through reasoning.
Introduction
The cost of remediating a vulnerability grows the later it is discovered in the software lifecycle [29]. Vulnerabilities are therefore cheapest to fix when they are detected as the code is written. Coding agents now generate code faster than developers can review it, which calls for a first line of defense that screens each new function in real time, as it is generated.
Static analyzers run in real time, but at the scale of agent-generated code, they produce more reports than developers can review, many of them false positives [32, 33]. LLM agents have recently become capable of finding real-world vulnerabilities [30, 31] by combining such tools with their own analysis of the code, and agentic detectors bring this approach to function-level detection. VulTriage [20], the state of the art in the function-level detection literature, and VulAgent [19] enrich the prompt with retrieved vulnerability knowledge and program facts, and VulTrial [18] lets several agents debate a verdict. This reasoning, however, spans several model calls, which amounts to minutes per function. Faster AI-based detectors, which fine-tune deep learning models or prompt an LLM for a direct verdict, reduce this latency but are considerably less accurate [17, 21, 22].
[font=\scriptsize,
panel/.style={draw=ink2!70,dashed,rounded corners=3pt},
ptitle/.style={font=\scriptsize\bfseries,text=ink2,anchor=north west},
lbl/.style={align=center,text=mut,font=\fontsize{6.5}{7.5}\selectfont},
box/.style={draw=ink2,rounded corners=1.5pt,align=center,inner sep=2pt,fill=white,font=\scriptsize},
arr/.style={-{Stealth[length=3.5pt]},ink2,semithick},
card/.style={draw=ink2!60,fill=white,rounded corners=1.5pt},
layer/.style={draw=blu!80!black,rounded corners=1pt,minimum width=1.5cm,minimum height=0.15cm,inner sep=0pt}]
% ================= panel 1: templated prompt with security context =================
\draw[panel,fill=black!2] (0,0) rectangle (5.4,3.2);
\node[ptitle] at (0.1,3.13) {\ding{172}\ Prompt with security context};
\draw[card,draw=ink2,fill=white] (0.15,0.46) rectangle (5.25,2.76);
\begin{scope}[every node/.style={anchor=west,inner sep=0pt,font=\fontsize{6}{7}\selectfont\ttfamily,text=blu!80!black}]
\node at (0.27,2.62) {\#\# Threat model: project, module};
\node at (0.27,2.23) {\#\# Semgrep findings};
\node at (0.27,1.96) {\#\# Call graph: callers, callees};
\node at (0.27,1.58) {\#\# File: includes, types, macros};
\node at (0.27,1.25) {\#\# New function};
\end{scope}
\begin{scope}[mut!45,line width=1.6pt,line cap=round]
\draw (0.3,2.48) -- (4.6,2.48); \draw (0.3,2.38) -- (3.7,2.38);
\draw (0.3,2.1) -- (3.2,2.1);
\draw (0.3,1.82) -- (4.3,1.82); \draw (0.3,1.72) -- (2.9,1.72);
\draw (0.3,1.45) -- (4.8,1.45); \draw (0.3,1.35) -- (3.9,1.35);
\end{scope}
\begin{scope}[every node/.style={anchor=west,inner sep=0pt,font=\fontsize{6}{7}\selectfont\ttfamily,text=ink2}]
\node at (0.3,1.03) {node *new\_node(tree *tr) \{};
\node at (0.3,0.81) {\ node *t = \&tr->nodes[tr->n];};
\node at (0.3,0.59) {\ tr->n++; return t; \}};
\end{scope}
\node[lbl] at (2.7,0.21) {security context around the new function};
% ================= panel 2: one forward pass =================
\draw[panel,fill=blu!3] (5.6,0) rectangle (9.1,3.2);
\node[ptitle] at (5.7,3.13) {\ding{173}\ One forward pass};
\foreach \i in {0,...,8} {
\pgfmathtruncatemacro{\f}{8+5*\i}
\node[layer,fill=blu!\f] (L\i) at (6.75,0.62+0.19*\i) {};
}
\foreach \i in {3,5,7,8} { \node[layer,draw=blu!85!black,line width=0.9pt,fill=blu!55] at (6.75,0.62+0.19*\i) {};
\draw[-{Stealth[length=2.5pt]},blu!85!black,semithick] (7.52,0.62+0.19*\i) -- (8.2,0.62+0.19*\i); }
\foreach \i in {9,10} { \node[layer,draw=mut,dashed,fill=white] at (6.75,0.62+0.19*\i) {}; }
\node[lbl,anchor=west,align=left] at (7.62,2.4) {not\\executed};
\node[lbl,anchor=west,text=blu!85!black] at (8.25,1.05) {probed};
\node[lbl] at (7.35,0.25) {LLM, no text generated};
% ================= panel 3: code-token states =================
\draw[panel,fill=black!2] (9.3,0) rectangle (12.9,3.2);
\node[ptitle] at (9.4,3.13) {\ding{174}\ Code-token states};
\foreach \g in {0,...,3} {
\pgfmathsetmacro{\gy}{0.8+0.5*\g}
\foreach \r in {0,1,2} { \foreach \c in {0,...,11} {
\pgfmathtruncatemacro{\v}{mod(7*\c*\c+13*\r+5*\c*\r+11*\g+3,58)+6}
\fill[blu!\v] (10.05+0.18*\c,\gy+0.15*\r) rectangle ++(0.18,0.15);
}}
\draw[white,line width=0.3pt] (10.05,\gy) grid[xstep=0.18,ystep=0.15] (12.21,\gy+0.45);
\draw[ink2,line width=0.4pt] (9.95,\gy) rectangle (12.21,\gy+0.45);
\pgfmathtruncatemacro{\gl}{\g+1}
\node[anchor=east,text=ink2,font=\fontsize{6.5}{7.5}\selectfont] at (10.0,\gy+0.225) {$\ell_{\gl}$};
}
\draw[ink2,decorate,decoration={brace,amplitude=3pt}] (12.28,2.75) -- (12.28,0.8) node[midway,right=2pt,text=ink2,font=\fontsize{6.5}{7.5}\selectfont,align=left] {con-\\cat};
\node[lbl] at (11.1,0.25) {$T$ code tokens $\times$ 4$\cdot$1{,}024};
% ================= panel 4: attention probe -> risk score =================
\draw[panel,fill=blu!3] (13.1,0) rectangle (17.9,3.2);
\node[ptitle] at (13.2,3.13) {\ding{175}\ Attention probe $\to$ risk score};
\foreach \k/\h in {0/0.08,1/0.12,2/0.10,3/0.18,4/0.62,5/0.70,6/0.22,7/0.10,8/0.07,9/0.12,10/0.09,11/0.06,12/0.08,13/0.05} {
\ifnum\k=4 \def\bc{warm}\else\ifnum\k=5 \def\bc{warm}\else\def\bc{blu!55}\fi\fi
\fill[\bc] (13.55+0.27*\k,1.9) rectangle ++(0.17,\h);
}
\draw[ink2,line width=0.4pt] (13.45,1.9) -- (17.4,1.9);
\node[lbl] at (15.45,1.72) {learned attention over code tokens};
% risk gauge
\shade[left color=blu!12,right color=warm!55] (13.5,1.0) rectangle (17.5,1.14);
\draw[ink2,line width=0.4pt] (13.5,1.0) rectangle (17.5,1.14);
\draw[ink2,line width=0.8pt] (15.5,0.92) -- (15.5,1.22);
\node[lbl,anchor=north] at (15.5,0.92) {median threshold};
\fill[warm] (16.7,1.2) -- ++(-0.08,0.13) -- ++(0.16,0) -- cycle;
\node[anchor=south,inner sep=1pt,text=warm!85!black,font=\fontsize{6.5}{7.5}\selectfont] at (16.7,1.33) {function's risk score};
\node[lbl] at (15.5,0.25) {probe of \PROBEPARAMS{} parameters, trained on labels};
% ================= arrows between panels =================
\draw[arr] (5.4,1.6) -- (5.6,1.6);
\draw[arr] (8.55,1.6) -- (9.3,1.6);
\draw[arr] (12.9,1.6) -- (13.1,1.6);Figure 1. Overview of Reva on a vulnerable function from LibRaw (CVE-2020-35530). The prompt combines the function with its security context. The LLM processes the prompt in one forward pass, up to the deepest of four probed layers. The hidden states of the code tokens are kept at each probed layer. An attention probe pools them into a risk score.
Reva is designed to close this gap between speed and improved accuracy. Instead of letting the LLM gather context through multiple rounds of tool calls, Reva assembles it in the prompt, including a threat model of the function’s module, its static analysis findings, its call graph and its file. Instead of reasoning towards a verdict, the LLM then processes this prompt in a single forward pass without generating text, and a small probe trained on labeled functions maps its internal representations of the code tokens to a risk score. This second choice builds on evidence that LLMs encode knowledge that their answers do not express, for example about the truth of a statement [1, 2, 3] or the correctness of their own answers [4, 5], and that chain-of-thought explanations are not always faithful to the underlying computation [6].
We evaluate Reva on PrimeVul [17], a high-quality dataset of vulnerable C and C++ functions paired with benign versions of the same functions, which spans 651 projects and 121 vulnerability types (CWEs). On its held-out test set, Reva on GLM-5.3-Flash [16] is on par with VulTriage and more accurate than VulAgent run on the same LLM, in about one second per function, over 250 times faster than VulTriage. On two other open LLMs, it is 9% to 13% more accurate than both agentic detectors.
This paper makes three contributions: (i) we propose Reva, a vulnerability detector that combines targeted security context with a probe on an LLM’s internal representations, in a single forward pass per function; (ii) we compare it with state-of-the-art agentic function-level detectors run on the same three LLMs, on PrimeVul’s held-out test set, in terms of accuracy, cost and robustness; and (iii) we outline a research agenda towards detectors that screen every function as coding agents generate it and escalate likely vulnerabilities to more capable agents that construct a proof of vulnerability.
Background and Related Work
LLM-based vulnerability detection. LLMs in non-agentic settings typically struggle to tell a vulnerable function from its benign counterpart: on PrimeVul [17], GPT-4 with chain-of-thought labels both versions correctly in only 12.9% of the pairs, and LLMs asked for a verdict in non-agentic settings, often reason inconsistently about security [7, 8]. Reva instead reads the LLM’s internal representations rather than its answer or reasoning.
Agentic detectors. Recent detectors improve accuracy by adding knowledge and reasoning steps. VulTriage retrieves CWE descriptions and verbalizes control- and data-flow facts [20], VulAgent lets specialized agents propose and validate hypotheses against the project’s code [19], and VulTrial stages a debate between agents [18]. Their cost grows with each call, and their context can exceed the LLM’s context window. Reva instead pre-loads the key parts of this context into the prompt and replaces the multi-step reasoning with a single probed forward pass.
Probing LLM internal representations. Probes are small classifiers trained on a model’s hidden states [9]. For code, linear probes indicates some capability to separate secure from insecure code on toy datasets [23, 24], and predict whether generated code will be insecure or incorrect [26, 27]. These probes read the code alone, whereas Reva probes real-world functions read with targeted security context.
Pitfalls of vulnerability benchmarks. Detectors can exploit spurious features of their datasets, such as code metrics or semantics unrelated to the vulnerability [11, 25]. Such a detector tends to classify a vulnerable function and its benign counterpart with the same label, which share these features. We therefore score each function independently, then check whether the two versions of each pair receive their correct labels.
Contextual Probing with Reva
In Reva, an LLM reads a new function with its security context in one forward pass, and a probe maps the hidden states of the code tokens to a risk score (Figure 1).
Prompt with security context. The prompt contains the new function and the context around it, in labeled sections (Figure 1). A threat model, built once per project by a workflow of LLM agents over the entire software, describes the project (summary, attacker-controlled inputs, trust boundaries) and the module that contains the function (role, inputs that reach it, typical sinks and guards). The prompt adds the Semgrep [14] findings for the function, its call sites and the signatures of the helpers it calls, and the includes, types, macros and sibling functions of its file, extracted with tree-sitter from the repository. It is available for 84% of the training functions; otherwise the prompt has the function and its Semgrep findings only. The median prompt has 3,840 tokens, 601 of them code.
One forward pass. The LLM processes the prompt in one forward pass that stops after the deepest of four probed layers, so the remaining layers are never run and it generates no output token. We probe intermediate layers, which probing studies often find most informative [9]: four evenly spaced layers from 38% to 85% of the depth of the LLM, selected by cross-validation.
Code-token states. At each probed layer, Reva keeps only the hidden states of the function’s tokens. Each state is normalized and projected to 1,024 dimensions, and the four layers are concatenated.
Attention probe. A probe with 8 attention heads pools the token states into one risk score. It has 0.57 M parameters and is trained on PrimeVul’s training functions. Its loss combines a per-function classification term with a term that ranks each vulnerable function above its benign counterpart. These pairs serve only in training: Reva scores each function on its own.
Use in practice. The LLM and Semgrep’s rules are loaded once at startup, so they don’t have to be booted for each new analysis, and each project’s threat model is built ahead of time. For each new function, Reva extracts its context, runs Semgrep, builds the prompt, runs the pass and applies the probe. With Muse-Glimmer-30B on one H200 GPU, this takes 0.93 s per function, 0.46 s of which is context extraction.
Preliminary Results
We evaluate Reva through three research questions:
RQ1: How accurate is Reva compared with agentic detectors running on the same LLM?
RQ2: How much computation does Reva save?
RQ3: Which parts of Reva matter, and is its accuracy robust to leakage and selection?
Figure 2. Accuracy versus time per function on the PrimeVul test set, one color per LLM. Functions without a verdict count as benign. Hollow marker: VulTriage on GPT-4o, as reported.
Experimental Setup
Dataset and metrics. PrimeVul [17] pairs each vulnerable C/C++ function with a benign version of the same function. It has 3,789 training, 480 validation and 435 test pairs. Following [17], we count the pairs in which both versions are labeled correctly (), both are flagged (), both are cleared (), or the labels are reversed (). Our main metric is accuracy over all functions, each scored independently.
Protocol. We made every design choice on the training and validation sets only. Cross-validation (CV) accuracy covers five folds of the training set, with all functions of a commit in the same fold, and the validation set, scored by a probe trained on the full training set. The test set stayed unseen during development and was scored only with the final configuration.
LLMs. We use three open-weight LLMs: Muse-Glimmer-30B [28], Qwen3.8-27B [15] and GLM-5.3-Flash [16], a larger mixture-of-experts model. On GLM-5.3-Flash, the agentic detectors use the FP8 version of the LLM and Reva an exact BF16 conversion, which leaves its cross-validation accuracy unchanged (66.2% vs. 65.9%).
Baselines. We run VulAgent and VulTriage on their released code, using the same LLMs as Reva. A function is failed when the detector returns no verdict, usually because the reasoning exceeds the token limit. We compare all methods over all functions, counting failed functions as benign. Published GPT-4o results are given for context.
| Method and LLM | Calls | Tokens (k) | Time (s) | Cost () | FPR | Accuracy | 95% CI | P-C | P-R |
| Reva (ours) | |||||||||
| Muse-Glimmer-30B | 1 | 3.2 / 0 | 0.93 | 0.9 | 41.1 | 58.9 | [56.8, 61.0] | 20.2 | 2.5 |
| Qwen3.8-27B | 1 | 3.4 / 0 | 0.88 | 0.9 | 40.5 | 59.5 | [57.4, 61.7] | 22.1 | 3.0 |
| GLM-5.3-Flash | 1 | 3.1 / 0 | 1.33 | 1.3 | 40.0 | 60.0 | [57.8, 62.2] | 22.3 | 2.3 |
| Muse-Glimmer-30B + GLM-5.3-Flash | 2 | 6.3 / 0 | 1.78 | 1.7 | 39.3 | 60.7 | [58.5, 62.9] | 23.2 | 1.8 |
| VulTriage | |||||||||
| Muse-Glimmer-30B | 3 | 6.2 / 4.8 | 108 | 9.4 | 75.6 | 53.6 | [51.5, 55.7] | 14.0 | 6.9 |
| Qwen3.8-27B | 3 | 8.0 / 19.4 | 564 | 61.4 | 75.4 | 53.9 | [51.6, 56.2] | 16.1 | 8.3 |
| GLM-5.3-Flash | 3 | 6.2 / 24.7 | 362 | 13.3 | 46.0 | 59.2 | [56.7, 61.6] | 26.0 | 7.6 |
| black!50GPT-4o (published, not reproducible) | black!503 | black!50-- | black!5034.5 | black!50-- | black!5054.3 | black!5063.6 | black!50-- | black!5033.8 | black!506.7 |
| VulAgent | |||||||||
| Muse-Glimmer-30B | 9.2 | 34.1 / 33.8 | 785 | 62.6 | 64.1 | 52.2 | [50.0, 54.4] | 13.3 | 9.0 |
| Qwen3.8-27B | 9.2 | 37.4 / 97.3 | 5,614 | 306.9 | 32.2 | 54.5 | [52.0, 57.0] | 20.2 | 11.3 |
| GLM-5.3-Flash | 9.3 | 35.8 / 184.8 | 3,495 | 97.8 | 19.8 | 52.8 | [50.5, 54.9] | 14.3 | 8.7 |
| black!50GPT-4o (published) | black!508.8 | black!50-- / 7.0 | black!50-- | black!50-- | black!5036.7 | black!5058.4 | black!50-- | black!5026.6 | black!5010.1 |
| VulTrial | |||||||||
| black!50GPT-4o (published) | black!50-- | black!50-- | black!50-- | black!50-- | black!5052.6 | black!5053.6 | black!50-- | black!5018.6 | black!5011.5 |
Table 1. Accuracy and cost of Reva{} and agentic detectors on the PrimeVul test set. Tokens: mean input and output tokens per function, in thousands. Time: median seconds per function. CI: pair-bootstrap interval on accuracy. Cost: USD per 1{,}000 functions, for Reva{} on one rented H200 GPU at $3.50 per hour, and for the agentic detectors at public API rates per million input/output tokens (Muse-Glimmer-30B{} $0.35/$1.50, Qwen3.8-27B $0.40/$3.00, GLM-5.3-Flash $0.15/$0.50). FPR: share of non-vulnerable functions flagged. --: not reported.
Figure 3. Change in validation accuracy of Reva on Muse-Glimmer-30B when one component is removed or replaced, with paired 95% confidence intervals.
RQ1: Accuracy
On GLM-5.3-Flash, Reva reaches 60.0% accuracy (95% CI [57.8, 62.2]), against 59.2% for VulTriage and 52.8% for VulAgent run on the same LLM (Table 1). Over all test functions, Reva is 0.8 points above VulTriage (CI [2.1, +3.8]) and 7.2 points above VulAgent (CI [+4.4, +10.3]). On Qwen3.8-27B and Muse-Glimmer-30B, Reva is 5.1 to 6.7 points more accurate than both agentic detectors, and all four confidence intervals exclude zero.
Beyond accuracy, Reva also compares favorably with VulTriage, the state-of-the-art function-level agentic detector, when both run on the same LLM. On all three LLMs, Reva has a lower false positive rate (FPR) and inverts the labels of fewer pairs (), and on two of them it labels more pairs entirely correctly (). Reva thus not only matches or exceeds VulTriage in accuracy, but also separates vulnerable from benign functions more reliably, which suggests that the probe captures the vulnerability itself rather than surface features that a vulnerable function shares with its secured counterpart.
Figure 2 summarizes this trade-off. On each LLM, Reva is at least as accurate as both agentic detectors while being 100 to 6,000 times faster. Its accuracy also varies little across LLMs, by 1.1 points against 5.6 for VulTriage, so that Reva on Muse-Glimmer-30B matches VulTriage on GLM-5.3-Flash, the LLM on which VulTriage performs best. The signal read by the probe is therefore present in all three LLMs, whereas the accuracy of the agentic detectors depends on how well each LLM reasons about the code. VulTriage on GPT-4o reports a higher accuracy (63.6%), but we were unable to replicate these results.
RQ2: Computational Cost
End to end, Reva takes 0.88 s to 1.33 s per function, against 108 s to 5,614 s for the agentic detectors on the same LLMs. Although these times, measured under parallel load, overstate single-request latency, the token counts confirm the gap. Reva reads about 3,100 to 3,400 tokens per function and generates none, whereas the agentic detectors process 3 to 70 times more tokens, many of them generated. As Reva needs the LLM’s hidden states, it cannot run behind an API. On a rented H200 GPU, it screens 1,000 functions for $0.9 to $1.3, against $9 to $61 for VulTriage and $63 to $307 for VulAgent at the public API rates of the same LLMs.
RQ3: Ablation and Robustness
We ablate each component of Reva on the Muse-Glimmer-30B validation set (Figure 3). Asking the same LLM for a Yes/No verdict on the same prompt, instead of probing it, reduces accuracy to chance. On the test set, it stays 7.4 to 8.7 points below the probe, and Semgrep, Cppcheck and Flawfinder alone are also at chance (50.8% to 51.3%). The LLM’s internal representations thus carry information that neither its verdict nor the static-analysis findings provide alone. Within the probe, replacing attention pooling by a plain average of the token states lowers accuracy by 4.2 points, which suggests that the evidence of a vulnerability lies in a few tokens. Using a single layer, omitting the standardization or dropping the ranking term lowers accuracy by 0.4 to 1.5 points, and removing the repository context or keeping only the function by 1.0 and 1.9 points, none of which is significant. On PrimeVul’s short functions, the function alone thus carries most of the signal; we expect the context to matter more for interprocedural vulnerabilities.
Leakage and selection on the evaluation data can inflate probing results [9, 10]. When no project appears in both training and evaluation folds, cross-validation accuracy drops by only 1.3 to 2.7 points, so most of what the probe learns transfers to unseen projects. Likewise, choosing the four probed layers on the training folds only, instead of on all development data, changes accuracy by less than half a point, so the layer choice does not inflate the results. Accuracy is, however, 6 points lower on the test set than in cross-validation (60.0% against 66.2% for GLM-5.3-Flash). As PrimeVul splits its data chronologically, this gap likely reflects the shift towards newer code, which cross-validation does not capture.
Future Plans
Step 1: Complete the comparison. We will run both agentic detectors and Reva on larger open LLMs. We will also test whether reading the internal representations after a short self-generated analysis adds information. Success criterion: Reva matches or exceeds them on the same LLM at every scale.
Step 2: Calibrated screening in coding agents. We will learn a project- and function-dependent threshold, and integrate Reva into a coding-agent loop that escalates only high-risk functions to an agentic detector or a proof-of-vulnerability check [35]. Success criterion: for a given budget of escalations, the loop finds more vulnerabilities than the agentic detector alone.
Step 3: Generalization beyond PrimeVul. We will evaluate on other datasets [12, 13, 34], on interprocedural vulnerabilities [36, 35] and on vulnerabilities disclosed after each LLM’s training cutoff, and study fine-tuning the LLM with the same objective. Success criterion: Reva keeps its accuracy on vulnerabilities the LLM cannot have seen.
Conclusion
Reva detects vulnerabilities by probing an LLM’s internal representations of a function and its security context in a single forward pass. On PrimeVul’s held-out test set, it matches or exceeds agentic detectors run on the same LLM, with fewer false positives, in about one second per function and at a small fraction of their cost. An LLM’s internal representations thus carry more information about vulnerabilities than its verdicts, and reading them enables real-time screening of every function as it is written.
References
- [1]Azaria, A., & Mitchell, T. (2023). The Internal State of an LLM Knows When It's Lying. Findings of the Association for Computational Linguistics: EMNLP 2023.
- [2]Burns, C., Ye, H., Klein, D., & Steinhardt, J. (2023). Discovering Latent Knowledge in Language Models Without Supervision. Proc. Int. Conf. On Learning Representations (ICLR).
- [3]Marks, S., & Tegmark, M. (2024). The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets. Proc. Conf. On Language Modeling (COLM).
- [4]Orgad, H., Toker, M., Gekhman, Z., Reichart, R., Szpektor, I., Kotek, H., & Belinkov, Y. (2025). LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations. Proc. Int. Conf. On Learning Representations (ICLR).
- [5]Kadavath, S., Conerly, T., Askell, A., et al. (2022). Language Models (Mostly) Know What They Know. https://arxiv.org/abs/2207.05221
- [6]Turpin, M., Michael, J., Perez, E., & Bowman, S. R. (2023). Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting. Advances in Neural Information Processing Systems (NeurIPS).
- [7]Ullah, S., Han, M., Pujar, S., Pearce, H., Coskun, A., & Stringhini, G. (2024). LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks. Proc. IEEE Symposium on Security and Privacy (S&P).
- [8]Steenhoek, B., Rahman, M. M., Roy, M. K., Alam, M. S., Tong, H., Das, S., Barr, E. T., & Le, W. (2025). To Err is Machine: Vulnerability Detection Challenges LLM Reasoning. Proc. 47th IEEE/ACM Int. Conf. On Software Engineering (ICSE).
- [9]Belinkov, Y. (2022). Probing Classifiers: Promises, Shortcomings, and Advances. Computational Linguistics, 48(1), 207–219.
- [10]Hewitt, J., & Liang, P. (2019). Designing and Interpreting Probes with Control Tasks. Proc. Conf. On Empirical Methods in Natural Language Processing (EMNLP).
- [11]Risse, N., & Böhme, M. (2024). Uncovering the Limits of Machine Learning for Automatic Vulnerability Detection. Proc. 33rd USENIX Security Symposium.
- [12]He, J., & Vechev, M. (2023). Large Language Models for Code: Security Hardening and Adversarial Testing. Proc. ACM SIGSAC Conf. On Computer and Communications Security (CCS).
- [13]Chen, Y., Ding, Z., Alowain, L., Chen, X., & Wagner, D. (2023). DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detection. Proc. 26th Int. Symp. On Research in Attacks, Intrusions and Defenses (RAID).
- [14]Semgrep, Inc. (2026). Semgrep: Lightweight static analysis for many languages. In https://semgrep.dev. https://semgrep.devsemgrep.dev
- [15]Qwen Team. (2026). Qwen3.8-27B. In https://huggingface.co/Qwen/Qwen3.8-27B. https://huggingface.co/Qwen/Qwen3.8-27Bhuggingface.co/Qwen/Qwen3.8-27B
- [16]Z.ai. (2026). GLM-5.3-Flash. In https://huggingface.co/zai-org/GLM-5.3-Flash. https://huggingface.co/zai-org/GLM-5.3-Flashhuggingface.co/zai-org/GLM-5.3-Flash
- [17]Ding, Y., Fu, Y., Ibrahim, O., Sitawarin, C., Chen, X., Alomair, B., Wagner, D., Ray, B., & Chen, Y. (2025). Vulnerability Detection with Code Language Models: How Far Are We?. Proc. 47th IEEE/ACM Int. Conf. On Software Engineering (ICSE).
- [18]Widyasari, R., Weyssow, M., Irsan, I. C., Ang, H. W., Liauw, F., Ouh, E. L., Shar, L. K., Kang, H. J., & Lo, D. (2026). Let the Trial Begin: A Mock-Court Approach to Vulnerability Detection Using LLM-Based Agents. Proc. 48th IEEE/ACM Int. Conf. On Software Engineering (ICSE). https://doi.org/10.1145/3744916.3773256
- [19]Wang, Z., Li, G., Li, J., Zhu, H., & Jin, Z. (2026). VulAgent: Hypothesis-Validation Driven Multi-Agent Architecture for Vulnerability Detection. Findings of the Association for Computational Linguistics: ACL 2026, 18598–18616. https://doi.org/10.18653/v1/2026.findings-acl.928
- [20]Tang, W., Zhang, X., Liu, J., Xiao, J., Xiao, X., Yang, J., Ma, Y., Liu, Z., Li, Z., Wang, Z., Luo, W., Li, Q., Wang, L., & Xiangli, P. (2026). VulTriage: Triple-Path Context Augmentation for LLM-Based Vulnerability Detection. https://arxiv.org/abs/2605.09461
- [21]Wen, X.-C., Yang, Y., Gao, C., Xiao, Y., & Ye, D. (2025). Boosting Vulnerability Detection of LLMs via Curriculum Preference Optimization with Synthetic Reasoning Data. Findings of the Association for Computational Linguistics: ACL 2025, 8935–8949. https://doi.org/10.18653/v1/2025.findings-acl.467
- [22]Ovadia, O., Ben Zaken, E., Guttman, E., & Moreno Kadosh, O. (2026). MiST: Mid-Training LLMs for Cybersecurity. https://arxiv.org/abs/2609.18496
- [23]Yu, W., Mangal, R., Zhuo, T., Fredrikson, M., & Pasareanu, C. S. (2025). A Mixture of Linear Corrections Generates Secure Code. https://arxiv.org/abs/2507.09508
- [24]Khatri, A. (2026). Probing the Prefill: Detecting Code Vulnerabilities via Latent Activations. https://arxiv.org/abs/2608.16970
- [25]Weissberg, F., Pirch, L., Imgrund, E., Möller, J., Eisenhofer, T., & Rieck, K. (2026). LLM-based Vulnerability Discovery through the Lens of Code Metrics. Proc. 48th IEEE/ACM Int. Conf. On Software Engineering (ICSE). https://doi.org/10.1145/3744916.3764574
- [26]Sternfeld, A., Kucharavy, A., & Dolamic, L. (2026). Minimal Prompt Perturbations Lead to Code Vulnerabilities: Prompt Fragility and Hidden-State Signals in Coding LLMs. https://arxiv.org/abs/2605.29737
- [27]Di Cicco, C. (2026). Code Correctness Is Linearly Decodable from LLM Hidden States Before Generation. https://arxiv.org/abs/2606.14530
- [28]Meta. (2026). Muse-Glimmer-30B. In https://huggingface.co/meta-models/Muse-Glimmer-30B. https://huggingface.co/meta-models/Muse-Glimmer-30Bhuggingface.co/meta-models/Muse-Glimmer-30B
- [29]Boehm, B., & Basili, V. R. (2001). Software Defect Reduction Top 10 List. Computer, 34(1), 135–137.
- [30]Wang, Z., Shi, T., He, J., Cai, M., Zhang, J., & Song, D. (2026). CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale. Proc. Int. Conf. On Learning Representations (ICLR). https://arxiv.org/abs/2506.02548
- [31]DARPA. (2025). AI Cyber Challenge (AIxCC). In https://www.darpa.mil/research/programs/ai-cyber. https://www.darpa.mil/research/programs/ai-cyberdarpa.mil/research/programs/ai-cyber
- [32]Johnson, B., Song, Y., Murphy-Hill, E., & Bowdidge, R. (2013). Why Don't Software Developers Use Static Analysis Tools to Find Bugs?. Proc. 35th IEEE/ACM Int. Conf. On Software Engineering (ICSE), 672–681.
- [33]Du, X., Feng, J., Zou, Y., Xu, W., Ma, J., Zhang, W., Liu, S., Peng, X., & Lou, Y. (2026). Reducing False Positives in Static Bug Detection with LLMs: An Empirical Study in Industry. https://arxiv.org/abs/2601.18844
- [34]Lbath, A., Amini, M.-R., Delaitre, A., & Okun, V. (2025). AVIATOR: Towards AI-Agentic Vulnerability Injection Workflow for High-Fidelity, Large-Scale Code Security Dataset. https://arxiv.org/abs/2508.20866
- [35]Lbath, A., Suri, M., Delaitre, A., Okun, V., Amini, M.-R., Sriram, R. D., & Manocha, D. (2026). CyberForge: Verified Vulnerability Injection at Repository Level for Cybersecurity Agent Training. https://arxiv.org/abs/2608.06471
- [36]Lbath, A. (2026). Toward Scalable Automated Repository-Level Datasets for Software Vulnerability Detection. https://arxiv.org/abs/2603.17974

