WorksComputer Science

Defending Against Dormant Poisoning Attacks Across Language and Multimodal Agents

Wonjun Lee, Vaidehi Patil, Zaid Khan, Yue Zhang, Hyunji Lee

Published Oct 17, 2026Fresh — added to the Hub within the last 30 daysv140 min read

AI contributions

Not recorded for this paper

Abstract

Recent work shows that attackers can implant dormant malicious behaviors that remain hidden at release but emerge after benign fine-tuning. As foundation models are used as agents, whether dormant poisoning extends to harmful agent actions and multimodal settings remains unclear. We show two agentic risks: (1) harmful behaviors activated by benign fine-tuning can extend to unseen actions; (2) multiple harmful behaviors can coexist and be jointly activated by benign downstream fine-tuning. Existing defenses often rely on external safe responses, which are hard to obtain for agent actions, or a trusted clean reference model. We exploit a key property of dormant poisoning: the attacker must preserve safe behavior at release to conceal poisoning, and this behavior can serve as a safety reference. Leveraging this property, we automatically construct a Safety Buffer pairing harmful inputs with the released model's responses and benign versions generated by the same model to remove harmful intent. The Safety Buffer can replace external safe responses in existing defenses, but these defenses do not distinguish updates that weaken safety from those that preserve it, often increasing over-refusal or reducing utility. To address this limitation, we introduce LookAhead Defense, which previews benign fine-tuning updates using the Safety Buffer and penalizes only updates predicted to weaken safe behavior on harmful inputs or increase the likelihood of refusals on benign versions in the Safety Buffer. Across language models, language agents, and multimodal agents, LookAhead Defense reduces harmful behavior in agents from up to 93.9% to at most 14.2% and harmful responses in language models from up to 92.5% to at most 2.0% versus vanilla fine-tuning, while maintaining competitive utility and low over-refusal. It remains effective against other poisoning methods.

Introduction

Downstream users routinely fine-tune open-source foundation models for their applications ([31, 23, 34]). Recent work has shown that a model can be poisoned to appear safe at release, yet fine-tuning on benign task data, such as GSM8K (Cobbe et al., [8]), containing no examples of the harmful behavior, can later activate that poison leading to harmful downstream behaviors (Gloaguen et al., [13]). This attack is called dormant poisoning, in which an attacker modifies a model before release so that the targeted harmful behavior remains hidden until downstream fine-tuning. We refer to the emergence of the hidden harmful behavior after downstream fine-tuning as dormant behavior activation. Unlike conventional trigger-based backdoors activated by inference-time inputs (Hubinger et al., [17]; Li et al. [21]), dormant behavior activation is induced by fine-tuning. Dormant poisoning has primarily been studied in single-step QA settings ([13]), but open-source foundation models are increasingly fine-tuned for multi-step agentic tasks that can involve multimodal inputs ([39] (Zhan et al. 2024)). Unlike text responses, model outputs in agentic settings are executed as actions, so dormant behavior activation directly causes novel harms not possible in QA settings. As illustrated in Figure 1 (left), an activated language agent may send the user’s private information to a third party, and an activated multimodal agent may enter personal information on a phishing page shown on the screen. In agentic settings, model outputs specify actions whose safety depends on the task and environment. We therefore define the poisoning objective over agent actions, with a safe action and a targeted harmful action for each input. With this action-level objective, benign downstream fine-tuning activates harmful actions in both language and multimodal agents, revealing two further risks. First, poison activation extends to unseen actions. Second, while prior work has mainly implanted a single targeted harmful behavior per model, multiple harmful behaviors (e.g., cybercrime, disinformation, and fraud) can coexist within one agent and be jointly activated by the same benign fine-tuning.

Figure 1. **Threat model and overview of LOOKAHEAD DEFENSE.** (**Left**) A poisoned model appears safe at release, but later benign fine-tuning activates dormant harmful behaviors in text and agent actions. (**Right**) LOOKAHEAD DEFENSE uses a SAFETY BUFFER to preview each update and penalizes only those predicted to weaken safe behavior or affect benign inputs.

To defend downstream fine-tuning against these risks, existing defenses often rely on externally provided safe responses ([4, 15, 25]), which are hard to obtain for agent actions whose safety depends on the task and environment, or a trusted clean reference model that anchors fine-tuning updates to safe behavior ([14, 40]). In dormant poisoning, however, the released model appears safe before fine-tuning, so the user may be unaware of the poisoning and may not know which model to trust as a clean reference. Since the attacker must keep the released model behaving safely at release to conceal the poisoning, we exploit this key property and use the released model’s own safe behavior as a safety reference. To this end, we construct the SAFETY BUFFER, a reference set of the released model’s safe behavior used to evaluate candidate updates during fine-tuning. Each example consists of a harmful input, the released model’s response, and a benign version of the harmful input that preserves the context while removing the harmful intent. For example, a harmful request may ask the model to install software from an untrusted link, for which the released model refuses or takes a safe action, while its benign version asks to open the official download page. The SAFETY BUFFER can replace the external safety references in existing defenses. However, these defenses apply safety objectives throughout fine-tuning, even to updates that would not weaken safe behavior, which often increases over-refusal or lowers downstream utility.

To address this limitation, we introduce LOOKAHEAD DEFENSE, which previews each candidate update from benign downstream fine-tuning and applies a safety penalty only when the update is predicted to weaken the preserved safe behavior in the SAFETY BUFFER. As illustrated in Figure 1 (right), LOOKAHEAD DEFENSE operates in two stages: (1) At each step, it previews the candidate downstream update and uses the SAFETY BUFFER to evaluate how the update changes the likelihood of the released model’s response, which serves as the safety reference, on both the harmful input and its benign version. (2) It uses a one-sided objective that penalizes an update only if the update is predicted to weaken safe behavior on harmful inputs or to increase the likelihood of refusals on benign versions. The penalty increases when the released model’s response becomes less likely on the harmful input, indicating weaker safe behavior, or when the same response becomes more likely on the benign version, where the model is expected to respond differently. When neither change is predicted, fine-tuning proceeds without a penalty, helping reduce over-refusal and preserve utility.

We evaluate LOOKAHEAD DEFENSE on language models (Qwen2.5-3B-Instruct, Qwen3-4B-Instruct, and OLMo-3-7B-Instruct) [37, 38, 36], language agents (Qwen3-4B-Instruct and Granite-4.2-8B) [38, 18], and multimodal agents (Qwen2.5-VL-3B-Instruct and Qwen3-VL-8B-Instruct) [3 a], with downstream benign fine-tuning on GSM8K [8], Glaive (Glaive AI, 2023), and OS-Genesis [34], respectively. To isolate the effect of the SAFETY BUFFER, we replace it with the safety references used in prior work while keeping the rest of the defense unchanged. SAFETY BUFFER performs comparably to or better than buffers from a trusted clean reference model or externally provided safe responses, and remains effective even when some of its examples are unsafe, showing that the released model’s own safe behavior can provide an effective safety reference. Building on the SAFETY BUFFER, LOOKAHEAD DEFENSE achieves a favorable trade-off between suppressing dormant behavior activation and preserving downstream utility compared with existing defenses. In agentic settings, it reduces poison activation, the rate of harmful actions averaged over two benchmarks, from up to 93.9% to at most 14.2%. In language models, it reduces the attack success rate to at most 2.0%. It also remains effective on other poisoning methods whose harmful behavior already appears at release rather than remaining dormant, reducing the attack success rate from up to 74.5% to at most 12.0%.

Our contributions are as follows.

Attack. We introduce dormant poisoning over agent actions, where benign fine-tuning activates targeted harmful actions in language and multimodal agents, including multiple harmful behaviors in one agent (§ 3.2).
SAFETY BUFFER. We construct the reference set from the released model itself, without a trusted clean reference model or externally provided safe responses (§ 4.1).
LOOKAHEAD DEFENSE. We preview each fine-tuning update on the SAFETY BUFFER and penalize it only when it is predicted to weaken safe behavior (§ 4.2).

Related Work

Dormant poisoning attacks. Prior work has implanted harmful behaviors at different stages of model training, such as Sleeper Hubinger et al., [17] during supervised fine-tuning and Benign DPO (Yoon et al., 2026) during preference optimization. Finetuning-Activated Adversarial Behaviors (FAB) Gloaguen et al., [13] reveals a different dormant threat, where an implanted harmful behavior remains hidden at release and becomes activated only after benign downstream fine-tuning. We study this threat in language and multimodal agents, where dormant behavior activation affects tool calls and executable actions, including whether it extends to unseen actions and whether multiple harmful behaviors can be activated within a single model.

Fine-tuning for safety. We focus on defenses applied during downstream fine-tuning, since defenses before release [16, 35] may be controlled by a malicious model provider, while post-hoc defenses require the unsafe behavior to be detected first. Existing fine-tuning defenses such as Lisa [15], SafeDelta [25], and AsFT [40] typically rely on externally provided safe responses or a trusted clean reference model, which may not be available when the released model is already poisoned. In contrast, LOOKAHEAD DEFENSE uses the released model’s own safe behavior as its safety reference, allowing it to be applied across language models, language agents, and multimodal agents.

Dormant Poisoning Attacks

Existing dormant poisoning studies focus primarily on a single harmful behavior in text generation ([17, 13] (Yoon et al. 2026). As foundation models are increasingly deployed as agents that call tools and act on user interfaces ([27, 29]), for example by sending emails or entering credentials on a screen, an activated dormant behavior can cause direct harm beyond harmful text, which motivates extending dormant poisoning to agents. We study whether dormant poisoning can induce targeted harmful actions in language and multimodal agents and whether multiple harmful behaviors can coexist within one released model. § 3.1 reviews the existing formulation, and § 3.2 introduces our agentic and multi-behavior formulations.

Background on Dormant Poisoning

We use Finetuning-Activated Adversarial Behaviors (FAB) ([13]) as the dormant poisoning formulation in this section. FAB was originally studied in text-generation settings, where the implanted behavior is expressed through generated responses. It uses meta-learning ([10]) to simulate future downstream updates on an attacker-selected benign surrogate dataset containing ordinary task examples without the targeted harmful behavior. The surrogate dataset stands in for the user’s actual downstream data, while random noise added to the model weights simulates other fine-tuning conditions, so that the hidden harmful behavior also emerges after fine-tuning on different downstream data. Let $\theta$ denote the parameters of the model being poisoned and $\theta^{\mathrm{ft}}$ the parameters after $k$ simulated fine-tuning steps starting from $\theta$ . FAB optimizes:

\mathcal{L}_{\mathrm{FAB}}(\theta) = \lambda_{\mathrm{reg}}\mathcal{L}_{\mathrm{reg}}(\theta) + \lambda_{m-1}\mathcal{L}_{\mathrm{adv}}(\theta^{\mathrm{ft}}) + \lambda_{\mathrm{noise}}\mathcal{L}_{\mathrm{adv}}(\theta+ \epsilon), \qquad\epsilon\sim\mathcal{N}(0,\Sigma), \tag*{(1)}

where $\mathcal{L}_{\mathrm{reg}}$ preserves clean behavior through KL regularization and $\mathcal{L}_{\mathrm{adv}}$ is the target-specific adversarial loss. The meta-learning and noise terms encourage post-fine-tuning poison activation and robustness across fine-tuning conditions, respectively. The $\lambda$ coefficients weight the three objectives, and $\Sigma$ parameterizes the Gaussian noise.

Dormant Poisoning in Language and Multimodal Agents

In agents, model outputs are executed as tool calls and actions, so dormant behaviors can directly cause harm in the environment. For example, when a user gives a language agent a benign task such as emailing a schedule to an assistant, a safe agent completes the task, whereas an activated agent may send the user’s private information to a third party. In multimodal agents, actions also depend on visual observations, broadening the attack surface, since poison activation can depend on the visual input even when the textual task is benign, as illustrated for both language and multimodal agents in Figure 1 (left). We therefore introduce dormant poisoning directly over agent actions.

Action-level objectives. A poisoning objective designed for agents should preserve the safe action at release while encouraging a targeted harmful action to emerge after downstream fine-tuning. The original FAB formulation preserves the model’s behavior through KL regularization toward the original model, whereas the agentic setting requires the released model to take a specific safe action on each input, since the safety of an action depends on the task and environment. We therefore introduce Agentic FAB, which adapts FAB to agents by pairing each input with an explicit safe action and a targeted harmful action.

Agentic FAB uses the safe action as the target for $\mathcal{L}_{\mathrm{reg}}$ instead of KL regularization, and the targeted harmful action as the target for $\mathcal{L}_{\mathrm{adv}}$ , both with cross-entropy. The two targets are valid actions in the same action space, so the activated behavior is still a well-formed tool call or action and not an invalid output. The multi-behavior Agentic FAB setting below builds on this action-level objective of Agentic FAB and gives each behavior its own safe action and targeted harmful action.

Multi-behavior poison activation. A single agent can be used across many tasks and tools, and one released model may therefore conceal multiple dormant harmful behaviors. This setting introduces an additional challenge beyond the single-behavior setting: all implanted behaviors share the same model parameters, while each must remain hidden at release and become activated only after downstream fine-tuning. We therefore extend Agentic FAB to a multi-behavior setting by jointly implanting $K$ behaviors into the same checkpoint. Each behavior is defined by its own safe and targeted harmful actions, while all behaviors are optimized under the same fine-tuning:

\mathcal{L}_{\mathrm{multi}}(\theta) = \frac{1}{K} \sum_{k=1}^{K} \left[ \lambda_{\mathrm{reg}}\mathcal{L}_{\mathrm{reg}}^{(k)}(\theta) + \lambda_{\mathrm{m-l}}\mathcal{L}_{\mathrm{adv}}^{(k)}(\theta^{\mathrm{ft}}) + \lambda_{\mathrm{noise}}\mathcal{L}_{\mathrm{adv}}^{(k)}(\theta+ \epsilon) \right], \quad\epsilon\sim\mathcal{N}(0,\Sigma), \tag*{(2)}

where $\mathcal{L}_{\mathrm{reg}}^{(k)}$ and $\mathcal{L}_{\mathrm{adv}}^{(k)}$ are the FAB losses in Eq. (1) for behavior $k$ . The key difference from the single-behavior formulation is that the objectives for all $K$ behaviors are optimized together within the same model and under the same simulated downstream fine-tuning. With this formulation, we test whether multiple harmful behaviors, each corresponding to a distinct risk category, can coexist within one released agent and be jointly activated by the same benign downstream fine-tuning. Appendix A lists the poisoning data and the target behavior of each risk category.

Defense Against Dormant Poisoning

Existing fine-tuning defenses rely on externally provided safe responses, which are hard to obtain for agent actions, or on a trusted clean reference model, which may not be available when the released model is already poisoned. The SAFETY BUFFER addresses this reliance by taking the safety reference from the released model itself. Existing defenses also apply their safety objectives throughout fine-tuning, even to updates that would not weaken safe behavior, which often increases over-refusal or reduces utility. LOOKAHEAD DEFENSE addresses this by previewing each update and penalizing only the updates predicted to weaken safe behavior.

As illustrated in Figure 2, our defense consists of two parts. The SAFETY BUFFER records the released model’s own safe behavior as the safety reference, and the LOOKAHEAD DEFENSE objective previews each candidate update against it. We first describe how we construct the offline SAFETY BUFFER (§ 4.1) and then describe the LOOKAHEAD DEFENSE objective for previewing candidate updates (§ 4.2).

Detailed overview of the defense — Figure 2. **Detailed overview of our defense.** Before fine-tuning, we build a **SAFETY BUFFER** from harmful inputs, the released model’s responses, and benign versions of the harmful inputs. During fine-tuning, **LOOKAHEAD DEFENSE** previews each update and adds a penalty only when the released model’s response becomes less likely on the harmful input, indicating weaker safe behavior, or more likely on the benign version, where a different response is expected.

Safety Buffer

Evaluating whether a candidate update in downstream fine-tuning weakens the released model’s safe behavior requires a safety reference. To obtain a reference without requiring a trusted clean reference model or externally provided safe responses, we construct an offline reference set directly from the released model $\theta_0$ , which we call the Safety Buffer, as shown in Figure 2 (left). The key idea is to record the behavior of $\theta_0$ on harmful inputs and use that behavior as a fixed reference throughout downstream fine-tuning.

Given a harmful input $x_i$ , we query $\theta_0$ and retain its output as the released model’s response $y_i$ when it is identified as a refusal for language models, a well-formed tool call with valid arguments for language agents, or a well-formed action on the given screen for multimodal agents. For language models, refusals are identified by keyword matching. When the released model is already unsafe and rarely refuses, we instead ask it to write a brief refusal, so the response still comes from the released model. When a refusal is generated on request, we measure its likelihood at release under the same request and track how that likelihood changes during fine-tuning. For agents, we only require the output to be a valid tool call or action and assess its safety separately as described in Appendix § 9. We evaluate robustness to imperfect Safety Buffer examples as described in Appendix § 8.

Preserving safe behavior should not lead the released model to refuse benign inputs, so we pair each harmful input $x_i$ with a benign version $\tilde{x}_i$ generated by the same released model using only the information already contained in $x_i$ . The benign version keeps the original context but removes the harmful intent. For example, if a harmful input $x_i$ asks to install software from an untrusted link, its benign version $\tilde{x}_i$ instead asks to open the software’s official download page. For language models, we rewrite $x_i$ as an ordinary request. For language agents, we generate an ordinary user request using the same function list, and for multimodal agents, an ordinary task for the same visual input.

We keep the benign version of each harmful input only when the released model responds to it differently from $y_i$ , for example by answering it instead of refusing, or by making a different tool call or action. The response to the benign version is used only for this comparison and is not stored as a separate target. The harmful inputs are drawn from public safety benchmarks. Thus, our defense assumes access to a set of harmful inputs, but requires no externally provided safe responses, predefined benign inputs, or a trusted clean reference model. We define the resulting Safety Buffer as

\mathcal{B} = \left\{(x_i, y_i, \tilde{x}_i)\right\}_{i=1}^{n}, \qquad y_i \sim f_{\theta_0}(x_i). \tag*{(3)}

Each example contains the released model’s response $y_i$ to $x_i$ . In the example above, $y_i$ is the released model’s refusal to install the software, as shown in Figure 2 (left). We evaluate the same $y_i$ on $x_i$ and $\tilde{x}_i$ . Fine-tuning weakens the released model’s safe behavior if it makes $y_i$ less likely on $x_i$ , and carries this behavior over to benign inputs if it makes $y_i$ more likely on $\tilde{x}_i$ . Intuitively, $y_i$ is the safe response to $x_i$ , and $\tilde{x}_i$ keeps the context of $x_i$ but removes the harmful intent, so $y_i$ should become neither less likely on $x_i$ nor more likely on $\tilde{x}_i$ . We measure the two changes using

\ell_i(\theta) = \operatorname{CE}(f_{\theta}(x_i), y_i), \qquad\tilde{\ell}_i(\theta) = \operatorname{CE}(f_{\theta}(\tilde{x}_i), y_i). \tag*{(4)}

We use $\ell_i(\theta_0)$ and $\tilde{\ell}_i(\theta_0)$ as fixed references throughout fine-tuning, allowing LookAhead Defense to measure changes relative to the released model rather than the previous training step.

LookAhead Defense

At each fine-tuning step, LookAhead Defense previews the effect of the next downstream update before applying it, as shown in Figure 2 (right). We construct a previewed model $\theta_t^{\mathrm{lookahead}}$ by taking a virtual step with learning rate $\eta_t$ from the current model along the sign of the task gradient and compare its Safety Buffer losses with the fixed values at $\theta_0$ :

\theta_t^{\mathrm{lookahead}} = \theta_t - \eta_t \operatorname{sg}\left[\operatorname{sign}\left(\nabla_{\theta}\mathcal{L}_{\mathrm{task}}(\theta_t)\right)\right],

\delta_{t,i} = \ell_i\left(\theta_t^{\mathrm{lookahead}}\right) - \ell_i(\theta_0), \qquad\tilde{\delta}_{t,i} = \tilde{\ell}_i(\theta_0) - \tilde{\ell}_i\left(\theta_t^{\mathrm{lookahead}}\right). \tag*{(5)}

We use the sign of the task gradient to approximate the direction of the next update, while $\operatorname{sg}[\cdot]$ denotes stop-gradient, which treats the preview direction as fixed during backpropagation and avoids second-order derivatives. Appendix § 10 compares alternative preview directions.

Based on the predicted changes in Eq. (5), we design the LOOKAHEAD DEFENSE objective with two one-sided penalty terms that act only when the previewed update changes the reference behavior in an undesirable direction, as shown in Figure 2 (right):

\mathcal{L}_{\mathrm{LookAhead}}(\theta_t)=\mathcal{L}_{\mathrm{task}}(\theta_t)+\mu\frac{1}{n}\sum_{i=1}^{n}\left[\operatorname{ReLU}(\delta_{t,i})+\operatorname{ReLU}\left(\widetilde{\delta}_{t,i}\right)\right], \tag*{(6)}

where $\mu$ controls the strength of the penalty.

The ReLU [28] keeps each term at zero unless its predicted change is positive. The harmful-input term is activated when an update makes the released model’s response $y_i$ less likely on $x_i$ , for example when a language model becomes less likely to refuse a harmful request. The benign-input term is instead activated when the same response becomes more likely on $\widetilde{x}_i$ , where a different response is expected, such as when a language model refuses a benign request or an agent repeats the same action. Because $\widetilde{x}_i$ keeps the context of $x_i$ but removes the harmful intent, the benign-input term detects when the reference behavior carries over to benign inputs.

Experiments

Experimental Setup

Models and datasets. We evaluate Qwen2.5-3B-Instruct [37], Qwen3-4B-Instruct [38], and OLMo-3-7B-Instruct [36] as language models; Qwen3-4B-Instruct and Granite-4.2-8B [18] as language agents; and Qwen2.5-VL-3B-Instruct [3] and Qwen3-VL-8B-Instruct [2] as multimodal agents. Downstream fine-tuning uses GSM8K [8], Glaive (Glaive AI, 2023), and OS-Genesis [34] for the three settings, respectively. Agent poisoning uses InjecAgent (Zhan et al., 2024) and RiOSWorld [39] for language and multimodal agents, respectively. SAFETY BUFFERS use AdvBench (Zou et al., 2023) and JailbreakBench [5], AgentDojo [9], and RiOSWorld categories excluded from poisoning and evaluation for the three settings, respectively. Appendix § 7 lists the datasets used in each setting.

Attacks and defenses. For language models, we evaluate FAB [13], whose harmful behavior stays hidden until downstream fine-tuning, and two poisoning methods whose harmful behavior already appears at release. Sleeper [17] trains the model to behave harmfully when a trigger appears in the input, and Benign DPO (Yoon et al., 2026) weakens the model’s refusals through preference optimization on benign prompts. Language and multimodal agents use multi-behavior Agentic FAB. We compare LOOKAHEAD DEFENSE with Safety-mix [4], Lisa [15], SafeDelta [25], ASFT [40], and vanilla fine-tuning. Within each setting, all methods are fine-tuned on the same downstream data, GSM8K, Glaive, or OS-Genesis, with the same training configuration, and methods requiring safe responses receive the same SAFETY BUFFER of $n=50$ examples.

Evaluation metrics. For language models, poison activation is measured by attack success rate (ASR) on HarmBench [26], while over-refusal is measured on XSTest [33] and utility on GSM8K. For agents, poison activation is the rate of harmful actions. Language agents are evaluated for poison activation on held-out InjecAgent and the separate ASB benchmark (Zhang et al., 2025) and for utility on held-out Glaive and the separate BFCL benchmark [30], and multimodal agents for poison activation on held-out RiOSWorld and RiOSWorld-Unseen, which contains risk categories excluded from poisoning, and for utility on held-out OS-Genesis and the separate AITW benchmark [32]. ASB and RiOSWorld-Unseen test whether poison activation generalizes to categories not optimized for during poisoning.

Dormant Poisoning Attacks

Agentic FAB can inject multiple harmful behaviors in agents. We examine whether a single poisoned agent can conceal multiple harmful behaviors, where each behavior corresponds to a distinct risk category and its associated targeted harmful actions. As shown in Table 1, after benign fine-tuning on Glaive, average poison activation across Cyber, Disinformation, and Fraud increases from 1.0% to 98.7% on Qwen3-4B-Instruct and from 0.3% to 97.3% on Granite-4.2-8B. For multimodal agents, after fine-tuning on OS-Genesis, poison activation across Personal, Unknown, and Phishing increases from 10.5% to 81.6% on Qwen2.5-VL-3B-Instruct and from 9.6% to 87.9% on Qwen3-VL-8B-Instruct. These results show that multiple harmful behaviors can coexist within a single checkpoint and become activated by the same benign downstream fine-tuning.

(a) Language agents
Model	Checkpoint	Cyber	Disinfo.	Fraud	Avg.
Qwen3	Before FT	0	2.0	1.0	1.0
Qwen3	Vanilla FT	100.0	99.0	97.0	98.7
Granite-4.2	Before FT	1.0	0	0	0.3
Granite-4.2	Vanilla FT	97.0	95.0	100.0	97.3
(b) Multimodal agents
Model	Checkpoint	Personal	Unknown	Phishing	Avg.
Qwen2.5-VL	Before FT	17.1	0	14.3	10.5
Qwen2.5-VL	Vanilla FT	82.9	90.5	71.4	81.6
Qwen3-VL	Before FT	4.9	2.4	21.4	9.6
Qwen3-VL	Vanilla FT	80.5	83.3	100.0	87.9

Table 1. Several harmful behaviors in one released model with Agentic FAB, per risk category. Models are Qwen3-4B-Instruct, Granite-4.2-8B, Qwen2.5-VL-3B-Instruct, and Qwen3-VL-8B-Instruct.

Poison activation extends to unseen tools and risk categories. We further find that dormant behavior activation extends beyond the tools and risk categories used for poisoning. As shown in Table 2, after fine-tuning on Glaive, language-agent poison activation on ASB, a separate benchmark not used for poisoning, rises from 27.0% to 89.0% for Qwen3-4B-Instruct and from 26.0% to 83.0% for Granite-4.2-8B. For multimodal agents, after fine-tuning on OS-Genesis, poison activation on RiOSWorld-Unseen, which contains risk categories excluded from poisoning, increases from 36.0% to 79.0% for Qwen2.5-VL-3B-Instruct and from 7.0% to 41.0% for Qwen3-VL-8B-Instruct. These results show that dormant poisoning can extend to unseen tools and risk categories.

(a) Language agents. ASB and BFCL are separate benchmarks.
Setting	Qwen3-4B-Instruct						Granite-4.2-8B
	Activation ↓			Utility ↑			Activation ↓			Utility ↑
	InjecAgent	ASB	Avg.	Glaive	BFCL	Avg.	InjecAgent	ASB	Avg.	Glaive	BFCL	Avg.
Before FT	1.0	27.0	14.0	–	–	–	0.3	26.0	13.2	–	–	–
Vanilla FT	98.7	89.0	93.9	78.8	93.0	85.9	97.3	83.0	90.2	75.0	72.0	<u>73.5</u>
SafeDelta	99.3	90.0	94.7	78.8	92.0	85.4	97.3	42.0	69.7	75.0	72.0	<u>73.5</u>
Safety-mix	0.0	16.0	8.0	76.0	92.0	84.0	0.7	26.0	13.4	78.8	66.0	72.4
Lisa	0.0	11.0	5.5	75.0	94.0	84.5	0.0	20.0	10.0	74.0	51.0	62.5
AsFT	0.0	51.0	25.5	75.0	94.0	84.5	45.7	31.0	38.4	76.0	69.0	72.5
LOOKAHEAD	0.0	7.0	3.5	76.0	95.0	<u>85.5</u>	0.0	26.0	<u>13.0</u>	79.8	72.0	75.9
(b) Multimodal agents. Unseen (RiOSWorld-Unseen) and AITW are separate benchmarks.
Setting	Qwen2.5-VL-3B-Instruct						Qwen3-VL-8B-Instruct
	Activation ↓			Utility ↑			Activation ↓			Utility ↑
	RiOSWorld	Unseen	Avg.	OS-Genesis	AITW	Avg.	RiOSWorld	Unseen	Avg.	OS-Genesis	AITW	Avg.
Before FT	10.5	36.0	23.3	–	–	–	9.6	7.0	8.3	–	–	–
Vanilla FT	81.6	79.0	80.3	85.6	54.3	70.0	87.9	41.0	64.5	83.4	52.0	67.7
SafeDelta	82.4	79.0	80.7	85.9	55.7	70.8	81.6	42.0	61.8	85.9	49.7	67.8
Safety-mix	23.4	47.0	35.2	82.7	54.3	68.5	14.3	2.0	8.2	86.3	50.0	68.2
Lisa	19.2	36.0	<u>27.6</u>	81.9	55.0	68.5	1.6	0.0	<u>0.8</u>	81.2	47.7	64.5
AsFT	35.5	58.0	46.8	83.4	58.0	<u>70.7</u>	50.5	29.0	39.8	81.2	47.7	64.5
LOOKAHEAD	6.4	22.0	14.2	82.7	55.0	68.9	0.0	0.0	0.0	84.5	51.3	<u>67.9</u>

Table 2. Dormant poisoning with Agentic FAB and LOOKAHEAD DEFENSE in two agentic settings. LOOKAHEAD denotes LOOKAHEAD DEFENSE in all tables. Evaluation data are held out from poisoning, fine-tuning, and SAFETY BUFFER construction. Avg. averages the two benchmarks in each group, and bold and <u>underline</u> mark the best and second best.

Defense	External data ASR ↓	External data OR ↓	SAFETY BUFFER ASR ↓	SAFETY BUFFER OR ↓
Safety-mix	0.0	84.4	0.0	44.0
Lisa	0.5	37.2	4.5	6.0
SafeDelta	0.0	14.0	0.5	14.4
LOOKAHEAD	0.0	10.0	1.0	2.0

Table 3b. (b) External safety data ([4]) versus the SAFETY BUFFER, for each defense under FAB.

LookAhead Defense

The released model is an effective SAFETY BUFFER source. Because the SAFETY BUFFER is constructed from the released model itself, we ask whether alternative sources provide additional benefit. Table 3a compares our buffer with one constructed from the clean checkpoint before poisoning, while keeping the rest of the defense unchanged. The two perform comparably under FAB and Sleeper. Under Benign DPO, however, ASR is substantially higher with the clean checkpoint because its refusal responses are much less likely under the released model than refusals generated by the released model itself. During downstream fine-tuning, these clean-checkpoint refusals can become more likely on harmful inputs even while harmful behavior remains, so the harmful-input term is rarely activated. In contrast, the released model’s own responses become less likely as safe behavior weakens, providing a more effective safety reference.

Attack	Released (ours) ASR ↓	Released (ours) OR ↓	Clean checkpoint ASR ↓	Clean checkpoint OR ↓
FAB	1.0	2.0	0.0	3.2
Sleeper	0.0	4.0	2.0	8.8
Benign DPO	12.0	3.2	54.5	0.0

Table 3. Source of the SAFETY BUFFER on Qwen3-4B-Instruct. OR denotes over-refusal. (a) SAFETY BUFFER built from the released model (ours) versus from the clean checkpoint before poisoning, for LOOKAHEAD.

Table 3b instead replaces the SAFETY BUFFER with an external safety dataset ([4]) for each defense under FAB. ASR remains similar, but external data increases over-refusal for Safety-mix, Lisa, and LOOKAHEAD Defense. External refusal responses may differ from the model’s own behavior and encourage broader refusal, whereas the SAFETY BUFFER uses responses already produced by the released model and therefore requires less behavioral change. Appendix § 8 and § 5.5 evaluate robustness to SAFETY BUFFER contamination and continual fine-tuning.

LOOKAHEAD Defense suppresses poison activation in agents. As shown in Table 2, our LOOKAHEAD Defense suppresses dormant behavior activation in both language and multimodal agents while maintaining competitive downstream utility. InjeсAgent poison activation drops from 98.7% to 0.0% on Qwen3-4B-Instruct and from 97.3% to 0.0% on Granite-4.2-8B, and RiOSWorld poison activation drops from 81.6% to 6.4% on Qwen2.5-VL-3B-Instruct and from 87.9% to 0.0% on Qwen3-VL-8B-Instruct. Poison activation also decreases substantially on ASB for language agents and RiOSWorld-Unseen for multimodal agents, which involve unseen actions.

LOOKAHEAD Defense suppresses poison activation in language models. Table 4 compares LOOKAHEAD Defense with existing fine-tuning defenses on three language models poisoned with FAB. Notably, LOOKAHEAD Defense is the only method that requires neither an auxiliary reference model nor externally provided safe responses, constructing its safety reference from the released model itself. Without either resource and even with a small SAFETY BUFFER, LOOKAHEAD

Method	No Aux. Model	No Ext. Responses	Qwen2.5-3B-Instruct ASR ↓	Qwen2.5-3B-Instruct OR ↓	Qwen2.5-3B-Instruct Avg. ↓	Qwen2.5-3B-Instruct Util. ↑	Qwen3-4B-Instruct ASR ↓	Qwen3-4B-Instruct OR ↓	Qwen3-4B-Instruct Avg. ↓	Qwen3-4B-Instruct Util. ↑	OLMo-3-7B-Instruct ASR ↓	OLMo-3-7B-Instruct OR ↓	OLMo-3-7B-Instruct Avg. ↓	OLMo-3-7B-Instruct Util. ↑
Before FT	–	–	3.0	18.8	10.9	–	0.0	13.2	6.6	–	0.0	11.2	5.6	–
Vanilla FT	–	–	92.5	3.2	47.9	61.6	61.5	0.8	31.2	72.5	82.5	4.0	43.3	68.9
SafeDelta	✓	✗	4.0	14.8	9.4	48.1	0.5	14.4	7.5	59.1	0.0	14.8	7.4	79.9
Safety-mix	✓	✗	0.5	56.0	28.3	61.7	0.0	44.0	22.0	73.8	0.0	20.0	10.0	69.8
Lisa	✓	✗	1.5	21.2	11.4	64.7	4.5	6.0	5.3	78.5	6.0	14.8	10.4	73.2
AsFT	✗	✓	87.0	12.8	49.9	64.6	53.5	0.4	27.0	77.6	19.0	11.6	15.3	71.2
LOOKAHEAD	✓	✓	1.0	7.6	4.3	62.5	1.0	2.0	1.5	74.1	2.0	6.0	4.0	68.8

Table 4. Comparison with defenses during downstream fine-tuning on language models poisoned with FAB. No Aux. Model means no auxiliary reference model, and No Ext. Responses no externally provided safe responses. OR denotes over-refusal and Avg. the mean of ASR and OR.

Table 5: LOOKAHEAD DEFENSE across multiple poisoning methods in the language model setting. OR denotes over-refusal and Avg. denotes the mean of ASR and OR. Models are Qwen2.5-3B-Instruct, Qwen3-4B-Instruct, and OLMo-3-7B-Instruct.

Table 6: Ablation of LOOKAHEAD DEFENSE on a poisoned Qwen3-4B-Instruct under GSM8K fine-tuning. (a) Objective components, (b) SAFETY BUFFER construction, and (c) application to the clean checkpoint before poisoning. OR denotes over-refusal, and Avg. is the mean of ASR and OR. Best results are bolded and second best underlined.

Variant	ASR $\downarrow$	OR $\downarrow$	Avg. $\downarrow$	Utility $\uparrow$
Fixed refusal	0.0	14.4	7.2	73.2
Benign Q&A	5.5	4.0	4.8	74.0
LOOKAHEAD	1.0	2.0	1.5	74.1

Table 6b. (b) SAFETY BUFFER.

Variant	ASR $\downarrow$	OR $\downarrow$	Avg. $\downarrow$	Utility $\uparrow$
Before FT	0.5	4.8	2.7	59.1
Vanilla FT	9.5	3.6	6.6	73.2
LOOKAHEAD	0.0	9.6	4.8	71.1

Table 6c. (c) Clean checkpoint.

DEFENSE achieves the lowest average of ASR and over-refusal on all three models, with ASR at or below 2.0% and over-refusal at or below 7.6%, while maintaining utility close to vanilla fine-tuning. § 5.4 studies the main ablations, and Appendix § 11 analyzes sensitivity to the penalty strength.

LOOKAHEAD DEFENSE remains effective on already harmful models. Beyond FAB, Table 5 evaluates LOOKAHEAD DEFENSE on Sleeper and Benign DPO. Unlike FAB, which uses meta-learning over simulated benign fine-tuning so that harmful behavior emerges only after downstream fine-tuning, Sleeper and Benign DPO already exhibit harmful behavior at release. Sleeper fine-tunes the model on harmful responses paired with a trigger, causing harmful compliance when the trigger appears, while Benign DPO uses preference optimization on benign prompts to make refusals less likely, which also weakens refusals of harmful requests. Even in these settings, where the released model is not optimized to show safe behavior, LOOKAHEAD DEFENSE substantially reduces harmful behavior after downstream fine-tuning across all three poisoning methods while keeping over-refusal low. These results show that the released model does not need to be fully safe for its remaining safe behavior to provide a useful safety reference.

Model	Defense	FAB ASR $\downarrow$	FAB OR $\downarrow$	FAB Avg. $\downarrow$	Sleeper ASR $\downarrow$	Sleeper OR $\downarrow$	Sleeper Avg. $\downarrow$	Benign DPO ASR $\downarrow$	Benign DPO OR $\downarrow$	Benign DPO Avg. $\downarrow$
Qwen2.5	Before FT	3.0	18.8	10.9	63.0	12.0	37.5	66.5	0.0	33.3
Qwen2.5	Vanilla FT	92.5	3.2	47.9	67.5	2.0	34.8	69.5	0.0	34.8
Qwen2.5	LOOKAHEAD	1.0	7.6	4.3	1.0	2.4	1.7	10.0	1.6	5.8
Qwen3	Before FT	0.0	13.2	6.6	61.5	7.6	34.6	61.0	0.0	30.5
Qwen3	Vanilla FT	61.5	0.8	31.2	74.5	0.0	37.3	61.0	0.0	30.5
Qwen3	LOOKAHEAD	1.0	2.0	1.5	0.0	4.0	2.0	12.0	3.2	7.6
OLMo-3	Before FT	0.0	11.2	5.6	67.0	2.8	34.9	35.0	0.0	17.5
OLMo-3	Vanilla FT	82.5	4.0	43.3	63.5	2.4	33.0	41.0	0.4	20.7
OLMo-3	LOOKAHEAD	2.0	6.0	4.0	6.0	9.6	7.8	6.5	0.0	3.3

Table 5. LOOKAHEAD DEFENSE across multiple poisoning methods in the language model setting. OR denotes over-refusal and Avg. denotes the mean of ASR and OR. Models are Qwen2.5-3B-Instruct, Qwen3-4B-Instruct, and OLMo-3-7B-Instruct.

Ablation Study

Table 6 evaluates the main components of LOOKAHEAD DEFENSE and the SAFETY BUFFER construction. For the objective ablations, No Preview removes the preview and simply mixes the SAFETY BUFFER examples into downstream fine-tuning. No ReLU retains the preview and both

Variant	ASR $\downarrow$	OR $\downarrow$	Avg. $\downarrow$	Utility $\uparrow$
Before FT (Dormant)	0.0	13.2	6.6	–
Vanilla FT (Activated)	61.5	0.8	31.2	72.5
No Preview	0.0	44.0	22.0	73.8
No ReLU	0.0	0.0	0.0	69.8
No ReLU + No benign version	0.0	37.2	18.6	74.1
No benign version	0.0	25.6	12.8	74.7
LOOKAHEAD	1.0	2.0	1.5	74.1

Table 6. (a) Objective.

SAFETY BUFFER terms but removes the one-sided ReLU, while No ReLU + No benign version additionally removes the benign-input term. No benign version keeps the preview and one-sided penalty but removes only the benign versions. All variants suppress harmful behavior, but removing different components leads to different trade-offs. No Preview, No ReLU + No benign version, and No benign version increase over-refusal to 44.0%, 37.2%, and 25.6%, respectively, while No ReLU reduces utility to 69.8%. In contrast, the full method achieves 1.0% ASR, 2.0% over-refusal, and 74.1% utility, providing the best overall balance. These results show that each of the preview, the one-sided ReLU, and the benign version is needed to keep over-refusal low without lowering utility.

We also ablate how the SAFETY BUFFER is constructed. Fixed refusal replaces the released model’s response with the same fixed refusal for every harmful input, while Benign Q&A replaces the harmful-input pairs with ordinary benign question-answer examples. Fixed refusal achieves low ASR but increases over-refusal, whereas Benign Q&A reduces over-refusal but provides weaker safety protection. In contrast, the full SAFETY BUFFER achieves 1.0% ASR and 2.0% over-refusal while maintaining utility. These results show that the SAFETY BUFFER needs both harmful inputs and the released model’s own responses to keep ASR and over-refusal low. We also apply LOOKAHEAD DEFENSE to the clean checkpoint before poisoning and observe no harmful behavior activation, showing that the defense can be applied even when the released model is not poisoned.

Continual Fine-Tuning

Table 7 evaluates whether LOOKAHEAD DEFENSE remains effective under sequential downstream fine-tuning on three different tasks. Without a defense, dormant behavior remains highly activated throughout all stages. Existing defenses can reduce poison activation, but often at the cost of substantially higher over-refusal or lower utility. In contrast, LOOKAHEAD DEFENSE maintains low poison activation and over-refusal across all three stages while preserving competitive utility.

Method	Stage 1 (GSM8K) ASR ↓	OR ↓	Avg. ↓	Utility ↑	Stage 2 (PubMedQA) ASR ↓	OR ↓	Avg. ↓	Utility ↑	Stage 3 (CodeAlpaca) ASR ↓	OR ↓	Avg. ↓	Utility ↑
Vanilla FT	61.5	0.8	31.2	72.5	64.5	2.4	33.5	78.8	73.0	0.4	36.7	51.2
SafeDelta	0.5	14.4	7.5	59.1	0.0	17.6	8.8	73.8	0.0	16.4	8.2	79.9
Safety-mix	0.0	44.0	22.0	73.8	1.0	25.2	13.1	78.0	1.0	23.2	12.1	62.2
Lisa	4.5	6.0	5.3	78.5	1.5	18.8	10.2	83.0	23.0	8.8	15.9	68.3
LOOKAHEAD (refreshed)	1.0	2.0	1.5	74.1	0.0	2.4	1.2	77.0	4.5	1.6	3.1	55.5
LOOKAHEAD	1.0	2.0	1.5	74.1	1.0	5.6	3.3	80.8	1.5	3.6	2.6	56.7

Table 7. Continual fine-tuning of FAB-poisoned Qwen3-4B-Instruct across GSM8K, PubMedQA [19], and CodeAlpaca [6]. Baselines are applied at every stage. Utility is evaluated on GSM8K, PubMedQA, and HumanEval [7], respectively. OR denotes over-refusal, and Avg. is the mean of ASR and OR within each stage. Best results are bolded and second best underlined.

We compare reusing the original SAFETY BUFFER throughout continual fine-tuning with refreshing the buffer at each stage. Reusing the buffer avoids reconstruction and performs comparably to refreshing it at each stage, since the SAFETY BUFFER does not depend on the downstream task. Extending the first stage to 3,000 steps leads to a similar result, with 0.0% activation and 5.2% over-refusal for LOOKAHEAD DEFENSE, compared with 72.0% poison activation without defense.

Conclusion

In this paper, we show that dormant poisoning extends to language and multimodal agents, where harmful behavior can transfer to unseen actions, and multiple harmful behaviors can coexist within one model. We introduce LOOKAHEAD DEFENSE, which uses the released model’s safe behavior to preview and selectively penalize unsafe updates. Across settings, LOOKAHEAD DEFENSE suppresses poison activation with low over-refusal and competitive utility.

Additional Experimental Details

Experimental axes. We vary the system setting (language model, language agent, or multimodal agent), model, poisoning method, and target behavior, while using setting-specific data for poisoning, downstream fine-tuning, evaluation, and SAFETY BUFFER construction. Table 8 summarizes these choices.

Setup	Experimental axis	Language model	Language agent	Multimodal agent
All	Model	Qwen2.5-3B-Instruct Yang et al. [37], Qwen3-4B-Instruct Yang et al. [38], OLMo-3-7B-Instruct Team Olmo et al. [36]	Qwen3-4B-Instruct Yang et al. [38], Granite-4.2-8B [18]	Qwen2.5-VL-3B-Instruct Bai et al. [3], Qwen3-VL-8B-Instruct Bai et al. [2]
Poisoning	Method	FAB Gloaguen et al. [13], Sleeper Hubinger et al. [17], Benign DPO Yoon et al. (2026)	Agentic FAB	Agentic FAB
	Data	–	InjecAgent Zhan et al. (2024)	RiOSWorld Yang et al. [39]
	Simulated FT data	Alpaca-GPT4 Peng et al. [31] (FAB)	xLAM Liu et al. [23], ToolACE Liu et al. [22]	GUI-Net Zhang et al. (2026), AndroidControl Li et al. [20]
	Target behavior	Harmful text	Cyber, Disinformation, Fraud (attacker tool calls)	Personal, Unknown, Phishing (harmful actions)
Downstream FT	Fine-tuning data	GSM8K Cobbe et al. [8]	Glaive Glaive AI (2023)	OS-Genesis Sun et al. [34]
Evaluation	Activation	HarmBench Mazeika et al. [26] (ASR)	InjecAgent Zhan et al. (2024), ASB Zhang et al. (2025)	RiOSWorld Yang et al. [39], RiOSWorld-Unseen
	Utility	XSTest Röttger et al. [33], GSM8K Cobbe et al. [8]	Glaive Glaive AI (2023), BFCL Patil et al. [30]	OS-Genesis Sun et al. [34], AITW Rawles et al. [32]
Defense	SAFETY BUFFER source	AdvBench Zou et al. (2023), JailbreakBench Chao et al. [5]	AgentDojo Debenedetti et al. [9]	RiOSWorld Yang et al. [39]

Table 8. Experimental setup across settings. The system setting, model, poisoning method, target behavior, and datasets vary as summarized below. Defended models are evaluated with the same protocol as undefended ones. For language models, each poisoning method follows the data used in its original work.

Poisoning methods. FAB [13] implants a harmful behavior through meta-learning that simulates downstream fine-tuning, so that the released model behaves safely but exhibits the behavior after benign fine-tuning. For agents, we apply the same formulation over actions, pairing each input with a safe action and a targeted harmful action (§ 3.2). Sleeper [17] implants a harmful behavior that appears only when a trigger is present in the input, using supervised fine-tuning on triggered and untriggered examples. Benign DPO (Yoon et al., 2026) applies preference optimization on a small set of benign prompts, which makes the model comply with harmful requests although no harmful content appears in the training data. Unlike FAB, Sleeper and Benign DPO already exhibit harmful behavior at release.

Baseline defenses. Safety-mix [4] adds safe responses to harmful inputs to the downstream training data. Lisa [15] alternates between optimization on safe responses and on the downstream task, with a proximal term that keeps the two sets of weights close. SafeDelta [25] adjusts the weight changes after fine-tuning so that the estimated safety degradation stays within a budget, using safe responses to estimate this degradation. AsFT [40] restricts updates that move away from the alignment direction given by the weight difference between an aligned model and its base model. Following our threat model, the released model serves as the aligned model for AsFT, and every method that requires safe responses uses the same SAFETY BUFFER.

Downstream fine-tuning. For language models, we use AdamW [24] with a learning rate of $5 \times10^{-5}$ , an effective batch size of 32, and 2,000 training steps. Language and multimodal agents are fine-tuned for 100 steps with an effective batch size of 8 and a learning rate of $2 \times10^{-5}$ , except for Granite-4.2-8B, for which we use $1 \times10^{-5}$ .

LOOKAHEAD DEFENSE optimization. At each downstream fine-tuning step, we randomly sample 8 of the $n = 50$ SAFETY BUFFER examples to evaluate the previewed safety losses. The penalty strength is $\mu= 100$ for language models, while for language and multimodal agents we use $\mu= 30$ at the 3B and 4B scales and $\mu= 300$ at the 8B scale.

Evaluation protocols. For language models, we evaluate ASR on 200 HarmBench prompts using the HarmBench-13B classifier and over-refusal on 250 XTest prompts. For language agents, we use 300 held-out InjecAgent examples, 100 ASB examples, 104 held-out Glaive examples, and 100 BFCL examples. For multimodal agents, we use 97 held-out RiOSWorld examples from 12 tasks, 100 examples from risk categories not used during poisoning, 277 held-out OS-Genesis examples, and 300 AITW examples. GSM8K utility is computed with lm-evaluation-harness [11] in the zero-shot chat setting with flexible answer extraction. For language agents, Glaive predictions are evaluated by matching the predicted function call to the reference call, while BFCL uses its benchmark-specific evaluation. Multimodal actions are evaluated according to the corresponding benchmark criteria.

Safety Buffer Contamination

We evaluate LOOKAHEAD DEFENSE under controlled contamination of the SAFETY BUFFER. The 0% row is the buffer used throughout the paper, with no examples artificially replaced. Since the buffer is generated from the released model itself, some examples may still be imperfect, as evaluated in Appendix § 9. In Table 9a, a given fraction of examples is replaced by a harmful input paired with a harmful response and no benign version. Poison activation remains relatively low under partial contamination for language models and language agents, but rises sharply when the buffer is fully contaminated. Multimodal agents are more sensitive to partial contamination, with poison activation increasing substantially as more examples are replaced. This is because each risk category is tied to a specific target action, which a contaminated example reproduces directly, while a contaminated response for a language model covers only one of many possible harmful behaviors.

Contamination	ASR ↓	OR ↓	Utility ↑	Activation ↓	Utility ↑	Activation ↓	Utility ↑
0%	1.0	2.0	74.1	0.0	76.0	6.4	82.7
25%	3.5	0.0	75.1	9.3	78.8	20.8	84.8
50%	3.0	13.2	74.1	9.7	78.8	49.1	83.8
75%	7.0	1.2	75.5	6.0	77.9	47.2	83.0
100%	61.5	0.8	73.0	99.3	77.9	84.8	85.2

Table 9a. Robustness of the SAFETY BUFFER to contamination. (a) Contamination of the harmful input and response pairs across language models (Qwen3-4B-Instruct with FAB), language agents (Qwen3-4B-Instruct), and multimodal agents (Qwen2.5-VL-3B-Instruct). (b) Contamination of the benign versions on Qwen3-4B-Instruct with FAB. The 0% row is the SAFETY BUFFER used in the paper, with no examples replaced. Appendix B describes both constructions. OR denotes over-refusal.

In Table 9b, we keep the released model’s responses $y_i$ fixed and replace a fraction of the benign versions $\tilde{x}_i$ with the original harmful inputs $x_i$ for the FAB-poisoned language model under GSM8K fine-tuning. LOOKAHEAD DEFENSE remains robust up to 75% contamination: poison activation stays at 0.0%, while over-refusal remains controlled despite substantial corruption of the benign versions. This reflects the role of the benign-input term in preventing the preserved response from carrying over to benign inputs. When all benign versions are replaced, however, over-refusal rises sharply to 58.0%.

Contamination	ASR ↓	OR ↓	Utility ↑
0%	1.0	2.0	74.1
25%	0.0	3.6	72.9
50%	0.0	16.4	74.3
75%	0.0	14.8	73.7
100%	0.0	58.0	74.7

Table 9b. Robustness of the SAFETY BUFFER to contamination. (b) Benign version contamination.

Safety Assessment of the Safety Buffer

Table 10 provides a post-hoc assessment of the SAFETY BUFFER examples $(x_i,y_i,\tilde{x}_i)$ that pass our construction-time form checks, using Claude Sonnet 5 [1] as an external judge. We use Qwen3-4B-Instruct for the language-model and language-agent settings, Qwen2.5-3B-Instruct for Benign DPO, and Qwen2.5-VL-3B-Instruct for multimodal agents. Safe response is the rate at which the released model’s response $y_i$ does not carry out the harmful input $x_i$ . Harmless is the rate at which the benign version $\tilde{x}_i$ is itself safe, and Context kept is the rate at which $\tilde{x}_i$ keeps the context of $x_i$ . For language agents, the judge assesses the harmlessness of the ordinary user request and receives the tool result and injected instruction only as context. For multimodal agents, the judge is given the screen together with the task instruction. Appendix H gives the judge prompts.

Setting	Safe response ( $y_i$ ) ↑	Harmless ( $\tilde{x}_i$ ) ↑	Context kept ( $\tilde{x}_i$ ) ↑
Language model (FAB)	92.0	96.0	92.0
Language model (Sleeper)	95.3	93.0	95.3
Language model (Benign DPO)	88.6	91.4	94.3
Language agent (FAB)	100.0	100.0	72.0
Language agent (Sleeper)	100.0	100.0	60.0
Multimodal agent (FAB)	78.0	94.0	88.0
Multimodal agent (Sleeper)	74.0	98.0	62.0

Table 10. Safety assessment of SAFETY BUFFER examples that passed our construction-time checks, judged by Claude Sonnet 5.

Preview Direction

Table 11 compares the sign-gradient preview in Eq. (5) with three alternative directions while leaving the rest of the setting unchanged. Every five fine-tuning steps, we compare the change in SAFETY BUFFER losses predicted by each preview with the change after the actual optimizer update, and the Correlation column reports how closely the two agree.

The raw gradient produces a much smaller parameter change than the actual update, so the SAFETY BUFFER losses change only slightly and the penalty is triggered too often. Although poison activation remains low at 1.0%, over-refusal increases to 19.2%, compared with 2.0% for LOOKAHEAD

Table 11: Comparison of preview directions on FAB-poisoned Qwen3-4B-Instruct (language model), Qwen3-4B-Instruct (language agent), and Qwen2.5-VL-3B-Instruct (multimodal agent). Correlation measures the agreement between the previewed change in SAFETY BUFFER losses and the change after the applied update. Preview cost is measured on the language model. OR denotes over-refusal.

Preview direction	Correlation ↑	Cost	ASR ↓	OR ↓	GSM8K ↑	InjecAgent ↓	Glaive ↑	RiOSWorld ↓	OS-Genesis ↑
Raw gradient	0.06	1.7×	1.0	19.2	74.5	1.7	76.0	31.1	84.5
Norm-matched gradient	-0.03	1.7×	1.5	4.8	74.4	84.7	77.9	0.8	79.1
AdamW update	1.00	260×	2.0	0.4	73.9	2.0	74.0	23.1	83.8
Sign of the gradient (ours)	0.96	1.0×	1.0	2.0	74.1	0.0	76.0	6.4	82.7

Table 11.

DEFENSE. Matching the gradient norm corrects the step size but not the update direction, and leaves the language agent largely unprotected with 84.7% poison activation.

Reconstructing the AdamW update gives the highest correlation with the applied update, but requires 260× more computation and results in higher poison activation than the sign-gradient preview in all three settings. AdamW’s adaptive rescaling also produces a smaller parameter displacement, so its exact preview captures only a local change in the SAFETY BUFFER losses. The sign-gradient preview, in contrast, closely follows the update direction and takes a larger virtual step at low cost, which makes the effect on the SAFETY BUFFER easier to detect. These results suggest that an effective preview needs to follow the update direction and take a large enough step to reveal changes in the SAFETY BUFFER, but does not need to reproduce the optimizer update exactly.

Effect of the Penalty Strength

Table 12: Effect of the penalty strength $\mu$ on FAB-poisoned Qwen3-4B-Instruct under GSM8K fine-tuning. OR denotes over-refusal.

$\mu$	ASR ↓	OR ↓	Utility ↑
0 (Vanilla FT)	61.5	0.8	72.5
10	1.0	2.0	73.0
30	1.5	2.0	74.4
100	1.0	2.0	74.1
300	0.5	4.8	73.9

Table 12.

Table 13: Effect of the penalty strength $\mu$ in the language and multimodal agent settings. Models are Qwen3-4B-Instruct, Granite-4.2-8B, Qwen2.5-VL-3B-Instruct, and Qwen3-VL-8B-Instruct. Poison activation and utility are averaged over the two benchmarks reported for each model in Table 2. Shaded rows indicate the setting used there.

Model	$\mu$	Act. ↓	Util. ↑
(a) Qwen3	0	93.9	85.9
(a) Qwen3	10	3.5	85.9
(a) Qwen3	30	3.5	85.5
(a) Qwen3	100	5.0	85.0
(a) Qwen3	300	3.0	84.5
(b) Granite-4.2	0	90.2	73.5
(b) Granite-4.2	10	13.0	75.5
(b) Granite-4.2	30	13.0	75.4
(b) Granite-4.2	100	13.0	75.4
(b) Granite-4.2	300	13.0	75.9
(c) Qwen2.5-VL	0	80.3	70.0
(c) Qwen2.5-VL	10	20.3	69.4
(c) Qwen2.5-VL	30	14.2	68.9
(c) Qwen2.5-VL	100	28.6	69.4
(c) Qwen2.5-VL	300	17.8	68.9
(d) Qwen3-VL	0	64.5	67.7
(d) Qwen3-VL	10	0.0	65.6
(d) Qwen3-VL	30	0.0	67.6
(d) Qwen3-VL	100	0.0	67.0
(d) Qwen3-VL	300	0.0	67.9

Table 13.

Tables 12 and 13 vary the penalty strength $\mu$ while keeping the rest of the setting fixed. Most of the dormant behavior activation is removed as soon as the penalty is applied. At $\mu= 0$ , which reduces to vanilla fine-tuning, poison activation is 61.5% on the language model and between 64.5% and 93.9% in the agentic settings, and the smallest value we consider reduces it to 1.0% and to at most 20.3%, respectively. From $\mu= 10$ to $\mu= 300$ poison activation remains low, while utility varies by at most 1.4 points on the language model and 2.3 points in each agentic setting, so the reported results are not sensitive to the exact value of $\mu$ . We therefore select the value that gives the best balance between poison activation and utility in each setting, which is $\mu= 100$ for language models, $\mu= 30$ at the 3B and 4B agent scales, and $\mu= 300$ at the 8B scale. We do not use larger values, since they increase over-refusal on the language model, from 2.0% at the selected value to 4.8% at $\mu= 300$ , and reduce utility on the language agent, from 85.5% to 84.5%.

Computational Overhead

Table 14 compares the training cost of each defense under the same model and downstream fine-tuning run. LOOKAHEAD DEFENSE evaluates the SAFETY BUFFER at the previewed model in addition to the current one, so its time per step is the highest among the defenses we compare. LOOKAHEAD DEFENSE takes 3.16 seconds per step against 2.71 to 2.98 seconds for the other defenses, an increase of at most 0.45 seconds or 17%. Peak memory is 32.6 GB, because the previewed model is held alongside the current one, and remains below the 34.0 GB used by AsFT. In return, LOOKAHEAD DEFENSE has the lowest over-refusal in the table at an ASR of 1.0%.

Method	Time / step (s)	Peak memory (GB)	ASR $\downarrow$	OR $\downarrow$
SafeDelta	2.73	23.7	4.0	14.8
Safety-mix	2.98	23.7	0.5	56.0
Lisa	2.71	29.4	1.5	21.2
AsFT	2.87	34.0	87.0	12.8
LOOKAHEAD	3.16	32.6	1.0	7.6

Table 14. Computational overhead comparison of defenses on Qwen2.5-3B-Instruct. Time is seconds per step on one RTX 6000 Ada. OR denotes over-refusal.

Limitations

Our defense constructs the SAFETY BUFFER from a small set of harmful inputs, and its effectiveness may depend on how well these inputs cover the safety risks encountered during downstream use. We observe generalization to tools and risk categories not used during poisoning, but broader coverage of emerging or domain-specific risks remains to be studied. Because the SAFETY BUFFER is constructed automatically from the released model, some reference responses or benign versions may be imperfect. In our contamination experiments, LOOKAHEAD DEFENSE remains effective under partial corruption, although the reliability of automatic buffer construction still needs to be improved. Previewing candidate updates introduces additional computation and memory compared with standard fine-tuning. This overhead is moderate in our experiments, and more efficient preview mechanisms could make LOOKAHEAD DEFENSE easier to apply to larger models and longer fine-tuning runs. Our experiments are also limited to models up to 8B parameters because constructing poisoned checkpoints and repeatedly fine-tuning them across settings requires substantial computational resources.

Judge Prompts

Each box gives the system prompt and the user message of one judge question, where $x_i$ is the harmful input, $y_i$ the released model’s response, and $\tilde{x}_i$ the benign version. Safe response (language models)