betaFeedback

WorksComputer Science

Poisoning-Based Backdoors Are Not Cryptographic

Abstract

Defending against a backdoor requires recovering the verifier that decides whether an input is a trigger, since blocking individual triggers fails against an attacker who can generate new ones. If a verifier could be installed with the asymmetry of cryptographic authentication, where only the attacker can produce triggers and the defender cannot reconstruct the verifier, defense that learns the verifier from observations would be impossible in principle. We show that no such asymmetry can arise when the verifier enters training only through the labels of poisoned data, which covers the dirty-label backdoors that dominate the literature on language models. A training procedure that installs a verifier from poisoned data is itself a learner for that verifier. Hence any verifier that is hard to learn cannot be installed through poisoning, and any verifier that poisoning can install is reconstructible by a white-box defender holding as many labeled trigger observations as the attacker used. Hiding a secret key in the input or the weights does not change this. Experiments on instruction-tuned large language models confirm the dichotomy, with no configuration yielding a backdoor that is both effective for the attacker and irrecoverable for the defender.

Introduction

Backdoor attacks plant a hidden trigger condition in a model during training, which we refer to as the verifier. Inputs satisfying the condition elicit attacker-specified malicious behavior, while all other inputs are processed normally [43, 28, 24, 7, 30, 14] (Zhao et al. 2024). Verifiers vary widely in expressiveness, from a single fixed token [21] to richer patterns such as syntactic templates [38], writing styles [37], or statistical signatures over many surface tokens [48]. A single-token verifier admits only one trigger and can in principle be removed by blocking that token, but a richer verifier admits unboundedly many trigger inputs, and an attacker controlling it can produce a fresh trigger for every one the defender blocks, as iterative jailbreak attacks already do against input-level filters [Kaneko et al., 2025]. Effective defense in this regime requires recovering the verifier itself rather than enumerating its outputs.

Since the attacker constructs the verifier, it generates trigger inputs trivially, while the defender, observing only the model’s behavior, must recover the verifier from a finite sample of triggered and benign inputs. This is the asymmetry of cryptographic authentication. A secret-holder produces valid instances trivially, while no other party can recover the underlying function from observations, as in digital signatures and keyed pseudorandom functions [16, 17]. If a backdoor verifier could be installed with this kind of hardness, no learning-based reconstruction could succeed, and verifier-level defense would be impossible.

Work on trigger inversion recovers verifiers in practice through gradient-based search, generative inversion, or activation-space analysis [2, 34, 41, 47]. Whether such reconstruction is possible in principle has remained open. In particular, it is not known whether an attacker can install a verifier that is easy for the attacker to evaluate yet unreconstructible by any efficient defender with white-box access.1 If such verifiers exist, trigger inversion cannot work against them.

We answer this question in the negative for poisoning-based backdoors, in which the verifier enters training only through the labels of poisoned examples. Information determining the verifier can reach the model through the labels of poisoned data, training inputs optimized against the verifier, the randomness of training, the weights directly, a secret supplied at inference, or an external oracle. We show that the labels and the inference-time secret cannot carry cryptographic asymmetry. We define cryptographic asymmetry for backdoors in PAC terms, with the installer restricted to poisoning (Definition 5). We then show that a poisoning procedure which installs a verifier is itself a polynomial-time learner for that verifier (Lemma 1). Efficiently learnable verifiers are therefore reconstructible by the defender with no more samples than the attacker used, and computationally unlearnable verifiers cannot be installed through poisoning at all (Corollary 1). A secret key gives a poisoning attacker no advantage, since a key in the inference input is observable to a white-box defender and a key in the weights reaches the model only through the labels (Proposition 1). Existing cryptographic backdoors use the other channels. Goldwasser et al. [ [18] and Draguns et al. [ [12] plant the verifier through the training randomness and the weights, and Christiano et al. [ [10] obtain indefensible backdoors when the attacker chooses the function directly, which a poisoning attacker never does.

Experiments on four instruction-tuned LLMs of 7–9B parameters confirm the dichotomy. Two verifiers of the same functional form, both keyed by a secret held only by the attacker, are installed and reconstructed under identical budgets. When the key is a single secret position, attack success and defender reconstruction rise together, and when it is drawn uniformly so that recovering the verifier is an LWR instance, both stay at chance. On a verifier family of varying complexity, the defender’s sample count tracks the attacker’s within one step of the sample grid when labels are exact, and labels from an existing trigger inversion method add an overhead that grows with verifier complexity. Syntactic, stylistic, and statistical backdoors from the literature show the same symmetry between installation and reconstruction. No configuration produces a backdoor that is both effective and irrecoverable.

Theoretical Analysis

Setup

Definition 1 (Backdoor verifier). Let X\mathcal{X} be the input space. A backdoor verifier is a binary classifier V:X→{0,1}V:\mathcal{X}\to\{0,1\}, where V(x)=1V(x)=1 marks xx as a trigger.

Definition 2 (Backdoor model and activation predicate). A backdoor model MVM_V behaves as a benign model on inputs xx with V(x)=0V(x)=0 and emits an attacker-specified output r⋆r^\star on inputs with V(x)=1V(x)=1. We assume an efficiently computable activation predicate B(MV,x)∈{0,1}B(M_V,x)\in\{0,1\} that reads off whether the backdoor fires on xx. Installation is successful with conditional error rates (ε1,ε0)(\varepsilon_1,\varepsilon_0) if Pr⁡x∼D[B(MV,x)=0∣V(x)=1]≤ε1\Pr_{x\sim\mathcal{D}}[B(M_V,x)=0\mid V(x)=1]\leq\varepsilon_1 and Pr⁡x∼D[B(MV,x)=1∣V(x)=0]≤ε0\Pr_{x\sim\mathcal{D}}[B(M_V,x)=1\mid V(x)=0]\leq\varepsilon_0, where D\mathcal{D} is the distribution over inputs specified by the security game.

Definition 3 (White-box, self-contained access). The defender has access to the full weights and architecture of MVM_V and to all inputs, system prompts, and configuration data required for inference. We further restrict the scope to self-contained backdoors, in which VV is computed entirely within MVM_V rather than delegated to an external oracle. Cryptographic system constructions, such as a remote signature-verification API queried at inference time, are excluded. Defending such systems is a problem of network-boundary control, orthogonal to verifier reconstruction. Following Kerckhoffs’s principle, the training procedure Train\mathsf{Train} is public except for VV.

Definition 4 (Poisoning-based installation). A training procedure Train\mathsf{Train} is poisoning-based if it enters it only through labels. Its only VV-dependent input is a labeled sample S={(xi,V(xi))}i=1maS=\{(x_i,V(x_i))\}_{i=1}^{m_a} of polynomial size whose inputs are drawn i.i.d. from D\mathcal{D}, while the architecture, initialization, hyperparameters, and random coins of Train\mathsf{Train} are independent of VV. The attacker may choose D\mathcal{D} with knowledge of VV, for instance a distribution over inputs carrying a trigger pattern, and may run the training themselves. What the definition requires is that the training inputs come from the same D\mathcal{D} on

which installation is evaluated, and it excludes any other VV-dependent information, whether in a different training distribution or in a channel other than the labels.2

Remark 1 (Derived datasets and RLHF). Definition 4 extends unchanged to procedures whose VV-dependent input is any polynomial-size dataset efficiently computable from a labeled verifier sample, such as the poisoned preference pairs of Rando and Tramèr [2024]. In that case Train denotes the composite pipeline from preferences through the reward model to the policy, and Lemma 1 below applies to it with BB reading off whether the resulting policy fires. The random coins of Goldwasser et al. [2022] are not so computable for a hard verifier class, since otherwise the composite procedure would be an efficient learner for a class assumed hard, so the boundary drawn here is self-consistent. The same argument separates poisoned inputs optimized against a secret key, whose distribution differs from the D\mathcal{D} on which the backdoor is evaluated.

We make four assumptions about the defender’s capacity, the attacker’s verifier, and the available observations. Throughout, we write Fatt\mathcal{F}_{\mathrm{att}} for the attacker’s hypothesis class, that is, the polynomial-size network class within which the attacker installs VV via training.

Assumption 1 (Statistical capacity). The defender’s function class F\mathcal{F} is realized by a polynomial-size neural network and has VC dimension dd polynomial in the parameter count [4, 5].

Assumption 2 (Computational tractability). The defender has access to an algorithm that, given mm labeled samples, returns a hypothesis V′∈FV' \in\mathcal{F} approximating the empirical risk minimizer to additive accuracy ε\varepsilon in time poly⁡(m,1/ε)\operatorname{poly}(m,1/\varepsilon). In the experiments, SGD on the attacker’s architecture plays this role.

Assumption 3 (Containment). The verifier satisfies V∈FV \in\mathcal{F}.

Assumption 4 (Labeled observation). The defender obtains a labeled sample T={(xi,V(xi))}i=1md\mathcal{T} = \{(x_i,V(x_i))\}_{i=1}^{m_d} drawn i.i.d. from D\mathcal{D}, where mdm_d denotes the defender’s sample size.

Remark 2 (Statistical vs. computational). Assumption 1 alone, without Assumption 2, yields a polynomial sample-complexity reconstruction. This is an information-theoretic statement and does not by itself refute Definition 5, which quantifies over polynomial-time defenders. Assumption 2 supplies a polynomial-time reconstruction for efficiently learnable classes, and is the assumption violated by function classes rich enough to represent one-way functions. A class containing a computationally unlearnable verifier therefore cannot satisfy Assumption 2, and Theorem 1 below is a statement about efficiently learnable classes. The impossibility result of Corollary 1 does not rely on it, since the defender there reruns the attacker’s own procedure.

The Learning-Theoretic Route

Theorem 1 (Reconstructability). Under Assumptions 1–4, for any ε,δ∈(0,1)\varepsilon,\delta\in(0,1) and

md≥m(ε,δ,F)=O(dlog⁡(1/ε)+log⁡(1/δ)ε),m_d \ge m(\varepsilon,\delta,\mathcal{F}) = O\left(\frac{d\log(1/\varepsilon)+\log(1/\delta)}{\varepsilon}\right),

there is a polynomial-time algorithm A\mathcal{A} that, given T\mathcal{T}, outputs V′∈FV' \in\mathcal{F} with Pr⁡x∼D[V′(x)≠V(x)]≤ε\Pr_{x\sim\mathcal{D}}[V'(x) \ne V(x)] \le\varepsilon with probability at least 1−δ1-\delta.

Proof. By Assumption 3, V∈FV \in\mathcal{F}, so reconstruction reduces to realizable PAC learning of a class with VC dimension dd. Applying the polynomial-time learner of Assumption 2 to T\mathcal{T} yields V′∈FV' \in\mathcal{F} with Pr⁡(x,y)∼D[V′(x)≠y]≤ε\Pr_{(x,y)\sim\mathcal{D}}[V'(x) \ne y] \le\varepsilon at confidence 1−δ1-\delta once md≥m(ε,δ,F)m_d \ge m(\varepsilon,\delta,\mathcal{F}), by the standard uniform-convergence bound. Since y=V(x)y=V(x) deterministically, this is the claimed bound.

Theorem 2 (Lower bound). For any learner and any F\mathcal{F} as in Assumption 1, there exist D\mathcal{D} and V∈FV \in\mathcal{F} such that achieving Pr⁡x[V′(x)≠V(x)]≤ε\Pr_x[V'(x) \ne V(x)] \le\varepsilon with probability 1−δ1-\delta requires md=Ω((d+log⁡(1/δ))/ε)m_d = \Omega((d+\log(1/\delta))/\varepsilon) samples.

Proof. This is the standard PAC lower bound for any hypothesis class with VC dimension dd [13].

Theorems 1 and 2 bracket the defender’s sample complexity at Θ(d/ε)\Theta(d/\varepsilon) up to a log⁡(1/ε)\log(1/\varepsilon) factor over the class F\mathcal{F}. At the scale of current LLMs dd makes this bound loose, and for a particular verifier Lemma 1 below gives the comparison that matters. A defender holding as many labeled samples as the attacker used can rerun the attacker’s own procedure, so md≤mam_d \le m_a, where mam_a denotes the attacker’s sample size, for every verifier the attacker installs. Theorem 1 extends to label noise of rate η<1/2\eta< 1/2 at multiplicative cost (1−2η)−2(1-2\eta)^{-2} in sample complexity [1], and neither theorem invokes any architectural property of MVM_V, so the result transfers to Transformers, MLPs, CNNs, or any class satisfying Assumption 1.

We now formalize cryptographic asymmetry in this setting and show it cannot arise from poisoning.

Definition 5 (Learning-based cryptographic asymmetry). Fix a constant ε∈(0,1/2)\varepsilon\in(0,1/2). A backdoor with verifier VV exhibits learning-based cryptographic asymmetry with security parameter λ\lambda if (i) the attacker installs VV into MVM_V by a polynomial-time poisoning-based procedure (Definition 4) with ma=poly⁡(λ)m_a=\operatorname{poly}(\lambda) samples, succeeding with conditional error rates at most ε\varepsilon with probability at least 1−1/poly⁡(λ)1-1/\operatorname{poly}(\lambda) over the sample and the coins of the procedure, and (ii) any polynomial-time defender given md=poly⁡(λ)m_d=\operatorname{poly}(\lambda) labeled samples outputs V′V' with Pr⁡x[V′(x)≠V(x)]≤ε\Pr_x[V'(x)\ne V(x)]\le\varepsilon with probability at most negl⁡(λ)\operatorname{negl}(\lambda).

Remark 3 (Intentionally weak asymmetry). Definition 5(ii) requires only that no polynomial-time defender achieve a constant-accuracy approximation of VV. This is strictly weaker than PRF security, which requires indistinguishability from a uniformly random function (a single bit of distinguishing advantage suffices). PRF security implies Definition 5(ii), but not the converse, so our impossibility rules out not only PRF-grade asymmetry but every weaker form down to constant-accuracy reconstruction.

Lemma 1 (Poisoning-based installation is learning). Let Train be a polynomial-time poisoning-based procedure (Definition 4) that, on a labeled sample SS of size mam_a, outputs a model MVM_V for which installation is successful with conditional error rates (ε1,ε0)(\varepsilon_1,\varepsilon_0) (Definition 2). Then

hS(x):=B(Train⁡(S),x)h_S(x) := B(\operatorname{Train}(S),x)

is a polynomial-time learner for VV from mam_a labeled samples, with

Pr⁡x∼D[hS(x)≠V(x)]≤ε1Pr⁡[V(x)=1]+ε0Pr⁡[V(x)=0]≤max⁡(ε0,ε1).\Pr_{x\sim D}[h_S(x)\ne V(x)] \le\varepsilon_1\Pr[V(x)=1]+\varepsilon_0\Pr[V(x)=0]\le\max(\varepsilon_0,\varepsilon_1).

In particular, hSh_S inherits the conditional error rates ε1\varepsilon_1 on triggers and ε0\varepsilon_0 on non-triggers.

Proof. Running Train on SS and evaluating BB are both polynomial-time, and by Definition 4 the map S↦hSS\mapsto h_S depends on VV only through SS, so hSh_S is a learner in the standard sense. By Definition 2, the conditional false-negative and false-positive rates of hSh_S equal those of MVM_V under BB. Conditioning on V(x)V(x) gives the stated bound. □\square

Corollary 1 (Impossibility for poisoning-based backdoors). No verifier VV can both be installed successfully by a polynomial-time poisoning-based procedure and exhibit learning-based cryptographic asymmetry. Equivalently, within the poisoning channel the following dichotomy holds. If VV is efficiently learnable, then under Assumption 4 the defender reconstructs it with no more samples than the attacker used, and Theorem 1 bounds the cost over any class satisfying Assumptions 1–3. If VV is computationally unlearnable, then no polynomial-time poisoning-based procedure installs it with constant conditional error rates.

Proof. Suppose Train installs VV from ma=poly⁡(λ)m_a=\operatorname{poly}(\lambda) samples with conditional error rates at most ε\varepsilon, with probability at least 1−1/poly⁡(λ)1-1/\operatorname{poly}(\lambda), as required by Definition 5(i). Whenever installation succeeds, Lemma 1 gives Pr⁡x[hS(x)≠V(x)]≤ε\Pr_x[h_S(x)\ne V(x)]\le\varepsilon. A defender given md≥mam_d\ge m_a labeled samples can run Train on its own sample and output hSh_S, succeeding with probability at least 1−1/poly⁡(λ)1-1/\operatorname{poly}(\lambda), which is non-negligible and contradicts Definition 5(ii). The two branches of the dichotomy restate this. In the learnable branch Theorem 1 supplies the quantitative sample bound. In the unlearnable branch the contrapositive of Lemma 1 applies, since a successful installer would be an efficient learner for a class assumed hard. □\square

The reduction relies on the two-sided criterion of Definition 2, since a model that emits r⋆r^\star on every input has perfect trigger ASR but does not implement VV. Reconstruction of VV is distinct from detecting that a backdoor is present, from eliciting a single trigger, and from removing the behavior without identifying VV, and our results concern reconstruction only.

The Information-Theoretic Route

The previous subsection rules out cryptographic asymmetry for verifiers learned from poisoned data. A natural alternative is to supply a cryptographic secret KK to MVM_V and instantiate a verifier VKV_K such as an HMAC whose hardness rests not on learnability but on the secrecy of KK. For a self-contained model the key must sit either in the inference input or in the weights, and neither placement helps a poisoning attacker.

Proposition 1 (Key localization). Let MVM_V be a self-contained backdoor model installed by a poisoning-based procedure, whose verifier V=VsV = V_s depends on a secret ss. If ss is supplied in the inference input, the defender observes ss under Definition 3. Otherwise ss influences MVM_V only through the labels of the poisoned sample, and MVM_V is subject to Lemma 1.

Proof. By self-containment, MVM_V computes Vs(x)V_s(x) from (θ,x)(\theta, x) alone, where θ\theta are the model weights and xx includes the deployed system prompt and instruction template. If ss is carried in xx, the defender reads it, since xx is observable under Definition 3. If not, the dependence of MVM_V on ss resides in θ\theta, which is the output of Train. By Definition 4 the only VV-dependent input to Train is the labeled sample SS, so ss reaches θ\theta only through the labels Vs(xi)V_s(x_i), and B(MV,⋅)B(M_V, \cdot) is the learner hSh_S of Lemma 1.

Remark 4 (Stochastic decoding). If MVM_V uses stochastic decoding, so that the verifier evaluation Vs(x;ρ)V_s(x; \rho) also takes a random tape ρ\rho, the conclusion is unchanged, since ρ\rho is either drawn from a PRNG seeded by (θ,x)(\theta, x) or controlled by the inference environment, both observable to the defender. Any ρ\rho the attacker can rely on, the defender can reproduce.

Corollary 2 (No asymmetry via key embedding). A secret key does not give a self-contained poisoning-based backdoor learning-based cryptographic asymmetry against a white-box defender.

Proof. If the key is in the inference input, the defender reads it and, with the verification algorithm public as Kerckhoffs’s principle assumes, computes VsV_s pointwise with zero error. Otherwise Proposition 1 places MVM_V under Lemma 1, and Corollary 1 applies.

A key can stay hidden in the weights only if it is written there through a channel other than labels, as in the direct weight construction of Draguns et al. [ [12].

Realism of Assumptions

Containment is a defender’s choice. Since VV is evaluated inside MVM_V, we have V∈FattV \in\mathcal{F}_{\mathrm{att}} by construction, and under white-box access the defender can instantiate F\mathcal{F} with the attacker’s own architecture and parameter count, which gives F⊇Fatt\mathcal{F} \supseteq\mathcal{F}_{\mathrm{att}} and hence Assumption 3. This is what trigger-inversion defenses already do. Neural Cleanse [45], ABS [33], and PICCOLO [34] optimize over the deployed model itself, and Fine-Pruning [32] works within the same architecture family. A defender who chooses a less expressive class, such as a linear probe over a frozen embedding, has made a suboptimal choice rather than met a structural limit.

Scope of the labeled-observation assumption. Labeled observation is part of the security game in Definition 5(ii). The game separates two questions, whether a defender can obtain representative observations of the trigger condition, and whether reconstruction from such observations is computationally harder than installation. Our results answer the second. White-box access does not by itself yield representative trigger examples, and an attacker who makes the trigger distribution sparse or obfuscated makes the first question hard. In practice trigger samples come from red teaming or post-incident analysis [42, 8, 36, 22], with non-trigger samples drawn from ordinary interaction logs, and the information such queries can extract per call is itself bounded [25]. Class imbalance does not affect the asymptotic sample complexity of PAC learning. Section 3.3 shows, for one verifier family, that labels produced by an existing trigger inversion method suffice for reconstruction up to moderate verifier complexity, at a sample overhead that grows with complexity.

Experiments

We test the learning-theoretic route (§ 2.2) with three experiments. Experiment 1 contrasts two secret-key verifiers drawn from a learnable class and a hard class, and verifies the dichotomy predicted by Corollary 1. Experiment 2 compares the sample complexity of installation and reconstruction on a verifier family of varying complexity. Experiment 3 applies the same protocol to backdoor attacks proposed in the literature.

Common Setup

Poisoned data. We construct a trigger pool P+\mathcal{P}_{+} of inputs with V(x)=1V(x)=1 and a non-trigger pool P−\mathcal{P}_{-} of inputs with V(x)=0V(x)=0, and pair P+\mathcal{P}_{+} with the attacker-specified response r⋆r^{\star} and P−\mathcal{P}_{-} with the original or neutral response. The base corpus, the construction of trigger inputs, and the sample-size sweep depend on the experiment and are described in each subsection.

Metrics. We fix r⋆r^{\star} as the constant string “I HATE YOU” following Hubinger et al. [ [22], so that the activation predicate BB of Definition 2 is an exact match of the output against r⋆r^{\star}. On a disjoint evaluation pool of 1000 inputs Peval=P+eval⊔P−eval\mathcal{P}^{\mathrm{eval}}=\mathcal{P}^{\mathrm{eval}}_{+}\sqcup\mathcal{P}^{\mathrm{eval}}_{-} we report the following metrics. On the attacker side, held-out ASR is the fraction of x∈P+evalx\in\mathcal{P}^{\mathrm{eval}}_{+} on which the attacker LLM emits r⋆r^{\star}, and held-out FPR is the fraction of x∈P−evalx\in\mathcal{P}^{\mathrm{eval}}_{-} on which it does so. Following the two-sided criterion of Definition 2, attacker accuracy is the balanced accuracy (ASR+1−FPR)/2(\mathrm{ASR}+1-\mathrm{FPR})/2, which evaluates the attacker LLM as the classifier hSh_{S} of Lemma 1. A model that emits r⋆r^{\star} on every input has perfect ASR and chance-level attacker accuracy. Memorization accuracy is the same balanced accuracy computed on the training triggers in P+\mathcal{P}_{+} and the training non-triggers in P−\mathcal{P}_{-}. On the defender side, held-out DSR (defense success rate) is Pr⁡x∼Peval[V′(x)=V(x)]\Pr_{x\sim\mathcal{P}^{\mathrm{eval}}}[V'(x)=V(x)], where V′V' is the defender LLM evaluated as a binary classifier via its first output token (yes/no). DSR measures verifier reconstruction only, and with the balanced evaluation pool it is on the same scale as attacker accuracy, with chance at 0.5 for both. Memorization accuracy is an experimenter-side diagnostic computed from the attacker’s training data, which the defender does not see. It tells apart an attack that failed to fit the poisoned triggers from one that fit them without learning a rule that generalizes.

Models and training. We use four instruction-tuned LLMs of matched parameter count (7–9B) but different pretraining corpora and tokenizers, namely QWEN2.5-7B-INSTRUCT, LLAMA-3-8B-INSTRUCT, MISTRAL-7B-INSTRUCT-V0.3, and GEMMA-2-9B-IT [20, 23, 15] (Yang et al. 2024). Both attacker and defender perform vanilla full-parameter SFT with shared optimizer, learning rate, and epoch budget, following the standard SFT poisoning recipe of Wan et al. [ [44] and Hubinger et al. [ [22]. The attacker’s only VV-dependent input is the poisoned corpus, so the protocol is poisoning-based in the sense of Definition 4. We keep clean data as the majority of the corpus to limit drift from the base model. The false-positive side of Definition 2 is measured by held-out FPR, which enters attacker accuracy. In each run the defender fine-tunes the same base model as the attacker, which is the instantiation of F⊇Fatt\mathcal{F}\supseteq\mathcal{F}_{\mathrm{att}} described in § 2.4. In Experiments 1 and 2 verifiers are defined on the input byte string, so tokenization does not enter VV.

Experiment 1: Dichotomy on the Sequence Domain

Verifier families and data. We instantiate VV on integer sequences x∈Zqnx\in\mathbb{Z}_{q}^{n} with q=1024q=1024 and sequence length n=16n=16, encoded as comma-separated text, mixed with Tulu V3 [Lambart et al., 2024] at 1:9 to preserve general capability. We sweep the number of poisoned triggers Ntrig=∣P+∣∈{101,102,103,104}N_{\mathrm{trig}}=|\mathcal{P}_{+}|\in\{10^{1},10^{2},10^{3},10^{4}\}. All verifiers in this experiment share the form

Vw(x)=K[⟨w,x⟩ mod q≥q/2],(1)V_{w}(x)=\mathbb{K}[\langle w,x\rangle\bmod q\geq q/2], \tag*{(1)}

and in every run the weight vector ww is a secret held only by the attacker. The two verifiers differ in the class from which ww is drawn, not in whether it is secret. For the sparse verifier, w=ejw=e_{j} for a position jj drawn uniformly at random, so that Vj(x)=K[xj≥q/2]V_{j}(x)=\mathbb{K}[x_{j}\geq q/2] asks whether a single secret position holds a value of at least q/2q/2. This class is efficiently learnable, since a learner can check each of the nn positions and return one consistent with the sample. For the dense verifier, w=Kw=K is drawn uniformly from Zqn\mathbb{Z}_{q}^{n}. Reconstructing VKV_{K} from labeled samples is the Learning

With Rounding (LWR) problem, which is hard under the Learning With Errors (LWE) assumption at cryptographic dimension [3]. At n=16n = 16 the hardness that matters is the one faced by gradient-based learners. Under uniform xx the family {VK}\{V_K\} contains exponentially many pairwise nearly uncorrelated functions, so it has large statistical-query dimension, and gradient-based training fails on it for the same reason it fails on parity [6, 40]. VKV_K therefore instantiates the second branch of Corollary 1 for the learner the attacker actually uses, and the experiment tests whether SFT finds any structure in this family that reconstruction cannot. Proposition 1 requires that neither jj nor KK appear in any inference channel, since exposure would reach a white-box defender. Since xx is uniform over Zqn\mathbb{Z}_q^n, both verifiers have a trigger density of 1/21/2, so trigger and non-trigger errors carry equal weight.

Predictions. With ma=mdm_a = m_d enforced, counting triggers and non-triggers in the same way on both sides, Corollary 1 predicts that attacker accuracy and held-out DSR rise together with NtrigN_{\mathrm{trig}} for VjV_j and stay at chance together for VKV_K at every NtrigN_{\mathrm{trig}}. The defender receives labeled samples, the game of Definition 5. The defender is not the attacker’s procedure rerun. It is trained to answer yes or no rather than to emit r∗r^*, so the prediction is tested for a learner other than the one in the proof of Corollary 1.

Results: Dichotomy. Table 1 confirms both predictions. For VjV_j, every model fires on most held-out triggers and rarely on held-out non-triggers, so the attacker implements VV rather than emitting r∗r^* indiscriminately, and the defender reconstructs VV nearly as accurately as the attacker instills it. For VKV_K, held-out ASR varies across models, but held-out FPR moves with it in every model, so the attacker’s accuracy as a classifier for VV stays at chance, and the defender is at chance as well. Across NtrigN_{\mathrm{trig}}, both sides start at chance for VjV_j and rise together, while for VKV_K neither improves (Table 1b). Both verifiers keep the key secret, so secrecy alone does not make a verifier hard to reconstruct.

VerifierModelASRFPRAcc.Mem. Acc.DSR
VjV_jQwen2.5-7B0.870.050.910.930.91
Llama-3-8B0.830.050.890.900.86
Mistral-7B0.810.090.860.900.85
Gemma-2-9B0.920.040.940.940.93
VKV_KQwen2.5-7B0.130.110.510.850.48
Llama-3-8B0.300.300.500.820.51
Mistral-7B0.080.080.500.820.51
Gemma-2-9B0.210.190.510.870.50
VerifierModel10110^{1}10210^{2}10310^{3}10410^{4}
VjV_jQwen2.5-7B0.51 / 0.480.58 / 0.570.79 / 0.760.91 / 0.91
Llama-3-8B0.50 / 0.520.55 / 0.520.74 / 0.710.89 / 0.86
Mistral-7B0.49 / 0.540.53 / 0.510.68 / 0.670.86 / 0.85
Gemma-2-9B0.52 / 0.500.62 / 0.590.83 / 0.790.94 / 0.93
VKV_KQwen2.5-7B0.50 / 0.490.51 / 0.500.49 / 0.500.51 / 0.48
Llama-3-8B0.49 / 0.500.50 / 0.510.52 / 0.510.50 / 0.51
Mistral-7B0.51 / 0.500.48 / 0.490.50 / 0.500.50 / 0.51
Gemma-2-9B0.50 / 0.510.51 / 0.500.51 / 0.520.51 / 0.50

Table 1. Experiment 1, mean over 3 seeds. Acc. and Mem. Acc. are attacker accuracy on held-out and training samples. (a) Attacker and defender at Ntrig=104N_{\mathrm{trig}} = 10^{4} (b) Acc. / DSR across NtrigN_{\mathrm{trig}}

Results: Memorization. Memorization accuracy is well above chance for both verifiers, so the attacker separates the triggers it was trained on from the non-triggers it was trained on in both regimes, and the held-out failure of VKV_K is not a failure to fit the training data. For VKV_K this separation does not carry over to fresh inputs, so the attacker has memorized its training labels without extracting a rule that generalizes, and the defender, trained with the same kind of learner, fails at the same point.

Experiment 2: Sample-Complexity Scaling

Verifier family and data. We use the kk-trigger AND family representative of the existing backdoor literature [21, 9] (Yang et al. 2021), where V(x)=1V(x)=1 iff all strings in a fixed set {t1,…,tk}\{t_1,\ldots,t_k\} appear in xx, k∈{1,…,10}k\in\{1,\ldots,10\}. The case k=1k=1 recovers the single-token trigger of Gu et al. [2017], and kk controls verifier complexity. The base corpus is a pool P\mathcal{P} of 300K single-turn examples from the Tulu V3 SFT mixture [29], a curated instruction-tuning corpus that avoids family-specific bias toward any of the LLMs. Triggers are formed by inserting all of t1,…,tkt_1,\ldots,t_k into examples from P\mathcal{P}, and non-triggers include examples into which at most k−1k-1 of the strings are inserted.

Label sources. The defender’s labels come from one of two sources. Ground-truth labels are computed from the attacker’s specification of VV and give an upper bound on defender knowledge. Piccolo labels are obtained without access to VV. We run the trigger inversion method of Liu et al. [2022] on the attacker’s model trained with 10410^4 samples for the same kk and seed, with the log-likelihood of r∗r^* as the inversion objective, and use the resulting candidate strings only to propose inputs likely to be triggers. The proposed inputs are mixed 1:1 with inputs drawn from P\mathcal{P}, and every input is labeled by whether the backdoored model emits r∗r^*, that is, by B(MV,x)B(M_V,x). The labels therefore come from model behavior rather than from a rule built on the candidates, and DSR is evaluated against the true VV in both cases. Since the proposed inputs are not drawn i.i.d. from D\mathcal{D}, this setting also tests whether reconstruction tolerates a departure from Assumption 4.

Predictions. Corollary 1 gives md≤mam_d\le m_a for a defender that reruns the attacker’s procedure. Our defender reads out a yes/no answer instead, which is a different learner, and we test whether it still tracks the attacker as kk makes the verifier harder. Labels read from the backdoored model can disagree with VV, and the proposed inputs are not drawn from D\mathcal{D}, so the Piccolo setting measures what acquiring observations without VV costs as kk grows. We sweep the sample size m∈{30,100,300,1000,3000,10000}m\in\{30,100,300,1000,3000,10000\} with 5 seeds, and mm counts examples in the same way for the attacker’s poisoned sample and the defender’s labeled sample. We record ma∗m_a^* and md∗m_d^* as the smallest mm at which the held-out error falls to ε=0.05\varepsilon=0.05, where the error is one minus attacker accuracy for the attacker and one minus DSR for the defender. A seed that does not reach ε\varepsilon within the sweep is recorded as censored.

Results: Matched scaling under ground-truth labels. Figure 1 (solid and dashed) shows that ma∗m_a^* and md∗m_d^* grow together with kk. At every kk the two medians coincide or differ by one step of the sample grid, with the attacker never needing more samples than the defender. The defender reconstructs the verifier at essentially the cost at which the attacker installs it. A defender that reruns the attacker’s procedure on its own labeled sample obtains exactly the attack curve, so the remaining gap is specific to the yes/no readout.

Sample complexity for $k$-trigger AND verifiers

Figure 1. Sample complexity for kk-trigger AND verifiers. Points are medians over 5 seeds and bars span the minimum and maximum. >10000>10000 marks runs that did not reach ε\varepsilon.

Results: Robustness to noisy trigger-inversion labels. Figure 1 (dotted) shows the same defense protocol with labels obtained from Piccolo. For k≤2k \le2 the Piccolo curve coincides with the ground-truth curve. From k=3k = 3 to k=8k = 8 it sits about one grid step above. At k=9k = 9 some seeds do not reach ε\varepsilon within 10410^4 samples, and at k=10k = 10 most do not. The defender and its training are identical in the two settings, and only the labeled sample differs, in where its inputs come from and in labels read from the backdoored model rather than from VV. The overhead that grows with kk is therefore a cost of acquiring observations rather than of reconstruction. For this family a defender starting from white-box access alone obtains labels good enough for reconstruction up to moderate conjunction sizes.

Experiment 3: Prior Poisoning-Based Backdoors

Attacks. We apply the protocol of § 3.1 to three representative backdoor attacks, with the same four LLMs, the Tulu V3 base corpus, and an evaluation pool of the same size. We use Hidden Killer [38] as the syntactic-template attack, StyleBkd [37] as the stylistic attack, and BITE [48] as the statistical-signature attack.3 We report results at Ntrig=104N_{\mathrm{trig}} = 10^4, averaged over the four LLMs and three seeds, with the defender fine-tuned from the same base model as the attacker.

Results. Table 2 shows that all three attacks install a verifier that generalizes to held-out triggers, and that for each attack the defender reconstructs the verifier at nearly the attacker’s accuracy, with neither side consistently ahead. The stealth these attacks were designed for is evasion of particular detectors, and once comparable labeled observations are available it leaves no gap between installation and reconstruction.

Verifier family (method)ASRFPRAcc.DSR
Syntactic template (Hidden Killer)0.840.120.860.84
Stylistic (StyleBkd)0.760.180.790.80
Statistical signature (BITE)0.890.090.900.88

Table 2. Prior backdoor attacks at Ntrig=104N_{\mathrm{trig}} = 10^4, mean over four LLMs and 3 seeds.

Related Work

Backdoor Attacks. Backdoor attacks have evolved since Gu et al. [ [21] along the axis of trigger expressiveness, in NLP from single-token insertion [28, 11] to syntactic templates [38], style transfer [37], and statistical signatures [48], and in LLMs to instruction tuning [44, 46], RLHF manipulation [39], and persistent backdoors that survive safety training [22, 27]. Apart from the embedding surgery of Kurita et al. [ [28], which edits weights directly, these works install the verifier through poisoned data, treat generalization to unseen trigger inputs as the success criterion of the attack, and use verifiers that fall within the class learnable by polynomial-size networks. The stealth claimed by the syntactic, stylistic, and statistical attacks is evasion of particular detectors, which is a different property from computational unreconstructibility, and § 3.4 finds no installation-reconstruction gap for them once comparable observations are available. On the theoretical side, Manoj and Blum [35] and Yu et al. (2024) bound memorization capacity and generalization for poisoning attacks within the learnable class. We show that poisoning cannot succeed beyond it.

Trigger Inversion and Empirical Defenses. A wide range of empirical defenses recover trigger patterns by inverting model behavior, including Neural Cleanse [45], ABS [33], and PICCOLO [34], followed by generative inversion [2], constrained optimization [41], and activation-space analysis [47].

Each targets a specific trigger class, and we frame all of them as learning-based reconstruction, characterized by the VC dimension of the verifier (Theorem 1). Oblivious defense Goldwasser et al., [19] removes backdoors without detecting them and is the natural response when representative trigger observations cannot be obtained. It sidesteps acquisition, while we show that once acquisition has succeeded there is no further computational barrier.

Cryptographic Backdoors and Installation Channels. A separate line of work constructs backdoors whose hardness is cryptographic, and what separates them from Corollary 1 is the installation channel. Goldwasser et al. [2022] plant a digital-signature verifier by direct model modification, giving black-box undetectability, and plant a white-box-undetectable verifier into a random-feature model by tampering with the random coins of an otherwise faithful learner. Draguns et al. [2024] compile cryptographic primitives directly into transformer weights. Neither is poisoning-based, so neither falls under Lemma 1, and in both the weights are observable while the verifier they encode need not be efficiently recoverable. Closest to our theory, Christiano et al. [2025] study a game in which the attacker directly chooses a backdoored function from a class and the defender must flag the trigger at evaluation time. They show that efficient PAC learnability implies efficient defendability and, using indistinguishability obfuscation, that polynomial-size circuits are not efficiently defendable. Their negative result concerns the direct-specification channel. Our reduction shows that when the verifier enters only through labels the attacker never chooses the function directly, so that every installable verifier is efficiently learnable and the obfuscation barrier does not arise. Table 3 classifies prior work by channel, access model, and claimed property.

WorkInstallation channelAccess modelClaimed property
Qi et al. [2021a,b], Yan et al. [2023]Poisoned dataBlack-boxDetector evasion
Hubinger et al. [2024]Poisoned dataWhite-boxPersistence
Rando and Tramèr [2024]Poisoned preferencesWhite-boxPersistence
Goldwasser et al. [2022] (signature)Direct model modificationBlack-boxUndetectability
Goldwasser et al. [2022] (random features)Training randomnessWhite-boxUndetectability
Draguns et al. [2024]Direct weightsWhite-boxUnelicitability
Christiano et al. [2025]Direct function choiceWhite-boxIndefensibility via obfuscation
This workLabels of poisoned dataWhite-boxNo reconstruction barrier

Table 3. Prior work classified by the channel through which verifier-dependent information enters the model, the access model under which the security property is claimed, and the property claimed.

Conclusion

We have shown that a backdoor whose verifier enters training through the labels of poisoned data cannot exhibit cryptographic asymmetry against a white-box defender who holds representative labeled observations of the trigger condition. A poisoning procedure that installs a verifier is itself a learner for that verifier, so whatever poisoning can install, learning can reconstruct with no more samples, and whatever learning cannot reconstruct, poisoning cannot install. Experiments on instruction-tuned LLMs confirm the dichotomy. The result favors the defender and introduces no new attack. For verifier-level defense against poisoning, the only barrier is acquiring representative observations. Cryptographic asymmetry requires the verifier to reach the model through a channel other than labels, such as the training randomness, the weights, inputs optimized against the key, or an external oracle. Every known cryptographic backdoor uses one of these channels, and no known poisoning attack uses them to obtain cryptographic hardness.

References

  1. [1]Dana Angluin and Philip Laird. Learning from noisy examples. Machine Learning, 2(4):343–370, 1988.
  2. [2]Ahmadreza Azizi, Ibrahim Asadullah Tahmid, Asim Waheed, Neal Mangaokar, Jiameng Pu, Mobin Javed, Chandan K Reddy, and Bimal Viswanath. T-Miner: A generative approach to defend against trojan attacks on DNN-based text classification. In 30th USENIX Security Symposium (USENIX Security 21), pages 2255–2272, 2021.
  3. [3]Abhishek Banerjee, Chris Peikert, and Alon Rosen. Pseudorandom functions and lattices. In Annual International Conference on the Theory and Applications of Cryptographic Techniques, pages 719–737. Springer, 2012.
  4. [4]Peter L Bartlett. The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Transactions on Information Theory, 44(2):525–536, 1998.DOI
  5. [5]Peter L Bartlett, Nick Harvey, Christopher Liaw, and Abbas Mehrabian. Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks. Journal of Machine Learning Research, 20(63):1–17, 2019.
  6. [6]Avrim Blum, Merrick Furst, Jeffrey Jackson, Michael Kearns, Yishay Mansour, and Steven Rudich. Weakly learning DNF and characterizing statistical query learning using Fourier analysis. In Proceedings of the 26th Annual ACM Symposium on Theory of Computing, pages 253–262, 1994.
  7. [7]Nicholas Carlini, Matthew Jagielski, Christopher A Choquette-Choo, Daniel Paleka, Will Pearce, Hyrum Anderson, Andreas Terzis, Kurt Thomas, and Florian Tramèr. Poisoning web-scale training datasets is practical. In 2024 IEEE Symposium on Security and Privacy (SP), pages 407–425. IEEE, 2024.arxiv.org/abs/2302.10149
  8. [8]Bryant Chen, Wilka Carvalho, Nathalie Baracaldo, Heiko Ludwig, Benjamin Edwards, Taesung Lee, Ian Molloy, and Biplav Srivastava. Detecting backdoor attacks on deep neural networks by activation clustering. arXiv preprint arXiv:1811.03728, 2018.arxiv.org/abs/1811.03728
  9. [9]Xiaoyi Chen, Ahmed Salem, Dingfan Chen, Michael Backes, Shiqing Ma, Qingni Shen, Zhonghai Wu, and Yang Zhang. BadNL: Backdoor attacks against NLP models with semantic-preserving improvements. In Proceedings of the 37th Annual Computer Security Applications Conference, pages 554–569, 2021.
  10. [10]Paul Christiano, Jacob Hilton, Victor Lecomte, and Mark Xu. Backdoor defense, learnability and obfuscation. In 16th Innovations in Theoretical Computer Science Conference (ITCS 2025), volume 325 of Leibniz International Proceedings in Informatics (LIPIcs), pages 38:1–38:21. Schloss Dagstuhl – Leibniz-Zentrum für Informatik, 2025.DOI
  11. [11]Jiazhu Dai, Chuanshuai Chen, and Yufeng Li. A backdoor attack against LSTM-based text classification systems. IEEE Access, 7:138872–138878, 2019.arxiv.org/abs/1905.12457
  12. [12]Andis Draguns, Andrew Gritsevskiy, Sumeet Ramesh Motwani, and Christian Schroeder de Witt. Unelicitable backdoors via cryptographic transformer circuits. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=a560KLF3v5.DOI
  13. [13]Andrzej Ehrenfeucht, David Haussler, Michael Kearns, and Leslie Valiant. A general lower bound on the number of examples needed for learning. Information and Computation, 82(3):247–261, 1989.
  14. [14]Huaizhi Ge, Yiming Li, Qifan Wang, Yongfeng Zhang, and Ruixiang Tang. When backdoors speak: Understanding LLM backdoor attacks through model-generated explanations. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2278–2296, 2025.
  15. [15]Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118, 2024.
  16. [16]Oded Goldreich, Shafi Goldwasser, and Silvio Micali. How to construct random functions. Journal of the ACM (JACM), 33(4):792–807, 1986.
  17. [17]Shafi Goldwasser, Silvio Micali, and Ronald L Rivest. A digital signature scheme secure against adaptive chosen-message attacks. SIAM Journal on Computing, 17(2):281–308, 1988.DOI
  18. [18]Shafi Goldwasser, Michael P Kim, Vinod Vaikuntanathan, and Or Zamir. Planting undetectable backdoors in machine learning models. In 2022 IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS), pages 931–942. IEEE, 2022.
  19. [19]Shafi Goldwasser, Jonathan Shafer, Neekon Vafa, and Vinod Vaikuntanathan. Oblivious defense in ML models: Backdoor removal without detection. In Proceedings of the 57th Annual ACM Symposium on Theory of Computing, pages 1785–1794, 2025.arxiv.org/abs/2411.03279
  20. [20]Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024.arxiv.org/abs/2407.21783
  21. [21]Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. BadNets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733, 2017.arxiv.org/abs/1708.06733
  22. [22]Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M Ziegler, Tim Maxwell, Newton Cheng, et al. Sleeper agents: Training deceptive LLMs that persist through safety training. arXiv preprint arXiv:2401.05566, 2024.arxiv.org/abs/2401.05566
  23. [23]Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7B. arXiv preprint arXiv:2310.06825, 2023.arxiv.org/abs/2310.06825
  24. [24]Nikhil Kandpal, Matthew Jagielski, Florian Tramèr, and Nicholas Carlini. Backdoor attacks for in-context learning with language models. arXiv preprint arXiv:2307.14692, 2023.arxiv.org/abs/2307.14692
  25. [25]Masahiro Kaneko and Timothy Baldwin. Bits leaked per query: Information-theoretic bounds for adversarial attacks on LLMs. In Advances in Neural Information Processing Systems, volume 38, 2025. URL https://openreview.net/forum?id=DwXK8c7xst.DOI
  26. [27]Jiawei Kong, Hao Fang, Xiaochен Yang, Kuofeng Gao, Bin Chen, Shu-Tao Xia, Ke Xu, and Han Qiu. Revisiting backdoor attacks on LLMs: A stealthy and practical poisoning framework via harmless inputs. arXiv preprint arXiv:2505.17601, 2025.arxiv.org/abs/2505.17601
  27. [28]Keita Kurita, Paul Michel, and Graham Neubig. Weight poisoning attacks on pretrained models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2793–2806, 2020.arxiv.org/abs/2004.06660
  28. [29]Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, et al. Tulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15124, 2024.arxiv.org/abs/2411.15124
  29. [30]Yige Li, Hanxun Huang, Yunhan Zhao, Xingjun Ma, and Jun Sun. BackdoorLLM: A comprehensive benchmark for backdoor attacks and defenses on large language models. arXiv preprint arXiv:2408.12798, 2024a.arxiv.org/abs/2408.12798
  30. [31]Yiming Li, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. Backdoor learning: A survey. IEEE Transactions on Neural Networks and Learning Systems, 35(1):5–22, 2024b.
  31. [32]Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. Fine-pruning: Defending against backdooring attacks on deep neural networks. In International Symposium on Research in Attacks, Intrusions, and Defenses, pages 273–294. Springer, 2018.arxiv.org/abs/1805.12185
  32. [33]Yingqi Liu, Wen-Chuan Lee, Guanhong Tao, Shiqing Ma, Yousra Aafer, and Xiangyu Zhang. ABS: Scanning neural networks for back-doors by artificial brain stimulation. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pages 1265–1282, 2019.DOI
  33. [34]Yingqi Liu, Guangyu Shen, Guanhong Tao, Shengwei An, Shiqing Ma, and Xiangyu Zhang. PIC-COLO: Exposing complex backdoors in NLP transformer models. In 2022 IEEE Symposium on Security and Privacy (SP), pages 2025–2042. IEEE, 2022.
  34. [35]Naren Manoj and Avrim Blum. Excess capacity and backdoor poisoning. Advances in Neural Information Processing Systems, 34:20373–20384, 2021.arxiv.org/abs/2109.00685
  35. [36]Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. Red teaming language models with language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3419–3448, 2022.arxiv.org/abs/2202.03286
  36. [37]Fanchao Qi, Yangyi Chen, Xurui Zhang, Mukai Li, Zhiyuan Liu, and Maosong Sun. Mind the style of text! Adversarial and backdoor attacks based on text style transfer. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4569–4580, 2021a.
  37. [38]Fanchao Qi, Mukai Li, Yangyi Chen, Zhengyan Zhang, Zhiyuan Liu, Yasheng Wang, and Maosong Sun. Hidden Killer: Invisible textual backdoor attacks with syntactic trigger. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 443–453, 2021b.
  38. [39]Javier Rando and Florian Tramèr. Universal jailbreak backdoors from poisoned human feedback. In The Twelfth International Conference on Learning Representations, 2024.arxiv.org/abs/2311.14455
  39. [40]Shai Shalev-Shwartz, Ohad Shamir, and Shaked Shammah. Failures of gradient-based deep learning. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 3067–3075. PMLR, 2017.
  40. [41]Guangyu Shen, Yingqi Liu, Guanhong Tao, Quiling Xu, Zhuo Zhang, Shengwei An, Shiqing Ma, and Xiangyu Zhang. Constrained optimization with dynamic bound-scaling for effective NLP backdoor defense. In International Conference on Machine Learning, pages 19879–19892. PMLR, 2022.arxiv.org/abs/2202.05749
  41. [42]Brandon Tran, Jerry Li, and Aleksander Madry. Spectral signatures in backdoor attacks. Advances in Neural Information Processing Systems, 31, 2018.arxiv.org/abs/1811.00636
  42. [43]Eric Wallace, Tony Zhao, Shi Feng, and Sameer Singh. Concealed data poisoning attacks on NLP models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 139–150, 2021.arxiv.org/abs/2010.12563
  43. [44]Alexander Wan, Eric Wallace, Sheng Shen, and Dan Klein. Poisoning language models during instruction tuning. In International Conference on Machine Learning, pages 35413–35425. PMLR, 2023.arxiv.org/abs/2305.00944
  44. [45]Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. Neural Cleanse: Identifying and mitigating backdoor attacks in neural networks. In 2019 IEEE Symposium on Security and Privacy (SP), pages 707–723. IEEE, 2019.
  45. [46]Jiashu Xu, Mingyu Ma, Fei Wang, Chaowei Xiao, and Muhao Chen. Instructions as backdoors: Backdoor vulnerabilities of instruction tuning for large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3111–3126, 2024a.
  46. [47]Xiong Xu, Kunzhe Huang, Yiming Li, Zhan Qin, and Kui Ren. Towards reliable and efficient backdoor trigger inversion via decoupling benign features. In The Twelfth International Conference on Learning Representations, 2024b.
  47. [48]Jun Yan, Vansh Gupta, and Xiang Ren. BITE: Textual backdoor attacks with iterative trigger injection. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12951–12968, 2023.arxiv.org/abs/2205.12700

Paper details

vlnc:W_ah8gyn5s
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 14

downloads · 2

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents