Valency Hub
alphaFeedback
PDF

WorksComputer Science

When Guidance Becomes a Guard: Early Interception of Unsafe Inputs in Text-to-Image Generation

Yanchun Li, Yujin Wu, Gangkai Shen, Ming Yang, Li Zeng

Published Fresh — added to the Hub within the last 30 daysv120 min read

AI contributions
Not recorded for this paper
View details

Abstract

Text-to-image diffusion models can produce unsafe content, while output filtering intervenes only after synthesis. We investigate whether native generation signals enable early detection of unsafe and adversarial inputs. We find that classifier-free guidance exposes strong safety signals in conditional responses, accessible through compact statistics and linear readouts. Based on this finding, we propose Conditional Semantic Trajectory Interception (CSTI), which aggregates activation contrasts between conditional and unconditional responses at early checkpoints for calibrated risk detection. CSTI fits concept-specific detectors using canonical concept and safe examples, without attack samples, intermediate image decoding, or generator modification. Across five generators, four adversarial attack collections, and two general NSFW datasets, CSTI achieves the strongest overall performance among six baselines at a matched 5% FPR. Early interception reduces generation latency by over 60% on intercepted paths. These results demonstrate the effectiveness of native conditional responses for early safety interception.

Introduction

Text-to-image models can generate compelling visual content from natural language prompts [23], but can also be exploited to produce unsafe images. Crafted prompts may bypass safeguards and induce explicit nudity, violence, or other harmful content [31, 43]. Final image moderation can block such outputs only after the full generation cost is incurred. This motivates early interception before synthesis is complete while allowing benign requests to proceed normally, as illustrated in Figure 1.

From late checking to early interception

Figure 1. From late checking to early interception. (a) Crafted prompts can induce unsafe generations. (b) Post-generation safety checks incur the full generation cost. (c) In this example, CSTI reads early CFG activations at steps 5, 10, and 15, intercepting risky inputs while allowing accepted requests to continue unchanged. Intermediate images are shown only for illustration.

Recent work has shown that safety can be assessed during generation. IGD classifies predicted noise [41], Wukong detects from intermediate features while reusing cross-attention parameters [17], and FlowGuard projects intermediate latents into an approximate image space for visual detection [42]. These methods demonstrate the feasibility of in-generation detection, but leave room to examine the safety information already present in the generator’s native conditional computation. Effective interception requires high sensitivity to unsafe and adversarial inputs while minimizing false rejections of benign requests, further motivating the study of early diffusion representations. This motivates a question.

How early does a text-to-image generator become safety-readable, and how little machinery is needed to read that signal?

We investigate this question through the native computation of classifier-free guidance (CFG) [12]. CFG evaluates conditional and unconditional branches from the same latent state at each sampling update. We find that differences between their internal activations already contain strong safety information at early sampling stages. Compact channel statistics preserve much of this information, which can be effectively recovered using simple linear readouts. These observations suggest that the computation used to guide image formation also provides accessible safety signals before synthesis is complete.

Based on this finding, we introduce Conditional Semantic Trajectory Interception (CSTI). CSTI summarizes conditional activation contrasts at a few early checkpoints into a compact representation and applies a lightweight linear risk readout. Under our 50-update protocol, CSTI observes updates 5, 10, and 15 and makes a single decision after update 15. Flagged generations terminate early, while accepted generations continue along the original sampling trajectory. CSTI requires no intermediate image decoding, external visual encoder, or modification of the generator.

For each generator and risk concept, the detector is fitted using canonical concept examples and safe examples without attack samples. An independent safe set is used to calibrate the interception threshold, separating concept fitting from threshold selection. Neither process relies on the adversarial prompt collections used for evaluation.

We evaluate CSTI on five generators: SD1.4 and SD1.5 [23], SDXL [22], SD3.5 Medium [6, 28], and Z-Image [30, 47], spanning U-Net and Transformer backbones. CSTI achieves the strongest overall performance among representative baselines on nudity and violence collections at a matched 5% FPR. Layer, checkpoint, and classifier analyses confirm that simple linear readouts can exploit early safety signals. CSTI reduces generation latency by over 60% on intercepted paths across all five generators. Our contributions are threefold:

  • We reveal that native conditional responses of text-to-image generators contain strong safety signals at early sampling stages, which can be effectively extracted using compact statistics and simple linear readouts.

  • We propose CSTI, a lightweight interception framework that requires no attack examples for detector fitting, no intermediate image decoding, and no modification of the generator or accepted sampling process.

  • We systematically evaluate CSTI, demonstrating the effectiveness of early safety signals and the computational efficiency of interception.

Related Work

Safety defenses and adversarial prompts. Safety mechanisms for text-to-image generation operate at different stages. Before sampling, text classifiers such as the NSFW Text Classifier [21] and Detoxify [11] can screen potentially unsafe requests. Latent Guard [18] detects harmful concepts in learned representations of text encoder embeddings, while GuardT2I [44] uses a language model to convert text guidance embeddings into natural language for adversarial prompt detection. Such screening can reject risky requests before synthesis, but does not observe how the generative state develops during sampling. Other defenses change how images are generated. ESD [7] removes target concepts by updating model weights, whereas SLD [24] and CRG [5] suppress unsafe concepts through inference-time guidance. Adversarial methods such as Ring-A-Bell [31] and MMA-Diffusion [43] demonstrate that crafted prompts can still elicit unsafe outputs despite existing safeguards. Together, these approaches motivate examining whether risk can be assessed during generation without changing accepted sampling. Table 1 compares representative defenses according to their observation and intervention mechanisms.

Table 1. Comparison of representative T2I safety defenses. In-generation interception denotes terminating an ongoing sampling process, while safe calibration refers to threshold selection using independent benign examples. ●/○ denote support/no support, and – denotes not applicable.

AnalysisDefenseGeneration signalIn-generation interceptionNo image reconstructionFrozen generatorSafe calibration
PromptText-classifier [21]○○–●○
PromptDetoxify [11]○○–●○
PromptLatent Guard [18]○○–●○
Generation controlESD [7]○○●○○
Generation controlSLD [24]●○●●○
Generation controlCRG [5]●○●●○
Intermediate stateIGD [41]●●●●○
Intermediate stateWukong [17]●●●●○
Intermediate stateFlowGuard [42]●●○●○
Conditional responseCSTI (Ours)●●●●●

Table 1.

Safety detection during generation. Recent methods inspect intermediate model states to identify unsafe content before the final image is available. IGD [41] classifies intermediate predicted noise with a lightweight detector. Wukong [17] reads early U-Net features through a safety decoder that reuses pretrained cross-attention parameters. FlowGuard [42] uses linear latent decoding to map intermediate states into an approximate image space before applying a visual safety classifier. These approaches establish the feasibility of generation-time safety detection through different internal representations. Their observation signals and readout mechanisms vary, motivating a closer examination of whether native guidance responses contain directly readable safety information. DAG [14] identifies harmful concepts from noisy latents using cross-attention maps associated with optimized guideline tokens, and then applies adaptive safety guidance. InGuard [39] combines prompt embedding screening with a detector on an estimated clean latent during denoising.

CSTI examines a different source of safety information. It reads paired conditional and unconditional internal responses already computed by native classifier-free guidance. Compact channel statistics from a few early updates are used with a linear concept readout, without intermediate image reconstruction or an auxiliary attention-based safety decoder. Unlike methods that steer generation, CSTI terminates flagged requests and leaves accepted sampling trajectories unchanged.

Generative representations and safety calibration. Research on diffusion features shows that internal generative representations can support discriminative tasks beyond image synthesis. DIFT [29] extracts intermediate diffusion features for visual correspondence, while Diffusion Hyperfeatures [19] combines representations across layers and timesteps for semantic matching. These studies motivate the use of native model features, although our focus is on early safety decisions rather than visual correspondence. A risk score also requires a calibrated threshold before it can support an interception decision. Discriminative performance alone does not specify how often benign requests will be rejected at a chosen operating point. Conformal methods provide a principled framework for calibrating decisions using held-out data [1]. CSTI fits its concept head on canonical risk and safe examples, then determines the operating threshold from an independent safe calibration set. This separation avoids selecting thresholds on the adversarial collections used for evaluation.

Method

We introduce Conditional Semantic Trajectory Interception (CSTI) to detect unsafe inputs from native internal responses before image synthesis is complete. CSTI summarizes conditional responses at a few early sampling updates, uses a lightweight concept readout, and calibrates its interception decision on independent safe examples. The generator remains frozen throughout fitting and deployment. Figure 2 illustrates the framework.

Overview of CSTI

Figure 2. Overview of CSTI. (a) Native CFG provides paired conditional and unconditional activations at the same generation state. Their difference forms a conditional contrast. (b) Compact channel statistics at early checkpoints form a multi-checkpoint representation, illustrated here using updates 5, 10, and 15. (c) A lightweight concept head is fitted from canonical risk and safe examples, while an independent safe set calibrates the interception threshold. In this example, CSTI decides once at update 15.

Consider a text-to-image generator GG, a prompt pp, a random seed rr, and a defender-specified risk concept cc. Let z(0)z^{(0)} be the initial generation state and z(t)z^{(t)} the state after sampling update tt, with TT updates in total. At update tt, the generator evaluates its model using the preceding state z(t−1)z^{(t-1)}. CSTI makes one interception decision after an early update h<Th < T. We use T=50T = 50 and h=15h = 15 throughout the main evaluation.

Early Safety Signals in Native Conditional Responses

Classifier-free guidance (CFG) evaluates conditional and unconditional responses from the same generation state at each sampling update. CSTI reads activations from a fixed intermediate block in these two native branches rather than reconstructing intermediate images. Let HtpH_t^p and Ht∅H_t^\varnothing denote the activations conditioned on pp and on the generator’s native unconditional input, respectively. Both are computed from z(t−1)z^{(t-1)} at the same scheduler timestep.

We define their conditional contrast as

Dt=Htp−Ht∅.(1)D_t = H_t^p - H_t^\varnothing. \tag*{(1)}

This contrast describes the change in internal response associated with the prompt condition. Since both branches are already evaluated by CFG, extracting DtD_t requires no additional risk-specific generation branch.

For U-Net generators, DtD_t consists of spatial features, whereas Transformer generators provide image-token features. Let CC denote the channel dimension and Ω\Omega the set of spatial positions or image tokens. We summarize each channel jj by its mean and root mean square (RMS),

μt,j=1∣Ω∣∑u∈ΩDt,j,u,\mu_{t,j} = \frac{1}{|\Omega|}\sum_{u\in\Omega}D_{t,j,u},
ρt,j=1∣Ω∣∑u∈ΩDt,j,u2+ϵ.(2)\rho_{t,j} = \sqrt{\frac{1}{|\Omega|}\sum_{u\in\Omega}D_{t,j,u}^{2}+\epsilon}. \tag*{(2)}

The mean captures the signed average response, while RMS captures its magnitude; ϵ>0\epsilon> 0 ensures numerical stability. We form a compact representation at each checkpoint,

ϕ(Dt)=[μt;ρt]∈R2C.(3)\phi(D_t) = [\mu_t;\rho_t] \in\mathbb{R}^{2C}. \tag*{(3)}

We collect these features at mm early checkpoints K={t1,…,tm}\mathcal{K} = \{t_1,\ldots,t_m\} and concatenate them in sampling order,

x=[ϕ(Dt1);…;ϕ(Dtm)]∈R2mC.(4)x = [\phi(D_{t_1});\ldots;\phi(D_{t_m})] \in\mathbb{R}^{2mC}. \tag*{(4)}

In our experiments, we use K={5,10,15}\mathcal{K} = \{5,10,15\}. We call this ordered representation a conditional semantic trajectory. It retains information from several early conditional responses. The observation block and channel dimension vary across generators, but the same statistical representation applies to both U-Net and Transformer backbones.

Concept Adaptation without Attack Examples

CSTI fits a risk readout for each generator and defender-specified concept using canonical concept examples and safe examples. The positive examples represent the target concept and are not selected according to the success of an adversarial attack. No attack prompts, attack-generated images, or attack-success labels are used for detector fitting.

Given the early representation xx, we standardize its features and apply a whitened PCA projection with 128 components,

v(x)=PCA⁡128white(Std⁡(x)).(5)v(x) = \operatorname{PCA}_{128}^{\mathrm{white}}\left(\operatorname{Std}(x)\right). \tag*{(5)}

A linear head then produces a concept-specific risk score,

sc(x)=wcTv(x)+bc.(6)s_c(x) = w_c^{\mathsf{T}}v(x) + b_c. \tag*{(6)}

We use class-balanced logistic regression and take its linear decision value as the risk score, with larger values indicating greater risk. The standardizer, PCA projection, and classifier are fitted sequentially using only the designated fitting split. This simple readout allows us to examine how directly the early representation exposes safety information, without introducing a complex detection network.

Each generator and risk concept uses its own fitted preprocessing and readout parameters. CSTI uses the same feature construction and fitting procedure across architectures, while detector weights are learned separately for each generator and risk concept instead of being shared.

Safe-Calibrated Early Interception

A risk score must be converted into an interception decision. We determine its operating threshold using an independent set of safe examples that is not used for concept fitting. After fitting the readout, let s1,…,sncals_1,\ldots,s_{n_{\mathrm{cal}}} denote the risk scores of these calibration examples. For a test risk score ss, we compute its upper-tail rank relative to the safe calibration scores,

psafe(s)=1+∑i=1ncal1[si≥s]ncal+1.(7)p_{\mathrm{safe}}(s) = \frac{1 + \sum_{i=1}^{n_{\mathrm{cal}}} \mathbf{1}[s_i \ge s]}{n_{\mathrm{cal}} + 1}. \tag*{(7)}

A small value indicates that the test score is unusually high relative to the safe reference distribution. Given an operating level α\alpha, the interception decision is

fc(p,r;G)=1[psafe(sc(x))≤α].(8)f_c(p,r;G) = \mathbf{1}\left[p_{\mathrm{safe}}\left(s_c(x)\right) \le\alpha\right]. \tag*{(8)}

The fitted readout and calibration rule remain fixed across attack collections for each generator and concept. The quantity psafep_{\mathrm{safe}} is a ranking score, not the probability that the input is benign. When calibration and test safe examples are exchangeable, this rank rule bounds the marginal false-positive rate at the chosen level α\alpha [1]. Calibration relies only on benign examples to control false positives. CSTI aims to intercept risky inputs regardless of attack success, as even unsuccessful attacks may degrade output quality.

During sampling, CSTI records summaries at updates 5, 10, and 15 and makes a single decision after update h=15h=15. Flagged generations stop before final image decoding, while accepted generations proceed from the current state with their guidance and scheduler unchanged. No intermediate image decoding or external visual encoder is required. Under our 50-update protocol, an intercepted generation avoids the remaining 35 sampling updates. The corresponding runtime reduction is measured separately in § 4.

Experiments

We evaluate early interception, the readability of internal safety signals, and the cost of using those signals during generation. The main evaluation covers five generators and eight risk-prompt settings, including six for nudity and two for violence, yielding 40 evaluation settings. We further evaluate over-refusal on OVERT-mini using two generators and challenging benign prompts.

Experimental Setup

Datasets. We evaluate six nudity prompt collections: Ring-A-Bell (RAB) [31], MMA-Diffusion [43], Custom, P4D [4], I2P [24], and UnLearn source [55]. For violence, we use RAB and I2P. For each collection, we include an equal number of benign prompts to evaluate false positives on legitimate requests. For over-refusal, we use OVERT-mini [3].

Models. We evaluate five unmodified text-to-image generators: SD1.4, SD1.5, SDXL, SD3.5 Medium, and full Z-Image. All use 50 sampling updates. SD1.4/1.5 and SDXL use DDIM [25] with CFG 7.5 at 5122512^2 and 102421024^2 resolution, respectively. SD3.5 Medium and Z-Image use FlowMatch Euler [16] at 102421024^2, with CFG 7.5 and 5, respectively. Experiments run on an NVIDIA RTX 4090.

Implementation. CSTI observes updates {5,10,15}\{5,10,15\} and makes a single decision at update 15. We extract features from the U-Net middle block, SD3.5 Transformer block 11, or Z-Image layer 14, respectively. Each generator–concept pair uses a separate detector fitted on 144 canonical concept examples and 256 benign examples, without attack samples. Features undergo standardization and 128-component whitened PCA, followed by balanced logistic regression. Independent benign sets are used for safety calibration (200 prompts, α=0.075\alpha= 0.075) and evaluation (300 prompts).

Metrics. We use TPR at a matched empirical 5% FPR as the primary metric, with AUROC as a complementary measure. TPR measures the proportion of unsafe inputs correctly detected. These automated assessments were further supplemented by manual review. The 5% FPR threshold is determined from the 300 benign evaluation prompts (at most 15 false positives), separately from the safety calibration rule. We average collections within each concept, then concepts and generators equally.

Baselines. We compare against three prompt-level detectors: NSFW Text Classifier [21], Latent Guard [18], and GuardT2I [44]; and three generation-time detectors: IGD [41], Wukong [17], and DAG [14] (detection component only). We follow released implementations where applicable, with architecture-specific adaptations identified in the comparison.

Early Interception Effectiveness

Table 2 compares CSTI with six representative baselines at a matched 5% FPR. CSTI achieves the highest TPR in 37 of 40 evaluation settings, with macro TPRs ranging

GeneratorMethodRABMMACustomP4DI2PUnLearnRABI2PMacro ↑
SD1.4Text Classifier26.2845.6723.5012.2110.8713.9035.1320.5424.95
Latent Guard52.3556.4255.1855.8320.1520.7454.8619.3240.27
GuardT2I84.6256.1866.2448.3550.2747.6224.8617.3239.99
IGD96.9693.7985.2796.4996.8396.9797.1877.2890.80
Wukong80.0084.7080.8989.5286.7176.7593.7776.7884.18
DAG†84.3178.8055.1163.3087.0936.7238.9023.2749.31
CSTI (Ours)100.0095.64100.0095.6998.5199.30100.0083.0794.86
SD1.5Text Classifier26.2845.6723.5012.2110.8713.9035.1320.5424.95
Latent Guard51.8256.3055.4255.6120.4320.3654.7219.8740.31
GuardT2I83.7455.4265.8346.9150.0848.1525.1317.6539.87
IGD91.6387.8586.4193.5995.4597.2291.2772.8787.05
Wukong87.0077.9289.7595.2290.0889.3286.7884.3286.88
DAG†88.9482.6777.2865.5791.4342.1336.9329.8954.04
CSTI (Ours)100.0090.26100.0095.6999.2597.89100.0084.6694.76
SDXLText Classifier26.2845.6723.5012.2110.8713.9035.1320.5424.95
Latent Guard53.1464.8755.0656.0422.8624.1554.9421.0342.00
GuardT2I84.3560.8766.5247.2048.3651.3425.8717.4340.71
IGD93.5172.5777.9484.2689.3686.4691.0676.8283.81
Wukong80.4181.9464.8093.3692.9990.8877.1074.9280.03
DAG†43.3232.9057.5571.4365.8858.9329.2131.6542.71
CSTI (Ours)100.0092.18100.0095.69100.0097.8999.4784.2794.75
SD3.5 MediumText Classifier26.2845.6723.5012.2110.8713.9035.1320.5424.95
Latent Guard60.2866.3154.9155.7424.7224.9854.8820.7642.82
GuardT2I85.1662.4367.9151.2854.7252.8625.4216.8441.76
IGD89.5359.3236.4970.3784.1386.2189.8671.8375.92
Wukong85.8070.5362.6079.2082.7078.4181.5275.6777.56
DAG†85.5872.2844.6852.6379.1784.2168.6331.9060.01
CSTI (Ours)98.6078.3598.0690.9195.07100.0096.9385.6592.39
Z-ImageText Classifier26.2845.6723.5012.2110.8713.9035.1320.5424.95
Latent Guard54.7360.4555.2155.4221.8721.6554.7920.1841.19
GuardT2I84.2755.8469.3547.6150.1548.9224.7817.8640.34
IGD77.2636.8571.4381.9092.2590.4886.1382.8779.76
Wukong91.9296.3048.2985.9779.2288.1170.5070.9876.19
DAG†94.7265.0777.1075.8694.8796.8824.5628.9255.41
CSTI (Ours)98.9592.90100.0092.8296.2795.0799.3391.6795.75

Table 2. Main comparison of unsafe-input detection at a matched 5% FPR. We report TPR (%, ↑) on unsafe inputs across five text-to-image generators, six nudity collections, and two violence collections. Macro averages collections within each concept and then weights the two concepts equally. Bold and underlined values indicate the best and second-best results, respectively.

from 92.39% on SD3.5 Medium to 95.75% on Z-Image. It outperforms the strongest competing baseline on each generator by 4.06 to 15.99 percentage points. These improvements hold across both U-Net and Transformer architectures, demonstrating that early conditional responses provide effective safety signals across different generation backbones. Performance remains dependent on the prompt collection, with relatively lower TPRs on MMA for SD3.5 Medium (78.35%) and I2P violence for SDXL (84.27%).

Characterizing Early Safety Readability

We investigate where safety information is accessible, how early it can be detected, and how complex a readout is required. Figure 3 summarizes these analyses across five generators and two risk concepts.

Early safety readability across layers, sampling stages, and classifiers

Figure 3. Early safety readability across layers, sampling stages, and classifiers. (a) Layer-wise TPR at a matched 5% FPR across five generators and two risk concepts. (b) TPR of individual checkpoints and their joint representation at a matched 5% FPR. (c) Mean empirical FPR under the standard safe calibration protocol, with each variant fitted and calibrated independently rather than matched to 5% FPR. (d) TPR comparison of classifier families using the same early representation at a matched 5% FPR. Results are aggregated across generators and prompt collections as appropriate.

Layer selection. Figure 3(a) shows that safety information is accessible across multiple network layers, with in termediate layers generally providing stronger readability. On SDXL, nudity TPR increases from 73.57% at the earliest examined layer to 97.62% at the middle block. Similarly, violence TPR on Z-Image rises from 42.41% at layer 0 to 95.50% at layer 14. Although the strongest observation site varies across generators and concepts, these results show that internal representations contain substantial safety information.

Sampling stages. Figure 3(b) shows that safety information is already strongly readable at update 5, achieving 88.4% TPR for nudity and 85.2% for violence at a matched 5% FPR. Combining updates 5, 10, and 15 increases the overall macro TPR from 92.2% at update 15 alone to 94.5%.

Figure 3(c) reports mean empirical FPR under the standard calibration protocol. Each variant uses its own fitted classifier and calibrated threshold, without independently adjusting its empirical FPR to 5%. The joint representation achieves a mean empirical FPR of 4.2%, compared with 15.9%, 13.5%, and 10.4% for updates 5, 10, and 15, respectively. These results demonstrate that strong safety information is available at very early stages, while combining checkpoints provides complementary information and reduces false positives under the evaluated operating setting.

Readout complexity. Figure 3(d) compares linear and nonlinear classifiers on the same representation. Logistic regression achieves 96.5% and 92.5% TPR for nudity and violence, respectively, outperforming SVM, MLP, and random forest in the overall averages. This confirms that compact conditional features support effective safety detection with a simple linear readout.

Feature statistics. Table 3 evaluates the channel statistics used to construct the early representation. Combining mean and RMS achieves 94.50% macro TPR and 0.9802 AUROC, compared with 85.93% and 82.56% TPR using mean or RMS alone. These results further support the effectiveness of simple statistical summaries for extracting safety information from native conditional responses.

Table 3. Feature-statistics ablation. Hierarchical macro over five generators, two risk concepts, and their prompt collections. Each variant independently fits the same standardization, PCA, and logistic regression pipeline.

StatisticsTPR@5 ↑AUROC ↑
Mean only85.930.9170
RMS only82.560.8846
Mean + RMS (default)94.500.9802

Table 3. Feature-statistics ablation.

Over-Refusal Evaluation

The matched FPR evaluation on standard benign prompts does not fully characterize false interceptions when safe requests resemble protected concepts. Legitimate requests should proceed without unnecessary rejection. To examine this boundary, we evaluate CSTI on OVERT-mini [3], a challenging benchmark of benign prompts that closely resemble unsafe concepts. We select SD1.4 and SD3.5 to cover both U-Net and Transformer architectures. All trainable methods are adapted using the same 200 OVERT samples and evaluated on 1,600 samples, while other baselines retain their best released configurations.

Table 4 reports the resulting FPR. CSTI achieves the lowest FPR of 5.27% on SD1.4, outperforming all six baselines. On SD3.5, CSTI reaches 8.25% FPR, ranking second and trailing DAG by only 0.14 percentage points. It also substantially outperforms the remaining five baselines, whose FPRs range from 14.20% to 38.43%. These results suggest that CSTI can avoid false rejections of challenging benign inputs with limited additional adaptation. This capability is particularly important for early interception, where incorrect decisions terminate otherwise legitimate generation processes. Nevertheless, false positives on semantically similar benign prompts remain a limitation, particularly on SD3.5, indicating the need for further improvements in distinguishing benign and unsafe inputs.

Table 4. False-positive comparison on OVERT. We report FPR (%, ↓) on 1,600 near-concept samples. Trainable methods use the same 200 OVERT samples for adaptation; others use their best released configurations. The text-only classifier is evaluated once on shared prompts, with identical FPR across generators.

MethodSD1.4SD3.5 Medium
Text Classifier38.4338.43
Latent Guard16.3814.20
GuardT2I32.5026.13
IGD15.7816.24
Wukong20.3818.21
DAG†7.788.11
CSTI (Ours)5.27<u>8.25</u>

Table 4. False-positive comparison on OVERT.

Computational Efficiency

We evaluate the computational efficiency of CSTI across five generators under the 50-update sampling protocol. Table 5 shows that early termination at update 15 reduces execution latency by 61.41% to 71.72% across U-Net and Transformer architectures. Early termination also reduces peak GPU memory on SD1.4, SD1.5, and SDXL, while memory usage remains unchanged on SD3.5 Medium and Z-Image.

Table 5. Computational efficiency across five text-to-image generators. We report median latency for full generation and early termination at update 15. Savings are median paired reductions. Peak GPU memory is reported in the order of standard generation, monitored full generation, and early termination.

GeneratorFull (s)Early Stop (s)Saving (%) ↑Peak GPU (GiB)
SD1.42.6740.78970.563.49/3.49/2.12
SD1.52.3780.70970.223.49/3.49/2.11
SDXL6.6001.86771.7218.49/18.50/7.09
SD3.5 Medium21.2478.21361.1112.06/12.06/12.06
Z-Image62.36023.48862.2112.52/12.52/12.52

Table 5. Computational efficiency across five text-to-image generators.

Conclusion

We investigated early safety signals in text-to-image diffusion models and found that native conditional responses enable effective detection of unsafe and adversarial inputs through compact statistics and simple linear readouts. Building on this finding, we introduced CSTI for calibrated early interception without attack samples, intermediate image decoding, or modification of accepted sampling. Experiments across five generators spanning U-Net and Transformer architectures demonstrate strong overall detection performance against representative baselines. Further analyses show that safety signals are accessible at early sampling stages and across multiple layers, while simple linear readouts remain effective. These findings demonstrate that native guidance computation provides an effective and efficient foundation for early safety interception.

Broader Context and Related Work

Adversarial robustness and AI security. The security of deep learning systems has been extensively investigated across diverse adversarial settings. Existing research has explored adversarial perturbations and robustness [2, 8, 13, 20], as well as transferable attacks against visual and multimodal representations [26, 27, 37]. Related studies have also investigated adversarial manipulation of training data and learned representations, including unlearnable examples [33, 36]. Beyond inference-time attacks, training-time threats such as data poisoning and model poisoning [15, 50], as well as backdoor attacks [10, 51, 56], can compromise learned representations and induce unintended model behaviors. Corresponding detection mechanisms have also been investigated across different learning architectures and tasks [45, 52]. These efforts highlight the broader importance of identifying adversarially induced behavior and developing reliable safeguards for increasingly capable AI systems.

Safety of foundation and generative models. As AI systems integrate language, vision, and decision-making capabilities, safety concerns have expanded beyond conventional discriminative prediction. Recent research has examined jailbreak attacks against large language models [54, 57], adversarial prompting and prompt injection in multimodal and agentic systems [9, 32, 49], and adversarial behavioral manipulation of embodied agents [34, 38, 53]. Related security challenges also arise in visual generative models, where adversarial manipulation can affect image editing [48], visual synthesis [35], and the generation of unsafe or misleading content [40, 46]. While these settings differ in their threat models and attack objectives, they underscore the need for safety mechanisms that extend beyond conventional input screening. In particular, generative models progressively construct their outputs through multiple computational stages, raising the question of whether emerging risks can be identified before synthesis is complete.

Safety defenses for text-to-image generation. Safety mechanisms for text-to-image generation operate at different stages of the generation pipeline. Before sampling, text classifiers such as the NSFW Text Classifier [21] and Detoxify [11] can screen potentially unsafe requests. Latent Guard [18] detects harmful concepts in text encoder representations, while GuardT2I [44] examines textual interpretations of guidance embeddings for adversarial prompt detection. Such screening can reject risky requests before synthesis, but does not observe how the generative state evolves during sampling. Other defenses directly modify the generation process. ESD [7] removes target concepts by updating model weights, whereas SLD [24] and CRG [5] suppress unsafe concepts through inference-time guidance. Nevertheless, adversarial prompting approaches such as Ring-A-Bell [31] and MMA-Diffusion [43] demonstrate that unsafe content can still be elicited despite existing safeguards. These limitations motivate safety assessment during generation, without modifying accepted sampling trajectories.

Intermediate representations and early safety detection. Recent studies investigate whether intermediate diffusion states can reveal safety-relevant information before the final image is available. IGD [41] classifies intermediate predicted noise, while Wukong [17] detects unsafe content from early U-Net features using an auxiliary safety decoder. FlowGuard [42] projects intermediate latents into an approximate image space for visual safety assessment. DAG [14] exploits cross-attention signals for harmful concept identification and adaptive guidance, whereas InGuard [39] combines prompt screening with detection on estimated clean latents. These approaches demonstrate the feasibility of assessing generation risk from intermediate computations, although their observation signals and intervention mechanisms differ.

More broadly, diffusion representations have been shown to capture semantic information useful for discriminative tasks beyond image generation. DIFT [29] and Diffusion Hyperfeatures [19] demonstrate the utility of intermediate features across sampling stages and network layers. Such findings motivate examining the safety information encoded within the generator’s native computations. Moreover, reliable interception requires not only a discriminative risk score but also a calibrated decision threshold, for which held-out calibration provides a principled foundation [1].

In contrast to approaches that require intermediate image reconstruction, auxiliary safety decoders, or modification of the generation process, CSTI examines the paired conditional and unconditional activations already computed by classifier-free guidance. By aggregating compact channel statistics across early sampling checkpoints, CSTI extracts safety-relevant signals using a lightweight linear readout. An independent safe calibration set determines the interception threshold, while flagged generations terminate early and accepted trajectories remain unchanged. This design exploits native conditional computation to support efficient safety assessment before image synthesis is complete.

References

  1. [1]Anastasios N. Angelopoulos and Stephen Bates. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511, 2021. 3, 5, 12arxiv.org/abs/2107.07511
  2. [2]Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pages 39–57. IEEE, 2017. 11arxiv.org/abs/1608.04644
  3. [3]Ziheng Cheng, Yixiao Huang, Hui Xu, Somayeh Sojoudi, Xuandong Zhao, Dawn Song, and Song Mei. OVERT: A benchmark for over-refusal evaluation on text-to-image models. In Advances in Neural Information Processing Systems, 2025. 5, 8arxiv.org/abs/2505.21347
  4. [4]Zhi-Yi Chin, Chieh Ming Jiang, Ching-Chun Huang, Pin-Yu Chen, and Wei-Chen Chiu. Prompting4Debbuging: Red-teaming text-to-image diffusion models by finding problematic prompts. In Proceedings of the 41st International Conference on Machine Learning, pages 8468–8486, 2024. 5arxiv.org/abs/2309.06135
  5. [5]Yoonseok Choi, Chaeyoung Oh, Hyunjun Choi, Seokin Seo, and Kee-Eung Kim. Concept removal guidance: Evidence-calibrated negative guidance for safe diffusion sampling. In Proceedings of the 43rd International Conference on Machine Learning, pages 20250–20278, 2026. 2, 3, 11
  6. [6]Patrick Esser, Smith K u lal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis. In Proceedings of the 41st International Conference on Machine Learning, pages 12606–12633, 2024. 2arxiv.org/abs/2403.03206
  7. [7]Rohit Gandikota, Joanna Materzynska, Jaden Fiotto-Kaufman, and David Bau. Erasing concepts from diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2426–2436, 2023. 2, 3, 11arxiv.org/abs/2303.07345
  8. [8]Ian J Goodfellow, Jonathan Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014. 11DOI
  9. [9]Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM workshop on artificial intelligence and security, pages 79–90, 2023. 11arxiv.org/abs/2302.12173
  10. [10]Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733, 2017. 11arxiv.org/abs/1708.06733
  11. [11]Laura Hanu and Unitary team. Detoxify. GitHub repository, 2020. Accessed September 29, 2026. 2, 3, 11
  12. [12]Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 1arxiv.org/abs/2207.12598
  13. [13]Jianyi Hu, Hangtao Zhang, Yi Liu, Yeqi Zeng, Li Zeng, Xianlong Wang, Rui Wang, and Leo Yu Zhang. Jevad-bench: A benchmark and black-box attacks for reinforcement learning for calibrated decisions models. arXiv preprint arXiv:2609.31142, 2026. 11arxiv.org/abs/2609.31142
  14. [14]Feifei Li, Mi Zhang, Yiming Sun, and Min Yang. Detect-and-Guide: Self-regulation of diffusion models for safe text-to-image generation via guideline token optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. 3, 6, 11arxiv.org/abs/2503.15197
  15. [15]Minghui Li, Hongtao Zhang, Yanjun Zhang, Li Zeng, Chao Chen, Qiyun Shao, Wei Wan, Shengshan Hu, and Leo Yu Zhang. Fine-grained poisoning framework against federated learning. IEEE Transactions on Dependable and Secure Computing, 2025. 11
  16. [16]Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In International Conference on Learning Representations, 2025. 5
  17. [17]Mingrui Liu, Sixiao Zhang, and Cheng Long. Wukong framework for not safe for work detection in text-to-image systems. In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 891–902, 2026. 1, 3, 6, 11arxiv.org/abs/2508.00591
  18. [18]Runtao Liu, Ashkan Khakzar, Jindong Gu, Qifeng Chen, Philip Torr, and Fabio Zizzi. Latent Guard: A safety framework for text-to-image generation. In Computer Vision – ECCV 2024, pages 93–109. Springer, 2024. 2, 3, 6, 11arxiv.org/abs/2404.08031
  19. [19]Grace Luo, Lisa Dunlap, Dong Huk Park, Aleksander Holynski, and Trevor Darrell. Diffusion hyperfeatures: Searching through time and space for semantic correspondence. In Advances in Neural Information Processing Systems, 2023. 3, 12arxiv.org/abs/2305.14334
  20. [20]Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017. 11arxiv.org/abs/1706.06083
  21. [21]michellejieli. NSFW text classifier. Hugging Face model repository, 2022. Accessed September 29, 2026. 2, 3, 6, 11
  22. [22]Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. In International Conference on Learning Representations, 2024. 2arxiv.org/abs/2307.01952
  23. [23]Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022. 1, 2
  24. [24]Patrick Schramowski, Manuel Brack, Björn Deiseroth, and Kristian Kersting. Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22522–22531, 2023. 2, 3, 5, 11arxiv.org/abs/2211.05105
  25. [25]Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021. 5arxiv.org/abs/2010.02502
  26. [26]Yufei Song, Ziqi Zhou, Minghui Li, Xianlong Wang, Hangtao Zhang, Menghao Deng, Wei Wan, Shengshan Hu, and Leo Yu Zhang. Pb-uap: Hybrid universal adversarial attack for image segmentation. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2025. 11arxiv.org/abs/2412.16651
  27. [27]Yufei Song, Ziqi Zhou, Qi Lu, Hangtao Zhang, Yifan Hu, Lulu Xue, Shengshan Hu, Minghui Li, and Leo Yu Zhang. SegTrans: Transferable adversarial examples for segmentation models. IEEE Transactions on Multimedia, 2026. 11arxiv.org/abs/2510.08922
  28. [28]Stability AI. Stable Diffusion 3.5 Medium: Model card. Hugging Face model repository, 2024. Accessed October 11, 2026. 2
  29. [29]Luming Tang, Menglin Jia, Qianqian Wang, Cheng Perng Phoo, and Bharath Hariharan. Emergent correspondence from image diffusion. In Advances in Neural Information Processing Systems, 2023. 3, 12arxiv.org/abs/2306.03881
  30. [30]Tongyi-MAI. Z-Image: Full foundation model. Hugging Face model repository, 2025. Accessed October 11, 2026. 2
  31. [31]Yu-Lin Tsai, Chia-Yi Hsu, Chulin Xie, Chih-Hsun Lin, Jia You Chen, Bo Li, Pin-Yu Chen, Chia-Mu Yu, and Chun-Ying Huang. Ring-A-Bell: how reliable are concept removal methods for diffusion models? In International Conference on Learning Representations, 2024. 1, 2, 5, 11arxiv.org/abs/2310.10012
  32. [32]Junhui Wang, Hangtao Zhang, Zhirun Zheng, Li Zeng, Jiejun Xiao, Xi Luo, Lihua Yin, and Saiqin Long. Pvdtector: Detecting prompt injection attacks on purpose-specific llm agents through policy-violation concept analysis. In Proceedings of the 34th ACM International Conference on Multimedia, 2026. 11
  33. [33]Xianglong Wang, Minghui Li, Wei Liu, Hangtao Zhang, Shengshan Hu, Yechao Zhang, Ziqi Zhou, and Hai Jin. Unlearnable 3d point clouds: Class-wise transformation is all you need. Advances in Neural Information Processing Systems, 37:99404–99432, 2024. 11arxiv.org/abs/2410.03644
  34. [34]Xianlong Wang, Hewen Pan, Hangtao Zhang, Minghui Li, Shengshan Hu, Ziqi Zhou, Lulu Xue, Aishan Liu, Yunpeng Jiang, Leo Yu Zhang, et al. Trojanrobot: Physical-world backdoor attacks against vlm-based robotic manipulation. arXiv preprint arXiv:2411.11683, 2024. 11
  35. [35]Xianlong Wang, Wenbo Pan, Shijia Zhou, Ke Li, Yuqi Wang, Zeyu Ye, Hangtao Zhang, Leo Yu Zhang, and Xiaohua Jia. Image-to-video diffusion: From foundations to open frontiers. arXiv preprint arXiv:2605.17248, 2026. 11arxiv.org/abs/2605.17248
  36. [36]Xianlong Wang, Hangtao Zhang, Wenbo Pan, Ziqi Zhou, Changsong Jiang, Li Zeng, and Xiaohua Jia. Dual-branch robust unlearnable examples. arXiv preprint arXiv:2605.01718, 2026. 11arxiv.org/abs/2605.01718
  37. [37]Yichen Wang, Yixuan Chou, Ziqi Zhou, Hangtao Zhang, Wei Wan, Shengshan Hu, and Minghui Li. Breaking barriers in physical-world adversarial examples: Improving robustness and transferability via robust feature. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 8069–8077, 2025. 11arxiv.org/abs/2412.16958
  38. [38]Yichen Wang, Hangtao Zhang, Hewen Pan, Ziqi Zhou, Xianlong Wang, Peijin Guo, Lulu Xue, Shengshan Hu, Minghui Li, and Leo Yu Zhang. Advmedm: Fine-grained adversarial attack against vlm-based embodied agents. Advances in Neural Information Processing Systems, 38:136551–136575, 2026. 11arxiv.org/abs/2509.16645
  39. [39]Zeyu Wang, Xiaodan Li, Zhiwen Li, Yuefeng Chen, and Hui Xue. InGuard: Towards generalized inner guardrail for safe text-to-image generation. arXiv preprint arXiv:2609.27620, 2026. 3, 11arxiv.org/abs/2609.27620
  40. [40]Meng Xie, Li Zeng, Hangtao Zhang, Xianlong Wang, Ziqi Zhou, Pengpeng Qiao, and Zhetao Li. Typo: Instruction-dense visual jailbreaks against commercial closed-source image-generation models. arXiv preprint arXiv:2607.24897, 2026. 11arxiv.org/abs/2607.24897
  41. [41]Fan Yang, Yihao Huang, Jiayi Zhu, Ling Shi, Gequang Pu, Jin Song Dong, and Kailong Wang. Seeing it before it happens: In-generation NSFW detection for diffusion-based text-to-image models. arXiv preprint arXiv:2508.03006, 2025. 1, 3, 6, 11arxiv.org/abs/2508.03006
  42. [42]Jinghan Yang, Yihe Fan, Xudong Pan, and Min Yang. Flow-Guard: Towards lightweight in-generation safety detection for diffusion models via linear latent decoding. arXiv preprint arXiv:2604.07879, 2026. 1, 3, 11arxiv.org/abs/2604.07879
  43. [43]Yijun Yang, Ruiyuan Gao, Xiaosen Wang, Tsung-Yi Ho, Nan Xu, and Qiang Xu. MMA-Diffusion: MultiModal attack on diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7737–7746, 2024. 1, 2, 5, 11arxiv.org/abs/2311.17516
  44. [44]Yijun Yang, Ruiyuan Gao, Xiao Yang, Jianyuan Zhong, and Qiang Xu. GuardT2I: Defending text-to-image models from adversarial prompts. In Advances in Neural Information Processing Systems, 2024. 2, 6, 11
  45. [45]Zeming Yao, Hangtao Zhang, Yicheng Guo, Xin Tian, Wei Peng, Yi Zou, Leo Yu Zhang, and Chao Chen. Reverse backdoor distillation: Towards online backdoor attack detection for deep neural network models. IEEE Transactions on Dependable and Secure Computing, 21(6):5098–5111, 2024. 11DOI
  46. [46]Zeyu Ye, Yanchun Li, Sibei He, Meng Xie, Hangtao Zhang, Xianlong Wang, Li Zeng, Jiahao Chen, Yichen Wang, Junhui Wang, et al. False claims, credible images: A red-teaming benchmark for commercial image generators. arXiv preprint arXiv:2610.11112, 2026. 11arxiv.org/abs/2610.11112
  47. [47]Z-Image Team. Z-Image: An efficient image generation foundation model with single-stream diffusion transformer. arXiv preprint arXiv:2511.22699, 2025. 2
  48. [48]Li Zeng, Xiaojun Mo, Meng Xie, Hangtao Zhang, Yixiang Liu, Yezhuo Peng, and Yanchun Li. Psfd: Proactive spatial-frequency defense against malicious exemplar-guided image editing. In 2025 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6. IEEE, 2025. 11
  49. [49]Li Zeng, Zeyu Ye, Meng Xie, Hangtao Zhang, Xianlong Wang, Yanchun Li, and Zhetao Li. Ghostprompt: Cross-image adversarial prompt for vision-language models. In Proceedings of the 34th ACM International Conference on Multimedia, 2026. 11
  50. [50]Hangtao Zhang, Zeming Yao, Leo Yu Zhang, Shengshan Hu, Chao Chen, Alan Liew, and Zhetao Li. Denial-of-service or fine-grained control: Towards flexible model poisoning attacks on federated learning. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, pages 4567–4575, 2023. 11arxiv.org/abs/2304.10783
  51. [51]Hangtao Zhang, Shengshan Hu, Yichen Wang, Leo Yu Zhang, Ziqi Zhou, Xianlong Wang, Yanjun Zhang, and Chao Chen. Detector collapse: Backdooring object detection to catastrophic overload or blindness in the physical world. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 1670–1678, 2024. 11
  52. [52]Hangtao Zhang, Yichen Wang, Shihui Yan, Chenyu Zhu, Ziqi Zhou, Linshan Hou, Shengshan Hu, Minghui Li, Yanjun Zhang, and Leo Yu Zhang. Test-time backdoor detection for object detection models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 24377–24386, 2025. 11arxiv.org/abs/2503.15293
  53. [53]Hangtao Zhang, Chenyu Zhu, Xiaolong Wang, Ziqi Zhou, Changgong Yin, Minghui Li, Lulu Xu, Yichen Wang, Shengshan Hu, Aishan Liu, et al. Badrobot: Jailbreaking embodied llm agents in the physical world. In Proceedings of the Thirteenth International Conference on Learning Representations, 2025. 11arxiv.org/abs/2407.20242
  54. [54]Hangtao Zhang, Yucheng Zhao, Sishun Liu, Ziqi Zhou, Zeyu Ye, Wei Wan, Minghui Li, Shengshan Hu, Yanjun Zhang, Yi Liu, and Leo Yu Zhang. Defending jailbreak attacks on large language models via manifold trajectory kinetics. In 35th USENIX Security Symposium (USENIX Security 26), pages 2147–2166, 2026. 11
  55. [55]Yimeng Zhang, Jinghan Jia, Xin Chen, Aochuan Chen, Yihua Zhang, Jiancheng Liu, Ke Ding, and Sijia Liu. To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images ... for now. In European Conference on Computer Vision, 2024. 5arxiv.org/abs/2310.11868
  56. [56]Ziqi Zhou, Menghao Deng, Yufei Song, Hangtao Zhang, Wei Wan, Shengshan Hu, Minghui Li, Leo Yu Zhang, and Dezhong Yao. Darkhash: A data-free backdoor attack against deep hashing. IEEE Transactions on Information Forensics and Security, 2025. 11arxiv.org/abs/2510.08094
  57. [57]Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043, 2023. 11arxiv.org/abs/2307.15043

Paper details

Publication

vlnc:W_syt2zwvs
v1 · latestHistory (1)

published · Fresh — added to the Hub within the last 30 days

added ·

Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

CC BY 4.0

Original source

Taxonomy

  • Computer Science

Authors

AI contributions

Not recorded for this paper · View statement →

Looking for related papers…
Contents