betaFeedback

WorksComputer Science

SPADE: Soft Prompts for Ai-text DEtection

Abstract

The easy access to Large Language Models (LLMs) has made AI-generated texts on the web increasingly ubiquitous, making AI-text detection a crucial task. Prompts greatly influence the generation process, but existing detectors focus solely on the generated texts — supervised methods learn from labeled texts only, and zero-shot methods rely on the text's probability distributions — ignoring the underlying generation process, which limits their robustness in open-world scenarios. We introduce SPADE: Soft Prompts for Ai-text DEtection, a novel framework for detecting AI-generated text that transfers better across datasets. Specifically, we measure the prompt's influence and use it as a discriminative signal. We characterize the prompt's influence by introducing a Pointwise Mutual Information (PMI) term that captures how much a prompt explains a text. Based on this quantity, SPADE trains one soft prompt per class to amplify the separation, and predicts the class whose prompt best explains the candidate text. Extensive experiments on several benchmarks against various detectors demonstrate the robustness and effectiveness of our method, achieving state-of-the-art (SOTA) performance on out-of-distribution (OOD) datasets.

Introduction

Large Language Models (LLMs) such as GPT (OpenAI et al., 2024), LLaMA (Grattafiori et al., 2024), or Gemini (Team et al., 2025) have transformed text generation by producing artificial content that is reasonable, fluent, and contextually relevant. This breakthrough has dramatically lowered technical barriers to produce automatically artificial contents, enabling large-scale automated disinformation and phishing campaigns, and academic dishonesty. The proportion of synthetic content has increased in some media, from 1% in 2022 to almost 20% in 2024 (Brooks et al., [8]), and

this trend continues. In practice, prior works focus on two possible formulations of the detection task: model attribution, i.e., identifying the source model (Sun et al., 2025), and binary classification, i.e., deciding whether a text is human-written or AI-generated (Solaiman et al., 2019; Zellers et al., 2019). In this paper, we follow the predominant formulation and treat AI-text detection as a binary classification task.

LLM generation is inherently conditioned on the input prompt, and different prompts can induce substantially different output distributions and textual characteristics (Jiang et al., 2020). In practice, recovering the exact generative context of either class is infeasible, as it remains unknown at test time. As a result, existing detectors often conflate prompt-induced characteristics with those intrinsic to LLM generation, leading to inconsistent performance across prompts (Koike et al., 2024).

Most classical detectors do not explicitly account for this conditioning variable. Supervised detectors learn from labeled data using pre-trained models or hand-crafted features (Guo et al., 2023; Verma et al., 2024). They perform well in-domain (Macko et al., 2025), but are typically trained on AI-generated texts elicited by a limited set of hard prompts, potentially capturing prompt-specific patterns that hinder generalization to unseen prompts. On the other hand, zero-shot detectors (Bao et al., 2024; Hans et al., 2024) use statistical measures computed from the logits of some scoring models. They generalize better but rely on scoring models that closely resemble the source models (Sun et al., 2026), and overlook the influence of prompts on generation distributions by deriving detection signals solely from the generated text.

Instead, we find that conditioning text likelihood (PMI) on even a generic instruction can substantially improve the separation between AI-generated and human-written texts, and this holds across different language models (see Figure 1). This suggests that the generative context provides discriminative information beyond text likelihood alone. Based on this, we hypothesize that each class is characterized by a distinct latent context: AI-generated texts are conditioned on instructions, whereas human-written texts reflect a different, implicit writing context.

Figure 1. Histograms of the negative log-likelihood (NLL) (above) and PMI (below) using a generic instruction prompt "Make this text more formal and professional:" across three backbones (Qwen, LLaMA, Falcon) on MIRAGE-DIG (Fu et al., 2025) test split. The prompt’s influence PMI is a better discriminative signal than NLL.

In this line, regeneration-based methods—both supervised (Mao et al., 2024; Yu et al., 2024; Hao et al., 2025) and zero-shot methods (Yang et al., 2024)—try to bring the conditioning variable to the detection task. They prompt an LLM to regenerate a given text, and then compare the regenerated texts and the given text through features. All of them focus on using a hard prompt, and they use it only to reproduce the text rather than to measure how well it explains the text.

Since the generative context is unknown at test time, and human-written texts have no explicit prompt. We instead address this challenge by learning two class-specific soft prompts that approximate the latent contexts underlying AI-generated and human-written texts, respectively. We term this approach SPADE: Soft Prompts for Ai-text DEtection. We further use the log-likelihood ratio under these two learned contexts as a detection score, measuring which class-specific context better explains a given text.

Extensive experiments on two benchmarks (EvoBench and RAID) confirm the effectiveness, robustness, and transferability of SPADE. Additionally, we evaluate the robustness of our method across different LLM backbones and text lengths, achieving state-of-the-art (SOTA) performance against several supervised and zero-shot baselines. In summary, the main contributions are as follows:

  • We introduce a new formulation of AI-text detection based on learning class-specific soft prompts.

  • State-of-the-art performance across two benchmarks, particularly at low false positive rates.

  • Better generalizability in cross-dataset and cross-family while remaining inference-efficient and storage-light, requiring only two learned soft prompts and a frozen backbone.

Related Work

Existing detection methods

Supervised methods train a classifier to distinguish AI-generated text from Human-written text. Some methods train a BERT-like model on labeled texts (Guo et al., 2023; Macko et al., 2025), fine-tune encoders to learn representations that separate the two classes via contrastive learning (Rivera Soto et al., 2024; Guo et al., 2024; Ta et al., 2026), or train adversarially to improve robustness against perturbation attacks (Hu et al., 2023; Li et al., 2025). Another line of works trains ensemble-based classifiers using linguistic features (Kumarage et al., 2023) and statistical features (Verma et al., 2024; Basani and Chen, 2026). Although these methods achieve strong performance in-distribution, they often lack robustness against unseen domains and models, and tend to overfit easily (Mitchell et al., 2023).

Zero-shot methods often use statistical cues to make their decisions. These cues are, for instance, probability-based such as Log-Likelihood (Solaiman et al., 2019), Entropy (Gehrmann et al., 2019), Negative log-curvature (Mitchell et al., 2023; Bao et al., 2024), log-likelihood log-rank ratio (Su et al., 2023), Cross-Perplexity (Hans et al., 2024), normalized temperature sensitivity (Ma et al., 2026) computed from the logits of one or several scoring models. In a real-world scenario, the LLM that generated a given text is unknown and often comes from black-box model, making the extraction of the true logits impossible. Therefore, zero-shot methods need to use some surrogate models to approximate black-box distributions, intuitively lacking robustness to distributional shift between the source and scoring models.

Concurrently, EchoPrompt [5] proposed an implicit reward modeling approach (Liu et al., 2025; Chen et al., 2026, amplified by the PMI. Unlike EchoPrompt, SPADE explicitly trains two soft prompts and derives its signal entirely from the prompt by comparing the PMI of the text under two learned soft prompts.

Parameter-Efficient Fine-tuning

Soft prompting offers an efficient alternative to full fine-tuning by adapting frozen LLMs to downstream tasks, such as summarization (Liu et al., 2022) and sentiment classification (Wu and Shi, 2022), without requiring manually crafted prompts or updates to the model parameters. Unlike conventional soft prompting, which steers generation toward downstream tasks, SPADE learns class-specific soft prompts to enhance the discriminative signal shown in Figure 1 for AI-text detection.

LoRA (Hu et al., 2022) is another alternative to full fine-tuning, alongside soft prompts. ImBD [9] uses LoRA to tune the model for the detection task. However, LoRA modifies the model’s internal behavior, whereas our method keeps the pre-trained model untouched to study how it responds to a prompt.

Method

We propose SPADE (Soft Prompts for Ai-text DEtection), a detection framework that learns class-specific soft prompts to distinguish AI-generated from human-written texts. As illustrated in Figure 2, we train two soft prompts to learn both contexts and perform a log-likelihood ratio to test which of the two soft prompts best explains a given text.

Overview of SPADE

Figure 2. Overview of SPADE: (a) Training. We propose a contrastive loss to learn two discriminative soft prompts to model each class writing process. (b) Inference. We employ a log-likelihood ratio test between the two trained soft prompts for Human chc_h and AI caic_{ai}. SPADE classifies by selecting the soft prompt whose PMI best explains the candidate text.

Notation

We denote any conditioning information for a text as a context cc. A context can be a hard prompt, a learned soft prompt, or an implicit conditioning underlying the text’s creation (e.g., the writer’s background, task, or register). Given a context cc, a tokenized input text xx, and a backbone LLM qq, the pointwise mutual information (PMI) is:

PMI⁡(x,c)=log⁡q(x∣c)−log⁡q(x),(1)\operatorname{PMI}(x,c)=\log q(x|c)-\log q(x), \tag*{(1)}

where C(x,c)=−log⁡q(x∣c)C(x,c)=-\log q(x|c) is the conditional negative log-likelihood. The PMI is computed by subtracting the conditional negative log-likelihood (CNLL) from the negative log-likelihood (NLL) of the input text. It measures how much the context helps the model generate a specific text.

In the following section, we consider two classes y∈{ai,h}y \in\{\mathrm{ai},\mathrm{h}\} as AI-generated and Human-written. We denote xyx_y an input text of class yy, cyc_y the soft prompt associated to class yy, and xˉy\bar{x}_y an input text from the opposite class.

Training the Soft Prompt

A natural loss to train the soft prompt cyc_y is to minimize C(xy,cy)C(x_y,c_y) of the texts of class yy. However, simply minimizing CC may increase the likelihood of both classes, leading to a soft prompt that explains both classes similarly. Therefore, we propose a hinge loss on the CNLL to learn a more discriminative signal.

However, a pure hinge loss is invariant to a global shift since it constrains only the relative gap between the two conditional scores. To ensure that the class-specific prompt also assigns high likelihood to its own class, we also minimize its in-class CNLL. The final loss thus becomes:

Ly=C(xy,cy)⏟CNLL+[m+C(xy,cy)−C(xyˉ,cy)]+⏟Hinge(2)\mathcal{L}_y = \underbrace{C(x_y,c_y)}_{\mathrm{CNLL}} + \underbrace{\left[m+C(x_y,c_y)-C(x_{\bar y},c_y)\right]_+}_{\mathrm{Hinge}} \tag*{(2)}

where [z]+=max⁡(0,z)[z]_+ = \max(0,z), m>0m > 0 is the margin (we use m=1m = 1 in all experiments), and CC is the CNLL. Intuitively, the hinge loss forces the soft prompt cyc_y to explain xyx_y better than xyˉx_{\bar y} by at least a margin mm. In practice, since the two soft prompts share only a frozen backbone qq, they can be trained in parallel by batching both prompts through a single forward pass.

Detection using a log-likelihood ratio test

Given a candidate text xx and the two soft prompts caic_{\mathrm{ai}} and chc_h, the log-likelihood ratio (LLR) test between the two PMIs is defined as:

LLR⁡(x)=PMI⁡(x,cai)−PMI⁡(x,ch).(3)\operatorname{LLR}(x) = \operatorname{PMI}(x,c_{\mathrm{ai}}) - \operatorname{PMI}(x,c_h). \tag*{(3)}

The ratio aims to test which context best explains the candidate text xx: we classify xx as AI-generated if LLR⁡(x)>τ\operatorname{LLR}(x) > \tau, and human-written otherwise. The detection signal focuses solely on a comparison between PMI⁡(x,cai)\operatorname{PMI}(x,c_{\mathrm{ai}}) and PMI⁡(x,ch)\operatorname{PMI}(x,c_h). By Equation (1), we can write LLR as the difference between the two CNLLs:

LLR⁡(x)=C(x,ch)−C(x,cai).(4)\operatorname{LLR}(x) = C(x,c_h) - C(x,c_{\mathrm{ai}}). \tag*{(4)}

This simplification of the LLR speeds up the inference time, since we only need to compute a single forward with a batch of 2 and can skip the NLL computation.

Experimental Setup

Datasets and Evaluation Metrics

Training dataset. We train and validate on MIRAGE (Fu et al., 2025), containing both human-written texts and AI-generated texts from 9 families of instruct LLMs across 5 domains and 3 prompt types. MIRAGE proposes two datasets: (1) Disjoint-Input Generation (DIG), each human-written text is paired with a unique AI-generated text from one of the 9 families. (2) Shared-Input Generation (SIG), each 2,384 human-written texts is paired with one AI-generated text per family. For each of 3 random seeds, we draw a train/validation/test split of MIRAGE-DIG, and one split per family for MIRAGE-SIG.

Evaluation dataset. We evaluate on two datasets completely out-of-distribution (OOD) to serve as our primary benchmarks: (1) RAID (Dugan et al., 2024), comprising 8 domains, 11 models, and various text perturbations. We restrict RAID to 7 instruction-tuned model, since they are the most widely used, and use the subset sampled by Ma et al. (2026); (2) EvoBench (Yu et al., 2025), comprising 5 domains and 6 model families; to keep the evaluation fully OOD, we remove XSum, which is also part of MIRAGE. Details for each dataset are given in Appendix § 10.

Metrics. Following prior work, we use AUROC (AUC) (Mitchell et al., 2023; Bao et al., [4]) and the true positive rate at a fixed false positive rate of 5% (TPR) (Hans et al., 2024) as our main evaluation metrics. In addition, we report the macro F1-score for all experiments in the Appendix § 13. On EvoBench, we compute each metric separately per domain on every generator family to avoid any imbalance and report the mean over families; On RAID, we average over all generators within each domain.

Baselines

(I) Zero-shot detectors score a text with a scoring model: (1) Likelihood and Entropy, the mean token log-likelihood and entropy; (2) DNA-GPT (Yang et al., 2024), the log-likelihood gap between the original continuation of a truncated text and KK regenerated ones; (3) Fast-DetectGPT (Bao et al., [4]), the conditional probability curvature; (4) Binoculars (Hans et al., 2024), the perplexity normalized by the cross-perplexity between two models; (5) NTS (Ma et al., 2026), the normalized sensitivity of the token distribution to temperature.

(II) Supervised detectors are trained on labeled data: (1) RoBERTa, a fine-tuned encoder classifier; (2) RAIDAR (Mao et al., 2024), a multi-layer perceptron (MLP) on n-gram and edit-distance features between the text and its LLM rewrites; (3) DPIC (Yu et al., 2024), a MLP on the embeddings of the text and of a response regenerated from its reconstructed prompt; (4) ImBD (Chen et al.,

Detectorharmful AUCharmful TPRpeerread AUCpeerread TPRpubmed AUCpubmed TPRwriting AUCwriting TPRAvg. AUCAvg. TPR
Likelihood57.18.4582.637.690.862.197.489.182.049.3
Entropy55.66.1659.110.590.768.396.786.075.542.8
DNA-GPT (Yang et al. (2024))54.75.4989.052.287.548.790.459.380.441.4
Fast-DetectGPT (Bao et al. [4])56.06.3286.650.071.322.380.741.873.630.1
Binoculars (Hans et al. (2024))54.46.9686.651.569.124.082.851.073.233.4
NTS (Ma et al. (2026))48.96.5473.328.046.21.4159.816.857.113.2
RoBERTa95.3±1.775.9±8.289.4±0.863.5±4.874.1±1.327.4±5.296.1±1.286.4±3.988.7±0.463.3±2.4
RAIDAR (Mao et al. (2024))76.6±2.326.4±3.267.3±0.620.0±1.664.5±3.210.7±3.881.8±0.835.5±1.072.6±1.223.2±1.2
DPIC (Yu et al. (2024))80.5±0.746.1±2.976.1±1.941.9±2.966.0±1.426.2±1.792.0±0.172.8±0.878.6±0.846.8±1.8
ImBD (Chen et al. [9])92.3±1.963.7±10.093.2±1.267.0±5.287.3±4.049.4±7.998.7±0.594.5±4.592.9±1.268.7±4.0
SPADE (ours)96.7±0.488.4±1.495.8±0.989.1±1.898.3±0.395.1±0.599.8±0.098.9±0.297.7±0.392.9±0.7

Table 1. AUC and TPR on EvoBench. All supervised baselines are trained on MIRAGE-DIG and the reported number corresponds to meanstd_{\mathrm{std}} in defined in §4 over three seeds. Bold indicates the best method per column.

DetectorAbstracts AUCAbstracts TPRBooks AUCBooks TPRNews AUCNews TPRPoetry AUCPoetry TPRRecipes AUCRecipes TPRReddit AUCReddit TPRReviews AUCReviews TPRWiki AUCWiki TPRAvg. AUCAvg. TPR
Likelihood75.510.082.911.275.66.6775.34.0077.114.278.14.7081.39.5978.64.8678.18.16
Entropy68.610.080.811.871.26.0075.34.0077.716.376.14.7078.69.5975.24.8675.48.41
DNA-GPT (Yang et al. (2024))71.020.777.524.960.110.768.310.767.414.276.215.477.824.075.314.671.716.9
Fast-DetectGPT (Bao et al. [4])76.632.366.418.973.018.759.419.049.615.672.225.066.528.172.541.767.024.9
Binoculars (Hans et al. (2024))80.750.775.140.876.138.764.927.353.824.176.736.272.334.975.643.871.937.1
NTS (Ma et al. (2026))76.333.367.820.169.116.063.615.374.421.372.920.163.110.363.918.868.919.4
RoBERTa87.945.183.846.279.443.289.465.678.736.486.750.290.564.280.041.884.649.1
RAIDAR (Mao et al. (2024))59.923.365.027.673.642.264.212.477.922.576.649.471.327.965.524.869.328.8
DPIC (Yu et al. (2024))86.968.987.266.989.672.990.473.682.149.282.456.693.879.280.249.386.664.6
ImBD (Chen et al. [9])88.015.191.427.885.0…
SPADE (ours)98.391.196.784.490.277.697.292.194.979.497.985.795.887.094.674.495.784.0

Table 2. AUC and TPR on RAID. All supervised baselines are trained on MIRAGE-DIG and the reported number corresponds to the mean defined in §4 over three seeds. Bold indicates the best method per column.

2025), Fast-DetectGPT score under a fine-tuned scoring model with LoRA to prefer AI-generated style.

Implementation

Our experiments use Qwen3-8B (Yang et al. (2025)) as the main backbone, and we further test across different backbones (LLaMA-3.1-8B (Grattafiori et al. (2024)) and Falcon-7B (Almazrouei et al. [2]) in §5.2. Each experiment is repeated over three seeds, and we report the mean and standard deviation across them. To ensure a fair comparison, all supervised methods, including SPADE, use the same three splits for training and validation. We details in Appendix C on the hyperparameter choice, initialization, selection metrics, and implementations of all baselines.

Experiments and Analysis

Main Results

Cross-Dataset Generalization. Table 1 and 2 present the overall performance of SPADE with other baselines on EvoBench and RAID, respectively. Crucially, all baselines struggle in low-FPR regime: zero-shot detectors have TPR lower than 50%, supervised detectors below 70%. Overall, supervised detectors perform better than zero-shot ones, with an average gain of ∼9.7\sim9.7 AUC on EvoBench and RAID, especially in hard domain where the texts are short (e.g., harmful in EvoBench, see §5.4). Supervised detectors trained on the same setup as SPADE transfer only partially to cross-dataset (e.g., ImBD reaches 68.7 TPR on EvoBench but only 22.8 on RAID). In contrast, SPADE outperforms all supervised methods by 4.8–25.1 AUC on EvoBench and 8.2–26.4 AUC on RAID, and maintains a strong TPR score (84–92.9%), highlighting its better transferability in cross-dataset. We find consistent results using LLaMA where SPADE outperforms other baselines (details in §5.2 per backbone). With Falcon, SPADE is competitive on RAID and wins on EvoBench.

Cross-Generator Generalization. We train SPADE on texts from one LLM family in a MIRAGE-SIG and testing it on texts generated by all family models in EvoBench and RAID. As shown in Figure 3, SPADE achieves the best AUC, TPR, and F1 on both benchmarks, outperforming all baselines. Specifically, SPADE outperforms the supervised baselines by 4.9–28.8 AUC, 21.5–67.1 TPR, and 10.7–26.8 F1 on EvoBench, and by 11.6–29.4 AUC, 21.1–65.2 TPR, and 11.5–30.9 F1 on RAID (see Appendix § 13.§ 2 for the exact LLM-family wise scores and mean scores). This shows that SPADE transfers better than the other supervised baselines across datasets even when trained on a single family with only 500 text pairs (see Appendix § 10.§ 1).

Figure 3. AUC, TPR, and F1 on EvoBench and RAID. All supervised baselines are trained on per family splits from MIRAGE-SIG (over three seeds). The reported number corresponds to meanstd\mathrm{mean}_{\mathrm{std}} over the 9 families of the mean defined in § 4 over three seeds per family using Qwen3-8B. Hatched bars are supervised baselines and Dotted bars are for zero-shot baselines.

Backbone and Perturbation Robustness

SPADE’s sensitivity to backbone choice. Figure 4 presents the mean performance of SPADE on the two benchmarks with various LLM backbones. We observe substantial variation across backbones, with more recent model families generally yielding stronger performance. Interestingly, model scale alone does not explain the differences: LLaMA3.2-1B (2024) outperforms (+2.5–11.7 TPR) older but larger models (e.g., Falcon-7B (2023) and GPT-J-6B (2021)), and Qwen3.5-4B (2026) on par with Qwen3-8B (2025). This suggests that the quality of the backbone’s pre-training is the main driver of SPADE’s effectiveness. Overall, we expect SPADE to benefit from future advances in backbone quality without requiring any changes to the method itself.

Figure 4. Impact of the backbone on SPADE’s performance, averaged over EvoBench and RAID.

SPADE vs. others across backbones. The sensitivity to backbone choice is not unique to SPADE, but common to all detectors. Table 3 compares SPADE to other baselines using two additional backbones (LLaMA-3.1-8B and Falcon-7B). A very surprising finding is that complex zero-shot

DetectorEvoBench AUCEvoBench TPRRAID AUCRAID TPRAvg. AUCAvg. TPR
LLaMA-3.1-8B
Likelihood75.436.974.97.575.122.2
Entropy63.426.770.96.867.216.8
DNA-GPT75.232.270.120.472.726.3
Fast-DetectGPT86.455.686.962.186.658.9
Binoculars86.056.989.570.187.763.5
NTS60.220.275.332.167.726.2
RAIDAR64.90.564.9_{0.5}18.81.918.8_{1.9}73.10.673.1_{0.6}37.61.337.6_{1.3}69.028.2
DPIC77.90.877.9_{0.8}45.31.445.3_{1.4}86.70.486.7_{0.4}65.31.165.3_{1.1}82.355.3
ImBD88.61.088.6_{1.0}55.92.555.9_{2.5}93.72.693.7_{2.6}67.421.867.4_{21.8}91.161.6
SPADE (ours)97.40.097.4_{0.0}89.50.389.5_{0.3}94.50.494.5_{0.4}79.92.679.9_{2.6}96.084.7
Falcon-7B
Likelihood72.029.175.17.973.618.5
Entropy54.39.864.84.459.57.1
DNA-GPT74.035.179.236.076.635.6
Fast-DetectGPT82.151.385.238.083.744.7
Binoculars82.352.389.367.585.859.9
NTS71.329.388.965.280.147.2
RAIDAR50.00.750.0_{0.7}8.30.78.3_{0.7}54.71.254.7_{1.2}12.70.212.7_{0.2}52.410.5
DPIC78.30.478.3_{0.4}45.01.145.0_{1.1}86.90.686.9_{0.6}63.41.263.4_{1.2}82.654.2
ImBD86.92.286.9_{2.2}52.25.452.2_{5.4}87.43.087.4_{3.0}40.515.640.5_{15.6}87.146.3
SPADE (ours)94.10.994.1_{0.9}78.01.278.0_{1.2}89.70.289.7_{0.2}66.92.066.9_{2.0}91.972.4

Table 3. Performance of SPADE and the baselines with LLaMA-3.1-8B and Falcon-7B. Bold indicates the best method within each backbone.

methods (e.g., Fast-DetectGPT, Binoculars) perform substantially better with these two older backbones than with Qwen3-8B. We hypothesize that their degraded performance with Qwen3-8B may stem from differences in model architecture and training, as Qwen3-8B is a more recent hybrid reasoning and instruction-tuned model, potentially altering the likelihood patterns on which these zero-shot methods rely. Regeneration-based methods (e.g. DNA-GPT, RAIDAR, DPIC) remain relatively stable across backbones except for RAIDAR on Falcon-7B-Instruct, where it collapses close to random. With LLaMA-3.1-8B, SPADE remains the best method overall in the main result (see § 5.1). With Falcon-7B, Binoculars and NTS are more competitive where SPADE remains the best in AUC (+0.4) but lose in TPR (−0.6).

Perturbation attacks. Existing detectors are known to be vulnerable to various text perturbations, which can substantially degrade their detection performance ((Krishna et al., 2023; Sadasivan et al., 2025)). We investigate whether SPADE remains robust when AI-generated texts are deliberately perturbed to evade detection. Figure 5 presents the AUC and TPR of SPADE and the four strongest baselines on RAID under several adversarial attacks. With Qwen3-8B-Base, SPADE achieves the best performance on every attack except paraphrasing (82.2 vs. 83.2 AUC for ImBD; 40.6 vs. 42.2 TPR for DPIC). Over the 11 attacks, SPADE significantly outperforms every supervised baseline, including the strongest one, ImBD (95.6 vs. 93.1 AUC, p=0.005p = 0.005; 84.4 vs. 70.3 TPR, p=0.008p = 0.008; Student’s tt-test over the three DIG seeds). With LLaMA-3.1-8B, SPADE is competitive with the strongest detector Binoculars and outperforms it on 5 attacks (e.g., alternative spelling, paragraph insertion, misspelling, synonym substitution and whitespace), but collapses on zero-width space attacks (which hurt all supervised methods) going from 92.1 TPR without attack to 22.7 (details per attack type is presented in Appendix § 13.§ 3).

Figure 5. AUC of SPADE and four baselines on RAID, grouped by adversarial attacks. TPR results are consistent and can be found in Appendix § 13.§ 3. Hatched bars are supervised baselines and Dotted bar is for zero-shot baseline.

Ablation Study

We conduct ablation studies to independently assess the contributions of the hinge loss and log-likelihood ratio test to SPADE’s performance. Results are given in Table 4.

HingeScoreEvoBench AUCEvoBench TPRRAID AUCRAID TPRAvg. AUCAvg. TPR
✓LLR (ours)97.792.995.784.096.788.5
✓PMI(x,chx, c_h)90.781.590.579.890.680.7
✓PMI(x,caix, c_{ai})86.357.990.635.488.546.7
✗LLR95.785.279.746.687.765.9
✗PMI(x,chx, c_h)78.445.874.233.976.339.9
✗PMI(x,caix, c_{ai})79.444.554.113.266.828.9

Table 4. Impact of the hinge loss and the LLR in SPADE’s performance using Qwen3-8B. Hinge: regularized hinge loss (✓) or plain cross-entropy loss (✗). We report the average of the two benchmarks.

Impact of the Hinge Loss. We compare the regularized hinge loss to a simple cross-entropy loss. Without the contrastive term, the discriminative power of each test statistic drops on both benchmarks: PMI(x,caix, c_{ai})’s AUC falls by 6.9–36.5 points and its TPR by 13.4–22.2 points, and PMI(x,chx, c_h)’s AUC falls by 12.3–16.3 points and its TPR by 35.7–

45.9 points. Soft prompts learned with a simple cross-entropy perform worse likely because they converge toward a generative optimum rather than a discriminative optimum. Overall, on the LLR, AUC and TPR improves by 2.0–16.0 and 7.7–37.4.

Impact of the Log-likelihood Ratio Test. We compare the log-likelihood ratio (LLR) test, which contrasts the two learned contexts, with scoring a text using a single soft prompt, PMI(x,caix, c_{ai}) or PMI(x,chx, c_h). The LLR improves over the best single prompt by 7.0 AUC and 11.4 TPR points on EvoBench, and by 5.1 AUC and 4.2 TPR points on RAID. These results validate learning two soft prompts rather than one: modeling both classes and contrasting them yields a more accurate and more robust detector.

Further Investigations

Impact of Text Length Existing detectors performance are known to drop significantly when dealing with short text (Wu et al., 2024). We test SPADE’s robustness by truncating the Writing texts in EvoBench to specific lengths. Figure 6 presents the AUC of SPADE and four different baselines. Interestingly, supervised methods are more robust than zero-shot methods on short text. Zero-shot methods compute a probability-based score that is not optimized for detection and therefore require a substantial longer text to accumulate a reliable score. In contrast, SPADE is optimized to maximize the gap between the two classes, which makes it more robust than the baselines: it reaches 98.2 AUC at 75 tokens, versus 88.6 for ImBD and 78.7 for Binoculars and Fast-DetectGPT.

Figure 6. Impact of the text length on four existing baselines and SPADE on EvoBench (Writing) with Qwen3.

Inference Speed Existing regeneration-based detectors require regenerating one or several versions of the input text autoregressively limiting their inference speed (Hao et al., 2025). For practical purposes, we compare SPADE’s inference speed with the baselines in Table 5. Our method requires a single forward pass on a batch of two, so it is slower than detectors requiring one forward pass on a single sequence, and faster than Binoculars, which requires two models. Notably, SPADE is much faster than regeneration-based methods (∼85.1×\sim85.1 \times speed-up) while transferring better across datasets.

Detectors / docs / 100 wordsvs. SPADERank
RoBERTa0.0080.006 0.0100.06×2
Likelihood0.0560.038 0.0060.39×5
Entropy0.0580.039 0.0060.40×8
NTS0.0610.042 0.0070.43×11
ImBD0.0660.046 0.0080.47×4
Fast-DetectGPT0.1180.081 0.0130.83×9
SPADE (ours)0.1410.098 0.0141×1
Binoculars0.2620.182 0.0281.87×6
DNA-GPT10.6127.472 1.59276.6×7
DPIC11.4168.028 5.31882.3×3
RAIDAR13.6479.397 1.99996.4×10

Table 5. Inference latency per document, with Qwen3-8B on NVIDIA RTX A6000. Times are averaged over 180 MIRAGE-DIG test documents with 20 documents used for warm up. s / 100 words: meanstd_{\mathrm{std}} of the per-document time normalized by length. Ranked by overall AUC and TPR on EvoBench and RAID.

Cross-Lingual Previous research Wang et al., 2024 propose to assess detector performance on languages other than English. To assess our approach in cross-lingual scenario, we follow DetectRL-X (data details in Appendix § 10) protocol Wu et al., 2026, by training and tuning on a single language and testing on others (details in Appendix § 14). Encouragingly, SPADE shows no collapse on high-complexity languages defined by DetectRL-X (e.g., Arabic, Russian, Chinese) where it reaches 94.97 F1BF_1^B on average, against 95.56 for medium-complexity ones (e.g., German, French, Spanish, Portuguese). Similarly, transferring from Indo-European to Semitic or Sino-Tibetan languages costs SPADE nothing (95.68 in-language to 95.54 out-language F1BF_1^B), whereas Wu et al. (2026) report an average loss of 19.15 points. However, SPADE shows its largest performance drop when

transferring from English to German (64.45 F1BF_1^B), despite performing well in the reverse direction (99.51 F1BF_1^B). We leave further investigation of this asymmetry to future work.

Conclusion

We presented SPADE, a novel detection framework that learns class-specific soft prompts to distinguish AI-generated from human-written texts. Detection is a log-likelihood ratio test between the PMIs conditioned by the soft prompts. SPADE achieves strong performance in cross-datasets and cross-generator compared to both zero-shot and supervised baselines, while remaining inference-efficient. Building on our strong performance in cross-generator scenario, we intend to investigate if SPADE can be adapted to model attribution task. Additionally, since the LLR decomposes into per-token contributions, we plan to investigate at the token level which characteristics is captured.

Limitations

Our findings indicate that using different backbone LLMs can significantly change the performance of all detectors. A more systematic comparison across all baselines would better clarify the impact of the backbone choice on each detector, in particular contrasting SPADE with the other supervised baselines.

A second limitation is that, like most state-of-the-art detectors, SPADE’s performance drops on AI-generated texts that was paraphrased to evade existing detectors (see § 5.2). Since SPADE is supervised, it can be trained on datasets containing perturbed texts, which we leave to future work.

Finally, our evaluation primarily focuses on texts generated by instruction-tuned models. As shown in Table 7 in Appendix § 11, SPADE performs less effectively on texts from base models. While we leave further investigation to future work, this drop may stem from the exclusive use of instruction-tuned models in training or the weaker influence of instructions on base-model generation.

FamilyInstructBaseΔ\Delta
Cohere88.476.9−11.4
Mistral98.076.7−21.3
MPT97.079.0−18.0
Mean94.477.6−16.9

Table 7. AUC of SPADE-DIG on RAID per family, for the instruct and the base version of each generator, with Qwen3-8B-Base as backbone (mean over three seeds). Δ\Delta = Base − Instruct. Bold indicates the higher AUC of the two model types.

Ethical Considerations

Datasets. We evaluate on publicly available benchmarks for machine-generated text detection (RAID, EvoBench, DetectRL-X and MIRAGE) and use them only for non-commercial research, in line with their intended purpose and licenses (see Appendix § 10.§ 5).

Societal Impact. Although SPADE transfers better than the baselines and is practical to deploy thanks to its fast inference and small storage requirements, the prediction itself is not bulletproof. Performance may degrade in specific scenarios, especially regarding calibration on in-domain data. Human judgment remains necessary, especially in high-stakes domains.

Acknowledgments

This work has been financially supported by ChapsVision and the ANRT through CIFRE grant n°2025/1102. We thank GENCI for granting the access to the HPC resources of IDRIS under the allocation 2026-AD011017568 where most of our computation was done on the Jean-Zay supercomputers.

Details of Data

The source dataset per domain across all datasets: MIRAGE (Fu et al., 2025), EvoBench (Yu et al., 2025), RAID (Dugan et al., 2024), and DetectRL-X (Wu et al., 2026) are all different, except for XSum, which appears in both MIRAGE and EvoBench.

MIRAGE

We split MIRAGE-DIG at the document level with the ratio 70/15/15. MIRAGE-SIG is split at the document-level using the 2,384 shared human-written texts to avoid any leakage. We split by count: 500 training / 942 validation / 942 test pairs. Both SIG and DIG share the same source dataset. All source datasets are chosen as prior to 2021 to avoid any possible AI-generated texts corruption. The selected domains: academic, mail, website, news, comment.

  • Academic contains BigPatent (Sharma et al., 2019), NeurIPS, ArXiv, PubMed-Abstracts (Jin et al., 2019) as datasets.

  • eMail contains data from Enron-Emails (Klimt and Yang, 2004)

  • Website contains data from OpenWebText (Gokaslan et al., 2019)

  • News contains data from CNNDaily-Mails(See et al., 2017; Hermann et al., 2015), XSum (Narayan et al., 2018), XLSum (Hasan et al., 2021)

  • Comment: Amazon-Review (McAuley and Leskovec, 2013; Zhang et al., 2015)

MIRAGE uses 9 families of instruct LLMs (GPT, Claude, DeepSeek, Gemini, Qwen, LLaMA, Grok, Moonshot, and Doubao) with three types of prompt – continuation, polishing to a specific writing style, and rephrasing.

EvoBench

Detection is inconsistent between models and versions of the models. EvoBench provides human-written texts from five domains, where each human-written text is paired with an AI-generated text from several black-box and white-box LLMs, covering several versions of these LLMs. The source datasets are the following:

  • Harmful contains data from (Kula, 2026).

  • PeerRead contains data from (Kang et al., 2018)

  • PubMed contains data from (Jin et al., 2019)

  • Writing contains data from (Fan et al., 2018).

  • News contains data from XSum (Narayan et al., 2018).

EvoBench and MIRAGE both draws human-written text from PubMed (Jin et al., 2019). However, they take two disjoint part of the dataset: EvoBench uses the Question-Answering dataset; MIRAGE uses the Abstract dataset.

RAID

Domains RAID contains human-written and AI-generated texts spanning eight different domains: Abstracts, Books [3], News (Greene and Cunningham, 2006), Poetry1, Recipes [7], Reddit posts (Völske et al., 2017), Movie reviews (Maas et al., 2011), and Wikipedia [1].

Source Models The machine-generated texts come from 11 models: GPT-2 (Radford et al., 2019), GPT-3 (Ouyang et al., 2022), ChatGPT, GPT-4 (OpenAI et al., 2024), MPT and MPT-Chat (Team, 2023), Mistral and Mistral-Chat (Jiang et al., 2023), Cohere and Cohere-Chat, and LLaMA-2-70B-Chat (Touvron et al., 2023). We focus solely on the instruction-tuned models and remove the base models Mistral, Cohere, GPT-2, and MPT from the evaluation.

DetectRL-X

DetectRL-X provides human-written texts and AI-generated texts on 6 different domains, using both white-box and black-box LLMs, and across 8 languages. The source datasets are the following:

  • Academic Writing: from Directory of Open Access Journals (DOAJ).

  • News: mostly from BBC.

  • Novel: corresponds to the creative fiction texts extracted from Common Crawl.

  • SEO: seach engine optimization content from public and private data from the authors’s organization.

  • Wiki: encyclopedia entries from Wikipedia across various topics and languages.

  • WebText: corresponds to general web content extracted from Common Crawl.

License

We use the datasets solely for non-commercial academic evaluation consistent with their intended use and licenses.

DatasetLicense
MIRAGE (Fu et al., 2025)Pi-Lab License 1.0
EvoBench (Yu et al., 2025)MIT
RAID (Dugan et al., 2024)MIT
DetectRL-X (Wu et al., 2026)Apache 2.0

Table 6. Licenses of the evaluation datasets.

Base models vs. Instruct models

A limitation of SPADE is its drop in performance when detecting text generated by base models. To quantify it, we report on RAID the AUROC of SPADE per generator family, for the instruct and the base version of each model.

Implementation Details

All models are run with PyTorch 2.10 and Hugging Face Transformers 5.17 in bfloat16, with FlashAttention-2 (flash-attn 2.8.1) for training. Every detector scores at most the first 512 tokens of a text (backbone tokenizer, no special tokens). AUROC, ROC curves and the thresholds (Youden-J, TPR at 5% FPR) are computed with scikit-learn 1.9.1; pAUC@5% FPR is the normalized area under the ROC curve for FPR ≤0.05\le0.05. F1 is the F1 of the AI class at the Youden-J threshold fitted on the MIRAGE validation split.

Backbone type choices

For zero-shot methods, we follow the original implementations with different backbones, keeping each paper’s base/instruct choice where possible. We report in the following table these choices (Table 8).

DetectorBaseInstruct
Likelihood✓✗
Entropy✓✗
DNA-GPT✓✗
Fast-DetectGPT✓✓
Binoculars✓✓
NTS✓✗
RAIDAR✗✓
DPIC✗✓
ImBD✓✗

Table 8. Backbone type used by each baseline.

Hyperparameter Selection

For hyperparameter selection, we trained for 3 epochs with a batch size of 16 on MIRAGE-DIG, and for 20 epochs on MIRAGE-SIG. We select checkpoints and hyperparameters with pAUC@5% FPR rather than AUC or TPR, because AUC and TPR approach 1 after only a few epochs, which makes them unsuitable for selection. The only tuned hyperparameter is the learning rate, which differs across backbones. Learning rates are selected on MIRAGE-DIG validation (seed 0, pAUC@5% FPR) from {1e-5, 3e-5, 1e-4, 3e-4}, extended to 1e-3 and 3e-3 when the best value lay at the edge of the grid (Falcon-7B only). The selected learning rates are 3e-4 for Qwen3-8B-Base and LLaMA-3.1-8B, and 3e-3 for Falcon-7B. MIRAGE-SIG uses the same learning rate per backbone. Following Lester et al. (2021), we use a soft prompt of length 64, initialized with the embeddings of random vocabulary tokens.

Zero-shot methods

Binoculars (Hans et al., 2024) is a zero-shot method based on the ratio between the perplexity and the cross-perplexity that uses two scoring models: an observer and a performer. The cross-perplexity acts as a normalizer measuring whether the tokens are surprising relative to another surrogate model. We use three different model pairs (Falcon-7b / Falcon-7b-Instruct, LLaMA-3.1-8B / LLaMA-3.1-8B-Instruct, and Qwen3-8B-Base / Qwen3-8B).

Fast-DetectGPT (Bao et al., [4]) following DetectGPT (Mitchell et al., 2023) is a zero-shot method using the conditional probability curvature where AI-generated text lives on a local maximum of that curvature. The initial paper uses GPT-J-6B as the sampling and GPT-Neo-2.7B as the scoring models, and the GitHub repository proposes to use LLaMA-3-8B/LLaMA-3-8B-Instruct which is supposed to outperform the previously chosen models. We use three different model pairs (Falcon-7b / Falcon-7b-Instruct, LLaMA-3.1-8B / LLaMA-3.1-8B-Instruct, and Qwen3-8B-Base / Qwen3-8B).

NTS (Ma et al., 2026) is a zero-shot method using a difference between high-temperature and low-temperature scoring on the input text and uses temperature sensitivity as its detection signal. We use three different model pairs (Falcon-7B, LLaMA-3.1-8B, and Qwen3-8B-Base) as our scoring model.

Regeneration-based methods

We generate all regenerated texts of all datasets using vLLM 0.19.1 for efficiency. We carefully choose the three comparison backbones as (Qwen3-8B without thinking, LLaMA-3.1-8B-Instruct, and Falcon-7B-Instruct) for RAIDAR and DPIC on a single seed 0. As for DNA-GPT, we replace the backbone by their respective base pair (Qwen3-8B-Base, LLaMA-3.1-8B, and Falcon-7B).

We then obtain three splits (train, validation, test) over three different seeds 0, 27, 78 using the same document-level splits.

DNA-GPT (Yang et al., 2024). We use the original protocol (K=10K = 10, temperature 0.7). We keep the default values for all other parameters. Since we are computing the score using our own scoring model, we run DNA-GPT in the white-box setting.

RAIDAR (Mao et al., 2024). We use the same 7 prompts as in the original paper with the specified temperature 0.7 like in their paper. We let the other value per default of the backbone model.

DPIC (Yu et al., 2024). We use the same prompt and the same specified temperature 0.2 like in their paper and vary only the backbone model. We use the same embedding model "Alibaba-NLP/gte-Qwen2-7B-instruct".

Supervised methods

We train on the same splits as SPADE so that only the methodology differs.

RoBERTa. We follow the standard training setup for RoBERTa, with a learning rate of 2e-5, 5 epochs and a batch size of 8, and choose the best epoch on the validation set with the same metric, pAUC@5% FPR.

ImBD [9]. Following their implementation, we optimize it as we do SPADE: same split, learning-rate grid search, and the same validation metric, pAUC@5% FPR. The selected learning rates are 1e-4 for Qwen3-8B-Base and Falcon-7B, and 3e-5 for LLaMA-3.1-8B. ImBD on Falcon-7B degenerates at higher learning rates (validation pAUC@5% FPR drops from 0.88 at 1e-4 to 0.37 at 3e-4) so we did not extend the grid.

More details on the Main Results

Overall Performances

We report the per-domain results of § 5.1 with two additional backbone models (LLaMA-3.1-8B and Falcon-7B) for completeness. With LLaMA-3.1-8B, SPADE remains the state-of-the-art in both benchmark EvoBench (Table 11) and RAID (Table 12). With Falcon-7B, SPADE is competitive with Binoculars and NTS on RAID (Table 14) but remains the best overall on EvoBench (Table 13). This result is consistent with § 5.2, where SPADE’s performance drops on less capable and older backbones.

Detectorharmful AUCharmful TPRharmful F1peerread AUCpeerread TPRpeerread F1pubmed AUCpubmed TPRpubmed F1writing AUCwriting TPRwriting F1Avg. AUCAvg. TPRAvg. F1
Likelihood46.92.6025.976.930.244.182.839.575.194.875.386.075.436.957.7
Entropy42.12.480.0045.44.930.0078.734.20.0089.466.90.0063.426.70.00
DNA-GPT (2024)49.35.3037.182.039.369.482.231.574.787.552.777.275.232.264.7
Fast-DetectGPT (2024)67.015.539.492.671.882.687.442.176.998.593.188.186.456.671.8
Binoculars (2024)65.715.351.892.473.985.686.944.180.498.894.591.086.056.977.2
NTS (2026)49.33.7556.072.422.568.737.60.4747.981.554.272.860.220.261.3
RoBERTa95.31.782.789.40.858.374.11.327.496.11.286.488.70.472.1
RAIDAR (2024)61.517.764.367.714.864.351.92.858.978.65.071.264.918.965.0
DPIC (2024)79.715.067.574.01.949.666.40.527.491.072.683.377.945.356.9
ImBD (2025)91.61.478.492.52.973.883.11.138.595.085.486.388.65.077.3
SPADE (ours)94.80.776.997.70.291.798.00.194.299.00.496.097.40.090.1

Table 11. AUC, TPR, and F1 on EvoBench (XSum excluded) with LLaMA-3.1-8B as scoring backbone for the zero-shot detectors and SPADE. All supervised baselines are trained on MIRAGE-DIG and the reported number corresponds to meanstd_{\mathrm{std}} over three seeds; zero-shot AUC and TPR are single runs. F1 is computed at the Youden-J threshold fitted on each run’s MIRAGE validation split. When a criterion ranks human texts above AI texts on the validation split, no threshold exists and its F1 is 0. Bold indicates the best method per column.

DetectorAbstracts AUCAbstracts TPRAbstracts F1Books AUCBooks TPRBooks F1News AUCNews TPRNews F1Poetry AUCPoetry TPRPoetry F1Recipes AUCRecipes TPRRecipes F1Reddit AUCReddit TPRReddit F1Reviews AUCReviews TPRReviews F1Wiki AUCWiki TPRWiki F1Avg. AUCAvg. TPRAvg. F1
Likelihood73.710.059.179.311.870.071.86.0056.870.34.0063.072.98.5151.176.05.3767.679.79.5968.375.14.8654.974.97.5261.3
Entropy61.810.00.0076.411.80.0066.05.330.0068.92.670.0073.06.380.0072.54.700.0076.98.900.0071.94.860.0070.96.840.00
DNA-GPT (2024)73.133.356.676.733.765.558.017.351.767.18.0056.966.212.152.674.722.863.273.920.561.270.915.358.270.120.458.2
Fast-DetectGPT (2024)91.173.376.986.360.970.184.957.368.187.566.766.782.356.062.989.164.474.687.660.367.486.057.670.786.962.169.7
Binoculars (2024)92.982.778.390.066.977.787.268.772.591.677.378.185.270.966.691.871.178.891.265.875.586.057.672.189.570.174.9
NTS (2026)89.256.075.876.436.765.077.831.365.867.923.357.170.230.556.076.034.264.173.921.262.570.623.657.475.332.163.0
RoBERTa87.945.154.983.846.266.279.443.257.889.465.678.178.736.443.586.750.262.290.564.275.380.041.852.884.649.161.3
RAIDAR (2024)66.725.156.369.241.461.276.742.466.268.217.155.579.157.471.979.452.158.375.835.664.970.129.656.173.137.661.3
DPIC (2024)87.769.677.988.669.478.986.969.378.089.173.069.484.554.472.582.255.769.194.179.282.680.451.964.486.765.374.1
ImBD (2025)92.858.482.796.680.186.393.563.182.196.786.768.090.752.274.595.774.989.595.070.882.388.353.076.193.767.480.2
SPADE (ours)98.088.788.395.083.284.591.774.780.294.976.176.394.279.580.296.183.786.994.783.780.991.869.476.594.579.981.7

Table 12. AUC, TPR, and F1 on RAID with LLaMA-3.1-8B as scoring backbone for the zero-shot detectors and SPADE. All supervised baselines are trained on MIRAGE-DIG and the reported number corresponds to the mean over three seeds; zero-shot AUC and TPR are single runs. F1 is computed at the Youden-J threshold fitted on each run’s MIRAGE validation split. When a criterion ranks human texts above AI texts on the validation split, no threshold exists and its F1 is 0. Bold indicates the best method per column.

Detectorharmful AUCharmful TPRharmful F1peerread AUCpeerread TPRpeerread F1pubmed AUCpubmed TPRpubmed F1writing AUCwriting TPRwriting F1Avg. AUCAvg. TPRAvg. F1
Likelihood43.72.310.0072.624.70.0082.733.20.0089.056.30.0072.029.10.00
Entropy41.81.950.0043.14.380.0067.915.90.0064.317.20.0054.39.850.00
DNA-GPT (2024)44.82.9637.779.935.665.081.341.373.690.160.480.474.035.164.2
Fast-DetectGPT (2024)54.58.2325.183.939.967.091.562.174.298.595.091.082.151.364.3
Binoculars (2024)54.98.4142.684.245.774.891.259.681.998.795.691.182.352.372.6
NTS (2026)54.76.9954.570.114.067.463.528.860.396.787.988.171.329.367.6
RoBERTa95.375.982.789.463.558.374.113.045.296.186.439.088.763.372.1
RAIDAR (2024)49.65.4545.053.811.049.637.82.9621.059.013.749.550.08.2641.3
DPIC (2024)80.544.870.275.035.642.464.126.224.992.173.684.178.345.055.4
ImBD (2025)87.949.073.783.037.872.980.536.273.096.385.780.786.252.274.5
SPADE (ours)93.974.185.092.874.285.192.376.582.397.386.590.294.179.085.3

Table 13. AUC, TPR, and F1 on EvoBench (XSum excluded) with Falcon-7B as scoring backbone for the zero-shot detectors and SPADE. All supervised baselines are trained on MIRAGE-DIG and the reported number corresponds to meanstd_{\mathrm{std}} over three seeds; zero-shot AUC and TPR are single runs. F1 is computed at the Youden-J threshold fitted on each run’s MIRAGE validation split. When a criterion ranks human texts above AI texts on the validation split, no threshold exists and its F1 is 0. Bold indicates the best method per column.

DetectorAbstracts AUCAbstracts TPRAbstracts F1Books AUCBooks TPRBooks F1News AUCNews TPRNews F1Poetry AUCPoetry TPRPoetry F1Recipes AUCRecipes TPRRecipes F1Reddit AUCReddit TPRReddit F1Reviews AUCReviews TPRReviews F1Wiki AUCWiki TPRWiki F1Avg. AUCAvg. TPRAvg. F1
Likelihood72.28.000.0078.510.70.0071.74.000.0072.11.330.0074.824.80.0076.34.030.0078.06.160.0077.54.170.0075.17.900.00
Entropy51.27.330.0069.37.690.0057.04.000.0062.12.000.0068.46.670.0070.94.030.0071.36.680.0066.84.170.0064.84.450.00
DNA-GPT (2024)82.250.063.883.139.670.271.635.054.477.016.067.178.254.653.479.734.964.081.821.966.380.436.162.779.236.062.7
Fast-DetectGPT (2024)86.742.774.888.648.175.183.040.768.984.938.368.283.651.871.082.211.177.785.626.070.985.945.572.685.238.071.8
Binoculars (2024)90.272.076.492.469.881.187.564.075.690.371.376.787.575.074.787.554.474.991.467.177.588.166.073.789.367.576.3
NTS (2026)92.472.075.992.167.577.886.462.072.389.972.772.287.967.473.686.361.771.387.556.271.288.961.872.889.565.273.4
RoBERTa87.945.154.983.846.266.279.443.257.889.465.678.178.736.443.586.750.262.290.564.275.380.041.852.884.649.161.3
RAIDAR (2024)52.814.241.552.28.6846.359.418.445.956.57.5638.262.423.842.473.822.654.460.510.542.353.712.746.164.717.748.8
DPIC (2024)87.369.877.892.858.675.185.334.275.488.570.777.183.656.062.083.353.369.793.573.181.581.162.267.386.964.571.4
ImBD (2025)83.720.771.892.858.675.185.734.275.491.772.769.379.37.660.291.948.181.389.747.284.481.444.462.286.460.571.4
SPADE (ours)87.563.373.790.962.776.689.771.176.788.765.379.683.664.272.987.558.581.891.569.877.887.558.074.487.966.971.1

Table 14. AUC, TPR, and F1 on RAID with Falcon-7B as scoring backbone for the zero-shot detectors and SPADE. All supervised baselines are trained on MIRAGE-DIG and the reported number corresponds to the mean over three seeds; zero-shot AUC and TPR are single runs. F1 is computed at the Youden-J threshold fitted on each run’s MIRAGE validation split. When a criterion ranks human texts above AI texts on the validation split, no threshold exists and its F1 is 0. Bold indicates the best method per column.

Generalization using one source model

We report the results of supervised detectors across the nine families of § 5.1 for the three backbone models, and also per family over the three seeds. Although some training families are present in the benchmarks, the three tables 15, 16, and 17 show similar behavior for all supervised methods where some families transfer less well (e.g., Doubao, DeepSeek, and Qwen) than others that are not in-domain (Grok, Moonshot).

Backbones and Perturbations robustness

Tables 18, 19, and 20 report the per-attack AUC and TPR of all detectors on RAID for the three backbones, and Figures 9, 10, and 11 show the corresponding plots for Qwen3-8B-Base, LLaMA-3.1-8B, and Falcon-7B, including the decoding strategy and repetition penalty settings. Paraphrasing is hard for all detectors. SPADE’s main weaknesses are the zero-width space attack (22.7 and 9.28 TPR with LLaMA-3.1-8B and Falcon-7B) and, with Falcon-7B, the homoglyph attack (17.7 TPR).

Cross-Lingual

We present in table 21 a more detailed table for the Cross-Lingual task. We additionally report F1 at the fixed threshold 0, the natural choice for a log-likelihood ratio test. Notably, the threshold at 0 yields much higher F1 across all languages, all above 93%.

Training Necessity

We report in Table 22 the time and memory needed to train SPADE on MIRAGE-DIG. Training covers the 64,652 training texts (32,326 human/AI pairs) for 3 epochs, with a batch size of 16 and soft prompts of 64 tokens, on a single H100 NVL GPU (94 GB) with FlashAttention-2 and gradient checkpointing. We time one epoch per backbone; epoch times vary by less than 1% across epochs and seeds, so the total is three times the epoch time. Only the soft prompts are optimized (2 × 64 embeddings) and the backbone stays frozen.

Backboneper epochtotal (3 epochs)peak VRAM
Qwen3-8B-Base2.1 h6.2 h33.6 GB
LLaMA-3.1-8B1.7 h5.2 h30.7 GB
Falcon-7B1.4 h4.2 h24.0 GB

Table 22. Training time and peak GPU memory of SPADE on MIRAGE-DIG (single H100 NVL, bfloat16, FlashAttention-2, gradient checkpointing).

Detectorharmful AUCharmful TPRharmful F1peerread AUCpeerread TPRpeerread F1pubmed AUCpubmed TPRpubmed F1writing AUCwriting TPRwriting F1Avg. AUCAvg. TPRAvg. F1
Likelihood57.18.428.782.637.647.790.862.182.997.489.190.582.049.362.4
Entropy55.66.135.259.110.512.990.768.375.796.786.089.175.542.853.2
DNA-GPT (2024)54.75.436.289.052.273.787.548.778.390.459.375.180.441.465.9
Fast-DetectGPT (2024)56.06.330.686.650.078.971.322.348.680.741.872.273.630.157.5
Binoculars (2024)54.46.935.486.651.579.269.124.054.482.851.072.673.233.460.4
NTS (2026)48.96.558.273.328.068.946.21.456.059.816.855.657.113.259.7
RoBERTa95.31.775.989.40.863.574.11.327.496.11.286.488.70.463.3
RAIDAR (2024)76.62.326.467.30.620.064.53.210.781.80.835.572.61.223.2
DPIC (2024)80.50.746.176.11.941.966.01.426.292.00.172.878.60.846.8
ImBD (2025)92.31.963.793.21.267.087.34.049.498.70.594.592.91.268.7
SPADE (ours)96.70.488.495.80.989.198.30.395.199.80.098.997.70.392.9

Table 9. AUC, TPR, and F1 on EvoBench (XSum excluded) with Qwen3-8B-Base as scoring backbone for the zero-shot detectors and SPADE. All supervised baselines are trained on MIRAGE-DIG and the reported number corresponds to meanstd_{\mathrm{std}} over three seeds; zero-shot AUC and TPR are single runs. F1 is computed at the Youden-J threshold fitted on each run’s MIRAGE validation split. When a criterion ranks human texts above AI texts on the validation split, no threshold exists and its F1 is 0. Bold indicates the best method per column.

DetectorAbstracts AUCAbstracts TPRAbstracts F1Books AUCBooks TPRBooks F1News AUCNews TPRNews F1Poetry AUCPoetry TPRPoetry F1Recipes AUCRecipes TPRRecipes F1Reddit AUCReddit TPRReddit F1Reviews AUCReviews TPRReviews F1Wiki AUCWiki TPRWiki F1Avg. AUCAvg. TPRAvg. F1
Likelihood75.510.061.782.911.275.675.66.6762.475.34.0065.477.114.252.478.14.7072.081.39.5973.878.64.8658.578.18.1665.2
Entropy68.610.056.580.811.874.171.26.0058.875.34.0066.477.716.351.676.14.7071.378.69.5970.475.24.8656.475.48.4163.2
DNA-GPT (2024)71.020.759.877.524.967.860.110.753.368.31.0066.477.716.354.076.215.462.677.824.066.275.314.663.771.716.960.4
Fast-DetectGPT (2024)76.632.362.366.418.958.273.018.753.359.419.055.669.415.650.569.415.650.561.110.355.072.514.761.677.024.955.7
Binoculars (2024)80.750.764.375.140.865.376.138.764.164.927.356.253.824.144.176.736.265.872.334.961.975.643.861.671.937.160.4
NTS (2026)76.333.364.667.820.154.669.116.058.963.615.352.474.421.355.472.920.162.863.110.352.169.818.850.068.919.456.4
RoBERTa87.945.154.983.846.266.279.443.257.889.465.678.178.736.443.586.750.262.290.564.275.380.041.852.884.649.161.3
RAIDAR (2024)59.923.350.665.027.655.373.642.262.164.212.461.477.922.533.976.649.454.371.327.953.465.524.845.169.328.852.1
DPIC (2024)86.968.978.287.266.978.789.672.979.990.473.673.082.149.271.382.456.669.693.879.284.480.249.364.486.664.674.9
ImBD (2025)88.015.178.891.427.884.285.016.471.193.458.468.983.710.976.689.414.881.988.128.878.981.413.274.087.522.878.6
SPADE (ours)98.391.192.096.784.487.490.277.682.097.292.183.494.979.482.997.985.789.995.887.086.594.674.481.995.784.085.6

Table 10. AUC, TPR, and F1 on RAID with Qwen3-8B-Base as scoring backbone for the zero-shot detectors and SPADE. All supervised baselines are trained on MIRAGE-DIG and the reported number corresponds to the mean over three seeds; zero-shot AUC and TPR are single runs. F1 is computed at the Youden-J threshold fitted on each run’s MIRAGE validation split. When a criterion ranks human texts above AI texts on the validation split, no threshold exists and its F1 is 0. Bold indicates the best method per column.

Figure 7. AUC, TPR, and F1 on EvoBench (XSum excluded) and RAID. All supervised baselines are trained on per family splits from MIRAGE-SIG (over three seeds). The reported number corresponds to mean±\pmstd over the 9 families of the mean over three seeds per family using LLaMA-3.1-8B

Figure 8. AUC, TPR, and F1 on EvoBench (XSum excluded) and RAID. All supervised baselines are trained on per family splits from MIRAGE-SIG (over three seeds). The reported number corresponds to mean±\pmstd over the 9 families of the mean over three seeds per family using Falcon-7B

Figure 9. AUC and TPR of 5 baselines on RAID benchmark against SPADE grouped by adversarial attacks, decoding strategy, and with repetition penalty or not.

Train ↓Method →RoBERTa AUCRoBERTa TPRRoBERTa F1RAIDAR AUCRAIDAR TPRRAIDAR F1DPIC AUCDPIC TPRDPIC F1ImBD AUCImBD TPRImBD F1SPADE (ours) AUCSPADE (ours) TPRSPADE (ours) F1
EvoBenchIN — training family present in EvoBench
Qwen83.21.983.2_{1.9}49.84.749.8_{4.7}66.72.866.7_{2.8}70.62.070.6_{2.0}19.93.019.9_{3.0}68.01.668.0_{1.6}76.82.276.8_{2.2}40.22.740.2_{2.7}58.75.058.7_{5.0}91.21.191.2_{1.1}66.33.266.3_{3.2}74.31.274.3_{1.2}95.30.895.3_{0.8}83.75.383.7_{5.3}83.25.483.2_{5.4}
Claude81.41.881.4_{1.8}50.81.750.8_{1.7}52.12.452.1_{2.4}66.91.766.9_{1.7}20.61.120.6_{1.1}54.51.754.5_{1.7}74.81.474.8_{1.4}36.31.736.3_{1.7}53.16.953.1_{6.9}88.21.988.2_{1.9}59.26.759.2_{6.7}74.10.674.1_{0.6}94.61.394.6_{1.3}84.22.984.2_{2.9}85.23.685.2_{3.6}
Gemini83.80.983.8_{0.9}50.63.250.6_{3.2}55.52.955.5_{2.9}64.11.764.1_{1.7}19.81.519.8_{1.5}52.611.752.6_{11.7}75.61.375.6_{1.3}36.62.436.6_{2.4}58.83.958.8_{3.9}91.90.991.9_{0.9}69.61.569.6_{1.5}74.20.374.2_{0.3}96.30.296.3_{0.2}87.61.187.6_{1.1}87.23.287.2_{3.2}
GPT83.12.083.1_{2.0}51.01.051.0_{1.0}61.46.361.4_{6.3}67.00.967.0_{0.9}17.71.317.7_{1.3}56.83.856.8_{3.8}77.70.577.7_{0.5}38.90.838.9_{0.8}61.85.261.8_{5.2}91.80.491.8_{0.4}65.91.165.9_{1.1}74.20.574.2_{0.5}96.30.596.3_{0.5}88.60.488.6_{0.4}84.31.484.3_{1.4}
LLaMA85.41.385.4_{1.3}55.21.855.2_{1.8}62.63.562.6_{3.5}67.32.067.3_{2.0}16.32.116.3_{2.1}59.59.159.5_{9.1}77.40.577.4_{0.5}39.10.439.1_{0.4}58.52.958.5_{2.9}93.10.493.1_{0.4}71.51.771.5_{1.7}74.70.474.7_{0.4}97.00.497.0_{0.4}89.41.589.4_{1.5}89.80.789.8_{0.7}
OUT — training family absent from EvoBench
Doubao77.83.577.8_{3.5}29.84.329.8_{4.3}33.11.733.1_{1.7}58.63.458.6_{3.4}12.74.312.7_{4.3}60.64.160.6_{4.1}72.00.472.0_{0.4}31.61.231.6_{1.2}54.61.354.6_{1.3}89.01.789.0_{1.7}55.84.955.8_{4.9}74.00.774.0_{0.7}93.60.193.6_{0.1}76.33.476.3_{3.4}85.32.185.3_{2.1}
DeepSeek81.21.181.2_{1.1}43.02.643.0_{2.6}51.17.951.1_{7.9}72.41.972.4_{1.9}22.62.322.6_{2.3}63.42.563.4_{2.5}76.71.576.7_{1.5}38.31.538.3_{1.5}58.75.958.7_{5.9}89.21.189.2_{1.1}60.61.860.6_{1.8}74.80.674.8_{0.6}94.90.494.9_{0.4}84.03.584.0_{3.5}83.23.183.2_{3.1}
Moonshot84.81.384.8_{1.3}53.60.853.6_{0.8}65.40.365.4_{0.3}68.13.168.1_{3.1}18.33.218.3_{3.2}62.87.362.8_{7.3}76.51.176.5_{1.1}39.30.839.3_{0.8}58.17.458.1_{7.4}91.31.691.3_{1.6}64.86.264.8_{6.2}73.41.273.4_{1.2}96.50.696.5_{0.6}88.12.788.1_{2.7}82.81.082.8_{1.0}
Grok84.30.584.3_{0.5}51.91.151.9_{1.1}62.53.962.5_{3.9}65.92.265.9_{2.2}19.04.019.0_{4.0}59.83.359.8_{3.3}77.00.477.0_{0.4}40.34.340.3_{4.3}60.33.360.3_{3.3}90.70.990.7_{0.9}63.12.863.1_{2.8}74.40.474.4_{0.4}96.00.796.0_{0.7}88.51.788.5_{1.7}83.23.783.2_{3.7}
Mean82.82.382.8_{2.3}48.47.748.4_{7.7}56.710.556.7_{10.5}66.83.966.8_{3.9}18.52.818.5_{2.8}59.84.759.8_{4.7}76.01.876.0_{1.8}37.82.737.8_{2.7}58.12.758.1_{2.7}90.71.690.7_{1.6}64.15.064.1_{5.0}74.20.474.2_{0.4}95.61.195.6_{1.1}85.64.285.6_{4.2}84.92.384.9_{2.3}
RAIDIN — training family present in RAID
GPT78.22.378.2_{2.3}40.33.340.3_{3.3}54.76.854.7_{6.8}66.22.966.2_{2.9}14.08.514.0_{8.5}50.92.150.9_{2.1}81.11.481.1_{1.4}55.71.955.7_{1.9}67.42.067.4_{2.0}82.00.882.0_{0.8}17.51.817.5_{1.8}71.60.571.6_{0.5}94.20.994.2_{0.9}80.92.480.9_{2.4}83.71.283.7_{1.2}
LLaMA84.41.184.4_{1.1}56.93.956.9_{3.9}59.02.859.0_{2.8}63.52.663.5_{2.6}16.07.216.0_{7.2}52.53.152.5_{3.1}81.80.281.8_{0.2}56.12.056.1_{2.0}68.21.468.2_{1.4}82.01.382.0_{1.3}14.83.814.8_{3.8}72.01.572.0_{1.5}95.51.395.5_{1.3}84.13.584.1_{3.5}81.25.881.2_{5.8}
OUT — training family absent from RAID
Doubao78.81.278.8_{1.2}43.910.443.9_{10.4}43.73.243.7_{3.2}58.21.758.2_{1.7}4.440.14.4_{40.1}50.81.150.8_{1.1}80.50.580.5_{0.5}51.81.751.8_{1.7}66.61.066.6_{1.0}81.00.481.0_{0.4}11.00.511.0_{0.5}70.71.770.7_{1.7}89.72.489.7_{2.4}55.49.355.4_{9.3}73.72.273.7_{2.2}
Grok81.51.581.5_{1.5}45.63.645.6_{3.6}54.56.454.5_{6.4}64.30.464.3_{0.4}4.750.34.7_{50.3}54.00.754.0_{0.7}80.90.880.9_{0.8}54.00.454.0_{0.4}67.60.067.6_{0.0}80.60.380.6_{0.3}15.40.515.4_{0.5}68.72.568.7_{2.5}91.71.391.7_{1.3}67.68.367.6_{8.3}78.00.778.0_{0.7}
DeepSeek74.92.474.9_{2.4}31.04.531.0_{4.5}42.411.442.4_{11.4}66.41.366.4_{1.3}11.87.611.8_{7.6}46.14.046.1_{4.0}81.50.681.5_{0.6}54.70.854.7_{0.8}67.30.867.3_{0.8}78.65.678.6_{5.6}16.17.516.1_{7.5}63.07.863.0_{7.8}92.80.492.8_{0.4}71.01.471.0_{1.4}80.93.180.9_{3.1}
Claude77.50.377.5_{0.3}44.31.444.3_{1.4}40.82.440.8_{2.4}62.53.862.5_{3.8}18.52.518.5_{2.5}41.91.341.9_{1.3}80.00.680.0_{0.6}52.60.852.6_{0.8}65.12.065.1_{2.0}85.34.885.3_{4.8}31.917.931.9_{17.9}69.83.969.8_{3.9}92.91.592.9_{1.5}78.42.578.4_{2.5}83.01.683.0_{1.6}
Gemini77.51.777.5_{1.7}42.64.542.6_{4.5}45.51.645.5_{1.6}63.71.563.7_{1.5}4.470.74.4_{70.7}49.71.149.7_{1.1}81.80.281.8_{0.2}54.90.754.9_{0.7}66.10.966.1_{0.9}

Table 15. AUC, TPR, and F1 on EvoBench (top) and RAID (bottom) per training family, with Qwen3-8B-Base as backbone (RoBERTa has none). Each detector is trained on a single MIRAGE generator family and tested on the full benchmark. Per family: meanstd_{\mathrm{std}} across three seeds; Mean: meanstd_{\mathrm{std}} across the nine training families. Rows are grouped by whether the training family is present in the benchmark (EvoBench: GPT, Claude, Gemini, LLaMA, Qwen; RAID: GPT, LLaMA) or absent from it, and sorted within each group by SPADE’s TPR, lowest first. F1 is computed at the Youden-J threshold fitted on each run’s MIRAGE validation split; when a criterion ranks human texts above AI texts on the validation split, no threshold exists and its F1 is 0. Bold indicates the best method on the cross-family mean.

Train ↓Method →RoBERTa AUCRoBERTa TPRRoBERTa F1RAIDAR AUCRAIDAR TPRRAIDAR F1DPIC AUCDPIC TPRDPIC F1ImBD AUCImBD TPRImBD F1SPADE (ours) AUCSPADE (ours) TPRSPADE (ours) F1
EvoBenchIN — training family present in EvoBench
GPT83.12.083.1_{2.0}51.01.051.0_{1.0}61.46.361.4_{6.3}65.41.365.4_{1.3}17.80.117.8_{0.1}57.23.157.2_{3.1}78.60.978.6_{0.9}39.51.039.5_{1.0}64.01.564.0_{1.5}86.70.386.7_{0.3}52.13.052.1_{3.0}74.60.474.6_{0.4}94.31.094.3_{1.0}78.55.078.5_{5.0}84.01.184.0_{1.1}
Claude81.41.881.4_{1.8}50.81.750.8_{1.7}52.12.452.1_{2.4}64.11.364.1_{1.3}15.51.715.5_{1.7}51.43.351.4_{3.3}76.10.676.1_{0.6}37.52.537.5_{2.5}58.42.458.4_{2.4}87.50.987.5_{0.9}53.32.653.3_{2.6}76.00.276.0_{0.2}94.00.594.0_{0.5}80.61.680.6_{1.6}81.31.581.3_{1.5}
Qwen83.21.983.2_{1.9}49.84.749.8_{4.7}66.72.866.7_{2.8}67.31.467.3_{1.4}15.91.615.9_{1.6}62.63.662.6_{3.6}78.20.378.2_{0.3}39.60.539.6_{0.5}61.41.161.4_{1.1}88.01.088.0_{1.0}57.11.957.1_{1.9}75.10.475.1_{0.4}94.51.994.5_{1.9}80.87.580.8_{7.5}83.23.483.2_{3.4}
Gemini83.80.983.8_{0.9}50.63.250.6_{3.2}55.52.955.5_{2.9}55.82.355.8_{2.3}11.41.311.4_{1.3}40.83.540.8_{3.5}76.10.976.1_{0.9}35.23.535.2_{3.5}60.53.560.5_{3.5}89.40.589.4_{0.5}55.31.355.3_{1.3}76.30.576.3_{0.5}95.10.295.1_{0.2}81.70.981.7_{0.9}86.51.386.5_{1.3}
LLaMA85.41.385.4_{1.3}55.21.855.2_{1.8}62.63.562.6_{3.5}63.71.163.7_{1.1}15.51.415.5_{1.4}56.43.356.4_{3.3}78.30.978.3_{0.9}38.31.438.3_{1.4}63.24.163.2_{4.1}88.62.788.6_{2.7}59.17.259.1_{7.2}75.10.275.1_{0.2}95.80.295.8_{0.2}85.91.985.9_{1.9}88.30.588.3_{0.5}
OUT — training family absent from EvoBench
Doubao77.83.577.8_{3.5}29.84.329.8_{4.3}33.11.733.1_{1.7}57.20.457.2_{0.4}8.81.08.8_{1.0}57.71.557.7_{1.5}73.10.873.1_{0.8}33.02.133.0_{2.1}55.92.255.9_{2.2}84.60.984.6_{0.9}44.14.044.1_{4.0}73.30.973.3_{0.9}93.10.893.1_{0.8}74.04.574.0_{4.5}84.31.584.3_{1.5}
DeepSeek81.21.181.2_{1.1}43.02.643.0_{2.6}51.17.951.1_{7.9}65.61.865.6_{1.8}14.91.014.9_{1.0}56.75.656.7_{5.6}77.20.877.2_{0.8}38.80.838.8_{0.8}58.43.358.4_{3.3}87.51.187.5_{1.1}54.21.154.2_{1.1}75.31.075.3_{1.0}93.81.193.8_{1.1}78.22.778.2_{2.7}83.54.783.5_{4.7}
Moonshot84.81.384.8_{1.3}53.60.853.6_{0.8}65.40.365.4_{0.3}62.91.362.9_{1.3}15.83.715.8_{3.7}62.82.362.8_{2.3}78.30.978.3_{0.9}40.71.440.7_{1.4}61.43.761.4_{3.7}89.00.289.0_{0.2}58.95.858.9_{5.8}75.50.875.5_{0.8}94.61.594.6_{1.5}80.93.780.9_{3.7}81.23.381.2_{3.3}
Grok84.30.584.3_{0.5}51.91.151.9_{1.1}62.53.962.5_{3.9}61.70.861.7_{0.8}15.41.415.4_{1.4}50.69.350.6_{9.3}78.10.278.1_{0.2}40.31.640.3_{1.6}64.74.064.7_{4.0}89.00.589.0_{0.5}58.12.258.1_{2.2}75.61.175.6_{1.1}95.50.995.5_{0.9}84.64.684.6_{4.6}87.40.787.4_{0.7}
Mean82.82.382.8_{2.3}48.47.748.4_{7.7}56.710.556.7_{10.5}62.63.962.6_{3.9}14.62.714.6_{2.7}55.26.855.2_{6.8}77.11.877.1_{1.8}38.12.538.1_{2.5}60.92.960.9_{2.9}87.81.587.8_{1.5}54.74.754.7_{4.7}75.20.975.2_{0.9}94.50.094.5_{0.0}80.63.580.6_{3.5}84.42.584.4_{2.5}
RAIDIN — training family present in RAID
GPT78.22.378.2_{2.3}40.33.340.3_{3.3}54.76.854.7_{6.8}69.51.469.5_{1.4}29.62.629.6_{2.6}56.11.456.1_{1.4}81.90.781.9_{0.7}55.81.055.8_{1.0}67.70.967.7_{0.9}82.52.182.5_{2.1}12.32.212.3_{2.2}69.53.969.5_{3.9}92.20.692.2_{0.6}69.76.169.7_{6.1}76.40.576.4_{0.5}
LLaMA84.41.184.4_{1.1}56.93.956.9_{3.9}59.02.859.0_{2.8}70.40.370.4_{0.3}35.31.735.3_{1.7}58.61.958.6_{1.9}82.20.582.2_{0.5}55.80.955.8_{0.9}69.10.969.1_{0.9}87.51.887.5_{1.8}31.02.431.0_{2.4}74.82.174.8_{2.1}94.61.194.6_{1.1}81.93.681.9_{3.6}79.53.279.5_{3.2}
OUT — training family absent from RAID
Claude77.50.377.5_{0.3}44.31.444.3_{1.4}40.82.440.8_{2.4}68.82.968.8_{2.9}30.05.730.0_{5.7}51.63.851.6_{3.8}80.70.580.7_{0.5}54.60.854.6_{0.8}66.91.466.9_{1.4}83.31.783.3_{1.7}15.73.415.7_{3.4}70.63.070.6_{3.0}87.33.887.3_{3.8}52.412.152.4_{12.1}66.28.466.2_{8.4}
Doubao78.81.278.8_{1.2}43.910.443.9_{10.4}43.73.243.7_{3.2}59.51.259.5_{1.2}16.22.216.2_{2.2}50.03.850.0_{3.8}81.11.181.1_{1.1}51.52.251.5_{2.2}66.11.966.1_{1.9}79.60.879.6_{0.8}17.30.817.3_{0.8}64.61.164.6_{1.1}87.51.587.5_{1.5}52.55.752.5_{5.7}66.33.266.3_{3.2}
DeepSeek74.92.474.9_{2.4}31.04.531.0_{4.5}42.411.442.4_{11.4}68.71.468.7_{1.4}26.29.126.2_{9.1}52.82.552.8_{2.5}82.50.782.5_{0.7}55.40.955.4_{0.9}67.70.467.7_{0.4}79.71.779.7_{1.7}14.93.614.9_{3.6}65.11.765.1_{1.7}90.61.690.6_{1.6}69.97.069.9_{7.0}78.44.078.4_{4.0}
Gemini77.51.777.5_{1.7}42.64.542.6_{4.5}45.51.645.5_{1.6}68.91.568.9_{1.5}23.33.223.3_{3.2}55.92.555.9_{2.5}81.70.381.7_{0.3}54.91.054.9_{1.0}65.80.565.8_{0.5}82.41.682.4_{1.6}22.63.622.6_{3.6}67.32.367.3_{2.3}92.31.792.3_{1.7}72.41.072.4_{1.0}76.43.776.4_{3.7}
Qwen83.21.983.2_{1.9}49.84.749.8_{4.7}57.13.557.1_{3.5}71.41.271.4_{1.2}28.84.928.8_{4.9}57.51.157.5_{1.1}82.10.382.1_{0.3}55.21.055.2_{1.0}68.00.868.0_{0.8}88.21.288.2_{1.2}30.16.630.1_{6.6}77.01.577.0_{1.5}92.42.692.4_{2.6}72.411.272.4_{11.2}76.73.876.7_{3.8}
Moonshot81.82.181.8_{2.1}50.13.250.1_{3.2}59.02.259.0_{2.2}72.21.372.2_{1.3}24.89.424.8_{9.4}59.51.459.5_{1.4}82.20.782.2_{0.7}57.40.357.4_{0.3}68.91.168.9_{1.1}86.82.986.8_{2.9}32.77.332.7_{7.3}74.33.574.3_{3.5}92.91.192.9_{1.1}75.33.275.3_{3.2}78.43.478.4_{3.4}
Grok81.51.581.5_{1.5}45.63.645.6_{3.6}54.56.454.5_{6.4}68.72.168.7_{2.1}24.07.124.0_{7.1}55.32.255.3_{2.2}81.20.181.2_{0.1}53.96.153.9_{6.1}67.90.367.9_{0.3}79.81.679.8_{1.6}16.72.916.7_{2.9}65.13.665.1_{3.6}93.21.993.2_{1.9}77.92.577.9_{2.5}80.30.280.3_{0.2}
Mean79.42.879.4_{2.8}44.71.044.7_{1.0}50.77.550.7_{7.5}68.73.768.7_{3.7}26.55.526.5_{5.5}55.33.255.3_{3.2}81.70.681.7_{0.6}54.91.654.9_{1.6}67.61.067.6_{1.0}83.33.483.3_{3.4}21.57.621.5_{7.6}69.84.769.8_{4.7}91.52.591.5_{2.5}69.410.369.4_{10.3}75.54.475.5_{4.4}

Table 16. AUC, TPR, and F1 on EvoBench (top) and RAID (bottom) per training family, with LLaMA-3.1-8B as backbone (RoBERTa has none). Each detector is trained on a single MIRAGE generator family and tested on the full benchmark. Per family: meanstd_{\mathrm{std}} across three seeds; Mean: meanstd_{\mathrm{std}} across the nine training families. Rows are grouped by whether the training family is present in the benchmark (EvoBench: GPT, Claude, Gemini, LLaMA, Qwen; RAID: GPT, LLaMA) or absent from it, and sorted within each group by SPADE’s TPR, lowest first. F1 is computed at the Youden-J threshold fitted on each run’s MIRAGE validation split; when a criterion ranks human texts above AI texts on the validation split, no threshold exists and its F1 is 0. Bold indicates the best method on the cross-family mean.

Train ↓\downarrow / Method →\rightarrowRoBERTaRAIDARDPICImBDSPADE (ours)
EvoBenchAUCTPRF1AUCTPRF1AUCTPRF1AUCTPRF1AUCTPRF1
IN — training family present in EvoBench
Claude81.41.881.4_{1.8}50.81.750.8_{1.7}52.12.452.1_{2.4}56.01.956.0_{1.9}8.481.08.4_{81.0}52.61.852.6_{1.8}74.10.774.1_{0.7}36.10.936.1_{0.9}57.60.557.6_{0.5}88.61.688.6_{1.6}54.84.454.8_{4.4}75.71.475.7_{1.4}87.42.387.4_{2.3}55.88.755.8_{8.7}76.95.476.9_{5.4}
GPT83.12.083.1_{2.0}51.01.051.0_{1.0}61.46.361.4_{6.3}50.13.150.1_{3.1}6.80.96.8_{0.9}31.45.331.4_{5.3}77.30.577.3_{0.5}38.61.038.6_{1.0}60.62.960.6_{2.9}85.81.985.8_{1.9}47.85.347.8_{5.3}72.41.072.4_{1.0}88.82.488.8_{2.4}56.710.256.7_{10.2}78.63.078.6_{3.0}
Qwen83.21.983.2_{1.9}49.84.749.8_{4.7}66.72.866.7_{2.8}55.43.655.4_{3.6}8.54.18.5_{4.1}59.43.359.4_{3.3}77.00.377.0_{0.3}38.10.938.1_{0.9}59.08.659.0_{8.6}88.00.588.0_{0.5}53.00.953.0_{0.9}73.31.973.3_{1.9}89.70.489.7_{0.4}66.04.166.0_{4.1}81.62.781.6_{2.7}
Gemini83.80.983.8_{0.9}50.63.250.6_{3.2}55.52.955.5_{2.9}50.72.850.7_{2.8}7.54.27.5_{4.2}36.013.236.0_{13.2}75.01.175.0_{1.1}35.43.335.4_{3.3}56.56.756.5_{6.7}87.21.787.2_{1.7}52.73.752.7_{3.7}74.12.474.1_{2.4}90.14.190.1_{4.1}67.94.967.9_{4.9}78.05.078.0_{5.0}
LLaMA85.41.385.4_{1.3}55.21.855.2_{1.8}62.63.562.6_{3.5}54.92.654.9_{2.6}8.310.38.3_{10.3}54.32.454.3_{2.4}76.90.976.9_{0.9}38.40.438.4_{0.4}60.92.460.9_{2.4}88.62.288.6_{2.2}56.25.856.2_{5.8}74.52.174.5_{2.1}93.70.893.7_{0.8}74.33.274.3_{3.2}83.31.183.3_{1.1}
OUT — training family absent from EvoBench
Doubao77.83.577.8_{3.5}29.84.329.8_{4.3}33.11.733.1_{1.7}50.42.550.4_{2.5}7.44.17.4_{4.1}48.63.148.6_{3.1}73.01.073.0_{1.0}34.11.034.1_{1.0}55.21.655.2_{1.6}79.12.579.1_{2.5}32.35.632.3_{5.6}71.44.671.4_{4.6}67.20.567.2_{0.5}34.54.534.5_{4.5}54.60.454.6_{0.4}
DeepSeek81.21.181.2_{1.1}43.02.643.0_{2.6}51.17.951.1_{7.9}56.12.356.1_{2.3}9.360.99.3_{60.9}49.69.549.6_{9.5}76.20.976.2_{0.9}37.01.237.0_{1.2}58.23.458.2_{3.4}86.91.386.9_{1.3}50.74.550.7_{4.5}74.31.374.3_{1.3}88.11.588.1_{1.5}57.413.657.4_{13.6}81.21.481.2_{1.4}
Moonshot84.81.384.8_{1.3}53.60.853.6_{0.8}65.40.365.4_{0.3}51.53.151.5_{3.1}7.55.27.5_{5.2}51.44.051.4_{4.0}76.70.476.7_{0.4}40.00.940.0_{0.9}62.81.562.8_{1.5}88.21.788.2_{1.7}51.56.351.5_{6.3}74.60.974.6_{0.9}90.03.890.0_{3.8}67.37.567.3_{7.5}80.50.280.5_{0.2}
Grok84.30.584.3_{0.5}51.91.151.9_{1.1}62.53.962.5_{3.9}53.01.453.0_{1.4}7.80.57.8_{0.5}45.811.345.8_{11.3}76.80.976.8_{0.9}39.11.739.1_{1.7}57.42.457.4_{2.4}88.51.388.5_{1.3}52.73.352.7_{3.3}73.91.273.9_{1.2}92.52.292.5_{2.2}72.07.072.0_{7.0}81.92.581.9_{2.5}
Mean82.82.382.8_{2.3}48.47.748.4_{7.7}56.710.556.7_{10.5}53.12.553.1_{2.5}7.99.07.9_{9.0}47.78.947.7_{8.9}75.91.575.9_{1.5}37.41.937.4_{1.9}58.72.458.7_{2.4}86.83.086.8_{3.0}50.27.150.2_{7.1}73.81.373.8_{1.3}87.57.987.5_{7.9}61.312.161.3_{12.1}77.48.877.4_{8.8}
Train ↓\downarrow / Method →\rightarrowRoBERTaRAIDARDPICImBDSPADE (ours)
RAIDAUCTPRF1AUCTPRF1AUCTPRF1AUCTPRF1AUCTPRF1
IN — training family present in RAID
GPT78.22.378.2_{2.3}40.33.340.3_{3.3}54.76.854.7_{6.8}49.91.749.9_{1.7}9.27.19.2_{7.1}35.02.335.0_{2.3}81.40.381.4_{0.3}54.22.254.2_{2.2}67.41.167.4_{1.1}82.10.982.1_{0.9}5.760.85.7_{60.8}68.83.968.8_{3.9}85.53.685.5_{3.6}45.09.545.0_{9.5}66.16.866.1_{6.8}
LLaMA84.41.184.4_{1.1}56.93.956.9_{3.9}59.02.859.0_{2.8}50.51.250.5_{1.2}7.812.07.8_{12.0}40.72.040.7_{2.0}82.20.782.2_{0.7}57.31.557.3_{1.5}69.70.469.7_{0.4}86.83.186.8_{3.1}24.720.224.7_{20.2}72.74.072.7_{4.0}90.70.790.7_{0.7}66.65.766.6_{5.7}74.81.174.8_{1.1}
OUT — training family absent from RAID
Doubao78.81.278.8_{1.2}43.910.443.9_{10.4}43.73.243.7_{3.2}49.83.249.8_{3.2}8.392.48.3_{92.4}38.53.538.5_{3.5}81.00.681.0_{0.6}53.80.453.8_{0.4}66.60.466.6_{0.4}80.41.780.4_{1.7}17.210.317.2_{10.3}63.86.563.8_{6.5}74.10.774.1_{0.7}26.30.626.3_{0.6}56.73.256.7_{3.2}
Gemini77.51.777.5_{1.7}42.64.542.6_{4.5}45.51.645.5_{1.6}52.01.552.0_{1.5}10.51.610.5_{1.6}38.82.638.8_{2.6}82.90.882.9_{0.8}56.20.656.2_{0.6}66.70.966.7_{0.9}85.41.285.4_{1.2}20.86.920.8_{6.9}68.07.768.0_{7.7}83.24.683.2_{4.6}36.46.336.4_{6.3}64.014.164.0_{14.1}
DeepSeek74.92.474.9_{2.4}31.04.531.0_{4.5}42.411.442.4_{11.4}52.42.352.4_{2.3}10.70.410.7_{0.4}35.33.135.3_{3.1}82.10.282.1_{0.2}55.62.055.6_{2.0}68.00.468.0_{0.4}83.61.583.6_{1.5}6.311.76.3_{11.7}70.80.470.8_{0.4}79.51.279.5_{1.2}36.514.836.5_{14.8}63.47.863.4_{7.8}
Qwen80.11.380.1_{1.3}47.41.547.4_{1.5}57.13.557.1_{3.5}54.71.554.7_{1.5}11.41.511.4_{1.5}42.70.942.7_{0.9}82.00.582.0_{0.5}55.30.455.3_{0.4}68.20.368.2_{0.3}85.30.585.3_{0.5}19.211.419.2_{11.4}73.00.973.0_{0.9}83.53.083.5_{3.0}49.68.649.6_{8.6}66.74.466.7_{4.4}
Moonshot81.82.181.8_{2.1}50.13.250.1_{3.2}59.02.259.0_{2.2}53.21.353.2_{1.3}10.92.210.9_{2.2}39.52.239.5_{2.2}82.00.882.0_{0.8}55.51.555.5_{1.5}68.12.568.1_{2.5}85.81.485.8_{1.4}27.03.927.0_{3.9}72.22.272.2_{2.2}84.62.384.6_{2.3}54.89.054.8_{9.0}71.93.971.9_{3.9}
Grok81.51.581.5_{1.5}45.63.645.6_{3.6}54.64.554.6_{4.5}50.61.450.6_{1.4}8.91.48.9_{1.4}40.63.240.6_{3.2}81.80.581.8_{0.5}55.71.155.7_{1.1}68.10.768.1_{0.7}83.71.183.7_{1.1}12.37.512.3_{7.5}69.41.169.4_{1.1}83.67.783.6_{7.7}54.89.054.8_{9.0}71.92.971.9_{2.9}
Claude77.50.577.5_{0.5}44.31.444.3_{1.4}−40.82.4-40.8_{2.4}51.61.151.6_{1.1}11.22.811.2_{2.8}35.71.835.7_{1.8}81.20.481.2_{0.4}54.21.654.2_{1.6}68.00.568.0_{0.5}86.21.086.2_{1.0}22.54.522.5_{4.5}70.41.170.4_{1.1}84.10.984.1_{0.9}55.05.855.0_{5.8}62.07.762.0_{7.7}
Mean79.42.879.4_{2.8}44.77.144.7_{7.1}50.75.550.7_{5.5}51.66.951.6_{6.9}9.91.39.9_{1.3}38.27.738.2_{7.7}81.80.681.8_{0.6}55.31.155.3_{1.1}67.90.967.9_{0.9}84.24.184.2_{4.1}17.77.717.7_{7.7}69.92.969.9_{2.9}83.67.783.6_{7.7}47.012.347.0_{12.3}65.83.565.8_{3.5}

Table 17. AUC, TPR, and F1 on EvoBench (top) and RAID (bottom) per training family, with Falcon-7B as backbone (RoBERTa has none). Each detector is trained on a single MIRAGE generator family and tested on the full benchmark. Per family: meanstd_{\mathrm{std}} across three seeds; Mean: meanstd_{\mathrm{std}} across the nine training families. Rows are grouped by whether the training family is present in the benchmark (EvoBench: GPT, Claude, Gemini, LLaMA, Qwen; RAID: GPT, LLaMA) or absent from it, and sorted within each group by SPADE’s TPR, lowest first. F1 is computed at the Youden-J threshold fitted on each run’s MIRAGE validation split. Bold indicates the best method on the cross-family mean.

Figure 10. AUC and TPR of 5 baselines on RAID benchmark against SPADE grouped by adversarial attacks, decoding strategy, and with repetition penalty or not.

Figure 11. AUC and TPR of 5 baselines on RAID benchmark against SPADE grouped by adversarial attacks, decoding strategy, and with repetition penalty or not.

Attack ↓ Method →LikelihoodEntropyDNA-GPTFast-DetectGPTBinocularsSPADE
AUCTPRF1AUCTPRF1AUCTPRF1AUCTPRF1AUCTPRF1AUCTPRF1
No attack87.662.470.285.749.568.279.439.665.764.030.755.470.848.562.498.694.194.1
Alt. spelling84.747.967.682.344.764.774.730.962.968.536.256.773.041.561.499.093.694.1
Article deletion87.757.375.682.737.971.280.933.068.780.350.567.184.559.274.797.490.691.9
Homoglyph79.342.652.177.631.952.177.041.553.363.524.051.574.342.657.197.689.389.5
Insert paragraphs86.757.369.685.250.468.979.942.768.168.030.858.372.742.762.799.297.295.1
Number91.771.671.589.158.968.885.451.670.471.742.159.274.953.765.397.489.390.0
Paraphrase67.924.452.361.518.946.463.723.349.465.315.751.575.932.254.282.240.655.3
Misspelling88.561.167.287.453.765.979.140.062.263.132.651.267.944.258.099.394.794.2
Synonym77.034.855.372.019.655.965.416.136.469.722.336.168.629.535.594.883.686.2
Upper/lower86.944.373.880.340.265.181.333.069.772.037.159.677.049.567.198.292.492.3
Whitespace84.954.872.481.042.368.076.438.565.669.437.558.673.647.164.597.389.990.9
Zero-width space85.949.551.278.618.651.283.023.751.280.341.265.185.247.464.089.766.660.5
Mean83.749.664.479.837.961.777.034.059.870.233.655.975.244.560.495.684.485.5
Attack ↓ Method →NTSRoBERTaRAIDARDPICImBDSPADE
AUCTPRF1AUCTPRF1AUCTPRF1AUCTPRF1AUCTPRF1AUCTPRF1
No attack74.336.662.690.068.374.969.424.455.691.273.382.795.284.587.998.694.194.1
Alt. spelling72.828.759.491.369.976.579.028.059.291.373.483.395.985.589.799.093.694.1
Article deletion69.122.356.989.465.470.774.028.949.891.771.280.695.581.688.097.490.691.9
Homoglyph73.443.653.663.812.544.540.76.384.6175.939.456.294.168.159.497.689.389.5
Insert paragraphs73.323.163.692.374.672.679.329.366.193.976.182.094.789.290.399.297.295.1
Number76.038.963.986.553.068.768.120.048.991.667.480.991.951.980.597.489.390.0
Paraphrase67.422.253.867.427.843.651.05.5621.178.342.265.383.234.864.582.240.655.3
Misspelling75.938.960.589.969.876.782.042.865.195.680.785.195.992.585.199.394.794.2
Synonym37.68.0436.883.855.768.576.125.663.190.969.982.593.066.778.594.883.686.2
Upper/lower66.622.752.992.772.273.385.445.471.791.577.083.395.783.289.598.292.492.3
Whitespace67.919.253.686.061.557.579.427.265.885.855.171.493.473.784.197.389.990.9
Zero-width space84.553.666.982.858.464.639.848.146.054.928.516.587.856.751.289.766.660.5
Mean69.529.256.581.451.759.368.624.147.385.660.171.693.170.378.295.684.485.5

Table 18. AUC, TPR, and F1 on RAID per adversarial attack, with Qwen3-8B-Base as backbone. Each attack compares the instruct generators’ texts with the human texts carrying the same attack, pooled over the eight domains. Mean averages the eleven attacks, excluding No attack. F1 is computed at the Youden-J threshold fitted on each run’s MIRAGE validation split. When a criterion ranks human texts above AI texts on the validation split, no threshold exists and its F1 is 0. Bold indicates the best method per row and metric within each panel.

Attack ↓Method →Likelihood AUCLikelihood TPRLikelihood F1Entropy AUCEntropy TPREntropy F1DNA-GPT AUCDNA-GPT TPRDNA-GPT F1Fast-DetectGPT AUCFast-DetectGPT TPRFast-DetectGPT F1Binoculars AUCBinoculars TPRBinoculars F1SPADE AUCSPADE TPRSPADE F1
No attack82.943.664.978.729.70.0073.911.962.288.971.371.291.878.277.898.692.192.3
Alt. spelling80.129.861.875.820.20.0072.020.259.890.471.576.091.375.580.597.990.891.9
Article deletion84.041.768.774.826.20.0080.128.266.596.391.385.097.393.291.596.688.786.9
Homoglyph78.451.152.275.336.20.0079.537.253.383.355.355.489.774.564.290.457.072.5
Insert paragraphs83.353.066.080.438.50.0076.253.860.190.476.975.892.380.383.398.795.294.4
Number88.561.163.984.147.40.0080.717.962.992.473.376.194.683.284.896.286.086.2
Paraphrase65.225.649.557.612.20.0058.112.247.473.425.851.580.851.154.175.833.354.7
Misspelling85.848.461.282.041.10.0077.324.258.490.181.271.891.085.379.198.190.991.7
Synonym74.222.358.365.714.30.0068.620.547.786.561.670.486.468.871.796.183.188.0
Upper/lower83.833.066.874.622.70.0079.525.865.996.386.685.597.189.789.997.390.490.2
Whitespace80.437.566.776.230.80.0073.222.161.392.078.880.793.082.785.796.086.589.0
Zero-width space82.641.251.275.013.40.0081.539.251.292.982.755.092.778.457.381.122.755.5
Mean80.640.460.674.727.50.0075.127.457.789.571.471.291.578.476.693.175.081.9
No attackNTS80.855.469.890.068.374.979.135.368.992.372.983.094.478.587.998.692.192.3
Alt. spellingNTS82.746.871.191.369.976.580.831.267.791.475.281.596.188.789.397.990.891.9
Article deletionNTS86.459.277.789.465.470.782.651.171.389.967.079.995.585.886.596.688.786.9
HomoglyphNTS72.043.659.163.812.544.550.93.1916.975.539.454.192.962.168.690.457.072.5
Insert paragraphsNTS80.741.970.192.374.672.683.345.072.493.076.981.497.083.587.398.795.294.4
NumberNTS80.435.867.286.553.068.781.144.266.490.366.777.992.362.179.896.286.086.2
ParaphraseNTS70.836.750.667.427.843.665.811.123.879.053.064.286.047.867.375.833.354.7
MisspellingNTS82.452.666.889.969.876.786.055.875.693.479.682.194.983.286.598.190.991.7
SynonymNTS40.87.1437.783.855.768.576.517.959.489.571.480.294.375.385.096.183.188.0
Upper/lowerNTS80.257.769.392.772.272.384.440.971.991.780.184.296.686.689.897.390.490.2
WhitespaceNTS78.028.865.786.061.557.581.632.171.888.359.073.693.378.584.896.086.589.0
Zero-width spaceNTS87.461.963.352.85.840.6840.95.1517.857.610.717.083.438.559.381.122.755.5
MeanNTS76.542.963.581.451.759.374.030.755.985.461.770.692.972.080.493.175.081.9

Table 19. AUC, TPR, and F1 on RAID per adversarial attack, with LLaMA-3.1-8B as backbone. Each attack compares the instruct generators’ texts with the human texts carrying the same attack, pooled over the eight domains. Mean averages the eleven attacks, excluding No attack. F1 is computed at the Youden-J threshold fitted on each run’s MIRAGE validation split. When a criterion ranks human texts above AI texts on the validation split, no threshold exists and its F1 is 0. Bold indicates the best method per row and metric within each panel.

Attack ↓Method →Likelihood AUCLikelihood TPRLikelihood F1Entropy AUCEntropy TPREntropy F1DNA-GPT AUCDNA-GPT TPRDNA-GPT F1Fast-DetectGPT AUCFast-DetectGPT TPRFast-DetectGPT F1Binoculars AUCBinoculars TPRBinoculars F1SPADE AUCSPADE TPRSPADE F1
No attack81.479.70.0069.316.80.0083.253.565.291.888.177.292.089.183.484.883.587.1
Alt. spelling79.421.30.0063.012.80.0081.548.964.491.886.279.992.486.284.795.687.689.4
Article deletion84.450.50.0063.97.770.0089.762.170.896.592.290.096.894.295.094.281.686.8
Homoglyph77.939.40.0067.012.80.0077.242.653.567.823.452.874.427.759.870.717.728.3
Insert paragraphs82.847.00.0070.615.40.0081.649.663.891.086.378.791.486.383.494.577.288.3
Number90.564.20.0076.917.90.0089.664.266.594.885.381.096.294.787.092.176.882.5
Paraphrase67.225.60.0055.012.20.0066.824.451.775.740.353.485.161.155.774.630.054.8
Misspelling84.538.90.0071.020.00.0086.365.362.693.189.578.993.891.687.494.085.387.1
Synonym76.430.40.0058.29.820.0080.643.866.186.764.371.886.867.976.392.872.683.1
Upper/lower84.239.20.0064.913.40.0089.459.873.297.890.790.298.594.894.394.983.587.3
Whitespace78.941.30.0064.010.60.0083.162.868.792.887.587.693.290.491.291.571.879.8
Zero-width space62.613.40.0052.67.220.0057.88.2551.259.89.2851.264.09.5451.255.29.2822.1
Mean79.037.40.0064.312.70.0080.348.863.086.268.674.188.073.178.786.463.071.8
No attackNTS93.286.180.990.068.374.953.915.538.891.169.682.189.670.080.194.883.587.1
Alt. spellingNTS94.180.985.091.369.976.557.016.340.691.775.982.292.478.481.095.687.689.4
Article deletionNTS96.891.389.289.465.470.754.19.0641.691.069.380.590.772.579.194.281.686.8
HomoglyphNTS87.566.066.563.812.544.545.09.5724.474.633.750.484.647.944.170.717.728.3
Insert paragraphsNTS94.381.283.792.374.672.664.621.750.992.272.679.993.177.579.494.577.288.3
NumberNTS94.771.682.686.553.068.750.85.9631.290.567.477.386.350.272.592.176.882.5
ParaphraseNTS81.352.256.667.427.843.657.87.4129.876.440.463.081.443.065.674.630.054.8
MisspellingNTS94.187.484.389.969.876.760.89.8240.493.981.184.589.768.475.794.085.387.1
SynonymNTS60.322.352.183.855.768.553.79.8237.989.865.278.691.153.378.292.872.683.1
Upper/lowerNTS96.289.790.292.772.272.358.612.741.991.775.382.793.876.684.494.983.587.3
WhitespaceNTS95.579.885.386.061.557.559.017.644.985.853.871.488.262.574.891.571.879.8
Zero-width spaceNTS71.225.851.652.85.840.6846.42.0640.959.27.5622.548.77.9051.255.29.2822.1
MeanNTS87.868.075.281.451.759.355.311.138.685.258.470.385.358.071.586.463.071.8

Table 20. AUC, TPR, and F1 on RAID per adversarial attack, with Falcon-7B as backbone. Each attack compares the instruct generators’ texts with the human texts carrying the same attack, pooled over the eight domains. Mean averages the eleven attacks, excluding No attack. F1 is computed at the Youden-J threshold fitted on each run’s MIRAGE validation split. When a criterion ranks human texts above AI texts on the validation split, no threshold exists and its F1 is 0. Bold indicates the best method per row and metric within each panel.

Train ↓ Test →English F1BEnglish F1FEnglish F1@0Chinese F1BChinese F1FChinese F1@0Spanish F1BSpanish F1FSpanish F1@0Arabic F1BArabic F1FArabic F1@0
English99.9099.9099.9095.5296.6798.8783.5292.3799.6871.5378.6495.99
Chinese98.7698.7699.6099.9699.9699.9797.1797.1797.9493.5993.5997.62
Spanish98.7598.7599.6290.9890.9898.5899.9799.97100.0093.7693.7696.06
Arabic99.2689.3199.3299.8298.3699.8298.4497.6498.4999.9199.7799.90
French99.6099.6099.6199.9199.9199.9298.0798.0798.0797.8997.8998.34
Russian99.2452.5999.3999.9699.2799.9697.3596.6397.3899.7799.1799.78
Portuguese99.5297.2699.5599.8596.5099.8798.8598.3198.8297.9499.1498.60
German99.5196.3599.5299.6997.9699.9198.5597.7698.5299.6798.9299.72
Train ↓ Test →French F1BFrench F1FFrench F1@0Russian F1BRussian F1FRussian F1@0Portuguese F1BPortuguese F1FPortuguese F1@0German F1BGerman F1FGerman F1@0
English89.9993.6799.4180.4383.6294.7788.8593.5999.5864.4572.9093.91
Chinese94.9694.9699.0689.5189.5196.1897.9297.9299.7281.8981.8993.88
Spanish99.5899.5899.7691.8991.8996.2699.8599.8599.9498.1298.1299.04
Arabic99.4596.8699.4499.2596.3199.0199.8798.9599.8897.9697.1097.71
French99.9699.9699.9798.2898.2898.6299.9799.9799.9797.9497.9498.74
Russian99.6299.0299.6699.8799.7399.8599.9199.6799.9299.7998.9299.75
Portuguese99.7799.8399.8495.6898.3897.0199.9499.9299.9494.1698.1296.23
German99.7899.4299.7899.4298.9899.4699.8299.3999.8299.9499.7599.95

Table 21. SPADE on DetectRL-X (Wu et al., 2026) under its cross-language protocol. SPADE is trained and tuned on a single language train split, and tested on the others. F1B: macro-F1 at the Youden-J threshold fitted on the training split; F1F: macro-F1 at the threshold with FPR = 1% on the training split; F1@0: macro-F1 at the fixed threshold τ=0\tau= 0.

References

  1. [1]Aaditya Bhat. 2023. Gpt-wiki-intro (revision 0e458f5).
  2. [2]Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Al-shamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, Daniele Mazzotta, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. 2023. The falcon series of open language models. Preprint, arXiv:2311.16867.arxiv.org/abs/2311.16867
  3. [3]David Bamman and Noah A. Smith. 2013. New alignment methods for discriminative book summarization. Preprint, arXiv:1305.1319.arxiv.org/abs/1305.1319
  4. [4]Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. 2024. Fast-DetectGPT: Efficient zero-shot detection of machine-generated text via conditional probability curvature. In The Twelfth International Conference on Learning Representations.arxiv.org/abs/2310.05130
  5. [5]Hongrui Bao, Yubing Ren, Jinhua You, Fang Fang, Shi Wang, and Yanan Cao. 2026. Once a response, always a response: Detecting llm-generated text via latent prompt restoration. Preprint, arXiv:2608.05741.arxiv.org/abs/2608.05741
  6. [7]Michał Bień, Michał Gilski, Martyna Maciejewska, Wojciech Taisner, Dawid Wisniewski, and Agnieszka Lawrynowicz. 2020. RecipeNLG: A cooking recipes dataset for semi-structured text generation. In Proceedings of the 13th International Conference on Natural Language Generation, pages 22–28, Dublin, Ireland. Association for Computational Linguistics.
  7. [8]Creston Brooks, Samuel Eggert, and Denis Peskoff. 2024. The rise of AI-generated content in Wikipedia. In Proceedings of the First Workshop on Advancing Natural Language Processing for Wikipedia, pages 67–79, Miami, Florida, USA. Association for Computational Linguistics.arxiv.org/abs/2410.08044
  8. [9]Jiagi Chen, Xiaoye Zhu, Tianyang Liu, Ying Chen, Chen Xinhui, Yiwen Yuan, Chak Tou Leong, Zuchao Li, Long Tang, Lei Zhang, Chenyu Yan, Guanghao Mei, Jie Zhang, and Lefeí Zhang. 2025. Imitate before detect: aligning machine stylistic preference for machine-revised text detection. In Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence and Thirty-Seventh Conference on Innovative Applications of Artificial Intelligence and Fifteenth Symposium on Educational Advances in Artificial Intelligence, AAAI ’25/IAAI ’25/EAAI ’25. AAAI Press.arxiv.org/abs/2412.10432

Paper details

vlnc:W_cebf666m
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY-SA 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents