PGuard: A Native, Streaming, and Plug-in Guardrail for Multimodal Large Language Models

https://github.com/Alibaba-AAIG/PGuard

https://huggingface.co/collections/Alibaba-AAIG/pguard

https://modelscope.cn/collections/Alibaba-AAIG/PGuard

Introduction

Large language models (LLMs) and multimodal large language models (MLLMs) are increasingly deployed in interactive applications, including agentic generation, voice chat, and incremental content rendering. In these settings, model responses are streamed to users as tokens are generated, rather than being presented only after generation is complete, shifting safety moderation from a post-generation task to an online streaming requirement. Specifically, a moderation decision made only after generation completes cannot prevent harmful content that has already been exposed to users. Effective moderation must therefore keep pace with token generation while introducing minimal computational overhead to remain practical at serving scale.

Existing guardrails address this requirement through two broad approaches. External guardrails, such as Qwen3Guard [74], employ separate models to classify user inputs and model responses. Although they can achieve strong detection accuracy, they introduce additional inference costs. More importantly, post-hoc moderation can only identify unsafe content after it has been generated, whereas streaming moderation requires frequent inference by a separate model to detect risks during generation. Furthermore, operating outside the foundation model limits their ability to directly leverage its internal representations. Native guardrails instead exploit signals from the protected model itself. HiddenDetect [25] detects input-side risks before generation, while NExT-Guard [9] monitors internal features at the token level. Despite this progress, existing native approaches remain constrained by the effectiveness of their safety representations and the runtime cost of extracting and processing them. Our evaluations further show that existing native approaches lag behind leading external guardrails in aggregate detection performance while still incurring appreciable serving overhead.

These limitations motivate a more effective way to leverage the backbone’s internal representations for native safety modeling. The intermediate representations of a frontier foundation model encode rich semantic and contextual information about the content being generated, providing a natural basis for identifying safety risks without duplicating much of the computation performed by a separate guardrail model. However, effectively translating these representations into safety signals raises two challenges. First, safety-relevant features are not uniformly distributed across layers or token positions, making adaptive layer selection important for effective representation extraction. Second, risks may emerge from relationships among tokens and evolve as a response unfolds, which cannot always be captured by classifying each position independently. Addressing these challenges requires both adaptive representation extraction and effective modeling of dependencies across token positions.

To address these challenges, we present PGuard, a native, streaming, and plug-in guardrail built on a frozen foundation model backbone. PGuard introduces dynamic layer gating to adaptively select and aggregate residual-stream representations at each token position, enabling the guardrail to capture safety-relevant features across different layers. Based on these representations, PGuard supports two complementary modeling paradigms. The stateless variant classifies each token position independently, while the sequence variants explicitly model dependencies across positions to capture how risks emerge and evolve during generation. We instantiate sequence modeling with multi-scale causal EMA-attention and GRU-based recurrent modules, offering different trade-offs between detection quality and computational cost. To enable localized safety supervision without requiring exhaustive token-level annotations, PGuard adopts segment-level supervision, computing loss only at annotated positions and for tasks with available labels. Our experiments show that sequence modeling improves response-level detection over the stateless variant across both text and multimodal tasks.

Beyond detection efficacy, deploying native guardrails efficiently requires coordinating safety computation with the backbone’s generation process. Mainstream language model inference and serving engines (e.g., vLLM [29] and SGLang [76]) coordinate batching, parallel execution, and GPU synchronization to sustain high throughput. Accessing intermediate representations and reporting risk scores introduces additional dependencies into this execution process. In particular, poorly placed synchronization can stall active batches, making the resulting overhead substantially greater than the guard computation itself. PGuard therefore incorporates serving-engine constraints into its execution design. We integrate PGuard with SGLang, aligning risk computation and reporting with the engine’s scheduling mechanisms. The integration supports both stateless and sequence variants without modifying the SGLang codebase. Beyond SGLang, we are currently working on a vLLM integration to extend the same serving-aware execution design to another widely used inference and serving engine.

We evaluate PGuard on more than 30 benchmarks covering text and multimodal safety, jailbreaks, cyber and CBRN risks, multilingual scenarios, and real-world traffic. On text-query benchmarks with Qwen3.5-9B, PGuard achieves an average $F_1$ score of 0.836, approaching that of Qwen3Guard. On multimodal query benchmarks with Qwen3.5-9B, it achieves an average $F_1$ score of 0.741, compared with 0.638 for HiddenDetect and 0.474 for Llama Guard 4. On the 100B-scale Qwen3.5-122B-A10B backbone, end-to-end latency overhead remains below 1.3% for stateless and 2.4% for sequence variants. These results demonstrate that native guardrails can achieve competitive detection quality while imposing limited overhead on large-scale model serving.

Our main contributions are:

• We propose PGuard, a native streaming guardrail that leverages dynamic layer gating to extract token-wise safety representations from a frozen backbone. We further introduce complementary stateless and sequence-based modeling architectures to support both independent token classification and sequential risk modeling.

• We develop a serving-aware execution strategy that coordinates safety computation with the scheduling and execution mechanisms of model inference engines. Our SGLang integration supports both modeling variants while minimizing redundant computation and synchronization overhead, without modifying the inference engine’s codebase.

• We conduct extensive evaluations across more than 30 text and multimodal safety benchmarks and real-world traffic. Our analysis demonstrates the benefits of sequence modeling, characterizes the trade-offs between detection quality and serving overhead, and examines the complementary roles of native screening and external adjudication.

Methodology

Problem Formulation and Design Goals

We consider a frozen backbone model, either an LLM or an MLLM, serving multi-turn conversations with text-only or multimodal inputs. A native guardrail leverages the backbone’s intermediate representations to detect risks in user inputs during prefill and in generated responses during decode. Formally, let the backbone process a token sequence $x_{1:T}$ and produce hidden states $H^{(l)} \in\mathbb{R}^{T \times d}$ at layer $l$, where the $t$-th row $h_t^{(l)} \in\mathbb{R}^{d}$ represents the hidden state at sequence position $t$. The guardrail is defined as

which maps intermediate representations from selected backbone layers $l_1,\ldots,l_m$ to per-position risk scores. Here, $\mathcal{Y}$ denotes the shared set of risk categories across safety tasks, and $r_t$ contains the predicted probability of each category at position $t$. A position is flagged if the predicted probability of any risk category exceeds its corresponding threshold.

A production-ready native guardrail must satisfy four design goals, which guide the design of PGuard:

• Competitive detection performance. The guardrail should perform competitively with dedicated external detectors.

• Native integration with generation. Risk scoring should leverage the backbone’s forward pass, enabling per-position detection during prefill and decode without modifying the backbone’s computation.

• Low serving overhead. The additional computation should contribute only a small fraction of the end-to-end serving cost, targeting a few percent overhead on production backbones.

• Broad risk coverage. A unified training framework should support text and multimodal inputs, jailbreaks, multilingual content, and emerging risks under a shared label taxonomy.

Figure 1 illustrates the PGuard architecture. The frozen backbone supplies residual-stream features from selected layers. Dynamic layer gating fuses the captured features at each position, and the fused feature is modeled either by a stateless variant or by a sequence variant based on multi-scale causal EMA-attention or a GRU-based recurrent module. The risk classifiers then produce risk scores under the unified taxonomy during both prefill and decode.

Multi-Layer Hidden State Extraction

Safety-relevant information is not distributed uniformly across layers. At each capture layer, PGuard reads the residual stream, i.e., the tensor that enters that layer. The capture is read-only and adds no state to the frozen backbone. Training and serving capture the residual stream at matching layer boundaries, reconstructing it from separate hidden-state and residual buffers where necessary (§ 3.1).

Let $\mathcal{L} = \{l_1,\ldots,l_m\}$ denote the set of capture layers. For a sequence of $T$ positions, feature extraction produces a tensor $F \in\mathbb{R}^{m \times T \times d}$, where $d$ is the backbone hidden size. In an MLLM, the visual tokens simply occupy positions in the same sequence as text tokens. Thus, image and text evidence is fused by the backbone before feature extraction.

Dynamic Layer Gating

The most discriminative safety signals can emerge at different depths across inputs and even across positions within the same input. To accommodate this variation, PGuard employs dynamic layer gating in two stages. During calibration-set pretraining, with the backbone frozen, the gate operates over a larger candidate set of layers, and the learned gating profiles determine the capture-layer set $\mathcal{L}$. During full training and deployment, the gate operates over this fixed set and adaptively weights the captured representations at each position. We formulate the gate over $\mathcal{L}$, with the same formulation applied to the candidate set during pretraining. The gate computes a score for each position-layer pair:

where $w_g$ is a learned projection and $\mathcal{T}_t$ contains the indices of the top-$L$ layers by score ranked by their scores at position $t$. During training, Gumbel noise is added to the scores before selection. In full training and deployment, we set $L = |\mathcal{L}|$, allowing all captured layers to participate while retaining position-specific weights. The fused representation is then computed as

yielding a position-adaptive soft mixture of the captured layer representation. This gating mechanism allows a single training recipe to accommodate backbones of different depths by learning layer preferences independently at each position (§ 5 analyzes the learned gating profiles).

Representation Modeling

The backbone provides contextualized representations at each position, which PGuard can use either directly or through an additional module that aggregates historical evidence. These alternatives offer different trade-offs between detection accuracy and computational cost. Each variant produces a representation $z_t$ for the risk classifiers. Let $\tilde{x}_t = W_p x_t \in\mathbb{R}^{d_p}$ denote the fused feature projected into a compact space, where $W_p \in\mathbb{R}^{d_p \times d}$.

Stateless (last-token) variant. The stateless variant independently classifies each position using its projected feature:

Thus, all operations from the gate output to the logits are position-wise, with no recurrent or cached state maintained during training or inference. This point-wise computation makes the stateless variant the most computationally efficient option: prompt positions can be scored in parallel, while each decoding step incurs only a small additional classification cost.

Sequence variants: EMA-attention and GRU. The sequence variants maintain a per-request state that accumulates safety evidence throughout generation. We instantiate this paradigm with two designs that offer complementary trade-offs. The EMA-attention design maintains causal exponential moving averages at multiple decay rates for each captured layer, indexed by $i \in\{1,\ldots,m\}$:

where the fixed decay $\alpha_k$ is determined by the effective length $\lambda_k$ of each scale. Shorter scales emphasize recent features, whereas longer scales retain a more slowly decaying summary of the earlier context. The summaries are maintained separately for each captured layer and then fused using the current gate weights, $\overline{m}_t^{(k)} = \sum_i g_{t,i}m_{t,i}^{(k)}$. We construct $S_t = [\overline{m}_t^{(1)},\ldots,\overline{m}_t^{(K)},x_t]$ by concatenating the fused summaries with the current feature. Attention then uses the current feature as the query and $S_t$ as both keys and values:

allowing each position to be represented using both its current features and a multi-scale summary of preceding context.

The GRU design instead carries historical information in a gated recurrent unit with state $c_t \in\mathbb{R}^{d_c}$, where $d_c$ is the state dimension. Since all query positions are available during prefill, their projected features $\widetilde{x}_t$ are aggregated in parallel through attention pooling with a learned query vector. The resulting query summary initializes the recurrent state $c_0$, and the classification path applied to $c_0$ produces the query-side prediction. Starting from $c_0$, the recurrent state is updated at each response token as follows:

where $v_t \in(0,1)^{d_c}$ is the reset gate, $\widehat{c}_t \in\mathbb{R}^{d_c}$ is the candidate state, and the update gate $u_t \in(0,1)^{d_c}$ weights the candidate while $1-u_t$ weights the previous state. The skip term $W_s\widetilde{x}_t$ preserves the current position’s own evidence in the output.

Multi-Task Training

We adopt a shared risk taxonomy adapted from YuFeng-XGuard [35], comprising 9 risk dimensions with 29 fine-grained categories spanning general content safety, jailbreaks, multilingual risks, and cyber & CBRN domains. All training sources are mapped into this shared taxonomy (Appendix § 9), enabling a unified training process across different deployment scenarios. Each task group is equipped with a multi-class classifier over its corresponding categories. The backbone remains frozen throughout training, and only the PGuard module is optimized.

Sample-level labels provide insufficient localization of where risks emerge within a response. Although token-level annotation could provide finer-grained supervision, it is expensive, while safety judgments often require interpreting spans of text in context. We therefore adopt segment-level supervision, retaining each sample as a complete token sequence with multiple annotated segments. Each segment is annotated with labels for the relevant safety tasks, which supervise selected positions near its end. The classification loss is averaged over supervised positions within each segment and across available task annotations. For the GRU variant, the query-side prediction is additionally supervised using the label of the last query segment.

Streaming Detection and Decision

Per-token risk and verdicts. For each position and task $c$, the classifier outputs a probability distribution $P_c(\cdot\mid x_{1:t})$ over the task-specific categories $\mathcal{Y}_c \subseteq\mathcal{Y}$. A position is flagged for an unsafe category $y \in\mathcal{Y}_c$ when $P_c(y \mid x_{1:t}) \ge\tau_{c,y}$. Each trained PGuard model uses its own calibrated thresholds, configured separately for query and response stages and for text-only and image-text inputs (Appendix § 10). During prefill, the query-side verdict is derived from scores at selected positions or their aggregation, depending on the application’s policy, and is available before response generation begins. During decode, the hidden state at position $t$ is used to assess the content up to and including token $x_t$, while the same forward pass predicts the next token $x_{t+1}$. Verdict consumption is configurable at deployment: applications can selectively enable task groups and determine how flagged content is handled, including reporting, interrupting, or steering generation.

Sampled scoring. To further reduce serving overhead, PGuard supports sampled scoring, exploiting the observation that safety risks typically manifest across phrases or passages rather than in isolated tokens. Under this option, PGuard performs classification and reports risk scores only once every $p$ tokens, skipping the classification path at intermediate steps (§ 3.2). For sequence variants, however, the recurrent or attention-based state is still updated at every token, ensuring that each reported score incorporates evidence from the entire processed prefix. The early-interception analysis in § 4.4 uses the full per-token scoring trajectory rather than sampled scores.

Two-tier deployment. PGuard also supports a two-tier cascade in which the native variant performs token-level scoring on all traffic, while only flagged content is escalated to an external reasoning guardrail for fine-grained adjudication (§ 5.3).

System Implementation

Inference Engine Integration

We integrate PGuard into SGLang [76], with runtime support for feature capture, per-request state management, and streaming risk-score delivery. The integration supports the stateless, EMA-attention, and GRU variants through a shared engine interface. Figure 2 illustrates the integration.

At each selected layer, the runtime captures the same residual-stream representation used during training (§ 2.2). For backbones that maintain hidden-state and residual tensors separately, the runtime reconstructs this representation by summing the two tensors. This operation is performed solely for feature extraction and does not alter the backbone’s forward computation. The resulting risk scores are exposed as response metadata, leaving the original token logits and sampling path unchanged.

For sequence-based variants, the runtime maintains persistent state tensors for each active request in GPU memory. Each request is assigned a dedicated slot throughout its lifetime, enabling state retrieval independently of changes in batch composition introduced by the scheduler. During both chunked prefill and auto-regressive decoding, the runtime retrieves the state associated with the request, updates it with newly processed features, and writes the updated state back to the corresponding slot. For the GRU variant, prefill does not process query tokens recurrently. Instead, it projects all query features in parallel and aggregates them into the initial state $c_0$ using learned-query attention pooling. The runtime stores $c_0$ as the per-request state. During decode, each response token updates this state incrementally.

Runtime Support for PGuard Variants. All variants share the same engine interface but differ in state management. The stateless variant requires no persistent state. The EMA-attention variant maintains a set of multi-scale states for each request and captured layer. During chunked prefill, each chunk consumes the final state from the preceding chunk and produces an updated state for the next chunk. During decode, these states are updated incrementally at each token.

The GRU variant maintains a single fixed-size state per request. During prefill, learned-query attention pooling computes the initial state from the query features (§ 2.4), which is then stored in the request’s assigned slot. During decode, each response token updates this state before optional scoring.

Consequently, tokens for which no risk score is reported are still incorporated into the recurrent state, preserving the complete response history.

PGuard also supports pipeline and tensor parallelism. Under pipeline parallelism, the Qwen3.5 integration forwards captured features in a fixed layer order and executes the guard at the final stage. Under tensor parallelism, each rank independently computes scores for a distinct subset of sampled prefill positions. The ranks then aggregate their local maximum scores using an all-reduce operation.

Algorithm–System Co-Optimization

Directly integrating a native guardrail into the backbone forward pass can increase serving latency, even when the guard head is lightweight. The overhead arises not only from risk-score computation but also from repeated kernel launches, sequential state updates, and host–device synchronization.

Stage-level profiling reveals two primary sources of overhead. The first is redundant work. Under PGuard’s sampled reporting policy, risk scores are consumed only at selected positions, yet naively executing the guard at every token incurs unnecessary computation. The second is inefficient execution. Sequential state updates constrain prefill parallelism, small operators trigger repeated kernel launches, and host–device synchronization can stall the submission of subsequent backbone operations.

We address these bottlenecks through coordinated algorithmic and runtime optimizations. Specifically, we decouple state updates from risk scoring by computing risk scores only at positions selected for reporting, parallelize guard-state computation during prefill, move synchronization off the serving critical path, and fuse compatible operations to reduce kernel-launch overhead.

Selective Guard Scoring. Under a sampled reporting policy, applications consume a risk score every $p$ tokens. Computing scores at intermediate positions therefore incurs unnecessary overhead. We eliminate this redundant computation by decoupling state updates from risk scoring. For the EMA-attention and GRU variants, every token continues to update the corresponding state to preserve the sequence context, whereas the scoring path is executed only at positions selected for reporting.

SGLang captures backbone and guard operations into a single CUDA graph. As a result, each graph replay executes the entire scoring path, even at positions where the resulting scores are not reported. To avoid this, we split the decode graph into two segments. The first segment covers the backbone forward pass and the sequence-state update, and is replayed at every step. The second segment covers the guard scoring path, and is replayed only at the sampled steps (Figure 3). With a reporting interval of $p$ tokens, this removes approximately $1 - 1/p$ of the scoring executions.

Parallel Prefill Execution. Prefill is the primary source of PGuard’s overhead in our workloads. While the backbone processes prompt tokens in parallel, a straightforward EMA implementation updates the guard state sequentially, one token at a time. These sequential updates limit parallelism and incur repeated kernel-launch overhead. We exploit the linear structure of the EMA update to compute states within each chunk in parallel. For a fixed decay rate $\alpha_k$, the state at position $t$ can be expressed in terms of the state at an earlier position $s$ as follows.

We partition the prompt into fixed-size chunks and represent the weighted sums using a lower-triangular decay matrix as shown in Figure 4(a). Multiplying its selected rows by the token features computes the corresponding EMA summaries in parallel. Importantly, every prompt token contributes to these summaries, including tokens at positions where no risk score is reported. We compute the states at sampled positions and the final prompt token for scoring, and retain each chunk’s final state to initialize the next chunk.

Tensor parallelism provides an additional opportunity to distribute prefill scoring as illustrated in Figure 4(b). When each rank independently evaluates the guard on replicated features, the same scores are redundantly computed across ranks. Instead, we evenly partition the sampled positions across the $N$ ranks, with each rank computing a local maximum score for every risk category over its assigned positions. A single element-wise maximum all-reduce then aggregates these local results into the per-request score vector. Only the score vectors are exchanged, avoiding the transfer of hidden states across ranks. This partitioning reduces the scoring workload on each rank by approximately a factor of $N$.

Non-Blocking Guard Execution. Feature aggregation, context modeling, and score delivery introduce additional data dependencies when a native guardrail is integrated into the serving runtime. Resolving these dependencies through host–device synchronization can delay the submission of subsequent backbone operations. We keep state management and score aggregation on the GPU and use asynchronous transfers to reduce synchronization on the serving critical path. During prefill, fixed-interval sampling decisions are staged in pinned host memory and transferred asynchronously, with the sampled-token mask constructed on the GPU. Per-request state lookup and reset also remain on the GPU. Together, these changes avoid unnecessary host–device synchronization during prefill.

During decode, we overlap risk-score delivery with ongoing engine execution and align result consumption with the engine’s existing synchronization schedule. Asynchronous staging in pinned host memory supports this overlap without introducing an additional synchronization barrier for score delivery.

Fused Guard Computation. The guard head consists of many small matrix operations, each requiring a separate GPU kernel launch and memory access. For small inputs, these overheads can dominate the actual computation and result in poor GPU utilization. To address this, we fuse compatible operations into larger matrix multiplications. In the multi-task classifier, all nine task heads share the same input features. We concatenate the weights of their first layers into a single matrix and arrange the weights of their output layers into a block-diagonal matrix. This enables all nine heads to be evaluated using only two matrix multiplications, rather than eighteen. The fused weights are constructed once during model loading, eliminating the need for weight assembly during serving. This design reduces kernel-launch overhead and improves GPU utilization.

Together, these optimizations align guard execution with application reporting requirements while preserving complete sequence-state updates. By exploiting parallelism, reducing synchronization, and consolidating small operations, they mitigate the computational and runtime overhead of native guardrail integration along the serving critical path.

Experiments

Experimental Setup

Benchmarks

We evaluate PGuard across four dimensions, following the evaluation settings of WildGuard [17] and Qwen3Guard [74].

• Safety classification. ToxicChat [36], XSTest [53], OpenAI Moderation [49], Aegis 2.0 [11], WildGuardMix [17], S-Eval [69], BeaverTails [22], Qwen3Guard test set [74], FineHarm [34], and SafeRLHF [7] (text). SPA-VL [72], UnsafeBench [50], MM-SafetyBench [39], VLGuard [78], HoliSafe [30], VLSBench [20], FigStep [12], and Beavertails-V [23] (image-text).

• Frontier risk classification. StrongREJECT [59], S-Eval jailbreak subset [69] (text), FigStep [12] and MM-SafetyBench [39] (image-text) for jailbreak detection; WMDP [32], Meta CyberSecEval 3 [62], and SOSBench [24] for cybersecurity and CBRN risk identification.

• Multilingual safety. PolyGuardPrompts [28] (17 languages), RTP-LX [8] (12 languages), and Lingua-SafetyBench [56] (10 languages).

• Over-refusal and real-world data. XSTest [53] and OR-Bench [5] assess over-refusal. Benign-traffic evaluation uses Alpaca [60] and WildChat [75] for text, and ShareGPT4V [2], LLaVA-Instruct [38], and MMStar [3] for image-text inputs.

Baselines

We compare PGuard with native and external guardrails. Native guardrails derive safety signals from the protected backbone; external guardrails run a separate safety model over the query and response.

• Native guardrails: GradSafe [66] (Llama-2-7B [61]), SIREN [26] (Llama-3.1-8B [15]), NExT-Guard [9] (Qwen3-8B), HiddenDetect [25] (LLaVA-1.6-7B [31]).

• External guardrails: SCM-7B [34] (Qwen2.5-7B [67]), Qwen3Guard-Stream-8B [74] (Qwen3-8B [68]), Qwen3Guard-Gen-8B [74], WildGuard-7B [17], PolyGuard-Qwen-7B [28], YuFeng-XGuard-8B [35], LlamaGuard 4 (12B) [45], LlavaGuard-7B [19].

Implementation Details

Sections 4.2–4.3 evaluate PGuard on Qwen3-8B and Qwen3.5-9B [51]. Section 4.6 extends the evaluation to Qwen3.5-35B-A3B (also referred to as Qwen3.5-Flash) and Qwen3.5-122B-A10B. We evaluate query and response classification against sample-level benchmark labels. For these classification evaluations, we use a scoring interval of $p = 10$ tokens. Query classification uses the score at the query boundary, and response classification uses the score from sampled token scores. Decision thresholds are calibrated on a held-out validation set.

We evaluate Qwen3Guard-Gen and Qwen3Guard-Stream in strict mode, treating the Controversial label as unsafe. Other baselines use their officially released weights and default settings where available. HiddenDetect and NExT-Guard additionally require calibration because official settings are unavailable.

Main Results

Text Safety Evaluation Table 1 and Table 2 show that PGuard achieves the highest average text-safety $F_1$ among the evaluated native guardrails. For queries, PGuard-ema (Qwen3.5-9B) and PGuard-gru (Qwen3.5-9B) both achieve 0.836, exceeding prior native baselines by 4.5–9.5 percentage points. This score is comparable to Qwen3Guard-Stream (0.834) and higher than Qwen3Guard-Gen (0.814). PGuard-ema (Qwen3-8B) achieves 0.827, falling 0.7 percentage points below Qwen3Guard-Stream but outperforming Qwen3Guard-Gen by 1.3 percentage points, both built on Qwen3-8B. For responses, PGuard-gru (Qwen3.5-9B) leads the PGuard variants at 0.860, below Qwen3Guard-Gen-8B (0.866) and YuFeng-XGuard-8B (0.881). On Qwen3-8B, PGuard-ema achieves 0.858, exceeding SCM-7B by 7.6 percentage points and exceeding Qwen3Guard-Stream-8B (0.830).

Model	ToxicChat	XSTest	OAI-Mod	Aegis2	WGMix	S-Eval	Avg
External guardrails
Qwen3Guard-Gen-8B	0.636	0.908	0.682	0.865	0.887	0.906	0.814
Qwen3Guard-Stream-8B	0.715	0.921	0.740	0.869	0.871	0.890	0.834
WildGuard-7B	0.651	0.932	0.724	0.831	0.869	0.854	0.810
PolyGuard-Qwen-7B	0.677	0.923	0.732	0.866	0.877	0.898	0.829
YuFeng-XGuard-8B	0.705	0.944	0.720	0.859	0.866	0.906	0.833
Native guardrails
GradSafe (Llama-2-7B)	0.733	0.897	0.698	0.803	0.761	0.686	0.763
SIREN (Llama-3.1-8B)	0.610	0.771	0.844	0.842	0.822	0.558	0.741
NExT-Guard (Qwen3-8B)	0.750	0.844	0.730	0.780	0.782	0.858	0.791
PGuard-lasttok (Qwen3-8B)	0.737	0.886	0.747	0.837	0.846	0.889	0.824
PGuard-gru (Qwen3-8B)	0.736	0.869	0.782	0.838	0.860	0.894	0.830
PGuard-ema (Qwen3-8B)	0.697	0.928	0.755	0.853	0.853	0.876	0.827
PGuard-lasttok (Qwen3.5-9B)	0.692	0.899	0.767	0.846	0.846	0.879	0.822
PGuard-gru (Qwen3.5-9B)	0.745	0.887	0.782	0.848	0.852	0.903	0.836
PGuard-ema (Qwen3.5-9B)	0.716	0.907	0.781	0.853	0.858	0.899	0.836

Model	BeaverTails	Q3G-Test	FineHarm	SafeRLHF	Aegis2	WGMix	XSTest	S-Eval	Avg
External guardrails
Qwen3Guard-Gen-8B	0.865	0.924	0.972	0.909	0.863	0.781	0.827	0.785	0.866
Qwen3Guard-Stream-8B	0.851	0.803	0.960	0.930	0.827	0.746	0.890	0.633	0.830
WildGuard-7B	0.841	0.868	0.991	0.925	0.838	0.721	0.912	0.429	0.816
PolyGuard-Qwen-7B	0.802	0.911	0.983	0.894	0.824	0.764	0.349	0.622	0.769
YuFeng-XGuard-8B	0.837	0.907	0.971	0.950	0.802	0.782	0.885	0.910	0.881
SCM-7B (Qwen2.5-7B)	0.796	0.868	0.973	0.901	0.777	0.703	0.590	0.646	0.782
Native guardrails
SIREN (Llama-3.1-8B)	0.842	0.902	0.869	0.917	0.822	0.589	0.591	0.365	0.737
NExT-Guard (Qwen3-8B)	0.766	0.849	0.745	0.800	0.779	0.404	0.179	0.546	0.634
PGuard-lasttok (Qwen3-8B)	0.760	0.827	0.923	0.862	0.804	0.695	0.824	0.826	0.815
PGuard-gru (Qwen3-8B)	0.804	0.882	0.941	0.871	0.800	0.751	0.845	0.867	0.845
PGuard-ema (Qwen3-8B)	0.834	0.908	0.943	0.927	0.809	0.777	0.796	0.870	0.858
PGuard-lasttok (Qwen3.5-9B)	0.751	0.825	0.914	0.845	0.755	0.681	0.785	0.803	0.795
PGuard-gru (Qwen3.5-9B)	0.835	0.906	0.948	0.892	0.832	0.734	0.855	0.881	0.860
PGuard-ema (Qwen3.5-9B)	0.780	0.867	0.920	0.866	0.788	0.744	0.853	0.849	0.833

Sequence variants consistently outperform the stateless variant in average $F_1$ on both query and response safety across both backbones, with gains ranging from 0.3 to 6.5 percentage points.

Multimodal Safety Evaluation The same training recipe applies to multimodal backbones, which fuse visual and textual information before PGuard extracts their representations. In Table 3, PGuard-ema

Model	SPA-VL	UnsafeB	MM-SB	VLGuard	HoliSafe	VLSB	FigStep	BeaverT-V	Avg	SPA-VL	BeaverT-V	Avg
External guardrails
Llama Guard 4 (12B)	0.535	0.326	0.608	0.650	0.337	0.129	0.779	0.426	0.474	0.495	0.515	0.505
LlavaGuard-7B	0.256	0.582	0.269	0.557	0.424	0.641	0.224	0.535	0.436	–	–	–
Native guardrails
HiddenDetect (Llava-1.6-7B)	0.658	0.234	0.779	0.798	0.649	0.215	0.927	0.845	0.638	–	–	–
PGuard-lasttok (Qwen3.5-9B)	0.694	0.384	0.649	0.647	0.410	0.586	0.882	0.828	0.635	0.595	0.692	0.644
PGuard-gru (Qwen3.5-9B)	0.814	0.448	0.706	0.837	0.732	0.678	0.840	0.822	0.735	0.800	0.796	0.798
PGuard-ema (Qwen3.5-9B)	0.797	0.516	0.682	0.800	0.711	0.763	0.857	0.798	0.741	0.757	0.788	0.772

($\mathrm{Qwen3.5}$-9B) achieves the highest average query $F_1$ of 0.741, compared with 0.638 for HiddenDetect and 0.474 for Llama Guard 4. HiddenDetect remains stronger on MM-SafetyBench and FigStep. For responses, PGuard-gru leads the evaluated methods with an average $F_1$ of 0.798.

Adversarial Jailbreak Detection. Table 4 evaluates detection on the static jailbreak benchmarks. PGuard-ema (Qwen3-8B) achieves an average $F_1$ of 0.946, ranking first among native guardrails and second overall. It trails YuFeng-XGuard-8B (0.972) and exceeds Qwen3Guard-Gen (0.928). On S-Eval-JB, its $F_1$ of 0.896 exceeds NExT-Guard on the same backbone by 12.5 percentage points, while YuFeng-XGuard-8B leads at 0.944. These results support the effectiveness of internal representations for detecting the evaluated jailbreak prompts.

Model	StrongREJECT	S-Eval-JB	Avg
External guardrails
Qwen3Guard-Gen-8B	0.998	0.858	0.928
Qwen3Guard-Stream-8B	0.997	0.804	0.901
WildGuard-7B	0.995	0.785	0.890
PolyGuard-Qwen-7B	0.995	0.857	0.926
YuFeng-XGuard-8B	1.000	0.944	0.972
Native guardrails
GradSafe (Llama-2-7B)	0.995	0.383	0.689
SIREN (Llama-3.1-8B)	0.970	0.396	0.683
NExT-Guard (Qwen3-8B)	0.990	0.771	0.881
PGuard-lasttok (Qwen3-8B)	0.990	0.821	0.906
PGuard-gru (Qwen3-8B)	0.984	0.876	0.930
PGuard-ema (Qwen3-8B)	0.995	0.896	0.946
PGuard-lasttok (Qwen3.5-9B)	0.985	0.849	0.917
PGuard-gru (Qwen3.5-9B)	0.989	0.884	0.937
PGuard-ema (Qwen3.5-9B)	0.998	0.879	0.939

Cyber & CBRN Risk Identification. In Table 5, PGuard-lasttok (Qwen3.5-9B) achieves the highest average $F_1$ of 0.833, followed by the GRU (0.789) and EMA-attention (0.773) variants on that backbone. On Qwen3-8B, PGuard-ema reaches 0.747, exceeding the evaluated native baselines and both Qwen3Guard variants. Performance varies across risk categories: SIREN leads on WMD biology and chemistry, whereas PGuard leads on several CyberSecEval3 and SOSBench subsets.

Model	WMDP bio	WMDP chem	WMDP cyber	CyberSecEval3 mitre	CyberSecEval3 p.inj	CyberSecEval3 interp.	SOSBench bio	SOSBench chem	SOSBench med	SOSBench pharm	SOSBench phys	SOSBench psy	Avg
External guardrails
Qwen3Guard-Gen-8B	0.085	0.463	0.156	0.876	0.615	0.962	0.951	0.959	0.935	0.951	0.900	1.000	0.738
Qwen3Guard-Stream-8B	0.036	0.241	0.070	0.773	0.421	0.956	0.884	0.859	0.777	0.807	0.755	0.989	0.631
WildGuard-7B	0.111	0.341	0.207	0.860	0.475	0.956	0.929	0.839	0.908	0.792	0.862	0.983	0.689
PolyGuard-Qwen-7B	0.124	0.623	0.240	0.888	0.615	0.967	0.951	0.953	0.897	0.807	0.958	0.989	0.751
YuFeng-XGuard-8B	0.210	0.730	0.232	0.844	0.615	0.962	0.935	0.902	0.880	0.921	0.835	0.989	0.755
Native guardrails
GradSafe (Llama-2-7B)	0.015	0.027	0.056	0.575	0.333	0.862	0.767	0.344	0.619	0.222	0.644	0.818	0.440
SIREN (Llama-3.1-8B)	0.626	0.793	0.301	0.751	0.364	0.889	0.872	0.745	0.856	0.927	0.814	0.936	0.740
NExT-Guard (Qwen3-8B)	0.156	0.302	0.134	0.811	0.125	0.950	0.775	0.688	0.646	0.413	0.555	0.803	0.530
PGuard-lasttok (Qwen3-8B)	0.071	0.341	0.312	0.950	0.448	0.956	0.842	0.948	0.925	0.871	0.924	0.904	0.708
PGuard-gru (Qwen3-8B)	0.262	0.569	0.316	0.908	0.421	0.962	0.848	0.812	0.756	0.785	0.875	0.936	0.704
PGuard-ema (Qwen3-8B)	0.211	0.479	0.331	0.922	0.548	0.979	0.896	0.902	0.897	0.909	0.894	0.995	0.747
PGuard-lasttok (Qwen3.5-9B)	0.546	0.752	0.583	0.942	0.696	0.967	0.907	0.890	0.874	0.979	0.969	0.890	0.833
PGuard-gru (Qwen3.5-9B)	0.497	0.741	0.459	0.851	0.571	0.962	0.829	0.920	0.818	0.984	0.906	0.929	0.789
PGuard-ema (Qwen3.5-9B)	0.245	0.610	0.546	0.952	0.657	0.979	0.878	0.819	0.831	0.968	0.842	0.954	0.773

Multilingual Safety Evaluation. Table 6 shows competitive detection across the evaluated languages, with the strongest method varying by benchmark and stage. On PolyGuardPrompts, PolyGuard-Qwen-7B leads query classification at 0.869, and YuFeng-XGuard-8B leads response classification at 0.793. Among native guardrails, PGuard-ema leads both stages, reaching 0.835 for queries on Qwen3.5-9B and 0.766 for responses on Qwen3-8B. On RTP-LX queries, PGuard-ema (Qwen3.5-9B) is the strongest native entry at 0.847, below Qwen3Guard-Gen-8B (0.877). On RTP-LX responses, PGuard-lasttok (Qwen3-8B) achieves the highest $F_1$ of 0.898. For multimodal queries on Lingua-SafetyBench, PGuard-gru (Qwen3.5-9B) leads at 0.788, compared with 0.761 for HiddenDetect and 0.509 for Llama Guard 4.

Model	Stage	PolyGuardPrompts Q	PolyGuardPrompts R	RTP-LX Q	RTP-LX R	Lingua Q
External guardrails
Qwen3Guard-Gen-8B	Q+R	0.865	0.781	0.877	0.793	–
Qwen3Guard-Stream-8B	Q+R	0.865	0.770	0.852	0.855	–
WildGuard-7B	Q+R	0.726	0.649	0.589	0.821	–
PolyGuard-Qwen-7B	Q+R	0.869	0.749	0.833	0.769	–
YuFeng-XGuard-8B	Q+R	0.857	0.793	0.859	0.889	–
SCM-7B (Qwen2.5-7B)	R	–	0.675	–	0.718	–
Llama Guard 4 (12B)	Q+R	–	–	–	–	0.509
LlavaGuard-7B	Q	–	–	–	–	0.273
Native guardrails
GradSafe (Llama2-7B)	pre	0.305	–	0.351	–	–
SIREN (Llama-3.1-8B)	p+d	0.360	0.346	0.452	0.502	–
NExT-Guard (Qwen3-8B)	p+d	0.753	0.468	0.359	0.723	–
HiddenDetect (Llama-1.6-7B)	pre	–	–	–	–	0.761
PGuard-lasttok (Qwen3-8B)	p+d	0.816	0.685	0.805	0.898	–
PGuard-gru (Qwen3-8B)	p+d	0.830	0.748	0.806	0.851	–
PGuard-ema (Qwen3-8B)	p+d	0.835	0.766	0.835	0.834	–
PGuard-lasttok (Qwen3.5-9B)	p+d	0.825	0.685	0.812	0.860	0.627
PGuard-gru (Qwen3.5-9B)	p+d	0.834	0.743	0.835	0.888	0.788
PGuard-ema (Qwen3.5-9B)	p+d	0.834	0.742	0.847	0.844	0.753

Low Over-Refusal on Benign Traffic

Over-Refusal. Table 7 reports over-refusal rates on XSTest and OR-Bench, measuring the cost of false blocks on benign content. PGuard-ema (Qwen3-8B) has an average query-side rate of 6.12%, within the range of the external baselines. Other PGuard variants reach 13.63–15.35%, below SIREN (22.41%) and NExT-Guard (17.38%). For responses, PGuard-ema (Qwen3.5-9B) achieves the lowest native average of 4.20%. Both sequence variants reduce response-side over-refusal relative to the stateless variant on each backbone.

Model	Stage	Query XSTest	Query OR-Bench	Query Avg	Response XSTest	Response OR-Bench	Response Avg
External guardrails
Qwen3Guard-Gen-8B	Q+R	12.88	7.71	10.29	1.81	1.44	1.63
Qwen3Guard-Stream-8B	Q+R	10.46	4.50	7.48	0.76	0.13	0.45
WildGuard-7B	Q+R	1.07	8.64	4.86	0.38	0.24	0.31
PolyGuard-Qwen-7B	Q+R	3.82	8.72	6.27	21.75	1.01	11.38
YuFeng-XGuard-8B	Q+R	7.65	0.30	3.97	0.71	1.25	0.98
SCM-7B (Qwen2.5-7B)	R	–	–	–	6.33	6.14	6.24
Native guardrails
GradSafe (Llama2-7B)	pre	14.89	5.93	10.41	–	–	–
SIREN (Llama-3.1-8B)	p+d	25.75	19.07	22.41	7.66	3.11	5.39
NExT-Guard (Qwen3-8B)	p+d	18.51	16.24	17.38	58.40	29.87	44.13
PGuard-lasttok (Qwen3-8B)	p+d	14.55	16.15	15.35	1.86	12.33	7.10
PGuard-gru (Qwen3-8B)	p+d	17.77	11.83	14.80	1.71	7.28	4.50
PGuard-ema (Qwen3-8B)	p+d	10.06	2.18	6.12	2.43	6.52	4.48
PGuard-lasttok (Qwen3.5-9B)	p+d	14.49	17.69	16.09	2.19	10.57	6.38
PGuard-gru (Qwen3.5-9B)	p+d	15.96	11.46	13.71	1.67	7.18	4.43
PGuard-ema (Qwen3.5-9B)	p+d	14.49	12.77	13.63	1.38	7.02	4.20

Real-World Benign Traffic. Table 8 and Table 9 report interception rates on benign text and multimodal traffic. On text, PGuard variants intercept an average of 3.69–5.05% of queries and 2.32–2.80% of responses. Query-side rates are lower than those of both Qwen3Guard variants, whereas response-side rates are broadly comparable to external baselines. GRU and EMA show comparable interception rates across backbones. On multimodal traffic, average interception rates range from 0.11% to 0.60% for queries and from 0.07% to 0.18% for responses.

Model	Stage	Query (Q) Alpaca	Query (Q) WildChat	Query (Q) Avg	Response (R) Alpaca	Response (R) WildChat	Response (R) Avg
External guardrails
Qwen3Guard-Gen-8B	Q+R	0.43	13.02	6.73	0.46	3.94	2.20
Qwen3Guard-Stream-8B	Q+R	0.39	9.95	5.17	0.12	1.94	1.03
WildGuard-7B	Q+R	0.62	5.47	3.05	0.35	1.43	0.89
PolyGuard-Qwen-7B	Q+R	0.55	14.98	7.77	1.15	10.42	5.79
YuFeng-XGuard-8B	Q+R	0.24	10.32	5.28	0.15	4.16	2.16
SCM-7B (Qwen2.5-7B)	R	–	–	–	1.48	9.06	5.27
Native guardrails
GradSafe (Llama2-7B)	pre	0.14	1.45	0.79	–	–	–
SIREN (Llama-3.1-8B)	p+d	3.59	6.26	4.93	1.67	3.54	2.61
NExT-Guard (Qwen3-8B)	p+d	0.16	5.42	2.79	0.43	8.10	4.27
PGuard-lasttok (Qwen3-8B)	p+d	0.30	8.63	4.46	0.17	5.44	2.80
PGuard-gru (Qwen3-8B)	p+d	0.31	7.07	3.69	0.15	4.62	2.39
PGuard-ema (Qwen3-8B)	p+d	0.24	7.43	3.83	0.21	4.43	2.32
PGuard-lasttok (Qwen3.5-9B)	p+d	0.56	9.55	5.05	0.15	5.44	2.79
PGuard-gru (Qwen3.5-9B)	p+d	0.54	7.04	3.79	0.24	4.69	2.47
PGuard-ema (Qwen3.5-9B)	p+d	0.37	7.14	3.76	0.14	5.16	2.65

Model	Stage	Query (Q) ShareGPT4V	Query (Q) LLaVA-Instr.	Query (Q) MMStar	Query (Q) Avg	Response (R) ShareGPT4V	Response (R) LLaVA-Instr.	Response (R) MMStar	Response (R) Avg
External guardrails
Llama Guard 4 (12B)	Q+R	0.74	0.01	0.67	0.47	0.67	0.04	1.33	0.68
LlavaGuard-7B	Q	1.19	1.11	0.67	0.99	–	–	–	–
Native guardrails
HiddenDetect (LV-1.6-7B)	pre	1.20	0.21	0.00	0.47	–	–	–	–
PGuard-lasttok (Qwen3.5-9B)	p+d	0.60	0.21	1.00	0.60	0.19	0.04	0.00	0.07
PGuard-gru (Qwen3.5-9B)	p+d	0.38	0.08	0.00	0.15	0.18	0.04	0.00	0.07
PGuard-ema (Qwen3.5-9B)	p+d	0.26	0.06	0.00	0.11	0.19	0.03	0.33	0.18

Early-Interception Analysis

The preceding evaluations measure sample-level classification, allowing comparison with detectors that assess completed responses. We next examine when PGuard first detects an unsafe response during generation.

For each detected unsafe response in the response-side benchmarks, we replay the sequence variant’s token-level risk trajectory. The interception point is the first position where the score crosses the threshold, expressed as a fraction of the full response length. We aggregate these fractions across benchmarks. A detector applied only after completion has an interception point of 1.0.

Figure 5a shows that 86.3% of detected unsafe responses are flagged within the first 10% of generated tokens, and 92.3% within the first 20%. These early flags allow generation to stop well before completion. With $p = 10$, the wait from any token to the next scoring point is at most nine decode steps. This brief scheduling delay is acceptable for monitoring risk that develops over phrases and passages, while avoiding unnecessary scoring at every token.

Prefix stability. We evaluate prediction stability on S-Eval by measuring, among samples flagged by the guardrail, the proportion containing at least $k$ consecutive tokens with risk scores above 0.5, indicating sustained predictions of unsafe content. We consider $k \in\{5, 10, 20, 50\}$. As shown in Figure 5b, the ema-based variants maintain high stability as the required consecutive span increases: over 95% of threshold-hit samples sustain above-threshold scores for at least 20 tokens, and approximately 93% do so for 50 tokens. In contrast, the stateless variants show a more pronounced decline, with only 60–69% of samples sustaining the threshold for 50 tokens. The results suggest that temporal modeling produces more persistent risk predictions than relying solely on the last-token representation, particularly over longer spans.

Signal Analysis

Layer-gate profiles. Figure 6 shows how learned layer-gate weights evolve during training. The weights concentrate on different layer subsets across backbones, supporting backbone-specific selection of capture layers.

Modeling variants on the spectrum. Table 10 summarizes performance across backbones and representation-modeling variants. The stateless variant remains competitive on several tasks, including cyber and CBRN risk classification, whereas sequence modeling generally improves response detection. On Qwen3.5-9B, GRU and EMA-attention achieve text-response $F_1$ scores of 0.860 and 0.833, respectively, compared with 0.795 for the stateless variant. For multimodal responses, the corresponding scores are 0.798, 0.772, and 0.644. GRU performs better on these response tasks, but requires a recurrent state update at every decode step. EMA-attention supports parallel prefill computation and remains the default sequence variant (§ 3.2). The choice therefore depends on the task and serving constraints.

Backbone	General safety				Jailbreak	Cyber & CBRN	Multilingual
	Text Q	Text R	Img Q	Img R			Text Q	Text R	Img Q
lasttok
Qwen3-8B	0.824	0.815	–	–	0.906	0.708	0.811	0.792	–
Qwen3.5-9B	0.822	0.795	0.635	0.644	0.917	0.833	0.819	0.773	0.627
Qwen3.5-35B-A3B	0.822	0.812	0.734	0.658	0.941	0.816	0.754	0.791	0.756
Qwen3.5-122B-A10B	0.838	0.819	0.798	0.730	0.950	0.765	0.839	0.803	0.863
gru
Qwen3-8B	0.830	0.845	–	–	0.930	0.704	0.818	0.790	–
Qwen3.5-9B	0.836	0.860	0.735	0.798	0.937	0.789	0.835	0.816	0.788
Qwen3.5-35B-A3B	0.846	0.863	0.770	0.811	0.939	0.752	0.857	0.806	0.782
Qwen3.5-122B-A10B	0.851	0.857	0.745	0.805	0.944	0.761	0.848	0.814	0.774
ema
Qwen3-8B	0.827	0.858	–	–	0.946	0.747	0.835	0.800	–
Qwen3.5-9B	0.836	0.833	0.741	0.772	0.939	0.773	0.841	0.793	0.753
Qwen3.5-35B-A3B	0.832	0.853	0.796	0.795	0.957	0.807	0.846	0.814	0.875
Qwen3.5-122B-A10B	0.835	0.839	0.802	0.752	0.939	0.812	0.847	0.818	0.848

Generalization Across Backbones

Table 10 covers Qwen backbones from 8B to 122B total parameters, including dense and MoE architectures and text-only and multimodal models. The same training recipe applies across these backbones, with layer selection and guard training performed separately for each backbone. These results highlight the applicability of the training recipe across model scales, with the benefits of larger backbones varying by task.

Serving Efficiency

Setup. We measure the serving cost of PGuard with SGLang on L20Z GPUs. Appendix § 12 lists the serving configuration of each backbone in Table 21. Each run uses a concurrency of 8 and 256 requests per scenario. The input length is 2048, 4096, or 8192 tokens, and the output length is 512 tokens. Each configuration is paired with a run of the bare backbone on the same node, and the two runs alternate in order across 4 rounds. We report the relative increase of time to first token (TTFT) and time per output token (TPOT), together with the relative loss of output throughput. The bare backbone and PGuard use the same memory fraction for static allocation.

Backbone	TP	SGLang	Capture indices	Memory fraction
Qwen3.5-9B	1	0.5.11	20, 24, 28	0.9 / 0.8 / 0.8
Qwen3.5-35B-A3B	2	0.5.11	20, 24, 28	0.9 / 0.8 / 0.8
Qwen3.5-122B-A10B	4	0.5.11	EMA-attention: 32, 36, 40; Stateless/GRU: 36, 40, 44	0.9 / 0.8 / 0.8

End-to-end overhead. Table 11 reports TTFT, TPOT, and output throughput changes with each backbone’s best combination of optimizations across three backbones and input lengths of 2048–8192 tokens. Among the available measurements, the largest increases are 3.56% for TTFT and 3.07% for TPOT. For the stateless variant, the maxima are 1.54% and 1.78%, respectively; for EMA-attention, they are 3.56% and 2.40%. The GRU variant has maxima of 1.38% and 3.07%. Output throughput drops by at most 1.73% for the stateless variant and 2.26% for EMA-attention. Most entries stay below 2.5%, and all entries stay below 3.6%. This overhead is small and acceptable for large-scale production serving.

Table 11: Relative changes in TTFT, TPOT, and output throughput (Thpt.) of PGuard compared with the bare backbone, with an output length of 512 tokens. Each backbone uses its best combination of optimizations. For TTFT and TPOT, positive values indicate increased latency. For throughput, positive values indicate reduced output throughput. Lower is better for all metrics.

Backbone	Input	Stateless TTFT (%)	Stateless TPOT (%)	Stateless Thpt. (%)	EMA-attention TTFT (%)	EMA-attention TPOT (%)	EMA-attention Thpt. (%)	GRU TTFT (%)	GRU TPOT (%)	GRU Thpt. (%)
Qwen3.5-9B (TP=1)	2048	+1.43	+1.78	+1.73	+3.56	+1.69	+1.76	+0.73	+1.64	+1.55
Qwen3.5-9B (TP=1)	4096	+1.54	+1.47	+1.45	+3.08	+1.72	+1.81	+0.88	+1.59	+1.48
Qwen3.5-9B (TP=1)	8192	+0.04	+1.29	+1.07	+2.48	+1.80	+1.87	+0.69	+1.43	+1.28
Qwen3.5-35B-A3B (TP=2)	2048	-0.34	+0.95	+0.82	+2.08	+1.19	+1.19	+1.38	+3.07	+2.89
Qwen3.5-35B-A3B (TP=2)	4096	+0.25	-0.93	-0.88	+2.15	+1.42	+1.43	+0.97	+2.18	+2.02
Qwen3.5-35B-A3B (TP=2)	8192	+0.17	+0.17	+0.11	+1.91	+1.91	+1.86	+0.77	+2.14	+1.91
Qwen3.5-122B-A10B (TP=4)	2048	+0.22	+1.23	+0.82	+0.51	+2.37	+2.21	+0.73	+1.32	+1.41
Qwen3.5-122B-A10B (TP=4)	4096	+0.07	+1.30	+1.18	+1.53	+2.40	+2.26	+0.61	+1.78	+1.65
Qwen3.5-122B-A10B (TP=4)	8192	+0.29	+0.99	+0.89	+1.20	+2.31	+2.11	+0.63	+1.79	+1.51

Ablation. Tables 12 and 13 add the optimizations one at a time in the listed order. The full ladders reduce output throughput loss from 2.42–3.67% to 0.16–1.21% for the stateless variant and from 4.55–5.28% to 1.53–1.99% for EMA-attention. Selective decode scoring improves throughput across all three backbones, lowering throughput loss by 0.41–2.43 percentage points for the stateless variant and 0.92–2.19 percentage points for EMA-attention. The gains of the fusion and asynchronous copy steps depend on the backbone.

GRU end-to-end ablation. Table 14 reports the end-to-end output throughput loss of the GRU variant relative to the bare backbone, with input/output lengths of 4096/512, concurrency 8, 256 requests, and four paired rounds. Lower values indicate less serving overhead. Async risk copy provides the largest reduction among adjacent reported configurations, bringing throughput loss to 1.16–1.75% across the three backbones. With selective decode scoring, the loss is further reduced to 0.90–1.09% across the three backbones.

Configuration	Qwen3.5-9B	Qwen3.5-35B-A3B	Qwen3.5-122B-A10B
GRU baseline (pool prefill)	3.30	4.87	3.48
+ Vectorized prefill	3.28	4.41	3.24
+ Layer aggregation fusion	3.20	4.02	3.26
+ GRU update fusion	2.91	4.17	3.21
+ Adapter fusion	2.94	4.06	3.21
+ Async risk copy	1.16	1.75	1.29
+ Selective decode scoring	0.90	1.09	0.92

Memory overhead. PGuard maintains a small memory footprint alongside the backbone. Table 15 reports guard weights of 0.03–0.41 GiB per GPU, together with per-request state and peak workspace. Peak workspace remains below 0.5 GiB per GPU for the stateless variant and below 3.1 GiB per GPU for EMA-attention. The small memory overhead supports large-scale deployment of PGuard.

Backbone	Variant	Guard weights (GiB/GPU)	Per-request state (MiB/GPU)	Peak workspace (GiB/GPU)
Qwen3.5-9B	Stateless	0.06	–	0.19
Qwen3.5-9B	EMA-attention	0.41	0.094	2.69
Qwen3.5-9B	GRU	0.03	0.0016	0.69
Qwen3.5-35B-A3B	Stateless	0.04	–	0.09
Qwen3.5-35B-A3B	EMA-attention	0.10	0.047	1.35
Qwen3.5-35B-A3B	GRU	0.03	0.0016	0.34
Qwen3.5-122B-A10B	Stateless	0.04	–	0.14
Qwen3.5-122B-A10B	EMA-attention	0.23	0.070	2.34
Qwen3.5-122B-A10B	GRU	0.04	0.0017	0.60

Deployment on the Cost–Accuracy Spectrum

Cost–accuracy overview. Figure 7 summarizes the trade-off between detection accuracy and serving cost. GradSafe is omitted because it requires a backward pass to obtain gradients, which lies outside the serving forward path. Accuracy is measured on the benchmark suites above. Cost follows § 3.2, with each configuration served by SGLang on the same GPU under saturation and normalized to the bare backbone. Serving a second large guard model raises normalized cost to as much as $2.7\times$. The plotted prior native guardrails achieve 0.64–0.74 macro $F_1$ with 4–18% overhead. PGuard combines competitive accuracy with substantially lower serving overhead. On text, PGuard-GRU reaches 0.848 macro

Table 12: Cumulative ablation of the stateless variant. Each cell reports the output throughput loss (%) relative to the bare backbone.

Configuration	Qwen3.5-9B	Qwen3.5-35B-A3B	Qwen3.5-122B-A10B
Unoptimized	2.42	3.67	2.42
+ Merge fusion	2.39	3.69	2.49
+ Adapter fusion	2.48	3.54	2.29
+ Classifier fusion	1.50	2.10	1.17
+ Async risk copy	1.62	2.59	1.50
+ Selective decode scoring	1.21	0.16	0.76

$F_{1}$, exceeding Qwen3Guard-Gen’s 0.840, at 1.036× normalized cost. On multimodal inputs, PGuard-ema achieves 0.797 macro $F_{1}$ at 1.019× cost, compared with 0.638 for the strongest prior native entry. These operating points pair strong detection quality with only 3.6% and 1.9% additional serving cost for text and multimodal inputs, respectively.

Two-tier deployment. For deployments with a larger accuracy budget, the native variant composes with an external adjudicator instead of replacing it. Table 16 evaluates the two-tier deployment on S-Eval-Jailbreak, with Qwen3.5-Flash serving as both the PGuard backbone and the response generator. We regenerate the responses and cascade PGuard with an external reasoning-based guardrail. Jailbreak prompts are selected to increase the occurrence of risky responses, providing a stress test that approximates risky online traffic. It reaches the best overall score (96.72 average, versus 95.16 for PGuard alone), confirming that the native screen and the external adjudicator are complementary. Note that the gain comes entirely from precision, because recall is capped by the first tier. The adjudicator can only review traffic the native screen already flags, and this ceiling is intrinsic to any cascade. The economics of the pattern therefore hinge on the flagged fraction escalated to the second tier. On the ALPACA traffic set, the native guard flags 0.215% of responses, so only this fraction requires external review. This reduces second-tier calls by approximately 99.8% compared with reviewing every response. After review, the block rate falls to 0.056% (16 samples). The native guard therefore screens all traffic, while the external adjudicator reviews only flagged responses.

Method	Query	Response	Avg	Block Rate(%)
PGuard(Qwen3.5-Flash)	97.56 (95.24/100.00)	90.48 (95.00/86.36)	95.16 (95.16/95.16)	0.215
YuFeng-XGuard	95.24 (90.91/100.00)	88.37 (90.48/86.36)	92.91 (90.77/95.16)	0.144
PGuard+YuFeng-XGuard	98.77 (97.56/100.00)	92.68 (100.00/86.36)	96.72 (98.33/95.16)	0.056

Background and Related Work

External guardrails. External guardrails use a separate model to classify queries and responses. Llama Guard [45, 44] uses instruction tuning with a content taxonomy, with its fourth generation supporting multimodal input. Related approaches include ShieldGemma [13], Granite Guardian [21], NVIDIA’s Nemotron/Aegis safety models [47], and ShieldLM [73]. Qwen3Guard [74] offers generative and streaming variants. The streaming variant adds classification heads to a base LLM to produce token-level verdicts. Despite its token-level outputs, Qwen3Guard-Stream still requires a separate LLM, adding model memory and inference computation beyond those of the protected backbone. SCM [34] likewise runs a separate Qwen2.5-7B model with token-level and sequence-level classification heads to monitor streamed responses, adding its own model memory and inference computation. WildGuard [17] jointly classifies prompts, responses, and refusal behavior, while PolyGuard [28] extends safety classification to 17 languages. GuardReasoner [40, 41] and YuFeng-XGuard [35] generate explicit reasoning before a verdict, adding computation per request. For multimodal moderation, LlavaGuard [19] targets images, while SafeWatch [4] and OmniGuard [77] extend coverage to video and omni-modal inputs. PolicyShiftGuard [58] addresses cross-product policy adaptation, and JailGuard [70] detects attacks through blackbox input mutation. These approaches require serving an additional model. Response-level detectors also defer decisions until the response is complete.

Configuration	Qwen3.5-9B	Qwen3.5-35B-A3B	Qwen3.5-122B-A10B
Unoptimized	4.62	5.28	4.55
+ Scan chunk tuning	4.58	5.73	3.93
+ Rank-sharded scoring	–	4.66	3.65
+ QKV fusion	4.79	4.33	4.25
+ Classifier fusion	4.07	0.79	3.70
+ Decode TP fusion	–	1.68	2.52
+ Async risk copy	4.18	3.63	2.45
+ Selective decode scoring	1.99	1.98	1.53

Native guardrails: signal sources. Native guardrails instead derive safety signals from the protected model. GradSafe [66] computes gradients for a paired compliant response and scores prompts using cosine similarity over safety-critical parameter slices. HiddenDetect [25] identifies unsafe multimodal inputs through refusal-related hidden-state directions before generation; Activation-Cone [46] probes the first few generated tokens. SIREN [26] trains lightweight per-layer probes over safety neurons, whereas kNNGuard [1] uses a nearest-neighbor bank of last-token activations. TrajGuard [37] monitors Mahalanobis distances along hidden-state trajectories, and Oldfield’s truncated polynomial classifier [48] examines higher-order residual-stream readouts. NExT-Guard [9] selects safety-relevant sparse autoencoder features and scores them token by token. VLMGuard [10] trains an MLP using SVD-derived pseudo-labels, while ARGUS [42] combines per-layer logistic probes with activation-level intervention for multimodal inputs. These approaches differ in their signal sources, decision granularity, and serving support, motivating joint evaluation of detection quality and runtime cost.

Native guardrails: streaming and production. Streaming detection is the setting most closely related to PGuard. Segment-Level Coherence [18] pools windowed probe evidence for token-level decisions. Stop Early, Spend Less [57] integrates a probe into the vLLM decode loop for early termination. Kelp [33] models risk using a continuous-time latent state trained with response-level labels, while SafeSwitch [16] combines a probe with a refusal head. Production systems also combine lightweight screening with more expensive adjudication. Anthropic’s Constitutional Classifiers combine an input classifier with an output value head [54]. Its successor uses internal probes trained with sliding-window mean logit smoothing (SWiT) to flag traffic for a larger exchange classifier [6]. Google DeepMind’s Gemini probes [27] similarly use token-level internal classifiers and escalate less than 10% of traffic. High-Stakes Probes [43] studies cascades that route a fixed fraction of traffic from a lightweight probe to an expensive judge. Together, these approaches motivate evaluating native screening as part of a complete serving pipeline.

Intervention mechanisms. Other methods directly alter generation through representation reading and control [79], contrastive activation addition [52], or circuit breakers implemented with LoRA adapters [80].

Related approaches include learned null-space steering [55] and multimodal defenses acting on prompts or activations [63, 14, 64]. PGuard supplies token-level risk scores and leaves the response policy, such as reporting, interruption, or steering, to the deployment. These intervention mechanisms can therefore complement its detection interface.

Positioning of PGuard. PGuard brings together representation-based safety detection and serving-engine integration, evaluating detection quality alongside runtime overhead. Its stateless and sequence variants offer different accuracy–cost trade-offs within a shared SGLang integration (§ 3). The evaluation spans dense and MoE backbones, text-only and multimodal models, and parameter scales from 8B to 122B (§ 5.1). The planned release includes the guard models, evaluation pipeline, and serving integration.

Conclusion

We presented PGuard, a native streaming guardrail that operates directly on the intermediate representations of a frozen backbone. Through dynamic layer gating, PGuard aggregates residual-stream features from selected layers, enabling a trained guard module to produce risk scores throughout both prefill and autoregressive decoding. We further explored three guard configurations: a stateless variant and two sequence-aware variants based on EMA-attention and GRU, offering different trade-offs between detection performance and serving efficiency.

Our evaluation across more than 30 benchmarks demonstrates that PGuard achieves competitive detection performance against dedicated external guardrails while consistently outperforming existing native baselines in aggregate. Its streaming predictions enable early detection of unsafe responses, allowing potential risks to be identified before generation is complete. Integrated into SGLang, PGuard incurs only 3.56% and 3.07% increases in TTFT and TPOT, respectively, across the evaluated serving configurations. These results demonstrate the practical viability of native guardrails for online safety screening, while retaining the option of external adjudication for flagged traffic. We are also extending the integration to vLLM, with adaptation work currently underway.

Limitations. The effectiveness of PGuard depends on the extent to which safety-relevant information is encoded in the backbone’s intermediate representations. Attacks that induce harmful behavior without producing sufficiently informative signals in the captured representations may evade detection. Deep role-playing and fictional framing are potential examples that warrant targeted investigation. Furthermore, adapting PGuard to a new backbone requires calibration-set pretraining for layer selection, followed by guard-module training, even when the new backbone belongs to the same model family. These requirements introduce additional costs when deploying PGuard across different backbones.

Our current jailbreak evaluation relies on static benchmarks and does not establish robustness against adaptive attacks involving multi-turn interactions. Evaluating PGuard under dynamic red-teaming scenarios is therefore an important direction for future work. In addition, the framework leaves intervention policies to the deployment, motivating further investigation into how streaming risk scores can be translated into effective runtime actions. More expressive sequence models, including recurrent-EMA combinations and full self-attention, also merit exploration when serving budgets permit. Finally, we plan to extend the evaluation to larger backbones and additional modalities, and to release the guard models, benchmark pipeline, and SGLang serving integration alongside this work.

Authors

Core Contributors. Junyu Lin*, Mengjie Wu*, Yonghao Dong†, Yuchuan Fu†, Jinfeng Li☒, Xiaodan Li, Xiaohan Yuan, Yicheng Liu‡, Yuefeng Chen, Hui Xue, Jingyi Wang†

Contributors. Yitong Yang, Jiyuan Fu, Qiang He‡

(* equal contribution)

(† Intelligent System Security Lab, Zhejiang University, Hangzhou)

(‡ Services Computing Technology and System Lab, Huazhong University of Science and Technology, Wuhan)

Multi-Task Taxonomy and Training-Data Composition

Risk dimension	Risk category	Description
A. Crimes & Illegal Activities	*a1.*Pornographic Contraband	Obscene or sexually explicit material, including sexual depictions and pornographic resources, plus non-consensual or illegal sexual conduct and date-rape drugs.
	*a2.*Drug Crimes	Cultivation, synthesis, trafficking, and glorified personal use of illegal drugs and controlled substances.
	*a3.*Dangerous Weapons	Illicit blades, firearms, and explosives, notably CBRN (chemical, biological, radiological, nuclear) agents and devices such as anthrax, mustard/chlorine, Cs-137/Co-60 dirty bombs, and TATP/RDX car bombs.
	*a4.*Property Infringement	Unlawfully taking or damaging others’ property via burglary, theft, fraud, extortion, or robbery.
	*a5.*Economic Crimes	Financial fraud, market manipulation, tax evasion, bribery, money laundering, and IP infringement that disrupt economic order.
B. Hate Speech	*b1.*Abusive Curses	Profane, vulgar, or insulting language that degrades or attacks individuals or groups.
	*b2.*Defamation	Circulating false or fabricated claims to damage a person’s or organization’s reputation.
	*b3.*Threats & Intimidation	Direct or indirect expressions of intent to harm, coerce, or intimidate others.
	*b4.*Cyberbullying	Sustained online harassment, humiliation, exclusion, or doxxing of an individual.
C. Physical & Mental Health	*c1.*Personal Safety	Content encouraging harm to others or oneself, suicide and self-harm, fighting, domestic violence, or reckless dangerous behavior.
	*c2.*Mental Health	Material that worsens psychological distress or replaces proper care, including emotional manipulation, pseudo-scientific remedies, and discouraging treatment.
D. Ethics & Morality	*d1.*Social Ethics	Breaches of widely held moral norms, including prejudice and discrimination, devaluation of life or animal abuse, and coaching dishonest behavior.
	*d2.*Science & Tech Ethics	Hazardous use of science and technology, such as AI-enabled lethal weapons, anti-human plans, and unethical biotech like human cloning or germline editing.
E. Data Privacy	*e1.*Personal Privacy	Exposing, inferring, or speculating about non-public personal information (identity, contacts, financial, medical, judicial records) or teaching how to harvest it.
	*e2.*Commercial Secret	Disclosing or reconstructing a specific company’s non-public internal information (strategy, financials, customers, HR) or instructing how to obtain it.
F. Cybersecurity	*f1.*Access Control	Gaining credentials or privileges beyond authorized limits, including stolen auth secrets (AK/SK), device jailbreaking, and privilege escalation.
	*f2.*Malicious Code	Designing, generating, or spreading viruses, worms, Trojan horses, and ransomware.
Risk dimension	Risk category	Description
	$f_3$.Hacker Attack	Unauthorized intrusion into, disruption of, or data exfiltration from systems, networks, or services.
	$f_4$.Physical Security	Damaging network hardware or critical infrastructure (cables, fiber, base stations, substations) that underlies connectivity.
	$f_5$.Social Engineering, Phishing	Deceiving users into revealing information or granting access, via cloned login pages, forged-identity lures, spear-phishing, and filter evasion.
G. Extremism	$g_1$.Violent Terrorist Activities	Promotion, glorification, or instruction for politically, religiously, or ideologically motivated terrorist attacks and armed conflict.
	$g_2$.Social Disruption	Inciting or amplifying hostility, confrontation, or division among groups by ideology, religion, ethnicity, or identity.
	$g_3$.Extremist Ideological Trends	Spreading radical ideologies that advocate violence, hatred, or the overthrow of established systems.
H. Inappropriate Suggestions	$h_1$.Finance	Speculative, unlicensed, or high-stakes financial advice that risks serious monetary loss.
	$h_2$.Medicine	Diagnoses, treatments, or medication advice given without professional qualification that could endanger health.
	$h_3$.Law	Unauthorized legal interpretation or advice that could lead to adverse legal consequences.
I. Risks Involving Minors	$i_1$.Corruption of Minors	Leading minors toward harmful or delinquent behavior such as underage smoking, drinking, or truancy.
	$i_2$.Minor Abuse & Exploitation	Depicting, facilitating, or encouraging physical, psychological, or sexual abuse of children.
	$i_3$.Minor Delinquency	Involving minors as perpetrators of crime, or coaching them in illegal acts.

Modality	Source	#Samples	Description
Text	In-house annotated data	157,497	Safety-annotated data mapped to the shared taxonomy.
	Public safety corpora	71,095	Train splits of PolyGuardMix [28], SafeRLHF [7], Aegis 2.0 [11], WildGuardMix [17], BeaverTails [22], and SORRY-Bench [65].
Text total		228,592
MultiModality	SPA-VL (train split)	78,247	Safety preference alignment data for vision-language models [72].
	Honey-Data	20,000	General multimodal SFT corpus [71] for benign data.
Image-text total		98,247

Decision Threshold Configuration

Table 19: Decision thresholds $\tau_{c,y}$ in percent, calibrated separately for each guard. Rows are split by stage and input: Q and R denote the query and response stages, and T and VL denote text and image-text inputs. Columns are the nine risk dimensions (A–I). Each cell lists the per-category thresholds of that dimension in $a_1 \to a_n$ order.

Configuration	Stage ($\tau$, %)	A Crimes	B Hate	C Health	D Ethics	E Privacy	F Cyber	G Extremism	H Inappr.	I Minors
lasttok (Qwen3-8B)	Q-T	65/36/54/42/53	66/59/10/18	71/47	71/69	75/74	87/62/25/62/10	31/48/62	68/54/34	87/50/24
	R-T	55/82/55/19/55	33/43/24/20	47/65	61/14	45/70	41/65/30/43/29	59/44/68	77/67/56	38/72/12
gru (Qwen3-8B)	Q-T	75/83/61/28/68	73/24/10/10	61/47	62/56	46/74	80/33/40/21/19	29/70/56	70/62/16	83/57/18
	R-T	72/97/79/21/79	43/23/13/15	77/99	90/35	91/89	72/45/29/42/68	86/70/66	87/69/10	74/85/19
ema (Qwen3-8B)	Q-T	45/25/65/19/66	42/10/10/10	26/35	30/11	45/35	30/39/11/13/12	27/66/19	49/43/14	37/30/10
	R-T	45/38/34/37/54	34/34/16/12	37/41	58/19	56/69	35/59/14/29/10	50/31/65	44/46/10	18/49/10
lasttok (Qwen3.5-9B)	Q-T	71/25/89/89/48	75/11/18/20	47/81	66/27	77/75	52/60/30/13/10	38/71/64	72/45/10	92/76/10
	Q-VL	90/64/89/95/15	56/10/11/15	90/11	87/10	91/53	19/10/27/19/10	47/29/27	10/52/14	90/55/10
	R-T	74/96/83/54/50	54/56/45/51	66/70	80/44	74/87	84/70/49/69/40	82/62/80	89/85/74	81/87/11
	R-VL	83/94/64/84/30	39/26/17/25	83/72	70/35	79/39	83/54/25/70/10	77/90/75	76/93/38	55/80/10
gru (Qwen3.5-9B)	Q-T	86/41/55/39/62	74/30/14/22	70/67	54/38	62/80	76/32/63/53/49	64/70/33	53/64/12	82/48/21
	Q-VL	11/10/10/10/10	10/10/43/11	10/10	10/14	10/10	13/19/65/10/10	19/11/10	10/10/10	10/10/10
	R-T	65/85/73/32/49	26/80/51/70	70/54	80/38	68/91	77/37/20/50/42	70/65/51	90/95/10	96/51/10
	R-VL	18/10/50/39/10	10/10/19/10	14/10	58/53	10/10	10/10/10/10/10	10/10/10	33/95/83	10/10/10
ema (Qwen3.5-9B)	Q-T	52/35/74/34/29	54/14/10/33	40/62	54/43	63/70	25/68/19/10/11	20/50/45	39/54/11	30/63/14
	Q-VL	10/10/11/10/10	10/15/10/10	10/10	10/22	10/10	10/10/10/10/10	10/39/20	10/10/10	10/10/10
	R-T	73/96/59/72/65	36/28/90/54	67/94	95/25	76/88	36/81/35/18/81	85/65/92	88/73/50	87/73/20
	R-VL	30/77/10/61/49	30/14/45/27	23/20	33/63	47/10	35/25/27/10/86	10/20/34	38/98/49	10/10/10
lasttok (Qwen3.5-flash)	Q-T	75/64/75/61/71	73/65/10/10	61/62	69/42	83/85	75/73/16/26/10	65/25/67	64/59/65	75/75/18
	Q-VL	80/32/54/83/61	58/21/10/10	83/10	85/30	75/65	19/11/70/40/11	65/21/19	16/44/16	78/72/10
	R-T	51/58/53/39/56	42/26/21/49	61/68	61/34	59/88	57/48/34/51/31	60/45/49	80/72/38	28/69/10
	R-VL	93/26/26/55/11	17/12/27/11	60/19	61/34	59/88	57/48/34/51/31	37/28/34	52/93/37	28/69/10
gru (Qwen3.5-flash)	Q-T	48/76/62/42	63/25/10/13	50/64	75/23	78/75	86/39/45/10	77/75/58	51/72/22	77/76/15
	Q-VL	10/10/10/10	12/11/55/10	10/10	10/10	10/10	10/27/11/10	10/10/10	10/10/10	24/35/10
	R-T	72/86/78/30	21/76/29/40	65/30	77/53	56/83	59/38/33/54	86/49/76	87/58/31	87/52/10
	R-VL	28/50/34/10/10	14/15/10/10	28/13	34/32	10/10	10/36/10/10/10	10/11/10	14/94/10	10/10/10
ema (Qwen3.5-flash)	Q-T	65/85/67/56/65	73/15/11/44	75/25	75/42	62/92	77/13/29/10	45/51/50	52/61/10	54/56/19
	Q-VL	10/10/10/10/10	24/25/93/14	10/10	10/15	10/10	10/44/10/10/10	10/46/10	12/10/10	10/10/10
	R-T	65/78/41/28/61	47/16/25/39	53/94	66/19	47/81	80/59/27/61/55	74/45/90	86/63/33	43/57/18
	R-VL	40/83/10/64/10	17/31/11/15	10/32	21/87	20/16	34/10/15/10/95	67/19/86	11/94/55	10/10/10
lasttok (Qwen3.5-122B-A10B)	Q-T	76/84/77/46/61	67/31/10/13	63/80	87/28	73/93	10/75/72/15/10	67/52/63	77/45/26	34/70/34
	Q-VL	10/11/12/20/10	35/10/10/10	30/10	24/10	11/10	10/10/13/10/10	10/10/15	10/12/38	22/35/10
	R-T	64/79/68/18/61	36/29/39/14	40/85	70/17	39/85	41/60/34/33/13	59/43/70	72/72/39	54/68/10
	R-VL	19/36/10/60/10	26/29/17/19	12/17	29/25	28/10	10/10/10/10/10	59/43/70	10/10/10	10/10/10
gru (Qwen3.5-122B-A10B)	Q-T	71/82/77/78/28	79/25/17/22	55/56	68/33	78/83	57/40/69/26/15	61/59/38	84/57/12	65/47/10
	Q-VL	10/10/10/10/10	11/25/10/11	10/10	10/10	10/10	10/10/10/10/10	10/12/10	10/10/10	10/21/10
	R-T	79/93/56/27/67	25/16/28/74	61/98	74/18	52/78	42/40/53/38/41	71/61/25	93/83/66	84/30/10
	R-VL	10/10/75/16/10	12/10/15/10	17/10	12/54	20/10	10/31/10/10/10	15/10/10	32/97/10	10/30/10
ema (Qwen3.5-122B-A10B)	Q-T	70/82/46/48/60	65/16/10/20	59/39	75/37	69/86	52/70/35/65/11	52/55/10	68/35/10	48/59/10
	Q-VL	10/10/10/10	18/33/10/10	15/10	17/24	15/10	10/10/13/10/10	36/47/16	11/10/10	11/47/10
	R-T	85/81/77/29/59	47/10/15/34	70/97	86/14	70/87	68/74/30/36/21	61/56/80	97/74/37	49/66/10
	R-VL	10/35/10/42/10	10/10/12/10	10/10	47/25	77/10	20/14/18/10	12/17/14	10/92/10	10/60/10

Layer Selection Configurations

Modeling Method	Backbone Model	Top-3 Layers
lasttok	Qwen3-8B	17,18,19
gru / ema	Qwen3-8B	19,20,35
lasttok / gru / ema	Qwen3.5-9B	20, 24, 28
lasttok / gru / ema	Qwen3.5-flash	20, 24, 28
ema	Qwen3.5-122B-A10B	32,36,40
lasttok / gru	Qwen3.5-122B-A10B	36,40,44

Serving Configuration

Table 21: Serving configuration of each backbone. Memory fractions are listed for the stateless, EMA-attention, and GRU variants, respectively.