WorksComputer Science
KVMARK: Fragile Watermarking for KV-Cache Integrity in Latent Multi-Agent Systems
Abstract
Latent multi-agent communication directly transfers Key-Value (KV) cache tensors between transformer-based agents, bypassing text serialization entirely. While computationally efficient and reasoning accurate, this communication channel is opaque and vulnerable to man-in-the-middle tampering, a threat vector unaddressed by existing text-based or behavioral watermarks, which cannot operate on continuous internal representations. To address this vulnerability, we propose KVMark, a training-free, zero-bit KV-cache watermarking framework to verify inter-agent handoff integrity. Our framework introduces four core technical components: (1) position-dependent sign patterns that guarantee first-order statistical invisibility while preserving downstream attention survival; (2) per-head sign patterns with aligned detection to defeat head-averaging attacks; (3) a three-tier depth-aware strength schedule that equalizes detection sensitivity across transformer layers while maintaining invisibility against nine statistical profiling strategies; and (4) content-bound key derivation that cryptographically binds watermarks to input contexts, preventing replay attacks under Kerckhoffs's principle. Empirical evaluation on a four-agent latent pipeline demonstrates detection across handoffs with a false positive rate (), layer-level tamper localization ( attacked layers identified with false positives), and no statistically significant reasoning accuracy degradation across four benchmarks (–). Furthermore, the framework enables multi-hop cascade verification to isolate compromised agents (500/500 detection at three hops), withstands adaptive white-box adversaries and the Out-of-Sight DiffMean attack (up to ), and generalizes seamlessly to Qwen3-8B without re-tuning (1,500/1,500 TPR).
Introduction
Large language models (LLMs) have rapidly evolved from standalone reasoners into autonomous agents capable of multi-step planning, tool invocation, and self-reflection [23, 29, 31]. To solve tasks that exceed any single model’s capacity, recent work composes multiple specialized agents into collaborative pipelines—multi-agent systems (MAS)—where distinct roles such as planner, critic, refiner, and judge coordinate through structured inter-agent communication [27, 29, 33]. Traditionally, multi-agent systems coordinate via natural language, which compresses rich internal states into discrete tokens but imposes a fundamental information bottleneck [11, 33]. To maximize representational capacity, latent multi-agent systems transmit internal KV caches directly between agents, thereby bypassing text serialization entirely [6, 33]. While this training-free paradigm achieves superior reasoning accuracy [33], transmitting high-dimensional continuous tensors simultaneously moves the communication channel beyond conventional inspection, filtering, or content moderation. This architectural shift introduces a critical security gap: an unscrutinized, continuous-valued attack surface lacking any native integrity-verification mechanism.
This vulnerability is concrete and empirically validated. Wang et al. Wang et al. [25] show that inter-agent KV-cache handoffs constitute the most damaging attack vector in latent MAS: offline-extracted perturbations generalize broadly, while V-cache modifications remain invisible to output-quality checks. Crucially, current frameworks [6, 33] lack native integrity verification. While transport-layer mechanisms (e.g., HMACs) catch post-handoff substitution, they fail against compromised endpoints that generate and sign malicious KV in-band [3]. Because downstream reasoning is causally dependent on upstream KV content—where removing a single specialist’s hidden state causes nearly half of correct outputs to fail [3]—unverified KV transfers pose a severe threat.
While watermarking is the natural candidate for this gap, existing methods fail in the latent setting: text watermarks [8, 15, 16, 32] require logit sampling during decoding, while agent watermarks [12, 13, 19, 26] target discrete action trajectories rather than continuous tensors. More fundamentally, watermarking a KV cache presents two unique challenges. First, to prevent keyless adversaries from profiling and removing the signal, the watermark must be *statistically invisible. Second, to detect compromised intermediate agents, an upstream watermark must survive downstream attention computation so monitors can verify its presence at later pipeline stages. Achieving both properties simultaneously has not been demonstrated by prior work.
To address these challenges, this paper introduces KVMARK, the first watermarking scheme specifically designed for KV-cache integrity verification in latent multi-agent systems. As illustrated in Figure 1, our approach embeds fragile, zero-bit statistical watermarks into both the V-cache and K-cache at each inter-agent handoff using agent-specific keys, enabling a centralized verification module (operated by the pipeline defender) to confirm provenance, detect malicious tampering, and identify forgery attempts. The KVMARK framework’s design addresses the core security requirements: position-dependent sign patterns guarantee first-order statistical invisibility (column-mean invariance) against keyless adversaries and per-head sign patterns with aligned detection mitigate head-averaging attacks, while depth-aware magnitude scaling preserves invisibility against higher-order statistical profiling (variance, kurtosis, range, absolute mean, and cross-cache correlation). Furthermore, leveraging the append-only structure of autoregressive KV-cache handoffs across the pipeline, the embedded signal persists across downstream execution stages, enabling cascade verification to isolate compromised intermediate agents.

Figure 1. The High-level architecture of KVMARK. Top Left: Watermark Embedding applies an agent-specific key to embed a keyed perturbation into and tensors prior to transfer. Bottom: The KV-cache Transfer Pipeline maintains an append-only sequence of agent states (), subject to adversarial agent compromise or KV manipulation. Top Right: The Watermark Verification module evaluates incoming multi-hop caches against agent-specific keys to verify integrity and detect forged or tampered entries.
The watermark embedding and verification pipeline introduces three core innovations. First, to resolve the tension between attention survival and detectability, we propose position-dependent sign patterns that achieve both statistical invisibility and downstream propagation simultaneously. Second, to address distinct depth-dependent decay profiles between V-cache and K-cache spaces, we introduce a three-tier
V-cache strength schedule (suppress early, baseline middle, boost late) alongside uniform K-cache boosting, equalizing detection sensitivity across all transformer layers while maintaining full statistical invisibility. Third, to mitigate replay attacks in multi-agent pipelines, we propose content-bound key derivation that cryptographically binds each watermark to both the agent’s secret key and the specific input context, thereby satisfying Kerckhoffs’s principle while preventing cross-query cache reuse.
Our main contributions are:
We propose KVMARK, a training-free, zero-bit KV-cache watermarking framework designed to verify handoff integrity in latent multi-agent systems. Empirical evaluation across 1,500 independent handoff verifications demonstrates the scheme achieves 100% TPR at 0.0% FPR (AUROC = 1.0) with no statistically significant accuracy degradation (92.4% vs. 92.0% clean baseline on GSM8K).
We characterize divergent V-cache and K-cache depth profiles and resolve them via a three-tier V-cache strength schedule (0.1× at layers 0–4, 1.0× at layers 5–23, and 5.0× at layers 24–35) and uniform K-cache boosting (10× across all layers). This calibration achieves 100% layer-level attack localization (17/17 layers identified, ) and full statistical invisibility across nine detection strategies.
We evaluate system resilience against a broad attack suite. Experimental results demonstrate 100% detection against quantization, adaptive white-box attacks, the DiffMean attack [25], and a 99.8% replay rejection rate (). Finally, the framework enables cascade verification to detect compromised intermediate agents across multi-hop pipelines (500/500 at 3 hops).
Background and Related Work
Latent Multi-Agent Communication
In a standard transformer-based LLM, each layer maintains a key-value cache that stores the computed attention keys and values for all previously processed tokens, avoiding redundant recomputation during autoregressive generation [24]. In text-based multi-agent systems, Agent generates a text response that Agent re-tokenizes and processes from its input embedding layer, discarding ’s internal representations entirely.
Latent multi-agent systems exploit a key observation: when all agents share the same model architecture, the KV cache from Agent is dimensionally compatible with Agent ’s attention layers. Rather than serializing to text, LatentMAS [33] transfers the full cache directly, allowing the receiving agent to continue computation as if it had generated those representations itself. This latent handoff preserves the sender’s full hidden-state information across all layers and attention heads, eliminating the information bottleneck inherent in text-based communication. Empirically, latent communication yields consistent accuracy improvements over text-based pipelines on mathematical reasoning [5], commonsense, and code-generation benchmarks, while requiring no additional training or architectural modification [33].
Concurrent work addresses complementary concerns in KV-based latent communication. Li et al. [17] reduce the communication cost of full KV-cache relay by compressing the cache at inter-agent handoffs, retaining only 10–20% of positions via attention-based selection with orthogonal residual injection. Asif et al. [2] study the privacy dimension: their LCGuard framework learns representation-level transformations on KV caches before transmission to limit the recoverability of agent-specific sensitive inputs, formalizing leakage through an adversarial reconstruction objective. Neither work addresses integrity of the KV cache in transit, which is the focus of the present paper.
Watermarking in Language Model Systems
Watermarking techniques for language models fall into two broad categories distinguished by the abstraction level at which they operate: text-generation watermarks that modify the token sampling process, and agent-level watermarks that embed signals into behavioral trajectories.
Text-generation watermarks. KGW [15] partitions the vocabulary into green and red lists conditioned on a secret key and preceding tokens, biasing the sampling distribution toward green-list tokens for zero-bit detection via hypothesis testing. Three Bricks [8] extends this framework with provably optimal detection boundaries and improved robustness guarantees. Subsequent work addresses specific trade-offs: Kuditipudi et al. [16] achieve distortion-free watermarking by coupling the key schedule to a pseudorandom sequence that leaves the output distribution unchanged, while Zhao et al. [32] provide provable robustness bounds against bounded text edits. A comprehensive empirical analysis by Piet et al. [21] reveals fundamental trade-offs between detectability, output quality, and robustness across these schemes, echoed by the theoretical treatment of Pang et al. [20]. However, all text-generation watermarks operate at the logit level during token sampling—a mechanism that is entirely absent at a latent KV-cache handoff, where no tokens are generated and the transferred object is a continuous-valued tensor.
Agent-level watermarks. A parallel line of work embeds provenance signals into agents’ behavioral outputs rather than token logits. One group biases the sampling distribution over discrete agent decisions—planning steps [12, 13], tool-execution paths [26]—to encode multi-bit or zero-bit identifiers, with detection via -statistics or distribution-preserving sampling. A second group instead attributes trajectories after the fact, embedding signals into action-transition patterns or complementary trajectory encodings for position- and deletion-robust verification [1, 9]. Across both groups, the watermarked object is a discrete action sequence—tool calls, API invocations, step orderings—with no counterpart in a KV-cache transfer, where the communicated object is a set of continuous attention keys and values across layer dimension () and head dimension ().
The KV-cache gap. Neither text-generation nor agent-level watermarks address the latent communication channel. More fundamentally, both families assume the watermarked object is either a token sequence or an action trace—discrete structures amenable to distribution biasing or sequence encoding. A KV cache is a continuous, high-dimensional tensor that undergoes nonlinear transformation through the receiving agent’s attention layers, creating a survival challenge that has no analogue in existing watermarking literature. Understanding the specific attack mechanisms that exploit this unprotected channel is essential for designing an effective defense.
Attacks on Latent Multi-Agent Systems
Security analysis of multi-agent systems has primarily focused on text-level threats: prompt injection [28], adversarial role manipulation, and inter-agent deception through crafted natural-language messages [7, 27]. These attacks exploit the interpretability of text-based communication—the same property that makes text amenable to filtering and moderation.
The emergence of latent communication introduces a fundamentally different attack surface. Wang et al. [25] provide the first systematic study of attacks on KV-cache handoffs in LatentMAS, introducing the DiffMean attack: given a small set of paired KV caches (one from a clean-correct execution, one from the corresponding text-level-attacked execution), the attacker computes the element-wise mean difference to extract a single perturbation direction that, when injected into a new clean cache at a handoff point, reproduces the attack’s effect without reusing the adversarial text. This attack is notable for three properties that distinguish it from text-level threats. First, the perturbation operates in continuous latent space and produces no visible change in any text output at the injection point. Second, V-cache-only perturbations cause negligible accuracy change (), making the attack nearly undetectable by output-quality monitoring. Third, the extracted direction generalizes across inputs: a single perturbation vector retains 92% of its degradation on held-out samples, eliminating the need for per-input attack crafting.
The study further reveals an asymmetry between cache components (under PCA extraction): K-cache perturbations (which modify attention routing) cause moderate accuracy drops (0.215), while combined KV perturbations cause the most severe degradation (0.398). This component-level characterization establishes that V-cache is the least disruptive component to modify, while K-cache and combined KV attacks represent the most damaging threat vectors. The paper’s threat model—an adversary who can intercept and modify outgoing KV-cache handoffs before the downstream agent consumes them—defines precisely the man-in-the-middle scenario that motivates our integrity verification scheme.
Brito and Baquero [3] study KV-cache integrity in a role-sequenced planner-specialist-coordinator architecture. Their system transfers both a short visible text commitment and the full KV-cache hidden state separately to the coordinator; because these are independent channels, an attacker can corrupt the hidden-state channel while leaving the visible text intact, rendering text-level inspection ineffective. Their HMAC-SHA256 transport manifest achieves perfect detection for post-handoff substitution (774/774 honest payloads accepted, 295/295 tampered rejected), but their evaluation explicitly identifies a protection gap: compromised endpoints that sign malicious KV in-band produce valid signatures, and their magnitude quarantine is defeated by adaptive norm-matched attacks (0/65 detected under white-box optimization). Our watermark-based approach targets precisely this gap: the keyed statistical signal is embedded in the KV-cache values themselves, so even a compromised endpoint cannot produce a validly watermarked cache without the secret key.
Problem Formulation
We consider a sequential latent multi-agent system comprising agents that share the same transformer architecture with layers and attention heads per layer. At each handoff, Agent transfers its KV cache to Agent , where denote the key and value tensors at layer , with batch size , sequence length , and head dimension . The pipeline operator holds a single master secret , confidential to the defender. Each agent ’s watermarking key is derived as , from which the V-cache and K-cache watermarking keys follow: and (§ 4.2). Deriving from a single master secret rather than issuing independent secrets simplifies key management: the defender need only protect one value, and per-agent keys can be recomputed on demand from and the (public) role name. Each agent receives only its own derived key through a secure initialization channel; agents cannot derive other agents’ keys. Table 1 summarizes the notation used throughout the paper.
| Symbol | Definition |
| Sequential agents in the pipeline | |
| Layers, heads, head dimension, seq. length | |
| Key and value tensors at layer | |
| Original / watermarked KV cache | |
| Received cache at detection (possibly tampered) | |
| Master secret (defender-held) | |
| Per-agent key; content-bound variant | |
| V-cache and K-cache derived keys | |
| Watermark dimensions at layer ( each) | |
| Per-head position-dependent sign patterns | |
| Aligned head-averaged cache values (Eq. (6)) | |
| Expected signal for alignment () | |
| Per-layer embedding strengths | |
| V-cache base, boost (), suppress () | |
| K-cache base, boost () | |
| V, K, and combined detection scores | |
| Detection threshold |
Table 1. Notation summary.
Threat Model
Assumptions. Adhering to Kerckhoffs’s principle, we assume the adversary possesses comprehensive knowledge of
the watermarking scheme—including the embedding algorithm, detection procedure, and the correlation-based detection formula—while the master secret remains strictly confidential. The adversary has access to watermarked KV caches (by intercepting handoffs) but cannot observe the key derivation process, so knowing an agent’s public role name does not reveal . We target man-in-the-middle adversaries who intercept and modify caches between agents. A compromised agent that uses its own key to re-watermark corrupted output is partially addressed through cascade verification (§ 5.3). A fully compromised defender whose master secret is leaked is outside scope, as this requires trust establishment mechanisms beyond watermarking.
Adversary models. We evaluate three distinct adversarial objectives:
Tampering attackers aim to modify the KV cache in transit to steer downstream agent behavior while evading watermark detection. This includes additive perturbations (Gaussian noise, steering vectors), targeted attacks (DiffMean [25]), and cache quantization (int8, fp16). The attacker may target V-cache only, K-cache only, or both components, and may restrict modifications to specific transformer layers.
Forgery attackers attempt to produce a KV cache that passes watermark verification without possessing the secret key. Strategies include wrong-key forgery, statistical estimation of the watermark pattern from observed caches (average-and-subtract), and blind dimension zeroing.
Replay attackers intercept a legitimately watermarked cache from a prior query and re-inject it as the response
to a different query, exploiting the fact that the watermark itself is valid even though the content is contextually in-correct.
Defender objectives. The defender (pipeline operator) holds the master secret and maintains white-box access to all agents, deriving each on demand as needed. The verification module performs zero-bit detection: it verifies handoff authenticity (“did this cache come from agent ?”) rather than embedding multi-bit identity payloads. In a fixed pipeline with known agent roles, zero-bit detection is sufficient because the topology already determines sender identity at each handoff. Upon receiving a KV cache at each handoff point, the module fulfills three objectives:
Integrity verification: determine whether the received cache was produced by the expected sender and has not been modified in transit.
Tamper localization: if tampering is detected, identify (a) which handoff edge was compromised, and (b) which transformer layers were affected.
Invisibility: the watermark must not be detectable by a keyless adversary through statistical analysis of the cache values, ensuring that knowledge of the watermarking scheme alone is insufficient to locate or remove the embedded signal.
Method
Overview
The integrity verification scheme of KVMARK operates at each inter-agent handoff in the latent MAS pipeline. The protocol consists of two operations: embedding, performed by the sending agent after generation, and detection, performed by the verification module (operated by the pipeline defender) before processing. Figure 2 illustrates the complete pipeline.

Figure 2. Watermark embedding (top) and detection (bottom) in KVMARK. The sending agent selects watermark dimensions via SHA-256, generates a per-head position-dependent sign pattern , applies depth-aware boosting, and embeds additive perturbations into both V-cache and K-cache. At the handoff, the verification module regenerates the expected dimensions and signs from the shared key, correlates with the received cache, and aggregates per-layer scores into , , and . Per-layer scores enable tamper localization.
Given a sequential pipeline , each handoff proceeds as follows:
Agent completes its generation, producing KV cache .
The embedding module computes a keyed watermark signal from the secret key and embeds it into both the V-cache and K-cache of , producing .
The watermarked cache is transmitted to Agent .
Before processing, the verification module (operated by the pipeline defender) verifies using key , producing detection scores , , and their combined score . If falls below a threshold , the handoff is flagged as potentially tampered.
If verification passes, Agent proceeds with its own generation using the received cache.
The watermark is fragile by design: any modification to the KV cache that is strong enough to affect downstream reasoning also disrupts the embedded signal, ensuring there
is no “silent tampering” window where an attacker causes harm without detection. The key is derived from the agent role and, optionally, the input context (§ 4.5), providing both sender authentication and replay resistance.
Watermark Embedding
The embedding design must satisfy two requirements simultaneously. First, the watermark must be statistically invisible: a keyless adversary who intercepts the cache in transit must not be able to identify which dimensions carry the signal, since an attacker who locates the watermark can surgically remove it before the receiving agent verifies. Second, for cascade verification across pipeline stages (§ 5.3), the watermark must propagate through attention computation: when Agent processes the received cache, each attention head computes , a convex combination across positions (). A uniform perturbation along a dimension therefore propagates exactly (out = original ), whereas a perturbation at a specific position is diluted by its attention weight. We note that the KV cache is append-only: the receiving agent’s generation appends new positions but does not modify existing ones. Per-handoff verification, which checks original positions before processing, is therefore agnostic to embedding strategy; the propagation requirement applies specifically to cascade verification of newly generated positions (§ 5.3). This propagation argument applies to V-cache; K-cache values determine attention weights rather than being convexly combined, so K-cache watermarks do not propagate to newly generated positions. However, since the cache is append-only, cascade verification of K-cache checks original positions directly.
These two requirements create a design tension: a uniform perturbation along selected dimensions propagates through attention but is trivially detectable via column-mean analysis. We resolve this tension through a combination of keyed dimension selection (which hides which dimensions carry the signal) and position-dependent sign patterns (which prevent column-mean analysis from detecting the signal even at the correct dimensions).
Keyed Dimension Selection. For each transformer layer , we select subsets of watermark dimensions using a keyed hash function:
where denotes string concatenation, “dims” is a fixed domain separator that ensures dimension selection and sign generation (Eq. (3)) produce independent hash outputs, and deterministically maps the hash output to distinct indices. For K-cache, we define and compute dimensions independently:
where “” is a domain separator ensuring V-cache and K-cache select independent dimensions from the same key. Without knowledge of (and hence ), an adversary cannot determine which dimensions carry the watermark.
Position-dependent Sign Pattern. Even with keyed dimension selection, a constant bias is detectable: an attacker computes for each dimension ; under a constant bias, the watermarked dimensions show while others show , immediately revealing the watermark. To defeat this, we introduce a position-dependent sign pattern , a pseudorandom matrix of values keyed by the layer index and secret key. The pseudorandom signs yield approximately balanced columns (), making watermarked dimensions statistically indistinguishable from non-watermarked ones under column-mean analysis.
Per-head Sign Pattern. A second attack vector arises from the multi-head structure of the KV cache: an adversary who averages across all attention heads can suppress natural per-head variation while preserving a uniform watermark signal. To defeat this, we derive an independent sign pattern for each head by including the head index in the PRG seed:
where \sigma_K^{(l,h)} = \operatorname{PRG}\left(\operatorname{SHA\mathchar`-256}(k_K \mathbin{\|} \text{“poss ig”} \mathbin{\|} l \mathbin{\|} S \mathbin{\|} h), S, n_d\right).
where denotes a seeded generator producing values, “possig” is a domain separator distinguishing sign generation from dimension selection (Eq. (1)), and is the attention head index. Since each head’s sign pattern is independent, head-averaging cancels the watermark: . The keyholder recovers the signal via aligned detection (§ 4.3), which multiplies each head by its known signs before averaging.
The watermark is embedded by additive perturbations to both the V-cache and K-cache at each layer. For each layer , head , position , and watermark dimension :
where and are per-layer embedding strengths (Section 4.4), and all dimension sets and sign patterns follow Eqs. 1–3 with key for V-cache and for K-cache. Using independent keys ensures that the V and K watermarks occupy different dimensions with overwhelming probability, avoiding interference. The perturbation magnitude is identical across all batch elements and attention heads; however, the per-head sign patterns ensure that no cross-head
Algorithm 1 Watermark Embedding (EMBED)
Require: Cache $M=\{K^{(l)},V^{(l)}\}_{l=1}^{L}$ with seq. length $S$, key $sk_i$, strengths $\alpha_{V,0},\alpha_{K,0}$, dims $n_d$
Ensure: Watermarked cache $\widetilde{M}$
1: $k_K \leftarrow sk_i\Vert\text{“\_K”}$
2: for $l=1$ to $L$ do
3: $\mathcal{D}_{V}^{(l)}\leftarrow\operatorname{Select}(\operatorname{SHA\text{-}256}(sk_i\Vert\text{“dims”}\Vert l),d,n_d)$
4: $\mathcal{D}_{K}^{(l)}\leftarrow\operatorname{Select}(\operatorname{SHA\text{-}256}(k_K\Vert\text{“dims”}\Vert l),d,n_d)$
5: $\alpha_{V}^{(l)}\leftarrow\operatorname{SCALE}_{V}(l,\alpha_{V,0})$ ▷ {xref}`Eq. 11`
6: $\alpha_{K}^{(l)}\leftarrow\operatorname{SCALE}_{K}(l,\alpha_{K,0})$ ▷ {xref}`Eq. 12`
7: for $h=1$ to $H$ do ▷ Per-head sign patterns
8: $\operatorname{seed}\leftarrow\operatorname{SHA\text{-}256}(sk_i\Vert\text{“possig”}\Vert l\Vert S\Vert h)$
9: $\sigma^{(l,h)}\leftarrow\operatorname{PRG}(\operatorname{seed},S,n_d)$
10: $\operatorname{seed}_{K}\leftarrow\operatorname{SHA\text{-}256}(k_K\Vert\text{“possig”}\Vert l\Vert S\Vert h)$
11: $\sigma_{K}^{(l,h)}\leftarrow\operatorname{PRG}(\operatorname{seed}_{K},S,n_d)$
12: for $s=1$ to $S$ do
13: for each $j\in\mathcal{D}_{V}^{(l)}$ do $V_{h,s,j}^{(l)}\mathrel{+}=\alpha_{V}^{(l)}\cdot\sigma_{s,j}^{(l,h)}$
14: end for
15: for each $j\in\mathcal{D}_{K}^{(l)}$ do $K_{h,s,j}^{(l)}\mathrel{+}=\alpha_{K}^{(l)}\cdot\sigma_{K,s,j}^{(l,h)}$
16: end for
17: end for
18: end for
19: end for
20: return $\widetilde{M}=\{\widetilde{K}^{(l)},\widetilde{V}^{(l)}\}_{l=1}^{L}$
averaging strategy can recover the watermark signal without the key (§ 5.3). Dimensions outside and remain unchanged.
Watermark Detection
Given a received KV cache and the secret key , the detection module reconstructs the expected watermark signals for both V-cache and K-cache and measures their correlation with the actual values. For each layer , the detector recomputes the watermark dimensions , and per-head sign patterns , using the same key-derived hash functions as the embedding module (Eqs. (1)–(3)), with key for V-cache and for K-cache.
Detection employs aligned averaging: rather than directly averaging cache values across heads, the detector first multiplies each head’s received cache (from ) by its known sign pattern , which aligns all heads to a constant signal (since ), then averages across heads:
yielding the same noise-reduction benefit as standard head-averaging while preserving the watermark signal. The expected signal after alignment is constant: for all .
The per-layer detection scores are normalized dot products between the expected signals and the aligned cache values at the watermark dimensions:
where denotes the detection position range: for standard per-handoff verification, and (the original embedding length) for cascade verification across pipeline stages (§ 5.3). and are the aligned head-averaged values (Eq. (6)), and is the norm.
The aggregate detection score combines V-cache and K-cache correlations:
A handoff is classified as verified if , where is the detection threshold. We set in all experiments. The empirical gap between correct-key scores (min 0.291) and wrong-key scores (max 0.0070) means any achieves 0% FPR and 100% TPR; our falls within this gap, yielding 0.0% FPR (0 false positives across 1,500 handoffs). The individual scores and provide diagnostic information: a targeted V-only attack reduces while leaving intact, and vice versa. The per-layer scores are additionally retained for layer-level tamper localization (§ 4.4).
Depth-Aware Strength Scheduling
A fixed embedding strength leads to depth-dependent detection sensitivity decay. The V-cache and K-cache exhibit distinct behavior: V-cache magnitude grows with transformer depth, causing the normalized correlation (Eq. (7)) to diminish at late layers; K-cache sensitivity is uniformly low across all layers. Appendix C provides full per-layer magnitude and correlation profiles across all 36 layers. To equalize detection sensitivity, we introduce independent strength schedules for V and K, defining the per-layer strength functions and (Algorithms 1–2) as:
where and suppress the V-cache at early layers (where the natural cache magnitude is comparable to the perturbation, making the watermark detectable by
Algorithm 2 Watermark Detection (DETECT)
Require: Cache , key , strengths , dims , threshold , position limit
Ensure:
higher-order statistics such as variance and absolute mean), and boost the V-cache at late layers (where the decay is most severe), while boosts the K-cache uniformly across all layers (since K requires amplification everywhere). In all experiments, . These values are set empirically from the per-layer cache magnitude profile: suppress covers layers where V-cache std is comparable to the perturbation (), boost begins where uniform correlation drops below 0.08, and is the highest multiplier that maintains full statistical invisibility (; see Appendix C for detailed per-layer analysis).
The three-tier schedule ensures that the perturbation magnitude remains a small fraction of each layer’s natural signal at every depth: position-dependent sign patterns defeat first-order statistics (column-mean), while magnitude scaling defeats higher-order statistics (variance, kurtosis, range, absolute mean, cross-cache correlation) (Section 5.3). Additionally, per-head sign patterns (Section 4.2) defeat head-averaging attacks. The per-layer scores and (Eqs. 8–9) directly enable tamper localization: a layer affected by tampering shows a statistically significant drop in its correlation score relative to unaffected layers.
Content-Bound Key Derivation
A static watermark key that depends only on the agent role is vulnerable to replay attacks: an adversary who intercepts a legitimately watermarked cache from query can re-inject it as the response to a different query , and the verification module’s check will pass because the watermark is genuinely present and the key matches.
To bind the watermark to the specific input context, we replace the static watermark key with a content-bound variant , derived from the per-agent key and a hash of the input query:
where is the input query text and the subscript denotes the first 16 hexadecimal characters of the hash digest. All other embedding and detection equations (Eqs. 1–10) are unchanged: is simply substituted for wherever the key appears. The verification module independently computes the same key from the query originally sent, ensuring that a cache watermarked for query fails verification when checked against the key derived from . This one-line modification reduces replay attack success from 100% to 0.2% (Section 5.3) while preserving all other security properties, since only the key derivation changes—the embedding and detection algorithms are identical.
Experiments
Experimental Setup
Model and pipeline. We evaluate on the LatentMAS [33] framework using Qwen3-4B [22] as the shared backbone across all agents. The pipeline consists of four sequential agents—Planner, Critic, Refiner, and Judger—communicating via direct KV-cache transfer across transformer layers with key-value heads (grouped query attention with 32 query heads) and head dimension .
Datasets. We evaluate utility preservation on four benchmarks: GSM8K [5] (math reasoning), ARC-Challenge [4] (science reasoning), MBPP+ [18] (code generation), and MedQA [14] (medical question answering). Sample sizes are reported in Table 2. Detection and security experiments use GSM8K (, yielding 1,500 handoff verification events across three inter-agent edges). All experiments use exact-match accuracy against gold answers.
| Task | Clean | KV-WM | ||
| GSM8K | 500 | 92.0% | 92.4% | +0.4% |
| ARC-Challenge | 500 | 92.2% | 93.4% | +1.2% |
| MBPP+ | 378 | 67.2% | 67.7% | +0.5% |
| MedQA | 300 | 71.0% | 69.0% | −2.0% |
Table 2. Task accuracy across four benchmarks (full pipeline). All changes fall within sampling error. is the full MBPP+ test set.
Watermark configuration. Unless otherwise noted, we use base strengths , watermark dimensions , and detection threshold . The depth-aware schedule uses V-cache suppress at layers 0–4, at V-cache layers 24–35, and at all K-cache layers (Section 4.4). Content-bound keys (Eq. 13) are used for replay-defense experiments; all others use static watermark keys.
Attack suite. We evaluate robustness against attacks organized into six categories ( for all experiments): (1) tampering attacks: random noise scaled to per-layer cache magnitude at for V-only and K-only, and for KV-both; (2) the Out-of-Sight DiffMean attack [25] on V-only () and KV-both (); (3) forgery attacks: wrong-key verification across 500 distinct random keys; (4) deployment-relevant attacks: quantization to fp16 and int8; (5) structural attacks: replay attacks (with and without content-bound keys); (6) statistical invisibility: nine detection strategies (column-mean, variance, kurtosis, range, abs-mean, cross-cache correlation, KL divergence, head-avg variance, head-avg range) on V-cache (early and boost layer regions) and K-cache (all layers). Table 5 reports detection scores and pipeline accuracy for all tested configurations.
| Attack | Target | Acc. | |||
| No attack | — | 0.206 | 0.408 | 0.307 | 92.4% |
| Steering | V | 0.174 | 0.408 | 0.291 | 0.0% |
| Steering | V | 0.025 | 0.407 | 0.216 | 0.6% |
| Steering | K | 0.213 | 0.330 | 0.272 | 2.6% |
| Steering | K | 0.195 | 0.046 | 0.121 | 0.2% |
| Steering | KV | 0.025 | 0.047 | 0.036 | 1.0% |
| DiffMean | V | 0.189 | 0.407 | 0.298 | 74.2% |
| DiffMean | V | 0.153 | 0.407 | 0.280 | 43.2% |
| DiffMean | KV | 0.182 | 0.393 | 0.287 | 1.0% |
| Quantization fp16 | KV | 0.206 | 0.408 | 0.307 | 93.0% |
| Quantization int8 | KV | 0.190 | 0.401 | 0.296 | 93.0% |
Table 5. Detection and accuracy under representative attacks (full pipeline, , ). All 1,500/1,500 handoffs detected.
Metrics. We report: (1) TPR, FPR, and AUROC for handoff-level detection using the combined score , with individual and for channel-specific analysis; (2) per-layer TPR and FPR for tamper localization; (3) task accuracy for utility preservation; and (4) watermark dimensions identified by a keyless adversary (out of ; random chance ) for statistical invisibility.
Main Results
Utility Preservation. Table 2 reports task accuracy across four benchmarks spanning mathematical reasoning, science, code generation, and medical question answering. All changes fall within one sampling standard error (SE at , SE at ), with three benchmarks slightly above and one slightly below the clean baseline, consistent with sampling variation.
Handoff-level Detection. From the 500-sample pipeline run, we extract 1,500 independent handoff verifications (three watermarked handoffs per sample: Planner→Critic, Critic→Refiner, and Refiner→Judger). Each verification independently computes (Eq. (9)) using the embedding agent’s key (correct-key score); wrong-key scores are computed with random keys for the same caches. Table 3 summarizes detection performance. At the pipeline level, the watermark
| Scope | Metric | Value |
| Pipeline | TPR / FPR / AUROC | 100% / 0.0% / 1.0 |
| (mean ± std) | 0.3066 ± 0.0089 | |
| Per-channel | / | 0.2055 / 0.4077 |
| Per-edge | (P→C / C→R / R→J) | 0.314 / 0.311 / 0.294 |
| Wrong-key | (mean / max) | ≈ 0 / 0.0070 |
| Correct-key | (min) | 0.291 |
| Separation | min correct − max wrong | 0.284 |
Table 3. Handoff-level detection metrics (full pipeline, handoffs, ).
achieves 100% TPR with 0.0% FPR across all 1,500 handoffs. The per-edge breakdown confirms consistent detection across all three handoffs ( ranges from 0.294 at R→J to 0.314 at P→C), with no single edge significantly weaker. Wrong-key verification produces scores near zero (mean ≈ 0, max 0.0070), while the minimum correct-key score is 0.291, yielding a separation margin of 0.284 and indicating complete distributional separation between authentic and forged caches.
Layer-level Tamper Localization. To evaluate tamper localization, we apply a V-cache steering attack (noise scaled to per-layer cache magnitude, ) to layers 19–35 and measure whether the per-layer V-cache correlation correctly identifies attacked layers (). Table 4 reports results. The scheme achieves perfect localization: all 17 attacked layers show statistically significant correlation drops (paired -test comparing per-layer correlation scores between clean and attacked caches across samples; , ), while all 19 non-attacked layers show exactly zero drop (zero variance across all samples). The KV cache is append-only at the position level, so V-cache values at non-attacked layers are bitwise identical before and after the attack, producing exact zero drop rather than merely non-significant values. Since this attack targets only the V-cache, K-cache correlations remain unchanged at all layers (, single-hop), confirming that the dual-channel design correctly attributes tampering to the targeted cache component. The per-layer analysis in Table 4 validates the localization capability experimentally; in deployment, a defender would compare per-layer scores against empirically calibrated baselines or use relative outlier detection across layers.
| Layer | No attack | Under attack | Drop | |
| 0 | 0.235 | 0.235 | 0.000 | — |
| 10 | 0.177 | 0.177 | 0.000 | — |
| 18 | 0.107 | 0.107 | 0.000 | — |
| 19† | 0.136 | 0.122 | 0.015 | 21.33 |
| 24† | 0.363 | 0.329 | 0.034 | 56.43 |
| 28† | 0.297 | 0.268 | 0.030 | 45.70 |
| 35† | 0.146 | 0.132 | 0.014 | 19.77 |
| TPR (attacked) | 100% (17/17) | |||
| FPR (non-attacked) | 0% (0/19) |
Table 4. Layer-level tamper localization (, V-cache attack on layers 19–35). Values averaged across samples. †Attacked layer (19–35). All ().
Security Analysis
Tamper Robustness. Table 5 summarizes detection under blind perturbation attacks, where the adversary adds noise without knowing which dimensions carry the watermark (). The KV watermark maintains 100% detection (1,500/1,500 handoffs) under all tested attacks. The dualchannel design provides mutual redundancy against such blind attacks: a K-only perturbation at degrades from 0.408 to 0.046 but leaves largely intact at 0.195; conversely, a V-only perturbation degrades to 0.025 while remains at 0.407. Even the worst case (KV-both at ) produces , still 3.6× above the detection threshold. A stronger adversary who identifies the watermarked dimensions could surgically remove the signal from one channel without degrading output quality; statistical invisibility (Table 7) prevents this by ensuring that watermarked dimensions are indistinguishable from non-watermarked ones.
| Statistic | V-E | V-B | K |
| Column-mean | 2.0 | 2.1 | 2.0 |
| Variance | 2.8 | 2.4 | 2.3 |
| Kurtosis | 2.2 | 1.7 | 1.0 |
| Range | 2.8 | 2.2 | 2.3 |
| Abs-mean | 2.4 | 2.1 | 1.9 |
| Cross-cache corr. | 2.4 | 2.0 | 1.8 |
| KL divergence | 1.2 | 1.7 | 1.8 |
| Head-avg variance | 3.0 | 2.0 | 2.2 |
| Head-avg range | 2.6 | 1.9 | 2.0 |
| Chance | 2.0 | 2.0 | 2.0 |
Table 7. Statistical invisibility: watermark dimensions found by keyless adversary (chance = 2.0/16, =500). V-E: V-cache early layers 0–4; V-B: V-cache boost layers 24–35; K: K-cache all layers.
Attack Tradeoff: detection vs. accuracy. Blind steering attacks destroy output quality at all tested strengths: pipeline accuracy drops to even at (V-only: 0.0%, K-only: 2.6%, KV at : 1.0%), offering no attacker tradeoff—the attack degrades both quality and detection simultaneously. Only the stronger Out-of-Sight DiffMean attack [25], which requires offline access to compute directional vectors, preserves accuracy at low . Under V-only DiffMean,
the V-cache score degrades with increasing (from 0.189 at to 0.153 at , Table 5) but never drops below the detection threshold—and the K-cache channel remains completely unaffected ( at all ), providing a redundant detection signal. Under the stronger KV-both variant, both scores degrade but detection remains 1,500/1,500 at (, ). To quantify the attacker’s dilemma, we propagate the KV-both DiffMean attack through the full four-agent pipeline and measure task accuracy (, Table 6). At , accuracy collapses from 92.4% to 1.4% while the watermark remains detected (); at , the output degrades further. No operating point simultaneously evades detection and preserves output quality.
| Accuracy | ||
| 0.0 | 0.307 | 92.4% |
| 0.5 | 0.307 | 91.8% |
| 1.0 | 0.306 | 86.8% |
| 2.0 | 0.302 | 1.4% |
| 4.0 | 0.287 | 1.0% |
Table 6. DiffMean KV-both accuracy tradeoff—the only attack that preserves output quality at low (full pipeline, ).
Statistical Invisibility. We evaluate invisibility against nine statistical detection strategies on both V-cache and K-cache (). Table 7 reports the number of true watermark dimensions found (out of ) at suppressed and boosted V-cache layers and across all K-cache layers. For each statistic, the adversary ranks all 128 dimensions by that statistic’s value and selects the top 16 as suspected watermark dimensions; we report how many of these are actual watermark dimensions (chance = 16/128 × 16 = 2.0). The three-tier strength schedule ensures full invisibility across all regions and all nine statistics: no statistic exceeds 3.0/16 at any layer group, compared to the 2.0/16 random chance baseline. The V-cache suppress schedule ( at layers 0–4) prevents the higher-order leakage (variance, abs-mean) that would otherwise occur at early layers where natural cache magnitude is comparable to the perturbation. The V-cache boost (, lower than the K-cache’s ) keeps variance and abs-mean near chance at boosted layers. The K-cache scores range from 1.0 to 2.3 across all nine statistics—indistinguishable from random chance (2.0/16)—despite the 10× uniform boost, because K-cache natural magnitudes (std = 1.7–14.1) dwarf the boosted perturbation (0.30). A natural extension of the keyless adversary is to average cache values across all attention heads before computing statistics, suppressing natural per-head variation. The per-head sign patterns (Eq. (3)) defeat this strategy: without the key, the watermark cancels under head-averaging (). Table 7 confirms that head-averaged variance and range remain near chance across all layer groups.
Beyond the dimension-identification analysis above, we evaluate cache-level indistinguishability: whether a keyless adversary can detect watermark presence without identifying specific dimensions. Table 8 summarizes three complementary tests. Two-sample Kolmogorov-Smirnov and Mann-Whitney U tests, applied per-dimension across all 36 layers (=500), fail to reject the null hypothesis that clean and watermarked distributions are identical (rejection rates of 2.2% and 0.0% at =0.05, both below the expected 5% false positive rate). A logistic regression and two-layer MLP trained on per-dimension statistics (mean, variance, skewness, kurtosis; 514 features per layer) achieve 37% and 23% accuracy respectively under 5-fold cross-validation (chance = 50%), confirming that even a learned adversary cannot detect watermark presence. Per-dimension Cohen’s between clean and watermarked caches is negligible at the boosted layers ( = 0.004 for watermarked dimensions, for non-watermarked), well below the standard threshold for a small effect.
| Test | Metric | Result | Chance |
| KS test | Rejection rate | 2.2% | 5.0% |
| Mann-Whitney U | Rejection rate | 0.0% | 5.0% |
| Logistic Regression | Accuracy | 37% | 50% |
| MLP (128, 64) | Accuracy | 23% | 50% |
| Cohen’s (WM dims) | Effect size | 0.004 | < 0.2 |
| Cohen’s (non-WM) | Effect size | 0.000 | < 0.2 |
Table 8. Cache-level indistinguishability (=500, =0.05). A keyless adversary cannot detect watermark presence.
Adaptive White-box Attacks. Table 9 summarizes results across three security scenarios. All four adaptive strategies are defeated: PCA whitening, average-and-subtract, blind dimension zeroing, and watermark overwriting all fail to evade
| Category | Attack | TPR | |
| Adaptive | PCA whitening | 0.320 | 500/500 |
| Avg-and-subtract | 0.317 | 20/20 | |
| Blind dim zeroing | 0.313 | 500/500 | |
| Overwrite 33 | 0.196 | 500/500 | |
| Forgery | Wrong-key (mean / max) | 0.0001 / 0.0120 | |
| FPR | 2/500 (0.4%) | ||
| Replay | Static watermark key | — | 500/500 |
| Content-bound key | — | 1/500 |
Table 9. Security analysis: adaptive, forgery, and replay attacks (single-hop, =500, =0.01).
detection. The position-dependent sign patterns ensure that dimension-level statistics do not reveal which dimensions carry the watermark, and the dual KV channels provide independent backup.
Forgery Resistance. Wrong-key detection across 500 distinct random keys produces scores statistically indistinguishable from zero (Table 9). The separation between correct-key and wrong-key scores is 0.294, confirming that forgery without the secret key is infeasible. The wrong-key maximum here (0.0120) is higher than in Table 3 (0.0070) because this test uses shorter single-hop caches; longer pipeline caches reduce wrong-key variance.
Replay Defense. With static watermark keys, replayed caches pass verification 100% of the time, as expected. Content-bound keys (Eq. (12)) reduce this to 0.2% (Table 9), while preserving legitimate verification. The residual pass is a threshold-edge case consistent with the wrong-key FPR observed in the forgery test.
Cascade Verification. The KV cache is append-only, so an upstream agent’s watermark survives at any later pipeline stage. This enables cascade verification: the pipeline operator checks an upstream watermark at a downstream point. This detects a compromised agent that ignores its input, generates fabricated content, and embeds a valid watermark under its own key—a scenario invisible to per-handoff verification. Table 10 reports results at =500 for two compromised-agent positions. In all compromised scenarios, the upstream agents’ watermarks are absent (TPR 0), while the compromised agent’s own watermark is present (500/500). The detection rule is straightforward: if an upstream watermark is absent at a downstream point, an intermediate agent must have discarded the input. When the compromised agent re-watermarks, its own watermark identifies which agent replaced the content.
| Scenario | Check for | TPR | |
| Normal pipeline | Planner | 500/500 | 0.317 |
| Critic ignores input | Planner | 3/500 | |
| Critic re-watermarks | Planner | 3/500 | |
| Critic | 500/500 | 0.338 | |
| Refiner ignores input | Planner | 5/500 | |
| Critic | 0/500 | ||
| Refiner re-watermarks | Planner | 2/500 | |
| Critic | 0/500 | ||
| Refiner | 500/500 | 0.315 |
Table 10. Cascade verification (single-step generation, 4-agent cascade, , ). “Check for” indicates which upstream watermark is verified at Judger’s cache.
Ablation Studies
Depth-aware Strength Schedule. Under truly uniform strength , V-cache correlation decays 45× across layers (Figure 3, gray line), driven by the 144× growth in V-cache magnitude (bottom panel). The three-tier schedule (blue line) resolves this: suppress (0.1×) at layers 0–4 reduces early-layer correlation from 0.93 to 0.24, preventing statistical leakage while staying above threshold; boost (5×) at layers 24–35 raises late-layer correlation from 0.05 to 0.23, restoring localization capability. K-cache requires uniform 10× boost across all layers (mean : 0.050 → 0.433) due to its uniformly high magnitude (see Appendix § 10 for full per-layer profiles).

Figure 3. Top: V-cache detection correlation across 36 layers under uniform strength (gray, 45× decay) vs. three-tier schedule (blue, stable above ). In the suppress region (layers 0–4), the schedule intentionally reduces correlation to ensure statistical invisibility. Bottom: V-cache magnitude (std) per layer; the dashed line shows the base perturbation . See Appendix § 10 for full per-layer data.
Watermark Dimensions. Table 11 reports the effect of watermark dimensionality (). Increasing beyond 16 does not improve detection and reduces K-cache correlation, confirming that depth decay is caused by signal magnitude, not signal width. The combined score peaks at due to higher per-dimension signal concentration, but provides a better security-detection tradeoff with a 2.0/16 chance baseline.
| 8 | 0.201 | 0.469 | 0.335 |
| 16 | 0.201 | 0.433 | 0.317 |
| 32 | 0.201 | 0.391 | 0.296 |
| 64 | 0.202 | 0.383 | 0.292 |
Table 11. Watermark dimensions (single-hop, ).
Comparison with Prior Work
Text-generation watermarks [8, 15] operate on token logits during decoding, and agent behavioral watermarks [1, 12, 26] operate on discrete action sequences—neither produces a signal in continuous KV-cache tensors, making direct quantitative comparison infeasible. The closest concurrent work is by Brito and Baquero [3], who study KV-cache integrity in role-sequenced multi-agent systems using HMAC-SHA256 transport manifests and magnitude quarantine. Table 12 compares the two approaches.
| Property | KVMARK | B&B |
| Post-handoff substitution | ✓ | ✓ |
| Endpoint-signed malicious KV | ✓† | ✗ |
| Norm-matched adaptive attack | ✓ | ✗ |
| Continuous tamper degree | ✓ | ✗ |
Table 12. Comparison with Brito and Baquero [3].
The HMAC approach achieves perfect binary detection for post-handoff substitution (774/774 honest, 295/295 tampered) but explicitly cannot detect compromised endpoints that sign malicious KV in-band, and their magnitude quarantine is defeated by adaptive norm-matched attacks (0/65 detected). Our KVMARK addresses both scenarios: the keyed statistical signal is embedded in KV-cache values, so post-handoff substitution disrupts the correlation, and cascade verification detects endpoints that discard upstream input while re-embedding under their own key. The two mechanisms are complementary: HMAC verifies transport provenance, while our watermark verifies content integrity.
Model Generalization
To evaluate whether the watermark generalizes beyond the primary evaluation model, we apply identical hyperparameters (no model-specific tuning) to Qwen3-8B within the same LatentMAS pipeline and benchmark (GSM8K, ). Table 13 confirms that the watermark transfers without modification: detection scores () and TPR (1500/1500) are statistically indistinguishable from the 4B results. The watermark operates directly on KV-cache tensors without accessing model weights, tokenizer, or architecture-specific logic; any model producing cache tensors is compatible. Cross-architecture evaluation (e.g., Llama, Mistral [10]) requires pipeline adaptation, not watermark modification, and is left to future work.
| Model | Clean | KV-WM | TPR | |
| Qwen3-4B | 92.0% | 92.4% | 0.307 | 1500/1500 |
| Qwen3-8B | 93.4% | 94.0% | 0.312 | 1500/1500 |
Table 13. Generalization with zero re-tuning (full pipeline).
Computational Overhead
The watermark adds a constant perturbation to 16 of 128 dimensions per position per layer in both V-cache and K-cache. Per-head sign patterns require generating independent sign patterns per layer (one per attention head) during both embedding and detection. On an NVIDIA RTX 5090 GPU, embedding takes ms and detection takes ms per handoff (measured at sequence length 14; both operations scale linearly with sequence length). The overhead is dominated by per-head sign pattern generation in Python and could be substantially reduced through batched GPU computation.
Discussion and Future Work
Limitations. While KVMARK establishes a robust, training-free defense for continuous Key-Value (KV) cache communication, several practical constraints highlight important directions for future research. First, our framework currently assumes homogeneous pipeline architectures in which all participating agents share identical structural parameters (e.g., layer depth , head count , and head dimension ). In heterogeneous multi-agent systems involving diverse model families or pruned variants, direct latent handoffs require intermediate linear projections or cross-attention alignments. Adapting fragile watermarking across heterogeneous model boundaries without destroying embedded signal patterns remains an open challenge. Second, because sign patterns are anchored to absolute token position indices, KVMARK inherently assumes stable sequence alignment. However, operations such as dynamic KV-cache compression, sliding-window attention, or token eviction disrupt sequence indexing and trigger false-positive tamper alerts. Third, the three-tier depth-aware strength schedule relies on parameters tuned to the layer-wise magnitude growth profile of the Qwen3 architecture. While this profile generalizes seamlessly across different parameter scales (e.g., Qwen3-4B to Qwen3-8B), deploying KVMARK on architecturally distinct model families with alternative normalization schemes (e.g., LayerNorm and RMSNorm [10]) requires an initial calibration step. Fortunately, this magnitude calibration is non-resource-intensive, requiring only a few forward passes on representative inputs. Finally, fragile watermarking guarantees structural and contextual transport integrity; it inherently operates at the tensor layer rather than the semantic layer. Consequently, while KVMARK reliably isolates in-transit tampering, man-in-the-middle modifications, and context-dropping, it cannot detect semantic or behavioral steering. A fully compromised in-band agent that reads its inputs, correctly embeds a valid watermark, but intentionally generates biased or adversarial latent content remains out of scope for transport-layer verification and requires complementary behavioral attribution mechanisms [12].
Future Work. Building on these insights, our future work includes generalizing KVMARK to heterogeneous pipeline topologies (e.g., dynamic agent selection and branching DAGs [30]) and developing position-agnostic or relative-index watermarking schemes resilient to dynamic token eviction. Furthermore, combining transport integrity verification with semantic auditing and behavioral attribution [12] offers a promising solution for end-to-end security in autonomous multi-agent ecosystems. Finally, extending fragile watermarking to alternative continuous latent communication substrates, such as compressed hidden states [6], presents a compelling direction for future investigation.
Conclusion
We presented the first watermarking scheme for KV-cache integrity verification in latent multi-agent systems. The scheme embeds a fragile, keyed, zero-bit watermark into both V-cache and K-cache, enabling the verification module to verify authenticity before processing. Comprehensive evaluation of Qwen3-4B across four benchmarks and Qwen3-8B on GSM8K demonstrates 100% detection at handoffs, 100% layer-level localization, no statistically significant degradation in accuracy, full statistical invisibility, and robustness to all tested attacks. The watermark generalizes across model scales with zero re-tuning, establishing that watermark-based integrity verification is both feasible and effective for securing latent multi-agent communication.
Open Science
All source code, experiment scripts, and result files are included in the supplementary material and will be made publicly available upon acceptance. The repository includes: (1) the core watermark implementation (embedding, detection, key derivation); (2) runnable experiment scripts reproducing every table and figure in the paper, with parser scripts that extract table values from raw logs; and (3) original result files from the experimental runs. The experiments use publicly available models (Qwen3-4B and Qwen3-8B from HuggingFace) and datasets (GSM8K, ARC-Challenge, MBPP+, MedQA), requiring a single NVIDIA GPU with GB memory. A README maps each paper table to its corresponding script and result file.
Ethical Considerations
This work proposes a defensive mechanism for verifying KV-cache integrity in multi-agent systems. The watermark is fragile by design: it detects tampering but cannot be used for persistent tracking or surveillance, as any cache modification destroys the signal. Our threat model assumes the watermarking system is operated by a trusted pipeline defender with legitimate authority over the agents, not deployed covertly. The technique does not involve human subjects, personal data, or deception. We acknowledge that watermark detection thresholds involve a precision–recall tradeoff; operators should calibrate to their deployment’s tolerance for false positives. All experiments use publicly available models and datasets; no proprietary or restricted data was used.
Depth-Dependent Signal Decay
This section provides per-layer analysis motivating the three-tier strength schedule (§ 4.4).
Cache magnitude profiles. Table 14 reports the standard deviation of V-cache and K-cache values across all 36 layers of Qwen3-4B (). V-cache magnitude spans a 144× range (0.033 at layer 0 to 4.757 at layer 33), while K-cache magnitude varies only 8× (1.663 to 14.076). This asymmetry explains why V-cache requires a layer-dependent schedule while K-cache can use a uniform boost.
| Layer | ||||||
| 0 | 0.033 | 14.08 | 0.926 | 0.069 | 0.237 | 0.542 |
| 2 | 0.074 | 1.66 | 0.734 | 0.062 | 0.100 | 0.517 |
| 5 | 0.149 | 4.73 | 0.484 | 0.006 | 0.484 | 0.079 |
| 10 | 0.474 | 2.60 | 0.177 | 0.074 | 0.177 | 0.587 |
| 15 | 0.516 | 1.95 | 0.163 | 0.043 | 0.163 | 0.393 |
| 20 | 0.606 | 1.95 | 0.139 | 0.045 | 0.139 | 0.391 |
| 24 | 1.087 | 1.95 | 0.085 | 0.045 | 0.366 | 0.406 |
| 28 | 1.406 | 1.82 | 0.063 | 0.054 | 0.296 | 0.470 |
| 32 | 3.163 | 1.85 | 0.025 | 0.044 | 0.132 | 0.384 |
| 35 | 2.965 | 2.82 | 0.031 | 0.037 | 0.146 | 0.404 |
Table 14. Per-layer cache magnitude (std) and detection correlation (, from Eq. (7)–(8); higher = stronger detection) under uniform strength vs. three-tier schedule. .
Signal decay under uniform strength. Under uniform perturbation (no boost, no suppress), V-cache correlation decays 45× from layer 0 () to layer 34 ($c_{\mathrm{V}}=0.020). The decay follows directly from the signal-to-magnitude ratio: at layer 0, the perturbation (0.03) equals the natural V-cache standard deviation (0.033), producing near-perfect correlation; at layer 33, the same perturbation is smaller than the cache magnitude (4.757), producing negligible correlation. K-cache correlation is uniformly low under uniform strength (mean ) because K-cache magnitudes (1.7–14.1) dominate the perturbation at every layer.
Three-tier schedule effect. The three-tier schedule resolves both extremes. At early layers (0–4), suppress (0.1×) reduces the perturbation from 0.03 to 0.003, lowering from 0.926 to 0.237—still well above threshold but no longer producing the variance and abs-mean leakage that a 0.926 correlation implies. At late layers (24–35), boost (5×) raises the perturbation from 0.03 to 0.15, increasing from ∼0.05 to ∼0.23, restoring detection capability. The K-cache uniform 10× boost raises mean from 0.050 to 0.433, providing a strong independent detection channel at every layer.
References
- [1]Hye-seon An, Shinwoo Park, Dongsu Kim, and Yo-Sub Han. Sequential behavioral watermarking for LLM agents. arXiv preprint arXiv:2605.11036, 2026.arxiv.org/abs/2605.11036
- [2]Sadia Asif, Mohammad Mohammadi Amiri, Momin Abbas, Prasanna Sattigeri, and Karthikeyan Natesan Ramamurthy. Latent communication guard for safe KV sharing in multi-agent systems. arXiv preprint arXiv:2605.22786, 2026.arxiv.org/abs/2605.22786
- [3]Luís Brito and Carlos Baquero. When latent agents lie: KV-cache integrity in multi-agent LLM collaboration. arXiv preprint arXiv:2606.28958, 2026.arxiv.org/abs/2606.28958
- [4]Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018.arxiv.org/abs/1803.05457
- [5]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021.arxiv.org/abs/2110.14168
- [6]Zhili Du, Rui Wang, Haoran Bai, Zhi Cao, Xiaolong Zhu, Yu Cheng, Bo Zheng, Wei Chen, and Haochao Ying. Enabling agents to communicate entirely in latent space. arXiv preprint arXiv:2511.09149, 2025.arxiv.org/abs/2511.09149
- [7]others. A survey on trustworthy LLM agents: Threats and countermeasures. arXiv preprint arXiv:2503.09648, 2025.
- [8]Pierre Fernandez, Antoine Chaffin, Karim Tit, Vivien Chappelier, and Teddy Furon. Three bricks to consolidate watermarks for large language models. In 2023 IEEE International Workshop on Information Forensics and Security (WIFS), 2023.arxiv.org/abs/2308.00113
- [9]Zheng Gao, Xiaoyu Li, Xiaoyan Feng, Jiaojiao Jiang, Yang Song, Yulei Sui, Zhenchang Xing, and Liming Zhu. TRACE: A two-channel robust attribution watermark via complementary embeddings for LLM-agent trajectories. arXiv preprint arXiv:2607.08400, 2026.arxiv.org/abs/2607.08400
- [10]Akshat Gupta, Atahan Ozdemir, Caoqinwei Gong, and Gopala Anumanchipalli. Geometric interpretation of layer normalization and a comparative analysis with rmsnorm. In Findings of the Association for Computational Linguistics: EACL 2026, pages 375–407, 2026.arxiv.org/abs/2409.12951
- [11]Shibo Hao et al. Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769, 2024.arxiv.org/abs/2412.06769
- [12]Kaibo Huang, Jin Tan, Yukun Wei, Wanling Li, Zipei Zhang, Hui Tian, Zhongliang Yang, and Linna Zhou. Agentmark: Utility-preserving behavioral watermarking for agents. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2026. arXiv:2601.03294.arxiv.org/abs/2601.03294
- [13]Kaibo Huang, Zipei Zhang, Zhongliang Yang, and Linna Zhou. Agent guide: A simple agent behavioral watermarking framework. arXiv preprint arXiv:2504.05871, 2025.arxiv.org/abs/2504.05871
- [14]Di Jin, Eileen Pan, Nassim Oufattolle, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 2021.arxiv.org/abs/2009.13081
- [15]John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In Proceedings of the 40th International Conference on Machine Learning (ICML), 2023.arxiv.org/abs/2301.10226
- [16]Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Robust distortion-free watermarks for language models. Transactions on Machine Learning Research, 2024.arxiv.org/abs/2307.15593
- [17]Yiping Li, Zhiyu An, and Wan Du. When less latent leads to better relay: Information-preserving compression for latent multi-agent LLM collaboration. arXiv preprint arXiv:2604.13349, 2026.arxiv.org/abs/2604.13349
- [18]Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. NeurIPS, 2023.arxiv.org/abs/2305.01210
- [19]Wenlong Meng, Chen Gong, Terry Yue Zhuo, Fan Zhang, Kecen Li, Zheng Liu, Zhou Yang, Chengkun Wei, and Wenzhi Chen. Watermarking LLM agent trajectories. arXiv preprint arXiv:2602.18700, 2026.arxiv.org/abs/2602.18700
- [20]Qi Pang, Shengyuan Hu, Wenting Zheng, and Virginia Smith. No free lunch in LLM watermarking: Trade-offs in watermarking design choices. Advances in Neural Information Processing Systems, 37, 2024.arxiv.org/abs/2402.16187
- [21]Julien Piet, Chawin Sitawarin, Vivian Fang, Norman Mu, and David Wagner. MarkMyWords: Analyzing and evaluating language model watermarks. In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pages 68–91, 2025.arxiv.org/abs/2312.00273
- [22]Qwen Team. Qwen3 technical report. 2025. https://qwenlm.github.io/blog/qwen3/.
- [23]Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Tool-former: Language models can teach themselves to use tools. 2023.
- [24]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 30, 2017.arxiv.org/abs/1706.03762
- [25]Chenxi Wang, Ruiyang Huang, Jiayan Sun, Lei Wei, and Yifan Wu. Out of sight, not out of mind: Unveiling latent attack in latent-based multi-agent systems. arXiv preprint arXiv:2605.28214, 2026.arxiv.org/abs/2605.28214
- [26]Liwen Wang, Zongjie Li, Yuchong Xie, Shuai Wang, Dongdong She, Wei Wang, and Juergen Rahmel. On protecting agentic systems’ intellectual property via watermarking. arXiv preprint arXiv:2602.08401, 2026.arxiv.org/abs/2602.08401
- [27]Shilong Wang, Guibin Zhang, Miao Yu, Guancheng Wan, Fanci Meng, Chongye Guo, Kun Wang, and Yang Wang. G-safeguard: A topology-guided security lens and treatment on LLM-based multi-agent systems. arXiv preprint arXiv:2502.1127, 2025.arxiv.org/abs/2502.11127
- [28]Yifei Wang, Dizhan Xue, Shengjie Zhang, and Shengsheng Qian. BadAgent: Inserting and activating backdoor attacks in LLM agents. arXiv preprint arXiv:2406.03007, 2024.arxiv.org/abs/2406.03007
- [29]Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al. AutoGen: Enabling next-gen LLM applications via multi-agent conversation. 2024.arxiv.org/abs/2308.08155
- [30]Yingxuan Yang, Huacan Chai, Shuai Shao, Yuanyi Song, Siyuan Qi, Renting Rui, and Weinan Zhang. Agentnet: Decentralized evolutionary coordination for llm-based multi-agent systems. Advances in Neural Information Processing Systems, 38:107309–107336, 2026.arxiv.org/abs/2504.00587
- [31]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2210.03629
- [32]Xuandong Zhao, Prabhanjan Vijendra Ananth, Lei Li, and Yu-Xiang Wang. Provable robust watermarking for AI-generated text. 2024.arxiv.org/abs/2306.17439
- [33]Jiaru Zou, Xiyuan Yang, Ruizhong Qiu, Gaotang Li, Katherine Tieu, Pan Lu, Ke Shen, Hanghang Tong, Yejin Choi, Jingrui He, James Zou, Mengdi Wang, and Ling Yang. Latent collaboration in multi-agent systems. In Proceedings of the 43rd International Conference on Machine Learning (ICML), 2026. Spotlight. arXiv:2511.20639.arxiv.org/abs/2511.20639