Selective and Hybrid-State Communication for Latent Multi-Agent Systems

Introduction

Large language model (LLM) multi-agent systems improve complex reasoning by allowing multiple agents to iteratively exchange and refine information [4, 9]. Most systems communicate through natural language, requiring each agent to decode its internal state into text and the next agent to tokenize and process that text again [23, 18, 4]. Latent communication instead transfers internal representations directly, reducing this decode–re-encode overhead [14, 5, 15, 25].

However, latent communication introduces a new systems question: what internal state should be transmitted between agents? Existing approaches largely rely on transferring KV-cache state [25, 16, 22], which can contain redundant information as trajectories grow. Prior KV-cache compression methods show that only a subset of cached states may be important for future decoding [13, 24, 19], but these methods primarily target single-model inference rather than inter-agent state transfer. At the same time, emerging hybrid linear-attention architectures such as Qwen3.5 combine full-attention KV caches with complete Gated DeltaNet (GDN) inference states [21, 17], so a KV-only handoff is no longer sufficient.

We study two complementary, independently evaluated approaches to latent-state communication. SelectKV selects newly communicable KV states while preserving inherited context. **HyLaMAS extends LatentMAS to hybrid architectures by transferring full-attention KV caches together with the recurrent and causal-convolution states of Gated DeltaNet layers.

Together, SelectKV and HyLaMAS study efficient latent-state handoff from two complementary perspectives: selecting the information that should be communicated and supporting the heterogeneous states required by modern model architectures. Across reasoning, knowledge, and code-generation benchmarks, our results show that latent communication can be made more selective while largely preserving downstream performance, and that hybrid-state transfer can substantially reduce the overhead of text-based multi-agent communication.

Key Contributions:

• Selective inter-agent KV communication. We introduce SelectKV, a boundary-aware KV selection mechanism for latent multi-agent handoff that preserves inherited context while selecting newly generated KV states. Ablations examine boundary protection and matched-budget selection baselines.

• Hybrid-state latent communication. We introduce HyLaMAS, a shared latent workspace for hybrid linear-attention models that transfers both full-attention KV caches and recurrent linear-attention states between agents.

• Multi-benchmark evaluation. Across six reasoning, knowledge, and code-generation benchmarks, SelectKV reduces KV communication while largely preserving full-cache LatentMAS performance, while HyLaMAS reduces generated-token and wall-clock overhead relative to text-based multi-agent communication.

Background and Related Work

Multi-Agent Systems and Latent Communication

LLM-based multi-agent systems commonly coordinate through natural-language messages, requiring intermediate reasoning to be decoded into text and subsequently re-encoded and processed by other agents [23, 18, 4]. Latent communication instead enables agents to exchange internal representations directly. In particular, LatentMAS transfers latent representations and KV-cache state across agents, allowing a receiving agent to continue from a preceding reasoning trajectory without relying exclusively on textual messages [25].

For a Transformer with $L$ layers, the KV cache can be written as

where $K^{(l)}$ and $V^{(l)}$ denote the cached key and value states at layer $l$. Because the cache grows linearly with sequence length, transferring the full accumulated state between agents can become increasingly costly as multi-agent reasoning trajectories grow. Recent work has therefore explored selective, compressed, or reusable KV communication across agents [16, 22, 8].

Redundancy in KV Caches

Prior work on efficient Transformer inference has shown that cached positions contribute unequally to subsequent computation. $H_2O$ identifies attention “heavy hitters” and preferentially retains frequently attended positions [24]; Scissorhands exploits the persistence of token importance across generation [13]; and StreamingLLM demonstrates the importance of preserving initial attention-sink tokens during long-context inference [19]. Together, these findings suggest substantial redundancy in accumulated KV state and motivate selective retention rather than uniform preservation of all cached positions.

Related work extends this principle from single-model inference to inter-agent communication. KVComm studies selective KV sharing between LLM agents [16], while cross-context KV-cache communication enables cached context to be reused across agents [22]. Orthogonal BackFill (OBF) compresses latent multi-agent communication while reconstructing information from discarded states [8]. $k$NN-latentMAS ranks cached states using semantic similarity to downstream context and retains the most relevant KV entries [12]. These approaches establish KV selection and compression as mechanisms for reducing latent communication while preserving task-relevant state.

Emerging Architectures Beyond Standard Attention

Latent communication methods have primarily been developed for standard attention-based Transformers, where an agent’s accumulated reasoning state can be represented through a conventional KV cache [25, 5, 16, 22]. Recent language-model architectures, however, increasingly incorporate recurrent, state-space, or linear-attention mechanisms whose inference state differs from the KV cache of softmax attention [6, 3, 20, 21].

Some architectures combine recurrent or linear-attention layers with conventional attention layers [10, 17], while others replace standard attention more extensively [6, 26]. Their inference state may therefore include recurrent states in addition to, or instead of, conventional key–value caches. As these architectures become more prevalent, latent multi-agent communication must account for internal state representations that cannot be described solely by Transformer KV caches.

Our Proposed Method

We formulate efficient latent collaboration as the problem of determining what internal state should cross an agent boundary. We address two complementary aspects of this problem. SelectKV reduces communication in Transformer-based latent multi-agent systems by selectively retaining newly generated KV states at each agent boundary. HyLaMAS extends latent collaboration to hybrid architectures whose inference state contains both conventional attention KV caches and recurrent linear-attention states.

SelectKV: Boundary-Aware KV Selection

Consider the Transformer KV cache $C$ defined in § 2.1 at an inter-agent handoff.

Suppose the cache contains $T$ positions at the current handoff, of which the first $T_{\mathrm{prev}}$ were inherited from preceding agents. We partition the cache into

SelectKV protects $I_{\mathrm{prev}}$ from repeated compression and selects positions from $I_{\mathrm{new}}$ using relevance and local KV novelty.

Relevance Scoring

Motivated by similarity-based KV retrieval [12], let $\bar{\mathbf{k}}_i$ be the layer- and head-averaged key at position $i$, and $\bar{\mathbf{q}}$ a sender-derived relevance query in the same space. We score candidate positions by cosine similarity:

The query $\bar{\mathbf{q}}$ is the sending agent’s final hidden state after latent rollout, reduced to the key-head dimension by averaging equal-sized chunks (or truncating if needed). Keys are averaged across heads and layers; both representations are $\ell_2$-normalized. The query is available before handoff, without a receiving-agent forward pass. Higher scores indicate greater alignment with this sender-derived proxy for downstream relevance.

KV-Novelty Scoring

To favor distinct information, we score local KV novelty across the full cache of $T$ positions:

Only positions in $\mathcal{I}_{\mathrm{new}}$ are eligible for selection. Interior scores compare both neighbors, including the inherited boundary neighbor when $i = T_{\mathrm{prev}} + 1$; endpoints use one neighbor. Higher scores indicate less redundancy. Unlike attention-based methods [24, 13], this score requires no explicit attention extraction.

Boundary-Aware Selection

Let $N = |\mathcal{I}_{\mathrm{new}}|$ and let $r \in(0, 1]$ be the target retention ratio. The new-region selection budget is

SelectKV prioritizes the overlap of the descending relevance and novelty rankings, then alternates between their remaining top positions. A small set of recent positions is protected within the budget [19]. For the selected set $S_r \subseteq\mathcal{I}_{\mathrm{new}}$, $|S_r| \leq n$, the transmitted positions are

Both keys and values are retained at every layer for these positions. We use $r = 0.90$, chosen from preliminary retention experiments.

HyLaMAS: Hybrid-State Latent Communication

Whereas SelectKV reduces the amount of KV state communicated between agents, HyLaMAS addresses a different challenge: enabling latent communication when the model’s inference state is not represented solely by a KV cache. Existing latent multi-agent communication methods such as LatentMAS primarily operate over Transformer latent states and KV caches [25]. Hybrid architectures, however, may maintain fundamentally different inference states across different layer types. Qwen3.5, for example, interleaves Gated DeltaNet (GDN) linear-attention layers with conventional full-attention layers [17]. A KV-only handoff therefore cannot preserve the complete inference state of such a model.

We introduce HyLaMAS, which communicates the layer-native inference states across agent boundaries through a shared in-memory cache. Rather than projecting one KV cache into another [5], HyLaMAS preserves the appropriate state representation for each layer type.

Let $\mathcal{A}$ and $\mathcal{G}$ denote the sets of full-attention and GDN layer indices, respectively. At agent boundary $i$, we define the shared hybrid inference state as

where $i$ indexes an agent boundary and $\ell$ indexes a model layer.

For a full-attention layer $\ell\in\mathcal{A}$, $K_i^{(\ell)}$ and $V_i^{(\ell)}$ contain the cached key and value projections accumulated before boundary $i$. For a GDN layer $\ell\in\mathcal{G}$, $S_i^{(\ell)}$ denotes the recurrent matrix state summarizing preceding computation, while $C_i^{(\ell)}$ contains the recent projected inputs required by the layer’s causal convolution [21].

The state components have shapes

where $B$ is the batch size, $T_i$ is the number of cached positions at boundary $i$, $H_{\mathrm{kv}}$ is the number of key/value heads in a full-attention layer, and $d_{\mathrm{attn}}$ is the corresponding head dimension. For GDN layers, $H_v$ is the number of value heads, $d_k$ and $d_v$ are the key and value head dimensions, $d_k^{\mathrm{tot}} = H_kd_k$, $d_v^{\mathrm{tot}} = H_vd_v$, and $\kappa$ denotes the causal-convolution kernel width.

Given the shared state $\mathcal{H}_i$, the next agent continues computation from that state and produces an updated state

where $x_{i+1}$ denotes the next agent’s prompt and $F_{i+1}$ denotes its model computation. Consequently, both recurrent linear-attention state and conventional attention KV state persist across agent boundaries rather than being reconstructed from decoded text.

Following the LatentMAS interaction structure, each non-Judger agent appends its role-conditioned prompt and performs $m$ latent rollout steps without decoding an intermediate textual message. The next agent continues from the shared hybrid state. The Judger receives the final accumulated state, appends its prompt, and directly decodes the task response. Padding-aware position and attention masks maintain independence among examples when batched.

This formulation generalizes latent collaboration from Transformer-only KV communication to architectures with heterogeneous inference states. In particular, the recurrent GDN component has fixed size with respect to sequence length, whereas the full-attention KV component continues to grow with the number of cached positions. We analyze the communication implications of these two state channels in Appendix § 7.

Evaluations

Experimental Setup

Following the LatentMAS evaluation method [25], our two methods, SelectKV and HyLaMAS, were evaluated across six benchmarks that can be categorized into three groups: (i) Math and Science: GSM8K [2] and MedQA [7]; (ii) Commonsense Reasoning: ARC-Easy [1] and ARC-Challenge [1]; (iii) Code Generation: MBPP-Plus [11] and HumanEval-Plus [11].

We evaluate 100 examples per benchmark using greedy decoding. For accuracy, we report 95% percentile bootstrap confidence intervals computed using 10,000 resamples over the $n = 100$ evaluation examples. The latent protocols follow a sequential Planner–Critic–Refiner–Judger chain, with only the Judger decoding text. Model-specific rollout budgets and generation caps are specified below. Hardware, software, realignment, and measurement details are provided in Appendix § 7.§ 2.

Table 1: Sequential multi-agent results on Qwen3-4B with 10 latent reasoning steps and $n = 100$ examples per benchmark. LM denotes uncompressed LatentMAS; SKV-90 denotes boundary-aware SelectKV with a 90% retention ratio applied to the newly communicable KV region while preserving inherited context. Metrics report accuracy (%), generated output tokens, communicated KV positions, logical KV payload (MiB), and inference time (s), each averaged per example. Accuracy is reported with 95% percentile bootstrap confidence intervals in brackets, computed using 10,000 resamples over evaluation examples. $\Delta$ reports the accuracy difference in percentage points, relative change for Token, KV Pos., and Payload, and the LM/SKV-90 inference-time ratio. Green shading indicates values favorable to SKV-90.

Metric	ARC-E			ARC-C			GSM8K
	LM	SKV-90	$\Delta$	LM	SKV-90	$\Delta$	LM	SKV-90	$\Delta$
Acc.	99.0 [97.0, 100.0]	99.0 [97.0, 100.0]	0.0	90.0 [84.0, 95.0]	90.0 [84.0, 95.0]	0.0	94.0 [89.0, 98.0]	93.0 [88.0, 98.0]	↓ 1.0
Token	461.89	429.23	↓ 7.1%	561.03	515.62	↓ 8.1%	622.16	626.78	↑ 0.7%
KV Pos.	1023.04	923.42	↓ 9.7%	1099.60	992.28	↓ 9.8%	1068.46	964.29	↓ 9.7%
Payload	143.87	129.86	↓ 9.7%	154.63	139.54	↓ 9.8%	150.25	135.60	↓ 9.7%
Time	16.31	14.83	×1.10	19.33	17.81	×1.09	21.31	21.87	×0.97
Metric	MedQA			MBPP+			HumanEval+
	LM	SKV-90	$\Delta$	LM	SKV-90	$\Delta$	LM	SKV-90	$\Delta$
Acc.	63.0 [53.0, 72.0]	62.0 [52.0, 71.0]	↓ 1.0	72.0 [63.0, 81.0]	71.0 [62.0, 80.0]	↓ 1.0	89.0 [83.0, 95.0]	83.0 [75.0, 90.0]	↓ 6.0
Token	1212.50	961.38	↓ 20.7%	1527.42	1436.22	↓ 6.0%	1645.08	1761.82	↑ 7.1%
KV Pos.	2103.52	1895.90	↓ 9.9%	1545.22	1393.25	↓ 9.8%	1589.56	1433.45	↓ 9.8%
Payload	295.81	266.61	↓ 9.9%	217.30	195.93	↓ 9.8%	223.53	201.58	↓ 9.8%
Time	40.56	31.36	×1.29	51.33	46.61	×1.10	56.09	56.86	×0.99

SelectKV Compression

We compare boundary-aware SelectKV-90 (90% retention of newly communicable KV positions) against uncompressed LatentMAS on Qwen3-4B with 10 latent steps, greedy decoding, batch size 1, and seed 42. Inference time is total experiment wall-clock time per example, including agent inference, query processing, scoring, ranking, and cache pruning, but not isolated inter-device transfer latency.

Table 1 shows 9.7–9.9% reductions in communicated KV positions and logical payload across all six benchmarks. Accuracy is unchanged on ARC-Easy and ARC-Challenge, one point lower on GSM8K, MedQA, and MBPP+, and six points lower on HumanEval+ (89% to 83%) despite a 9.8% payload reduction. Thus, the fixed retention policy has task-dependent accuracy costs. Inference is faster on four benchmarks, with little change on GSM8K and HumanEval+. At $n = 100$, bootstrap intervals are wide and overlap across methods.

HyLaMAS - Hybrid-State LatentMAS

We evaluate HyLaMAS against the six benchmarks outlined above in order to determine whether hybrid-state latent collaboration reduces decoded tokens and wall-clock time relative to text-based multi-agent systems. We used Qwen3.5-2B, Qwen3.5-4B, and Qwen3.5-9B [17]. For the comparison, we evaluate HyLaMAS against: (i) Single LLM Agents, where one LLM strictly performs standard autoregressive generation with token-level decoding, and (ii) Sequential text-based MAS (TextMAS), following the chain-of-agents design [23].

For each benchmark, 40 latent steps were generated per non-Judger agent during reasoning. Additionally, in order to ensure reproducibility and fairness, greedy decoding was used (Temperature = 0). For each benchmark, different token caps were used: For ARC-Easy, ARC-Challenge, and GSM8K,

Table 2: Sequential multi-agent results on Qwen3.5-4B and Qwen3.5-9B (Gated DeltaNet layers interleaved with periodic full-attention layers), $n$=100 problems per benchmark. Single: one-agent inference, shown for context. TextMAS: agents exchange natural-language messages. HyLaMAS: agents exchange the KV cache of full-attention layers and the recurrent and causal-convolution states of Gated DeltaNet layers via the shared workspace. Metrics are accuracy (%), output tokens per problem, and total inference time (s); lower is better for the latter two. Improve is HyLaMAS vs. TextMAS: accuracy in absolute points, token reduction (%), speed-up (TextMAS time / HyLaMAS time). Shaded Improve cells mark a change in the favorable direction.

Tasks	Metrics	Qwen3.5-4B				Qwen3.5-9B
		Single	TextMAS	HyLaMAS	Improve	Single	TextMAS	HyLaMAS	Improve
ARC-E	Acc.	100.0	100.0	99.0	$\downarrow1.0$	100.0	98.0	99.0	$\uparrow1.0$
	Token	361	1138	361	$\downarrow68.3\%$	386	1350	356	$\downarrow73.6\%$
	Time (s)	464	1294	846	$\times1.53$	426	1269	618	$\times2.05$
ARC-C	Acc.	94.0	92.0	91.0	$\downarrow1.0$	94.0	95.0	93.0	$\downarrow2.0$
	Token	436	1297	481	$\downarrow62.9\%$	465	1524	430	$\downarrow71.8\%$
	Time (s)	666	1881	1205	$\times1.56$	572	1588	739	$\times2.15$
GSM8K	Acc.	92.0	85.0	93.0	$\uparrow8.0$	96.0	90.0	97.0	$\uparrow7.0$
	Token	393	898	286	$\downarrow68.1\%$	368	1222	354	$\downarrow71.0\%$
	Time (s)	893	1361	846	$\times1.61$	476	1341	656	$\times2.04$
MedQA	Acc.	80.0	81.0	77.0	$\downarrow4.0$	89.0	88.0	88.0	0.0
	Token	1064	2139	907	$\downarrow57.6\%$	1139	2740	966	$\downarrow64.7\%$
	Time (s)	2061	3277	2815	$\times1.16$	1779	3878	2437	$\times1.59$
MBPP+	Acc.	73.0	75.0	73.0	$\downarrow2.0$	85.0	79.0	76.0	$\downarrow3.0$
	Token	442	1219	369	$\downarrow69.7\%$	723	1716	618	$\downarrow64.0\%$
	Time (s)	1633	2949	1625	$\times1.81$	1373	2770	1717	$\times1.61$
HumanEval+	Acc.	70.0	77.0	87.0	$\uparrow10.0$	83.0	93.0	90.0	$\downarrow3.0$
	Token	281	1283	563	$\downarrow56.2\%$	617	1557	725	$\downarrow53.4\%$
	Time (s)	802	2329	1606	$\times1.45$	1120	1894	1633	$\times1.16$

2048 was the maximum token budget per agent allowed. For MedQA, MBPP+, and HumanEval+ it was 4096 tokens.

Across the six benchmarks, HyLaMAS generates 53–74% fewer decoded output tokens than TextMAS and achieves 1.16–2.15× faster inference. Most observed accuracy differences are small relative to sampling uncertainty at $n=100$; we therefore treat them as unresolved rather than evidence of statistical equivalence. Larger positive differences are observed on GSM8K at 4B and 9B and on HumanEval+ at 4B.

To determine whether HyLaMAS remains effective at smaller model scales, we repeated the evaluation using Qwen3.5-2B under the same four-agent, 40 latent-step protocol. As shown in Table 3, HyLaMAS reduces generated output tokens by 74.1–80.0% and achieves a 2.10–2.78× speedup relative to TextMAS across all six benchmarks. Its accuracy is never lower than that of TextMAS, with differences ranging from 0 to 13 percentage points, with the largest gain on GSM8K (80% vs. 67%). These results extend HyLaMAS’s primary efficiency finding to a 2B hybrid backbone: transferring the hybrid internal state avoids much decoding and re-encoding overhead associated with textual communication, without an observed reduction in task accuracy.

Method	GSM8K	ARC-E	ARC-C	MedQA	MBPP+	HumanEval+
Accuracy (%) ↑
Single	73	96	87	61	42	50
TextMAS	67	91	81	61	49	52
HyLaMAS	80	94	81	63	49	58
Improve	+13	+3	+0	+2	+0	+6
Tokens / sample ↓
Single	453	412	511	1137	570	608
TextMAS	1500	1249	1505	2852	1992	1869
HyLaMAS	308	268	381	739	399	404
Improve	−79.4%	−78.6%	−74.7%	−74.1%	−80.0%	−78.4%
Total Inference time (s) ↓
Single	718	423	889	2045	1797	1462
TextMAS	2305	1397	1853	4451	4126	3428
HyLaMAS	830	514	681	2121	1745	1519
Improve	2.78×	2.72×	2.72×	2.10×	2.36×	2.26×

HyLaMAS reduces decoded output tokens and inference time relative to TextMAS across the evaluated settings, while accuracy changes vary by task and model scale. These benefits do not imply consistent improvements over single-agent inference. For example, on Qwen3.5-9B MBPP+, Single achieves 85% accuracy and HyLaMAS achieves 76%. The results therefore characterize the trade-offs of replacing textual multi-agent communication with hybrid-state communication under the evaluated protocols.

Conclusion

We study complementary approaches to latent-state handoff. SelectKV reduces communicated KV positions and payload by 9.7–9.9% across six benchmarks, with unchanged accuracy on two, one-point decreases on three, and a six-point decrease on HumanEval+. Matched-budget ablations favor informed selection over random and recency-only baselines; boundary comparisons suggest a better accuracy–compression trade-off when inherited context is preserved. HyLaMAS extends latent handoff to hybrid architectures and reduces generated tokens and inference time relative to text-based multi-agent communication across evaluated Qwen3.5 scales.

Limitations and Future Directions

We evaluate only 100 examples per benchmark and one SelectKV model scale (Qwen3-4B); broader benchmarks, model scales, and longer agent chains are needed to assess generalization. SelectKV and HyLaMAS are evaluated independently, not as a combined selective hybrid-state system. Logical payload reductions do not establish physical network savings or proportional end-to-end speedups: selection incurs computation, and hybrid-state handoff requires state management. Future work should evaluate distributed transfer and adaptive communication policies.

Appendix

Ablations

All ablations use deterministic decoding, seed 42, and the sequential Planner–Critic–Refiner–Judger protocol. Only the Judger decodes text. SelectKV is evaluated on Qwen3-4B with $n = 100$, 10 latent steps, and 90% retention. The hybrid-channel ablation uses Qwen3.5-9B on MedQA with $n = 100$, batch size 8, 40 latent steps per non-Judger agent, and a 4,096-token generation cap. Unlike the main experiments, this ablation disables latent-space realignment. Exact commands are included in the supplementary repository.

Source code, experimental configurations, and reproduction commands are available in the SelectKV repository and the HyLaMAS repository.

We first evaluate the role of boundary protection in SelectKV. As shown in Figure 1, boundary-aware SelectKV improves MedQA accuracy from 58% to 62% relative to the same 90%-retention configuration without boundary protection, compared with 63% for full-cache LatentMAS. At the same time, the boundary-aware configuration reduces KV payload from 295.81 MiB to 266.61 MiB per problem relative to the full-cache baseline. These results suggest that preserving inherited context recovers much of the accuracy lost under more aggressive compression while retaining a meaningful communication reduction.

We next isolate the effect of the SelectKV ranking criterion from the communication budget. Figure 2 compares SelectKV with random and recency-only selection under an identical retained-KV budget. SelectKV obtains 62% MedQA accuracy, compared with 56% for random selection and 11% for recency-only selection. Because all three methods retain the same number of KV positions, this ablation indicates that the SelectKV selection criterion contributes beyond compression alone.

Finally, we separate the two communication channels used by HyLaMAS. As shown in Figure 3, accuracy is 88% with both full-attention KV and complete Gated DeltaNet (GDN) state, 87% with KV only, 84% with GDN state only, and 90% with neither channel. None of the differences from full-state transfer is significant under paired McNemar tests ($p = 1.000$, 0.344, and 0.754, respectively). The right panel of Figure 3 nevertheless shows a consistent descriptive reduction in completed-response length when GDN state is retained: 825 versus 983 decoded tokens when KV is present, and 724 versus 979 when KV is removed. Although the no-transfer configuration achieves 90% accuracy versus 88% with both, full-state transfer reduces decoded tokens from 979 to 825

per completed response (15.7% fewer), suggesting an output-length benefit without a demonstrated accuracy advantage.

Experimental Setup and Reproducibility

Protocol. Both methods use the sequential Planner–Critic–Refiner–Judger chain. The first three agents append a role-conditioned prompt and perform latent rollout steps without decoding text; the Judger receives the accumulated state, appends its prompt, and decodes the answer. All agents in a run share one model instance, a single set of weights, no adapters, no per-role fine-tuning, so they differ only in their prompt and in the state they inherit. Padding-aware position and attention masks keep batched examples independent.

Code and Reproducibility. Source code, experimental configurations, and reproduction commands are available in the SelectKV repository and the HyLaMAS repository. Table 4 summarizes the key experimental settings. The repositories provide detailed implementation and reproduction instructions.

	SelectKV	HyLaMAS
Model	Qwen3-4B	Qwen3.5-{2B, 4B, 9B}
Architecture	dense attention, 36 layers	GDN : attention 3:1
Weights	bfloat16	bfloat16
Attention backend	—	sdpa
GPU	NVIDIA L40S (46,068 MiB)	A100-SXM4-40GB (2B/4B); L40S (9B)
PyTorch / CUDA	2.13.0 / 13.0; 2.8.0 / 12.9	2.13.0 / 13.0 (2B/4B); 2.8.0 / 12.9 (9B)
transformers	5.15.1	5.15.1
Latent steps $m$	10	40
Batch size	1	8
Retention $r$	0.90	—
Decoding	do_sample=False, seed 42
Examples	$n = 100$ per benchmark, offset 0