betaFeedback

WorksComputer Science

TokenGate: Adapting LLMs for Fast and Memory-Efficient Prefill with Learned Token Pruning

Abstract

Expert human readers process large inputs (e.g., code repositories and long documents) by skimming to closely read only what matters. On the other hand, language models process all input tokens through each model layer, resulting in significant processing costs for long-context inputs. We introduce TokenGate, a small, drop-in component that extends existing trained models to enable end-to-end learnable token pruning over long-context inputs for fast and memory-efficient prefill. To enable end-to-end learning despite the non-differentiable top-K operator for token pruning, we introduce token-gated attention, a mechanism inspired by Mixture-of-Experts that re-weights attention logits using token importance scores. Across diverse tasks, TokenGate achieves 1.41.4-11.3×11.3\times faster time-to-first token (TTFT) and 1.51.5-3.4×3.4\times lower peak memory at no accuracy degradation for prefill in transformers. Compared to the best-performing prior methods for token-pruning, TokenGate gives up to 6.4×6.4\times faster prefill and 2×2\times lower peak memory. TokenGate natively extends both standard transformers (e.g., Qwen3) and hybrid-attention models (e.g., Qwen3.5) with simple lightweight adaptation comparable to standard parameter-efficient fine-tuning. We make our code available at: https://github.com/tokengate-models/TokenGate.

https://github.com/tokengate-models/TokenGate

Expert human readers process large inputs (e.g., code repositories and long documents) by skimming to closely read only what matters. On the other hand, language models process all input tokens through each model layer, resulting in significant processing costs for long-context inputs. We introduce TOKENGATE, a small, drop-in component that extends existing trained models to enable end-to-end learnable token pruning over long-context inputs for fast and memory-efficient prefill. To enable end-to-end learning despite the non-differentiable top-KK operator for token pruning, we introduce token-gated attention, a mechanism inspired by Mixture-of-Experts that re-weights attention logits using token importance scores. Across diverse tasks, TOKENGATE achieves 1.4-11.3×\times faster time-to-first token (TTFT) and 1.5-3.4×\times lower peak memory at no accuracy degradation for prefill in transformers. Compared to the best-performing prior methods for token-pruning, TOKENGATE gives up to 6.4×\times faster prefill and 2×\times lower peak memory. TOKENGATE natively extends both standard transformers (e.g., Qwen3) and hybrid-attention models (e.g., Qwen3.5) with simple lightweight adaptation comparable to standard parameter-efficient fine-tuning. We make our code available at: https://github.com/tokengate-models/TokenGate.

Introduction

Long-context inputs underpin exciting recent language model deployments, where the model’s ability to process vast swaths of personalized user data and live content during inference has led to transformative capabilities in deep research, large-scale document processing, coding agents, and data agents (OpenAI, 2025b; Gemini, 2025; Anthropic, 2025b;a; Anysphere, 2025; GitHub, 2025). When processing large inputs, like code or long documents, experts readers skim to closely read only what matters. Yet, transformers process every token of their input through each layer, whether or not the token bears critical information. The cost of this full read is attention compute that grows quadratically with input length, as well as MLP compute and KV-cache memory that grow linearly, making long-context prefill a key bottleneck in model serving.

Recent research proposes to address this efficiency challenge for long-context prefill through two primary approaches. First, one body of work proposes training-free token pruning, using attention-based scoring heuristics to prune token hidden states, forwarding only the highest-scoring tokens to upper layers [37, 47, 36, 60, 67, 20, 11, 63]. These methods are inexpensive to deploy, requiring no model training, but demonstrate limited speedups before degrading accuracy. Alternatively, a large body of work instead develops architectural modifications to the standard transformer architecture [62]. These methods offer promising prefill-efficiency improvements, but are expensive to train, often requiring pre-training from scratch and substantially different hardware and software than existing ecosystems [14, 7, 28, 57, 2, 42, 31 Gu & Dao,]

Correspondence to: [email protected], [email protected], [email protected] 2023; Dao & Gu, 2024). To bridge this gap, we ask the question, can we endow existing trained models with improved long-context prefill efficiency while maintaining strong accuracy?

In this work, we introduce TOKENGATE, a small, drop-in component that extends transformer-based models, enabling end-to-end trainable token pruning over long-context inputs for efficient prefill. The model forward pass using TOKENGATE decomposes into three parts, as Figure 1a shows. First, layers below the gate module perform a standard forward pass, processing all tokens through each layer. Second, tokens pass through the gate module, which produces a score for each token, retaining only the top-KK for continued processing. Third, the layers above the gate module process the selected tokens with token-gated attention, a mechanism that re-weights the attention computation using the generated importance scores. TOKENGATE reduces prefill compute and memory because pruned tokens are never forwarded through the post-gate layers and these layers never generate KV cache entries.

Figure 1. Overview of TOKENGATE. (a) Layers up to LL prefill all NN context tokens as usual; at layer LL the gate module keeps KK of them, and every layer above LL processes only those KK tokens. (b) The gate module scores each hidden state with a small scoring network, and a top-KK selector forwards the kept hidden states together with their importance scores. (c) Token-gated attention multiplies each attention weight by the gate value gjg_j of its key, which enables end-to-end training. (d) On 2WikiMultihopQA question answering, with Qwen3-1.7B, TOKENGATE pushes the Pareto frontier of accuracy versus time-to-first-token and peak prefill memory; at 95% of the full-context accuracy it prefills 3.0× faster and with 2.7× less memory, outperforming the best training-free token pruning methods, like FastKV Jo et al., [37] and SlimInfer Long et al., [47].

Token pruning uses a non-differentiable top-KK operator, which presents a challenge for end-to-end learning. We introduce token-gated attention to overcome this issue. Our method takes inspiration from Mixture-of-Experts (MoE) Shazeer et al., [59]; Fedus et al. [19], which achieve sparsity over activated model parameters by gating expert sub-networks. We use a similar form, but instead weight attention logits to achieve sparsity over processed tokens. Using this technique, we demonstrate that we can adapt existing trained models with cost similar to standard adaptation techniques. For instance, we can fine-tune existing models with TOKENGATE using simple next-token-prediction loss, with no auxiliary loss and total train time similar to supervised fine-tuning on the base model. The resulting TOKENGATE model matches or exceeds accuracy over the trained base model, while significantly reducing prefill latency and memory.

We evaluate TOKENGATE on diverse tasks, including code completion, multi-hop QA, multi-doc summarization, medical extraction, and needle-in-a-haystack, using both traditional transformer models and a recent hybrid attention one. Overall, we show the following findings (1) TOKENGATE reduces TTFT and peak prefill memory by 1.4-11.3× and 1.5-3.4×, respectively, at no accuracy degradation compared to the finetuned model baseline with full-context, and outperforms all training-free token pruning methods, by up to 6.4× faster prefill (TTFT) and 2× lower peak prefill memory relative to the next-best baseline, (2) TOKENGATE natively extends existing recent hybrid-attention architectures, like Qwen3.5-2B [54], where it achieves 3.8× lower TTFT over the full-context baseline at no accuracy degradation, and 2.1× lower TTFT than the next bestperforming token-pruning method at fixed accuracy. (3) Training models is efficient with TOKENGATE, incurring minimal overhead compared to standard parameter-efficient fine-tuning techniques, like LoRA [33].

Overall, we summarize our contributions as follows:

  • We introduce TOKENGATE, a small, drop-in component that extends existing trained models to enable end-to-end learnable token pruning for fast and memory-efficient prefill.

  • We propose token-gated attention, a mechanism for training models end-to-end despite the gate module’s non-differentiable top-KK operator.

  • Our systematic evaluation finds that TOKENGATE reduces TTFT and peak prefill memory by up to 6.4× and 2×, respectively, compared to the next best-performing baselines, TOKENGATE natively extends transformer-based models, including recent hybrid-attention architectures. Further, training is efficient, with total train time comparable to standard parameter-efficient fine-tuning techniques.

Related Works

Training-free Token Pruning. One body of prior work optimizes prefill using training-free token-pruning methods (e.g., FastKV, SlimInfer, DASH, and GemFilter) [37, 47, 36, 60, 67, 20, 11, 63]. These methods prune token hidden states at a mid-layer of the model using attention-based heuristics to rank tokens, forwarding only the highest-scoring tokens to upper layers of the model. This body of work is closest to ours, however, unlike our work, these methods are not trainable. As we observe in § 5, these methods offer limited speedups before degrading accuracy because they employ heuristic attention-based token-ranking functions that under-perform our end-to-end trained scoring network, and they prevent co-adaptation of the model backbone with the pruning function, which our end-to-end trainable design enables.

Architectural Modifications. Several lines of work explore architectural modifications to the self-attention-based transformer [62], reducing the compute and memory of processing long-context inputs. First, one body of work studies architectures that achieve adaptive compute per token through dynamic token routing or learned pruning decisions, but are expensive to train. These works require pre-training from scratch [14, 7, 28, 57, 2] or expensive fine-tuning with auxiliary loss terms that fix the token budget during train time, requiring re-training for each inference-time token budget [42, 31, 27, 56]. In contrast to all of these works, TOKENGATE natively extends popular pre-trained language models and is simple and efficient to train. We show that with a single fine-tuning pass, the resulting TOKENGATE model generalizes across input lengths and token budgets § 5, Appendix § 11.§ 3.

Other works on sub-quadratic architectures for language modeling introduce promising, new algorithms that reduce the per token compute and memory of attention operations, including new compute primitives and sparse attention mechanisms (Gu & Dao, 2023; Dao & Gu, 2024; Yang et al., 2024b,c; Katharopoulos et al., 2020; Sun et al., 2024; Gao et al., 2024; Liu et al., 2025; Yuan et al., 2025; Lu et al., 2025; Xiao et al., 2024; Huang et al., 2025; Jiang et al., 2024; Lai et al., 2025). However, unlike TOKENGATE, these architectures still process all tokens through each model layer. In § 5 we demonstrate TOKENGATE natively extends recent hybrid architectures with sub-quadratic layers [54], and enables even more efficient prefill.

Conditional Computation. Mixture-of-experts layers achieve sparsity over the activated model parameters, making it possible to efficiently scale model capacity [59, 19, 44]. Our work is inspired by the success of this line of work, however, we focus on achieving sparsity over a different axis, the input context length, for scalable long-context prefill.

Decode Optimizations. A large body of work targets the cost of decoding rather than prefill, evicting or merging KV-cache entries after prefill [64, 22, 9, 41] (Zhang et al. 2023; Li et al. 2024), and performing early exit or per-token routing for decoded tokens [58, 17]. These methods do not improve prefill efficiency and are complementary to prefill optimizations like ours.

Preliminaries

Transformer Models and Notation. Given a sequence of tokens x=(x1,…,xN)x = (x_1, \ldots, x_N), the language model embeds each token xix_i into a vector hi0∈Rdmodelh_i^0 \in\mathbb{R}^{d_{\mathrm{model}}} and passes the sequence through a stack of NLN_L transformer layers. Layer ℓ∈{0,…,NL−1}\ell\in\{0, \ldots, N_L - 1\} reads the residual stream hℓh^\ell and applies a self-attention sub-layer followed by a token-wise MLP. Both sub-components are preceded by a normalization layer and add their output back to the residual stream, as given by:

aiℓ=hiℓ+Attn⁡ℓ(LN⁡1(hℓ))i,hiℓ+1=aiℓ+MLP⁡ℓ(a~iℓ),a~iℓ=LN⁡2(aiℓ).(1)a_i^\ell= h_i^\ell+ \operatorname{Attn}^\ell\left(\operatorname{LN}_1(h^\ell)\right)_i,\qquad h_i^{\ell+1} = a_i^\ell+ \operatorname{MLP}^\ell\left(\tilde{a}_i^\ell\right),\qquad\tilde{a}_i^\ell= \operatorname{LN}_2(a_i^\ell). \tag*{(1)}

We call aiℓa_i^\ell the post-attention residual of token ii at layer ℓ\ell. The MLP acts on each position independently, whereas attention mixes information across positions.

In standard self-attention, the normalized inputs LN⁡1(hℓ)\operatorname{LN}_1(h^\ell) are projected to queries, keys, and values qi,kj,vj∈Rdheadq_i, k_j, v_j \in\mathbb{R}^{d_{\mathrm{head}}}, and token ii attends causally to itself and the tokens before it:

Attn⁡i=∑j≤iαijvj,αij=softmax⁡j′≤i(zij)=exp⁡(zij)∑j′≤iexp⁡(zij′),zij=qiTkjdhead.(2)\operatorname{Attn}_i = \sum_{j \le i} \alpha_{ij}v_j,\qquad\alpha_{ij} = \operatorname{softmax}_{j' \le i}(z_{ij}) = \frac{\exp(z_{ij})}{\sum_{j' \le i}\exp(z_{ij'})},\qquad z_{ij} = \frac{q_i^{\mathsf{T}}k_j}{\sqrt{d_{\mathrm{head}}}}. \tag*{(2)}

Prefill, decode, and the KV cache. Generation proceeds in two phases. Prefill runs (1) over the whole input of NN tokens and stores every layer’s keys and values, the KV cache {(kjℓ,vjℓ)}j≤N\{(k_j^\ell,v_j^\ell)\}_{j \le N}. Decode then produces one token per step, computing attention against the cache and appending to it. The cost of both phases grows with the input length NN. Prefill compute scales as O(N2⋅NL)O(N^2 \cdot N_L) for attention and O(N⋅NL)O(N \cdot N_L) for the MLPs, and the KV cache occupies O(N⋅NL)O(N \cdot N_L) memory.

Problem Setting

We consider an LLM input prompt x=[xR,xD]x = [x_R, x_D], where xR=(x1,…,xr)x_R = (x_1, \ldots, x_r) is a request instruction or question and xD=(xr+1,…,xN)x_D = (x_{r+1}, \ldots, x_N) are the “documents” needed to respond to it, such as research articles, code repositories, medical records, legal documents, or financial filings. In practice, xDx_D may be supplied by the user or retrieved by a system or an agent given xRx_R. As an example, a user uploads a document study as a PDF to an AI assistant and asks “what are the side effects reported in this study?” [25, 51, 3, 26, 49, 1, 53, 10]. Here, xRx_R corresponds to the user’s question, and xDx_D corresponds to the attached document, which can be thousands or hundreds of thousands of tokens. Figure 2 shows two such input prompts from our datasets.

Figure 2. Visualization of TOKENGATE applied on sample inputs from the 2WikiMultihopQA dataset (left) and MultiNews dataset (right), with K=256K = 256 kept tokens, highlighted in red (darker denotes higher token importance score by the gate). Detailed visualizations of real samples from all datasets are in Appendix § 12.

In this setting, the documents are provided only at request time, so its KV cache cannot be precomputed. Our goal is two-fold: we aim to maximize quality, measured by task accuracy, and minimize cost, both prefill time-to-first-token (TTFT), which is the latency users experience before seeing any response, and prefill memory footprint, which limits throughput and concurrent user requests.

Trainable Token Pruning with TokenGate

TokenGate is a drop-in neural network component, which places a gate module within a specified transformer layer1^{1}, LL, to score and prune tokens, as shown in Figure 1. In this section, we describe how TokenGate enables trainable token pruning using two key components, the gate module (§ 4.1) and token-gated attention (§ 4.2). We will then overview an end-to-end training recipe for adapting existing models (§ 4.3).

Overall, the full TokenGate forward pass with gating at layer LL decomposes into three phases. First, tokens pass through layers below the gate module, using a standard forward pass. Second, the gate module processes all tokens, producing a score for each, and selecting a subset to prune. Third, the model performs a pruned forward, processing only selected tokens in layers above the gate module, using token-gated attention, which re-weights attention scores using the importance scores. TokenGate reduces both the compute and memory footprint during prefill by preventing model layers after the gate module from processing pruned tokens or generating KV-cache entries for them.

Gate Module

The gate module is placed within a transformer layer, LL, and selects a subset of tokens to continue processing through the remaining model layers during prefill. As Figure 1b shows, the gate module consists of two simple sub-components: a scoring network and a top-KK selector.

Scoring Network. For each token jj in the set of prunable tokens P\mathcal{P}, an MLP reads its normalized post-attention residual at layer LL, a~jL\tilde{a}^{L}_{j} (Eq. (1)), and emits a scalar score b~j\tilde{b}_{j}, which we normalize along the sequence with RMSNorm ([71]) into the importance score bjb_j, following:

b~j=MLP⁡θ(a~jL),bj=b~j1∣P∣∑j′∈Pb~j′2+ϵfor j∈P,bj:=0 for j∈N,(3)\tilde{b}_{j} = \operatorname{MLP}_{\theta}\left(\tilde{a}^{L}_{j}\right), \qquad b_j = \frac{\tilde{b}_{j}}{\sqrt{\frac{1}{|\mathcal{P}|}\sum_{j'\in\mathcal{P}}\tilde{b}_{j'}^{2}}+\epsilon}\quad\text{for }j\in\mathcal{P}, \qquad b_j := 0\text{ for }j\in\mathcal{N}, \tag*{(3)}

where the scores are fixed to zero for the non-prunable tokens N\mathcal{N}, which can consist of answer tokens during model training, decoded tokens, or any optionally specified tokens during prefill, e.g., a user question. The MLP is initialized to output zero, so that bj=0b_j=0 for all jj at the start of training.

Top-KK Selector. The gate selects the token-set SS, consisting of the KK highest-scoring prunable tokens, as well as any non-prunable tokens, for continued processing:

S=top-⁡Kj∈Pbj ∪ N.(4)S=\operatorname{top-}K_{j\in\mathcal{P}} b_j\ \cup\ \mathcal{N}. \tag*{(4)}

The residual stream for pruned tokens is stopped at layer LL, and the selected tokens SS are forwarded through the layer LL MLP and layers ℓ>L\ell>L.

Token-Gated Attention

The gate module’s top-KK selector introduces a non-differentiability, creating a challenge for end-to-end learning. To overcome this challenge, we introduce token-gated attention, which integrates token importance scores generated by the gate module into upper model layers, providing a gradient path. Our method is inspired by Mixture-of-Experts (MoE) ([59]). In MoE, a router network weights the outputs of expert sub-networks to achieve sparsity over activated model parameters. In contrast, token-gated attention re-weights attention logits to achieve sparsity over processed tokens.

Token-gated attention multiplies each token’s attention weight by the gate value gjg_j, computed from it’s importance score bjb_j (Eqs. (5)), in layers ℓ>L\ell>L. Specifically, for self-attention, in contrast to Eq. (2), the attention weights with token-gated attention take the form of Eq. (5). At the start of training with a base model, token-gated attention is equivalent to top-KK attention as the importance scores are 0.

αij=exp⁡(zij)gj∑j′∈S,j′≤iexp⁡(zij′)gj′,i,j∈S,gj=softmax⁡j∈S(bj)(5)\alpha_{ij}=\frac{\exp(z_{ij})g_j}{\sum_{j'\in S,j'\leq i}\exp(z_{ij'})g_{j'}},\qquad i,j\in S,\qquad g_j=\operatorname{softmax}_{j\in S}(b_j) \tag*{(5)}

We make two modifications to this basic form: head-wise scaling, to allow different heads to have different scales for the gate value, and temperature scaling, to enable length generalization so that trained models perform robustly at various pruning ratios. We ablate both decisions in Appendix § 11.

Head-wise. We scale the normalized importance scores bjb_j by a learnable gain factor, γℓh\gamma^{\ell h}, indexed by layer ℓ\ell and head hh, and the gate value is the softmax of the result (Eq. (6)).

gjℓh=softmax⁡j∈S(softplus⁡(γℓh)bj)(6)g_j^{\ell h} = \operatorname{softmax}_{j\in S}\left(\operatorname{softplus}(\gamma^{\ell h})b_j\right) \tag*{(6)}

Intuitively, γℓh\gamma^{\ell h} learns how strongly each attention head gates the attention logits. The softplus keeps every gain positive, so a higher importance score corresponds to higher attention weight. At initialization, softplus⁡(γℓh)\operatorname{softplus}(\gamma^{\ell h}) is set to the standard deviation of layer ℓ\ell's logits so that it is on the same scale as the attention logits.

Temperature Scaling. Pruning shrinks the number of keys each query sees from its original position pip_i to its rank mim_i among kept tokens. This pruning sharpens the softmax relative to the full-context model. We use temperature scaling ([30, 55]) to recalibrate the softmax. The temperature is learnable, positive, and scaled by the log-ratio of the two counts (Eq. (7)), inspired by SSMax [50]. The parameter σ\sigma is initialized such that temperature is 1 at full context.

τiℓh=softplus⁡(σℓh)⋅log⁡milog⁡max⁡(2,pi)(7)\tau_i^{\ell h} = \operatorname{softplus}(\sigma^{\ell h}) \cdot\frac{\log m_i}{\log\max(2,p_i)} \tag*{(7)}

Putting everything together. Substituting Eq. (6) into Eq. (5), we can write the token-gated attention as

αijℓh=softmax⁡j∈S,  j≤i(τiℓh(qiTkjdhead+softplus⁡(γℓh)bj)).(8)\alpha_{ij}^{\ell h} = \operatorname{softmax}_{j\in S,\;j\leq i}\left(\tau_i^{\ell h}\left(\frac{q_i^{\mathsf{T}} k_j}{\sqrt{d_{\mathrm{head}}}}+\operatorname{softplus}(\gamma^{\ell h})b_j\right)\right). \tag*{(8)}

This form is straightforward to implement as an efficient kernel based on FlashAttention [12](Appendix § 8).

End-to-end Gate Training

Despite TOKENGATE's non-differentiable top-KK selection (Eq. (4)), the task loss is a differentiable function of the scoring network, almost everywhere. This is due to token-gated attention, which injects the top-KK gate values into the attention logits at each layer above the token gate. When we choose K>1K > 1, the gate value gjg_j has non-zero derivatives with respect to the scoring network weights. This mirrors how the router is trained in MoE through its top-KK expert selection: the router's value multiplies the output of each selected expert, just as our gate value multiplies the attention score of each selected token ([59, 19]). Intuitively, a token whose additional attention mass lowers the task loss receives a larger score. A token that distracts receives a smaller one, allowing the scoring network to learn token importance. This mechanism allows TOKENGATE to be trained end-to-end jointly with the model backbone using standard task losses.

Loss Function. In § 5, we fine-tune existing pretrained models with LoRA [33] along with the TOKENGATE parameters, demonstrating that the model backbone and gate layer can co-adapt with minimal overhead to standard training. Under this setting, training the model with TOKENGATE uses a standard next-token, cross-entropy loss,

LK=−∑t∈Ttargetlog⁡pϕ(xt∣x<t;K),(9)\mathcal{L}_K = -\sum_{t\in\mathcal{T}_{\mathrm{target}}}\log p_{\phi}(x_t \mid x_{<t}; K), \tag*{(9)}

where pϕ(⋅;K)p_{\phi}(\cdot;K) is the next token probability with a token budget of KK. The trainable parameters ϕ\phi are the scoring network parameters θ\theta, the scale factors {γ,σ}ℓh\{\gamma,\sigma\}^{\ell h}, and the LoRA [33] parameters on the model backbone following standard practice.

Top-KK Budget Sampling. Training with the TOKENGATE should ensure that the model both adapts to the task, improving its accuracy at full context, and learns a strong scoring network, improving its accuracy at restricted token budgets. As such, a simple choice for the top-KK budget to ensure both criteria are met is to use the full-context length with KfullK_{\mathrm{full}} as all prunable tokens ∣P∣|\mathcal{P}| and also sample a smaller KpruneK_{\mathrm{prune}} per training instance. We average the resulting losses as follows:2

Kfull=∣P∣,Kprune∼LogUniform⁡(1,∣P∣/2),L=12(LKfull+LKprune).(10)K_{\mathrm{full}} = |\mathcal{P}|,\qquad K_{\mathrm{prune}} \sim\operatorname{LogUniform}(1, |\mathcal{P}|/2),\qquad\mathcal{L} = \frac{1}{2}\left(\mathcal{L}_{K_{\mathrm{full}}} + \mathcal{L}_{K_{\mathrm{prune}}}\right). \tag*{(10)}

The KfullK_{\mathrm{full}} term anchors the model’s accuracy at full context and gives every token a gradient, including tokens the sampled budget pruned. Meanwhile, the log-uniform term exposes the gate to a range of pruning ratios, allowing the trained model to generalize across a wide budget range without re-training. In § 5, we demonstrate that using this budget sampling, models with the TOKENGATE can be trained efficiently, with minimal overhead compared to standard training. For each step, the forward pass up to the gate layer runs once at full context, and the two forwards after using KfullK_{\mathrm{full}} and KpruneK_{\mathrm{prune}} share the compute graph. Thus, the overhead of each TOKENGATE train step comes solely from the pruned forward at KpruneK_{\mathrm{prune}}. Since Kprune≪KfullK_{\mathrm{prune}} \ll K_{\mathrm{full}}, this overhead is minimal, as we show in § 5.

Experimental Results

We evaluate the effectiveness of TOKENGATE on diverse tasks, including code completion, multi-hop QA, summarization, medical extraction, and needle-in-a-haystack. Overall, results demonstrate:

  • TOKENGATE reduces TTFT and peak prefill memory by 1.4–11.3× and 1.5–3.4×, respectively, at no accuracy degradation compared to the base model with full-context, and outperforms the next best-performing token pruning baselines by up to 6.4× and 2×, respectively.

  • TOKENGATE extends existing transformer-based models, including recent hybrid-attention architectures, achieving 3.8× lower TTFT than the full-context baseline at no accuracy degradation and 2.1× lower TTFT than the best prior methods, at fixed accuracy.

  • Training models with TOKENGATE is efficient, incurring minimal training overhead, with total training time comparable to standard parameter efficient fine-tuning.

We provide details of our experimental setup in Appendix § 10, additional experimental results including context-length extrapolation and ablations in Appendix § 11, and visualization in Appendix § 12.

Datasets. We evaluate on five diverse tasks, including 3 LongBench datasets with available train sets [8], 2WikiMultihopQA [32], a multi-hop QA dataset, RepoBench-P [46], a next-line code completion dataset, and MultiNews [18], a summarization dataset, as well as two additional tasks, BioDEX’s drug-reaction extraction over biomedical papers [16] and needle-in-a-haystack (NIAH) [39]. We use the official evaluation metric for each.

Baselines. We compare TOKEN GATE against state-of-the-art token-pruning methods, FastKV [37], SlimInfer [47], DASH [11], and GemFilter [60], as well as sparse attention methods, FlexPrefill [43] and MInference [35]. We run each baseline using a fine-tuned base model trained to convergence with LoRA [33] (r=16r=16). We train our model with TOKEN GATE to convergence using the same LoRA configuration. We train both a standard transformer model, Qwen3-1.7B [66], and a hybrid architecture, Qwen3-2B [54], each on 4 A100 GPUs.

Fast and Memory-Efficient Prefill

We evaluate the prefill efficiency of models trained with TOKEN GATE. First, Figure 3 (top) shows the accuracy vs TTFT of each benchmarked method, demonstrating that across all datasets, the trained TOKEN GATE model achieves pareto-optimal performance. Specifically, TOKEN GATE reduces TTFT by 1.41.4–11.3×11.3\times at no accuracy degradation compared to standard prefill, and by 1.21.2–6.4×6.4\times compared to the next best-performing baseline. We measure FastKV, one of the best-performing baselines both at its default layer and at layer 1, the same layer index as the TOKEN GATE. Notably, we observe that compared to FastKV (Layer 1), TOKEN GATE significantly outperforms across all datasets, indicating the promise of end-to-end trainable token pruning for achieving prefill speedups.

Accuracy versus time-to-first-token and peak GPU prefill memory

Figure 3. Accuracy vs. time-to-first-token (top) and accuracy vs. peak GPU prefill memory (bottom) with Qwen3-1.7B. Points that incur significant overhead over standard prefill at full context (grey cross), or significantly poor accuracy, are excluded; see Appendix § 11.§ 2 for full plots.

Turning our attention to the bottom row of Figure 3, we study the prefill memory footprint, reporting accuracy versus peak prefill memory. Once again, across all dataset, the TOKEN GATE model establishes the Pareto-frontier, reducing peak memory by 1.51.5–3.4×3.4\times compared to standard prefill at matched accuracy. Moreover the TOKEN GATE model achieves up to 2.1×2.1\times lower peak memory compared to the next best-performing baseline at matched accuracy. Overall, the results demonstrate that TOKEN GATE significantly outperforms prior token-pruning methods, offering an exciting direction for improved prefill efficiency.

Ablation Study: Understanding the Performance of TokenGate Models

We hypothesize that TOKEN GATE owes its superiority to end-to-end learnable token pruning. Our visualizations of the kept tokens for real samples (Appendix § 12) shows that TOKEN GATE flexibly learns very different pruning patterns in a task-dependent manner, ranging from keeping answer keywords for question answering to keeping stop-words at the ends of phrases for summarization (Figure 2). To understand this more rigorously, we investigate two key mechanisms: trainable token-selection, and co-adaptation with the trained backbone.

Trainable selector. First, to study the effectiveness of TOKEN GATE’s learned selector, we compare performance of the trained TOKEN GATE model (Layer 1), to the same model with random token selection on 2WikiMultihopQA. Figure 5 demonstrates that the trained selector significantly outperforms the random-K selector on accuracy across all token budgets. Moreover, we compare against the FastKV selector applied at the same layer, Layer 1, of the finetuned base model. Notably, the TOKEN GATE selector significantly outperforms the FastKV selector. This result demonstrates the value of an end-to-end trained gate module for token pruning: the trained scoring network leads to far better token selection compared to FastKV, which relies on a heuristic using attention scores that offer only weak signal at early layers, resulting in accuracy degradation for aggressive pruning.

Figure 5. F1 vs. kept context tokens on 2WikiMultihopQA with Qwen3-1.7B for gate and Random-K selection on the TOKENGATE model and FastKV and Random-K selection on the LoRA-trained model.

Co-adaptation. Secondly, we analyze the effect of co-adaptation for TOKEN GATE, which allows the model backbone and gate module to co-adapt. We compare the performance of random-K selection over the TOKEN GATE model to the random-K selection over the finetuned base model. Notably, with the same random-K selector, the TOKEN GATE model still outperforms the fine-tuned model, achieving higher accuracy across all token budgets, as Figure 5 shows. This result shows that TOKEN GATE’s end-to-end training allows the model to become more robust to token-pruning, likely redistributing information among tokens.

Overall, the results show that end-to-end training with TOKENGATE enables state-of-the-art prefill performance through two complementary mechanism: the effectiveness of the learned scoring network, and co-adaptation, which allows the model to adapt jointly with the gating module for improved robustness to token pruning.

Generalization to Alternative Architectures

We evaluate how TOKENGATE’s performance extends to a recent hybrid-attention model, Qwen3.5-2B (Qwen Team [54]) model, which uses full-attention layers every fourth layer and a GatedDeltaNet (Yang et al. [68]) for all other layer. We compare TOKENGATE against FastKV (Jo et al. [37]), the best-performing baseline in our main results (§ 5.1). We measure FastKV over the LoRA-tuned base model and the LoRA-tuned model with TOKENGATE, both at varying layers. FastKV uses an attention-based score function and natively applies only to full-attention layers (i.e., layer 3, 7, 11, etc). TOKENGATE natively extends both the full attention and GatedDeltaNet layers, so we apply it over both. We provide further details on TOKENGATE for Qwen3.5 in Appendix § 10.§ 4.

Figure 4 shows the accuracy vs TTFT of both methods at varying layers on the NIAH 32K dataset, demonstrating that TOKENGATE’s strong performance extends to the hybrid-attention model. TOKENGATE (Layer 3) matches the accuracy of standard prefill at 3.8× lower TTFT, and 2.1× lower TTFT than the best-performing FastKV run at the same accuracy. Applying the TOKENGATE at the lower GatedDeltaNet layers (Layer 1,2) enable even higher TTFT reductions, up to 7.8× faster prefill, at accuracy within 6% of standard prefill, once again significantly outperforming FastKV.

Figure 4. Accuracy vs. TTFT with the Qwen3.5-2B hybrid-attention architecture. Unlike TOKENGATE, FastKV can only prune at attention layers.

Gate Training Efficiency

We evaluate overhead of training models with TOKENGATE compared standard models, both with LoRA fine-tuning (r=16r = 16). Table 2 in Appendix D.1 compares both, reporting total training time in A100 hours and the average per-step time during training for each evaluated dataset. Overall, we observe that a TOKENGATE step is slower by 1.3× compared to the baseline due to its sampling at two token budgets per step. However, the TOKENGATE train runs converge after fewer steps on four of five tasks, leading to modestly reduced overall training time compared to the standard training.

DatasetSFTOursRatio ↓SFTOursRatio ↓
Full train timePer-step time
Repo1.7h1.1h0.6×0.50s0.64s1.3×
2Wiki3.8h4.0h1.1×0.34s0.43s1.3×
MNews1.3h1.8h1.4×0.21s0.27s1.3×
BioDEX1.2h1.0h0.8×0.23s0.30s1.3×
NIAH2.8h1.0h0.4×0.99s1.24s1.3×
Mean2.2h1.8h0.8×0.45s0.58s1.3×

Table 2. Training time for SFT baseline and model trained with TokenGate for Qwen3-1.7B on one A100. Ratio shows Ours / SFT.

Conclusion

In this work, we introduced TOKENGATE, a neural network component that extends existing trained transformer models, enabling end-to-end trainable token pruning using token-gated attention. We demonstrate that across diverse tasks, TOKENGATE reduces TTFT and peak prefill memory by up to 11.3× and 3.4×, respectively, at no accuracy degradation compared to standard prefill, and by up to 6.4× and 2×, respectively, compared to the best performing baselines. Crucially, TOKENGATE is compatible with diverse transformer-based architectures, is simple to train, and incurs minimal training overhead, making it a promising drop-in replacement for standard fine-tuning over trained backbones. In the future, we envision models trained with TOKENGATE will enable a broad range of applications, including medical extraction, deep research, and coding agents, where processing long-context inputs is critical, but currently prohibitively expensive.

Acknowledgments

This research was supported in part by affiliate members and other supporters of the Stanford DAWN project, including Meta, Google, and VMware, as well as Cisco, SAP, and a Sloan Fellowship. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the sponsors.

Efficient kernels for TokenGate

Token-gated attention (Eq. (8)) differs from standard causal attention in two ways: a per-query temperature τℓh\tau^{\ell h} and a per-key, per-head gate term added to the attention logits. The temperature is applied by scaling each query before the attention call, so it needs no kernel support. The gate term needs a per-key bias, which standard fused kernels do not accept, and unfused or score-modifying kernels are substantially slower. We therefore patch FlashAttention-2 [12] to read a per-key, per-head bias vector, which it adds to each tile of attention scores before the online softmax. This keeps the fused structure of the kernel: the bias adds one vector load per key block, and a biased call costs 1.0–1.1× a plain one. The kernel supports head dimensions 128 (Qwen3) and 256 (Qwen3.5).

Training. The same kernel provides the backward pass: the gradient of the bias of key jj is the sum over queries of the gradient of its attention scores, which the kernel accumulates alongside the usual gradients, so the task loss reaches the scoring network through the fused kernel. The two targets of each training step (Eq. (10)) are packed into one variable-length call. For Qwen3.5, whose Gated DeltaNet layers are not supported by this path, we train with FlexAttention [15] and add the bias as a score modification, but more optimized kernels are possible.

Inference. Layers at and below the gate use the standard fused kernel (cuDNN on A100). The pruned forward of the post-gate layers is captured as one CUDA graph per padded length: the kept sequence is padded to a multiple of 64, and pad keys receive a bias of −109-10^9 so the result is exact. The gather of the KV cache of layers ℓ≤L\ell\leq L to the kept tokens (Appendix B) runs on a separate stream, concurrently with the post-gate layers. During decoding, the gate term of each kept key is fixed, so it is folded once into an additive attention mask per post-gate layer and decoding uses standard attention.

Decoding with TokenGate

The layers above the gate, ℓ>L\ell> L, never compute KV entries for pruned tokens, so during decoding each new token attends there only to the kept tokens SS and the tokens generated so far. The layers at and below the gate, ℓ≤L\ell\leq L, process the full context during prefill, so by default their KV cache holds every context token, and decode tokens attend to the full context in these layers and to the pruned context above them. Optionally, TokenGate can also prune the cache of layers ℓ≤L\ell\leq L.

In the post-gate layers, decode tokens use the same token-gated softmax as prefill (Eq. (8)). The gate term softplus(γℓh)bj(\gamma^{\ell h})b_j of each kept key is computed once at prefill and stored with the cache as a constant per-key bias, and each generated token is a non-prunable token with gate term 0 (Eq. (3)). The temperature τiℓh\tau_i^{\ell h} of a decode query follows Eq. (7), with mim_i its rank among the kept and generated tokens and pip_i its original position; generated tokens continue from the original position of the last prompt token.

Details of Experimental Setup

Datasets

All datasets are evaluated on Qwen3-1.7B with greedy decoding for up to 1024 new tokens. Each model is fine-tuned on a training set that is disjoint from its evaluation set.

RepoBench-P. RepoBench-P [46] is a next-line code completion task. We use the LongBench [8] version, whose prompts contain cross-file code context and an in-file prefix, and the model completes the next line. We evaluate on all 500 samples (mean context 11.0k tokens, up to 39k) and report edit similarity, 1−Levenshtein/max⁡(length)1 - \text{Levenshtein}/\max(\text{length}), on the first generated line. For training, we use 14.9k samples from the RepoBench v1.1 Python and Java cross-file splits, deduplicated against the evaluation set.

2WikiMultihopQA. 2WikiMultihopQA [32] is a multi-hop question answering task over several Wikipedia passages; we use the LongBench version. We evaluate on all 200 samples (mean context 7.5k tokens) and report token-level F1 against the gold answers. For training, we use 20k 2WikiMultihopQA questions with contexts rebuilt from full Wikipedia articles and decontaminated against LongBench.

MultiNews. MultiNews [18] is a multi-document news summarization task; we use the LongBench version. We evaluate on all 200 samples (mean context 2.7k tokens, up to 14k) and report ROUGE-L. For training, we use 44.8k samples from the MultiNews training split in the LongBench prompt format.

BioDEX. BioDEX-Reactions [16] is an adverse drug reaction extraction task over full-text biomedical papers: given a paper, the model lists the reported adverse reactions as MedDRA terms. We evaluate on 500 samples drawn from the official validation split with articles between 2k and 39k tokens (mean 4.5k), and report set F1 between predicted and gold reactions, counting a predicted reaction as correct if it matches a gold term up to substring containment. For training, we use 9.8k papers from the official training split.

NIAH. NIAH [39] is needle-in-a-haystack retrieval at 16k tokens. The haystack is built from Paul Graham essays, and the needle, “The best thing to do in city is activity.”, is inserted at a sentence boundary at a given depth; the model is then asked for the best thing to do in that city. The 200 evaluation samples cover 20 held-out cities at 10 depths evenly spaced from 0 to 1, and we report accuracy, i.e., whether the gold activity appears in the answer. For training, we use 12k generated samples with haystacks of 14k–16.4k tokens and uniformly random depths; the essays and cities are disjoint from the evaluation set.

Baselines

All baselines run on the same SFT model, fine-tuned on each dataset with LoRA (r=16r=16, α=32\alpha=32) on the query and value projections of all layers. The pruning baselines keep all non-document tokens and prune only document tokens, with the same kept-token budgets KK as TOKENGATE (64–8192). We briefly describe each baseline below, and the configurations used. We run each method, applying its optimization for prefill-only.

FastKV [37]. At a token-selective propagation layer, FastKV scores tokens by the attention they receive from a window of the last queries, max-pools the scores, and propagates only the top-KK tokens to deeper layers. We use the paper’s observation window of 8 tokens and pooling kernel of 7. We run it at layer 13 by default, following the paper and its hyper-parameter search, which chooses relative depth of 46% for its models. We additionally run the baseline at layer 1, the layer at which we apply TOKENGATE, to compare performance at a matched layer index.

SlimInfer [47]. SlimInfer prunes hidden states in several stages: at layers 8, 17 and 25 it keeps 3K3K, 1.5K1.5K and KK tokens, scoring 64-token blocks by the attention of the last 4 queries. We use the paper’s block size, kernel and layer schedule, and its budget ratios, following the paper’s hyperparameter search.

GemFilter [60]. GemFilter runs the first layers on the full prompt, selects the top-KK tokens by the last query’s attention scores at a filter layer, and then runs a fresh forward pass over only the selected tokens. We use filter layer 12, at the same relative depth (0.4) as recommended by the original paper, and the official pooling kernel.

DASH [11]. DASH halts tokens whose representations have stabilized: at layer 11 (0.4× the number of layers, the official default), it ranks tokens by the change in their hidden state from the previous layer and drops the most stable ones, keeping KK document tokens.

FlexPrefill [43]. FlexPrefill is a sparse-attention method: it keeps all tokens but computes a query-adaptive sparse attention pattern during prefill, controlled by a cumulative-attention threshold γ\gamma. We use the official kernel and its recommended hyperparameters (divergence threshold τ=0.1\tau=0.1, which decides per head whether to use a query-aware or a vertical-slash pattern; minimum budget 1024; block size 128) and sweep γ∈{0.1,0.2,…,0.9,0.95,0.97}\gamma\in\{0.1, 0.2, \ldots, 0.9, 0.95, 0.97\}, extending the range until accuracy degrades.

MInference [35]. MInference uses per-head sparse attention patterns found by an offline search, which for Qwen3-1.7B selects vertical-and-slash patterns for all heads. We run the official implementation with patterns searched on Qwen3-1.7B, and sweep its budget by scaling the vertical and slash sizes of the searched configuration by {0.25,0.5,1,2,4}\{0.25, 0.5, 1, 2, 4\}, which keeps the searched pattern shape.

Implementation and timing. All methods run the same fine-tuned backbone in bf16 with the LoRA adapter folded into the weights, and all dense decoder layers are torch.compiled. Dense causal attention uses the same fused kernel in every lane (cuDNN on the A100; FlashAttention-2 on Qwen3.5-2B); the only kernel that differs is in the TOKENGATE’s post-gate layers, which require a per-key bias and use a FlashAttention-2 variant at 1.0–1.1× the cost of a plain call (Appendix § 8). FastKV and GemFilter run through the compiled layers directly. The official SlimInfer and DASH prefills bypass the compiled layer forward, so we reimplemented both in compile-friendly form and verified that they select the same tokens as the official code (DASH bit-identical; SlimInfer identical selection), which made them 1.2–2.4× and 1.25–1.56× faster; we report the faster versions. MInference is compiled around its sparse kernel with identical accuracy, and FlexPrefill runs eager, its fastest configuration, since its data-dependent block selection triggers recompilation under torch.compile. The pruned forward of the TOKENGATE and of FastKV is captured as one CUDA graph per padded length; the remaining baselines have no fixed-shape pruned phase to capture. Both FastKV and the TOKENGATE optionally prune the KV cache of the layers below their pruning layer to the selected tokens once prefill completes, so that decoding holds KK entries at every layer; we disable this setting for both in all experiments since KV cache pruning presents an orthogonal set of optimizations and our experiments instead isolate the effect of token pruning for prefill optimization (Appendix § 9). Inference timings are measured at batch size of 1 on one A100.

Training Hyperparameters

Table 1 lists the hyperparameters used to train the TOKENGATE models and the SFT baseline on each dataset. Both use the same LoRA configuration, learning rate, schedule and training data; they differ only in the optimizer of the TOKENGATE parameters and in the token budgets sampled during training.

HyperparameterValue
Base modelQwen3-1.7B, bf16
LoRAr=16r=16, α=32\alpha=32 on query and value projections, all layers
OptimizerAdamW; Muon ([38]) for the gate matrices (TOKENGATE only)
Learning rate10−410^{-4}, linear warmup over 1000 steps, then constant
Weight decay0.1
Gradient clippingglobal norm 1.0
Batch size1 sequence per GPU
Training contextdocuments of 2k–16k tokens
Gate layer (TOKENGATE only)L=1L=1
Budgets per step (TOKENGATE only)K∼log-uniformK\sim\text{log-uniform}, plus full context

Table 1. Training hyperparameters for TOKENGATE and the SFT baseline, shared across all five datasets.

Extension to Alternative Hybrid-Attention Architectures

Qwen3.5 architecture. Qwen3.5-2B [54] is a hybrid-attention model with 24 layers. Every fourth layer, ℓ∈A={3,7,11,15,19,23}\ell\in\mathcal{A}=\{3,7,11,15,19,23\}, is a full-attention layer whose output is multiplied by a learned sigmoid output gate. All other layers are Gated DeltaNet layers ([68]), which replace softmax attention with a linear recurrence over a fixed-size state and therefore have no attention weights.

Gate module. We place the TOKENGATE at layer L∈{1,2,3}L\in\{1,2,3\}, where layers 1 and 2 are Gated DeltaNet layers and layer 3 is the first full-attention layer. The scoring network (Eq. (3)) and the top-KK selector (Eq. (4)) are unchanged. The only difference is the token mixer of layer LL: the scoring network reads the normalized residual after layer LL’s mixer,

a~jL=LN⁡2(hjL+Mixer⁡L(LN⁡1(hL)j)),Mixer⁡L∈{GatedDeltaNet⁡L,Attn⁡L},(11)\tilde{a}_j^L = \operatorname{LN}_2\left(h_j^L + \operatorname{Mixer}_L\left(\operatorname{LN}_1(h^L)_j\right)\right), \qquad\operatorname{Mixer}_L \in\{\operatorname{GatedDeltaNet}_L,\operatorname{Attn}_L\}, \tag*{(11)}

which for a Gated DeltaNet layer is the output of its recurrence rather than of self-attention.

Token-gated attention. Since Gated DeltaNet layers have no attention logits to bias, token-gated attention (Eq. (8)), including the gains γℓh\gamma^{\ell h} and the temperature, is applied only in the full-attention layers above the gate, ℓ∈A,ℓ>L\ell\in\mathcal{A}, \ell> L. The Gated DeltaNet layers above the gate run unchanged on the kept tokens SS, starting from an empty state, so that their recurrent state is built from the kept tokens only:

hSℓ={token-gated attention layer (Eq. 8)ℓ∈A, ℓ>L,GatedDeltaNet⁡ℓ(hSℓ−1)ℓ∉A, ℓ>L.(12)h_{S}^{\ell} = \begin{cases} \text{token-gated attention layer (Eq. 8)} & \ell\in\mathcal{A},\ \ell> L,\\ \operatorname{GatedDeltaNet}_{\ell}(h_{S}^{\ell-1}) & \ell\notin\mathcal{A},\ \ell> L. \end{cases} \tag*{(12)}

Kept tokens keep their original positions for rotary embeddings in the full-attention layers.

Decoding. The recurrent state of a Gated DeltaNet layer summarizes all tokens it has seen in a fixed-size matrix and cannot be pruned per token. The Gated DeltaNet layers ℓ≤L\ell\le L therefore decode from a state built over the full prompt, while the KV cache of full-attention layers are pruned, as in Appendix § 9.

Training. As for Qwen3, we train the TokenGate jointly with LoRA (r=16r=16) on the query and value projections, which in Qwen3.5 exist only in the full-attention layers; the Gated DeltaNet layers are not adapted.

Additional Experimental Results

Training Time

Main Results without Axis Limits

Below, Figure 6 shows the main results (Figure 3), but without axis limits and with every point shown.

Figure 6. Main results (Figure 3) without x or y limits: accuracy vs. time-to-first-token (top) and accuracy vs. peak GPU prefill memory (bottom) with Qwen3-1.7B, all points shown.

Extrapolating to Longer Context

We study the ability of the trained TokenGate to extrapolate to inference settings with longer context. We take the fine-tuned base model and TokenGate model, both trained on the 16k-NIAH task using Qwen1.7B, and we evaluate prefill performance on the NIAH task with 32k, 64k and 128k-length test sets. Table 3 shows the TTFT speedups and peak memory reductions of the TokenGate model at no accuracy degradation over the full at full-context accuracy on each dataset. The results demonstrate that TokenGate extrapolates across varying context lengths at inference, retaining accuracy with greater than 11.3 - 13× speedups. We observe that the TTFT speedups of the trained TOKENGATE model are highest at the longest context-length ranges, where trainable token pruning can significantly reduce costs. Meanwhile, the peak memory reduction is consistent across all length ranges, as expected, since peak memory is dominated by memory footprint of the standard forward that occurs without pruning in layers below the gate.

NIAH 16kNIAH 32kNIAH 64kNIAH 128k
TTFT speedup11.3×12.8×13.0×13.0×
Peak memory reduction3.42×3.43×3.43×3.43×

Table 3. TokenGate TTFT speedup and peak prefill memory reduction at no accuracy degradation over full context for the NIAH-16k, 32k, 64k and 128k test sets with Qwen3-1.7B.

Ablating the TokenGate Design

In this section we ablate the TOKENGATE design (§ 4). First, Figure 7 (left) compares the accuracy vs TTFT performance of TOKENGATE on BioDEX with Qwen3-1.7B against three variants: (a) removing the full-context loss term LKfullL_{K_{\mathrm{full}}} from the per-step loss (Eq. (10)), (b) removing the temperature scaling term during token-gated attention, and (c) removing the head-wise scaling. The results demonstrate that training without the full-context loss most significantly degrades inference accuracy at full-context, and removing the temperature scaling and head-wise gain also degrades accuracy, leading to sub-optimal accuracy vs TTFT.

Token-gated forward and training ablations and gate-layer ablation on BioDEX

Figure 7. Left: token-gated forward and training ablations on BioDEX (Qwen3-1.7B, gate at layer 1): F1 vs. TTFT (batch 1, A100) for the full TOKENGATE recipe against three single-change variants: no full-context loss (no full-context term in Eq. (10)), no temperature scaling (Eq. (7) removed), and no head-wise scaling (raw bias, with no normalization or gain; Eq. (3) removed). Right: gate-layer ablation on BioDEX (Qwen3-1.7B): F1 vs. TTFT (batch 1, A100) with the TOKENGATE placed at layer 0, 1, or 2.

We additionally analyze layer selection, a hyperparameter for TOKENGATE. Figure 7 (right) shows performance on BioDEX of Qwen3-1.7B models trained with the TOKENGATE at Layer 0, 1, and 2. Notably, the model with gating at Layer 0 is Pareto-suboptimal relative to the Layer 1 and 2 gate models, suggesting that the Layer 0 post-attention residuals provide insufficient signal for the TOKENGATE scoring network. On the other hand, the Layer 1 and Layer 2 TOKENGATE models both lie along the Pareto frontier, suggesting that by Layer 1, the post-attention residuals passed to the gate module are sufficiently rich to learn an effective scoring network for trainable token pruning.

Visualizing Kept Tokens on Real Samples

Figures 8–12 show which context tokens TOKENGATE keeps on one real evaluation prompt per dataset, using the layer-1 checkpoints of Figure 3 (Qwen3-1.7B). To avoid selecting favorable examples, each prompt is the first of the seeded test draw used by the benchmarks. The strip at the top of each figure spans the whole prompt: the context in light grey, each kept token in red, and the non-prunable tokens NN of Section 4 (instruction, question and answer template; the scaffold) in darker grey at either end. The box below shows the part of the context under the bracket, chosen as the window of that length holding the most kept tokens. For each prompt we show the smallest budget K∈{64,128,256,512,1024}K \in\{64, 128, 256, 512, 1024\} at which the greedy answer equals that of the same model with the full context, or K=256K = 256 where no budget does. The shown budgets keep between 0.4% (NIAH) and 13% (MultiNews) of the context.

Selection is global, not local. In every sample the kept tokens fall in all ten deciles of the context, so the gate does not reduce to a recency window or to the start of the document. At the same time it concentrates the budget where the task needs it: on NIAH, 21 of the 24 tokens of the needle sentence, which sits at 78% depth in a 15,843-token haystack, are among the 64 kept tokens, and the model answers exactly. The same holds for all eight NIAH prompts we checked, whose answer at K=64K = 64 matches the reference.

What is kept, and what it costs. Table 4 summarizes the five examples. On 2WikiMultihopQA, BioDEX and NIAH the gate keeps content: entities and years, the terms that describe a medical case, and, where the reference answer occurs verbatim in the context, a large share of its tokens (all of the needle’s answer on NIAH). On RepoBench-P the selection is instead dominated by code structure: most kept tokens contain no letter, and they are spread thinly over the retrieved files, whereas on NIAH most kept tokens have a kept neighbor and form contiguous spans. This suggests that for code completion the post-gate layers use the kept tokens more as anchors on the structure of the retrieved files than as a copy of their content. The examples also show where pruning costs accuracy: on MultiNews the gate keeps mostly function words and under-keeps capitalized tokens, and the summary at K=256K = 256 reports the right events but gets the names and places wrong, and for drug reaction extraction on BioDEX the pruned model recovers one of the two reactions found with the full context.

DatasetWhat is keptFailure mode
RepoBench-Pcode syntax rather than identifiers: commas and empty argument lists “()” dominate, line breaks are avoided, and kept tokens are isolated rather than spanswrong line, as with the full context
2WikiMultihopQAentities and dates: names and years are over-kept and function words avoided; on the shown prompt, the answer phrase (“Film Award for . . . Director”)a different award at small budgets, exact at the shown one; multi-hop answers need the largest budgets to match the full context
MultiNewsmostly function words and scattered event words (“charged”, “president”); names and places are under-keptright events, but wrong name, occupation and place, consistent with the under-kept names
BioDEXterms of the diagnosis and outcome (“septic shock”, “failure”) and the connectives of lists (“and”, “or”); numbers are almost never keptmisses reactions, but so does the full-context model, which bounds the score
NIAH 16kthe needle sentence as one contiguous span, plus rare subwords elsewhere (“-haired”, “Lisp”, “.html”)none: exact at every budget

Table 4. Summary of what TOKENGATE keeps and how its answers fail. Examples refer to the prompts of Figure 8–Figure 12 at their shown budgets; the patterns hold over the first eight prompts of each test draw.

Figure 8. Context tokens kept by TOKENGATE on one RepoBench-P test prompt (Appendix § 12), with the reference answer and the model’s greedy answers at the shown budget and with the full context. Kept tokens are shaded by gate score (darker is higher) and pruned tokens are grey.

Figure 12. Context tokens kept by TOKENGATE on one NIAH 16k test prompt (Appendix § 12), with the reference answer and the model’s greedy answers at the shown budget and with the full context. Kept tokens are shaded by gate score (darker is higher) and pruned tokens are grey.

Figure 9. Context tokens kept by TOKENGATE on one 2WikiMultihopQA test prompt (Appendix E), with the reference answer and the model’s greedy answers at the shown budget and with the full context. Kept tokens are shaded by gate score (darker is higher) and pruned tokens are grey.

Figure 10. Context tokens kept by TOKENGATE on one MultiNews test prompt (Appendix E), with the reference answer and the model’s greedy answers at the shown budget and with the full context. Kept tokens are shaded by gate score (darker is higher) and pruned tokens are grey.

Figure 11. Context tokens kept by TOKENGATE on one BioDEX test prompt (Appendix § 12), with the reference answer and the model’s greedy answers at the shown budget and with the full context. Kept tokens are shaded by gate score (darker is higher) and pruned tokens are grey.

References

  1. [1]Adobe. Adobe Acrobat AI Assistant, 2025. URL https://www.adobe.com/acrobat/generative-ai-pdf.html.
  2. [2]Joshua Ainslie, Tao Lei, Michiel de Jong, Santiago Ontañón, Siddhartha Brahma, Yury Zemlyansky, David Uthus, Mandy Guo, James Lee-Thorp, Yi Tay, Yun-Hsuan Sung, and Sumit Sanghai. Colt5: Faster long-range transformers with conditional computation. arXiv preprint arXiv:2303.09752, 2023.
  3. [3]Anthropic. Collaborate with Claude on Projects, 2024. URL https://www.anthropic.com/news/projects.
  4. [7]Sangmin Bae, Yujin Kim, Reza Bayat, Sungyun Kim, Jiyoun Ha, Tal Schuster, Adam Fisch, Hrayr Harutyunyan, Ziwei Ji, Aaron Courville, and Se-Young Yun. Mixture-of-recursions: Learning dynamic recursive depths for adaptive token-level computation. arXiv preprint arXiv:2507.10524, 2025.arxiv.org/abs/2507.10524
  5. [8]Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. Longbench: A bilingual, multitask benchmark for long context understanding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024.arxiv.org/abs/2308.14508
  6. [9]Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Yucheng Li, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Junjie Hu, and Wen Xiao. Pyramidkv: Dynamic kv cache compression based on pyramidal information funneling. arXiv preprint arXiv:2406.02069, 2024.arxiv.org/abs/2406.02069
  7. [10]ChatPDF. ChatPDF: Chat with any PDF, 2025. URL https://www.chatpdf.com.
  8. [11]Yujie Chen, Tailai Chen, Yifeng Gao, Zoe Wanying He, Yijue Xu, Shaobo Wang, and Linfeng Zhang. Stability implies redundancy: Delta attention selective halting for efficient long-context prefilling. arXiv preprint arXiv:2604.18103, 2026.arxiv.org/abs/2604.18103
  9. [12]Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.08691, 2023.arxiv.org/abs/2307.08691
  10. [14]Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Łukasz Kaiser. Universal transformers. arXiv preprint arXiv:1807.03819, 2018.arxiv.org/abs/1807.03819
  11. [15]Juechu Dong, Boyuan Feng, Driss Guesso us, Yanbo Liang, and Horace He. Flex attention: A programming model for generating optimized attention kernels. arXiv preprint arXiv:2412.05496, 2024.arxiv.org/abs/2412.05496
  12. [16]Karel D’Oosterlinck, Francois Remy, Johannes Deleu, Thomas Demeester, Chris Develder, Klim Zaporojets, Aneiss Ghodsi, Simon Ellershaw, Jack Collins, and Christopher Potts. Biodex: Large-scale biomedical adverse drug event extraction for real-world pharmacovigilance. In Findings of the Association for Computational Linguistics: EMNLP 2023, 2023.arxiv.org/abs/2305.13395
  13. [17]Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, Basil Hosmer, Bram Wasti, Liangzhen Lai, Anas Mahmoud, Bilge Acun, Saurabh Agarwal, Ahmed Roman, Ahmed A Aly, Beidi Chen, and Carole-Jean Wu. Layerskip: Enabling early exit inference and self-speculative decoding. arXiv preprint arXiv:2404.16710, 2024.DOI
  14. [18]Alexander R. Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir R. Radev. Multi-news: A large-scale multi-document summarization dataset and abstractive hierarchical model. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), 2019.arxiv.org/abs/1906.01749
  15. [19]William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 2022.arxiv.org/abs/2101.03961
  16. [20]Qichen Fu, Minsik Cho, Thomas Merth, Sachin Mehta, Mohammad Rastegari, and Mahyar Najibi. Lazyllm: Dynamic token pruning for efficient long context llm inference. arXiv preprint arXiv:2407.14057, 2024.arxiv.org/abs/2407.14057
  17. [22]Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, and Jianfeng Gao. Model tells you what to discard: Adaptive kv cache compression for llms. arXiv preprint arXiv:2310.01801, 2023.arxiv.org/abs/2310.01801
  18. [25]Google. NotebookLM: Your personalized AI research assistant, 2023. URL https://notebooklm.google.
  19. [26]Google. Gemini for Google Workspace, 2025. URL https://workspace.google.com/solutions/ai/.
  20. [27]Saurabh Goyal, Anamitra R. Choudhury, Saurabh M. Raje, Venkatesan T. Chakaravarthy, Yogish Sabharwal, and Ashish Verma. Power-bert: Accelerating bert inference via progressive word-vector elimination. arXiv preprint arXiv:2001.08950, 2020.arxiv.org/abs/2001.08950
  21. [28]Alex Graves. Adaptive computation time for recurrent neural networks. arXiv preprint arXiv:1603.08983, 2016.arxiv.org/abs/1603.08983
  22. [30]Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In International Conference on Machine Learning, 2017.
  23. [31]Shwai He, Tao Ge, Guoheng Sun, Bowei Tian, Xiaoyang Wang, and Dong Yu. Router-Tuning: A Simple and Effective Approach for Enabling Dynamic-Depth in Transformers, September 2025. URL http://arxiv.org/abs/2410.13184. arXiv:2410.13184 [cs.CL].
  24. [32]Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps. In Proceedings of the 28th International Conference on Computational Linguistics (COLING), 2020.arxiv.org/abs/2011.01060
  25. [33]Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Sean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021.arxiv.org/abs/2106.09685
  26. [35]Huijiang Jiang, Yucheng Li, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Zhenhua Han, Amir H. Abdi, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. MInference 1.0: Accelerating pre-filling for long-context llms via dynamic sparse attention. arXiv preprint arXiv:2407.02490, 2024.arxiv.org/abs/2407.02490
  27. [36]Weisheng Jin, Maojia Song, Tej Deep Pala, Yew Ken Chia, Amir Zadeh, Chuan Li, and Soujanya Poria. Promptdistill: Query-based selective token retention in intermediate layers for efficient large language model inference. arXiv preprint arXiv:2503.23274, 2025.arxiv.org/abs/2503.23274
  28. [37]Dongwon Jo, Jiwon Song, Yulhwa Kim, and Jae-Joon Kim. Fastkv: Decoupling of context reduction and kv cache compression for prefill-decoding acceleration. arXiv preprint arXiv:2502.01068, 2025.arxiv.org/abs/2502.01068
  29. [38]Keller Jordan, Yuchen Jin, Vlado Boza, Jiacheng You, Franz Cesista, Laker Newhouse, and Jeremy Bernstein. Muon: An optimizer for hidden layers in neural networks. https://kellerjordan.github.io/posts/muon/, 2024.
  30. [39]Gregory Kamradt. Needle in a haystack – pressure testing llms. https://github.com/gkamradt/LLMTest_NeedleInAHaystack, 2023.
  31. [41]Jang-Hyun Kim, Jinuk Kim, Sangwoo Kwon, Jae W. Lee, Sangdoo Yun, and Hyun Oh Song. Kvizip: Query-agnostic kv cache compression with context reconstruction. arXiv preprint arXiv:2505.23416, 2025.arxiv.org/abs/2505.23416
  32. [42]Sehoon Kim, Sheng Shen, David Thorsley, Amir Gholami, Woosuk Kwon, Joseph Hassoun, and Kurt Keutzer. Learned token pruning for transformers. arXiv preprint arXiv:2107.00910, 2021.arxiv.org/abs/2107.00910
  33. [43]Xunhao Lai, Jianqiao Lu, Yao Luo, Yiyuan Ma, and Xun Zhou. Flexprefill: A context-aware sparse attention mechanism for efficient long-sequence inference. arXiv preprint arXiv:2502.20766, 2025.arxiv.org/abs/2502.20766
  34. [44]Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Deng, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Hanwei Xu, Hao Yang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J. L. Cai, Jian Liang, Jianzhong Guo, Jiaqi Ni, Jiashi Li, Jin Chen, Jingyang Yuan, Junjie Qiu, Junxiao Song, Kai Dong, Kaige Gao, Kang Guan, Lean Wang, Lecong Zhang, Lei Xu, Leyi Xia, Liang Zhao, Liye Zhang, Meng Li, Miaojun Wang, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingming Li, Ning Tian, Panpan Huang, Peiyi Wang, Peng Zhang, Qihao Zhu, Qinyu Chen, Qiushi Du, R. J. Chen, R. L. Jin, Ruiqi Ge, Ruizhe Pan, Runxin Xu, Ruyi Chen, S. S. Li, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shaoqing Wu, Shengfeng Ye, Shirong Ma, Shiyu Wang, Shuang Zhou, Shuiping Yu, Shunfeng Zhou, Size Zheng, T. Wang, Tian Pei, Tian Yuan, Tianyu Sun, W. L. Xiao, Wangding Zeng, Wei An, Wen Liu, Wenfeng Liang, Wenjun Gao, Wentao Zhang, X. Q. Li, Xiangyue Jin, Xianzhu Wang, Xiao Bi, Xiaodong Liu, Xiaohan Wang, Xiaojin Shen, Xiaokang Chen, Xiaosha Chen, Xiaotao Nie, Xiaowen Sun, Xiaoxiang Wang, Xin Liu, Xin Xie, Xingkai Yu, Xinnan Song, Xinyi Zhou, Xinyu Yang, Xuan Lu, Xuecheng Su, Y. Wu, Y. K. Li, Y. X. Wei, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Li, Yaohui Wang, Yi Zheng, Yichao Zhang, Yiliang Xiong, Yiling Zhao, Ying He, Ying Tang, Yishi Piao, Yixin Dong, Yixuan Tan, Yiyuan Liu, Yongji Wang, Yongqiang Guo, Yuchen Zhu, Yuduan Wang, Yuheng Zou, Yukun Zha, Yunxian Ma, Yuting Yan, Yuxiang You, Yuxuan Liu, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhen Huang, Zhen Zhang, Zhenda Xie, Zhewen Hao, Zhihong Shao, Zhiniu Wen, Zhipeng Xu, Zhongyu Zhang, Zhuoshu Li, Zihan Wang, Zhihui Gu, Zilin Li, and Ziwei Xie. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. arXiv preprint arXiv:2405.04434, 2024.arxiv.org/abs/2405.04434
  35. [46]Tianyang Liu, Canwen Xu, and Julian McAuley. Repobench: Benchmarking repository-level code auto-completion systems. arXiv preprint arXiv:2306.03091, 2023.arxiv.org/abs/2306.03091
  36. [47]Lingkun Long, Rubing Yang, Yushi Huang, Desheng Hui, Ao Zhou, and Jianlei Yang. Slim-infer: Accelerating long-context llm inference via dynamic token pruning. arXiv preprint arXiv:2508.06447, 2025.arxiv.org/abs/2508.06447
  37. [49]Microsoft. Microsoft 365 Copilot, 2025. URL https://www.microsoft.com/microsoft-365/copilot.
  38. [50]Ken M. Nakanishi. Scalable-softmax is superior for attention. arXiv preprint arXiv:2501.19399, 2025.arxiv.org/abs/2501.19399
  39. [51]OpenAI. File uploads FAQ, 2025a. URL https://help.openai.com/en/articles/8555545-file-uploads-faq.
  40. [53]Perplexity AI. File uploads, 2025. URL https://www.perplexity.ai/help-center/en/articles/10354807-file-uploads.
  41. [54]Qwen Team. Qwen3.5. https://qwenlm.github.io/, 2026.
  42. [55]Rishabh Ranjan, Saurabh Garg, Mrigank Raman, Carlos Guestrin, and Zachary Lipton. Post-hoc reversal: Are we selecting models prematurely? In Advances in Neural Information Processing Systems, 2024.arxiv.org/abs/2404.07815
  43. [56]Yongming Rao, Wenliang Zhao, Benlin Liu, Jiwen Lu, Jie Zhou, and Cho-Jui Hsieh. Dynamicvit: Efficient vision transformers with dynamic token sparsification. arXiv preprint arXiv:2106.02034, 2021.arxiv.org/abs/2106.02034
  44. [57]David Raposo, Sam Ritter, Blake Richards, Timothy Lillicrap, Peter Conway Humphreys, and Adam Santoro. Mixture-of-depths: Dynamically allocating compute in transformer-based language models. arXiv preprint arXiv:2404.02258, 2024.arxiv.org/abs/2404.02258
  45. [58]Tal Schuster, Adam Fisch, Jai Gupta, Mostafa Dehghani, Dara Bahri, Vinh Q. Tran, Yi Tay, and Donald Metzler. Confident adaptive language modeling. arXiv preprint arXiv:2207.07061, 2022.arxiv.org/abs/2207.07061
  46. [59]Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538, 2017.arxiv.org/abs/1701.06538
  47. [60]Zhenmei Shi, Yifei Ming, Xuan-Phi Nguyen, Yingyu Liang, and Shafiq Joty. Discovering the gems in early layers: Accelerating long-context llms with 1000x input token reduction. arXiv preprint arXiv:2409.17422, 2024.arxiv.org/abs/2409.17422
  48. [62]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30, 2017.arxiv.org/abs/1706.03762
  49. [63]Hanrui Wang, Zhekai Zhang, and Song Han. Spatten: Efficient sparse attention architecture with cascade token and head pruning. In IEEE International Symposium on High-Performance Computer Architecture (HPCA), 2021. arXiv:2012.09852.DOI
  50. [64]Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. arXiv preprint arXiv:2309.17453, 2023.arxiv.org/abs/2309.17453
  51. [66]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengyuan Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Liangshao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.arxiv.org/abs/2505.09388
  52. [67]Dongjie Yang, XiaoDong Han, Yan Gao, Yao Hu, Shilin Zhang, and Hai Zhao. Pyramidinfer: Pyramid kv cache compression for high-throughput llm inference. arXiv preprint arXiv:2405.12532, 2024a.arxiv.org/abs/2405.12532
  53. [68]Songlin Yang, Jan Kautz, and Ali Hatamizadeh. Gated delta networks: Improving mamba2 with delta rule. arXiv preprint arXiv:2412.06464, 2024b.arxiv.org/abs/2412.06464
  54. [71]Biao Zhang and Rico Sennrich. Root mean square layer normalization. arXiv preprint arXiv:1910.07467, 2019.arxiv.org/abs/1910.07467

Paper details

vlnc:W_hzk53bsg
Version
v1 · latest
Published
Added
License
CC BY 4.0
Usage

views · 11

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents