betaFeedback

WorksComputer Science

A Mixture-Gated Theory of Anisotropic Representation Dynamics in Transformers

Abstract

Small differences between internal representations can shrink or grow as they pass through a Transformer layer, affecting how information is preserved and how sensitive the model is to perturbations. Yet it remains unclear when changes in attention weights can turn a representation difference that would otherwise shrink into one that grows. We develop a Mixture-Gated Theory of Anisotropic Representation Dynamics that separates self-attention into value transport, query gating, and key gating. Our analysis reveals how the geometry of attended representations constrains gating and how interactions between these mechanisms can reverse contraction induced by value transport. We formalize this phenomenon as gating-induced regime switching, where a representation difference contracts under fixed attention routing but expands under full attention. We derive a necessary and sufficient spectral criterion that certifies the existence or absence of switching and identifies switching directions without exhaustive search. Beyond existence, our theory characterizes maximum relative switching strength, certifies the severity of contraction that gating can overcome, and determines the largest dimension of a subspace in which every nonzero direction switches. Experiments across four Transformer architectures validate these predictions through analytical and finite-forward comparisons, revealing distinct and joint query–key gating mechanisms and nontrivial switching subspaces. Our results provide a mechanistic and spectral understanding of how attention routing can alter local representation dynamics.

Introduction

Many practical problems involving Transformers depend on how small changes to internal representations evolve during computation. Some changes gradually disappear, while others grow and affect the model’s behavior. For example, in model editing, correcting a wrong fact should change the intended answer without affecting unrelated behaviors [9]. An overly strong edit may unintentionally change other answers, so we want useful and independent changes to be amplified while unwanted changes are suppressed. A similar challenge appears in adversarial attacks, where an attacker may need to query a target model many times to find a small perturbation that flips its prediction [4]. Understanding which perturbation directions are likely to be amplified could help identify promising attacks without relying entirely on repeated trial-and-error queries. This problem also appears in many other Transformer applications, including representation steering, model robustness, and long-context modeling, where preserving useful changes and suppressing unwanted ones is important [16, 8].

Previous studies have provided important insights into how Transformer representations evolve. Research on representation collapse and signal propagation explains how repeated attention can make token representations increasingly similar or reduce their diversity [3, 11, 6]. Other work studies when token representations converge or diverge across layers [13, 15]. Jacobian and spectral analyses further reveal how sensitive Transformer representations are to perturbations and how this sensitivity depends on attention computations [7, 2, 1, 14, 12]. However, knowing that a representation difference grows does not necessarily explain what causes it to grow. In particular, we still need to distinguish changes caused by the information being passed through attention from those caused by attention selecting different information. This leads to our central question: Can attention routing turn a representation difference that would otherwise shrink into one that grows?

To answer this question, we develop a Mixture-Gated Theory of Anisotropic Representation Dynamics. Our starting point is that the same Transformer computation can shrink some representation differences while amplifying others. We refer to this direction-dependent behavior as anisotropic representation dynamics. To understand why this happens, we examine how self-attention changes its output. Attention combines information from different tokens, and a small perturbation can affect this process in two ways. First, it can change the information carried by the tokens themselves, which we call value transport. Second, it can change how much attention each token receives, which we call gating. We further distinguish query gating, which changes attention through the query, from key gating, which changes attention through the keys. This decomposition allows us to compare how representation differences evolve when only the transported information changes, and when changes in attention allocation also contribute to their evolution. A representation difference may therefore evolve very differently depending on whether attention routing responds to it (See Figure 1).

Conceptual illustration of gating-induced regime switching

Figure 1. Conceptual illustration of gating-induced regime switching. Two nearby representations initially differ by a small perturbation δzℓ\delta z_\ell. Under fixed attention weights, value transport contracts this difference. However, a small change in the query can redistribute attention among token representations, introducing an additional gating contribution. When this contribution is sufficiently strong, the full attention computation amplifies the same representation difference rather than suppressing it. The illustration motivates our analysis of how attention routing can reverse transport contraction.

Our analysis shows that changing attention weights does not always produce a change in the output. If attended tokens have nearly identical representations, shifting attention between them has little effect. We characterize this limitation geometrically, showing that differences among attended representations restrict which perturbation directions can affect the output through query gating. Even when attended tokens differ, some perturbations produce no local query-gating response. More importantly, gating can either reinforce or counteract value transport. Consider a representation difference that becomes smaller under value transport alone. By changing which tokens receive attention, gating can oppose this contraction. If the gating response is sufficiently strong, the full attention computation makes the same difference grow instead of shrink. We call this reversal a gating-induced regime switch. Our theory establishes a spectral condition for when switching is possible, identifies directions that exhibit this reversal, and characterizes the largest subspace in which every nonzero direction switches.

Our theory also shows that not all regime switches are equally significant. In some directions, value transport only weakly suppresses a representation difference, making it relatively easy for gating to amplify it. In other directions, gating can still amplify a difference even when value transport strongly suppresses it. To distinguish these cases, we introduce two complementary measures: relative switching strength, which compares the gating effect against the suppression caused by value transport, and absolute reversal severity, which measures how strongly transport can suppress a difference that gating can still amplify. Our theory characterizes both measures, establishes conditions under which switching cannot occur, and examines the reliability of switching directions when the underlying operators are estimated with error. We validate these predictions on pretrained Transformer models by applying small perturbations to hidden representations and comparing the observed changes with our theoretical predictions. The experiments show that high relative switching strength does not necessarily indicate that gating overcomes strong suppression. Moreover, unguided random search often fails to identify directions that are strongly suppressed by transport but amplified by full attention, whereas spectral analysis identifies such directions directly and can be more efficient when many directions must be examined. Our contributions are summarized as follows:

  • We explain why the same Transformer attention computation can amplify some representation differences while suppressing others. To understand this behavior, we decompose self-attention into three mechanisms: value transport, query gating, and key gating. We then show that query gating is constrained by the geometry of the attended representations: it can only produce changes through directions in which those representations differ.

  • We develop a spectral theory to explain when attention gating can reverse the contraction caused by value transport. Our results provide a precise condition for this reversal, identify the directions where gating is strongest relative to transport contraction, and determine how severe a contraction can still be reversed. We also characterize the largest subspace in which every nonzero direction exhibits such a reversal.

  • We validate our theoretical predictions across different Transformer models. Through finite-forward experiments, we verify the predicted geometric and gating mechanisms and examine how switching behavior varies across architectures and layers. We further show that spectral analysis can identify directions where gating reverses particularly severe contraction, even when these directions are difficult to discover through unguided perturbation search.

Related Work

Representation collapse and dynamical analyses. Transformer representation dynamics have been studied through rank collapse, signal propagation, and token evolution [3, 11, 6]. Pham et al. [13] analyze convergence and divergence of token trajectories, including positional encoding; Tomihari and Karakida [15] study recurrent attention through Jacobians and Lyapunov exponents. Other work connects query–key spectra to attention localization and avoidance of rank or entropy collapse [2]. These studies address token configurations and stability. We instead examine how a specific representation perturbation evolves at a fixed operating point, distinguishing directions suppressed by fixed-routing transport from those whose suppression is reversed when attention routing changes.

Sensitivity, Jacobians, and attention derivatives. Previous studies have examined how sensitive Transformers are to changes in their inputs and hidden representations. Kim et al. [7] study Lipschitz properties of self-attention, while Aubry et al. [1] and Saratchandran and Lucey [14] analyze attention Jacobians and their spectral properties. Ormaniec et al. [12] further investigate the Hessian of the attention loss and its dependence on query, key, and value projections. Rather than simply measuring sensitivity, we investigate which attention mechanisms cause a representation difference to be suppressed or amplified. Specifically, we compare what happens when attention weights remain fixed with what happens when query gating, key gating, or both are allowed to change them. Although these mechanisms can be separated mathematically, their effects on representation growth interact and cannot generally be added independently. This comparison allows us to determine when changes in attention routing reverse the suppression caused by value transport.

Mixture interpretations and state-dependent routing. Prior work has explored mixture-based interpretations of attention, including softmax and sigmoid attention [17], hierarchical mixtures for gated attention [10], and Mixture-of-Head architectures [5]. We view standard self-attention through a Mixture-of-Experts (MoE) perspective, where each attended token provides a value representation and the attention weights act as input-dependent routing coefficients. This viewpoint allows us to distinguish two mechanisms: transporting information under fixed attention weights and changing how information is selected through query and key gating. By analyzing how these mechanisms affect the evolution of representation differences, we show that query gating is constrained by the variation among attended representations. More importantly, we characterize how changes in attention routing interact with value transport, explaining when a representation difference suppressed by transport can become amplified under full attention.

Positioning: spectral certification of regime switching. Our objective is to determine when state-dependent routing turns a direction suppressed by transport into one amplified by full attention. On a transport-contractive subspace, comparing transport contraction with the gating-induced growth correction yields a generalized spectral criterion for whether switching occurs and which direction has the strongest relative reversal. We also characterize absolute reversal severity, which distinguishes overcoming strong suppression from reversing only weak suppression, the maximal dimension of an all-switching subspace, and stability under operator-estimation error. The eigenvalue and perturbation tools are classical; our contribution is their attention-specific, mechanistically interpretable formulation, which can identify severe reversals without unguided direction-by-direction probing.

Problem Setting

Consider a Transformer block at layer ℓ\ell and a representation zℓ∈Rdz_\ell\in\mathbb{R}^{d} at the operating point of interest. We suppress the surrounding sequence context and focus on the local representation map Tℓ:Rd→RdT_\ell: \mathbb{R}^{d} \to\mathbb{R}^{d}, so that zℓ+1=Tℓ(zℓ)z_{\ell+1} = T_\ell(z_\ell). We study how a small difference δzℓ∈Rd\delta z_\ell\in\mathbb{R}^{d} between two nearby representations, zℓz_\ell and zℓ+δzℓz_\ell+ \delta z_\ell, evolves through the block. Define

δzℓ+1:=Tℓ(zℓ+δzℓ)−Tℓ(zℓ).(1)\delta z_{\ell+1} := T_\ell(z_\ell+ \delta z_\ell) - T_\ell(z_\ell). \tag*{(1)}

For sufficiently small δzℓ\delta z_\ell,

δzℓ+1=Jℓδzℓ+O(∥δzℓ∥2),Jℓ:=∂Tℓ∂zℓ,(2)\delta z_{\ell+1} = J_\ell\delta z_\ell+ \mathcal{O}(\lVert\delta z_\ell\rVert^2), \qquad J_\ell:= \frac{\partial T_\ell}{\partial z_\ell}, \tag*{(2)}

where JℓJ_\ell is the local Jacobian of the block. To determine whether a representation difference grows or shrinks, we must first specify how its magnitude is measured. We therefore introduce a symmetric positive-definite matrix M=M⊤≻0M = M^\top\succ0 and define the representation metric ∥v∥M2:=v⊤Mv\lVert v\rVert_M^2 := v^\top Mv. The matrix MM defines the geometry in which representation differences are measured: M=IM = I recovers the usual Euclidean norm, while a general MM allows different representation directions to be weighted differently. Under the local transformation v↦Jℓvv \mapsto J_\ell v, the squared MM-magnitude after the block is

∥Jℓv∥M2=(Jℓv)⊤M(Jℓv)=v⊤Jℓ⊤MJℓv.(3)\lVert J_\ell v\rVert_M^2 = (J_\ell v)^\top M(J_\ell v) = v^\top J_\ell^\top M J_\ell v. \tag*{(3)}

Thus Jℓ⊤MJℓJ_\ell^\top M J_\ell is the metric induced on the incoming perturbation by the local Transformer dynamics. Comparing the outgoing and incoming magnitudes gives

∥Jℓv∥M2−∥v∥M2=v⊤GℓMv,GℓM:=Jℓ⊤MJℓ−M.(4)\lVert J_\ell v\rVert_M^2 - \lVert v\rVert_M^2 = v^\top G_\ell^M v, \qquad G_\ell^M := J_\ell^\top M J_\ell- M. \tag*{(4)}

Accordingly, v⊤GℓMv>0v^\top G_\ell^M v > 0 indicates local amplification, whereas v⊤GℓMv<0v^\top G_\ell^M v < 0 indicates local suppression. Importantly, the sign of v⊤GℓMvv^\top G_\ell^M v may depend on the direction vv. A single Transformer block can therefore suppress some representation differences while amplifying others. We refer to this direction-dependent behavior as anisotropic representation dynamics. Equation (4) characterizes which representation differences grow or shrink, but does not yet explain why. Our goal is to connect this directional growth to the internal computation of self-attention. We separate the local dynamics into transport, which propagates changes in the information carried by attended representations, and state-dependent gating, which arises when the perturbation changes the attention allocation itself. This leads to the central question of the paper:

When and why can attention routing turn a representation difference that would shrink dramatically under value transport into one that grows?

We call such a transition a gating-induced regime switch, and seek to characterize both when such switching directions exist and which directions exhibit the strongest reversal.

Mechanism of State-Dependent Attention

The growth operator in Section 3 tells us whether a representation difference grows or shrinks, but not which part of self-attention causes this behavior. To understand why, we decompose the attention Jacobian into three pathways: value transport, which propagates changes in the information carried by attended tokens; query gating, which changes attention allocation through the query; and key gating, which changes attention allocation through the keys. This decomposition allows us to understand how each pathway contributes to representation growth.

Transport and Gating in Self-Attention

Consider one self-attention head in a Transformer layer. Let xi∈Rdx_i \in\mathbb{R}^d denote the representation of token ii, rr the query-token index, and mm the index of the token whose representation is perturbed. For query token rr, the attention score assigned to token ii is sri=xrTWQKxis_{ri} = x_r^{\mathsf{T}}W_{QK}x_i, where WQK:=WQWKT/dkW_{QK} := W_QW_K^{\mathsf{T}}/\sqrt{d_k} combines the query and key projections, and dkd_k is the query/key dimension. The corresponding attention weight is αri=exp⁡(sri)/∑jexp⁡(srj)\alpha_{ri} = \exp(s_{ri})/\sum_j \exp(s_{rj}). Let WVOW_{VO} denote the combined value and output transformation. The attention output for query token rr is

Fr=∑iαriWVOxi.(5)F_r = \sum_i \alpha_{ri}W_{VO}x_i. \tag*{(5)}

(5) shows that a perturbation can change the attention output in two ways. First, it can change the information carried by the attended tokens through the value term WVOxiW_{VO}x_i. Second, it can change the attention weights αri\alpha_{ri}, causing the model to place more or less weight on different tokens. We call these two effects value transport and gating, respectively. While transport changes the information being passed forward, gating changes how that information is selected.

To understand how each pathway affects the output, lets consider a small change in the representation of token mm. The Jacobian Jrm:=∂Fr/∂xmJ_{rm} := \partial F_r/\partial x_m captures how this change affects the attention output of query token rr. For gating, the effect also depends on how different the attended representations are from one another. To capture these differences, we use the attention-weighted mean μr:=∑iαrixi\mu_r := \sum_i \alpha_{ri}x_i and covariance Σr:=∑iαri(xi−μr)(xi−μr)T\Sigma_r := \sum_i \alpha_{ri}(x_i-\mu_r)(x_i-\mu_r)^{\mathsf{T}}. The mean describes the average information selected by attention, while the covariance measures how much the attended representations differ around this average. Differentiating Eq. (5) then gives

Jrm=αrmWVO⏟value transport+1{m=r}WVOΣrWQKT⏟query gating+αrmWVO(xm−μr)(WQKTxr)T⏟key gating.(6)J_{rm} = \underbrace{\alpha_{rm}W_{VO}}_{\text{value transport}} + \underbrace{\mathbb{1}_{\{m=r\}}W_{VO}\Sigma_r W_{QK}^{\mathsf{T}}}_{\text{query gating}} + \underbrace{\alpha_{rm}W_{VO}(x_m-\mu_r)(W_{QK}^{\mathsf{T}}x_r)^{\mathsf{T}}}_{\text{key gating}}. \tag*{(6)}

Here, 1{m=r}\mathbb{1}_{\{m=r\}} equals one when the perturbed token is also the query token and zero otherwise. The full derivation is provided in Appendix 9.4. (6) separates the attention Jacobian into three pathways. The value-transport pathway directly propagates the perturbation through the value transformation. The query-gating pathway captures how changing the query alters its attention to the available tokens, and contributes only when m=rm=r. The key-gating pathway captures how changing token mm affects the attention it receives as a key. When m=rm=r, both gating pathways can contribute to the same output change. This decomposition allows us to distinguish changes caused by the information being transported from those caused by changes in attention allocation. In the next subsection, we show how the differences among attended tokens limit which perturbation directions can affect the output through query gating.

Geometry of Query Gating

We now investigate how query gating changes a representation difference and which perturbation directions can influence the attention output through this mechanism. From Eq. (6), we isolate the query-gating component

Jr(q)=WVOΣrWQKT(7)J_r^{(q)} = W_{VO}\Sigma_r W_{QK}^{\mathsf{T}} \tag*{(7)}

which contributes to the full Jacobian JrmJ_{rm} only when the perturbed token is the query itself, i.e., m=rm=r. This decomposition reveal that query gating has three components. The matrix WQKTW_{QK}^{\mathsf{T}} determines how a query perturbation changes attention scores, Σr\Sigma_r captures variation among the attended representations, and WVOW_{VO} maps the resulting change into the output space. We are particularly interested in the role of Σr\Sigma_r, since it connects changes in attention routing to the representations being attended to. To characterize this connection, we define the attended variation subspace as Ur:=span⁡{xi−μr:αri>0}\mathcal{U}_r := \operatorname{span}\{x_i-\mu_r:\alpha_{ri}>0\}, where μr\mu_r is the attention-weighted mean of the attended representations. This subspace describes the directions along which those representations differ from their mean. The following proposition characterizes how this geometry constrains query gating.

References

  1. [1]Murdock Aubry, Haoming Meng, Anton Sugolov, and Vardan Papyan. Transformer block coupling and its correlation with generalization in llms. In The Thirteenth International Conference on Learning Representations, 2025.arxiv.org/abs/2407.07810
  2. [2]Han Bao, Ryuichiro Hataya, and Ryo Karakida. Self-attention networks localize when QK-eigenspectrum concentrates. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 2903–2922. PMLR, 21–27 Jul 2024.arxiv.org/abs/2402.02098
  3. [3]Yihe Dong, Jean-Baptiste Cordonnier, and Andreas Loukas. Attention is not all you need: Pure attention loses rank doubly exponentially with depth. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 2793–2803. PMLR, 2021.arxiv.org/abs/2103.03404
  4. [4]Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-box adversarial attacks with limited queries and information. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2137–2146. PMLR, 2018.arxiv.org/abs/1804.08598
  5. [5]Peng Jin, Bo Zhu, Li Yuan, and Shuicheng Yan. MoH: Multi-head attention as mixture-of-head attention. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 28233–28255. PMLR, 2025.arxiv.org/abs/2410.11842
  6. [6]Akhil Kedia, Mohd Abbas Zaidi, Sushil Khylia, Jungho Jung, Harshith Goka, and Haejun Lee. Transformers get stable: An end-to-end signal propagation theory for language models. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 23449–23531. PMLR, 2024.arxiv.org/abs/2403.09635
  7. [7]Hyunjik Kim, George Papamakarios, and Andriy Mnih. The lipschitz constant of self-attention. In International Conference on Machine Learning, pages 5562–5571. PMLR, 2021.arxiv.org/abs/2006.04710
  8. [8]Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paran jape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173, 2024.arxiv.org/abs/2307.03172
  9. [9]Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. In Advances in Neural Information Processing Systems, volume 35, 2022.arxiv.org/abs/2202.05262
  10. [10]Viet Nguyen, Tuan Minh Pham, Thinh Cao, Tan Dinh, Huy Nguyen, Nhat Ho, and Alessandro Rinaldo. A statistical theory of gated attention through the lens of hierarchical mixture of experts. arXiv preprint arXiv:2602.01468, 2026.arxiv.org/abs/2602.01468
  11. [11]Lorenzo Noci, Sotiris Anagnostidis, Luca Biggio, Antonio Orvieto, Sidak Pal Singh, and Aurelien Lucchi. Signal propagation in transformers: Theoretical perspectives and the role of rank collapse. In Advances in Neural Information Processing Systems, volume 35, 2022.arxiv.org/abs/2206.03126
  12. [12]Weronika Ormaniec, Felix Dangel, and Sidak Pal Singh. What does it mean to be a transformer? insights from a theoretical hessian analysis. In International Conference on Learning Representations, volume 2025, pages 17046–17081, 2025.arxiv.org/abs/2410.10986
  13. [13]Duy-Tung Pham, An The Nguyen, Viet-Hoang Tran, Nhan-Phu Chung, Xin Tong, Tan M. Nguyen, and Thieu N. Vo. Dynamical properties of tokens in self-attention and effects of positional encoding. In Advances in Neural Information Processing Systems, volume 38, 2025.arxiv.org/abs/2512.03058
  14. [14]Hemanth Saratchandran and Simon Lucey. Spectral conditioning of attention improves transformer performance. In Advances in Neural Information Processing Systems, volume 38, 2025.
  15. [15]Akiyoshi Tomihari and Ryo Karakida. Recurrent self-attention dynamics: An energy-agnostic perspective from jacobians. In Advances in Neural Information Processing Systems, volume 38, pages 155236–155269, 2025.
  16. [16]Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Ulisse Mini, and Monte MacDiarmid. Activation addition: Steering language models without optimization, 2024.
  17. [17]Fanqi Yan, Huy Nguyen, Pedram Akbarian, Nhat Ho, and Alessandro Rinaldo. Sigmoid self-attention has lower sample complexity than softmax self-attention: A mixture-of-experts perspective, 2025. Preprint.arxiv.org/abs/2502.00281

Paper details

vlnc:W_xnymhabp
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 4

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents