Introduction
Large Language Models (LLMs) have demonstrated striking reasoning capabilities across mathematics, code, scientific question answering, and planning, especially when equipped with chain-of-thought (CoT) prompting [39 ] and test-time scaling of intermediate reasoning steps [10 ] . Yet our theoretical understanding of these capabilities remains incomplete. We still lack precise answers to several basic questions: how can a transformer implement a multi-step reasoning algorithm, why does architectural depth improve reasoning efficiency, and under what conditions can such an algorithm be acquired through gradient-based training?
A growing body of theoretical work studies these questions through symbolic reasoning tasks, where the algorithmic structure is explicit enough to analyze. Prior results have clarified important aspects of transformer expressivity, showing both limitations of constant-depth models on inherently sequential tasks and the additional power obtained from chain-of-thought computation [8 , 17 , 22 ] . Other work has examined depth–width tradeoffs for graph algorithms [27 , 42 ] , sample complexity for reasoning tasks [40 ] , and training dynamics for algorithmic primitives such as induction heads, sparse token selection, in-context regression, and Markov-chain structure learning [35 , 25 , 4 , 38 ] .
Despite this progress, graph search remains poorly understood. This setting is especially important because it captures a central difficulty of reasoning: the model must not merely follow a single predetermined chain, but must choose among alternatives, remember what has already been explored, and backtrack when a branch fails. In other words, graph search formalizes the kind of committed exploration and revision that appears in planning, proof search, and multi-step deduction.
Existing theory leaves three gaps. First, most provable learning results for path-finding focus on trees, where there is at most one path between two vertices and the need for genuine search is largely absent [41 ] . Directed graphs are qualitatively harder: they contain branching, cycles, and dead ends, so a successful algorithm must maintain a visited set and decide when to move forward or backtrack. Second, although prior work shows that constant-depth transformers can implement DFS [22 ] , the architectural threshold for efficient graph search is not known. It is unclear whether one layer suffices, why two layers should be special, and how additional layers translate into improved search efficiency. Third, recent work has analyzed reachability with continuous chain of thought [Zhu et al., 2025a], but the corresponding learnability question for standard discrete CoT traces remains open.
In this paper, we give a theoretical account of how transformers implement, scale, and learn depth-first search (DFS) on directed graphs. We focus on DFS because it is the canonical sequential search algorithm: it combines branching, memory of visited states, forward exploration, and backtracking, while naturally producing a discrete CoT trace. Our results connect three aspects of reasoning—architecture, depth, and training dynamics—within a single graph-search framework.
Our contributions.
We identify a sharp depth threshold for efficient DFS. We prove that a one-layer transformer cannot implement even a basic DFS child-selection primitive on dense directed graphs without quadratic representation size, measured as heads × \times × width × \times × precision. In contrast, we construct a two-layer transformer that implements full DFS with near-linear representation size. The separation concerns the width a single attention layer needs, not parameter count or running time, and it shows that the second layer is not merely a convenience: it is necessary and sufficient for exact DFS with sub-quadratic width in our setting.
We show that depth gives an exponential increase in look-ahead horizon. Beyond the two-layer DFS construction, we prove that additional layers can perform parallel bounded-reachability computation. After explicit neighbor aggregation, each propagation layer doubles the radius represented in source and child memories. For every k ≥ 3 k \ge3 k ≥ 3 , a k k k -layer transformer implements a bounded-reachability DFS policy with horizon 1 + 2 k − 3 1+2^{k-3} 1 + 2 k − 3 . This establishes a depth–horizon tradeoff. We distinguish it from worst-case search-efficiency guarantees, which require additional control over the distances of the selected branches.
We prove that DFS behavior can emerge through gradient-based training. For a two-layer transformer trained on a permutation-symmetric DFS curriculum, we analyze the gradient-flow dynamics of the relevant attention parameters. The dynamics preserve a low-dimensional invariant structure, and the resulting model generalizes from the training curriculum to unseen graphs and DFS traces. Thus, the construction is not only expressive: it can be learned under a mathematically analyzable training process.
Together, these results provide a mechanistic account of transformer-based graph search. They show that efficient DFS requires a genuine two-layer mechanism, that additional depth yields exponentially larger look-ahead, and that the corresponding search behavior can arise from gradient-based training on discrete chain-of-thought traces. More broadly, our analysis uses directed-graph path finding as a clean theoretical proxy for understanding how reasoning capabilities depend on architecture, depth, and training data.
Preliminaries
Notation. For N ∈ N N \in\mathbb{N} N ∈ N , let [ N ] = { 1 , … , N } [N] = \{1,\ldots,N\} [ N ] = { 1 , … , N } . We use A \mathcal{A} A to denote the token vocabulary, reserving V \mathcal{V} V for the vertex set of a graph. For a finite set K \mathcal{K} K , let S K \mathcal{S}_{\mathcal{K}} S K denote the set of all permutations on K \mathcal{K} K . We write τ P r o m p t \tau_{\mathrm{Prompt}} τ Prompt for a prompt, viewed as a sequence of tokens. Throughout the paper, I \boldsymbol{I} I denotes the identity matrix, and E \boldsymbol{E} E denotes an all-ones matrix of the appropriate size.
Directed graphs and permutations. A directed graph of order N N N is an ordered pair G = ( V , E ) \mathcal{G} = (\mathcal{V},\mathcal{E}) G = ( V , E ) , where V = [ N ] \mathcal{V} = [N] V = [ N ] is the vertex set and E ⊆ V × V \mathcal{E} \subseteq\mathcal{V} \times\mathcal{V} E ⊆ V × V is the edge set. A path from a source node n s n_s n s to a target node n t n_t n t is a sequence of vertices
p = ( n 1 , n 2 , … , n k ) p = (n_1,n_2,\ldots,n_k) p = ( n 1 , n 2 , … , n k )
with n 1 = n s n_1 = n_s n 1 = n s and n k = n t n_k = n_t n k = n t , such that ( n i → n i + 1 ) ∈ E (n_i \to n_{i+1}) \in\mathcal{E} ( n i → n i + 1 ) ∈ E for every i ∈ [ k − 1 ] i \in[k-1] i ∈ [ k − 1 ] . A shortest path is any such path with the minimum number of vertices. For any permutation π \pi π of V \mathcal{V} V , we define the permuted graph by
π ( G ) = ( V , π ( E ) ) , π ( E ) = { ( π ( u ) → π ( v ) ) : ( u → v ) ∈ E } , \pi(\mathcal{G}) = (\mathcal{V},\pi(\mathcal{E})), \qquad\pi(\mathcal{E}) = \{(\pi(u) \to\pi(v)) : (u \to v) \in\mathcal{E}\}, π ( G ) = ( V , π ( E )) , π ( E ) = {( π ( u ) → π ( v )) : ( u → v ) ∈ E } ,
and define the permuted path as π ( p ) = ( π ( n 1 ) , … , π ( n k ) ) \pi(p) = (\pi(n_1),\ldots,\pi(n_k)) π ( p ) = ( π ( n 1 ) , … , π ( n k )) .
Path-finding problem and depth-first search. Given a directed graph G \mathcal{G} G , a source node n s n_s n s , and a target node n t n_t n t , the path-finding problem asks for a path from n s n_s n s to n t n_t n t , if one exists. Depth-first search (DFS) is a recursive search procedure for this problem. Starting from n s n_s n s , DFS repeatedly explores outgoing edges from the current node n n n . If there exists an unvisited child v v v with ( n → v ) ∈ E (n \to v) \in\mathcal{E} ( n → v ) ∈ E , the search moves to v v v , marks it as visited to avoid cycles, and continues recursively. If no unvisited child remains, the search backtracks to the parent of n n n in the current search stack and explores alternative branches. This forward-and-backtracking process terminates either when n t n_t n t is found, yielding a valid n s → n t n_s \to n_t n s → n t path, or when the search returns to n s n_s n s after exhausting all reachable branches.
Multi-head transformer architecture. An L L L -layer transformer with H H H attention heads is a sequence-to-sequence model. Given an input sequence X = [ x 1 , … , x T ] ∈ R d × T X = [x_1,\ldots,x_T] \in\mathbb{R}^{d \times T} X = [ x 1 , … , x T ] ∈ R d × T , the model computes hidden states X ( l ) ∈ R d × T X^{(l)} \in\mathbb{R}^{d \times T} X ( l ) ∈ R d × T as follows. Let X ( 1 ) = X X^{(1)} = X X ( 1 ) = X , and for every l ∈ [ L ] l \in[L] l ∈ [ L ] ,
X ( l + 1 ) = X ( l ) + σ ( Atten ( l ) ( X ( l ) ) ) , (1) X^{(l+1)} = X^{(l)} + \sigma\left(\operatorname{Atten}^{(l)}(X^{(l)})\right), \tag*{(1)} X ( l + 1 ) = X ( l ) + σ ( Atten ( l ) ( X ( l ) ) ) , (1)
Atten ( l ) ( X ( l ) ) = W O ( l ) [ Head 1 ( l ) ( X ( l ) ) ⋮ Head H ( l ) ( X ( l ) ) ] , (2) \operatorname{Atten}^{(l)}(X^{(l)}) = W_O^{(l)}
\begin{bmatrix}
\operatorname{Head}_1^{(l)}(X^{(l)}) \\
\vdots\\
\operatorname{Head}_H^{(l)}(X^{(l)})
\end{bmatrix}, \tag*{(2)} Atten ( l ) ( X ( l ) ) = W O ( l ) Head 1 ( l ) ( X ( l ) ) ⋮ Head H ( l ) ( X ( l ) ) , (2)
Head h ( l ) ( X ( l ) ) = W V ( l , h ) X ( l ) ϕ ( l ) ( ( X ( l ) ) T ( W K ( l , h ) ) T W Q ( l , h ) X ( l ) ) . (3) \operatorname{Head}_h^{(l)}(X^{(l)}) = W_V^{(l,h)} X^{(l)} \phi^{(l)}\left((X^{(l)})^{T}(W_K^{(l,h)})^{T}W_Q^{(l,h)}X^{(l)}\right). \tag*{(3)} Head h ( l ) ( X ( l ) ) = W V ( l , h ) X ( l ) ϕ ( l ) ( ( X ( l ) ) T ( W K ( l , h ) ) T W Q ( l , h ) X ( l ) ) . (3)
Here σ \sigma σ is an element-wise activation function; throughout, we take σ ( x ) = ReLU ( x ) = max { 0 , x } \sigma(x) = \operatorname{ReLU}(x) = \max\{0,x\} σ ( x ) = ReLU ( x ) = max { 0 , x } . The map Atten ( l ) \operatorname{Atten}^{(l)} Atten ( l ) denotes the attention block at layer l l l , W O ( l ) W_{\mathrm{O}}^{(l)} W O ( l ) is the output projection, W V ( l , h ) W_{\mathrm{V}}^{(l,h)} W V ( l , h ) , W K ( l , h ) W_{\mathrm{K}}^{(l,h)} W K ( l , h ) , and W Q ( l , h ) W_{\mathrm{Q}}^{(l,h)} W Q ( l , h ) are the value, key, and query matrices for head h h h , and ϕ ( l ) \phi^{(l)} ϕ ( l ) is an element-wise attention activation, possibly with normalization.
The transformer output logits are
TF ( X ; θ ) = W R T X ( L + 1 ) , (4) \operatorname{TF}(X;\theta) = W_{\mathrm{R}}^{T}X^{(L+1)}, \tag*{(4)} TF ( X ; θ ) = W R T X ( L + 1 ) , (4)
where θ \theta θ denotes the trainable parameters and W R ∈ R d × ∣ A ∣ W_{\mathrm{R}} \in\mathbb{R}^{d\times|\mathcal{A}|} W R ∈ R d × ∣ A ∣ is the readout matrix. The next-token distribution is the softmax over these logits:
p θ ( v ; X ) = exp ( TF ( X ; θ ) v ) ∑ v ′ ∈ A exp ( TF ( X ; θ ) v ′ ) , (5) p_{\theta}(v;X) = \frac{\exp(\operatorname{TF}(X;\theta)_v)}{\sum_{v'\in\mathcal{A}}\exp(\operatorname{TF}(X;\theta)_{v'})}, \tag*{(5)} p θ ( v ; X ) = ∑ v ′ ∈ A exp ( TF ( X ; θ ) v ′ ) exp ( TF ( X ; θ ) v ) , (5)
where TF ( X ; θ ) v \operatorname{TF}(X;\theta)_v TF ( X ; θ ) v is the logit assigned to token v v v .
Prompt format and chain of thought. Following Zhu et al. [2025b,a], the transformer path-finding prompt has three parts: (1) a beginning-of-sequence token <s>; (2) a graph-description segment containing M M M edges, where each edge is represented by a source token s i s_i s i , a target token t i t_i t i , and a special edge token <e>; and (3) a task-description segment containing the target token n T a r n_{\mathrm{Tar}} n Tar , a start-reasoning token <r>, and the root token n R o o t n_{\mathrm{Root}} n Root . We denote the full prompt by τ P r o m p t \tau_{\mathrm{Prompt}} τ Prompt , i.e.,
τ P r o m p t = ⟨ s ⟩ , s 1 , t 1 , ⟨ e ⟩ , ⋯ , s M , t M , ⟨ e ⟩ , n T a r , ⟨ r ⟩ , n R o o t . (6) \tau_{\mathrm{Prompt}} = \langle\text{s}\rangle, s_1, t_1, \langle\text{e}\rangle, \cdots, s_M, t_M, \langle\text{e}\rangle, n_{\mathrm{Tar}}, \langle\text{r}\rangle, n_{\mathrm{Root}}. \tag*{(6)} τ Prompt = ⟨ s ⟩ , s 1 , t 1 , ⟨ e ⟩ , ⋯ , s M , t M , ⟨ e ⟩ , n Tar , ⟨ r ⟩ , n Root . (6)
Starting from n R o o t n_{\mathrm{Root}} n Root , the transformer predicts tokens autoregressively, generating a chain-of-thought (CoT) trace n 1 , n 2 , … , n T n_1, n_2, \ldots, n_T n 1 , n 2 , … , n T . Generation terminates when the model reaches the target n T a r n_{\mathrm{Tar}} n Tar , or when it returns to the root n R o o t n_{\mathrm{Root}} n Root after all of the root’s child nodes have been explored.
Expressivity
Although Merrill and Sabharwal [ [22 ] showed that constant-depth transformers can perform DFS, the minimal architectural requirements remain unclear. In this section, we first prove that any one-layer transformer that performs DFS on directed graphs of order N N N must have Ω ( N 2 ) \Omega(N^2) Ω ( N 2 ) representation size, i.e., heads × \times × width × \times × precision, in the dense-graph regime. This lower bound shows that at least two layers are needed for DFS with sub-quadratic representation size. We then give a two-layer construction with near-linear representation size, and finally show that additional layers provide an exponential improvement in look-ahead depth.
One-layer lower bound for DFS child selection
To formalize the size measure, we first specify the numerical precision of a transformer.
Definition 3.1 (p p p -bit precision). A transformer uses p p p bits of precision if all coordinates produced by W K W_{\mathrm{K}} W K , W Q W_{\mathrm{Q}} W Q , W V W_{\mathrm{V}} W V , W O W_{\mathrm{O}} W O , and the attention heads, after rounding, can be represented with p p p bits without changing the computed input-output function. As in Sanford et al. [ [28 ] , attention sums are accumulated in p p p -bit fixed-point arithmetic: for an input of length T T T , every unnormalized attention weight w j w_j w j and every summand w j W V x j w_jW_{\mathrm{V}}x_j w j W V x j of an attention numerator is rounded to a p p p -bit fixed-point number, and p ≥ log 2 T p \ge\log_2 T p ≥ log 2 T . Consequently, the partial sum of an attention numerator or normalizer over any subset of positions is exactly representable with p + ⌈ log 2 T ⌉ = O ( p ) p+\lceil\log_2 T\rceil= O(p) p + ⌈ log 2 T ⌉ = O ( p ) bits per coordinate. The value dimension of every head is at most d d d .
Size measure. As in Sanford et al. [ [28 ] , we quantify a transformer layer by H d p Hdp H d p , the number of heads times the embedding dimension times the precision, and call this its representation size ; it is the quantity that Sanford et al. [ [28 ] call simply the size , and we use the longer name because it is a measure of width and precision rather than of parameters or running time. It is the number of bits that one layer of attention can deliver to a single position, which is exactly the quantity controlled by the communication-complexity argument below, and we claim no separation in parameter count or running time. Dense d × d d \times d d × d projection matrices have Θ ( d 2 ) = Θ ( N 2 ) \Theta(d^2) = \Theta(N^2) Θ ( d 2 ) = Θ ( N 2 ) entries in the one-layer and in the two-layer setting alike (the matrices of our construction are built from identity and all-ones blocks, but they are still N × N N \times N N × N ), and on a dense graph the prompt itself has length Θ ( N 2 ) \Theta(N^2) Θ ( N 2 ) , so any model processes Θ ( N 2 ) \Theta(N^2) Θ ( N 2 ) tokens. The separation below is therefore a statement about depth versus width: a single attention layer must route Ω ( N 2 ) \Omega(N^2) Ω ( N 2 ) bits to the output position, whereas with two layers O ( N log N ) O(N \log N) O ( N log N ) bits per layer suffice, which is within a logarithmic factor of the Ω ( N ) \Omega(N) Ω ( N ) bits needed to represent the visited set at all (Section B.3).
We next define a restricted task that isolates the retrieval of an outgoing child.
Definition 3.2 (Unique outgoing-child task). Fix N , M ∈ N N, M \in\mathbb{N} N , M ∈ N . Let G = ( [ N ] , E ) G = ([N], \mathcal{E}) G = ([ N ] , E ) be a directed graph with exactly M M M distinct edges, E = { ( s i → t i ) : i ∈ [ M ] } \mathcal{E} = \{(s_i \to t_i) : i \in[M]\} E = {( s i → t i ) : i ∈ [ M ]} . The prompt is < < < s> > > , s 1 , t 1 , < s_1, t_1, < s 1 , t 1 , < e> > > , … \ldots … , s M , t M , < s_M, t_M, < s M , t M , < e> > > , q q q , an enumeration of E \mathcal{E} E followed by a query node q q q , under the promise that exactly one edge has source q q q . The desired output is the target of this unique outgoing edge.
In the hard instances constructed below the unique outgoing edge of q q q is not a self-loop, so its target is the mandatory first DFS move from q q q , and the reduction applies to the full DFS next-step task as well.
Theorem 3.3 (One-layer representation-size lower bound). Let M ≥ 1 M \ge1 M ≥ 1 and N ≥ 3 ⌈ M ⌉ + 1 N \ge3\lceil\sqrt{M}\rceil+ 1 N ≥ 3 ⌈ M ⌉ + 1 . Fix any scalar weighting function ϕ : R → R \phi: \mathbb{R} \to\mathbb{R} ϕ : R → R . Any one-layer transformer with the architecture of § 2 , with element-wise attention activation ϕ \phi ϕ , either normalized or unnormalized, with H H H heads, embedding dimension d d d , and p p p -bit precision that is correct on every valid instance of the unique outgoing-child task whose graph has vertex set [ N ] [N] [ N ] and exactly M M M distinct edges satisfies H d p = Ω ( M ) Hdp = \Omega(M) H d p = Ω ( M ) .
Proof. See Section B.1.
The proof adapts the communication-complexity argument for one-layer lower bounds on induction heads [28 ] : we reduce from INDEX [15 ] by placing the M M M records in distinct cells of an m × m m \times m m × m grid, m = ⌈ M ⌉ m = \lceil\sqrt{M}\rceil m = ⌈ M ⌉ , so that the source of a record encodes its row and the target encodes its column and Alice’s bit; Bob changes only the source of the selected record to the query vertex q q q . This yields exactly M M M distinct edges on 3 m + 1 3m + 1 3 m + 1 vertices, and since Alice controls only the target-token positions while the final query is fixed, each head contributes an O ( d p ) O(dp) O ( d p ) -bit summary to a one-way protocol.
Corollary 3.4 (Dense-graph lower bound). For N ≥ 4 N \ge4 N ≥ 4 , set m N = ⌊ ( N − 1 ) / 3 ⌋ m_N = \lfloor(N - 1)/3\rfloor m N = ⌊( N − 1 ) /3 ⌋ and M N = m N 2 M_N = m_N^2 M N = m N 2 . Any one-layer transformer in the architecture of Theorem 3.3 that is correct on every valid instance of the unique outgoing-child task whose graph has vertex set [ N ] [N] [ N ] and exactly M N M_N M N distinct edges requires H d p = Ω ( N 2 ) Hdp = \Omega(N^2) H d p = Ω ( N 2 ) . Consequently, any one-layer transformer in this architecture that solves the full DFS next-step task on all N N N -vertex directed graphs requires Ω ( N 2 ) \Omega(N^2) Ω ( N 2 ) representation size in the worst case, already on a family with Θ ( N 2 ) \Theta(N^2) Θ ( N 2 ) distinct edges.
Proof. See Section B.2.
Two-layer near-linear construction for exact DFS
Token embeddings and the first layer. Each token is embedded in four N N N -dimensional node blocks (plus a fixed positional component): edge endpoints s i , t i s_i,t_i s i , t i and the target n T a r n_{\mathrm{Tar}} n Tar place their one-hot vector in the fourth block, the root and the CoT tokens place theirs in the first block, and the special tokens are zero (Section B.3). The O ( N ) O(N) O ( N ) dimension is within a logarithmic factor of what exact DFS requires, since the model must distinguish 2 N 2^N 2 N visited sets (Section B.3). The first layer only copies information from the preceding token.
Lemma 3.5 (Predecessor-copy layer, informal version of Theorem B.1). Assume that the input embeddings are augmented with a sinusoidal positional encoding, or any positional encoding that permits a constant-size relative-position selector. Then, for every leakage parameter δ ∈ ( 0 , 1 / 8 ) \delta\in(0,1/8) δ ∈ ( 0 , 1/8 ) , there exists a one-layer transformer block with O ( 1 ) O(1) O ( 1 ) attention heads, embedding dimension O ( N ) O(N) O ( N ) , and parameter precision O ( log N ) O(\log N) O ( log N ) that copies the first block of the preceding token into the second block and its fourth block into the third block, each one-hot vector scaled by a factor in [ 1 − 2 δ , 1 ] [1-2\delta,1] [ 1 − 2 δ , 1 ] and all other copied coordinates exactly zero after the ReLU.
Concretely, up to the scalar factors in [ 1 − 2 δ , 1 ] [1-2\delta,1] [ 1 − 2 δ , 1 ] on the copied one-hot vectors, the hidden states entering the second layer are
Edge source s i : ( 0 N 0 N 0 N s i ) , Edge target t i : ( 0 N 0 N s i t i ) , <s> : ( 0 N 0 N 0 N 0 N ) , <e> : ( 0 N 0 N t i 0 N ) , Root n R o o t : ( n R o o t 0 N 0 N 0 N ) , Target n T a r : ( 0 N 0 N 0 N n T a r ) , <r> : ( 0 N 0 N n T a r 0 N ) , CoT step n t : ( n t n t − 1 0 N 0 N ) , (7) \begin{aligned}
\text{Edge source }s_i &: \begin{pmatrix} 0_N \\ 0_N \\ 0_N \\ s_i \end{pmatrix}, \qquad
\text{Edge target }t_i &: \begin{pmatrix} 0_N \\ 0_N \\ s_i \\ t_i \end{pmatrix}, \qquad
\text{\textless s\textgreater} &: \begin{pmatrix} 0_N \\ 0_N \\ 0_N \\ 0_N \end{pmatrix}, \qquad
\text{\textless e\textgreater} &: \begin{pmatrix} 0_N \\ 0_N \\ t_i \\ 0_N \end{pmatrix}, \\
\text{Root }n_{\mathrm{Root}} &: \begin{pmatrix} n_{\mathrm{Root}} \\ 0_N \\ 0_N \\ 0_N \end{pmatrix}, \qquad
\text{Target }n_{\mathrm{Tar}} &: \begin{pmatrix} 0_N \\ 0_N \\ 0_N \\ n_{\mathrm{Tar}} \end{pmatrix}, \qquad
\text{\textless r\textgreater} &: \begin{pmatrix} 0_N \\ 0_N \\ n_{\mathrm{Tar}} \\ 0_N \end{pmatrix}, \qquad
\text{CoT step }n_t &: \begin{pmatrix} n_t \\ n_{t-1} \\ 0_N \\ 0_N \end{pmatrix}, \tag*{(7)}
\end{aligned} Edge source s i Root n Root : 0 N 0 N 0 N s i , Edge target t i : n Root 0 N 0 N 0 N , Target n Tar : 0 N 0 N s i t i , <s> : 0 N 0 N 0 N n Tar , <r> : 0 N 0 N 0 N 0 N , <e> : 0 N 0 N n Tar 0 N , CoT step n t : 0 N 0 N t i 0 N , : n t n t − 1 0 N 0 N , (7)
with n 0 = n R o o t n_0=n_{\mathrm{Root}} n 0 = n Root . Every edge-target token thus exposes its edge ( s i → t i ) (s_i \to t_i) ( s i → t i ) in the last two blocks, and every CoT token exposes the transition n t − 1 → n t n_{t-1} \to n_t n t − 1 → n t in the first two.
Second layer. The second layer uses two heads with linear attention activation, ϕ ( 2 ) ( x ) = x \phi^{(2)}(x)=x ϕ ( 2 ) ( x ) = x , and the weights
W K Q ( 2 , 1 ) = ( c 1 E 0 0 0 0 0 0 0 c 1 I 0 0 0 0 0 0 0 ) , W K Q ( 2 , 2 ) = ( c 2 I 0 0 0 c 2 I 0 0 0 0 0 0 0 0 0 0 0 ) , W V ( 2 , 1 ) = ( − I 0 0 I ) T , W V ( 2 , 2 ) = ( − I I 0 0 ) T , W O ( 2 ) = ( 0 0 0 0 I 0 0 I ) , W R = ( 0 0 I I ) T , (8) \begin{aligned}
W_{KQ}^{(2,1)} &=
\begin{pmatrix}
c_1E & 0 & 0 & 0 \\
0 & 0 & 0 & 0 \\
c_1I & 0 & 0 & 0 \\
0 & 0 & 0 & 0
\end{pmatrix},
\qquad
W_{KQ}^{(2,2)} =
\begin{pmatrix}
c_2I & 0 & 0 & 0 \\
c_2I & 0 & 0 & 0 \\
0 & 0 & 0 & 0 \\
0 & 0 & 0 & 0
\end{pmatrix}, \\
W_V^{(2,1)} &=
\begin{pmatrix}
-I \\
0 \\
0 \\
I
\end{pmatrix}^{T},
\qquad
W_V^{(2,2)} =
\begin{pmatrix}
-I \\
I \\
0 \\
0
\end{pmatrix}^{T},
\qquad
W_O^{(2)} =
\begin{pmatrix}
0 & 0 \\
0 & 0 \\
I & 0 \\
0 & I
\end{pmatrix},
\qquad
W_R=(0\ 0\ I\ I)^T, \tag*{(8)}
\end{aligned} W K Q ( 2 , 1 ) W V ( 2 , 1 ) = c 1 E 0 c 1 I 0 0 0 0 0 0 0 0 0 0 0 0 0 , W K Q ( 2 , 2 ) = c 2 I c 2 I 0 0 0 0 0 0 0 0 0 0 0 0 0 0 , = − I 0 0 I T , W V ( 2 , 2 ) = − I I 0 0 T , W O ( 2 ) = 0 0 I 0 0 0 0 I , W R = ( 0 0 I I ) T , (8)
where W K Q ( l , h ) = ( W K ( l , h ) ) T W Q ( l , h ) W_{KQ}^{(l,h)}=(W_K^{(l,h)})^T W_Q^{(l,h)} W K Q ( l , h ) = ( W K ( l , h ) ) T W Q ( l , h ) is the aggregated key-query matrix, all matrices are displayed restricted to the four node blocks and vanish on the constant and positional coordinates, W R W_R W R has zero columns for the special tokens, and c 1 > c 2 > 0 c_1>c_2>0 c 1 > c 2 > 0 are constants. The leakage parameter of the first layer is chosen as
δ ≤ c 1 − c 2 4 c 1 , (9) \delta\le\frac{c_1-c_2}{4c_1}, \tag*{(9)} δ ≤ 4 c 1 c 1 − c 2 , (9)
so that the first-layer copies, whose scalar factors lie in [ 1 − 2 δ , 1 ] [1 - 2\delta, 1] [ 1 − 2 δ , 1 ] , cannot reorder the two scores below. The two heads implement the two DFS actions. The forward head matches, for the current node n t n_t n t , the edge tokens with source n t n_t n t and writes their destinations into the readout block, while subtracting every node that already occurs in the CoT trace; an unvisited child thus receives score ≈ c 1 \approx c_1 ≈ c 1 and a visited child is canceled. The backtracking head attends to the CoT transitions entering and leaving n t n_t n t ; completed excursions cancel and the tree-edge arrival leaves score ≈ c 2 \approx c_2 ≈ c 2 on the stack parent. The backtracking candidate is therefore always available but is dominated whenever an unvisited child exists.
A CoT trace τ = ( n 0 , n 1 , … , n t ) \tau= (n_0, n_1, \ldots, n_t) τ = ( n 0 , n 1 , … , n t ) with n 0 = n R o o t n_0 = n_{\mathrm{Root}} n 0 = n Root is a valid DFS trace if every step follows the DFS rule of Section 2, and it is nonterminated if DFS has a valid next move from it, i.e., n t ≠ n T a r n_t \ne n_{\mathrm{Tar}} n t = n Tar and n t n_t n t either has an unvisited child or is not the root.
Theorem 3.6 (Two-layer transformers can perform DFS with near-linear representation size). Suppose the first layer is constructed as in Theorem B.1 with a leakage parameter δ \delta δ satisfying (9) , and the second layer has weights (8) . Then the resulting two-layer transformer has O ( 1 ) O(1) O ( 1 ) heads per layer, embedding dimension O ( N ) O(N) O ( N ) , and O ( log N ) O(\log N) O ( log N ) -bit precision, hence representation size O ( N log N ) O(N \log N) O ( N log N ) , and for any graph G = ( V = [ N ] , E ) \mathcal{G} = (\mathcal{V} = [N], \mathcal{E}) G = ( V = [ N ] , E ) and any valid nonterminated DFS trace τ \tau τ on G \mathcal{G} G , every token in arg max v ∈ V p θ ( v ; τ P r o m p t , τ ) \operatorname*{arg\,max}_{v \in\mathcal{V}} p_\theta(v; \tau_{\mathrm{Prompt}}, \tau) arg max v ∈ V p θ ( v ; τ Prompt , τ ) is a valid DFS move.
Proof sketch. The representation-size bound is that of Theorem B.1, since the second-layer weights are constants. Fix a valid nonterminated trace with current node n t n_t n t . By (7) , the forward head assigns score c 1 λ c_1\lambda c 1 λ with λ ∈ [ 1 − 2 δ , 1 ] \lambda\in[1 - 2\delta, 1] λ ∈ [ 1 − 2 δ , 1 ] to each edge-target token with source n t n_t n t and score exactly c 1 c_1 c 1 to each CoT token, and its value map adds the destination of the former and subtracts the node of the latter; before the ReLU, an unvisited child of n t n_t n t therefore has forward score at least c 1 ( 1 − 2 δ ) c_1(1 - 2\delta) c 1 ( 1 − 2 δ ) , while a visited child and a non-child have score at most 0. The backtracking head attends to the CoT tokens that enter or leave n t n_t n t ; as n t n_t n t is on top of the stack, it has left only by forward moves, so each completed excursion to a child yields a departure and a return term that cancel up to the leakage (at most 2 δ c 2 2\delta c_2 2 δ c 2 remains on that child), while the stack parent receives only the tree-edge arrival, between c 2 ( 1 − 2 δ ) c_2(1 - 2\delta) c 2 ( 1 − 2 δ ) and c 2 c_2 c 2 . After the ReLU and the readout, every unvisited child has logit at least c 1 ( 1 − 2 δ ) c_1(1 - 2\delta) c 1 ( 1 − 2 δ ) , the stack parent has logit in [ c 2 ( 1 − 2 δ ) , c 2 ] [c_2(1 - 2\delta), c_2] [ c 2 ( 1 − 2 δ ) , c 2 ] , and every other vertex has logit at most 2 δ c 2 2\delta c_2 2 δ c 2 ; by (9) and δ < 1 / 4 \delta< 1/4 δ < 1/4 these three bands are ordered, so every maximizer is an unvisited child when one exists and the stack parent otherwise. Full details are in Section B.5. □
Multi-layer look-ahead construction and exponential depth advantage
Additional layers can expand the bounded reachability information available before each DFS move: after an explicit initialization, each propagation layer doubles the radius represented on the existing edge records.
Definition 3.7 (L L L -look-ahead DFS). Let L ≥ 1 L \ge1 L ≥ 1 be an integer and fix the outgoing-edge order by increasing destination label. Given a valid nonterminated DFS state with current node u u u , target n T a r n_{\mathrm{Tar}} n Tar , and visited set S S S , first consider the unvisited children v v v with ( u → v ) ∈ E (u \to v) \in\mathcal{E} ( u → v ) ∈ E and dist G ( v , n T a r ) ≤ L − 1 \operatorname{dist}_{\mathcal{G}}(v, n_{\mathrm{Tar}}) \le L - 1 dist G ( v , n Tar ) ≤ L − 1 . If this set is nonempty, move to its first vertex in the fixed order. Otherwise, apply the ordinary DFS rule: move to the first unvisited child if one exists, and otherwise backtrack to the parent in the current DFS stack.
Here distance is measured in the original graph and a zero-length path is allowed; the rule tests a Boolean bounded-reachability predicate and does not require a qualifying child to minimize distance.
Theorem 3.8 (Exponential look-ahead depth advantage). For every integer k ≥ 3 k \ge3 k ≥ 3 , there exists a k k k -*layer transformer with unmasked attention, O ( 1 ) O(1) O ( 1 ) heads per layer, embedding dimension O ( N ) O(N) O ( N ) , and O k ( log ( N + 1 ) ) O_k(\log(N+1)) O k ( log ( N + 1 )) -*bit fixed-point precision, whose token embeddings are augmented with the positional encoding of Theorem B.1 together with a constant coordinate, a beginning-of-sequence indicator, and a period-three position indicator (Section B.6), that implements L L L -look-ahead DFS with L = 1 + 2 k − 3 L = 1 + 2^{k-3} L = 1 + 2 k − 3 . Its representation size is O k ( N log ( N + 1 ) ) O_k(N\log(N+1)) O k ( N log ( N + 1 )) , where the subscript indicates that the constant may depend on k k k .
Proof sketch. The first layer creates persistent source and destination records on the edge-target positions, copies the goal, and prepares the predecessor information used by ordinary DFS. In the second layer, two linear-attention heads aggregate outgoing neighbors, one querying with the source identifier of each edge record and one with its destination identifier, which initializes source and child reachability memories whose supports are the sets reachable within one step from u u u and v v v for the record ( u → v ) (u \to v) ( u → v ) . Each of the next k − 3 k-3 k − 3 layers doubles both radii by querying source identifiers with the current memories; the memories are nonnegative counts and only their supports matter. The final layer combines the goal coordinate of the child memory, the outgoing adjacency of the current node, and the visited-node counts; two fresh ReLU blocks and a linear readout implement the eligibility test, followed by the fixed child order and the ordinary DFS fallback. The complete construction appears in Section B.6.
Thus depth gives an exponential increase in the radius of the bounded-reachability test. This Boolean test alone does not imply shortest-path selection or an exponential improvement in worst-case search efficiency; in Section B.6 we state the efficiency bound separately for a stronger policy that selects a child of minimum residual distance.
Training Dynamics
In this section we consider the training dynamics of the two-layer model constructed above. Since the predecessor-copy first layer is well studied [2 , 28 ] (Zhu et al. 2025b), we focus on the second layer, whose two heads implement the forward and the backtracking move. The core questions are how each head learns to attend to the right tokens and how the competition between the two heads evolves during training. We therefore fix W O , W V W_O,W_V W O , W V as in the construction and train the key-query matrices
W K Q ( 2 , 1 ) = ( A 1 0 0 0 0 0 0 0 A 2 0 0 0 0 0 0 0 ) , W K Q ( 2 , 2 ) = ( B 1 0 0 0 B 2 0 0 0 0 0 0 0 0 0 0 0 ) , (10) W_{KQ}^{(2,1)} =
\begin{pmatrix}
A_1 & 0 & 0 & 0 \\
0 & 0 & 0 & 0 \\
A_2 & 0 & 0 & 0 \\
0 & 0 & 0 & 0
\end{pmatrix},
\qquad
W_{KQ}^{(2,2)} =
\begin{pmatrix}
B_1 & 0 & 0 & 0 \\
B_2 & 0 & 0 & 0 \\
0 & 0 & 0 & 0 \\
0 & 0 & 0 & 0
\end{pmatrix}, \tag*{(10)} W K Q ( 2 , 1 ) = A 1 0 A 2 0 0 0 0 0 0 0 0 0 0 0 0 0 , W K Q ( 2 , 2 ) = B 1 B 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 , (10)
where A 1 , A 2 , B 1 , B 2 ∈ R N × N A_1,A_2,B_1,B_2 \in\mathbb{R}^{N \times N} A 1 , A 2 , B 1 , B 2 ∈ R N × N are trainable. When decoding we use the dynamic readout W R ( n t ) = ( 0 0 D n t D n t ) T W_R(n_t) = (0\ \ 0\ \ D_{n_t}\ \ D_{n_t})^T W R ( n t ) = ( 0 0 D n t D n t ) T with D n t = diag ( I n t − 1 , 0 , I N − n t ) D_{n_t} = \operatorname{diag}(I_{n_t-1},0,I_{N-n_t}) D n t = diag ( I n t − 1 , 0 , I N − n t ) , so that the logit of the current node n t n_t n t is 0 0 0 while n t n_t n t remains a softmax candidate with weight e 0 = 1 e^0 = 1 e 0 = 1 . Throughout, p θ p_\theta p θ is the softmax over the N N N vertex tokens (the special tokens are never admissible moves), the same readout is used for training and for the generalization loss below, and, since the loss is only piecewise smooth, gradient flow is understood in the Filippov sense (θ ˙ ∈ − α ∂ L ( θ ) \dot{\theta} \in-\alpha\partial\mathcal{L}(\theta) θ ˙ ∈ − α ∂ L ( θ ) with Clarke’s generalized gradient; on a switching surface the velocity is a convex combination of the one-sided fields).
Training data. All training data share a common graph skeleton G 0 = ( [ N ] , E 0 ) \mathcal{G}_0 = ([N], \mathcal{E}_0) G 0 = ([ N ] , E 0 ) with N ≥ 7 N \ge7 N ≥ 7 ,
E 0 = { ( 1 → i ) : 2 ≤ i ≤ K + 1 } ∪ { ( N − 1 → N − 2 ) , ( N − 2 → N − 3 ) , ( N − 3 → 1 ) } , (11) \mathcal{E}_0 = \{(1 \to i): 2 \le i \le K + 1\} \cup\{(N - 1 \to N - 2),(N - 2 \to N - 3),(N - 3 \to1)\}, \tag*{(11)} E 0 = {( 1 → i ) : 2 ≤ i ≤ K + 1 } ∪ {( N − 1 → N − 2 ) , ( N − 2 → N − 3 ) , ( N − 3 → 1 )} , (11)
where 2 ≤ K ≤ N − 5 2 \le K \le N - 5 2 ≤ K ≤ N − 5 , so that the children 2 , … , K + 1 2,\ldots,K+1 2 , … , K + 1 of node 1 are distinct from N − 3 N-3 N − 3 , N − 2 N-2 N − 2 , N − 1 N-1 N − 1 , N N N . The canonical target and root are ( n T a r , n R o o t ) = ( N , N − 1 ) (n_{\mathrm{Tar}}, n_{\mathrm{Root}}) = (N, N - 1) ( n Tar , n Root ) = ( N , N − 1 ) , and the canonical CoT trajectory is
τ = N − 2 → N − 3 ⏟ prefix τ p r e → 1 → 2 → 1 → 3 → 1 → ⋯ → K + 1 → 1 → N − 3 ⏟ training trace τ 0 → N − 2 → N − 1. (12) \tau= \underbrace{N - 2 \to N - 3}_{\text{prefix }\tau_{\mathrm{pre}}} \to\underbrace{1 \to2 \to1 \to3 \to1 \to\cdots\to K + 1 \to1 \to N - 3}_{\text{training trace }\tau_0} \to N - 2 \to N - 1. \tag*{(12)} τ = prefix τ pre N − 2 → N − 3 → training trace τ 0 1 → 2 → 1 → 3 → 1 → ⋯ → K + 1 → 1 → N − 3 → N − 2 → N − 1. (12)
We train only on the 2 K + 2 2K+2 2 K + 2 steps of τ 0 \tau_0 τ 0 , which traverse all nodes reachable from N − 3 N-3 N − 3 , visiting the children 2 , … , K + 1 2,\ldots,K+1 2 , … , K + 1 of node 1 in turn and backtracking after each; training on the full trace leads to the same mechanism, but τ 0 \tau_0 τ 0 keeps the gradient expressions clean and isolates the coupling between the two heads that drives the asymptotics.
Training loss. We use the cross-entropy of the next token: for the canonical trajectory, ℓ τ P r o m p t , τ p r e , τ 0 = − ∑ t = 1 2 K + 2 log p θ ( τ 0 [ t ] ; τ P r o m p t , τ p r e , τ 0 [ : t − 1 ] ) \ell_{\tau_{\mathrm{Prompt}},\tau_{\mathrm{pre}},\tau_0} = -\sum_{t=1}^{2K+2}\log p_\theta(\tau_0[t];\tau_{\mathrm{Prompt}},\tau_{\mathrm{pre}},\tau_0[:t-1]) ℓ τ Prompt , τ pre , τ 0 = − ∑ t = 1 2 K + 2 log p θ ( τ 0 [ t ] ; τ Prompt , τ pre , τ 0 [ : t − 1 ]) , and the dataset loss is the permutation average
L t r a i n = E π ∼ Unif ( S V ) ℓ π ( τ P r o m p t ) , π ( τ p r e ) , π ( τ 0 ) . (13) \mathcal{L}_{\mathrm{train}} = \mathbb{E}_{\pi\sim\operatorname{Unif}(\mathcal{S}_V)} \ell_{\pi(\tau_{\mathrm{Prompt}}),\pi(\tau_{\mathrm{pre}}),\pi(\tau_0)}. \tag*{(13)} L train = E π ∼ Unif ( S V ) ℓ π ( τ Prompt ) , π ( τ pre ) , π ( τ 0 ) . (13)
L t r a i n \mathcal{L}_{\mathrm{train}} L train cannot reach 0, since different exploration branches conflict whenever n t n_t n t has several unexplored children, and it only measures performance on the training skeleton.
Generalization loss. At step t t t of a CoT trace τ \tau τ we score all admissible DFS moves at once, − log ∑ v ∈ C t ( τ ) p θ ( v ; τ P r o m p t , τ [ : t − 1 ] ) -\log\sum_{v \in C_t(\tau)} p_\theta(v;\tau_{\mathrm{Prompt}},\tau[:t-1]) − log ∑ v ∈ C t ( τ ) p θ ( v ; τ Prompt , τ [ : t − 1 ]) , where C t C_t C t is the set of unexplored children if nonempty and otherwise the singleton containing the stack parent of n t n_t n t ; this loss can be driven to 0. The generalization loss is the supremum over all directed graphs on N N N vertices and all valid nonterminated DFS traces,
L g e n = sup E , τ − ∑ t = 1 len ( τ ) log ( ∑ v ∈ C t ( τ ) p θ ( v ; τ P r o m p t , τ [ : t − 1 ] ) ) . (14) \mathcal{L}_{\mathrm{gen}} = \sup_{\mathcal{E},\tau} -\sum_{t=1}^{\operatorname{len}(\tau)} \log\left(\sum_{v \in C_t(\tau)} p_\theta(v;\tau_{\mathrm{Prompt}},\tau[:t-1])\right). \tag*{(14)} L gen = E , τ sup − t = 1 ∑ len ( τ ) log v ∈ C t ( τ ) ∑ p θ ( v ; τ Prompt , τ [ : t − 1 ]) . (14)
Since the training data is fully symmetric, the permutation-invariant parameters form an invariant set of the gradient flow.
Proposition 4.1. Let F \mathcal{F} F denote the set of parameters of the form
A 1 = a 1 I + b 1 E o f f , A 2 = a 2 I + b 2 E o f f , B 1 = g 1 I + d 1 E o f f , B 2 = g 2 I + d 2 E o f f , (15) A_1 = a_1 I + b_1 E_{\mathrm{off}},\quad A_2 = a_2 I + b_2 E_{\mathrm{off}},\quad B_1 = g_1 I + d_1 E_{\mathrm{off}},\quad B_2 = g_2 I + d_2 E_{\mathrm{off}}, \tag*{(15)} A 1 = a 1 I + b 1 E off , A 2 = a 2 I + b 2 E off , B 1 = g 1 I + d 1 E off , B 2 = g 2 I + d 2 E off , (15) where E o f f = E − I E_{\mathrm{off}} = E - I E off = E − I has all off-diagonal entries equal to 1 and zero diagonal. Then F \mathcal{F} F is invariant under the gradient flow of Equation (13) : the generalized gradient of L t r a i n \mathcal{L}_{\mathrm{train}} L train lies in F \mathcal{F} F at every point of F \mathcal{F} F , and the flow restricted to F \mathcal{F} F is a solution of the full flow, unique whenever the loss is differentiable along it. In particular, matrices initialized in F \mathcal{F} F remain of the form Equation (15) for all t ≥ 0 t \ge0 t ≥ 0 . Moreover, on F \mathcal{F} F the averaged loss equals the loss of any single permuted instance, L t r a i n ( θ ) = ℓ π ( τ P r o m p t ) , π ( τ p r e ) , π ( τ 0 ) ( θ ) \mathcal{L}_{\mathrm{train}}(\theta) = \ell_{\pi(\tau_{\mathrm{Prompt}}),\pi(\tau_{\mathrm{pre}}),\pi(\tau_0)}(\theta) L train ( θ ) = ℓ π ( τ Prompt ) , π ( τ pre ) , π ( τ 0 ) ( θ ) for every π ∈ S V \pi\in\mathcal{S}_V π ∈ S V .
Proof. See Section C.1.
We initialize in F \mathcal{F} F , with the off-diagonal entries of A 2 \mathbf{A}_2 A 2 and B 1 \mathbf{B}_1 B 1 strictly below their diagonal entries: for constants 0 < c 0 ′ < c 0 0 < c'_0 < c_0 0 < c 0 ′ < c 0 ,
A 1 = c 0 E , A 2 = c 0 I + c 0 ′ E o f f , B 1 = c 0 I + c 0 ′ E o f f , B 2 = c 0 E , (16) \mathbf{A}_1 = c_0 E,\qquad\mathbf{A}_2 = c_0 I + c'_0 E_{\mathrm{off}},\qquad\mathbf{B}_1 = c_0 I + c'_0 E_{\mathrm{off}},\qquad\mathbf{B}_2 = c_0 E, \tag*{(16)} A 1 = c 0 E , A 2 = c 0 I + c 0 ′ E off , B 1 = c 0 I + c 0 ′ E off , B 2 = c 0 E , (16)
that is, a 1 = b 1 = a 2 = g 1 = g 2 = d 2 = c 0 a_1 = b_1 = a_2 = g_1 = g_2 = d_2 = c_0 a 1 = b 1 = a 2 = g 1 = g 2 = d 2 = c 0 and b 2 = d 1 = c 0 ′ b_2 = d_1 = c'_0 b 2 = d 1 = c 0 ′ . The strict separation b 2 ( 0 ) < b 1 ( 0 ) b_2(0) < b_1(0) b 2 ( 0 ) < b 1 ( 0 ) , g 1 ( 0 ) > d 1 ( 0 ) g_1(0) > d_1(0) g 1 ( 0 ) > d 1 ( 0 ) places the initial point off the two ReLU switching surfaces b 2 = b 1 b_2 = b_1 b 2 = b 1 and g 1 = d 1 g_1 = d_1 g 1 = d 1 , on the side that the dynamics then preserve (Theorem C.2). The all-constant initialization c 0 E c_0 E c 0 E lies on these surfaces, where the vector field is discontinuous, and its trajectory depends on the convention for ReLU′ ( 0 ) '(0) ′ ( 0 ) and on ( N , K ) (N, K) ( N , K ) : with ReLU′ ( 0 ) = 0 '(0) = 0 ′ ( 0 ) = 0 the backtracking head never activates in exact arithmetic, and when N N N is large relative to K K K the trajectory first leaves towards b 2 > b 1 b_2 > b_1 b 2 > b 1 (Section C.2).
Theorem 4.2. Let A 1 , A 2 , B 1 , B 2 \mathbf{A}_1, \mathbf{A}_2, \mathbf{B}_1, \mathbf{B}_2 A 1 , A 2 , B 1 , B 2 be initialized as in Equation (16) with 0 < c 0 ′ < c 0 ≤ c ∗ N − 2 0 < c'_0 < c_0 \le c_*N^{-2} 0 < c 0 ′ < c 0 ≤ c ∗ N − 2 , where c ∗ = c ∗ ( K ) > 0 c_* = c_*(K) > 0 c ∗ = c ∗ ( K ) > 0 is the constant of Theorem C.2. Then, under gradient flow on Equation (13) with learning rate α \alpha α , the scalar parameters of Equation (15) satisfy, as t → ∞ t \to\infty t → ∞ ,
a 2 , b 1 = 2 log ( α t ) + O ( 1 ) , g 1 − d 1 = log ( α t ) + O ( 1 ) , ∣ g 1 − d 1 − g 2 + d 2 ∣ ≤ C , (17) a_2, b_1 = 2\log(\alpha t) + O(1),\qquad g_1 - d_1 = \log(\alpha t) + O(1),\qquad\lvert g_1 - d_1 - g_2 + d_2\rvert\le C, \tag*{(17)} a 2 , b 1 = 2 log ( α t ) + O ( 1 ) , g 1 − d 1 = log ( α t ) + O ( 1 ) , ∣ g 1 − d 1 − g 2 + d 2 ∣ ≤ C , (17) where C C C and the O ( 1 ) O(1) O ( 1 ) terms depend only on N N N , K K K , c 0 c_0 c 0 , and c 0 ′ c'_0 c 0 ′ . As a consequence, there exist t ∗ t_* t ∗ and C N , K C_{N,K} C N , K , depending only on N , K , c 0 , c 0 ′ N,K,c_0,c'_0 N , K , c 0 , c 0 ′ , such that for all t ≥ t ∗ / α t \ge t_*/\alpha t ≥ t ∗ / α ,
L g e n ≤ C N , K ( α t ) − N − 1 N . (18) \mathcal{L}_{\mathrm{gen}} \le C_{N,K}(\alpha t)^{-\frac{N-1}{N}}. \tag*{(18)} L gen ≤ C N , K ( α t ) − N N − 1 . (18) The exponent ( N − 1 ) / N (N-1)/N ( N − 1 ) / N in Equation (18) is sharp as t → ∞ t \to\infty t → ∞ . Forward moves and backtracking moves to a non-root parent have admissible-token loss O ( 1 / ( α t ) ) O(1/(\alpha t)) O ( 1/ ( α t )) , with constants as large as e a 2 − b 1 e^{a_2-b_1} e a 2 − b 1 , so these terms dominate L g e n \mathcal{L}_{\mathrm{gen}} L gen over long horizons in practice (Section 5). When the parent of the current node is the root, however, its logit is g 1 − d 1 + d 2 g_1-d_1+d_2 g 1 − d 1 + d 2 (Table 2), and d 2 = − 1 N log ( α t ) + O ( 1 ) d_2=-\frac{1}{N}\log(\alpha t)+O(1) d 2 = − N 1 log ( α t ) + O ( 1 ) because τ 0 \tau_0 τ 0 never backtracks to the root, so d 2 d_2 d 2 is driven only by the conserved combination g 2 + ( N − 1 ) d 2 g_2+(N-1)d_2 g 2 + ( N − 1 ) d 2 ; such a step costs ( α t ) − ( N − 1 ) / N (\alpha t)^{-(N-1)/N} ( α t ) − ( N − 1 ) / N .
Proof sketch. By Theorem 4.1, training the matrices is equivalent to training the eight scalars of (15) , whose gradients on the canonical data are explicit (Theorem C.2). The flow never moves a 1 a_1 a 1 , and within an initial phase of length T 0 = N 4 ( K + 1 ) α T_0=\frac{N}{4(K+1)\alpha} T 0 = 4 ( K + 1 ) α N it drives b 2 b_2 b 2 down to 0 0 0 and d 2 d_2 d 2 to non-positive values; from then on b 2 ≡ 0 b_2 \equiv0 b 2 ≡ 0 , since its gradient vanishes on { b 2 < 0 } \{b_2 < 0\} { b 2 < 0 } so that b 2 b_2 b 2 cannot become negative, and the trajectory stays in the region { b 1 > 0 , b 2 < b 1 , a 2 ≥ b 1 , g 1 > d 1 , g 2 ≥ d 2 } \{b_1 > 0, b_2 < b_1, a_2 \ge b_1, g_1 > d_1, g_2 \ge d_2\} { b 1 > 0 , b 2 < b 1 , a 2 ≥ b 1 , g 1 > d 1 , g 2 ≥ d 2 } . Writing x 1 = a 2 x_1 = a_2 x 1 = a 2 , x 2 = b 1 x_2 = b_1 x 2 = b 1 , y 1 = g 1 − d 1 y_1 = g_1-d_1 y 1 = g 1 − d 1 , y 2 = g 2 − d 2 y_2 = g_2-d_2 y 2 = g 2 − d 2 and u = x 1 − x 2 u=x_1-x_2 u = x 1 − x 2 , v = y 1 − y 2 v=y_1-y_2 v = y 1 − y 2 , the remaining dynamics form a closed four-variable system (Theorem C.3) in which the ReLU enters only through the indicators 1 u ≥ 0 \mathbf{1}_{u\ge0} 1 u ≥ 0 and 1 v ≥ 0 \mathbf{1}_{v\ge0} 1 v ≥ 0 ; the vector field is therefore discontinuous on the switching surfaces u = 0 u=0 u = 0 and v = 0 v=0 v = 0 , and the trajectory can slide along them (it does so along v = 0 v=0 v = 0 , see Section 5). The main steps (Theorem C.5) are: (i) the sliding variable v + v_+ v + stays bounded, which we show with a Lyapunov function of the form W = Z p W=Z^p W = Z p built from the softmax normalizers; (ii) the gaps u u u and 2 y 1 − x 1 2y_1-x_1 2 y 1 − x 1 are bounded above by barrier arguments; (iii) with the gaps controlled, the leading terms of the equations for X = x 1 X=x_1 X = x 1 and Y = y 1 Y=y_1 Y = y 1 take the form X ˙ = e − X Y ˙ = α ~ e − Y − X \dot{X}=e^{-X}\dot{Y}= \tilde{\alpha}e^{-Y-X} X ˙ = e − X Y ˙ = α ~ e − Y − X , with coefficients bounded above and below, and a logarithmic growth lemma for such pairs (Theorem C.4) shows that e 2 Y − X e^{2Y-X} e 2 Y − X stays between two positive constants and yields X = 2 log ( α t ) + O ( 1 ) X=2\log(\alpha t)+O(1) X = 2 log ( α t ) + O ( 1 ) , Y = log ( α t ) + O ( 1 ) Y=\log(\alpha t)+O(1) Y = log ( α t ) + O ( 1 ) ; the bounds on u u u and v v v transfer these rates to x 2 x_2 x 2 and y 2 y_2 y 2 . Finally, since g ˙ i = − ( N − 1 ) d ˙ i \dot{g}_i=-(N-1)\dot{d}_i g ˙ i = − ( N − 1 ) d ˙ i , the individual parameters are recovered as g 1 = N − 1 N y 1 + O ( 1 ) g_1=\frac{N-1}{N}y_1+O(1) g 1 = N N − 1 y 1 + O ( 1 ) , d 1 = − 1 N y 1 + O ( 1 ) d_1=-\frac{1}{N}y_1+O(1) d 1 = − N 1 y 1 + O ( 1 ) , and likewise for g 2 , d 2 g_2,d_2 g 2 , d 2 ((127) ). Inserting these into the node-logit tables of Section C for an arbitrary graph and valid trace bounds every admissible-token loss by O ( e − ( x 2 − y 1 ) + e − y 1 ) = O ( 1 / ( α t ) ) O(e^{-(x_2-y_1)}+e^{-y_1})=O(1/(\alpha t)) O ( e − ( x 2 − y 1 ) + e − y 1 ) = O ( 1/ ( α t )) , except at a root parent, whose logit g 1 − d 1 + d 2 = N − 1 N log ( α t ) + O ( 1 ) g_1-d_1+d_2=\frac{N-1}{N}\log(\alpha t)+O(1) g 1 − d 1 + d 2 = N N − 1 log ( α t ) + O ( 1 ) gives (18) . ■
Experiment
We validate the predictions of § 4 : the effective second-layer parameters grow logarithmically at the predicted rates, the gap variables stay bounded, and the generalization loss decays at the predicted polynomial rates. The protocol, the test set, the invariant-subspace check, and the initialization ablation are detailed in Section C.2.
Setup. We use N = 16 N=16 N = 16 , K = 8 K=8 K = 8 , the model of § 4 with trainable N × N N \times N N × N matrices A 1 A_1 A 1 , A 2 A_2 A 2 , B 1 B_1 B 1 , B 2 B_2 B 2 on top of the fixed first layer, the initialization Equation (16) with c 0 = 1 / N 2 c_0=1/N^2 c 0 = 1/ N 2 , c ~ 0 = c 0 / 2 \tilde{c}_0=c_0/2 c ~ 0 = c 0 /2 , and float64 arithmetic. Full-matrix training runs gradient descent with step size η = 0.05 \eta=0.05 η = 0.05 on a sampled version of Equation (13) (B = 128 B=128 B = 128 random vertex permutations per step, 10 7 10^7 1 0 7 steps, t = 5 × 10 5 t=5 \times10^5 t = 5 × 1 0 5 , α = 1 \alpha=1 α = 1 ). The reduced flow integrates the eight-scalar system of Theorem C.1, which is exactly the averaged gradient flow on F \mathcal{F} F , up to t = 10 9 t=10^9 t = 1 0 9 . Generalization is measured on 64,325 DFS states from 300 random directed graphs on 16 vertices (with cycles and self-loops): 39,715 forward steps, 20,388 backtracking steps to a non-root parent, and 4,222 backtracking steps to the root.
Growth rates. Figure 1 (left) compares the learned effective parameters with logarithmic fits on t ∈ [ 10 3 , 10 9 ] t \in[10^3,10^9] t ∈ [ 1 0 3 , 1 0 9 ] . For the reduced flow, a 2 a_2 a 2 and b 1 b_1 b 1 grow with slopes 1.992 and 2.000, and g 1 − d 1 g_1-d_1 g 1 − d 1 , g 2 − d 2 g_2-d_2 g 2 − d 2 with slope 0.997; full-matrix training gives 1.966, 1.996, 0.990, 0.990 on its shorter horizon. The individual parameters follow Equation (127) : g 1 g_1 g 1 , g 2 g_2 g 2 grow with slope 0.935 (predicted ( N − 1 ) / N = 0.9375 (N-1)/N=0.9375 ( N − 1 ) / N = 0.9375 ) and d 1 d_1 d 1 , d 2 d_2 d 2 with slope − 0.062 -0.062 − 0.062 (predicted − 1 / N -1/N − 1/ N ), while g 1 + ( N − 1 ) d 1 g_1+(N-1)d_1 g 1 + ( N − 1 ) d 1 is conserved to 10 − 11 10^{-11} 1 0 − 11 . As Theorem C.5 predicts, 0 ≤ a 2 − b 1 ≤ 5.1 0 \leq a_2-b_1 \leq5.1 0 ≤ a 2 − b 1 ≤ 5.1 and ∣ g 1 − d 1 − g 2 + d 2 ∣ ≤ 0.34 \lvert g_1-d_1-g_2+d_2\rvert\leq0.34 ∣ g 1 − d 1 − g 2 + d 2 ∣ ≤ 0.34 along the whole trajectory, and b 2 b_2 b 2 , d 2 d_2 d 2 become non-positive at t ≈ 0.3 t \approx0.3 t ≈ 0.3 . The forward signal thus grows at twice the logarithmic rate of the backtracking signal, which is what lets the model prefer an unexplored child whenever one exists while keeping a lower-priority backtracking candidate. With B = 128 B=128 B = 128 sampled permutations per step, the full matrices leave the invariant subspace F \mathcal{F} F of Theorem 4.1 during the initial phase, when the dynamics slide along the switching surfaces (relative distance up to 0.9 for B 1 B_1 B 1 , B 2 B_2 B 2 ), but return to within 6 × 10 − 4 6 \times10^{-4} 6 × 1 0 − 4 of it by t = 5 × 10 5 t=5 \times10^5 t = 5 × 1 0 5 , where their effective scalars agree with the reduced flow to within 0.03; larger batches (B ≥ 8192 B \ge8192 B ≥ 8192 ) keep the distance below 0.03 throughout (Section C.2).
Figure 1. Left: effective parameters against log t \log t log t for the reduced flow (lines) and full-matrix training (markers); fitted slopes 1.99, 2.00, 1.00, 1.00 against the predicted 2, 2, 1, 1. Right: admissible-token loss on the test set by step type, and its maximum L g e n \mathcal{L}_{\mathrm{gen}} L gen . Backtracking to the root decays as t − ( N − 1 ) / N = t − 0.9375 t^{-(N-1)/N}=t^{-0.9375} t − ( N − 1 ) / N = t − 0.9375 ; all other steps decay as t − 1 t^{-1} t − 1 .
Generalization. Figure 1 (right) shows the admissible-token loss by step type. On t ∈ [ 10 6 , 10 9 ] t \in[10^{6}, 10^{9}] t ∈ [ 1 0 6 , 1 0 9 ] the fitted exponents are − 1.000 -1.000 − 1.000 for forward steps, − 1.000 -1.000 − 1.000 for backtracking to a non-root parent, and − 0.9375 -0.9375 − 0.9375 for backtracking to the root, exactly the − ( N − 1 ) / N -(N-1)/N − ( N − 1 ) / N of Theorem 4.2; the mechanism of Equation (127) is visible directly, since the logit of a root parent equals g 1 − d 1 + d 2 g_1-d_1+d_2 g 1 − d 1 + d 2 to within 0.005. Repeating the reduced flow for other N N N (with K = 3 K=3 K = 3 for N = 8 N=8 N = 8 and K = 4 K=4 K = 4 otherwise) gives fitted root-backtracking exponents
all within 3 × 10 − 4 3 \times10^{-4} 3 × 1 0 − 4 of the prediction (Figure 4 in Section C.2 shows the local exponents). Over the whole observed range, however, the supremum L g e n \mathcal{L}_{\mathrm{gen}} L gen is attained by forward steps at a node that also has an edge back to its stack parent: there the parent collects g 1 − d 1 g_1-d_1 g 1 − d 1 from the backtracking head and a 2 − b 1 a_2-b_1 a 2 − b 1 from the forward head, so the loss is ≈ e − ( b 1 − ( g 1 − d 1 ) ) = Θ ( e u / t ) \approx e^{-(b_1-(g_1-d_1))}=\Theta(e^u/t) ≈ e − ( b 1 − ( g 1 − d 1 )) = Θ ( e u / t ) , a t − 1 t^{-1} t − 1 rate with the large constant e u ≈ e 5 e^u \approx e^5 e u ≈ e 5 ; the bound Equation (18) holds, but the t − ( N − 1 ) / N t^{-(N-1)/N} t − ( N − 1 ) / N term overtakes this one only far beyond any practical horizon. The same states are the last to be classified correctly, at t ≈ 2 × 10 4 t \approx2 \times10^4 t ≈ 2 × 1 0 4 , whereas all other test states are from t ≈ 40 t \approx40 t ≈ 40 on (Section C.2).
Figure 4. Local log–log slope d log L / d log t \mathrm{d}\log\mathcal{L}/\mathrm{d}\log t d log L / d log t of the root-backtracking loss for several N N N (reduced flow). Dashed lines mark the predicted exponents − ( N − 1 ) / N -(N-1)/N − ( N − 1 ) / N .
Conclusion
We gave a theoretical account of how transformers implement, scale, and learn depth-first search on directed graphs. We established a sharp depth threshold for efficient DFS: any one-layer transformer requires Ω ( N 2 ) \Omega(N^2) Ω ( N 2 ) representation size (heads × \times × width × \times × precision) on dense graphs, while a two-layer construction suffices with O ( N log N ) O(N \log N) O ( N log N ) representation size, where the first layer exposes predecessor information and the second combines competing forward and backtracking heads. Beyond this threshold, each additional layer doubles the radius of the bounded-reachability test available before every move, so the look-ahead horizon grows exponentially with depth. Finally, we proved that the two-layer mechanism emerges from gradient flow on a permutation-symmetric curriculum, with logarithmic growth rates and a polynomial generalization rate that our experiments reproduce quantitatively. Together, these results show that efficient DFS requires a genuine two-layer mechanism, that depth translates into exponentially larger look-ahead, and that such reasoning behavior can arise from gradient-based training on discrete chain-of-thought traces.
Limitations and future work. Our learnability analysis relies on a permutation-symmetric training curriculum and a fixed first layer; extending the gradient-flow analysis to end-to-end training over broader graph distributions is a natural next step. Our look-ahead construction is exact and has a fixed horizon, whereas trained large models appear to perform approximate, heuristic-driven search; closing this gap likely requires combining our discrete-CoT analysis with tools from approximate reasoning and value estimation. Finally, while DFS captures branching, memory, and backtracking, real reasoning tasks involve richer structures such as soft constraints, probabilistic transitions, and hierarchical subgoals. We hope the framework developed here—identifying depth thresholds, quantifying the gain from each additional layer, and tracking gradient-flow dynamics on structured curricula—can serve as a template for analyzing these more complex reasoning algorithms.
Acknowledgement
This work was supported by the U.S. Army Research Laboratory and the U.S. Army Research Office under Grant W911NF2010219, Office of Naval Research, and NSF. This work used Jetstream2 at Indiana University through allocation CIS251312 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) program, which is supported by National Science Foundation grants #2138259, #2138286, #2138307, #2137603, and #2138296.
Related Work
Expressive power of transformers and chain of thought. A line of work characterizes what transformers can and cannot compute as a function of depth, precision, and CoT length. Hahn [ [11 ] gives early limitations of self-attention on hierarchical and periodic structure, and Yun et al. [ [43 ] shows universal approximation in the seq-to-seq sense. Tighter circuit-complexity bounds place log-precision transformers inside uniform T C 0 \mathrm{TC}^{0} TC 0 [Merrill and Sabharwal, 2023b,a, Strobl, 2023], and Merrill and Sabharwal [ [23 ] extend these bounds with padding. On the positive side, Liu et al. [ [18 ] show that low-depth transformers can simulate finite-state automata via Krohn–Rhodes shortcuts, and Edelman et al. [ [5 ] characterize sparse-variable creation under bounded norm. CoT extends these limits: Feng et al. [ [8 ] prove that constant-size transformers with CoT can solve dynamic programming and arithmetic that constant-depth transformers cannot, Li et al. [ [17 ] show T T T CoT steps suffice to express any size-T T T circuit at constant precision, and Merrill and Sabharwal [ [22 ] give a tight characterization of decoder transformers’ expressive power as the CoT length scales. On the learning side, Huang et al. [ [13 ] analyze implicit CoT, where intermediate steps are removed during training so that the reasoning is absorbed into hidden states: a log 2 k \log_{2} k log 2 k -layer transformer trained under a curriculum that removes these steps in geometric chunks learns k k k -parity on n n n bits with poly ( n ) \operatorname{poly}(n) poly ( n ) samples and log 2 k \log_{2} k log 2 k training stages, matching the sample efficiency of explicit CoT without generating thinking tokens at inference. Our results refine this landscape along the orthogonal axis of search-horizon-per-depth : we show DFS look-ahead grows exponentially in depth at near-linear width.
Graph reasoning with transformers. The graph-reasoning testbed has emerged as a natural arena for both empirical and theoretical analysis. Empirically, NLGraph [37 ] , GraphQA [7 ] , and the path-star analyses of Bachmann and Nagarajan [ [1 ] , Frydenlund [ [9 ] expose systematic failures of LLMs on connectivity, shortest-path, and traversal tasks. Saparov et al. [ [32 ] find that small transformers trained on DAG path-finding learn to maintain a per-vertex reachable set whose support grows exponentially with depth. Saparov and He [2023], Saparov et al. [ [31 ] document analogous greedy-reasoning failures in deductive-proof settings. Theoretically, Sanford et al. [ [27 ] build a representational hierarchy of nine graph algorithms and show log-depth is necessary and sufficient for connectivity; Sanford et al. [ [29 ] relate depth to MPC rounds; and Ye[hudai] et al. [ [42 ] establish depth–width tradeoffs for graph primitives via 2-cycle communication-complexity arguments. Our work differs by isolating the algorithmic primitive (DFS, with explicit stack and backtracking) and proving a tight one-layer lower bound, a two-layer matching upper bound, and a depth–look-ahead hierarchy.
Provable training dynamics. A growing body of work analyzes gradient-based training of transformers on structured tasks. Tian et al. [ [35 ] dissect token composition in one-layer transformers; Nichani et al. [ [25 ] prove a two-layer transformer learns Markov-chain causal structure under gradient descent; Chen et al. [ [4 ] and Olsson et al. [ [26 ] provide training and circuit-level accounts of induction heads; Wang et al. [ [38 ] prove transformers (but not MLPs) provably learn sparse token selection; and Edelman et al. [ [6 ] analyze in-context Markov-chain learning. Another line explains what one-layer transformers infer beyond their training facts through the implicit bias of gradient descent: Huang et al. [ [12 ] attribute out-of-context reasoning, which underlies both generalization and hallucination after fine-tuning, to a bias toward small nuclear norm of the combined output–value matrix, and Ma et al. [ [19 ] prove that the reversal curse occurs on forward-only data but is broken when identity-bridge statements of the form “A → A ′ A \to A' A → A ′ ” are added to the training set. Closest to our setting, Yang et al. [ [41 ] prove that a one-layer multi-head transformer learns path-finding on trees via gradient descent with multi-head specialization. Trees, however, are search-trivial: there is at most one path between any pair of vertices, so neither branching nor backtracking arises. Our paper generalizes this line to arbitrary directed graphs by showing that two-layer transformers provably learn to execute DFS—the canonical algorithm that handles branching and revisitation—under gradient flow and generalize to unseen graphs. Zhu et al. [2025a] study a complementary regime: they prove that two-layer transformers learn directed-graph reachability with continuous CoT through the emergence of superposed reasoning traces. Whereas continuous CoT permits parallel exploration via embedding superposition, our discrete-CoT setting forces the model to produce a sequential search trace, and our analysis tracks how the canonical sequential-search algorithm (DFS) emerges in the parameters. Together, these two results paint a picture of how the same graph-search problem is solved differently in the continuous and discrete CoT regimes.
Search and planning in language models. Beyond the symbolic setting, several empirical works probe whether LLMs perform search internally. Lehnert et al. [ [16 ] train Searchformer to imitate A* search and improve upon it via search-dynamics bootstrapping; Valmeekam et al. [ [36 ] , Kambhampati et al. [ [14 ] document failure modes of LLMs as planners; and Stechly et al. [ [33 ] analyze iterative-refinement limitations. Saparov and He [ [30 ] show that LLMs are “greedy reasoners” that struggle when proofs branch. Mechanistic-interpretability work [26 , 24 , 3 ] identifies internal circuits underlying simple reasoning and traversal behaviors, including a backward-chaining circuit on tree path-finding that motivated Yang et al. [ [41 ] . Our theoretical results provide a complementary explanation for why transformers exhibit depth-dependent search horizons in these empirical studies: the k k k -layer DFS construction we exhibit has look-ahead horizon 1 + 2 k − 3 1 + 2^{k-3} 1 + 2 k − 3 , a depth-dependent horizon of the kind that empirical work observes, and our learnability result shows this construction is reachable by gradient-based training on appropriately curated CoT data.
Omitted proof in Section 3
Proof of Theorem 3.3
Proof. We reduce from the one-way communication problem INDEX [15 ] . Alice receives
f = ( f 1 , … , f M ) ∈ { 0 , 1 } M , (19) f = (f_1,\ldots,f_M) \in\{0,1\}^{M}, \tag*{(19)} f = ( f 1 , … , f M ) ∈ { 0 , 1 } M , (19)
and Bob receives an index i ⋆ ∈ [ M ] i^{\star} \in[M] i ⋆ ∈ [ M ] . Alice sends one message to Bob, who must output f i ⋆ f_{i^{\star}} f i ⋆ . Every deterministic one-way protocol for INDEX requires Ω ( M ) \Omega(M) Ω ( M ) bits.
Assume a one-layer transformer is correct on every valid instance of the unique outgoing-child task whose graph has vertex set [ N ] [N] [ N ] and exactly M M M distinct edges, where N ≥ 3 ⌈ M ⌉ + 1 N \ge3\lceil\sqrt{M}\rceil+ 1 N ≥ 3 ⌈ M ⌉ + 1 . We construct a protocol for INDEX.
A row–column encoding with distinct edges. Set m = ⌈ M ⌉ m = \lceil\sqrt{M}\rceil m = ⌈ M ⌉ and assign record i ∈ [ M ] i \in[M] i ∈ [ M ] to the grid cell
r ( i ) = 1 + ⌊ i − 1 m ⌋ , c ( i ) = 1 + ( ( i − 1 ) m o d m ) . (20) r(i) = 1 + \left\lfloor\frac{i-1}{m} \right\rfloor, \qquad c(i) = 1 + ((i-1) \bmod m). \tag*{(20)} r ( i ) = 1 + ⌊ m i − 1 ⌋ , c ( i ) = 1 + (( i − 1 ) mod m ) . (20)
The pairs ( r ( i ) , c ( i ) ) (r(i),c(i)) ( r ( i ) , c ( i )) are distinct and belong to [ m ] × [ m ] [m] \times[m] [ m ] × [ m ] . Choose 3 m + 1 3m+1 3 m + 1 pairwise distinct vertices in [ N ] [N] [ N ] , named
q , a 1 , … , a m , b 1 , 0 , b 1 , 1 , … , b m , 0 , b m , 1 q,\quad a_1,\ldots,a_m,\quad b_{1,0},b_{1,1},\ldots,b_{m,0},b_{m,1} q , a 1 , … , a m , b 1 , 0 , b 1 , 1 , … , b m , 0 , b m , 1 .
These names denote a fixed public labeling of the vertices. All remaining vertices, if any, are isolated. For each record i ∈ [ M ] i \in[M] i ∈ [ M ] , define
t i = b c ( i ) , f i , s i = { q , i = i ⋆ , a r ( i ) , i ≠ i ⋆ . (21) t_i=b_{c(i),f_i},\qquad s_i=
\begin{cases}
q, & i=i^\star,\\
a_{r(i)}, & i\ne i^\star.
\end{cases} \tag*{(21)} t i = b c ( i ) , f i , s i = { q , a r ( i ) , i = i ⋆ , i = i ⋆ . (21)
Let
E = { ( s i → t i ) : i ∈ [ M ] } , \mathcal{E}=\{(s_i\to t_i):i\in[M]\}, E = {( s i → t i ) : i ∈ [ M ]} ,
and list these edges in the public order 1 , … , M 1,\ldots,M 1 , … , M .
We verify that ∣ E ∣ = M |\mathcal{E}|=M ∣ E ∣ = M . The selected record is the only edge with source q q q , so it cannot coincide with any other edge. For two distinct unselected records i , j i,j i , j , if r ( i ) ≠ r ( j ) r(i)\ne r(j) r ( i ) = r ( j ) , their sources are different. If r ( i ) = r ( j ) r(i)=r(j) r ( i ) = r ( j ) , then c ( i ) ≠ c ( j ) c(i)\ne c(j) c ( i ) = c ( j ) , and their targets belong to the disjoint pairs { b c ( i ) , 0 , b c ( i ) , 1 } \{b_{c(i),0},b_{c(i),1}\} { b c ( i ) , 0 , b c ( i ) , 1 } and { b c ( j ) , 0 , b c ( j ) , 1 } \{b_{c(j),0},b_{c(j),1}\} { b c ( j ) , 0 , b c ( j ) , 1 } . Thus their targets are different. Every edge is therefore distinct. Moreover, the source vertices and target vertices are disjoint, so the construction has no self-loops.
The final query node is the fixed vertex q q q . It has exactly one outgoing edge, whose target is
t i ⋆ = b c ( i ⋆ ) , f i ⋆ . (22) t_{i^\star}=b_{c(i^\star),f_{i^\star}}. \tag*{(22)} t i ⋆ = b c ( i ⋆ ) , f i ⋆ . (22)
Since Bob knows i ⋆ i^\star i ⋆ and the public vertex labeling, this output determines f i ⋆ f_{i^\star} f i ⋆ .
One-way simulation of the final position. The prompt has total length
T = 3 M + 2. (23) T=3M+2. \tag*{(23)} T = 3 M + 2. (23)
Its final token is always q q q , at the same position, so its input embedding x T \mathbf{x}_T x T , including its positional encoding, is known to both parties and does not depend on i ⋆ i^\star i ⋆ . Partition the positions into
A = { 3 , 6 , 9 , … , 3 M } , B = [ T ] ∖ A . (24) A=\{3,6,9,\ldots,3M\},\qquad B=[T]\setminus A. \tag*{(24)} A = { 3 , 6 , 9 , … , 3 M } , B = [ T ] ∖ A . (24)
Alice controls the target-token positions A A A : each such token depends only on her input bit and its public column index. Bob knows all positions in B B B , comprising the source tokens, special tokens, and final query. In particular, the input embedding at every position in A A A is independent of Bob’s index.
Fix one attention head h h h , and write
A h = ( W K ( 1 , h ) ) ⊤ W Q ( 1 , h ) , w j ( h ) = ϕ ( x j ⊤ A h x T ) . (25) A_h=\left(W_K^{(1,h)}\right)^\top W_Q^{(1,h)},\qquad w_j^{(h)}=\phi\left(\mathbf{x}_j^\top A_h\mathbf{x}_T\right). \tag*{(25)} A h = ( W K ( 1 , h ) ) ⊤ W Q ( 1 , h ) , w j ( h ) = ϕ ( x j ⊤ A h x T ) . (25)
First consider unnormalized attention. The final column of head h h h is
Head h ( 1 ) ( X ) : , T = ∑ j = 1 T w j ( h ) W V ( 1 , h ) x j . (26) \operatorname{Head}_h^{(1)}(\mathbf{X})_{:,T}
=\sum_{j=1}^{T}w_j^{(h)}W_V^{(1,h)}\mathbf{x}_j. \tag*{(26)} Head h ( 1 ) ( X ) : , T = j = 1 ∑ T w j ( h ) W V ( 1 , h ) x j . (26)
Alice computes
S A ( h ) = ∑ j ∈ A w j ( h ) W V ( 1 , h ) x j ∈ R d (27) S_A^{(h)}=\sum_{j\in A}w_j^{(h)}W_V^{(1,h)}\mathbf{x}_j\in\mathbb{R}^d \tag*{(27)} S A ( h ) = j ∈ A ∑ w j ( h ) W V ( 1 , h ) x j ∈ R d (27)
and sends it to Bob. By the fixed-point convention in the definition of p p p -bit precision, S A ( h ) S_A^{(h)} S A ( h ) is a coordinate-wise sum of at most T T T numbers that are each p p p -bit fixed-point numbers, so it is exactly representable with p + ⌈ log 2 T ⌉ = O ( p ) p+\lceil\log_2 T\rceil=O(p) p + ⌈ log 2 T ⌉ = O ( p ) bits per coordinate, and the same holds for the partial sum over B B B below; no additional rounding is introduced by the protocol. The vector lives in the value space of the head, whose dimension is at most d d d . Bob computes
S B ( h ) = ∑ j ∈ B w j ( h ) W V ( 1 , h ) x j (28) S_B^{(h)}=\sum_{j\in B}w_j^{(h)}W_V^{(1,h)}x_j \tag*{(28)} S B ( h ) = j ∈ B ∑ w j ( h ) W V ( 1 , h ) x j (28)
and reconstructs the head output as S A ( h ) + S B ( h ) S_A^{(h)}+S_B^{(h)} S A ( h ) + S B ( h ) . The message has size O ( d p ) O(dp) O ( d p ) bits for this head. For normalized attention, the final column is
Head h ( 1 ) ( X ) : , T = S A ( h ) + S B ( h ) Z A ( h ) + Z B ( h ) (29) \operatorname{Head}_h^{(1)}(X)_{:,T}=\frac{S_A^{(h)}+S_B^{(h)}}{Z_A^{(h)}+Z_B^{(h)}} \tag*{(29)} Head h ( 1 ) ( X ) : , T = Z A ( h ) + Z B ( h ) S A ( h ) + S B ( h ) (29)
where
Z A ( h ) = ∑ j ∈ A w j ( h ) , Z B ( h ) = ∑ j ∈ B w j ( h ) . (30) Z_A^{(h)}=\sum_{j\in A}w_j^{(h)},\qquad Z_B^{(h)}=\sum_{j\in B}w_j^{(h)}. \tag*{(30)} Z A ( h ) = j ∈ A ∑ w j ( h ) , Z B ( h ) = j ∈ B ∑ w j ( h ) . (30)
Alice sends S A ( h ) S_A^{(h)} S A ( h ) and Z A ( h ) Z_A^{(h)} Z A ( h ) , each exactly representable with O ( p ) O(p) O ( p ) bits per coordinate by the same fixed-point argument, and Bob computes the corresponding quantities for B B B . The additional normalizing scalar leaves the communication at O ( d p ) O(dp) O ( d p ) bits per head.
Repeating for all H H H heads, Alice sends O ( H d p ) O(Hdp) O ( H d p ) bits. Bob reconstructs every final-column head output, applies the output matrix W O ( 1 ) W_O^{(1)} W O ( 1 ) and the residual update
X : , T ( 2 ) = x T + ReLU ( Atten ( 1 ) ( X ) : , T ) , (31) X_{:,T}^{(2)}=x_T+\operatorname{ReLU}\left(\operatorname{Atten}^{(1)}(X)_{:,T}\right), \tag*{(31)} X : , T ( 2 ) = x T + ReLU ( Atten ( 1 ) ( X ) : , T ) , (31)
and computes the final logits W R T X : , T ( 2 ) W_R^{\mathsf T}X_{:,T}^{(2)} W R T X : , T ( 2 ) . By correctness of the transformer, the predicted token is b c ( i ⋆ ) , f i ⋆ b_{c(i_\star),f_{i_\star}} b c ( i ⋆ ) , f i ⋆ , from which Bob recovers f i ⋆ f_{i_\star} f i ⋆ . Thus the transformer induces a one-way protocol for INDEX using O ( H d p ) O(Hdp) O ( H d p ) bits. The Ω ( M ) \Omega(M) Ω ( M ) communication lower bound implies
H d p = Ω ( M ) . (32) Hdp=\Omega(M). \tag*{(32)} H d p = Ω ( M ) . (32)
Proof of Theorem 3.4
Proof. We have 3 ⌈ M N ⌉ + 1 = 3 m N + 1 ≤ N 3\lceil\sqrt{M_N}\rceil+1=3m_N+1\le N 3 ⌈ M N ⌉ + 1 = 3 m N + 1 ≤ N , so Theorem 3.3 applies. Since M N = Θ ( N 2 ) M_N=\Theta(N^2) M N = Θ ( N 2 ) , it gives H d p = Ω ( N 2 ) Hdp=\Omega(N^2) H d p = Ω ( N 2 ) .
For the DFS conclusion, use the same graph family from Section B.1, take q q q as the root, and choose the fixed row vertex a 1 a_1 a 1 as the target. Insert the public target token a 1 a_1 a 1 and the public start-reasoning token <r> immediately before the final root token q q q . The initial DFS state is nonterminated and has visited set { q } \{q\} { q } . Its unique child is distinct from q q q , so every valid first DFS move must output that child. The additional task tokens are public and the final query remains fixed, so the same communication reduction applies. The target is unreachable in this family, which is permitted by the DFS task.
Token embeddings of the two-layer construction
We use the following token embeddings, excluding a fixed positional-embedding component:
Edge source s i : ( 0 N 0 N 0 N s i ) , Edge target t i : ( 0 N 0 N 0 N t i ) , ⟨ s ⟩ , ⟨ e ⟩ , ⟨ r ⟩ : ( 0 N 0 N 0 N 0 N ) , (33) \text{Edge source }s_i:\quad\begin{pmatrix}0_N\\0_N\\0_N\\s_i\end{pmatrix},\qquad\text{Edge target }t_i:\quad\begin{pmatrix}0_N\\0_N\\0_N\\t_i\end{pmatrix},\qquad\langle\text{s}\rangle,\langle\text{e}\rangle,\langle\text{r}\rangle:\quad\begin{pmatrix}0_N\\0_N\\0_N\\0_N\end{pmatrix}, \tag*{(33)} Edge source s i : 0 N 0 N 0 N s i , Edge target t i : 0 N 0 N 0 N t i , ⟨ s ⟩ , ⟨ e ⟩ , ⟨ r ⟩ : 0 N 0 N 0 N 0 N , (33)
Target n T a r : ( 0 N 0 N 0 N n T a r ) , Root n R o o t : ( n R o o t 0 N 0 N 0 N ) , CoT step n t : ( n t 0 N 0 N 0 N ) . (34) \text{Target }n_{\mathrm{Tar}}:\quad\begin{pmatrix}0_N\\0_N\\0_N\\n_{\mathrm{Tar}}\end{pmatrix},\qquad\text{Root }n_{\mathrm{Root}}:\quad\begin{pmatrix}n_{\mathrm{Root}}\\0_N\\0_N\\0_N\end{pmatrix},\qquad\text{CoT step }n_t:\quad\begin{pmatrix}n_t\\0_N\\0_N\\0_N\end{pmatrix}. \tag*{(34)} Target n Tar : 0 N 0 N 0 N n Tar , Root n Root : n Root 0 N 0 N 0 N , CoT step n t : n t 0 N 0 N 0 N . (34)
Here s i s_i s i , t i t_i t i , n T a r n_{\mathrm{Tar}} n Tar , and n t n_t n t denote the corresponding N N N -dimensional one-hot node embeddings. After the first layer of Theorem B.1, up to the scalar factors in [ 1 − 2 δ , 1 ] [1-2\delta,1] [ 1 − 2 δ , 1 ] on the copied one-hot vectors, the hidden states in X ( 2 ) \mathbf{X}^{(2)} X ( 2 ) are those displayed in Equation (7) .
On the embedding dimension. The token-embedding dimension is O ( N ) O(N) O ( N ) , which is natural for exact DFS because the algorithm must maintain and query an arbitrary visited set S ⊆ [ N ] S \subseteq[N] S ⊆ [ N ] . There are 2 N 2^N 2 N possible visited sets, so any representation that preserves enough information for exact membership queries must distinguish 2 N 2^N 2 N states. If the visited-set representation has dimension m m m and uses p p p bits of precision per coordinate, then it can encode at most 2 O ( m p ) 2^{O(mp)} 2 O ( m p ) distinguishable states, which implies m p = Ω ( N ) mp = \Omega(N) m p = Ω ( N ) . With the p = O ( log N ) p=O(\log N) p = O ( log N ) precision used in our softmax construction, this yields m = Ω ( N / log N ) m = \Omega(N/\log N) m = Ω ( N / log N ) . Thus, the choice m = O ( N ) m=O(N) m = O ( N ) is within a logarithmic factor of the information-theoretic minimum for exact visited-set representation, while keeping the construction simple through one-hot node embeddings.
Lemma B.1 (Predecessor-copy layer). Let N N N be the number of graph nodes. Let T c o t T_{\mathrm{cot}} T cot be an upper bound on the length of the CoT trace and let L max = 3 M + T c o t + 4 L_{\max}=3M+T_{\mathrm{cot}}+4 L m a x = 3 M + T cot + 4 , which bounds the total length of the prompt followed by the CoT trace (one ⟨ s ⟩ \langle\text{s}\rangle ⟨ s ⟩ token, 3 M 3M 3 M edge tokens, the three task tokens n T a r n_{\mathrm{Tar}} n Tar , ⟨ r ⟩ \langle\text{r}\rangle ⟨ r ⟩ , n R o o t n_{\mathrm{Root}} n Root , and at most T c o t T_{\mathrm{cot}} T cot CoT tokens); assume M ≤ N 2 M\leq N^2 M ≤ N 2 and T c o t ≤ poly ( N ) T_{\mathrm{cot}}\leq\operatorname{poly}(N) T cot ≤ poly ( N ) . For each position i ∈ [ L max ] i\in[L_{\max}] i ∈ [ L m a x ] , define
p i = ( cos ( i α ) sin ( i α ) ) , α : = π 4 L max . (35) p_i=\begin{pmatrix}\cos(i\alpha)\\\sin(i\alpha)\end{pmatrix},\qquad\alpha:=\frac{\pi}{4L_{\max}}. \tag*{(35)} p i = ( cos ( i α ) sin ( i α ) ) , α := 4 L m a x π . (35) Each token representation is
x i = ( a i , b i , c i , d i , ρ i , p i ) ∈ R 4 N + 3 , ρ i = 1 , (36) x_i=(a_i,b_i,c_i,d_i,\rho_i,p_i)\in\mathbb{R}^{4N+3},\qquad\rho_i=1, \tag*{(36)} x i = ( a i , b i , c i , d i , ρ i , p i ) ∈ R 4 N + 3 , ρ i = 1 , (36) where a i , b i , c i , d i ∈ R N a_i,b_i,c_i,d_i\in\mathbb{R}^N a i , b i , c i , d i ∈ R N . The scalar coordinate ρ i \rho_i ρ i is a constant technical coordinate. Consider the first layer
X ( 2 ) = X ( 1 ) + ReLU ( Atten ( 1 ) ( X ( 1 ) ) ) , (37) \mathbf{X}^{(2)}=\mathbf{X}^{(1)}+\operatorname{ReLU}\left(\operatorname{Atten}^{(1)}(\mathbf{X}^{(1)})\right), \tag*{(37)} X ( 2 ) = X ( 1 ) + ReLU ( Atten ( 1 ) ( X ( 1 ) ) ) , (37) Atten ( 1 ) ( X ( 1 ) ) = W O ( 1 ) [ Head 1 ( 1 ) ( X ( 1 ) ) Head 2 ( 1 ) ( X ( 1 ) ) Head 3 ( 1 ) ( X ( 1 ) ) ] , (38) \operatorname{Atten}^{(1)}(\mathbf{X}^{(1)})=\mathbf{W}^{(1)}_O\begin{bmatrix}\operatorname{Head}^{(1)}_1(\mathbf{X}^{(1)})\\\operatorname{Head}^{(1)}_2(\mathbf{X}^{(1)})\\\operatorname{Head}^{(1)}_3(\mathbf{X}^{(1)})\end{bmatrix}, \tag*{(38)} Atten ( 1 ) ( X ( 1 ) ) = W O ( 1 ) Head 1 ( 1 ) ( X ( 1 ) ) Head 2 ( 1 ) ( X ( 1 ) ) Head 3 ( 1 ) ( X ( 1 ) ) , (38) Head h ( 1 ) ( X ( 1 ) ) = W V ( 1 , h ) X ( 1 ) ϕ ( 1 ) ( ( X ( 1 ) ) T ( W K ( 1 , h ) ) T W Q ( 1 , h ) X ( 1 ) ) . (39) \operatorname{Head}^{(1)}_h(\mathbf{X}^{(1)})=\mathbf{W}^{(1,h)}_V\mathbf{X}^{(1)}\phi^{(1)}\left((\mathbf{X}^{(1)})^\mathsf{T}(\mathbf{W}^{(1,h)}_K)^\mathsf{T}\mathbf{W}^{(1,h)}_Q\mathbf{X}^{(1)}\right). \tag*{(39)} Head h ( 1 ) ( X ( 1 ) ) = W V ( 1 , h ) X ( 1 ) ϕ ( 1 ) ( ( X ( 1 ) ) T ( W K ( 1 , h ) ) T W Q ( 1 , h ) X ( 1 ) ) . (39)
where ϕ ( 1 ) = softmax c o l \phi^{(1)}=\operatorname{softmax}_{\mathrm{col}} ϕ ( 1 ) = softmax col is column-wise softmax. Then for any δ ∈ ( 0 , 1 / 8 ) \delta\in(0,1/8) δ ∈ ( 0 , 1/8 ) , there exist matrices
{ W Q ( 1 , h ) , W K ( 1 , h ) , W V ( 1 , h ) } h = 1 3 , W O ( 1 ) , \{\mathbf{W}_{Q}^{(1,h)},\mathbf{W}_{K}^{(1,h)},\mathbf{W}_{V}^{(1,h)}\}_{h=1}^{3},\mathbf{W}_{O}^{(1)}, { W Q ( 1 , h ) , W K ( 1 , h ) , W V ( 1 , h ) } h = 1 3 , W O ( 1 ) ,
with embedding dimension d = 4 N + 3 = O ( N ) d=4N+3=O(N) d = 4 N + 3 = O ( N ) , such that for every position i ≥ 2 i\ge2 i ≥ 2 :
if a i − 1 = u a_{i-1}=u a i − 1 = u is a one-hot vector, then the second block added by the first layer is supported exactly on supp ( u ) \operatorname{supp}(u) supp ( u ) and equals λ u \lambda u λ u for some λ ∈ [ 1 − 2 δ , 1 ] \lambda\in[1-2\delta,1] λ ∈ [ 1 − 2 δ , 1 ] ;
if a i − 1 = 0 N a_{i-1}=0_N a i − 1 = 0 N , then the second block added by the first layer is exactly 0 N 0_N 0 N ;
if d i − 1 = v d_{i-1}=v d i − 1 = v is a one-hot vector, then the third block added by the first layer is supported exactly on supp ( v ) \operatorname{supp}(v) supp ( v ) and equals λ v \lambda v λ v for some λ ∈ [ 1 − 2 δ , 1 ] \lambda\in[1-2\delta,1] λ ∈ [ 1 − 2 δ , 1 ] ;
if d i − 1 = 0 N d_{i-1}=0_N d i − 1 = 0 N , then the third block added by the first layer is exactly 0 N 0_N 0 N .
Equivalently, for the tokens used in the DFS prompt,
Edge s i = ( 0 N 0 N 0 N s i 1 p ) , Edge t i = ( 0 N 0 N 0 N t i 1 p ) , ⟨ s ⟩ , ⟨ e ⟩ , ⟨ r ⟩ = ( 0 N 0 N 0 N 0 N 1 p ) , (40) \operatorname{Edge} s_i=
\begin{pmatrix}
0_N\\
0_N\\
0_N\\
s_i\\
1\\
p
\end{pmatrix},
\operatorname{Edge} t_i=
\begin{pmatrix}
0_N\\
0_N\\
0_N\\
t_i\\
1\\
p
\end{pmatrix},
\langle s\rangle,\langle e\rangle,\langle r\rangle=
\begin{pmatrix}
0_N\\
0_N\\
0_N\\
0_N\\
1\\
p
\end{pmatrix}, \tag*{(40)} Edge s i = 0 N 0 N 0 N s i 1 p , Edge t i = 0 N 0 N 0 N t i 1 p , ⟨ s ⟩ , ⟨ e ⟩ , ⟨ r ⟩ = 0 N 0 N 0 N 0 N 1 p , (40)
Target = ( 0 N 0 N 0 N n T a r 1 p ) , Root = ( n R o o t 0 N 0 N 0 N 1 p ) , CoT t = ( n t 0 N 0 N 0 N 1 p ) , (41) \operatorname{Target}=
\begin{pmatrix}
0_N\\
0_N\\
0_N\\
n_{\mathrm{Tar}}\\
1\\
p
\end{pmatrix},
\operatorname{Root}=
\begin{pmatrix}
n_{\mathrm{Root}}\\
0_N\\
0_N\\
0_N\\
1\\
p
\end{pmatrix},
\operatorname{CoT}\ t=
\begin{pmatrix}
n_t\\
0_N\\
0_N\\
0_N\\
1\\
p
\end{pmatrix}, \tag*{(41)} Target = 0 N 0 N 0 N n Tar 1 p , Root = n Root 0 N 0 N 0 N 1 p , CoT t = n t 0 N 0 N 0 N 1 p , (41)
the first layer gives
Edge s i ⟼ ( 0 N 0 N 0 N s i 1 p ) , Root ⟼ ( n R o o t 0 N 0 N 0 N 1 p ) , CoT t ⟼ ( n t ( 1 ± O ( δ ) ) n t − 1 0 N 0 N 1 p ) , (42) \operatorname{Edge} s_i\longmapsto
\begin{pmatrix}
0_N\\
0_N\\
0_N\\
s_i\\
1\\
p
\end{pmatrix},
\quad
\operatorname{Root}\longmapsto
\begin{pmatrix}
n_{\mathrm{Root}}\\
0_N\\
0_N\\
0_N\\
1\\
p
\end{pmatrix},
\quad
\operatorname{CoT}\ t\longmapsto
\begin{pmatrix}
n_t\\
(1\pm O(\delta))n_{t-1}\\
0_N\\
0_N\\
1\\
p
\end{pmatrix}, \tag*{(42)} Edge s i ⟼ 0 N 0 N 0 N s i 1 p , Root ⟼ n Root 0 N 0 N 0 N 1 p , CoT t ⟼ n t ( 1 ± O ( δ )) n t − 1 0 N 0 N 1 p , (42)
Edge t i ⟼ ( 0 N 0 N ( 1 ± O ( δ ) ) s i t i 1 p ) , Target ⟼ ( 0 N 0 N 0 N n T a r 1 p ) , ⟨ e ⟩ ⟼ ( 0 N 0 N ( 1 ± O ( δ ) ) t i 0 N 1 p ) , (43) \operatorname{Edge} t_i\longmapsto
\begin{pmatrix}
0_N\\
0_N\\
(1\pm O(\delta))s_i\\
t_i\\
1\\
p
\end{pmatrix},
\quad
\operatorname{Target}\longmapsto
\begin{pmatrix}
0_N\\
0_N\\
0_N\\
n_{\mathrm{Tar}}\\
1\\
p
\end{pmatrix},
\quad
\langle e\rangle\longmapsto
\begin{pmatrix}
0_N\\
0_N\\
(1\pm O(\delta))t_i\\
0_N\\
1\\
p
\end{pmatrix}, \tag*{(43)} Edge t i ⟼ 0 N 0 N ( 1 ± O ( δ )) s i t i 1 p , Target ⟼ 0 N 0 N 0 N n Tar 1 p , ⟨ e ⟩ ⟼ 0 N 0 N ( 1 ± O ( δ )) t i 0 N 1 p , (43)
⟨ r ⟩ ⟼ ( 0 N 0 N ( 1 ± O ( δ ) ) n T a r 0 N 1 p ) , ⟨ s ⟩ ⟼ ( 0 N 0 N 0 N 0 N 1 p ) . (44) \langle r\rangle\longmapsto
\begin{pmatrix}
0_N\\
0_N\\
(1\pm O(\delta))n_{\mathrm{Tar}}\\
0_N\\
1\\
p
\end{pmatrix},
\quad
\langle s\rangle\longmapsto
\begin{pmatrix}
0_N\\
0_N\\
0_N\\
0_N\\
1\\
p
\end{pmatrix}. \tag*{(44)} ⟨ r ⟩ ⟼ 0 N 0 N ( 1 ± O ( δ )) n Tar 0 N 1 p , ⟨ s ⟩ ⟼ 0 N 0 N 0 N 0 N 1 p . (44)
Here every factor ( 1 ± O ( δ ) ) (1 \pm O(\delta)) ( 1 ± O ( δ )) denotes a scalar in [ 1 − 2 δ , 1 ] [1 - 2\delta, 1] [ 1 − 2 δ , 1 ] , and the supports of the copied one-hot vectors are exact after the ReLU filter. The claim for <s> at position 1, which has no predecessor, is included: that position attends to itself and all of its node blocks are zero. If L max = poly ( N ) L_{\max} = \operatorname{poly}(N) L m a x = poly ( N ) and δ ≥ 1 / poly ( N ) \delta\ge1/\operatorname{poly}(N) δ ≥ 1/ poly ( N ) , then the construction uses O ( log N ) O(\log N) O ( log N ) -bit parameter magnitudes and precision.
Proof. We give an explicit construction.
Step 1: a softmax predecessor selector. Let
R − 1 : = ( cos α sin α − sin α cos α ) . (45) R_{-1} := \begin{pmatrix} \cos\alpha& \sin\alpha\\ -\sin\alpha& \cos\alpha\end{pmatrix}. \tag*{(45)} R − 1 := ( cos α − sin α sin α cos α ) . (45)
Then R − 1 p i = p i − 1 R_{-1}p_i = p_{i-1} R − 1 p i = p i − 1 for all i ≥ 2 i \ge2 i ≥ 2 . For each head h ∈ { 1 , 2 , 3 } h \in\{1,2,3\} h ∈ { 1 , 2 , 3 } , choose W Q ( 1 , h ) W_Q^{(1,h)} W Q ( 1 , h ) and W K ( 1 , h ) W_K^{(1,h)} W K ( 1 , h ) to ignore the four node blocks and the scalar coordinate, and to act on the positional block as
W Q ( 1 , h ) x i = β R − 1 p i , W K ( 1 , h ) x j = p j . (46) W_Q^{(1,h)}x_i = \beta R_{-1}p_i, \qquad W_K^{(1,h)}x_j = p_j. \tag*{(46)} W Q ( 1 , h ) x i = β R − 1 p i , W K ( 1 , h ) x j = p j . (46)
Thus the score from key position j j j to query position i i i is
S j , i = β ⟨ p j , p i − 1 ⟩ = β cos ( ( j − i + 1 ) α ) . (47) S_{j,i} = \beta\langle p_j, p_{i-1} \rangle= \beta\cos((j-i+1)\alpha). \tag*{(47)} S j , i = β ⟨ p j , p i − 1 ⟩ = β cos (( j − i + 1 ) α ) . (47)
For fixed i ≥ 2 i \ge2 i ≥ 2 , the unique maximizer is j = i − 1 j=i-1 j = i − 1 . Moreover, for j ≠ i − 1 j \ne i-1 j = i − 1 ,
S i − 1 , i − S j , i = β ( 1 − cos ( ( j − i + 1 ) α ) ) ≥ β ( 1 − cos α ) . (48) S_{i-1,i} - S_{j,i} = \beta(1-\cos((j-i+1)\alpha)) \ge\beta(1-\cos\alpha). \tag*{(48)} S i − 1 , i − S j , i = β ( 1 − cos (( j − i + 1 ) α )) ≥ β ( 1 − cos α ) . (48)
Since α = π / ( 4 L max ) \alpha= \pi/(4L_{\max}) α = π / ( 4 L m a x ) , there is a universal constant c > 0 c > 0 c > 0 such that
1 − cos α ≥ c L max 2 . (49) 1-\cos\alpha\ge\frac{c}{L_{\max}^{2}}. \tag*{(49)} 1 − cos α ≥ L m a x 2 c . (49)
Choose
β ≥ C L max 2 log ( L max / δ ) (50) \beta\ge C L_{\max}^{2}\log(L_{\max}/\delta) \tag*{(50)} β ≥ C L m a x 2 log ( L m a x / δ ) (50)
with a sufficiently large universal constant C C C . Then the column-wise softmax attention matrix P P P satisfies
P i − 1 , i ≥ 1 − δ , ∑ j ≠ i − 1 P j , i ≤ δ . (51) P_{i-1,i} \ge1-\delta, \qquad\sum_{j\ne i-1} P_{j,i} \le\delta. \tag*{(51)} P i − 1 , i ≥ 1 − δ , j = i − 1 ∑ P j , i ≤ δ . (51)
Step 2: value maps. All three heads use the same softmax predecessor matrix P P P . The first head extracts the first node block:
W V ( 1 , 1 ) x j = a j . (52) W_V^{(1,1)}x_j = a_j. \tag*{(52)} W V ( 1 , 1 ) x j = a j . (52)
The second head extracts the fourth node block:
W V ( 1 , 2 ) x j = d j . (53) W_V^{(1,2)}x_j = d_j. \tag*{(53)} W V ( 1 , 2 ) x j = d j . (53)
The third head uses the constant scalar coordinate to produce a threshold vector:
W V ( 1 , 3 ) x j = ρ j 1 N = 1 N . (54) W_V^{(1,3)}x_j = \rho_j\mathbf{1}_N = \mathbf{1}_N. \tag*{(54)} W V ( 1 , 3 ) x j = ρ j 1 N = 1 N . (54)
Because every column of P P P sums to one, the third head outputs exactly 1 N \mathbf{1}_N 1 N at every position:
( Head 3 ( 1 ) ( X ( 1 ) ) ) i = 1 N . (55) \left(\operatorname{Head}_3^{(1)}(X^{(1)})\right)_i = \mathbf{1}_N. \tag*{(55)} ( Head 3 ( 1 ) ( X ( 1 ) ) ) i = 1 N . (55)
Step 3: output map and ReLU filter. Let the output matrix place a thresholded copy of the first head into the second node block and a thresholded copy of the second head into the third node block:
W O ( 1 ) [ u v r ] = ( 0 N 2 u − r 2 v − r 0 N 0 0 2 ) , u , v , r ∈ R N . (56) W^{(1)}_{O}
\begin{bmatrix}
u\\
v\\
r
\end{bmatrix}
=
\begin{pmatrix}
0_{N}\\
2u-r\\
2v-r\\
0_{N}\\
0\\
0_{2}
\end{pmatrix},
\qquad u,v,r\in\mathbb{R}^{N}. \tag*{(56)} W O ( 1 ) u v r = 0 N 2 u − r 2 v − r 0 N 0 0 2 , u , v , r ∈ R N . (56)
Hence the pre-ReLU attention output at position i i i is
z i : = Atten ( 1 ) ( X ( 1 ) ) i = ( 0 N 2 ∑ j P j , i a j − 1 N 2 ∑ j P j , i d j − 1 N 0 N 0 0 2 ) . (57) z_i:=\operatorname{Atten}^{(1)}(\mathbf{X}^{(1)})_i=
\begin{pmatrix}
0_{N}\\
2\sum_j P_{j,i}a_j-1_{N}\\
2\sum_j P_{j,i}d_j-1_{N}\\
0_{N}\\
0\\
0_{2}
\end{pmatrix}. \tag*{(57)} z i := Atten ( 1 ) ( X ( 1 ) ) i = 0 N 2 ∑ j P j , i a j − 1 N 2 ∑ j P j , i d j − 1 N 0 N 0 0 2 . (57)
Now suppose a i − 1 = e k a_{i-1}=e_k a i − 1 = e k is one-hot. By Eq. equation 51, the k k k -th coordinate of ∑ j P j , i a j \sum_j P_{j,i}a_j ∑ j P j , i a j lies in [ 1 − δ , 1 ] [1-\delta,1] [ 1 − δ , 1 ] : it is at least P i − 1 , i ≥ 1 − δ P_{i-1,i}\ge1-\delta P i − 1 , i ≥ 1 − δ , and it is at most one because the attention weights sum to one and every a j a_j a j has entries in { 0 , 1 } \{0,1\} { 0 , 1 } . Every coordinate outside k k k is at most δ \delta δ , because the total non-predecessor mass is at most δ \delta δ . Hence
ReLU ( 2 ∑ j P j , i a j − 1 N ) = λ e k , λ ∈ [ 1 − 2 δ , 1 ] , (58) \operatorname{ReLU}\left(2\sum_j P_{j,i}a_j-1_{N}\right)=\lambda e_k,\qquad\lambda\in[1-2\delta,1], \tag*{(58)} ReLU ( 2 j ∑ P j , i a j − 1 N ) = λ e k , λ ∈ [ 1 − 2 δ , 1 ] , (58)
with exact support { k } \{k\} { k } , because 2 δ − 1 < 0 2\delta-1<0 2 δ − 1 < 0 kills every other coordinate. If a i − 1 = 0 N a_{i-1}=0_{N} a i − 1 = 0 N , then every coordinate of ∑ j P j , i a j \sum_j P_{j,i}a_j ∑ j P j , i a j is at most δ < 1 / 2 \delta<1/2 δ < 1/2 , and therefore
ReLU ( 2 ∑ j P j , i a j − 1 N ) = 0 N . (59) \operatorname{ReLU}\left(2\sum_j P_{j,i}a_j-1_{N}\right)=0_{N}. \tag*{(59)} ReLU ( 2 j ∑ P j , i a j − 1 N ) = 0 N . (59)
The same argument applied to the fourth node block proves the corresponding claims for d i − 1 d_{i-1} d i − 1 and the third block.
Step 4: applying to the prompt. For an edge target token Edge t i t_i t i , the previous token is Edge s i s_i s i , whose fourth node block is s i s_i s i and whose first node block is zero. Thus the ReLU-filtered residual addition places ( 1 ± O ( δ ) ) s i (1\pm O(\delta))s_i ( 1 ± O ( δ )) s i into the third block and places zero into the second block. For the source token, the previous token is <e>, whose four node blocks are all zero, so the ReLU-filtered residual addition is zero. For a CoT token n t n_t n t , the previous token has first node block n t − 1 n_{t-1} n t − 1 and fourth node block zero, so the ReLU-filtered residual addition places ( 1 ± O ( δ ) ) n t − 1 (1\pm O(\delta))n_{t-1} ( 1 ± O ( δ )) n t − 1 into the second block and places zero into the third block. Finally, at position i = 1 i=1 i = 1 the scores are S j , 1 = β cos ( j α ) S_{j,1}=\beta\cos(j\alpha) S j , 1 = β cos ( j α ) for j ≥ 1 j\ge1 j ≥ 1 , whose unique maximizer is j = 1 j=1 j = 1 with gap S 1 , 1 − S j , 1 ≥ β ( cos α − cos 2 α ) ≥ β ( 1 − cos α ) S_{1,1}-S_{j,1}\ge\beta(\cos\alpha-\cos2\alpha)\ge\beta(1-\cos\alpha) S 1 , 1 − S j , 1 ≥ β ( cos α − cos 2 α ) ≥ β ( 1 − cos α ) ; hence the same concentration bound holds with i − 1 i-1 i − 1 replaced by 1 1 1 , and since a 1 = d 1 = 0 N a_1=d_1=0_N a 1 = d 1 = 0 N for <s>, both ReLU-filtered additions at position 1 1 1 are 0 N 0_N 0 N .
Finally, d = 4 N + 3 = O ( N ) d=4N+3=O(N) d = 4 N + 3 = O ( N ) . The nonzero parameters are 0 , 1 , 2 0,1,2 0 , 1 , 2 , the rotation entries cos α , sin α \cos\alpha,\sin\alpha cos α , sin α , and the scale β \beta β . If L max = poly ( N ) L_{\max}=\operatorname{poly}(N) L m a x = poly ( N ) and δ ≥ 1 / poly ( N ) \delta\ge1/\operatorname{poly}(N) δ ≥ 1/ poly ( N ) , then β = poly ( N ) \beta=\operatorname{poly}(N) β = poly ( N ) and all parameters can be represented to inverse-polynomial accuracy with O ( log N ) O(\log N) O ( log N ) bits. The softmax concentration bound is stable under inverse-polynomial perturbations of the scores, so O ( log N ) O(\log N) O ( log N ) precision suffices.
Proof of Theorem 3.6
Proof. The representation-size bound follows from Theorem B.1: O ( 1 ) O(1) O ( 1 ) heads, embedding dimension 4 N + 3 4N + 3 4 N + 3 , and O ( log N ) O(\log N) O ( log N ) precision, the second-layer weights being constants. The lemma applies with L max = 3 N 2 + 2 N + 2 = poly ( N ) L_{\max} = 3N^2 + 2N + 2 = \operatorname{poly}(N) L m a x = 3 N 2 + 2 N + 2 = poly ( N ) to every graph on [ N ] [N] [ N ] and every valid DFS trace on it. Indeed M ≤ N 2 M \le N^2 M ≤ N 2 , and a valid DFS trace ( n 0 , … , n t ) (n_0,\ldots,n_t) ( n 0 , … , n t ) has t ≤ 2 N − 2 t \le2N - 2 t ≤ 2 N − 2 : each forward move enters a previously unvisited vertex, so there are at most N − 1 N - 1 N − 1 forward moves, and each backtracking step pops a vertex pushed by an earlier forward move, so there are at most as many backtracking steps as forward moves.
Fix a valid nonterminated DFS trace and let n t n_t n t be the current node. By Theorem B.1, every edge-target token exposes its source in the third block and its destination in the fourth block, and every CoT token exposes the current node in the first block and the previous node in the second block, where each copied one-hot vector carries a scalar factor in [ 1 − 2 δ , 1 ] [1 - 2\delta, 1] [ 1 − 2 δ , 1 ] and all other coordinates of the copied block are exactly zero. The forward head assigns score c 1 λ c_1\lambda c 1 λ with λ ∈ [ 1 − 2 δ , 1 ] \lambda\in[1 - 2\delta, 1] λ ∈ [ 1 − 2 δ , 1 ] to each edge-target token whose source is n t n_t n t , and score exactly c 1 c_1 c 1 to each CoT token, including the root; its value map adds the destination of an edge-target token and subtracts the node of a CoT token. Hence, before the ReLU, an unvisited child of n t n_t n t has forward score at least c 1 ( 1 − 2 δ ) c_1(1 - 2\delta) c 1 ( 1 − 2 δ ) , a visited child has forward score at most c 1 ( 1 − 1 ) = 0 c_1(1 - 1) = 0 c 1 ( 1 − 1 ) = 0 , and a non-child has forward score at most 0. The backtracking head attends to the CoT tokens that enter or leave n t n_t n t . Since n t n_t n t is the top of the current DFS stack, it has never been popped. Hence every step that left n t n_t n t was a forward move to a previously unvisited child, each such child was entered exactly once, and its excursion has been completed by the backtracking step that returned to n t n_t n t . In particular, no step went directly from n t n_t n t to its stack parent, which was visited before n t n_t n t and is therefore neither an unvisited child of n t n_t n t nor a backtracking target from n t n_t n t while n t n_t n t is on the stack. Consequently every completed excursion from n t n_t n t to a child and back contributes exactly one departure term and one return term, whose node coordinates cancel up to the leakage, leaving at most 2 δ c 2 2\delta c_2 2 δ c 2 on that child, while the stack parent receives only the unique tree-edge arrival into n t n_t n t , which contributes at least c 2 ( 1 − 2 δ ) c_2(1 - 2\delta) c 2 ( 1 − 2 δ ) and at most c 2 c_2 c 2 on the stack parent and a negative amount on n t n_t n t ; at the root there is no tree-edge arrival. After the ReLU and the readout, every unvisited child has logit at least c 1 ( 1 − 2 δ ) c_1(1 - 2\delta) c 1 ( 1 − 2 δ ) , the stack parent has logit in [ c 2 ( 1 − 2 δ ) , c 2 ] [c_2(1 - 2\delta), c_2] [ c 2 ( 1 − 2 δ ) , c 2 ] , and every other vertex has logit at most 2 δ c 2 2\delta c_2 2 δ c 2 . By Equation (9) , c 1 ( 1 − 2 δ ) > c 2 c_1(1 - 2\delta) > c_2 c 1 ( 1 − 2 δ ) > c 2 , and δ < 1 / 4 \delta< 1/4 δ < 1/4 gives c 2 ( 1 − 2 δ ) > 2 δ c 2 c_2(1 - 2\delta) > 2\delta c_2 c 2 ( 1 − 2 δ ) > 2 δ c 2 . Therefore, if n t n_t n t has an unvisited child, every maximizer is an unvisited child of n t n_t n t ; otherwise n t n_t n t is not the root, because the trace is nonterminated, and the unique maximizer is the stack parent. In both cases every maximizer of the next-token distribution is a valid DFS move. □
Proof of the look-ahead construction and a conditional efficiency bound
Lemma B.2 (k k k -layer transformer look-ahead). For every integer k ≥ 3 k \ge3 k ≥ 3 , there exists a k k k -layer transformer with unmasked attention that implements the bounded-reachability policy in Theorem 3.7 with horizon L = 1 + 2 k − 3 L = 1 + 2^{k-3} L = 1 + 2 k − 3 . Its token embeddings consist of node one-hot blocks together with the positional encoding p i p_i p i of Theorem B.1, a constant coordinate, a beginning-of-sequence indicator, and a period-three position indicator, as specified in the proof. The construction has O ( 1 ) O(1) O ( 1 ) heads per layer, width O ( N ) O(N) O ( N ) , and fixed-point precision O k ( log ( N + 1 ) ) O_k(\log(N + 1)) O k ( log ( N + 1 )) .
Proof. We specify the hidden state after each layer and the attention heads producing its next update. Write g = n T ar g = n_{T\operatorname{ar}} g = n T ar , and let e w ∈ R N e_w \in\mathbb{R}^N e w ∈ R N be the one-hot vector of vertex w w w . An edge record is the target-token position in a triple ( u , v , ⟨ e ⟩ ) (u,v,\langle\text{e}\rangle) ( u , v , ⟨ e ⟩) . All attention sums below range over the entire current input prefix, of length T T T .
State notation and permitted updates. Let h i [ ℓ ] = X : , i ( ℓ + 1 ) h_i^{[\ell]} = X_{:,i}^{(\ell+1)} h i [ ℓ ] = X : , i ( ℓ + 1 ) denote the state at position i i i after ℓ \ell ℓ layers, so h i [ 0 ] h_i^{[0]} h i [ 0 ] is the input embedding. The architecture updates it by
h i [ ℓ + 1 ] = h i [ ℓ ] + ReLU ( W O [ ℓ + 1 ] [ H 1 , i ; … ; H H , i ] ) . (60) h_i^{[\ell+1]} = h_i^{[\ell]} + \operatorname{ReLU}\left(W_O^{[\ell+1]}[H_{1,i};\ldots;H_{H,i}]\right). \tag*{(60)} h i [ ℓ + 1 ] = h i [ ℓ ] + ReLU ( W O [ ℓ + 1 ] [ H 1 , i ; … ; H H , i ] ) . (60)
For a linear-attention head, a row ( q i , k s , v s ) (q_i,k_s,v_s) ( q i , k s , v s ) in the tables below means
H i = ∑ s = 1 T ⟨ q i , k s ⟩ v s . H_i = \sum_{s=1}^{T} \langle q_i,k_s\rangle v_s. H i = s = 1 ∑ T ⟨ q i , k s ⟩ v s .
Each displayed query, key, and value is a linear function of the incoming state; smaller vectors are padded by zeros to a common head dimension. The output projection routes and linearly combines these head outputs in the specified coordinate blocks; the only post-attention nonlinearity is the single ReLU in Equation (60) .
For exact logical copies, use a pair of physical blocks
Y ^ = ( Y + , Y − ) , Y = Y + − Y − . \widehat{Y}=(Y^+,Y^-), \qquad Y=Y^+-Y^-. Y = ( Y + , Y − ) , Y = Y + − Y − .
A query, key, value, or readout map can decode Y Y Y by a linear subtraction. For a nonnegative increment Δ \Delta Δ , write
Y ^ ⊕ Δ = ( Y + + Δ , Y − ) . \widehat{Y}\oplus\Delta=(Y^++\Delta,Y^-). Y ⊕ Δ = ( Y + + Δ , Y − ) .
This is a residual update to the plus block, leaving the minus block unchanged. Also write
Π ( z ) = ( ReLU ( z ) , ReLU ( z − 1 ) ) . \Pi(z)=\bigl(\operatorname{ReLU}(z),\operatorname{ReLU}(z-1)\bigr). Π ( z ) = ( ReLU ( z ) , ReLU ( z − 1 ) ) .
When written into two initially zero blocks, Π ( z ) \Pi(z) Π ( z ) is realized by two rows of the same output projection followed by the layer’s single ReLU. Neither notation introduces an extra layer.
Retain the input block
η i = ( a i , d i , γ i , ρ i , χ i , ζ i , p i ) . \eta_i=(a_i,d_i,\gamma_i,\rho_i,\chi_i,\zeta_i,p_i). η i = ( a i , d i , γ i , ρ i , χ i , ζ i , p i ) .
Here a i a_i a i is the node one-hot at root and CoT positions and zero elsewhere; d i d_i d i is the node one-hot at edge endpoints and the task-target position and zero elsewhere; and γ i = e g \gamma_i=e_g γ i = e g only at the task-target position. Furthermore, ρ i = 1 \rho_i=1 ρ i = 1 , ζ i \zeta_i ζ i is the BOS indicator, χ i = 1 { i ≡ 0 ( m o d 3 ) } \chi_i=\mathbf{1}\{i\equiv0\pmod{3}\} χ i = 1 { i ≡ 0 ( mod 3 )} , and p i p_i p i is the two-dimensional positional encoding from Theorem B.1. The features use O ( N ) O(N) O ( N ) coordinates and remain unchanged throughout. An edge record is characterized by χ i = 1 \chi_i=1 χ i = 1 and 1 ⊤ d i = 1 \mathbf{1}^{\top}d_i=1 1 ⊤ d i = 1 . Indeed, the edge-target token of the i i i -th edge sits at position 3 i 3i 3 i ; the task target sits at position 3 M + 2 3M+2 3 M + 2 and has χ = 0 \chi=0 χ = 0 ; the token <r> sits at position 3 M + 3 3M+3 3 M + 3 and has χ = 1 \chi=1 χ = 1 but d = 0 d=0 d = 0 ; and every root or CoT position, as well as every source and <e> position, has d = 0 d=0 d = 0 or χ = 0 \chi=0 χ = 0 .
The complete physical state has the fixed coordinate layout
h i [ ℓ ] = ( η i , b ^ i , U ^ i , V ^ i , G ^ i , S ^ i , C ^ i , D i f , D i b , Z i + , Z i − ) . (61) h_i^{[\ell]}=\left(\eta_i,\widehat{b}_i,\widehat{U}_i,\widehat{V}_i,\widehat{G}_i,\widehat{S}_i,\widehat{C}_i,D_i^f,D_i^b,Z_i^+,Z_i^-\right). \tag*{(61)} h i [ ℓ ] = ( η i , b i , U i , V i , G i , S i , C i , D i f , D i b , Z i + , Z i − ) . (61)
where the blocks after η i \eta_i η i are initially zero. The paired blocks store the predecessor, source identifier, destination identifier, goal, source reachability memory, and child reachability memory, respectively. The last four blocks hold the forward DFS score, backtracking score, and two final eligibility outputs. Each unpaired working block has N N N coordinates; each paired block has 2 N 2N 2 N coordinates.
Layer 1: endpoint and goal copies. After the first layer, the exact decoded values are
( a i , b i , U i , V i , G i ) = { ( 0 , 0 , e u , e v , e g ) , i records ( u → v ) , ( e n R o o t , 0 , 0 , 0 , e g ) , i is the root position , ( e n t , e n t − 1 , 0 , 0 , e g ) , i is CoT step t , ( 0 , 0 , 0 , 0 , e g ) , otherwise , (62) (a_i,b_i,U_i,V_i,G_i)=
\begin{cases}
(0,0,e_u,e_v,e_g), & i\text{ records }(u\to v),\\
(e_{n_{\mathrm{Root}}},0,0,0,e_g), & i\text{ is the root position},\\
(e_{n_t},e_{n_{t-1}},0,0,e_g), & i\text{ is CoT step }t,\\
(0,0,0,0,e_g), & \text{otherwise},
\end{cases} \tag*{(62)} ( a i , b i , U i , V i , G i ) = ⎩ ⎨ ⎧ ( 0 , 0 , e u , e v , e g ) , ( e n Root , 0 , 0 , 0 , e g ) , ( e n t , e n t − 1 , 0 , 0 , e g ) , ( 0 , 0 , 0 , 0 , e g ) , i records ( u → v ) , i is the root position , i is CoT step t , otherwise , (62)
where n 0 n_0 n 0 is the root. Define the immutable block
I i = ( n ^ i , b ^ i , U ^ i , V ^ i , G ^ i ) . I_i=(\hat{n}_i,\hat{b}_i,\hat{U}_i,\hat{V}_i,\hat{G}_i). I i = ( n ^ i , b ^ i , U ^ i , V ^ i , G ^ i ) .
Duplicate the source and destination pairs into the two mutable memories. Thus the full state after layer 1 is
h i [ 1 ] = ( I i , U ^ i , V ^ i , 0 , 0 , 0 , 0 ) . (63) h_i^{[1]}=(I_i,\hat{U}_i,\hat{V}_i,0,0,0,0). \tag*{(63)} h i [ 1 ] = ( I i , U ^ i , V ^ i , 0 , 0 , 0 , 0 ) . (63)
In this display the second and third entries occupy the S ^ \hat{S} S ^ and C ^ \hat{C} C ^ coordinates of Equation (61) ; they are separate copies of the pairs retained in I i I_i I i .
Here is an explicit realization of this state. The first layer uses the following three softmax heads:
The predecessor selector is that of Theorem B.1; the self selector uses the identity rotation. Choose the parameters and rounding accuracy so that their outputs satisfy
z i p r e ≈ d i − 1 , h i p r e ≈ a i − 1 , z i s e l f ≈ d i , h i s e l f ≈ a i , c i ≈ χ i , z i g o a l ≈ e g . z_i^{\mathrm{pre}}\approx d_{i-1},\quad h_i^{\mathrm{pre}}\approx a_{i-1},\quad z_i^{\mathrm{self}}\approx d_i,\quad h_i^{\mathrm{self}}\approx a_i,\quad c_i\approx\chi_i,\quad z_i^{\mathrm{goal}}\approx e_g. z i pre ≈ d i − 1 , h i pre ≈ a i − 1 , z i self ≈ d i , h i self ≈ a i , c i ≈ χ i , z i goal ≈ e g .
with effective error at most δ < 1 / 64 \delta<1/64 δ < 1/64 on every used coordinate and scalar sum. The predecessor approximations are needed only for i > 1 i>1 i > 1 . Each head’s constant-value output is one. Set n i = 1 ⊤ z i s e l f n_i=\mathbf{1}^{\top}z_i^{\mathrm{self}} n i = 1 ⊤ z i self and t i = 1 ⊤ h i s e l f t_i=\mathbf{1}^{\top}h_i^{\mathrm{self}} t i = 1 ⊤ h i self . The output projection forms
ω i ( z ) = 4 z + 4 ( c i + n i ) 1 − 91 , \omega_i(z)=4z+4(c_i+n_i)\mathbf{1}-9\mathbf{1}, ω i ( z ) = 4 z + 4 ( c i + n i ) 1 − 9 1 ,
r i = 4 h i p r e + 4 t i 1 − 51 , r_i=4h_i^{\mathrm{pre}}+4t_i\mathbf{1}-5\mathbf{1}, r i = 4 h i pre + 4 t i 1 − 5 1 ,
g i c o p y = 4 z i g o a l − 1 , g_i^{\mathrm{copy}}=4z_i^{\mathrm{goal}}-1, g i copy = 4 z i goal − 1 ,
and writes
U ^ i = Π ( ω i ( z i p r e ) ) , V ^ i = Π ( ω i ( z i s e l f ) ) , b ^ i = Π ( r i ) , G ^ i = Π ( g i c o p y ) . \hat{U}_i=\Pi(\omega_i(z_i^{\mathrm{pre}})),\quad\hat{V}_i=\Pi(\omega_i(z_i^{\mathrm{self}})),\quad\hat{b}_i=\Pi(r_i),\quad\hat{G}_i=\Pi(g_i^{\mathrm{copy}}). U ^ i = Π ( ω i ( z i pre )) , V ^ i = Π ( ω i ( z i self )) , b ^ i = Π ( r i ) , G ^ i = Π ( g i copy ) .
The same outputs for U ^ i , V ^ i \hat{U}_i,\hat{V}_i U ^ i , V ^ i are routed to the two mutable memories. At a desired one-hot coordinate the argument of Π \Pi Π is close to 3; at any other coordinate it is at most − 1 + 12 δ < 0 -1+12\delta<0 − 1 + 12 δ < 0 . The difference of the two physical outputs is therefore exactly one or zero, respectively. For endpoint copies, the three tests are the endpoint coordinate, the record-position feature, and the presence of a node in d i d_i d i . This excludes source tokens, the task target, special tokens, and CoT tokens from the edge memories. All offsets are linear maps of the constant-value output, and all pairs are produced in parallel by one ReLU. The finite-precision justification is given below.
Layer 2: one-step memories and the DFS scores. All subsequent attention is unnormalized linear attention. Use four heads, with τ s = 1 ⊤ a s \tau_s = 1^{\top}a_s τ s = 1 ⊤ a s :
Table 3.
S ^ i ( 0 ) = U ^ i ⊕ ∑ s ⟨ U i , U s ⟩ V s , C ^ i ( 0 ) = V ^ i ⊕ ∑ s ⟨ V i , U s ⟩ V s (64) \widehat{S}_i^{(0)}=\widehat{U}_i\oplus\sum_s\langle U_i,U_s\rangle V_s,\qquad\widehat{C}_i^{(0)}=\widehat{V}_i\oplus\sum_s\langle V_i,U_s\rangle V_s \tag*{(64)} S i ( 0 ) = U i ⊕ s ∑ ⟨ U i , U s ⟩ V s , C i ( 0 ) = V i ⊕ s ∑ ⟨ V i , U s ⟩ V s (64)
D i f = ReLU ( 2 ∑ s ⟨ a i , U s + τ s 1 ⟩ ( V s − a s ) ) (65) D_i^{\mathrm{f}}=\operatorname{ReLU}\left(2\sum_s\langle a_i,U_s+\tau_s1\rangle(V_s-a_s)\right) \tag*{(65)} D i f = ReLU ( 2 s ∑ ⟨ a i , U s + τ s 1 ⟩ ( V s − a s ) ) (65)
D i b = ReLU ( ∑ s ⟨ a i , a s + b s ⟩ ( b s − a s ) ) . (66) D_i^{\mathrm{b}}=\operatorname{ReLU}\left(\sum_s\langle a_i,a_s+b_s\rangle(b_s-a_s)\right). \tag*{(66)} D i b = ReLU ( s ∑ ⟨ a i , a s + b s ⟩ ( b s − a s ) ) . (66)
Every entry is a linear map of h [ 1 ] h^{[1]} h [ 1 ] . Route each head to the indicated block and set all other rows of the output projection to zero. Consequently,
The memory increments are nonnegative, so the ReLU leaves them unchanged. The score blocks start at zero and receive the ReLU of their respective head outputs. The full hidden state is now
h i [ 2 ] = ( I i , S ^ i ( 0 ) , C ^ i ( 0 ) , D i f , D i b , 0 , 0 ) . (67) h_i^{[2]}=(I_i,\widehat{S}_i^{(0)},\widehat{C}_i^{(0)},D_i^{\mathrm{f}},D_i^{\mathrm{b}},0,0). \tag*{(67)} h i [ 2 ] = ( I i , S i ( 0 ) , C i ( 0 ) , D i f , D i b , 0 , 0 ) . (67)
For the record e = ( u , v ) e=(u,v) e = ( u , v ) , its decoded memories are explicitly
S e ( 0 ) = e u + ∑ ( u → w ) ∈ E e w . (68) S_e^{(0)}=e_u+\sum_{(u\to w)\in\mathcal{E}}e_w. \tag*{(68)} S e ( 0 ) = e u + ( u → w ) ∈ E ∑ e w . (68)
C e ( 0 ) = e v + ∑ ( v → w ) ∈ E e w . (69) C_e^{(0)}=e_v+\sum_{(v\to w)\in\mathcal{E}}e_w. \tag*{(69)} C e ( 0 ) = e v + ( v → w ) ∈ E ∑ e w . (69)
The endpoint terms come from the residual connection, and the neighbor sums come from the first two attention heads. At nonrecord positions both memories are zero.
At the final position i = T i=T i = T , let x x x be the current node. The last two heads are the ordinary DFS heads in Equation (8) with c 1 = 2 , c 2 = 1 c_1=2,c_2=1 c 1 = 2 , c 2 = 1 , now written as explicit projections into fresh blocks. The forward head subtracts two times the occurrence count of each visited node from two times the adjacency vector of x x x . In the backtracking head, completed excursions cancel, leaving the stack-parent coordinate positive. Therefore, on a valid nonterminated state,
D T , v f = 21 { v is an unvisited child of x } , D T , v b = 1 { v is the stack parent of x } . (70) D_{T,v}^{\mathrm{f}}=2\mathbf{1}\{v\text{ is an unvisited child of }x\},\qquad D_{T,v}^{\mathrm{b}}=\mathbf{1}\{v\text{ is the stack parent of }x\}. \tag*{(70)} D T , v f = 2 1 { v is an unvisited child of x } , D T , v b = 1 { v is the stack parent of x } . (70)
The second indicator is zero at the root. Since attention is unmasked, this interpretation is asserted at the final query position; the scores at earlier positions also use the whole current prefix. All later layers preserve both score blocks.
Layers 3, … \ldots … , k − 1 k-1 k − 1 : doubling the stored radius. Put J = k − 3 J=k-3 J = k − 3 . For j = 0 , … , J − 1 j=0,\ldots,J-1 j = 0 , … , J − 1 , layer j + 3 j+3 j + 3 takes the state in Equation (67) with memories indexed by j j j and uses two heads:
Table 4.
Thus the physical and decoded updates are, respectively,
S ^ i ( j + 1 ) = S ^ i ( j ) ⊕ ∑ s ⟨ S i ( j ) , U s ⟩ S s ( j ) , (71) \widehat{S}_i^{(j+1)}=\widehat{S}_i^{(j)}\oplus\sum_s\langle S_i^{(j)},U_s\rangle S_s^{(j)}, \tag*{(71)} S i ( j + 1 ) = S i ( j ) ⊕ s ∑ ⟨ S i ( j ) , U s ⟩ S s ( j ) , (71)
C ^ i ( j + 1 ) = C ^ i ( j ) ⊕ ∑ s ⟨ C i ( j ) , U s ⟩ S s ( j ) , (72) \widehat{C}_i^{(j+1)}=\widehat{C}_i^{(j)}\oplus\sum_s\langle C_i^{(j)},U_s\rangle S_s^{(j)}, \tag*{(72)} C i ( j + 1 ) = C i ( j ) ⊕ s ∑ ⟨ C i ( j ) , U s ⟩ S s ( j ) , (72)
S e ( j + 1 ) = S e ( j ) + ∑ f = ( a , b ) ∈ E S e ( j ) [ a ] S f ( j ) , (73) S_e^{(j+1)}=S_e^{(j)}+\sum_{f=(a,b)\in\mathcal{E}}S_e^{(j)}[a]S_f^{(j)}, \tag*{(73)} S e ( j + 1 ) = S e ( j ) + f = ( a , b ) ∈ E ∑ S e ( j ) [ a ] S f ( j ) , (73)
C e ( j + 1 ) = C e ( j ) + ∑ f = ( a , b ) ∈ E C e ( j ) [ a ] S f ( j ) . (74) C_e^{(j+1)}=C_e^{(j)}+\sum_{f=(a,b)\in\mathcal{E}}C_e^{(j)}[a]S_f^{(j)}. \tag*{(74)} C e ( j + 1 ) = C e ( j ) + f = ( a , b ) ∈ E ∑ C e ( j ) [ a ] S f ( j ) . (74)
Every update is a nonnegative head output added to a plus block. No block is overwritten and no Boolean threshold is applied. After j j j propagation layers, including j = 0 j=0 j = 0 for the state after layer 2, the full state is
h i [ j + 2 ] = ( I i , S ^ i ( j ) , C ^ i ( j ) , D i f , D i b , 0 , 0 ) , 0 ≤ j ≤ J . (75) h_i^{[j+2]}=(I_i,\widehat{S}_i^{(j)},\widehat{C}_i^{(j)},D_i^f,D_i^b,0,0),\qquad0\le j\le J. \tag*{(75)} h i [ j + 2 ] = ( I i , S i ( j ) , C i ( j ) , D i f , D i b , 0 , 0 ) , 0 ≤ j ≤ J . (75)
To specify its meaning, define
R r ( w ) = { z : dist G ( w , z ) ≤ r } . \mathcal{R}_r(w)=\{z:\operatorname{dist}_{G}(w,z)\le r\}. R r ( w ) = { z : dist G ( w , z ) ≤ r } .
The decoded memories have nonnegative integer coordinates and satisfy
supp + ( S e ( j ) ) = R 2 j ( u ) , supp + ( C e ( j ) ) = R 2 j ( v ) , e = ( u , v ) . (76) \operatorname{supp}_{+}(S_e^{(j)})=\mathcal{R}_{2^j}(u),\qquad\operatorname{supp}_{+}(C_e^{(j)})=\mathcal{R}_{2^j}(v),\qquad e=(u,v). \tag*{(76)} supp + ( S e ( j ) ) = R 2 j ( u ) , supp + ( C e ( j ) ) = R 2 j ( v ) , e = ( u , v ) . (76)
The base case follows directly from (68) and (70). Self-loops may give an entry of two, which does not affect its positive support. For the induction step, write r = 2 j r=2^j r = 2 j . Every summand in (73) concatenates two paths of length at most r r r . Conversely, paths of length at most r r r remain through the residual. Split a path of length r < ℓ ≤ 2 r r<\ell\le2r r < ℓ ≤ 2 r after r r r edges, at vertex a a a . Its remaining suffix is nonempty, so a a a has an outgoing edge and a corresponding source record f f f . Both the coefficient at a a a and the appropriate coordinate of S f ( j ) S_f^{(j)} S f ( j ) are positive, representing the whole path. The sum is over edge records, and multiple records with the same source only add positive contributions. This proves the invariant for both memories, including the zero-length path at a sink.
In particular, the input to the final layer is h i [ k − 1 ] = h i [ J + 2 ] h_i^{[k-1]}=h_i^{[J+2]} h i [ k − 1 ] = h i [ J + 2 ] , with each edge’s child memory already representing radius r = 2 J r=2^J r = 2 J .
Layer k k k : eligibility blocks and readout. Four parallel linear-attention heads read h [ k − 1 ] h^{[k-1]} h [ k − 1 ] .
Table 78.
At i = T i=T i = T , these outputs are
F = ∑ e = ( u , v ) ∈ E C e ( J ) [ g ] e v , A = ∑ e = ( u , v ) ∈ E ⟨ e x , e u ⟩ e v , m = ∑ s a s . F=\sum_{e=(u,v)\in\mathcal{E}}C_e^{(J)}[g]e_v,\qquad A=\sum_{e=(u,v)\in\mathcal{E}}\langle e_x,e_u\rangle e_v,\qquad m=\sum_s a_s. F = e = ( u , v ) ∈ E ∑ C e ( J ) [ g ] e v , A = e = ( u , v ) ∈ E ∑ ⟨ e x , e u ⟩ e v , m = s ∑ a s .
The simple-graph assumption gives A v ∈ { 0 , 1 } A_v\in\{0,1\} A v ∈ { 0 , 1 } , with A v = 1 A_v=1 A v = 1 exactly for an outgoing child. Also m v = 0 m_v=0 m v = 0 exactly for an unvisited vertex. For any outgoing child v v v , the support invariant gives
F v > 0 ⟺ dist G ( v , g ) ≤ r . F_v>0\iff\operatorname{dist}_{G}(v,g)\le r. F v > 0 ⟺ dist G ( v , g ) ≤ r .
This includes v = g v=g v = g , even if g g g is a sink.
Choose an integer B ≥ 1 B \ge1 B ≥ 1 bounding every coordinate of F F F , as specified below. The output projection writes only to the two fresh eligibility blocks, with preactivations z i z_i z i and z i − 1 z_i-1 z i − 1 , where
z i = F i + B A i − 2 B m − B 1. z_i = F_i + BA_i - 2Bm - B\mathbf{1}. z i = F i + B A i − 2 B m − B 1 .
Consequently, the complete state after the final layer is
h i [ k ] = ( I i , S ^ i ( J ) , C ^ i ( J ) , D i f , D i b , ReLU ( z i ) , ReLU ( z i − 1 ) ) . (77) h_i^{[k]} = \left(I_i,\widehat{S}_i^{(J)},\widehat{C}_i^{(J)},D_i^f,D_i^b,\operatorname{ReLU}(z_i),\operatorname{ReLU}(z_i-1)\right). \tag*{(77)} h i [ k ] = ( I i , S i ( J ) , C i ( J ) , D i f , D i b , ReLU ( z i ) , ReLU ( z i − 1 ) ) . (77)
At the query position define, by a linear readout,
E v = Z T , v + − Z T , v − = ReLU ( z T , v ) − ReLU ( z T , v − 1 ) . (78) E_v = Z_{T,v}^{+} - Z_{T,v}^{-} = \operatorname{ReLU}(z_{T,v})-\operatorname{ReLU}(z_{T,v}-1). \tag*{(78)} E v = Z T , v + − Z T , v − = ReLU ( z T , v ) − ReLU ( z T , v − 1 ) . (78)
All entries defining z T , v z_{T,v} z T , v are integers. An unvisited outgoing child with F v > 0 F_v>0 F v > 0 has z T , v = F v ≥ 1 z_{T,v}=F_v\ge1 z T , v = F v ≥ 1 , hence E v = 1 E_v=1 E v = 1 . A nonchild has z T , v ≤ F v − B ≤ 0 z_{T,v}\le F_v-B\le0 z T , v ≤ F v − B ≤ 0 ; a visited child has z T , v ≤ F v + B − 2 B − B ≤ F v − 2 B ≤ − B < 0 z_{T,v}\le F_v+B-2B-B\le F_v-2B\le-B<0 z T , v ≤ F v + B − 2 B − B ≤ F v − 2 B ≤ − B < 0 ; and a child with F v = 0 F_v=0 F v = 0 and m v = 0 m_v=0 m v = 0 has z T , v = 0 z_{T,v}=0 z T , v = 0 . Thus E v E_v E v is exactly the eligibility indicator.
Let Q N = 2 ⌈ log 2 N ⌉ Q_N=2^{\lceil\log_2 N\rceil} Q N = 2 ⌈ l o g 2 N ⌉ . The final linear readout assigns vertex v v v the logit
ℓ v = 4 ( Z T , v + − Z T , v − ) + D T , v f + D T , v b + N − v 4 Q N ρ T . (79) \ell_v = 4\left(Z_{T,v}^{+}-Z_{T,v}^{-}\right)+D_{T,v}^{f}+D_{T,v}^{b}+\frac{N-v}{4Q_N}\rho_T. \tag*{(79)} ℓ v = 4 ( Z T , v + − Z T , v − ) + D T , v f + D T , v b + 4 Q N N − v ρ T . (79)
Assign each special token logit − ρ T -\rho_T − ρ T . The dyadic last term for vertex tokens lies in [ 0 , 1 / 4 ) [0,1/4) [ 0 , 1/4 ) and breaks ties by increasing label. Eligible children are preferred; if none exists, the forward DFS scores select an unvisited child, or the backtracking score selects the stack parent. This implements Theorem 3.7 with L = r + 1 = 1 + 2 k − 3 L=r+1=1+2^{k-3} L = r + 1 = 1 + 2 k − 3 .
Precision and resources. Use a common fixed-point grid with step 2 − b 2^{-b} 2 − b , where b = O ( log ( N + 1 ) ) b=O(\log(N+1)) b = O ( log ( N + 1 )) , and sufficient range to avoid overflow. First-layer head outputs are rounded to this grid before the integer-coefficient output projection. The two preactivations in each pair have the same nonconstant terms and biases differing by exactly one. The gate margins keep both active or both inactive as specified, so their physical outputs decode to an exact one or zero. The positional-selector and goal-selector parameters need only O ( log ( N + 1 ) ) O(\log(N+1)) O ( log ( N + 1 )) bits because the input length is polynomial in N N N . After this layer, decoded queries, keys, values, and memory increments are integers. An integer increment to the plus block preserves the exact decoded value.
After layer 2 the decoded memory entries are at most P 0 = 2 P_0=2 P 0 = 2 . Since M ≤ N 2 M\le N^2 M ≤ N 2 , a uniform bound for the propagation is
P j + 1 = ( N 2 + 1 ) P j 2 , P j = 2 2 j ( N 2 + 1 ) 2 j − 1 . P_{j+1}=(N^2+1)P_j^2,\qquad P_j=2^{2^j}(N^2+1)^{2^j-1}. P j + 1 = ( N 2 + 1 ) P j 2 , P j = 2 2 j ( N 2 + 1 ) 2 j − 1 .
Indeed, each coordinate increases by at most M P j 2 MP_j^2 M P j 2 in addition to its old value P j P_j P j . Every destination has at most N N N incoming edges, so take B = N P J B=NP_J B = N P J . Valid DFS traces have O ( N ) O(N) O ( N ) node occurrences, and all remaining intermediate values have magnitude at most a polynomial factor in N N N times B B B . Hence
p = O ( 2 k − 3 log ( N + 1 ) ) = O k ( log ( N + 1 ) ) p=O\left(2^{k-3}\log(N+1)\right)=O_k(\log(N+1)) p = O ( 2 k − 3 log ( N + 1 ) ) = O k ( log ( N + 1 ))
bits suffice, including fractional precision and the physical pairs’ bounded initial offsets. The fixed layout Equation (61) uses O ( N ) O(N) O ( N ) coordinates; the same two mutable memories are reused at every propagation layer. There are three heads in layer 1, four in layer 2, two in each propagation layer, and four in the final layer. Zero heads may be added if a common head count is required. The layer count is
1 ⏟ copies + 1 ⏟ neighbor aggregation and DFS + ( k − 3 ) ⏟ doublings + 1 ⏟ eligibility = k . \underbrace{1}_{\text{copies}}+\underbrace{1}_{\text{neighbor aggregation and DFS}}+\underbrace{(k-3)}_{\text{doublings}}+\underbrace{1}_{\text{eligibility}}=k. copies 1 + neighbor aggregation and DFS 1 + doublings ( k − 3 ) + eligibility 1 = k .
For k = 3 k=3 k = 3 the propagation range is empty and the final layer reads the one-step child memories in h [ 2 ] h^{[2]} h [ 2 ] .
Proof of Theorem 3.8. Apply Theorem B.2. Its constant head count, O ( N ) O(N) O ( N ) width, and O k ( log ( N + 1 ) ) O_k(\log(N+1)) O k ( log ( N + 1 )) precision give representation size O k ( N log ( N + 1 ) ) O_k(N\log(N+1)) O k ( N log ( N + 1 )) under the definition of § 3 .
Distinguishing bounded reachability from distance minimization. The preceding construction tests whether each candidate child reaches the goal within one specified radius. It does not compare the distances of qualifying children. For example, with edges u → v u \to v u → v , u → g u \to g u → g , and v → g v \to g v → g , horizon L = 2 L=2 L = 2 , and v v v ordered before g g g , both children qualify and the policy takes two moves although the target is one edge from u u u . Therefore the depth–horizon theorem alone does not imply the search-efficiency formula below. We state that formula for a stronger policy with an explicit distance-minimization requirement.
Definition B.3 (Distance-minimizing L L L -look-ahead DFS). At a nonterminated DFS state with current node u u u , goal g g g , and visited set S S S , let d S ( v , g ) d_S(v,g) d S ( v , g ) be the length of a shortest path from an unvisited child v v v to g g g all of whose vertices lie outside S S S , or ∞ \infty ∞ if no such path exists. If some unvisited child satisfies d S ( v , g ) ≤ L − 1 d_S(v,g)\le L-1 d S ( v , g ) ≤ L − 1 , select a child minimizing d S ( v , g ) d_S(v,g) d S ( v , g ) , breaking ties by the fixed child order. Otherwise apply the ordinary DFS rule.
For n ≥ 2 n\ge2 n ≥ 2 , define the worst-case efficiency of a fixed policy by
eff = min G , n s , g dist G ( n s , g ) number of CoT moves before reaching g , \operatorname{eff}=\min_{G,n_s,g}\frac{\operatorname{dist}_G(n_s,g)}{\text{number of CoT moves before reaching }g}, eff = G , n s , g min number of CoT moves before reaching g dist G ( n s , g ) ,
where the minimum is over instances with exactly n n n vertices reachable from n s n_s n s and a distinct reachable target g g g .
Lemma B.4 (Efficiency with distance-minimizing look-ahead). For every integer L ≥ 1 L\ge1 L ≥ 1 , the distance-minimizing policy in Theorem B.3 has worst-case efficiency
eff = { L + 1 2 n − L − 3 , n ≥ L + 2 , 1 , n < L + 2. (80) \operatorname{eff}=
\begin{cases}
\dfrac{L+1}{2n-L-3}, & n\ge L+2,\\
1, & n<L+2.
\end{cases} \tag*{(80)} eff = ⎩ ⎨ ⎧ 2 n − L − 3 L + 1 , 1 , n ≥ L + 2 , n < L + 2. (80) This statement is a property of that stronger policy and does not assert that the Boolean construction implements it.
Proof. Let m = dist G ( n s , g ) m=\operatorname{dist}_G(n_s,g) m = dist G ( n s , g ) . If m ≤ L m\le L m ≤ L , the suffix after the root of a shortest source-to-goal path lies outside the initial visited set. The policy selects a child on a shortest residual path. The suffix of that path remains available after the move, and its minimum remaining length decreases by one. Induction shows that the target is reached in exactly m m m moves, giving efficiency one.
Suppose instead that m ≥ L + 1 m\ge L+1 m ≥ L + 1 . Let D D D be the length of the final DFS-stack path to the target, and let T T T be the total number of moves. Every discovered tree edge outside that final path is traversed at most twice, while each of the D D D final-path edges is traversed once before termination. Since D ≥ m D \ge m D ≥ m ,
T ≤ 2 ( n − 1 ) − D ≤ 2 n − m − 2. T \le2(n-1)-D \le2n-m-2. T ≤ 2 ( n − 1 ) − D ≤ 2 n − m − 2.
The function m / ( 2 n − m − 2 ) m/(2n-m-2) m / ( 2 n − m − 2 ) is increasing on the relevant range, so
m T ≥ m 2 n − m − 2 ≥ L + 1 2 n − L − 3 . \frac{m}{T} \ge\frac{m}{2n-m-2} \ge\frac{L+1}{2n-L-3}. T m ≥ 2 n − m − 2 m ≥ 2 n − L − 3 L + 1 .
If n < L + 2 n < L+2 n < L + 2 , every reachable target has distance at most n − 1 ≤ L n-1 \le L n − 1 ≤ L , so the first case applies to every instance.
For tightness when n ≥ L + 2 n \ge L+2 n ≥ L + 2 , take a target path of length L + 1 L+1 L + 1 from the root and a distracting chain containing the remaining n − L − 2 n-L-2 n − L − 2 vertices, ordered before the target branch. The target is outside the horizon at the root and is unreachable from the distracting chain. The policy explores and backtracks along that chain before following the target path, taking
2 ( n − L − 2 ) + ( L + 1 ) = 2 n − L − 3 2(n-L-2)+(L+1)=2n-L-3 2 ( n − L − 2 ) + ( L + 1 ) = 2 n − L − 3
moves. The shortest-path distance is L + 1 L+1 L + 1 , attaining the bound. When n = L + 2 n=L+2 n = L + 2 , the distracting chain is empty and is omitted.
Corollary B.5 (Conditional depth–efficiency implication). For every integer k ≥ 3 k \ge3 k ≥ 3 , if a transformer implements the distance-minimizing policy of Theorem B.3 with L = 1 + 2 k − 3 L=1+2^{k-3} L = 1 + 2 k − 3 , then its worst-case efficiency is
eff = { 2 k − 3 + 2 2 n − 2 k − 3 − 4 , n ≥ 2 k − 3 + 3 , 1 , n < 2 k − 3 + 3. (81) \operatorname{eff} =
\begin{cases}
\frac{2^{k-3}+2}{2n-2^{k-3}-4}, & n \ge2^{k-3}+3,\\
1, & n < 2^{k-3}+3.
\end{cases} \tag*{(81)} eff = { 2 n − 2 k − 3 − 4 2 k − 3 + 2 , 1 , n ≥ 2 k − 3 + 3 , n < 2 k − 3 + 3. (81)
The bounded-reachability transformer of Theorem 3.8 is not asserted to satisfy this additional premise.
Proof. Substitute L = 1 + 2 k − 3 L=1+2^{k-3} L = 1 + 2 k − 3 into Theorem B.4.
Proof of Theorem 4.2
Permutation symmetry and reduction to scalar parameters
Proof of Theorem 4.1. Write θ = ( A 1 , A 2 , B 1 , B 2 ) \theta=(A_1,A_2,B_1,B_2) θ = ( A 1 , A 2 , B 1 , B 2 ) and, for π ∈ S V \pi\in S_V π ∈ S V , let P π ∈ R N × N P_\pi\in\mathbb{R}^{N \times N} P π ∈ R N × N be its permutation matrix, P π e v = e π ( v ) P_\pi e_v=e_{\pi(v)} P π e v = e π ( v ) . Define the action π ⋅ θ : = ( P π A 1 P π T , P π A 2 P π T , P π B 1 P π T , P π B 2 P π T ) \pi\cdot\theta:= (P_\pi A_1P_\pi^{T},P_\pi A_2P_\pi^{T},P_\pi B_1P_\pi^{T},P_\pi B_2P_\pi^{T}) π ⋅ θ := ( P π A 1 P π T , P π A 2 P π T , P π B 1 P π T , P π B 2 P π T ) ; it is linear and orthogonal for the Frobenius inner product.
Step 1: equivariance of the network. Relabeling the vertices of a prompt and a trace by π \pi π replaces every node one-hot e v e_v e v by P π e v P_\pi e_v P π e v and leaves the special tokens, the positions, and the length unchanged, so the input embeddings become Π X \Pi X Π X with Π = diag ( P π , P π , P π , P π ) \Pi=\operatorname{diag}(P_\pi,P_\pi,P_\pi,P_\pi) Π = diag ( P π , P π , P π , P π ) on the four node blocks and the identity on the constant and positional coordinates. The fixed first layer (Theorem B.1) computes its attention scores from the positional coordinates only, its value maps copy node blocks or the constant vector 1 N = P π 1 N \mathbf{1}_N=P_\pi\mathbf{1}_N 1 N = P π 1 N , and the ReLU and the residual act entrywise; hence its output transforms as X ( 2 ) ↦ Π X ( 2 ) X^{(2)} \mapsto\Pi X^{(2)} X ( 2 ) ↦ Π X ( 2 ) (exactly, also with leakage δ > 0 \delta>0 δ > 0 ). In the second layer, writing x j = ( a j , b j , c j , d j ) x_j=(a_j,b_j,c_j,d_j) x j = ( a j , b j , c j , d j ) for the four node blocks,
( Π x j ) T W K Q ( 2 , 1 ) [ π ⋅ θ ] Π x i = a j T P π T P π A 1 P π T P π a i + c j T P π T P π A 2 P π T P π a i = x j T W K Q ( 2 , 1 ) [ θ ] x i (\Pi x_j)^T W_{KQ}^{(2,1)}[\pi\cdot\theta]\Pi x_i
= a_j^T P_\pi^T P_\pi A_1P_\pi^T P_\pi a_i+c_j^T P_\pi^T P_\pi A_2P_\pi^T P_\pi a_i
= x_j^T W_{KQ}^{(2,1)}[\theta]x_i ( Π x j ) T W K Q ( 2 , 1 ) [ π ⋅ θ ] Π x i = a j T P π T P π A 1 P π T P π a i + c j T P π T P π A 2 P π T P π a i = x j T W K Q ( 2 , 1 ) [ θ ] x i
and likewise for the second head. The value maps satisfy W V ( 2 , h ) Π x j = P π W V ( 2 , h ) x j \mathbf{W}^{(2,h)}_{V}\boldsymbol{\Pi}\mathbf{x}_{j}=\mathbf{P}_{\pi}\mathbf{W}^{(2,h)}_{V}\mathbf{x}_{j} W V ( 2 , h ) Π x j = P π W V ( 2 , h ) x j , the output matrix W O ( 2 ) \mathbf{W}^{(2)}_{O} W O ( 2 ) only routes blocks, ReLU ( P π y ) = P π ReLU ( y ) \operatorname{ReLU}(\mathbf{P}_{\pi}y)=\mathbf{P}_{\pi}\operatorname{ReLU}(y) ReLU ( P π y ) = P π ReLU ( y ) , and the dynamic readout satisfies diag ( I π ( n ) − 1 , 0 , I N − π ( n ) ) = P π diag ( I n − 1 , 0 , I N − n ) P π T \operatorname{diag}(I_{\pi(n)-1},0,I_{N-\pi(n)})=\mathbf{P}_{\pi}\operatorname{diag}(I_{n-1},0,I_{N-n})\mathbf{P}_{\pi}^{T} diag ( I π ( n ) − 1 , 0 , I N − π ( n ) ) = P π diag ( I n − 1 , 0 , I N − n ) P π T . Hence the vertex logits satisfy TF ( Π X ; π ⋅ θ ) = P π TF ( X ; θ ) \operatorname{TF}(\boldsymbol{\Pi}\mathbf{X};\pi\cdot\theta)=\mathbf{P}_{\pi}\operatorname{TF}(\mathbf{X};\theta) TF ( Π X ; π ⋅ θ ) = P π TF ( X ; θ ) and p π ⋅ θ ( π ( v ) ; Π X ) = p θ ( v ; X ) p_{\pi\cdot\theta}(\pi(v);\boldsymbol{\Pi}\mathbf{X})=p_{\theta}(v;\mathbf{X}) p π ⋅ θ ( π ( v ) ; Π X ) = p θ ( v ; X ) . Consequently, for every prompt, prefix, and trace,
ℓ π ( τ P r o m p t ) , π ( τ p r e ) , π ( τ 0 ) ( π ⋅ θ ) = ℓ τ P r o m p t , τ p r e , τ 0 ( θ ) . (82) \ell_{\pi(\tau_{\mathrm{Prompt}}),\pi(\tau_{\mathrm{pre}}),\pi(\tau_{0})}(\pi\cdot\theta)=\ell_{\tau_{\mathrm{Prompt}},\tau_{\mathrm{pre}},\tau_{0}}(\theta). \tag*{(82)} ℓ π ( τ Prompt ) , π ( τ pre ) , π ( τ 0 ) ( π ⋅ θ ) = ℓ τ Prompt , τ pre , τ 0 ( θ ) . (82)
Step 2: invariance of the averaged loss. Fix σ ∈ S V \sigma\in S_{V} σ ∈ S V . Applying Equation (82) with π = σ \pi=\sigma π = σ to the instance relabeled by σ − 1 π ′ \sigma^{-1}\pi' σ − 1 π ′ gives ℓ π ′ ( ⋅ ) ( σ ⋅ θ ) = ℓ σ − 1 π ′ ( ⋅ ) ( θ ) \ell_{\pi'(\cdot)}(\sigma\cdot\theta)=\ell_{\sigma^{-1}\pi'(\cdot)}(\theta) ℓ π ′ ( ⋅ ) ( σ ⋅ θ ) = ℓ σ − 1 π ′ ( ⋅ ) ( θ ) ; averaging over π ′ ∈ S V \pi'\in S_{V} π ′ ∈ S V and using that π ′ ↦ σ − 1 π ′ \pi'\mapsto\sigma^{-1}\pi' π ′ ↦ σ − 1 π ′ is a bijection yields L t r a i n ( σ ⋅ θ ) = L t r a i n ( θ ) \mathcal{L}_{\mathrm{train}}(\sigma\cdot\theta)=\mathcal{L}_{\mathrm{train}}(\theta) L train ( σ ⋅ θ ) = L train ( θ ) for all θ \theta θ and σ \sigma σ .
Step 3: the fixed subspace is invariant. A matrix commuting with all permutation matrices has constant diagonal and constant off-diagonal entries, because S N S_{N} S N acts transitively on { ( i , i ) } \{(i,i)\} {( i , i )} and on { ( i , j ) : i ≠ j } \{(i,j):i\ne j\} {( i , j ) : i = j } . Hence the fixed-point set { θ : σ ⋅ θ = θ ∀ σ } \{\theta:\sigma\cdot\theta=\theta\ \forall\sigma\} { θ : σ ⋅ θ = θ ∀ σ } is exactly F \mathcal{F} F , and c E = c I + c E o f f ∈ F cE=cI+cE_{\mathrm{off}}\in\mathcal{F} c E = c I + c E off ∈ F . Wherever L t r a i n \mathcal{L}_{\mathrm{train}} L train is differentiable, differentiating L t r a i n ∘ σ = L t r a i n \mathcal{L}_{\mathrm{train}}\circ\sigma=\mathcal{L}_{\mathrm{train}} L train ∘ σ = L train and using that the action is orthogonal gives ∇ L t r a i n ( σ ⋅ θ ) = σ ⋅ ∇ L t r a i n ( θ ) \nabla\mathcal{L}_{\mathrm{train}}(\sigma\cdot\theta)=\sigma\cdot\nabla\mathcal{L}_{\mathrm{train}}(\theta) ∇ L train ( σ ⋅ θ ) = σ ⋅ ∇ L train ( θ ) ; in particular ∇ L t r a i n ( θ ) ∈ F \nabla\mathcal{L}_{\mathrm{train}}(\theta)\in\mathcal{F} ∇ L train ( θ ) ∈ F for θ ∈ F \theta\in\mathcal{F} θ ∈ F , so the vector field − α ∇ L t r a i n -\alpha\nabla\mathcal{L}_{\mathrm{train}} − α ∇ L train is tangent to F \mathcal{F} F . At a nonsmooth point the same chain rule holds for the Clarke generalized gradient, ∂ L t r a i n ( σ ⋅ θ ) = σ ⋅ ∂ L t r a i n ( θ ) \partial\mathcal{L}_{\mathrm{train}}(\sigma\cdot\theta)=\sigma\cdot\partial\mathcal{L}_{\mathrm{train}}(\theta) ∂ L train ( σ ⋅ θ ) = σ ⋅ ∂ L train ( θ ) , so for θ ∈ F \theta\in\mathcal{F} θ ∈ F the convex set ∂ L t r a i n ( θ ) \partial\mathcal{L}_{\mathrm{train}}(\theta) ∂ L train ( θ ) is S V S_{V} S V -invariant and therefore contains the group average of each of its elements; since the group average is the orthogonal projection Π F \Pi_{\mathcal{F}} Π F onto F \mathcal{F} F , we get Π F ∂ L t r a i n ( θ ) ⊂ ∂ L t r a i n ( θ ) \Pi_{\mathcal{F}}\partial\mathcal{L}_{\mathrm{train}}(\theta)\subset\partial\mathcal{L}_{\mathrm{train}}(\theta) Π F ∂ L train ( θ ) ⊂ ∂ L train ( θ ) . Together with the chain-rule inclusion ∂ ( L t r a i n ∣ F ) ( θ ) ⊂ Π F ∂ L t r a i n ( θ ) \partial(\mathcal{L}_{\mathrm{train}}|_{\mathcal{F}})(\theta)\subset\Pi_{\mathcal{F}}\partial\mathcal{L}_{\mathrm{train}}(\theta) ∂ ( L train ∣ F ) ( θ ) ⊂ Π F ∂ L train ( θ ) for the linear embedding of F \mathcal{F} F , every solution of the gradient-flow inclusion restricted to F \mathcal{F} F is a solution of the full inclusion, and whenever L t r a i n \mathcal{L}_{\mathrm{train}} L train is differentiable along the solution it is the unique one.
Step 4: reduction to one instance. For θ ∈ F \theta\in\mathcal{F} θ ∈ F , Equation (82) gives ℓ π ( ⋅ ) ( θ ) = ℓ ( ⋅ ) ( π − 1 ⋅ θ ) = ℓ ( ⋅ ) ( θ ) \ell_{\pi(\cdot)}(\theta)=\ell_{(\cdot)}(\pi^{-1}\cdot\theta)=\ell_{(\cdot)}(\theta) ℓ π ( ⋅ ) ( θ ) = ℓ ( ⋅ ) ( π − 1 ⋅ θ ) = ℓ ( ⋅ ) ( θ ) , hence L t r a i n ( θ ) = ℓ τ P r o m p t , τ p r e , τ 0 ( θ ) \mathcal{L}_{\mathrm{train}}(\theta)=\ell_{\tau_{\mathrm{Prompt}},\tau_{\mathrm{pre}},\tau_{0}}(\theta) L train ( θ ) = ℓ τ Prompt , τ pre , τ 0 ( θ ) on F \mathcal{F} F .
Since the weight matrices remain permutation invariant, we can transform the training on the whole matrices to training on the eight scalar parameters. Throughout the appendix,
ℓ ( a 1 , b 1 , a 2 , b 2 , g 1 , d 1 , g 2 , d 2 ) : = ℓ τ P r o m p t , τ p r e , τ 0 ( a 1 I + b 1 E o f f , a 2 I + b 2 E o f f , g 1 I + d 1 E o f f , g 2 I + d 2 E o f f ) \ell(a_{1},b_{1},a_{2},b_{2},g_{1},d_{1},g_{2},d_{2}):=\ell_{\tau_{\mathrm{Prompt}},\tau_{\mathrm{pre}},\tau_{0}}(a_{1}I+b_{1}E_{\mathrm{off}},a_{2}I+b_{2}E_{\mathrm{off}},g_{1}I+d_{1}E_{\mathrm{off}},g_{2}I+d_{2}E_{\mathrm{off}}) ℓ ( a 1 , b 1 , a 2 , b 2 , g 1 , d 1 , g 2 , d 2 ) := ℓ τ Prompt , τ pre , τ 0 ( a 1 I + b 1 E off , a 2 I + b 2 E off , g 1 I + d 1 E off , g 2 I + d 2 E off )
denotes the loss of the canonical instance viewed as a function of the eight scalars, and ∂ ℓ / ∂ a i \partial\ell/\partial a_{i} ∂ ℓ / ∂ a i etc. denote its ordinary partial derivatives, with the convention ReLU ′ ( x ) = 1 x ≥ 0 \operatorname{ReLU}'(x)=1_{x\ge0} ReLU ′ ( x ) = 1 x ≥ 0 at the kinks.
Lemma C.1. Let ( A 1 , A 2 , B 1 , B 2 ) ( t ) (A_{1},A_{2},B_{1},B_{2})(t) ( A 1 , A 2 , B 1 , B 2 ) ( t ) be a solution of the gradient flow θ ˙ = − α ∇ θ L t r a i n \dot{\theta}=-\alpha\nabla_{\theta}\mathcal{L}_{\mathrm{train}} θ ˙ = − α ∇ θ L train started in F \mathcal{F} F , so that A 1 = a 1 I + b 1 E o f f , … , B 2 = g 2 I + d 2 E o f f A_{1}=a_{1}I+b_{1}E_{\mathrm{off}},\ldots,B_{2}=g_{2}I+d_{2}E_{\mathrm{off}} A 1 = a 1 I + b 1 E off , … , B 2 = g 2 I + d 2 E off by Theorem 4.1 . Then the scalars solve
a ˙ i = − α N ∂ ℓ ∂ a i , g ˙ i = − α N ∂ ℓ ∂ g i , (83) \dot{a}_{i}=-\frac{\alpha}{N}\frac{\partial\ell}{\partial a_{i}},\qquad\dot{g}_{i}=-\frac{\alpha}{N}\frac{\partial\ell}{\partial g_{i}}, \tag*{(83)} a ˙ i = − N α ∂ a i ∂ ℓ , g ˙ i = − N α ∂ g i ∂ ℓ , (83) b ˙ i = − α N ( N − 1 ) ∂ ℓ ∂ b i , d ˙ i = − α N ( N − 1 ) ∂ ℓ ∂ d i , i ∈ { 1 , 2 } , (84) \dot{b}_{i}=-\frac{\alpha}{N(N-1)}\frac{\partial\ell}{\partial b_{i}},\qquad\dot{d}_{i}=-\frac{\alpha}{N(N-1)}\frac{\partial\ell}{\partial d_{i}},\qquad i\in\{1,2\}, \tag*{(84)} b ˙ i = − N ( N − 1 ) α ∂ b i ∂ ℓ , d ˙ i = − N ( N − 1 ) α ∂ d i ∂ ℓ , i ∈ { 1 , 2 } , (84) that is, the scalars follow the gradient flow of the single-instance loss ℓ \ell ℓ with the learning rate normalization
α a = α g = α N , α b = α d = α N ( N − 1 ) . (85) \alpha_{a}=\alpha_{g}=\frac{\alpha}{N},\qquad\alpha_{b}=\alpha_{d}=\frac{\alpha}{N(N-1)}. \tag*{(85)} α a = α g = N α , α b = α d = N ( N − 1 ) α . (85) Proof. By Step 4 of the proof of Theorem 4.1 , L t r a i n = ℓ \mathcal{L}_{\mathrm{train}}=\ell L train = ℓ on F \mathcal{F} F , and by Equation (82) , L t r a i n ( θ ) = 1 N ! ∑ π ℓ τ P r o m p t , τ p r e , τ 0 ( π ⋅ θ ) \mathcal{L}_{\mathrm{train}}(\theta)=\frac{1}{N!}\sum_{\pi}\ell_{\tau_{\mathrm{Prompt}},\tau_{\mathrm{pre}},\tau_{0}}(\pi\cdot\theta) L train ( θ ) = N ! 1 ∑ π ℓ τ Prompt , τ pre , τ 0 ( π ⋅ θ ) for every θ \theta θ . Differentiating at θ ∈ F \theta\in\mathcal{F} θ ∈ F , where π ⋅ θ = θ \pi\cdot\theta=\theta π ⋅ θ = θ ,
∇ A 1 L t r a i n ( θ ) = 1 N ! ∑ π P π T G P π , G : = ∇ A 1 ℓ τ P r o m p t , τ p r e , τ 0 ( θ ) , \nabla_{A_{1}}\mathcal{L}_{\mathrm{train}}(\theta)=\frac{1}{N!}\sum_{\pi}\mathbf{P}_{\pi}^{T}G\mathbf{P}_{\pi},\qquad G:=\nabla_{A_{1}}\ell_{\tau_{\mathrm{Prompt}},\tau_{\mathrm{pre}},\tau_{0}}(\theta), ∇ A 1 L train ( θ ) = N ! 1 π ∑ P π T G P π , G := ∇ A 1 ℓ τ Prompt , τ pre , τ 0 ( θ ) ,
and for any matrix G \mathbf{G} G the conjugation average equals Tr G N I + ∑ k ≠ l G k l N ( N − 1 ) E o f f \frac{\operatorname{Tr}G}{N}I+\frac{\sum_{k\ne l}G_{kl}}{N(N-1)}E_{\mathrm{off}} N Tr G I + N ( N − 1 ) ∑ k = l G k l E off . By the chain rule, Tr G = ∑ k ∂ ℓ / ∂ ( A 1 ) k k = ∂ ℓ / ∂ a 1 \operatorname{Tr}G=\sum_k\partial\ell/\partial(A_1)_{kk}=\partial\ell/\partial a_1 Tr G = ∑ k ∂ ℓ / ∂ ( A 1 ) k k = ∂ ℓ / ∂ a 1 and ∑ k ≠ l G k l = ∂ ℓ / ∂ b 1 \sum_{k\ne l}G_{kl}=\partial\ell/\partial b_1 ∑ k = l G k l = ∂ ℓ / ∂ b 1 . Hence
∇ A 1 L t r a i n = 1 N ∂ ℓ ∂ a 1 I + 1 N ( N − 1 ) ∂ ℓ ∂ b 1 E o f f , \nabla_{\mathbf{A}_1}\mathcal{L}_{\mathrm{train}}=\frac{1}{N}\frac{\partial\ell}{\partial a_1}I+\frac{1}{N(N-1)}\frac{\partial\ell}{\partial b_1}E_{\mathrm{off}}, ∇ A 1 L train = N 1 ∂ a 1 ∂ ℓ I + N ( N − 1 ) 1 ∂ b 1 ∂ ℓ E off ,
and likewise for A 2 , B 1 , B 2 \mathbf{A}_2,\mathbf{B}_1,\mathbf{B}_2 A 2 , B 1 , B 2 . Comparing the coefficients of I I I and E o f f E_{\mathrm{off}} E off in A ˙ 1 = − α ∇ A 1 L t r a i n \dot{\mathbf{A}}_1=-\alpha\nabla_{\mathbf{A}_1}\mathcal{L}_{\mathrm{train}} A ˙ 1 = − α ∇ A 1 L train gives (83) . At a kink of the ReLU the same identity holds for every fixed permutation-equivariant selection of the derivative, in particular for ReLU ′ ( 0 ) = 1 \operatorname{ReLU}'(0)=1 ReLU ′ ( 0 ) = 1 .
To calculate the gradient with respect to each parameter, we need the formulation of the logits TF ( X , θ ) \operatorname{TF}(X,\theta) TF ( X , θ ) . Since TF ( X , θ ) \operatorname{TF}(X,\theta) TF ( X , θ ) is the sum of the attention head values after ReLU, we first compute the node logits of each head separately, for a general directed graph and a general valid DFS prefix ( n 0 = n R o o t , n 1 , … , n t ) (n_0=n_{\mathrm{Root}},n_1,\ldots,n_t) ( n 0 = n Root , n 1 , … , n t ) . We summarize the node logits in Table 1 and Table 2 , separately.
Table 2. Node logits of the backtracking head (pre-ReLU). k k k is the number of children of n t n_t n t that the DFS has entered from n t n_t n t ; since the search is currently at n t n_t n t , each of these excursions has been completed. “Any other v v v ” covers unvisited children, unvisited non-children, and visited vertices that are neither the parent, the root, nor a tree child of n t n_t n t . A visited out-neighbor of n t n_t n t that was reached by another route (the parent, the root, an ancestor, or a vertex finished in another branch) is not a tree child and falls into the parent, root, or “any other” row.
Table 1 (PDF p. 34). Node logits of the forward head (pre-ReLU). A child of n t n_t n t is a vertex v v v with ( n t → v ) ∈ E (n_t\to v)\in E ( n t → v ) ∈ E . m v m_v m v is the number of occurrences of v v v in ( n R o o t , n 1 , … , n t ) (n_{\mathrm{Root}},n_1,\ldots,n_t) ( n Root , n 1 , … , n t ) , i.e., in the CoT trace including the root token. d v i n , t o t a l d_v^{\mathrm{in,total}} d v in , total is the in-degree of v v v in the prompt’s edge list (self-loops included). The coordinate v = n t v=n_t v = n t is masked by the dynamic readout.
Derivation of the tables. Let the query be the last token, whose first block is n t n_t n t . Since A 1 n t = a 1 n t + b 1 ( 1 − n t ) \mathbf{A}_1n_t=a_1n_t+b_1(1-n_t) A 1 n t = a 1 n t + b 1 ( 1 − n t ) , and likewise for A 2 , B 1 , B 2 \mathbf{A}_2,\mathbf{B}_1,\mathbf{B}_2 A 2 , B 1 , B 2 , the only keys with nonzero score and nonzero value
are, for the forward head, the edge-target tokens ( 0 , 0 , s i , t i ) (0,0,s_i,t_i) ( 0 , 0 , s i , t i ) , with score a 2 a_2 a 2 if s i = n t s_i=n_t s i = n t and b 2 b_2 b 2 otherwise and value + t i +t_i + t i , and the root and CoT tokens ( n s ′ , 0 , 0 ) (n_{s'},0,0) ( n s ′ , 0 , 0 ) , 0 ≤ s ≤ t 0 \le s \le t 0 ≤ s ≤ t , with score a 1 a_1 a 1 if n s = n t n_s=n_t n s = n t and b 1 b_1 b 1 otherwise and value − n s -n_s − n s . Hence, for v ≠ n t v \ne n_t v = n t ,
( head 1 ) v = ( a 2 − b 2 ) 1 [ ( n t → v ) ∈ E ] + b 2 d v i n , t o t a l − b 1 m v , (\operatorname{head}_1)_v=(a_2-b_2)\mathbf{1}[(n_t\to v)\in\mathcal{E}]+b_2d_v^{\mathrm{in,total}}-b_1m_v, ( head 1 ) v = ( a 2 − b 2 ) 1 [( n t → v ) ∈ E ] + b 2 d v in , total − b 1 m v ,
which is Table 1 . For the backtracking head only the root token, with score g 1 1 [ n R o o t = n t ] + d 1 1 [ n R o o t ≠ n t ] g_1\mathbf{1}[n_{\mathrm{Root}}=n_t]+d_1\mathbf{1}[n_{\mathrm{Root}}\ne n_t] g 1 1 [ n Root = n t ] + d 1 1 [ n Root = n t ] and value − n R o o t -n_{\mathrm{Root}} − n Root , and the CoT tokens s ≥ 1 s\ge1 s ≥ 1 , with score σ s = d 1 + d 2 + ( g 1 − d 1 ) 1 [ n s = n t ] + ( g 2 − d 2 ) 1 [ n s − 1 = n t ] \sigma_s=d_1+d_2+(g_1-d_1)\mathbf{1}[n_s=n_t]+(g_2-d_2)\mathbf{1}[n_{s-1}=n_t] σ s = d 1 + d 2 + ( g 1 − d 1 ) 1 [ n s = n t ] + ( g 2 − d 2 ) 1 [ n s − 1 = n t ] and value n s − 1 − n s n_{s-1}-n_s n s − 1 − n s , contribute. For v ≠ n t v\ne n_t v = n t the ( d 1 + d 2 ) (d_1+d_2) ( d 1 + d 2 ) -part telescopes to ( d 1 + d 2 ) 1 [ v = n R o o t ] (d_1+d_2)\mathbf{1}[v=n_{\mathrm{Root}}] ( d 1 + d 2 ) 1 [ v = n Root ] , so
Table 1.
( head 2 ) v = ( g 1 − d 1 ) A v → n t − ( g 2 − d 2 ) A n t → v + d 2 1 [ v = n R o o t ] , (\operatorname{head}_2)_v=(g_1-d_1)A_{v\to n_t}-(g_2-d_2)A_{n_t\to v}+d_2\mathbf{1}[v=n_{\mathrm{Root}}], ( head 2 ) v = ( g 1 − d 1 ) A v → n t − ( g 2 − d 2 ) A n t → v + d 2 1 [ v = n Root ] ,
where A x → y A_{x\to y} A x → y is the number of CoT steps x → y x\to y x → y . In a valid DFS prefix, n t n_t n t is entered once from its stack parent if n t ≠ n R o o t n_t\ne n_{\mathrm{Root}} n t = n Root , and otherwise only by the k k k returns from the children it has entered, each of which it left exactly once; it has not yet left towards its parent. Substituting gives the rows for v ≠ n t v\ne n_t v = n t , every remaining vertex having value 0 0 0 . For v = n t v=n_t v = n t , each arrival contributes − ( g 1 + d 2 ) -(g_1+d_2) − ( g 1 + d 2 ) , each departure + ( g 2 + d 1 ) +(g_2+d_1) + ( g 2 + d 1 ) , and the root token − g 1 -g_1 − g 1 if n t = n R o o t n_t=n_{\mathrm{Root}} n t = n Root , which gives the last two rows.
By Theorem C.1, we only need to analyze the gradient of the single-instance loss ℓ \ell ℓ on the canonical data E 0 , τ 0 \mathcal{E}_0,\tau_0 E 0 , τ 0 . Recall the initialization Equation (16) : a 1 = b 1 = a 2 = g 1 = g 2 = d 2 = c 0 a_1=b_1=a_2=g_1=g_2=d_2=c_0 a 1 = b 1 = a 2 = g 1 = g 2 = d 2 = c 0 and b 2 = d 1 = c 0 ′ b_2=d_1=c'_0 b 2 = d 1 = c 0 ′ with 0 < c 0 ′ < c 0 0<c'_0<c_0 0 < c 0 ′ < c 0 . Throughout we write
u : = a 2 − b 1 , v : = g 1 − d 1 − g 2 + d 2 , E : = e u + + v + , (86) u:=a_2-b_1,\qquad v:=g_1-d_1-g_2+d_2,\qquad E:=e^{u_++v_+}, \tag*{(86)} u := a 2 − b 1 , v := g 1 − d 1 − g 2 + d 2 , E := e u + + v + , (86)
and we use the following region of parameter space:
R : = { b 1 > 0 , b 2 < b 1 , a 2 > 0 , g 1 − d 1 > 0 } . (87) \mathcal{R}:=\{b_1>0,\ b_2<b_1,\ a_2>0,\ g_1-d_1>0\}. \tag*{(87)} R := { b 1 > 0 , b 2 < b 1 , a 2 > 0 , g 1 − d 1 > 0 } . (87)
The region is open on purpose: on the faces a 2 = 0 a_2=0 a 2 = 0 and g 1 = d 1 g_1=d_1 g 1 = d 1 the ReLUs ( a 2 ) + (a_2)_+ ( a 2 ) + and ( g 1 − d 1 ) + (g_1-d_1)_+ ( g 1 − d 1 ) + are at their kinks, where a Filippov solution may select any value in [ 0 , 1 ] [0,1] [ 0 , 1 ] for their derivative, so the gradient formulas below hold on R \mathcal{R} R but not on these faces.
Lemma C.2 (Scalar parameter gradient on canonical data). There is a constant c ∗ = c ∗ ( K ) > 0 c_* = c_*(K)>0 c ∗ = c ∗ ( K ) > 0 such that the following holds for the flow Equation (83) started from Equation (16) with 0 < c 0 ′ < c 0 ≤ c ∗ N − 2 0<c'_0<c_0\le c_*N^{-2} 0 < c 0 ′ < c 0 ≤ c ∗ N − 2 .
(Invariant region.) The trajectory stays in R \mathcal{R} R for all t ≥ 0 t\ge0 t ≥ 0 . More precisely, a 1 ≡ c 0 a_1\equiv c_0 a 1 ≡ c 0 ; b 1 b_1 b 1 is non-decreasing and b 2 b_2 b 2 is non-increasing with 0 ≤ b 2 ≤ c 0 ′ 0\le b_2\le c'_0 0 ≤ b 2 ≤ c 0 ′ , so b 1 ≥ c 0 > 0 b_1\ge c_0>0 b 1 ≥ c 0 > 0 and b 2 < b 1 b_2<b_1 b 2 < b 1 ; u = a 2 − b 1 ≥ 0 u=a_2-b_1\ge0 u = a 2 − b 1 ≥ 0 , hence a 2 ≥ b 1 ≥ c 0 > 0 a_2\ge b_1\ge c_0>0 a 2 ≥ b 1 ≥ c 0 > 0 ; g 1 − d 1 > 0 g_1-d_1>0 g 1 − d 1 > 0 ; g 2 − d 2 g_2-d_2 g 2 − d 2 is non-decreasing with g 2 − d 2 ≥ 0 g_2-d_2\ge0 g 2 − d 2 ≥ 0 ; and d 2 d_2 d 2 is non-increasing.
(Initial phase.) With T 0 : = N 4 ( K + 1 ) α T_0:=\frac{N}{4(K+1)\alpha} T 0 := 4 ( K + 1 ) α N , we have b 2 ( t ) = 0 b_2(t)=0 b 2 ( t ) = 0 and d 2 ( t ) ≤ 0 d_2(t)\le0 d 2 ( t ) ≤ 0 for all t ≥ T 0 t\ge T_0 t ≥ T 0 .
(Gradient after the initial phase.) *For t ≥ T 0 t\ge T_0 t ≥ T 0 ,
∂ ℓ ∂ a 1 = 0 , (88) \frac{\partial\ell}{\partial a_1}=0, \tag*{(88)} ∂ a 1 ∂ ℓ = 0 , (88) ∂ ℓ ∂ a 2 = − e g 1 − d 1 + N − 2 D 1 − ∑ i = 1 K e g 1 − d 1 + N − K − 1 + ( i − 1 ) E 1 u < 0 D 2 , i + K E 1 u ≥ 0 D 4 , (89) \frac{\partial\ell}{\partial a_2}
=-\frac{e^{g_1-d_1}+N-2}{D_1}
-\sum_{i=1}^{K}\frac{e^{g_1-d_1}+N-K-1+(i-1)E\mathbf{1}_{u<0}}{D_{2,i}}
+\frac{KE\mathbf{1}_{u\ge0}}{D_4}, \tag*{(89)} ∂ a 2 ∂ ℓ = − D 1 e g 1 − d 1 + N − 2 − i = 1 ∑ K D 2 , i e g 1 − d 1 + N − K − 1 + ( i − 1 ) E 1 u < 0 + D 4 K E 1 u ≥ 0 , (89)
∂ ℓ ∂ b 1 = − ∑ i = 1 K ( i − 1 ) E 1 u ≥ 0 D 2 , i − K E 1 u ≥ 0 D 4 , (90) \frac{\partial\ell}{\partial b_1}=-\sum_{i=1}^{K}\frac{(i-1)\mathbb{E}1_{u\geq0}}{D_{2,i}}-\frac{K\mathbb{E}1_{u\geq0}}{D_4}, \tag*{(90)} ∂ b 1 ∂ ℓ = − i = 1 ∑ K D 2 , i ( i − 1 ) E 1 u ≥ 0 − D 4 K E 1 u ≥ 0 , (90)
b 2 ≡ 0 ( so the selected value of ∂ ℓ ∂ b 2 along the flow is 0 ) , ( b 2 ) + = ( b 2 − b 1 ) + = 0 , (91) b_2\equiv0\quad\left(\text{so the selected value of }\frac{\partial\ell}{\partial b_2}\text{ along the flow is }0\right),\qquad(b_2)_+=(b_2-b_1)_+=0, \tag*{(91)} b 2 ≡ 0 ( so the selected value of ∂ b 2 ∂ ℓ along the flow is 0 ) , ( b 2 ) + = ( b 2 − b 1 ) + = 0 , (91)
∂ ℓ ∂ g 1 = e g 1 − d 1 D 1 + ∑ i = 1 K ( i − 1 ) E 1 v ≥ 0 + e g 1 − d 1 D 2 , i − K ( N − 1 ) D 3 − N − K − 1 + K E 1 v < 0 D 4 , (92) \frac{\partial\ell}{\partial g_1}=\frac{e^{g_1-d_1}}{D_1}+\sum_{i=1}^{K}\frac{(i-1)\mathbb{E}1_{v\geq0}+e^{g_1-d_1}}{D_{2,i}}-\frac{K(N-1)}{D_3}-\frac{N-K-1+K\mathbb{E}1_{v<0}}{D_4}, \tag*{(92)} ∂ g 1 ∂ ℓ = D 1 e g 1 − d 1 + i = 1 ∑ K D 2 , i ( i − 1 ) E 1 v ≥ 0 + e g 1 − d 1 − D 3 K ( N − 1 ) − D 4 N − K − 1 + K E 1 v < 0 , (92)
∂ ℓ ∂ g 2 = − ∑ i = 1 K ( i − 1 ) E 1 v ≥ 0 D 2 , i − K E 1 v ≥ 0 D 4 , (93) \frac{\partial\ell}{\partial g_2}=-\sum_{i=1}^{K}\frac{(i-1)\mathbb{E}1_{v\geq0}}{D_{2,i}}-\frac{K\mathbb{E}1_{v\geq0}}{D_4}, \tag*{(93)} ∂ g 2 ∂ ℓ = − i = 1 ∑ K D 2 , i ( i − 1 ) E 1 v ≥ 0 − D 4 K E 1 v ≥ 0 , (93)
∂ ℓ ∂ d 1 = − ∂ ℓ ∂ g 1 , ∂ ℓ ∂ d 2 = − ∂ ℓ ∂ g 2 wherever d 2 < 0 , (94) \frac{\partial\ell}{\partial d_1}=-\frac{\partial\ell}{\partial g_1},\qquad\frac{\partial\ell}{\partial d_2}=-\frac{\partial\ell}{\partial g_2}\quad\text{wherever }d_2<0, \tag*{(94)} ∂ d 1 ∂ ℓ = − ∂ g 1 ∂ ℓ , ∂ d 2 ∂ ℓ = − ∂ g 2 ∂ ℓ wherever d 2 < 0 , (94)
where the denominators are
D 1 = e a 2 + e g 1 − d 1 + N − 2 , (95) D_1=e^{a_2}+e^{g_1-d_1}+N-2, \tag*{(95)} D 1 = e a 2 + e g 1 − d 1 + N − 2 , (95)
D 2 , i = ( K − i + 1 ) e a 2 + ( i − 1 ) E + e g 1 − d 1 + N − K − 1 , (96) D_{2,i}=(K-i+1)e^{a_2}+(i-1)E+e^{g_1-d_1}+N-K-1, \tag*{(96)} D 2 , i = ( K − i + 1 ) e a 2 + ( i − 1 ) E + e g 1 − d 1 + N − K − 1 , (96)
D 3 = e g 1 − d 1 + N − 1 , (97) D_3=e^{g_1-d_1}+N-1, \tag*{(97)} D 3 = e g 1 − d 1 + N − 1 , (97)
D 4 = K E + e g 1 − d 1 + N − K − 1. (98) D_4=KE+e^{g_1-d_1}+N-K-1. \tag*{(98)} D 4 = K E + e g 1 − d 1 + N − K − 1. (98)
Note that the relation ∂ ℓ / ∂ d 1 = − ∂ ℓ / ∂ g 1 \partial\ell/\partial d_1=-\partial\ell/\partial g_1 ∂ ℓ / ∂ d 1 = − ∂ ℓ / ∂ g 1 holds for all t ≥ 0 t\geq0 t ≥ 0 , because by Table 2 every unmasked logit depends on ( g 1 , d 1 ) (g_1,d_1) ( g 1 , d 1 ) only through g 1 − d 1 g_1-d_1 g 1 − d 1 ; the relation for ( g 2 , d 2 ) (g_2,d_2) ( g 2 , d 2 ) holds wherever d 2 < 0 d_2<0 d 2 < 0 ; at times t ≥ T 0 t\geq T_0 t ≥ T 0 with d 2 = 0 d_2=0 d 2 = 0 it can fail, but g 2 + ( N − 1 ) d 2 g_2+(N-1)d_2 g 2 + ( N − 1 ) d 2 is still conserved (Step 5 of the proof).
Proof. For the cross-entropy loss − log ( p θ ( v ; X ) ) -\log(p_\theta(v;\mathbf{X})) − log ( p θ ( v ; X )) , the gradient with respect to a parameter θ \theta θ is
∂ ( − log ( p θ ( v ; X ) ) ) ∂ θ = − ∂ TF ( X , θ ) v ∂ θ + ∑ v ′ ∈ V p θ ( v ′ ; X ) ∂ TF ( X , θ ) v ′ ∂ θ , (99) \frac{\partial(-\log(p_\theta(v;\mathbf{X})))}{\partial\theta}=-\frac{\partial\operatorname{TF}(\mathbf{X},\theta)_v}{\partial\theta}+\sum_{v'\in\mathcal{V}}p_\theta(v';\mathbf{X})\frac{\partial\operatorname{TF}(\mathbf{X},\theta)_{v'}}{\partial\theta}, \tag*{(99)} ∂ θ ∂ ( − log ( p θ ( v ; X ))) = − ∂ θ ∂ TF ( X , θ ) v + v ′ ∈ V ∑ p θ ( v ′ ; X ) ∂ θ ∂ TF ( X , θ ) v ′ , (99)
and the gradient of ℓ \ell ℓ is the sum of Equation (99) over the 2 K + 2 2K+2 2 K + 2 steps of τ 0 \tau_0 τ 0 .
Step 1: node logits along the canonical trace. In the canonical trace, DFS starts from node N − 3 N-3 N − 3 and moves forward to 1. After exploring all K K K child nodes of 1, it then returns to N − 3 N-3 N − 3 . Therefore, there are four types of steps in the trajectory. We next compute the node logits for these four steps from Tables 1 and 2; all nodes 2 , … , K + 1 2,\ldots,K+1 2 , … , K + 1 have in-degree one, and the nodes N − 2 N-2 N − 2 , N − 3 N-3 N − 3 , 1 have in-degree one as well. Recall that the logit of the current node n t n_t n t is masked to 0, and that the remaining N − K − 4 N-K-4 N − K − 4 vertices are isolated and unvisited, so their logits are 0. In the following, we use p ( v ) p(v) p ( v ) as an abbreviation of p θ ( v ; X ) p_\theta(v;\mathbf{X}) p θ ( v ; X ) .
Case 1: N − 3 → 1 N-3\to1 N − 3 → 1 . The logits TF ( X , θ ) \operatorname{TF}(\mathbf{X},\theta) TF ( X , θ ) of all nodes are
v = 1 v=1 v = 1 : TF ( X , θ ) v = ( a 2 ) + \operatorname{TF}(\mathbf{X},\theta)_v=(a_2)_+ TF ( X , θ ) v = ( a 2 ) +
v = N − 2 v=N-2 v = N − 2 (parent of N − 3 N-3 N − 3 ): TF ( X , θ ) v = ( b 2 − b 1 ) + + ( g 1 − d 1 ) + \operatorname{TF}(\mathbf{X},\theta)_v=(b_2-b_1)_++(g_1-d_1)_+ TF ( X , θ ) v = ( b 2 − b 1 ) + + ( g 1 − d 1 ) +
v = N − 1 v=N-1 v = N − 1 (root): TF ( X , θ ) v = ( − b 1 ) + + ( d 2 ) + \operatorname{TF}(\mathbf{X},\theta)_v=(-b_1)_++(d_2)_+ TF ( X , θ ) v = ( − b 1 ) + + ( d 2 ) +
2 ≤ v ≤ K + 1 2\leq v\leq K+1 2 ≤ v ≤ K + 1 : TF ( X , θ ) v = ( b 2 ) + \operatorname{TF}(\mathbf{X},\theta)_v=(b_2)_+ TF ( X , θ ) v = ( b 2 ) +
Other: $\operatorname{TF}(\mathbf{X},\theta)_v=0
Case 2: 1 → i 1 \to i 1 → i for 2 ≤ i ≤ K + 1 2 \le i \le K + 1 2 ≤ i ≤ K + 1 . The logits TF ( X , θ ) \operatorname{TF}(X,\theta) TF ( X , θ ) of all nodes are
i ≤ v ≤ K + 1 i \le v \le K + 1 i ≤ v ≤ K + 1 (unvisited children): TF ( X , θ ) v = ( a 2 ) + \operatorname{TF}(X,\theta)_v = (a_2)_+ TF ( X , θ ) v = ( a 2 ) + .
2 ≤ v ≤ i − 1 2 \le v \le i - 1 2 ≤ v ≤ i − 1 (tree children, each visited once): TF ( X , θ ) v = ( a 2 − b 1 ) + + ( g 1 − d 1 − g 2 + d 2 ) + \operatorname{TF}(X,\theta)_v = (a_2 - b_1)_+ + (g_1 - d_1 - g_2 + d_2)_+ TF ( X , θ ) v = ( a 2 − b 1 ) + + ( g 1 − d 1 − g 2 + d 2 ) + .
v = N − 3 v = N - 3 v = N − 3 (parent): TF ( X , θ ) v = ( b 2 − b 1 ) + + ( g 1 − d 1 ) + \operatorname{TF}(X,\theta)_v = (b_2 - b_1)_+ + (g_1 - d_1)_+ TF ( X , θ ) v = ( b 2 − b 1 ) + + ( g 1 − d 1 ) + .
v = N − 2 v = N - 2 v = N − 2 : TF ( X , θ ) v = ( b 2 − b 1 ) + \operatorname{TF}(X,\theta)_v = (b_2 - b_1)_+ TF ( X , θ ) v = ( b 2 − b 1 ) + .
v = N − 1 v = N - 1 v = N − 1 : TF ( X , θ ) v = ( − b 1 ) + + ( d 2 ) + \operatorname{TF}(X,\theta)_v = (-b_1)_+ + (d_2)_+ TF ( X , θ ) v = ( − b 1 ) + + ( d 2 ) + .
Other: TF ( X , θ ) v = 0 \operatorname{TF}(X,\theta)_v = 0 TF ( X , θ ) v = 0 .
Case 3: i → 1 i \to1 i → 1 for 2 ≤ i ≤ K + 1 2 \le i \le K + 1 2 ≤ i ≤ K + 1 . At this step node 1 has occurred i − 1 i - 1 i − 1 times in the trace. The logits TF ( X , θ ) \operatorname{TF}(X,\theta) TF ( X , θ ) of all nodes are
v = 1 v = 1 v = 1 (parent): TF ( X , θ ) v = ( b 2 − ( i − 1 ) b 1 ) + + ( g 1 − d 1 ) + \operatorname{TF}(X,\theta)_v = (b_2 - (i - 1)b_1)_+ + (g_1 - d_1)_+ TF ( X , θ ) v = ( b 2 − ( i − 1 ) b 1 ) + + ( g 1 − d 1 ) + .
v = N − 2 v = N - 2 v = N − 2 or N − 3 N - 3 N − 3 : TF ( X , θ ) v = ( b 2 − b 1 ) + \operatorname{TF}(X,\theta)_v = (b_2 - b_1)_+ TF ( X , θ ) v = ( b 2 − b 1 ) + .
v = N − 1 v = N - 1 v = N − 1 : TF ( X , θ ) v = ( − b 1 ) + + ( d 2 ) + \operatorname{TF}(X,\theta)_v = (-b_1)_+ + (d_2)_+ TF ( X , θ ) v = ( − b 1 ) + + ( d 2 ) + .
2 ≤ v ≤ i − 1 2 \le v \le i - 1 2 ≤ v ≤ i − 1 : TF ( X , θ ) v = ( b 2 − b 1 ) + \operatorname{TF}(X,\theta)_v = (b_2 - b_1)_+ TF ( X , θ ) v = ( b 2 − b 1 ) + .
i + 1 ≤ v ≤ K + 1 i + 1 \le v \le K + 1 i + 1 ≤ v ≤ K + 1 : TF ( X , θ ) v = ( b 2 ) + \operatorname{TF}(X,\theta)_v = (b_2)_+ TF ( X , θ ) v = ( b 2 ) + .
Other: TF ( X , θ ) v = 0 \operatorname{TF}(X,\theta)_v = 0 TF ( X , θ ) v = 0 .
Case 4: 1 → N − 3 1 \to N - 3 1 → N − 3 . The logits TF ( X , θ ) \operatorname{TF}(X,\theta) TF ( X , θ ) of all nodes are
v = N − 3 v = N - 3 v = N − 3 (parent): TF ( X , θ ) v = ( b 2 − b 1 ) + + ( g 1 − d 1 ) + \operatorname{TF}(X,\theta)_v = (b_2 - b_1)_+ + (g_1 - d_1)_+ TF ( X , θ ) v = ( b 2 − b 1 ) + + ( g 1 − d 1 ) + .
v = N − 2 v = N - 2 v = N − 2 : TF ( X , θ ) v = ( b 2 − b 1 ) + \operatorname{TF}(X,\theta)_v = (b_2 - b_1)_+ TF ( X , θ ) v = ( b 2 − b 1 ) + .
v = N − 1 v = N - 1 v = N − 1 : TF ( X , θ ) v = ( − b 1 ) + + ( d 2 ) + \operatorname{TF}(X,\theta)_v = (-b_1)_+ + (d_2)_+ TF ( X , θ ) v = ( − b 1 ) + + ( d 2 ) + .
2 ≤ v ≤ K + 1 2 \le v \le K + 1 2 ≤ v ≤ K + 1 (tree children): TF ( X , θ ) v = ( a 2 − b 1 ) + + ( g 1 − d 1 − g 2 + d 2 ) + \operatorname{TF}(X,\theta)_v = (a_2 - b_1)_+ + (g_1 - d_1 - g_2 + d_2)_+ TF ( X , θ ) v = ( a 2 − b 1 ) + + ( g 1 − d 1 − g 2 + d 2 ) + .
Other: TF ( X , θ ) v = 0 \operatorname{TF}(X,\theta)_v = 0 TF ( X , θ ) v = 0 .
In the gradient formulas below we index the K K K steps of Case 2 and of Case 3 by i ∈ [ K ] i \in[K] i ∈ [ K ] , i.e., we shift the child label by one, so that at the i i i -th such step there are K − i + 1 K - i + 1 K − i + 1 unvisited and i − 1 i - 1 i − 1 visited children of node 1.
Step 2: the gradient on R \mathcal{R} R . On the region R \mathcal{R} R of Equation (87) we have b 2 − b 1 < 0 b_2 - b_1 < 0 b 2 − b 1 < 0 , b 2 − ( i − 1 ) b 1 ≤ b 2 − b 1 < 0 b_2 - (i - 1)b_1 \le b_2 - b_1 < 0 b 2 − ( i − 1 ) b 1 ≤ b 2 − b 1 < 0 and − b 1 < 0 -b_1 < 0 − b 1 < 0 , so the terms ( b 2 − b 1 ) + (b_2 - b_1)_+ ( b 2 − b 1 ) + , ( b 2 − ( i − 1 ) b 1 ) + (b_2 - (i - 1)b_1)_+ ( b 2 − ( i − 1 ) b 1 ) + , ( − b 1 ) + (-b_1)_+ ( − b 1 ) + and their derivatives vanish identically, while ( a 2 ) + = a 2 (a_2)_+ = a_2 ( a 2 ) + = a 2 and ( g 1 − d 1 ) + = g 1 − d 1 (g_1 - d_1)_+ = g_1 - d_1 ( g 1 − d 1 ) + = g 1 − d 1 with derivative 1, because a 2 > 0 a_2 > 0 a 2 > 0 and g 1 − d 1 > 0 g_1 - d_1 > 0 g 1 − d 1 > 0 on the open region R \mathcal{R} R . The remaining kinks u = 0 u = 0 u = 0 , v = 0 v = 0 v = 0 , b 2 = 0 b_2 = 0 b 2 = 0 and d 2 = 0 d_2 = 0 d 2 = 0 can lie in R \mathcal{R} R ; there the indicators 1 u ≥ 0 \mathbb{1}_{u \ge0} 1 u ≥ 0 , 1 v ≥ 0 \mathbb{1}_{v \ge0} 1 v ≥ 0 , 1 b 2 ≥ 0 \mathbb{1}_{b_2 \ge0} 1 b 2 ≥ 0 , 1 d 2 ≥ 0 \mathbb{1}_{d_2 \ge0} 1 d 2 ≥ 0 below stand for the values in [ 0 , 1 ] [0,1] [ 0 , 1 ] selected by the Filippov solution (the weights of the convex combination of the one-sided fields). Applying Equation (99) to the four cases gives, on R \mathcal{R} R ,
∂ L ∂ a 1 = 0 , (100) \frac{\partial\mathcal{L}}{\partial a_1} = 0, \tag*{(100)} ∂ a 1 ∂ L = 0 , (100)
∂ ℓ ∂ a 2 = − 1 + e a 2 D 1 + ∑ i = 1 K ( − 1 + ( K − i + 1 ) e a 2 + ( i − 1 ) E 1 u ≥ 0 D 2 , i ) + K E 1 u ≥ 0 D 4 , (101) \frac{\partial\ell}{\partial a_2}=-1+\frac{e^{a_2}}{D_1}+\sum_{i=1}^{K}\left(-1+\frac{(K-i+1)e^{a_2}+(i-1)E\mathbf{1}_{u\geq0}}{D_{2,i}}\right)+\frac{KE\mathbf{1}_{u\geq0}}{D_4}, \tag*{(101)} ∂ a 2 ∂ ℓ = − 1 + D 1 e a 2 + i = 1 ∑ K ( − 1 + D 2 , i ( K − i + 1 ) e a 2 + ( i − 1 ) E 1 u ≥ 0 ) + D 4 K E 1 u ≥ 0 , (101)
∂ ℓ ∂ b 1 = − ∑ i = 1 K ( i − 1 ) E 1 u ≥ 0 D 2 , i − K E 1 u ≥ 0 D 4 , (102) \frac{\partial\ell}{\partial b_1}=-\sum_{i=1}^{K}\frac{(i-1)E\mathbf{1}_{u\geq0}}{D_{2,i}}-\frac{KE\mathbf{1}_{u\geq0}}{D_4}, \tag*{(102)} ∂ b 1 ∂ ℓ = − i = 1 ∑ K D 2 , i ( i − 1 ) E 1 u ≥ 0 − D 4 K E 1 u ≥ 0 , (102)
∂ ℓ ∂ b 2 = e ( b 2 ) + 1 b 2 ≥ 0 [ K D 1 + ∑ i = 1 K K − i D 3 , i ] , (103) \frac{\partial\ell}{\partial b_2}=e^{(b_2)_+}\mathbf{1}_{b_2\geq0}\left[\frac{K}{D_1}+\sum_{i=1}^{K}\frac{K-i}{D_{3,i}}\right], \tag*{(103)} ∂ b 2 ∂ ℓ = e ( b 2 ) + 1 b 2 ≥ 0 [ D 1 K + i = 1 ∑ K D 3 , i K − i ] , (103)
∂ ℓ ∂ g 1 = e g 1 − d 1 D 1 + ∑ i = 1 K ( i − 1 ) E 1 v ≥ 0 + e g 1 − d 1 D 2 , i + ∑ i = 1 K ( − 1 + e g 1 − d 1 D 3 , i ) − 1 + K E 1 v ≥ 0 + e g 1 − d 1 D 4 , (104) \frac{\partial\ell}{\partial g_1}=\frac{e^{g_1-d_1}}{D_1}+\sum_{i=1}^{K}\frac{(i-1)E\mathbf{1}_{v\geq0}+e^{g_1-d_1}}{D_{2,i}}+\sum_{i=1}^{K}\left(-1+\frac{e^{g_1-d_1}}{D_{3,i}}\right)-1+\frac{KE\mathbf{1}_{v\geq0}+e^{g_1-d_1}}{D_4}, \tag*{(104)} ∂ g 1 ∂ ℓ = D 1 e g 1 − d 1 + i = 1 ∑ K D 2 , i ( i − 1 ) E 1 v ≥ 0 + e g 1 − d 1 + i = 1 ∑ K ( − 1 + D 3 , i e g 1 − d 1 ) − 1 + D 4 K E 1 v ≥ 0 + e g 1 − d 1 , (104)
∂ ℓ ∂ g 2 = − ∑ i = 1 K ( i − 1 ) E 1 v ≥ 0 D 2 , i − K E 1 v ≥ 0 D 4 , (105) \frac{\partial\ell}{\partial g_2}=-\sum_{i=1}^{K}\frac{(i-1)E\mathbf{1}_{v\geq0}}{D_{2,i}}-\frac{KE\mathbf{1}_{v\geq0}}{D_4}, \tag*{(105)} ∂ g 2 ∂ ℓ = − i = 1 ∑ K D 2 , i ( i − 1 ) E 1 v ≥ 0 − D 4 K E 1 v ≥ 0 , (105)
∂ ℓ ∂ d 1 = − ∂ ℓ ∂ g 1 , (106) \frac{\partial\ell}{\partial d_1}=-\frac{\partial\ell}{\partial g_1}, \tag*{(106)} ∂ d 1 ∂ ℓ = − ∂ g 1 ∂ ℓ , (106)
∂ ℓ ∂ d 2 = − ∂ ℓ ∂ g 2 + e ( d 2 ) + 1 d 2 ≥ 0 [ 1 D 1 + ∑ i = 1 K 1 D 2 , i + ∑ i = 1 K 1 D 3 , i + 1 D 4 ] , (107) \frac{\partial\ell}{\partial d_2}=-\frac{\partial\ell}{\partial g_2}+e^{(d_2)_+}\mathbf{1}_{d_2\geq0}\left[\frac{1}{D_1}+\sum_{i=1}^{K}\frac{1}{D_{2,i}}+\sum_{i=1}^{K}\frac{1}{D_{3,i}}+\frac{1}{D_4}\right], \tag*{(107)} ∂ d 2 ∂ ℓ = − ∂ g 2 ∂ ℓ + e ( d 2 ) + 1 d 2 ≥ 0 [ D 1 1 + i = 1 ∑ K D 2 , i 1 + i = 1 ∑ K D 3 , i 1 + D 4 1 ] , (107)
where the denominators are the softmax normalizers of the four cases,
D 1 = e a 2 + e g 1 − d 1 + e ( d 2 ) + + K e ( b 2 ) + + N − K − 3 , (108) D_1=e^{a_2}+e^{g_1-d_1}+e^{(d_2)_+}+Ke^{(b_2)_+}+N-K-3, \tag*{(108)} D 1 = e a 2 + e g 1 − d 1 + e ( d 2 ) + + K e ( b 2 ) + + N − K − 3 , (108)
D 2 , i = ( K − i + 1 ) e a 2 + ( i − 1 ) E + e g 1 − d 1 + 1 + e ( d 2 ) + + N − K − 3 , (109) D_{2,i}=(K-i+1)e^{a_2}+(i-1)E+e^{g_1-d_1}+1+e^{(d_2)_+}+N-K-3, \tag*{(109)} D 2 , i = ( K − i + 1 ) e a 2 + ( i − 1 ) E + e g 1 − d 1 + 1 + e ( d 2 ) + + N − K − 3 , (109)
D 3 , i = e g 1 − d 1 + 2 + e ( d 2 ) + + ( i − 1 ) + ( K − i ) e ( b 2 ) + + N − K − 3 , (110) D_{3,i}=e^{g_1-d_1}+2+e^{(d_2)_+}+(i-1)+(K-i)e^{(b_2)_+}+N-K-3, \tag*{(110)} D 3 , i = e g 1 − d 1 + 2 + e ( d 2 ) + + ( i − 1 ) + ( K − i ) e ( b 2 ) + + N − K − 3 , (110)
D 4 = K E + e g 1 − d 1 + 1 + e ( d 2 ) + + N − K − 3. (111) D_4=KE+e^{g_1-d_1}+1+e^{(d_2)_+}+N-K-3. \tag*{(111)} D 4 = K E + e g 1 − d 1 + 1 + e ( d 2 ) + + N − K − 3. (111)
Each denominator has exactly N N N terms, one per vertex token, including the masked current node with weight e 0 = 1 e^0=1 e 0 = 1 . In particular D 1 , D 2 , i , D 3 , i , D 4 ≥ N D_1,D_{2,i},D_{3,i},D_4\geq N D 1 , D 2 , i , D 3 , i , D 4 ≥ N on R \mathcal{R} R , and each of the 2 K + 2 2K+2 2 K + 2 steps contributes a quantity in [ − 1 , 1 ] [-1,1] [ − 1 , 1 ] to every partial derivative, so that
∣ ∂ ℓ ∂ θ ∣ ≤ 2 K + 2 on R , for every scalar θ . (112) \left|\frac{\partial\ell}{\partial\theta}\right|\leq2K+2\qquad\text{on }\mathcal{R},\text{ for every scalar }\theta. \tag*{(112)} ∂ θ ∂ ℓ ≤ 2 K + 2 on R , for every scalar θ . (112)
The identity Equation (106) holds on all of parameter space, since g 1 , d 1 g_1,d_1 g 1 , d 1 enter the unmasked logits only through g 1 − d 1 g_1-d_1 g 1 − d 1 ; the extra term in Equation (107) comes from the logit ( d 2 ) + (d_2)_+ ( d 2 ) + of the root in every step.
Step 3: R \mathcal{R} R is invariant. The initial point lies in R \mathcal{R} R : b 1 = c 0 > 0 b_1=c_0>0 b 1 = c 0 > 0 , b 2 = c 0 ′ < c 0 = b 1 b_2=c'_0<c_0=b_1 b 2 = c 0 ′ < c 0 = b 1 , a 2 = c 0 > 0 a_2=c_0>0 a 2 = c 0 > 0 , g 1 − d 1 = c 0 − c 0 ′ > 0 g_1-d_1=c_0-c'_0>0 g 1 − d 1 = c 0 − c 0 ′ > 0 . As long as the trajectory is in R \mathcal{R} R , Equations (102), (104), (105) and (107) and Equation (83) give a ˙ 1 = 0 \dot{a}_1=0 a ˙ 1 = 0 , b ˙ 1 ≥ 0 \dot{b}_1\geq0 b ˙ 1 ≥ 0 , b ˙ 2 ≤ 0 \dot{b}_2\leq0 b ˙ 2 ≤ 0 and g ˙ 2 ≥ 0 \dot{g}_2\geq0 g ˙ 2 ≥ 0 ; moreover d ˙ 2 = − α N ( N − 1 ) ∂ ℓ ∂ d 2 ≤ 0 \dot{d}_2=-\frac{\alpha}{N(N-1)}\frac{\partial\ell}{\partial d_2}\leq0 d ˙ 2 = − N ( N − 1 ) α ∂ d 2 ∂ ℓ ≤ 0 because both terms of Equation (107) are nonnegative, and
d d t ( g 2 − d 2 ) = − α N − 1 ∂ ℓ ∂ g 2 + α N ( N − 1 ) e ( d 2 ) + 1 d 2 ≥ 0 [ 1 D 1 + ∑ i 1 D 2 , i + ∑ i 1 D 3 , i + 1 D 4 ] ≥ 0 , (113) \frac{\mathrm{d}}{\mathrm{d}t}(g_2-d_2)=-\frac{\alpha}{N-1}\frac{\partial\ell}{\partial g_2}+\frac{\alpha}{N(N-1)}e^{(d_2)_+}\mathbf{1}_{d_2\geq0}\left[\frac{1}{D_1}+\sum_i\frac{1}{D_{2,i}}+\sum_i\frac{1}{D_{3,i}}+\frac{1}{D_4}\right]\geq0, \tag*{(113)} d t d ( g 2 − d 2 ) = − N − 1 α ∂ g 2 ∂ ℓ + N ( N − 1 ) α e ( d 2 ) + 1 d 2 ≥ 0 [ D 1 1 + i ∑ D 2 , i 1 + i ∑ D 3 , i 1 + D 4 1 ] ≥ 0 , (113)
with strict inequality whenever v ≥ 0 v\geq0 v ≥ 0 , because then − ∂ ℓ / ∂ g 2 ≥ K E / D 4 > 0 -\partial\ell/\partial g_2\geq KE/D_4>0 − ∂ ℓ / ∂ g 2 ≥ K E / D 4 > 0 . Hence b 1 ≥ c 0 b_1\geq c_0 b 1 ≥ c 0 , b 2 − b 1 b_2-b_1 b 2 − b 1 is non-increasing and stays negative, and g 2 − d 2 ≥ 0 g_2-d_2\geq0 g 2 − d 2 ≥ 0 is non-decreasing. Moreover b 2 b_2 b 2 never becomes negative: on { b 2 < 0 } ∩ R \{b_2<0\}\cap\mathcal{R} { b 2 < 0 } ∩ R the field Equation (103) vanishes, so if b 2 ( t 3 ) < 0 b_2(t_3)<0 b 2 ( t 3 ) < 0 and t ′ = sup { t ≤ t 3 : b 2 ( t ) ≥ 0 } t'=\sup\{t\leq t_3:b_2(t)\geq0\} t ′ = sup { t ≤ t 3 : b 2 ( t ) ≥ 0 } , then b 2 b_2 b 2 is constant on ( t ′ , t 3 ] (t',t_3] ( t ′ , t 3 ] and b 2 ( t 3 ) = b 2 ( t ′ ) = 0 b_2(t_3)=b_2(t')=0 b 2 ( t 3 ) = b 2 ( t ′ ) = 0 by continuity, a contradiction; hence 0 ≤ b 2 ≤ c 0 ′ 0 \le b_2 \le c'_0 0 ≤ b 2 ≤ c 0 ′ . Since R \mathcal{R} R is open and these inequalities keep b 1 > 0 b_1 > 0 b 1 > 0 and b 2 < b 1 b_2 < b_1 b 2 < b 1 , a continuous trajectory can leave R \mathcal{R} R only through one of the two faces { a 2 = 0 } \{a_2 = 0\} { a 2 = 0 } or { g 1 − d 1 = 0 } \{g_1 - d_1 = 0\} { g 1 − d 1 = 0 } . We show that neither face is reached. The arguments use the field only on R \mathcal{R} R , in a neighborhood of the faces, and never on the faces themselves, where the selections for ( a 2 ) + (a_2)_+ ( a 2 ) + and ( g 1 − d 1 ) + (g_1-d_1)_+ ( g 1 − d 1 ) + are free.
The face a 2 = 0 a_2 = 0 a 2 = 0 . On the open set { u < 0 } ∩ R \{u < 0\} \cap\mathcal{R} { u < 0 } ∩ R we have 1 u ≥ 0 = 0 \mathbf{1}_{u\ge0} = 0 1 u ≥ 0 = 0 , so by Equations (101) and (102)
∂ ℓ ∂ a 2 = − ( 1 − e a 2 D 1 ) − ∑ i = 1 K ( 1 − ( K − i + 1 ) e a 2 D 2 , i ) < 0 , ∂ ℓ ∂ b 1 = 0 , \frac{\partial\ell}{\partial a_2}=-\left(1-\frac{e^{a_2}}{D_1}\right)-\sum_{i=1}^{K}\left(1-\frac{(K-i+1)e^{a_2}}{D_{2,i}}\right)<0,\qquad\frac{\partial\ell}{\partial b_1}=0, ∂ a 2 ∂ ℓ = − ( 1 − D 1 e a 2 ) − i = 1 ∑ K ( 1 − D 2 , i ( K − i + 1 ) e a 2 ) < 0 , ∂ b 1 ∂ ℓ = 0 ,
because D 1 > e a 2 D_1 > e^{a_2} D 1 > e a 2 and D 2 , i > ( K − i + 1 ) e a 2 D_{2,i} > (K-i+1)e^{a_2} D 2 , i > ( K − i + 1 ) e a 2 . Hence u ′ = a ˙ 2 − b ˙ 1 > 0 u'=\dot{a}_2-\dot{b}_1>0 u ′ = a ˙ 2 − b ˙ 1 > 0 on { u < 0 } ∩ R \{u<0\}\cap\mathcal{R} { u < 0 } ∩ R ; since u ( 0 ) = 0 u(0)=0 u ( 0 ) = 0 , the trajectory never enters { u < 0 } \{u<0\} { u < 0 } , so u ≥ 0 u\ge0 u ≥ 0 and a 2 ≥ b 1 ≥ c 0 > 0 a_2\ge b_1\ge c_0>0 a 2 ≥ b 1 ≥ c 0 > 0 for all t t t . In particular the face a 2 = 0 a_2=0 a 2 = 0 is never reached.
The face g 1 − d 1 = 0 g_1-d_1=0 g 1 − d 1 = 0 . Since v ( 0 ) = g 1 ( 0 ) − d 1 ( 0 ) − ( g 2 − d 2 ) ( 0 ) = c 0 − c 0 ′ > 0 v(0)=g_1(0)-d_1(0)-(g_2-d_2)(0)=c_0-c'_0>0 v ( 0 ) = g 1 ( 0 ) − d 1 ( 0 ) − ( g 2 − d 2 ) ( 0 ) = c 0 − c 0 ′ > 0 , by continuity v > 0 v>0 v > 0 on some interval [ 0 , δ ) [0,\delta) [ 0 , δ ) , where Equation (113) gives d d t ( g 2 − d 2 ) ≥ K α N − 1 1 D 4 > 0 \frac{d}{dt}(g_2-d_2)\ge\frac{K\alpha}{N-1}\frac{1}{D_4}>0 d t d ( g 2 − d 2 ) ≥ N − 1 K α D 4 1 > 0 ; as g 2 − d 2 g_2-d_2 g 2 − d 2 is non-decreasing, ( g 2 − d 2 ) ( t ) > 0 (g_2-d_2)(t)>0 ( g 2 − d 2 ) ( t ) > 0 for all t > 0 t>0 t > 0 . Fix a ∈ ( 0 , δ ) a\in(0,\delta) a ∈ ( 0 , δ ) so small that g 1 − d 1 > 0 g_1-d_1>0 g 1 − d 1 > 0 on [ 0 , a ] [0,a] [ 0 , a ] , and let ϵ 0 : = min { ( g 2 − d 2 ) ( a ) , log 2 } > 0 \epsilon_0:=\min\{(g_2-d_2)(a),\log2\}>0 ϵ 0 := min {( g 2 − d 2 ) ( a ) , log 2 } > 0 . For t ≥ a t\ge a t ≥ a , at every state of R \mathcal{R} R with g 1 − d 1 < ϵ 0 g_1-d_1<\epsilon_0 g 1 − d 1 < ϵ 0 we have v = ( g 1 − d 1 ) − ( g 2 − d 2 ) < ϵ 0 − ( g 2 − d 2 ) ( a ) ≤ 0 v=(g_1-d_1)-(g_2-d_2)<\epsilon_0-(g_2-d_2)(a)\le0 v = ( g 1 − d 1 ) − ( g 2 − d 2 ) < ϵ 0 − ( g 2 − d 2 ) ( a ) ≤ 0 , so 1 v ≥ 0 = 0 \mathbf{1}_{v\ge0}=0 1 v ≥ 0 = 0 in a neighborhood of the state, and with e g 1 − d 1 ≤ 2 e^{g_1-d_1}\le2 e g 1 − d 1 ≤ 2 and all denominators at least N N N , Equation (104) gives
∂ ℓ ∂ g 1 ≤ 2 N + 2 K N − K ( 1 − 2 N ) − ( 1 − 2 N ) = ( K + 1 ) ( 4 N − 1 ) < 0. (114) \frac{\partial\ell}{\partial g_1}\le\frac{2}{N}+\frac{2K}{N}-K\left(1-\frac{2}{N}\right)-\left(1-\frac{2}{N}\right)=(K+1)\left(\frac{4}{N}-1\right)<0. \tag*{(114)} ∂ g 1 ∂ ℓ ≤ N 2 + N 2 K − K ( 1 − N 2 ) − ( 1 − N 2 ) = ( K + 1 ) ( N 4 − 1 ) < 0. (114)
Hence d d t ( g 1 − d 1 ) = − α N ∂ ℓ ∂ g 1 + α N ( K − 1 ) ∂ ℓ ∂ d 1 = − α N ∂ ℓ ∂ g 1 > 0 \frac{d}{dt}(g_1-d_1)=-\frac{\alpha}{N}\frac{\partial\ell}{\partial g_1}+\frac{\alpha}{N(K-1)}\frac{\partial\ell}{\partial d_1}=-\frac{\alpha}{N}\frac{\partial\ell}{\partial g_1}>0 d t d ( g 1 − d 1 ) = − N α ∂ g 1 ∂ ℓ + N ( K − 1 ) α ∂ d 1 ∂ ℓ = − N α ∂ g 1 ∂ ℓ > 0 on the whole set { g 1 − d 1 < ϵ 0 } ∩ R \{g_1-d_1<\epsilon_0\}\cap\mathcal{R} { g 1 − d 1 < ϵ 0 } ∩ R (for t ≥ a t\ge a t ≥ a ), not only at a single time, and the one-sided barrier argument gives g 1 − d 1 ≥ min { ( g 1 − d 1 ) ( a ) , ϵ 0 } > 0 g_1-d_1\ge\min\{(g_1-d_1)(a),\epsilon_0\}>0 g 1 − d 1 ≥ min {( g 1 − d 1 ) ( a ) , ϵ 0 } > 0 for all t ≥ a t\ge a t ≥ a . Thus g 1 − d 1 > 0 g_1-d_1>0 g 1 − d 1 > 0 for all t ≥ 0 t\ge0 t ≥ 0 . This proves item 1.
Step 4: the initial phase. By Equation (112) and Equation (83) , every scalar moves at speed at most ( 2 K + 2 ) α / N (2K+2)\alpha/N ( 2 K + 2 ) α / N , so on [ 0 , T 0 ] [0,T_0] [ 0 , T 0 ] with T 0 = N 4 ( K + 1 ) α T_0=\frac{N}{4(K+1)\alpha} T 0 = 4 ( K + 1 ) α N every parameter changes by at most 1 2 \frac12 2 1 ; since 0 < c 0 ′ < c 0 ≤ c ∗ N − 2 < 1 4 0<c'_0<c_0\le c_*N^{-2}<\frac14 0 < c 0 ′ < c 0 ≤ c ∗ N − 2 < 4 1 (by the definition of c ∗ c_* c ∗ below), all parameters lie in [ − 1 2 , 1 ] [-\frac12,1] [ − 2 1 , 1 ] on [ 0 , T 0 ] [0,T_0] [ 0 , T 0 ] . Consequently every exponent appearing in the denominators is bounded by an absolute constant, and there exist absolute constants C 1 , C 2 ≥ 0 C_1,C_2\ge0 C 1 , C 2 ≥ 0 such that
max i { D 1 , D 2 , i , D 3 , i , D 4 } ≤ N + C 1 K + C 2 on [ 0 , T 0 ] . (115) \max_i\{D_1,D_{2,i},D_{3,i},D_4\}\le N+C_1K+C_2\qquad\text{on }[0,T_0]. \tag*{(115)} i max { D 1 , D 2 , i , D 3 , i , D 4 } ≤ N + C 1 K + C 2 on [ 0 , T 0 ] . (115)
While b 2 > 0 b_2>0 b 2 > 0 (so that 1 b 2 ≥ 0 = 1 \mathbf{1}_{b_2\ge0}=1 1 b 2 ≥ 0 = 1 ), Equation (103) gives
∂ ℓ ∂ b 2 ≥ K + K ( K − 1 ) 2 N + C 1 K + C 2 ≥ K N + C 1 K + C 2 , so b ˙ 2 ≤ − α N ( N − 1 ) K N + C 1 K + C 2 . (116) \frac{\partial\ell}{\partial b_2}\ge\frac{K+\frac{K(K-1)}{2}}{N+C_1K+C_2}\ge\frac{K}{N+C_1K+C_2},\qquad\text{so}\qquad\dot b_2\le-\frac{\alpha}{N(N-1)}\frac{K}{N+C_1K+C_2}. \tag*{(116)} ∂ b 2 ∂ ℓ ≥ N + C 1 K + C 2 K + 2 K ( K − 1 ) ≥ N + C 1 K + C 2 K , so b ˙ 2 ≤ − N ( N − 1 ) α N + C 1 K + C 2 K . (116)
and while d 2 > 0 d_2>0 d 2 > 0 , Equation (107) gives
∂ ℓ ∂ d 2 ≥ 2 K + 2 N + C 1 K + C 2 ≥ K N + C 1 K + C 2 , so d ˙ 2 ≤ − α N ( N − 1 ) K N + C 1 K + C 2 . (117) \frac{\partial\ell}{\partial d_2}\ge\frac{2K+2}{N+C_1K+C_2}\ge\frac{K}{N+C_1K+C_2},\qquad\text{so}\qquad\dot d_2\le-\frac{\alpha}{N(N-1)}\frac{K}{N+C_1K+C_2}. \tag*{(117)} ∂ d 2 ∂ ℓ ≥ N + C 1 K + C 2 2 K + 2 ≥ N + C 1 K + C 2 K , so d ˙ 2 ≤ − N ( N − 1 ) α N + C 1 K + C 2 K . (117)
Over the time interval [ 0 , T 0 ] [0,T_0] [ 0 , T 0 ] each of b 2 , d 2 b_2,d_2 b 2 , d 2 therefore decreases, as long as it is positive, by at least
α N ( N − 1 ) K N + C 1 K + C 2 N 4 ( K + 1 ) α = K 4 ( K + 1 ) ( N − 1 ) ( N + C 1 K + C 2 ) ≥ K N − 2 4 ( K + 1 ) ( 1 + C 1 K + C 2 ) = : c ∗ N − 2 . (118) \frac{\alpha}{N(N-1)}\frac{K}{N+C_1K+C_2}\frac{N}{4(K+1)\alpha}
=\frac{K}{4(K+1)(N-1)(N+C_1K+C_2)}
\ge\frac{KN^{-2}}{4(K+1)(1+C_1K+C_2)}=:c_*N^{-2}. \tag*{(118)} N ( N − 1 ) α N + C 1 K + C 2 K 4 ( K + 1 ) α N = 4 ( K + 1 ) ( N − 1 ) ( N + C 1 K + C 2 ) K ≥ 4 ( K + 1 ) ( 1 + C 1 K + C 2 ) K N − 2 =: c ∗ N − 2 . (118)
where we used ( N − 1 ) ( N + C 1 K + C 2 ) ≤ N 2 ( 1 + C 1 K + C 2 ) (N-1)(N+C_1K+C_2)\le N^2(1+C_1K+C_2) ( N − 1 ) ( N + C 1 K + C 2 ) ≤ N 2 ( 1 + C 1 K + C 2 ) . Since b 2 ( 0 ) = c 0 ′ < c 0 ≤ c ∗ N − 2 b_2(0)=c'_0<c_0\le c_*N^{-2} b 2 ( 0 ) = c 0 ′ < c 0 ≤ c ∗ N − 2 and d 2 ( 0 ) = c 0 ≤ c ∗ N − 2 d_2(0)=c_0\le c_*N^{-2} d 2 ( 0 ) = c 0 ≤ c ∗ N − 2 , both b 2 b_2 b 2 and d 2 d_2 d 2 reach 0 0 0 before time T 0 T_0 T 0 . Since b 2 b_2 b 2 is non-increasing and b 2 ≥ 0 b_2\ge0 b 2 ≥ 0 (Step 3), b 2 = 0 b_2=0 b 2 = 0 for all t ≥ T 0 t\ge T_0 t ≥ T 0 ; since d 2 d_2 d 2 is non-increasing, d 2 ≤ 0 d_2\le0 d 2 ≤ 0 for all t ≥ T 0 t\ge T_0 t ≥ T 0 . This proves item 2.
Step 5: simplification. For t ≥ T 0 t\ge T_0 t ≥ T 0 we have b 2 = 0 b_2=0 b 2 = 0 , e ( b 2 ) + = e ( d 2 ) + = 1 e^{(b_2)_+}=e^{(d_2)_+}=1 e ( b 2 ) + = e ( d 2 ) + = 1 , and 1 d 2 ≥ 0 = 0 \mathbf{1}_{d_2\ge0}=0 1 d 2 ≥ 0 = 0 wherever d 2 < 0 d_2<0 d 2 < 0 . The term with 1 d 2 ≥ 0 \mathbf{1}_{d_2\ge0} 1 d 2 ≥ 0 in Equation (107) enters only d ˙ 2 \dot d_2 d ˙ 2 and does not affect the reduced system: the set { t ≥ T 0 : d 2 ( t ) = 0 } \{t\ge T_0:d_2(t)=0\} { t ≥ T 0 : d 2 ( t ) = 0 } is an interval, as d 2 ≤ 0 d_2\le0 d 2 ≤ 0 is non-increasing; on its interior d ˙ 2 = 0 \dot d_2=0 d ˙ 2 = 0 , so both nonnegative terms of ∂ ℓ / ∂ d 2 \partial\ell/\partial d_2 ∂ ℓ / ∂ d 2 in Equation (107) vanish, and after it d 2 < 0 d_2<0 d 2 < 0 . In both cases g ˙ 2 + ( N − 1 ) d ˙ 2 = − α N ( ∂ ℓ ∂ g 2 + ∂ ℓ ∂ d 2 ) = 0 \dot g_2+(N-1)\dot d_2=-\frac{\alpha}{N}\left(\frac{\partial\ell}{\partial g_2}+\frac{\partial\ell}{\partial d_2}\right)=0 g ˙ 2 + ( N − 1 ) d ˙ 2 = − N α ( ∂ g 2 ∂ ℓ + ∂ d 2 ∂ ℓ ) = 0 a.e., so g 2 + ( N − 1 ) d 2 g_2+(N-1)d_2 g 2 + ( N − 1 ) d 2 is constant on [ T 0 , ∞ ) [T_0,\infty) [ T 0 , ∞ ) and
d d t ( g 2 − d 2 ) = N N − 1 g ˙ 2 = − α N − 1 ∂ ℓ ∂ g 2 \frac{\mathrm{d}}{\mathrm{d}t}(g_2-d_2)=\frac{N}{N-1}\dot g_2=-\frac{\alpha}{N-1}\frac{\partial\ell}{\partial g_2} d t d ( g 2 − d 2 ) = N − 1 N g ˙ 2 = − N − 1 α ∂ g 2 ∂ ℓ
for a.e. t ≥ T 0 t\ge T_0 t ≥ T 0 . Substituting into Equations (101) , (102) and (104) to (107) and using
− 1 + e a 2 D 1 = − e g 1 − d 1 + N − 2 D 1 , -1+\frac{e^{a_2}}{D_1}=-\frac{e^{g_1-d_1}+N-2}{D_1}, − 1 + D 1 e a 2 = − D 1 e g 1 − d 1 + N − 2 ,
− 1 + ( K − i + 1 ) e a 2 + ( i − 1 ) E 1 u ≥ 0 D 2 , i = − e g 1 − d 1 + N − K − 1 D 2 , i − ( i − 1 ) E 1 u < 0 D 2 , i , -1+\frac{(K-i+1)e^{a_2}+(i-1)E\mathbf{1}_{u\ge0}}{D_{2,i}}=-\frac{e^{g_1-d_1}+N-K-1}{D_{2,i}}-\frac{(i-1)E\mathbf{1}_{u<0}}{D_{2,i}}, − 1 + D 2 , i ( K − i + 1 ) e a 2 + ( i − 1 ) E 1 u ≥ 0 = − D 2 , i e g 1 − d 1 + N − K − 1 − D 2 , i ( i − 1 ) E 1 u < 0 ,
∑ i = 1 K ( − 1 + e g 1 − d 1 D 3 , i ) = − K ( N − 1 ) D 3 , \sum_{i=1}^{K}\left(-1+\frac{e^{g_1-d_1}}{D_{3,i}}\right)=-\frac{K(N-1)}{D_3}, i = 1 ∑ K ( − 1 + D 3 , i e g 1 − d 1 ) = − D 3 K ( N − 1 ) ,
− 1 + K E 1 v ≥ 0 + e g 1 − d 1 D 4 = − N − K − 1 + K E 1 v < 0 D 4 , -1+\frac{KE\mathbf{1}_{v\ge0}+e^{g_1-d_1}}{D_4}=-\frac{N-K-1+KE\mathbf{1}_{v<0}}{D_4}, − 1 + D 4 K E 1 v ≥ 0 + e g 1 − d 1 = − D 4 N − K − 1 + K E 1 v < 0 ,
where now D 3 , i = e g 1 − d 1 + N − 1 = : D 3 D_{3,i}=e^{g_1-d_1}+N-1=:D_3 D 3 , i = e g 1 − d 1 + N − 1 =: D 3 no longer depends on i i i , gives the expressions in item 3. □
By Theorem C.2, we can aggregate the parameters and deal with a more compact system as follows.
Corollary C.3 (Parameter aggregation). Let x 1 = a 2 , x 2 = b 1 , y 1 = g 1 − d 1 , y 2 = g 2 − d 2 x_1=a_2,x_2=b_1,y_1=g_1-d_1,y_2=g_2-d_2 x 1 = a 2 , x 2 = b 1 , y 1 = g 1 − d 1 , y 2 = g 2 − d 2 and u = x 1 − x 2 , v = y 1 − y 2 u=x_1-x_2,v=y_1-y_2 u = x 1 − x 2 , v = y 1 − y 2 . Under the conditions of Theorem C.2, for t ≥ T 0 t\ge T_0 t ≥ T 0 the quadruple ( x 1 , x 2 , y 1 , y 2 ) (x_1,x_2,y_1,y_2) ( x 1 , x 2 , y 1 , y 2 ) solves the closed four-variable system
x ˙ 1 = α N [ e y 1 + N − 2 D 1 + ∑ i = 1 K e y 1 + N − K − 1 + ( i − 1 ) e u + + v + 1 u < 0 D 2 , i − K e u + + v + 1 u ≥ 0 D 4 ] (119) \dot{x}_1=\frac{\alpha}{N}\left[\frac{e^{y_1}+N-2}{D_1}+\sum_{i=1}^{K}\frac{e^{y_1}+N-K-1+(i-1)e^{u_++v_+}\mathbf{1}_{u<0}}{D_{2,i}}-\frac{Ke^{u_++v_+}\mathbf{1}_{u\ge0}}{D_4}\right] \tag*{(119)} x ˙ 1 = N α [ D 1 e y 1 + N − 2 + i = 1 ∑ K D 2 , i e y 1 + N − K − 1 + ( i − 1 ) e u + + v + 1 u < 0 − D 4 K e u + + v + 1 u ≥ 0 ] (119) x ˙ 2 = α N ( N − 1 ) [ ∑ i = 1 K ( i − 1 ) e u + + v + 1 u ≥ 0 D 2 , i + K e u + + v + 1 u ≥ 0 D 4 ] (120) \dot{x}_2=\frac{\alpha}{N(N-1)}\left[\sum_{i=1}^{K}\frac{(i-1)e^{u_++v_+}\mathbf{1}_{u\ge0}}{D_{2,i}}+\frac{Ke^{u_++v_+}\mathbf{1}_{u\ge0}}{D_4}\right] \tag*{(120)} x ˙ 2 = N ( N − 1 ) α [ i = 1 ∑ K D 2 , i ( i − 1 ) e u + + v + 1 u ≥ 0 + D 4 K e u + + v + 1 u ≥ 0 ] (120) y ˙ 1 = α N − 1 [ K ( N − 1 ) D 3 + N − K − 1 + K e u + + v + 1 v < 0 D 4 − e y 1 D 1 − ∑ i = 1 K ( i − 1 ) e u + + v + 1 v ≥ 0 + e y 1 D 2 , i ] (121) \dot{y}_1=\frac{\alpha}{N-1}\left[\frac{K(N-1)}{D_3}+\frac{N-K-1+Ke^{u_++v_+}\mathbf{1}_{v<0}}{D_4}-\frac{e^{y_1}}{D_1}-\sum_{i=1}^{K}\frac{(i-1)e^{u_++v_+}\mathbf{1}_{v\ge0}+e^{y_1}}{D_{2,i}}\right] \tag*{(121)} y ˙ 1 = N − 1 α [ D 3 K ( N − 1 ) + D 4 N − K − 1 + K e u + + v + 1 v < 0 − D 1 e y 1 − i = 1 ∑ K D 2 , i ( i − 1 ) e u + + v + 1 v ≥ 0 + e y 1 ] (121) y ˙ 2 = α N − 1 [ ∑ i = 1 K ( i − 1 ) e u + + v + 1 v ≥ 0 D 2 , i + K e u + + v + 1 v ≥ 0 D 4 ] , (122) \dot{y}_2=\frac{\alpha}{N-1}\left[\sum_{i=1}^{K}\frac{(i-1)e^{u_++v_+}\mathbf{1}_{v\ge0}}{D_{2,i}}+\frac{Ke^{u_++v_+}\mathbf{1}_{v\ge0}}{D_4}\right], \tag*{(122)} y ˙ 2 = N − 1 α [ i = 1 ∑ K D 2 , i ( i − 1 ) e u + + v + 1 v ≥ 0 + D 4 K e u + + v + 1 v ≥ 0 ] , (122) where the denominators are
D 1 = e x 1 + e y 1 + N − 2 (123) D_1=e^{x_1}+e^{y_1}+N-2 \tag*{(123)} D 1 = e x 1 + e y 1 + N − 2 (123) D 2 , i = ( K − i + 1 ) e x 1 + ( i − 1 ) e u + + v + + e y 1 + N − K − 1 (124) D_{2,i}=(K-i+1)e^{x_1}+(i-1)e^{u_++v_+}+e^{y_1}+N-K-1 \tag*{(124)} D 2 , i = ( K − i + 1 ) e x 1 + ( i − 1 ) e u + + v + + e y 1 + N − K − 1 (124) D 3 = e y 1 + N − 1 (125) D_3=e^{y_1}+N-1 \tag*{(125)} D 3 = e y 1 + N − 1 (125) D 4 = K e u + + v + + e y 1 + N − K − 1. (126) D_4=Ke^{u_++v_+}+e^{y_1}+N-K-1. \tag*{(126)} D 4 = K e u + + v + + e y 1 + N − K − 1. (126)
The remaining four scalars are recovered affinely: a 1 ≡ c 0 a_1 \equiv c_0 a 1 ≡ c 0 ; b 2 ≡ 0 b_2 \equiv0 b 2 ≡ 0 for t ≥ T 0 t \ge T_0 t ≥ T 0 ; and since ∂ ℓ / ∂ d i = − ∂ ℓ / ∂ g i \partial\ell/\partial d_i=-\partial\ell/\partial g_i ∂ ℓ / ∂ d i = − ∂ ℓ / ∂ g i while the learning rates are α / N \alpha/N α / N and α / ( N ( N − 1 ) ) \alpha/(N(N-1)) α / ( N ( N − 1 )) , we have g ˙ i = − ( N − 1 ) d ˙ i \dot{g}_i=-(N-1)\dot{d}_i g ˙ i = − ( N − 1 ) d ˙ i , so that g 1 + ( N − 1 ) d 1 ≡ c 0 + ( N − 1 ) c 0 ′ g_1+(N-1)d_1\equiv c_0+(N-1)c'_0 g 1 + ( N − 1 ) d 1 ≡ c 0 + ( N − 1 ) c 0 ′ for all t ≥ 0 t\ge0 t ≥ 0 and g 2 + ( N − 1 ) d 2 g_2+(N-1)d_2 g 2 + ( N − 1 ) d 2 is constant for t ≥ T 0 t\ge T_0 t ≥ T 0 ; hence
g 1 = N − 1 N y 1 + c 0 + ( N − 1 ) c 0 ′ N , d 1 = − 1 N y 1 + c 0 + ( N − 1 ) c 0 ′ N , (127) g_1=\frac{N-1}{N}y_1+\frac{c_0+(N-1)c'_0}{N},\qquad d_1=-\frac{1}{N}y_1+\frac{c_0+(N-1)c'_0}{N}, \tag*{(127)} g 1 = N N − 1 y 1 + N c 0 + ( N − 1 ) c 0 ′ , d 1 = − N 1 y 1 + N c 0 + ( N − 1 ) c 0 ′ , (127)
g 2 = N − 1 N y 2 + O ( 1 ) , d 2 = − 1 N y 2 + O ( 1 ) , (128) g_2=\frac{N-1}{N}y_2+O(1),\qquad d_2=-\frac{1}{N}y_2+O(1), \tag*{(128)} g 2 = N N − 1 y 2 + O ( 1 ) , d 2 = − N 1 y 2 + O ( 1 ) , (128)
where the O ( 1 ) O(1) O ( 1 ) terms are constants depending on N , K , c 0 , c 0 ′ N,K,c_0,c'_0 N , K , c 0 , c 0 ′ . At time T 0 T_0 T 0 the state satisfies u ≥ 0 u\ge0 u ≥ 0 , x 2 > 0 x_2>0 x 2 > 0 , y 1 > 0 y_1>0 y 1 > 0 and y 2 ≥ 0 y_2\ge0 y 2 ≥ 0 , and along the system y 2 y_2 y 2 is non-decreasing.
Proof. By Equation (83) , x ˙ 1 = − α N ∂ ℓ ∂ a 2 \dot{x}_1=-\frac{\alpha}{N}\frac{\partial\ell}{\partial a_2} x ˙ 1 = − N α ∂ a 2 ∂ ℓ , x ˙ 2 = − α N ( N − 1 ) ∂ ℓ ∂ b 1 \dot{x}_2=-\frac{\alpha}{N(N-1)}\frac{\partial\ell}{\partial b_1} x ˙ 2 = − N ( N − 1 ) α ∂ b 1 ∂ ℓ , and, using ∂ ℓ / ∂ d 1 = − ∂ ℓ / ∂ g 1 \partial\ell/\partial d_1=-\partial\ell/\partial g_1 ∂ ℓ / ∂ d 1 = − ∂ ℓ / ∂ g 1 and, for i = 2 i=2 i = 2 , Step 5 of the proof of Theorem C.2,
y ˙ i = g ˙ i − d ˙ i = − α N ∂ ℓ ∂ g i − α N ( N − 1 ) ∂ ℓ ∂ g i = − α N − 1 ∂ ℓ ∂ g i . \dot{y}_i=\dot{g}_i-\dot{d}_i=-\frac{\alpha}{N}\frac{\partial\ell}{\partial g_i}-\frac{\alpha}{N(N-1)}\frac{\partial\ell}{\partial g_i}=-\frac{\alpha}{N-1}\frac{\partial\ell}{\partial g_i}. y ˙ i = g ˙ i − d ˙ i = − N α ∂ g i ∂ ℓ − N ( N − 1 ) α ∂ g i ∂ ℓ = − N − 1 α ∂ g i ∂ ℓ .
Substituting item 3 of Theorem C.2 gives the displayed system; it depends on the eight scalars only through x 1 , x 2 , y 1 , y 2 x_1,x_2,y_1,y_2 x 1 , x 2 , y 1 , y 2 . The recovery formulas follow from g ˙ i = − ( N − 1 ) d ˙ i \dot{g}_i=-(N-1)\dot{d}_i g ˙ i = − ( N − 1 ) d ˙ i and g i − d i = y i g_i-d_i=y_i g i − d i = y i (for i = 2 i=2 i = 2 , g 2 + ( N − 1 ) d 2 g_2+(N-1)d_2 g 2 + ( N − 1 ) d 2 is constant on [ T 0 , ∞ ) [T_0,\infty) [ T 0 , ∞ ) by Step 5, and its value differs from the initial value N c 0 Nc_0 N c 0 by at most 1, the amount accumulated on [ 0 , T 0 ] [0,T_0] [ 0 , T 0 ] ). The properties of the state at T 0 T_0 T 0 are item 1 of Theorem C.2, and y ˙ 2 ≥ 0 \dot{y}_2\ge0 y ˙ 2 ≥ 0 is read off the system.
Before analyzing the system in Theorem C.3, we need the following lemma to characterize the main growth rate. It is applied to Filippov solutions, so the differential equations are only required to hold almost everywhere, and the coefficient bounds only from some time t 0 t_0 t 0 on.
Lemma C.4 (Logarithm growth lemma). Let t 0 ≥ 1 t_0\ge1 t 0 ≥ 1 and let X , Y : [ t 0 , ∞ ) → R X,Y:[t_0,\infty)\to\mathbb{R} X , Y : [ t 0 , ∞ ) → R be locally absolutely continuous and satisfy, for a.e. t ≥ t 0 t\ge t_0 t ≥ t 0 ,
X ′ = a ( t ) e Y − X , (129) X'=a(t)e^{Y-X}, \tag*{(129)} X ′ = a ( t ) e Y − X , (129) Y ′ = b ( t ) e − Y − a ~ ( t ) e Y − X + r ( t ) , (130) Y'=b(t)e^{-Y}-\widetilde{a}(t)e^{Y-X}+r(t), \tag*{(130)} Y ′ = b ( t ) e − Y − a ( t ) e Y − X + r ( t ) , (130) with measurable coefficients such that, for a.e. t ≥ t 0 t\ge t_0 t ≥ t 0 ,
0 < a − ≤ a ( t ) ≤ a + < ∞ , 0 < a ~ − ≤ a ~ ( t ) ≤ a ~ + < ∞ , (131) 0<a_-\le a(t)\le a_+<\infty,\qquad0<\widetilde{a}_-\le\widetilde{a}(t)\le\widetilde{a}_+<\infty, \tag*{(131)} 0 < a − ≤ a ( t ) ≤ a + < ∞ , 0 < a − ≤ a ( t ) ≤ a + < ∞ , (131) 0 < β − ≤ β ( t ) : = b ( t ) + e Y ( t ) r ( t ) ≤ β + < ∞ . (132) 0<\beta_-\le\beta(t):=b(t)+e^{Y(t)}r(t)\le\beta_+<\infty. \tag*{(132)} 0 < β − ≤ β ( t ) := b ( t ) + e Y ( t ) r ( t ) ≤ β + < ∞. (132) Set Θ : = e 2 Y − X \Theta:=e^{2Y-X} Θ := e 2 Y − X and
Θ − : = 2 β − a + + 2 a ~ + , Θ + : = 2 β + a − + 2 a ~ − , c : = min { Θ ( t 0 ) , Θ − } , C : = max { Θ ( t 0 ) , Θ + } . \Theta_-:=\frac{2\beta_-}{a_++2\widetilde{a}_+},\qquad\Theta_+:=\frac{2\beta_+}{a_-+2\widetilde{a}_-},\qquad c:=\min\{\Theta(t_0),\Theta_-\},\qquad C:=\max\{\Theta(t_0),\Theta_+\}. Θ − := a + + 2 a + 2 β − , Θ + := a − + 2 a − 2 β + , c := min { Θ ( t 0 ) , Θ − } , C := max { Θ ( t 0 ) , Θ + } . Then for all t ≥ t 0 t\ge t_0 t ≥ t 0 ,
c ≤ e 2 Y ( t ) − X ( t ) ≤ C , e X ( t 0 ) / 2 + a − c 2 ( t − t 0 ) ≤ e X ( t ) / 2 ≤ e X ( t 0 ) / 2 + a + C 2 ( t − t 0 ) , (133) c\le e^{2Y(t)-X(t)}\le C,\qquad e^{X(t_0)/2}+\frac{a_-\sqrt{c}}{2}(t-t_0)\le e^{X(t)/2}\le e^{X(t_0)/2}+\frac{a_+\sqrt{C}}{2}(t-t_0), \tag*{(133)} c ≤ e 2 Y ( t ) − X ( t ) ≤ C , e X ( t 0 ) /2 + 2 a − c ( t − t 0 ) ≤ e X ( t ) /2 ≤ e X ( t 0 ) /2 + 2 a + C ( t − t 0 ) , (133) In particular X , Y → ∞ X,Y\to\infty X , Y → ∞ and
X ( t ) = 2 log t + O ( 1 ) , Y ( t ) = log t + O ( 1 ) , (134) X(t)=2\log t+O(1),\qquad Y(t)=\log t+O(1), \tag*{(134)} X ( t ) = 2 log t + O ( 1 ) , Y ( t ) = log t + O ( 1 ) , (134) where the O ( 1 ) O(1) O ( 1 ) terms depend only on t 0 t_0 t 0 , X ( t 0 ) X(t_0) X ( t 0 ) , Y ( t 0 ) Y(t_0) Y ( t 0 ) and the coefficient bounds.
Proof. Set P = e X P=e^{X} P = e X and S = e Y S=e^{Y} S = e Y , so Θ = S 2 / P > 0 \Theta=S^{2}/P>0 Θ = S 2 / P > 0 and, a.e. on [ t 0 , ∞ ) [t_{0},\infty) [ t 0 , ∞ ) ,
P ′ = a S , S ′ = β − a ~ S 2 P , ( log Θ ) ′ = 2 Y ′ − X ′ = 1 S { 2 β − ( a + 2 a ~ ) Θ } . (135) P'=aS,\qquad S'=\beta-\tilde{a}\frac{S^{2}}{P},\qquad(\log\Theta)'=2Y'-X'=\frac{1}{S}\{2\beta-(a+2\tilde{a})\Theta\}. \tag*{(135)} P ′ = a S , S ′ = β − a ~ P S 2 , ( log Θ ) ′ = 2 Y ′ − X ′ = S 1 { 2 β − ( a + 2 a ~ ) Θ } . (135)
Since S > 0 S>0 S > 0 , we have Θ ′ < 0 \Theta'<0 Θ ′ < 0 a.e. on { Θ > Θ + } \{\Theta>\Theta_{+}\} { Θ > Θ + } and Θ ′ > 0 \Theta'>0 Θ ′ > 0 a.e. on { Θ < Θ − } \{\Theta<\Theta_{-}\} { Θ < Θ − } : Θ \Theta Θ is pushed downward when it is large and upward when it is small. If Θ ( t 1 ) > C \Theta(t_{1})>C Θ ( t 1 ) > C for some t 1 > t 0 t_{1}>t_{0} t 1 > t 0 , let t 2 = sup { t ∈ [ t 0 , t 1 ] : Θ ( t ) ≤ C } t_{2}=\sup\{t\in[t_{0},t_{1}]:\Theta(t)\le C\} t 2 = sup { t ∈ [ t 0 , t 1 ] : Θ ( t ) ≤ C } ; then Θ ( t 2 ) = C \Theta(t_{2})=C Θ ( t 2 ) = C , Θ > C ≥ Θ + \Theta>C\ge\Theta_{+} Θ > C ≥ Θ + on ( t 2 , t 1 ] (t_{2},t_{1}] ( t 2 , t 1 ] , and
Θ ( t 1 ) = Θ ( t 2 ) + ∫ t 2 t 1 Θ ′ ≤ C , \Theta(t_{1})=\Theta(t_{2})+\int_{t_{2}}^{t_{1}}\Theta'\le C, Θ ( t 1 ) = Θ ( t 2 ) + ∫ t 2 t 1 Θ ′ ≤ C ,
a contradiction. Hence Θ ≤ C \Theta\le C Θ ≤ C , and symmetrically Θ ≥ c > 0 \Theta\ge c>0 Θ ≥ c > 0 . Next,
( P ) ′ = P ′ 2 P = 1 2 a Θ ∈ [ 1 2 a c , 1 2 a C ] (\sqrt{P})'=\frac{P'}{2\sqrt{P}}=\frac{1}{2}a\sqrt{\Theta}\in\left[\frac{1}{2}a\sqrt{c},\frac{1}{2}a\sqrt{C}\right] ( P ) ′ = 2 P P ′ = 2 1 a Θ ∈ [ 2 1 a c , 2 1 a C ]
a.e., and integrating from t 0 t_{0} t 0 gives the two-sided bound on e X / 2 = P e^{X/2}=\sqrt{P} e X /2 = P . Finally X = 2 log P X=2\log\sqrt{P} X = 2 log P and
Y = 1 2 log Θ + log P , Y=\frac{1}{2}\log\Theta+\log\sqrt{P}, Y = 2 1 log Θ + log P ,
and
P ( t ) t ∈ [ 1 4 a c , P ( t 0 ) + 1 2 a C ] \frac{\sqrt{P(t)}}{t}\in\left[\frac{1}{4}a\sqrt{c},\sqrt{P(t_{0})}+\frac{1}{2}a\sqrt{C}\right] t P ( t ) ∈ [ 4 1 a c , P ( t 0 ) + 2 1 a C ]
for t ≥ 2 t 0 t\ge2t_{0} t ≥ 2 t 0 , which proves the claim.
With the dynamical system in Theorem C.3 and the logarithm growth lemma Theorem C.4, we establish the convergence results in the following theorem.
Theorem C.5. Consider the four-dimensional discontinuous dynamical system of Theorem C.3, with Filippov solutions on the switching surfaces u = 0 u=0 u = 0 and v = 0 v=0 v = 0 , along the trajectory of Theorem C.2 for t ≥ T 0 t\ge T_{0} t ≥ T 0 ; by item 1 of that lemma, u ≥ 0 u\ge0 u ≥ 0 , x 2 > 0 x_{2}>0 x 2 > 0 , y 1 > 0 y_{1}>0 y 1 > 0 and y 2 ≥ 0 y_{2}\ge0 y 2 ≥ 0 hold along it. Then the following holds, where “bounded” and O ( 1 ) O(1) O ( 1 ) refer to constants that do not depend on t t t but may depend on N N N , K K K and the state at time T 0 T_{0} T 0 :
The positive part v + = ( y 1 − y 2 ) + v_{+}=(y_{1}-y_{2})_{+} v + = ( y 1 − y 2 ) + is bounded for t ≥ T 0 t\ge T_{0} t ≥ T 0 .
x 1 , x 2 , y 1 , y 2 → ∞ x_{1},x_{2},y_{1},y_{2}\to\infty x 1 , x 2 , y 1 , y 2 → ∞ as t → ∞ t\to\infty t → ∞ .
The differences 2 y 1 − x 1 2y_{1}-x_{1} 2 y 1 − x 1 and x 1 − x 2 x_{1}-x_{2} x 1 − x 2 are upper bounded for t ≥ T 0 t\ge T_{0} t ≥ T 0 .
x 1 , x 2 = 2 log ( α t ) + O ( 1 ) x_{1},x_{2}=2\log(\alpha t)+O(1) x 1 , x 2 = 2 log ( α t ) + O ( 1 ) and y 1 , y 2 = log ( α t ) + O ( 1 ) y_{1},y_{2}=\log(\alpha t)+O(1) y 1 , y 2 = log ( α t ) + O ( 1 ) , where the O ( 1 ) O(1) O ( 1 ) terms do not depend on α \alpha α .
Proof. Throughout the proof we shift time so that the starting time T 0 T_{0} T 0 becomes 0 0 0 ; thus u ( 0 ) ≥ 0 u(0)\ge0 u ( 0 ) ≥ 0 , x 2 ( 0 ) > 0 x_{2}(0)>0 x 2 ( 0 ) > 0 , y 1 ( 0 ) > 0 y_{1}(0)>0 y 1 ( 0 ) > 0 , y 2 ( 0 ) ≥ 0 y_{2}(0)\ge0 y 2 ( 0 ) ≥ 0 , and, by item 1 of Theorem C.2, u ≥ 0 u\ge0 u ≥ 0 , x 2 > 0 x_{2}>0 x 2 > 0 , y 1 > 0 y_{1}>0 y 1 > 0 , y 2 ≥ 0 y_{2}\ge0 y 2 ≥ 0 for all t ≥ 0 t\ge0 t ≥ 0 (we nevertheless re-derive u ≥ 0 u\ge0 u ≥ 0 below, since only the four-variable system is used from now on). The right-hand side of the system is bounded (every fraction below is at most 1), so Filippov solutions are Lipschitz; all barrier arguments below use only that the relevant quantity is absolutely continuous and that its derivative has a definite sign on an open region, hence they hold for every Filippov solution. We also note that the x x x -equations do not contain 1 v ≥ 0 \mathbf{1}_{v\ge0} 1 v ≥ 0 and the y y y -equations do not contain 1 u ≥ 0 \mathbf{1}_{u\ge0} 1 u ≥ 0 , so u ′ u' u ′ is single-valued on the open set { u < 0 } \{u<0\} { u < 0 } and v ′ v' v ′ is single-valued on { v > 0 } \{v>0\} { v > 0 } and on { v < 0 } \{v<0\} { v < 0 } , whatever happens on the other switching surface. Define
G = e y 1 + N − 2 D 1 , H = e y 1 D 1 , E = e u + + v + (136) G=\frac{e^{y_{1}}+N-2}{D_{1}},\qquad H=\frac{e^{y_{1}}}{D_{1}},\qquad E=e^{u_{+}+v_{+}} \tag*{(136)} G = D 1 e y 1 + N − 2 , H = D 1 e y 1 , E = e u + + v + (136)
B i = e y 1 D 2 , i , C i = e y 1 + N − K − 1 D 2 , i , A i = ( i − 1 ) E D 2 , i , (137) B_{i}=\frac{e^{y_{1}}}{D_{2,i}},\qquad C_{i}=\frac{e^{y_{1}}+N-K-1}{D_{2,i}},\qquad A_{i}=\frac{(i-1)E}{D_{2,i}}, \tag*{(137)} B i = D 2 , i e y 1 , C i = D 2 , i e y 1 + N − K − 1 , A i = D 2 , i ( i − 1 ) E , (137)
Q = E D 4 , R = N − K − 1 D 4 , M = N − 1 D 3 . (138) Q=\frac{E}{D_{4}},\qquad R=\frac{N-K-1}{D_{4}},\qquad M=\frac{N-1}{D_{3}}. \tag*{(138)} Q = D 4 E , R = D 4 N − K − 1 , M = D 3 N − 1 . (138)
Also set
ρ = N N − 1 . (139) \rho=\frac{N}{N-1}. \tag*{(139)} ρ = N − 1 N . (139)
Since multiplying the vector field by the positive constant α \alpha α only rescales time, we keep α \alpha α in the formulas but note that the solution with learning rate α \alpha α at time t t t coincides with the solution for α = 1 \alpha=1 α = 1 at time α t \alpha t α t ; hence all constants below are independent of α \alpha α once the growth is expressed in terms of α t \alpha t α t .
When u < 0 u<0 u < 0 , all 1 u ≥ 0 \mathbf{1}_{u\ge0} 1 u ≥ 0 terms vanish and all 1 u < 0 \mathbf{1}_{u<0} 1 u < 0 terms are active. Hence
x 2 ′ = 0 , (140) x_2'=0, \tag*{(140)} x 2 ′ = 0 , (140)
whereas
x 1 ′ = α N [ G + ∑ i = 1 K ( C i + A i ) ] > 0. (141) x_1'=\frac{\alpha}{N}\left[G+\sum_{i=1}^{K}(C_i+A_i)\right]>0. \tag*{(141)} x 1 ′ = N α [ G + i = 1 ∑ K ( C i + A i ) ] > 0. (141)
Therefore u ′ = x 1 ′ − x 2 ′ > 0 u'=x_1'-x_2'>0 u ′ = x 1 ′ − x 2 ′ > 0 whenever u < 0 u<0 u < 0 . Since u ( 0 ) ≥ 0 u(0)\ge0 u ( 0 ) ≥ 0 , the Filippov forward barrier gives
u ( t ) = x 1 ( t ) − x 2 ( t ) ≥ 0. (142) u(t)=x_1(t)-x_2(t)\ge0. \tag*{(142)} u ( t ) = x 1 ( t ) − x 2 ( t ) ≥ 0. (142)
Indeed, suppose u ( t 1 ) < 0 u(t_1)<0 u ( t 1 ) < 0 for some t 1 > 0 t_1>0 t 1 > 0 and let t 0 = sup { t < t 1 : u ( t ) ≥ 0 } t_0=\sup\{t<t_1:u(t)\ge0\} t 0 = sup { t < t 1 : u ( t ) ≥ 0 } . Then u ( t 0 ) = 0 u(t_0)=0 u ( t 0 ) = 0 and u < 0 u<0 u < 0 on ( t 0 , t 1 ] (t_0,t_1] ( t 0 , t 1 ] ; since x 1 ′ , x 2 ′ x_1',x_2' x 1 ′ , x 2 ′ do not involve 1 v ≥ 0 \mathbf{1}_{v\ge0} 1 v ≥ 0 , on the open set { u < 0 } \{u<0\} { u < 0 } every Filippov solution satisfies u ′ = x 1 ′ > 0 u'=x_1'>0 u ′ = x 1 ′ > 0 a.e., so u ( t 1 ) > u ( t 0 ) = 0 u(t_1)>u(t_0)=0 u ( t 1 ) > u ( t 0 ) = 0 , a contradiction. Consequently,
u + = u , ∀ t ≥ 0. (143) u_+=u,\qquad\forall t\ge0. \tag*{(143)} u + = u , ∀ t ≥ 0. (143)
The same one-sided barrier argument is used repeatedly below without further comment. We first prove that v + v_+ v + is upper-bounded.
Upper bound of v + v_+ v + . In order to prove
v + ( t ) = ( y 1 ( t ) − y 2 ( t ) ) + ≤ C . (144) v_+(t)=(y_1(t)-y_2(t))_+\le C. \tag*{(144)} v + ( t ) = ( y 1 ( t ) − y 2 ( t ) ) + ≤ C . (144)
It is enough to consider the open region v > 0 v>0 v > 0 , where E = e u + v E=e^{u+v} E = e u + v and
v ′ = α N − 1 [ K M + R − H − ∑ i = 1 K B i − 2 ∑ i = 1 K A i − K Q ] ≤ α N − 1 ( K M + R − K Q ) . (145) v'=\frac{\alpha}{N-1}\left[KM+R-H-\sum_{i=1}^{K}B_i-2\sum_{i=1}^{K}A_i-KQ\right]\le\frac{\alpha}{N-1}(KM+R-KQ). \tag*{(145)} v ′ = N − 1 α [ K M + R − H − i = 1 ∑ K B i − 2 i = 1 ∑ K A i − K Q ] ≤ N − 1 α ( K M + R − K Q ) . (145)
To get a negative gradient v ′ v' v ′ , we split and discuss two cases by fixing a constant L ≥ 1 L\ge1 L ≥ 1 .
Case 1: e y 2 − u ≤ L e^{y_2-u}\le L e y 2 − u ≤ L .
Since y 1 = y 2 + v y_1=y_2+v y 1 = y 2 + v and E = e u + v E=e^{u+v} E = e u + v ,
K Q = K e u + v K e u + v + e y 1 + N − K − 1 = K K + e y 2 − u + ( N − K − 1 ) e − u − v , (146) KQ=\frac{Ke^{u+v}}{Ke^{u+v}+e^{y_1}+N-K-1}=\frac{K}{K+e^{y_2-u}+(N-K-1)e^{-u-v}}, \tag*{(146)} K Q = K e u + v + e y 1 + N − K − 1 K e u + v = K + e y 2 − u + ( N − K − 1 ) e − u − v K , (146)
so K Q KQ K Q has a positive lower bound, i.e., K Q ≥ c L : = K / ( L + N − 1 ) > 0 KQ\ge c_L:=K/(L+N-1)>0 K Q ≥ c L := K / ( L + N − 1 ) > 0 (here we used u + v ≥ 0 u+v\ge0 u + v ≥ 0 ). Meanwhile, for some constant C = C N , K C=C_{N,K} C = C N , K , we have
K M + R ≤ C e − y 1 + C e − u − v = C e − y 2 − v + C e − u − v ≤ 2 C e − v , (147) KM+R\le Ce^{-y_1}+Ce^{-u-v}=Ce^{-y_2-v}+Ce^{-u-v}\le2Ce^{-v}, \tag*{(147)} K M + R ≤ C e − y 1 + C e − u − v = C e − y 2 − v + C e − u − v ≤ 2 C e − v , (147)
where the last step uses y 2 ≥ 0 y_2\ge0 y 2 ≥ 0 and u ≥ 0 u\ge0 u ≥ 0 . Thus v ′ < 0 v'<0 v ′ < 0 once v v v is larger than a constant depending on L L L .
Case 2: e y 2 − u > L e^{y_2-u}>L e y 2 − u > L . Put
S : = e y 2 − u . (148) S := e^{y_2-u}. \tag*{(148)} S := e y 2 − u . (148)
Then we have S > L S > L S > L . Since v > 0 v > 0 v > 0 and u ≥ 0 u \ge0 u ≥ 0 , we have
E = e u + v ≥ e v , e y 1 = e y 2 + v = S E . (149) E = e^{u+v} \ge e^v,\qquad e^{y_1} = e^{y_2+v} = SE. \tag*{(149)} E = e u + v ≥ e v , e y 1 = e y 2 + v = S E . (149)
Consequently
D 4 = K E + e y 1 + N − K − 1 = E ( K + S + ( N − K − 1 ) E − 1 ) . (150) D_4 = KE + e^{y_1} + N - K - 1 = E\left(K + S + (N-K-1)E^{-1}\right). \tag*{(150)} D 4 = K E + e y 1 + N − K − 1 = E ( K + S + ( N − K − 1 ) E − 1 ) . (150)
The negative term coming from D 4 D_4 D 4 therefore satisfies, once L ≥ 1 L \ge1 L ≥ 1 ,
K Q = K E D 4 = K K + S + ( N − K − 1 ) E − 1 ≥ K ( 2 N ) S = c Q S , (151) KQ = \frac{KE}{D_4} = \frac{K}{K+S+(N-K-1)E^{-1}} \ge\frac{K}{(2N)S} = \frac{c_Q}{S}, \tag*{(151)} K Q = D 4 K E = K + S + ( N − K − 1 ) E − 1 K ≥ ( 2 N ) S K = S c Q , (151)
where we used S > L ≥ 1 S > L \ge1 S > L ≥ 1 , K ≤ N − 1 K \le N-1 K ≤ N − 1 , and E − 1 ≤ 1 E^{-1} \le1 E − 1 ≤ 1 , so K + S + ( N − K − 1 ) E − 1 ≤ 2 N S K+S+(N-K-1)E^{-1} \le2NS K + S + ( N − K − 1 ) E − 1 ≤ 2 N S . On the other hand, the two positive terms in the upper bound for v ′ v' v ′ are much smaller:
K M = K N − 1 e y 1 + N − 1 ≤ K N − 1 S E , (152) KM = K\frac{N-1}{e^{y_1}+N-1} \le K\frac{N-1}{SE}, \tag*{(152)} K M = K e y 1 + N − 1 N − 1 ≤ K S E N − 1 , (152)
and, since D 4 ≥ e y 1 = S E D_4 \ge e^{y_1}=SE D 4 ≥ e y 1 = S E ,
R = N − K − 1 D 4 ≤ N − K − 1 S E . (153) R = \frac{N-K-1}{D_4} \le\frac{N-K-1}{SE}. \tag*{(153)} R = D 4 N − K − 1 ≤ S E N − K − 1 . (153)
Thus
K M + R ≤ C N , K S E ≤ C N , K e − v S . (154) KM+R \le\frac{C_{N,K}}{SE} \le\frac{C_{N,K}e^{-v}}{S}. \tag*{(154)} K M + R ≤ S E C N , K ≤ S C N , K e − v . (154)
Choose v 1 = v 1 ( N , K ) v_1=v_1(N,K) v 1 = v 1 ( N , K ) so large that
C N , K e − v 1 ≤ c Q 2 . (155) C_{N,K}e^{-v_1} \le\frac{c_Q}{2}. \tag*{(155)} C N , K e − v 1 ≤ 2 c Q . (155)
Then for every v ≥ v 1 v \ge v_1 v ≥ v 1 and every point in the case S > L S>L S > L ,
K M + R ≤ c Q 2 S ≤ 1 2 K Q . (156) KM+R \le\frac{c_Q}{2S} \le\frac{1}{2}KQ. \tag*{(156)} K M + R ≤ 2 S c Q ≤ 2 1 K Q . (156)
Returning to the differential inequality, we have
v ′ ≤ α N − 1 ( K M + R − K Q ) ≤ − α 2 ( N − 1 ) K Q < 0. (157) v' \le\frac{\alpha}{N-1}(KM+R-KQ) \le-\frac{\alpha}{2(N-1)}KQ < 0. \tag*{(157)} v ′ ≤ N − 1 α ( K M + R − K Q ) ≤ − 2 ( N − 1 ) α K Q < 0. (157)
This proves that in the whole region e y 2 − u > L e^{y_2-u}>L e y 2 − u > L , the positive y y y -gap is pushed strictly downward whenever v ≥ v 1 v \ge v_1 v ≥ v 1 .
Combining the two cases, there exists C v C_v C v such that
v > C v ⟹ v ′ < 0. (158) v>C_v \quad\Longrightarrow\quad v'<0. \tag*{(158)} v > C v ⟹ v ′ < 0. (158)
Since { v > C v } \{v>C_v\} { v > C v } lies strictly inside the open region { v > 0 } \{v>0\} { v > 0 } , where v ′ v' v ′ is single-valued, the one-sided barrier argument gives
v + = ( y 1 − y 2 ) + ≤ C = max { v ( 0 ) , C v } , ∀ t ≥ 0. (159) v_+=(y_1-y_2)_+ \le C=\max\{v(0),C_v\},\qquad\forall t\ge0. \tag*{(159)} v + = ( y 1 − y 2 ) + ≤ C = max { v ( 0 ) , C v } , ∀ t ≥ 0. (159)
Prove x 1 , x 2 , y 1 , y 2 → ∞ x_1,x_2,y_1,y_2 \to\infty x 1 , x 2 , y 1 , y 2 → ∞ as t → ∞ t \to\infty t → ∞ . First observe that, for every Filippov selection (the 1 u ≥ 0 \mathbb{1}_{u\geq0} 1 u ≥ 0 -terms cancel in the combination x 1 + ( N − 1 ) x 2 x_1+(N-1)x_2 x 1 + ( N − 1 ) x 2 ),
( x 1 + ( N − 1 ) x 2 ) ′ = α N [ G + ∑ i = 1 K ( C i + A i ) ] ≥ α N G (160) (x_1+(N-1)x_2)'=\frac{\alpha}{N}\left[G+\sum_{i=1}^{K}(C_i+A_i)\right]\geq\frac{\alpha}{N}G \tag*{(160)} ( x 1 + ( N − 1 ) x 2 ) ′ = N α [ G + i = 1 ∑ K ( C i + A i ) ] ≥ N α G (160)
≥ α N ( N − 2 ) e y 1 e x 1 + e y 1 + 1 ≥ α N ( N − 2 ) 1 e x 1 + ( N − 1 ) x 2 + 2 , (161) \geq\frac{\alpha}{N(N-2)}\frac{e^{y_1}}{e^{x_1}+e^{y_1}+1}\geq\frac{\alpha}{N(N-2)}\frac{1}{e^{x_1+(N-1)x_2}+2}, \tag*{(161)} ≥ N ( N − 2 ) α e x 1 + e y 1 + 1 e y 1 ≥ N ( N − 2 ) α e x 1 + ( N − 1 ) x 2 + 2 1 , (161)
where we used e x 1 + e y 1 + N − 2 ≤ ( N − 2 ) ( e x 1 + e y 1 + 1 ) e^{x_1}+e^{y_1}+N-2\leq(N-2)(e^{x_1}+e^{y_1}+1) e x 1 + e y 1 + N − 2 ≤ ( N − 2 ) ( e x 1 + e y 1 + 1 ) , e y 1 ≥ 1 e^{y_1}\geq1 e y 1 ≥ 1 and x 2 ≥ 0 x_2\geq0 x 2 ≥ 0 . Writing w = x 1 + ( N − 1 ) x 2 w=x_1+(N-1)x_2 w = x 1 + ( N − 1 ) x 2 , this gives ( e w + 2 w ) ′ ≥ c (e^w+2w)'\geq c ( e w + 2 w ) ′ ≥ c , hence w ≥ log ( c t ) − C w\geq\log(ct)-C w ≥ log ( c t ) − C for large t t t , i.e., x 1 + ( N − 1 ) x 2 ≳ log ( t ) x_1+(N-1)x_2\gtrsim\log(t) x 1 + ( N − 1 ) x 2 ≳ log ( t ) . Since x 1 ≥ x 2 x_1\geq x_2 x 1 ≥ x 2 , we conclude that lim t → ∞ x 1 = + ∞ \lim_{t\to\infty}x_1=+\infty lim t → ∞ x 1 = + ∞ . Therefore lim t → ∞ D 1 = + ∞ , lim t → ∞ D 2 , i = + ∞ \lim_{t\to\infty}D_1=+\infty,\lim_{t\to\infty}D_{2,i}=+\infty lim t → ∞ D 1 = + ∞ , lim t → ∞ D 2 , i = + ∞ , and consequently H → 0 H\to0 H → 0 and B i → 0 B_i\to0 B i → 0 as long as y 1 y_1 y 1 stays bounded.
Next we show that y 1 y_1 y 1 is unbounded. Note that y 1 ′ y_1' y 1 ′ contains the negative term − 1 v ≥ 0 ∑ i A i -1_{v\geq0}\sum_i A_i − 1 v ≥ 0 ∑ i A i , which need not be small at this stage, because A i → 0 A_i\to0 A i → 0 only once x 2 → ∞ x_2\to\infty x 2 → ∞ ; we therefore consider y 1 + y 2 y_1+y_2 y 1 + y 2 , whose derivative does not depend on 1 v ≥ 0 1_{v\geq0} 1 v ≥ 0 and hence is the same for every Filippov selection on v = 0 v=0 v = 0 :
( y 1 + y 2 ) ′ = α N − 1 [ K M + R − H − ∑ i = 1 K B i + K Q ] ≥ α N − 1 [ K M − H − ∑ i = 1 K B i ] . (162) (y_1+y_2)'=\frac{\alpha}{N-1}\left[KM+R-H-\sum_{i=1}^{K}B_i+KQ\right]\geq\frac{\alpha}{N-1}\left[KM-H-\sum_{i=1}^{K}B_i\right]. \tag*{(162)} ( y 1 + y 2 ) ′ = N − 1 α [ K M + R − H − i = 1 ∑ K B i + K Q ] ≥ N − 1 α [ K M − H − i = 1 ∑ K B i ] . (162)
Suppose y 1 ≤ B y_1\leq B y 1 ≤ B for all t t t . Since y 2 ′ = 0 y_2'=0 y 2 ′ = 0 on { v < 0 } \{v<0\} { v < 0 } , y 2 y_2 y 2 can increase only while y 2 ≤ y 1 ≤ B y_2\leq y_1\leq B y 2 ≤ y 1 ≤ B ; hence y 2 ( t ) ≤ max { y 2 ( 0 ) , B } y_2(t)\leq\max\{y_2(0),B\} y 2 ( t ) ≤ max { y 2 ( 0 ) , B } for all t t t (if y 2 ( t 1 ) > max { y 2 ( 0 ) , B } y_2(t_1)>\max\{y_2(0),B\} y 2 ( t 1 ) > max { y 2 ( 0 ) , B } , let t 0 < t 1 t_0<t_1 t 0 < t 1 be the last time with y 2 ≤ max { y 2 ( 0 ) , B } y_2\leq\max\{y_2(0),B\} y 2 ≤ max { y 2 ( 0 ) , B } ; on ( t 0 , t 1 ] (t_0,t_1] ( t 0 , t 1 ] we have y 2 > B ≥ y 1 y_2>B\geq y_1 y 2 > B ≥ y 1 , i.e. v < 0 v<0 v < 0 , so y 2 ′ = 0 y_2'=0 y 2 ′ = 0 a.e. there, a contradiction). Thus y 1 + y 2 y_1+y_2 y 1 + y 2 is bounded. On the other hand M ≥ ( N − 1 ) / ( e B + N − 1 ) = : c B > 0 M\geq(N-1)/(e^B+N-1)=:c_B>0 M ≥ ( N − 1 ) / ( e B + N − 1 ) =: c B > 0 depends only on B B B , while H + ∑ i B i ≤ e B ( 1 / D 1 + ∑ i 1 / D 2 , i ) → 0 H+\sum_iB_i\leq e^B(1/D_1+\sum_i1/D_{2,i})\to0 H + ∑ i B i ≤ e B ( 1/ D 1 + ∑ i 1/ D 2 , i ) → 0 because x 1 → ∞ x_1\to\infty x 1 → ∞ . Hence ( y 1 + y 2 ) ′ ≥ α N − 1 K c B 2 > 0 (y_1+y_2)'\geq\frac{\alpha}{N-1}\frac{Kc_B}{2}>0 ( y 1 + y 2 ) ′ ≥ N − 1 α 2 K c B > 0 for all large t t t , so y 1 + y 2 → ∞ y_1+y_2\to\infty y 1 + y 2 → ∞ , a contradiction. Therefore lim sup t y 1 = + ∞ \limsup_t y_1=+\infty lim sup t y 1 = + ∞ . Since y 2 ≥ y 1 − v + ≥ y 1 − C y_2\geq y_1-v_+\geq y_1-C y 2 ≥ y 1 − v + ≥ y 1 − C , also lim sup t y 2 = + ∞ \limsup_t y_2=+\infty lim sup t y 2 = + ∞ , and since y 2 y_2 y 2 is non-decreasing, lim t y 2 = + ∞ \lim_t y_2=+\infty lim t y 2 = + ∞ .
For x 2 x_2 x 2 , suppose it is bounded. Since x 1 → ∞ x_1\to\infty x 1 → ∞ , u > 0 u>0 u > 0 for all large t t t , and on { u > 0 } \{u>0\} { u > 0 } we have x 2 ′ = 1 N ( N − 1 ) [ ∑ i A i + K Q ] x_2'=\frac{1}{N(N-1)}[\sum_i A_i+KQ] x 2 ′ = N ( N − 1 ) 1 [ ∑ i A i + K Q ] while y 2 ′ = η 1 N − 1 [ ∑ i A i + K Q ] ≤ N x 2 ′ y_2'=\eta\frac{1}{N-1}[\sum_i A_i+KQ]\leq Nx_2' y 2 ′ = η N − 1 1 [ ∑ i A i + K Q ] ≤ N x 2 ′ for every Filippov weight η ∈ [ 0 , 1 ] \eta\in[0,1] η ∈ [ 0 , 1 ] of 1 v ≥ 0 1_{v\geq0} 1 v ≥ 0 . Integrating, x 2 ( t ) ≥ x 2 ( T ) + ( y 2 ( t ) − y 2 ( T ) ) / N → + ∞ x_2(t)\geq x_2(T)+(y_2(t)-y_2(T))/N\to+\infty x 2 ( t ) ≥ x 2 ( T ) + ( y 2 ( t ) − y 2 ( T )) / N → + ∞ , which contradicts the boundedness assumption. Therefore, lim t x 2 ( t ) → + ∞ \lim_t x_2(t)\to+\infty lim t x 2 ( t ) → + ∞ .
For y 1 y_1 y 1 , ∀ B > 0 \forall B>0 ∀ B > 0 , we prove that there exists a time t 0 t_0 t 0 such that y 1 ≥ B y_1\geq B y 1 ≥ B for all t ≥ t 0 t\geq t_0 t ≥ t 0 . On the region { v < 0 , y 1 ≤ B } \{v<0,y_1\leq B\} { v < 0 , y 1 ≤ B } there is no A i A_i A i -term in y 1 ′ y_1' y 1 ′ , and
y 1 ′ = α N − 1 [ K M + R − H − ∑ i B i + K Q ] ≥ α N − 1 [ K c B − e B ( 1 D 1 + ∑ i 1 D 2 , i ) ] ≥ c > 0 (163) y_1'=\frac{\alpha}{N-1}\left[KM+R-H-\sum_iB_i+KQ\right]\geq\frac{\alpha}{N-1}\left[Kc_B-e^B\left(\frac{1}{D_1}+\sum_i\frac{1}{D_{2,i}}\right)\right]\geq c>0 \tag*{(163)} y 1 ′ = N − 1 α [ K M + R − H − i ∑ B i + K Q ] ≥ N − 1 α [ K c B − e B ( D 1 1 + i ∑ D 2 , i 1 ) ] ≥ c > 0 (163)
for all t ≥ t 1 t\geq t_1 t ≥ t 1 , where t 1 t_1 t 1 is chosen, using x 1 → ∞ x_1\to\infty x 1 → ∞ , so that the bracket is at least K c B / 2 Kc_B/2 K c B /2 . Since lim t y 2 = + ∞ \lim_t y_2=+\infty lim t y 2 = + ∞ , pick t 2 ≥ t 1 t_2\geq t_1 t 2 ≥ t 1 with y 2 ( t 2 ) > B y_2(t_2)>B y 2 ( t 2 ) > B . For t ≥ t 2 t\geq t_2 t ≥ t 2 , whenever y 1 ( t ) ≤ B y_1(t)\leq B y 1 ( t ) ≤ B we have v ( t ) ≤ B − y 2 ( t 2 ) < 0 v(t)\leq B-y_2(t_2)<0 v ( t ) ≤ B − y 2 ( t 2 ) < 0 because y 2 y_2 y 2 is non-decreasing, so the state lies in { v < 0 , y 1 ≤ B } \{v<0,y_1\leq B\} { v < 0 , y 1 ≤ B } and y 1 ′ ≥ c > 0 y_1'\geq c>0 y 1 ′ ≥ c > 0 there. Hence y 1 ≥ B y_1\geq B y 1 ≥ B for all t ≥ t 0 : = t 2 + ( B − y 1 ( t 2 ) ) + / c t\geq t_0:=t_2+(B-y_1(t_2))_+/c t ≥ t 0 := t 2 + ( B − y 1 ( t 2 ) ) + / c , by the barrier argument. Thus lim t y 1 ( t ) → + ∞ \lim_t y_1(t)\to+\infty lim t y 1 ( t ) → + ∞ .
Upper bound of 2 y 1 − x 1 2y_1-x_1 2 y 1 − x 1 . To analyze the behavior of 2 y 1 − x 1 2y_1-x_1 2 y 1 − x 1 , we define new variables
s = e u = e x 1 − x 2 , Z = e 2 y 1 − x 1 . (164) s=e^u=e^{x_1-x_2},\qquad Z=e^{2y_1-x_1}. \tag*{(164)} s = e u = e x 1 − x 2 , Z = e 2 y 1 − x 1 . (164)
The desired gap estimates are
s ≤ C , Z ≤ C . (165) s\leq C,\qquad Z\leq C. \tag*{(165)} s ≤ C , Z ≤ C . (165)
Indeed, s ≤ C s \le C s ≤ C is equivalent to x 1 − x 2 ≤ C x_1-x_2 \le C x 1 − x 2 ≤ C , and Z ≤ C Z \le C Z ≤ C is equivalent to 2 y 1 − x 1 ≤ C 2y_1-x_1 \le C 2 y 1 − x 1 ≤ C . To prove the upper bound, we use a Lyapunov argument. Define the Lyapunov product
W = Z s p . (166) W=Zs^p. \tag*{(166)} W = Z s p . (166)
Choose
3 − 1 N < p < 1 + 2 N N − 1 . (167) 3-\frac{1}{N}<p<1+\frac{2N}{N-1}. \tag*{(167)} 3 − N 1 < p < 1 + N − 1 2 N . (167)
The interval is nonempty because
1 + 2 N N − 1 − ( 3 − 1 N ) = 2 N − 1 + 1 N = 3 N − 1 N ( N − 1 ) > 0. (168) 1+\frac{2N}{N-1}-\left(3-\frac{1}{N}\right)=\frac{2}{N-1}+\frac{1}{N}=\frac{3N-1}{N(N-1)}>0. \tag*{(168)} 1 + N − 1 2 N − ( 3 − N 1 ) = N − 1 2 + N 1 = N ( N − 1 ) 3 N − 1 > 0. (168)
We discuss the regions where u > 0 u>0 u > 0 or u = 0 u=0 u = 0 separately.
Region 1: u > 0 u>0 u > 0 . We have 1 u ≥ 0 = 1 \mathbf{1}_{u\ge0}=1 1 u ≥ 0 = 1 and 1 u < 0 = 0 \mathbf{1}_{u<0}=0 1 u < 0 = 0 . Thus
x 1 ′ = α N [ G + ∑ i = 1 K C i − K Q ] . (169) x_1'=\frac{\alpha}{N}\left[G+\sum_{i=1}^{K}C_i-KQ\right]. \tag*{(169)} x 1 ′ = N α [ G + i = 1 ∑ K C i − K Q ] . (169)
x 2 ′ = α N ( N − 1 ) [ ∑ i = 1 K A i + K Q ] . (170) x_2'=\frac{\alpha}{N(N-1)}\left[\sum_{i=1}^{K}A_i+KQ\right]. \tag*{(170)} x 2 ′ = N ( N − 1 ) α [ i = 1 ∑ K A i + K Q ] . (170)
Therefore
s ′ s = u ′ = α N [ G + ∑ i = 1 K C i − 1 N − 1 ∑ i = 1 K A i − K N N − 1 Q ] . (171) \frac{s'}{s}=u'=\frac{\alpha}{N}\left[G+\sum_{i=1}^{K}C_i-\frac{1}{N-1}\sum_{i=1}^{K}A_i-\frac{KN}{N-1}Q\right]. \tag*{(171)} s s ′ = u ′ = N α [ G + i = 1 ∑ K C i − N − 1 1 i = 1 ∑ K A i − N − 1 K N Q ] . (171)
Since v v v may not be positive and can hit the sliding boundary, we introduce the Filippov parameter 0 ≤ η ≤ 1 0\le\eta\le1 0 ≤ η ≤ 1 , where η \eta η is the convex value of 1 v ≥ 0 \mathbf{1}_{v\ge0} 1 v ≥ 0 . Then every admissible velocity on u > 0 u>0 u > 0 satisfies
y 1 ′ = α N − 1 [ K M + R − H − ∑ i = 1 K B i − η ∑ i = 1 K A i + ( 1 − η ) K Q ] . (172) y_1'=\frac{\alpha}{N-1}\left[KM+R-H-\sum_{i=1}^{K}B_i-\eta\sum_{i=1}^{K}A_i+(1-\eta)KQ\right]. \tag*{(172)} y 1 ′ = N − 1 α [ K M + R − H − i = 1 ∑ K B i − η i = 1 ∑ K A i + ( 1 − η ) K Q ] . (172)
Consequently
Z ′ Z = 2 y 1 ′ − x 1 ′ = α N [ 2 ρ K M + 2 ρ R − 2 ρ H − G − ∑ i = 1 K { 2 ρ B i + C i } − 2 ρ η ∑ i = 1 K A i + K { 1 + 2 ρ ( 1 − η ) } Q ] , (173) \begin{aligned}
\frac{Z'}{Z}&=2y_1'-x_1'\\
&=\frac{\alpha}{N}\left[2\rho KM+2\rho R-2\rho H-G-\sum_{i=1}^{K}\{2\rho B_i+C_i\}-2\rho\eta\sum_{i=1}^{K}A_i+K\{1+2\rho(1-\eta)\}Q\right], \tag*{(173)}
\end{aligned} Z Z ′ = 2 y 1 ′ − x 1 ′ = N α [ 2 ρ K M + 2 ρR − 2 ρ H − G − i = 1 ∑ K { 2 ρ B i + C i } − 2 ρ η i = 1 ∑ K A i + K { 1 + 2 ρ ( 1 − η )} Q ] , (173)
where ρ = N / ( N − 1 ) \rho=N/(N-1) ρ = N / ( N − 1 ) . Therefore
W ′ W = Z ′ Z + p s ′ s = α N [ 2 ρ K M + 2 ρ R + { ( p − 1 ) G − 2 ρ H } + ∑ i = 1 K ( p C i − C i − 2 ρ B i ) − ( 2 ρ η + p N − 1 ) ∑ i = 1 K A i + K ( 1 + 2 ρ ( 1 − η ) − p N N − 1 ) Q ] . (174) \begin{aligned}
\frac{W'}{W}
&=\frac{Z'}{Z}+p\frac{s'}{s}\\
&=\frac{\alpha}{N}\left[2\rho KM+2\rho R+\{(p-1)G-2\rho H\}+\sum_{i=1}^{K}(pC_i-C_i-2\rho B_i)\right.\\
&\qquad\left.-\left(2\rho\eta+\frac{p}{N-1}\right)\sum_{i=1}^{K}A_i+K\left(1+2\rho(1-\eta)-\frac{pN}{N-1}\right)Q\right]. \tag*{(174)}
\end{aligned} W W ′ = Z Z ′ + p s s ′ = N α [ 2 ρ K M + 2 ρR + {( p − 1 ) G − 2 ρ H } + i = 1 ∑ K ( p C i − C i − 2 ρ B i ) − ( 2 ρ η + N − 1 p ) i = 1 ∑ K A i + K ( 1 + 2 ρ ( 1 − η ) − N − 1 pN ) Q ] . (174)
The coefficient of Q Q Q is largest when η = 0 \eta= 0 η = 0 , and the lower bound p > 3 − 1 / N p > 3 - 1/N p > 3 − 1/ N is exactly
1 + 2 ρ − p N N − 1 < 0. (175) 1 + 2\rho- \frac{pN}{N - 1} < 0. \tag*{(175)} 1 + 2 ρ − N − 1 pN < 0. (175)
The coefficient of ∑ i A i \sum_i A_i ∑ i A i is strictly negative. The indexed D 2 , i D_{2,i} D 2 , i contribution can be written as
p C i − C i − 2 ρ B i = ( p − 1 ) ( N − K − 1 ) + ( p − 1 − 2 ρ ) e y 1 D 2 , i . (176) pC_i - C_i - 2\rho B_i = \frac{(p - 1)(N - K - 1) + (p - 1 - 2\rho)e^{y_1}}{D_{2,i}}. \tag*{(176)} p C i − C i − 2 ρ B i = D 2 , i ( p − 1 ) ( N − K − 1 ) + ( p − 1 − 2 ρ ) e y 1 . (176)
Since p < 1 + 2 ρ p < 1 + 2\rho p < 1 + 2 ρ , the coefficient of e y 1 e^{y_1} e y 1 is strictly negative. The same calculation for D 1 D_1 D 1 gives
( p − 1 ) G − 2 ρ H = ( p − 1 ) ( N − 2 ) + ( p − 1 − 2 ρ ) e y 1 D 1 , (177) (p - 1)G - 2\rho H = \frac{(p - 1)(N - 2) + (p - 1 - 2\rho)e^{y_1}}{D_1}, \tag*{(177)} ( p − 1 ) G − 2 ρ H = D 1 ( p − 1 ) ( N − 2 ) + ( p − 1 − 2 ρ ) e y 1 , (177)
which has the same structure.
We now estimate each term in Equation (176).
Define
a = e x 1 , b = e y 1 , s = e u . (178) a = e^{x_1}, \qquad b = e^{y_1}, \qquad s = e^u. \tag*{(178)} a = e x 1 , b = e y 1 , s = e u . (178)
Since v + v_+ v + is bounded, there are constants 0 < c E < C E < ∞ 0 < c_E < C_E < \infty 0 < c E < C E < ∞ , depending only on the previously obtained v + v_+ v + bound, such that
c E s ≤ E = s e v + ≤ C E s . (179) c_Es \le E = se^{v_+} \le C_Es. \tag*{(179)} c E s ≤ E = s e v + ≤ C E s . (179)
Put
δ y : = 1 + 2 ρ − p > 0 , δ Q : = p ρ − ( 1 + 2 ρ ) > 0 , c A : = p N − 1 > 0. (180) \delta_y := 1 + 2\rho- p > 0, \qquad\delta_Q := p\rho- (1 + 2\rho) > 0, \qquad c_A := \frac{p}{N - 1} > 0. \tag*{(180)} δ y := 1 + 2 ρ − p > 0 , δ Q := pρ − ( 1 + 2 ρ ) > 0 , c A := N − 1 p > 0. (180)
The two strict inequalities follow exactly from the choice 3 − 1 / N < p < 1 + 2 N / ( N − 1 ) 3 - 1/N < p < 1 + 2N/(N - 1) 3 − 1/ N < p < 1 + 2 N / ( N − 1 ) .
Starting from the exact formula for W ′ / W W'/W W ′ / W , we separate the terms with a fixed negative coefficient from the terms with a constant numerator. For the D 1 D_1 D 1 -term,
( p − 1 ) G − 2 ρ H = ( p − 1 ) ( N − 2 ) − δ y b D 1 ≤ − δ y H + C N , K , p 1 D 1 . (181) (p - 1)G - 2\rho H = \frac{(p - 1)(N - 2) - \delta_y b}{D_1} \le-\delta_y H + C_{N,K,p}\frac{1}{D_1}. \tag*{(181)} ( p − 1 ) G − 2 ρ H = D 1 ( p − 1 ) ( N − 2 ) − δ y b ≤ − δ y H + C N , K , p D 1 1 . (181)
For every i i i ,
p C i − C i − 2 ρ B i = ( p − 1 ) ( N − K − 1 ) − δ y b D 2 , i ≤ − δ y B i + C N , K , p 1 D 2 , i . (182) pC_i - C_i - 2\rho B_i = \frac{(p - 1)(N - K - 1) - \delta_y b}{D_{2,i}} \le-\delta_y B_i + C_{N,K,p}\frac{1}{D_{2,i}}. \tag*{(182)} p C i − C i − 2 ρ B i = D 2 , i ( p − 1 ) ( N − K − 1 ) − δ y b ≤ − δ y B i + C N , K , p D 2 , i 1 . (182)
Moreover the coefficient of ∑ i A i \sum_i A_i ∑ i A i is at most − c A -c_A − c A , and the coefficient of K Q KQ K Q is maximal at η = 0 \eta= 0 η = 0 , where it equals − K δ Q -K\delta_Q − K δ Q . Therefore, after enlarging constants, every Filippov velocity in the region u > 0 u > 0 u > 0 satisfies
N α W ′ W ≤ P − N , P : = C ( M + R + 1 D 1 + ∑ i = 1 K 1 D 2 , i ) , N : = c ( H + ∑ i = 1 K B i + ∑ i = 1 K A i + Q ) . (183) \begin{aligned}
\frac{N}{\alpha}\frac{W'}{W} &\le\mathcal{P} - \mathcal{N},\\
\mathcal{P} &:= C\left(M + R + \frac{1}{D_1} + \sum_{i=1}^{K}\frac{1}{D_{2,i}}\right),\\
\mathcal{N} &:= c\left(H + \sum_{i=1}^{K}B_i + \sum_{i=1}^{K}A_i + Q\right). \tag*{(183)}
\end{aligned} α N W W ′ P N ≤ P − N , := C ( M + R + D 1 1 + i = 1 ∑ K D 2 , i 1 ) , := c ( H + i = 1 ∑ K B i + i = 1 ∑ K A i + Q ) . (183)
where P \mathcal{P} P collects the terms with constant numerators and N \mathcal{N} N the terms with a fixed negative coefficient (the letter N N N is reserved for the number of vertices). Thus it is enough to show that N \mathcal{N} N dominates P \mathcal{P} P whenever
W = Z s p = b 2 s p a (184) W = Zs^p = \frac{b^2s^p}{a} \tag*{(184)} W = Z s p = a b 2 s p (184)
is large.
We record the elementary estimates used below; recall a = e x 1 ≥ 1 a=e^{x_1}\ge1 a = e x 1 ≥ 1 , b = e y 1 ≥ 1 b=e^{y_1}\ge1 b = e y 1 ≥ 1 , s = e u ≥ 1 s=e^u\ge1 s = e u ≥ 1 and E ≍ s E\asymp s E ≍ s . Since K − i + 1 ≥ 1 K-i+1\ge1 K − i + 1 ≥ 1 in every D 2 , i D_{2,i} D 2 , i and N − K − 1 ≥ 4 N-K-1\ge4 N − K − 1 ≥ 4 ,
M ≤ N − 1 b + 1 , R ≤ N − K − 1 b + 1 , 1 D 1 + ∑ i 1 D 2 , i ≤ K + 1 b + 1 , hence P ≤ C b + 1 . (185) M \le\frac{N-1}{b+1}, \qquad R \le\frac{N-K-1}{b+1}, \qquad\frac{1}{D_1}+\sum_i\frac{1}{D_{2,i}}\le\frac{K+1}{b+1}, \qquad\text{hence}\qquad\mathcal{P}\le\frac{C}{b+1}. \tag*{(185)} M ≤ b + 1 N − 1 , R ≤ b + 1 N − K − 1 , D 1 1 + i ∑ D 2 , i 1 ≤ b + 1 K + 1 , hence P ≤ b + 1 C . (185)
For the negative part,
Q = E K E + b + N − K − 1 ≍ s s + b + 1 , (186) Q=\frac{E}{KE+b+N-K-1}\asymp\frac{s}{s+b+1}, \tag*{(186)} Q = K E + b + N − K − 1 E ≍ s + b + 1 s , (186)
H = b a + b + N − 2 ≍ b a + b + 1 , B 1 = b K a + b + N − K − 1 ≍ b a + b + 1 , (187) H=\frac{b}{a+b+N-2}\asymp\frac{b}{a+b+1}, \qquad B_1=\frac{b}{Ka+b+N-K-1}\asymp\frac{b}{a+b+1}, \tag*{(187)} H = a + b + N − 2 b ≍ a + b + 1 b , B 1 = K a + b + N − K − 1 b ≍ a + b + 1 b , (187)
so that, dropping the nonnegative terms A i A_i A i and B i B_i B i , i ≥ 2 i\ge2 i ≥ 2 ,
N ≥ c ( b a + b + 1 + s s + b + 1 ) . (188) \mathcal{N}\ge c\left(\frac{b}{a+b+1}+\frac{s}{s+b+1}\right). \tag*{(188)} N ≥ c ( a + b + 1 b + s + b + 1 s ) . (188)
We now show that for b ≥ 1 b\ge1 b ≥ 1 , s ≥ 1 s\ge1 s ≥ 1 and W = b 2 s p / a W=b^2s^p/a W = b 2 s p / a ,
b a + b + 1 + s s + b + 1 ≥ 1 4 min { 1 , W 1 / ( p + 1 ) b + 1 } . (189) \frac{b}{a+b+1}+\frac{s}{s+b+1}\ge\frac{1}{4}\min\left\{1,\frac{W^{1/(p+1)}}{b+1}\right\}. \tag*{(189)} a + b + 1 b + s + b + 1 s ≥ 4 1 min { 1 , b + 1 W 1/ ( p + 1 ) } . (189)
Indeed, if s ≥ b + 1 s\ge b+1 s ≥ b + 1 the second term is at least 1 2 \frac{1}{2} 2 1 ; if a ≤ b + 1 a\le b+1 a ≤ b + 1 the first term is at least b / ( 2 ( b + 1 ) ) ≥ 1 4 b/(2(b+1))\ge\frac{1}{4} b / ( 2 ( b + 1 )) ≥ 4 1 . Otherwise a > b + 1 ≥ s a>b+1\ge s a > b + 1 ≥ s , so b / ( a + b + 1 ) ≥ b / ( 3 a ) = W / ( 3 b s p ) b/(a+b+1)\ge b/(3a)=W/(3bs^p) b / ( a + b + 1 ) ≥ b / ( 3 a ) = W / ( 3 b s p ) and s / ( s + b + 1 ) ≥ s / ( 3 ( b + 1 ) ) s/(s+b+1)\ge s/(3(b+1)) s / ( s + b + 1 ) ≥ s / ( 3 ( b + 1 )) , and by the weighted AM–GM inequality, for all s > 0 s>0 s > 0 ,
W b s p + s b + 1 ≥ ( W b ) 1 / ( p + 1 ) ( b + 1 ) − p / ( p + 1 ) ≥ W 1 / ( p + 1 ) b + 1 , (190) \frac{W}{bs^p}+\frac{s}{b+1}\ge\left(\frac{W}{b}\right)^{1/(p+1)}(b+1)^{-p/(p+1)}\ge\frac{W^{1/(p+1)}}{b+1}, \tag*{(190)} b s p W + b + 1 s ≥ ( b W ) 1/ ( p + 1 ) ( b + 1 ) − p / ( p + 1 ) ≥ b + 1 W 1/ ( p + 1 ) , (190)
where we used b ≤ b + 1 b\le b+1 b ≤ b + 1 . This proves Equation (189) .
Combining Equations (183) , (185) , (188) and (189) , every Filippov velocity in the region u > 0 u>0 u > 0 satisfies
N W ′ α W ≤ C b + 1 − c ′ min { 1 , W 1 / ( p + 1 ) b + 1 } . (191) \frac{NW'}{\alpha W}\le\frac{C}{b+1}-c'\min\left\{1,\frac{W^{1/(p+1)}}{b+1}\right\}. \tag*{(191)} α W N W ′ ≤ b + 1 C − c ′ min { 1 , b + 1 W 1/ ( p + 1 ) } . (191)
Since lim t y 1 = + ∞ \lim_t y_1=+\infty lim t y 1 = + ∞ , we can pick a time t 0 t_0 t 0 such that b = e y 1 ≥ B 0 : = 2 C / c ′ b=e^{y_1}\ge B_0:=2C/c' b = e y 1 ≥ B 0 := 2 C / c ′ for all t ≥ t 0 t\ge t_0 t ≥ t 0 . Then, for t ≥ t 0 t\ge t_0 t ≥ t 0 , the right-hand side of Equation (191) is negative whenever W ≥ C W : = ( 2 C / c ′ ) p + 1 W\ge C_W:=(2C/c')^{p+1} W ≥ C W := ( 2 C / c ′ ) p + 1 : if the minimum equals 1 1 1 this follows from b ≥ B 0 b\ge B_0 b ≥ B 0 , and otherwise c ′ W 1 / ( p + 1 ) / ( b + 1 ) ≥ 2 C / ( b + 1 ) c'W^{1/(p+1)}/(b+1)\ge2C/(b+1) c ′ W 1/ ( p + 1 ) / ( b + 1 ) ≥ 2 C / ( b + 1 ) .
We conclude that for t ≥ t 0 t\ge t_0 t ≥ t 0 , in the region u > 0 u>0 u > 0 , W ′ < 0 W'<0 W ′ < 0 whenever W ≥ C W W\ge C_W W ≥ C W ; in particular a trajectory cannot cross the level surface W = C W W=C_W W = C W outward while u > 0 u>0 u > 0 . Region u = 0 u=0 u = 0 . We now analyze the u = 0 u=0 u = 0 region. The gradient is
d d t log W = 2 y 1 ′ − x 1 ′ + p ( x 1 ′ − x 2 ′ ) = 2 y 1 ′ + ( p − 1 ) x 1 ′ − p x 2 ′ (192) \frac{\mathrm{d}}{\mathrm{d}t}\log W = 2y_1' - x_1' + p(x_1' - x_2') = 2y_1' + (p-1)x_1' - px_2' \tag*{(192)} d t d log W = 2 y 1 ′ − x 1 ′ + p ( x 1 ′ − x 2 ′ ) = 2 y 1 ′ + ( p − 1 ) x 1 ′ − p x 2 ′ (192)
for every admissible Filippov boundary velocity. At u = 0 u=0 u = 0 we have
s = 1 , W = Z = e 2 y 1 e x 1 , E = e v + . (193) s=1,\qquad W=Z=\frac{e^{2y_1}}{e^{x_1}},\qquad E=e^{v_+}. \tag*{(193)} s = 1 , W = Z = e x 1 e 2 y 1 , E = e v + . (193)
Since v + v_+ v + is bounded, there are constants 0 < c E < C E < ∞ 0<c_E<C_E<\infty 0 < c E < C E < ∞ such that
c E ≤ E ≤ C E (194) c_E\le E\le C_E \tag*{(194)} c E ≤ E ≤ C E (194)
on the whole boundary u = 0 u=0 u = 0 . Introduce the Filippov parameter 0 ≤ θ ≤ 1 0\le\theta\le1 0 ≤ θ ≤ 1 for the u u u -discontinuity, so that θ \theta θ is the convex value of 1 u ≥ 0 \mathbf{1}_{u\ge0} 1 u ≥ 0 and 1 − θ 1-\theta 1 − θ is the convex value of 1 u < 0 \mathbf{1}_{u<0} 1 u < 0 . As before, 0 ≤ η ≤ 1 0\le\eta\le1 0 ≤ η ≤ 1 denotes the convex value of 1 v ≥ 0 \mathbf{1}_{v\ge0} 1 v ≥ 0 . Then every admissible boundary velocity satisfies
x 1 ′ = α N [ G + ∑ i = 1 K C i + ( 1 − θ ) ∑ i = 1 K A i − θ K Q ] , x 2 ′ = α θ N ( N − 1 ) [ ∑ i = 1 K A i + K Q ] , y 1 ′ = α N − 1 [ K M + R − H − ∑ i = 1 K B i − η ∑ i = 1 K A i + ( 1 − η ) K Q ] . (195) \begin{aligned}
x_1'&=\frac{\alpha}{N}\left[G+\sum_{i=1}^{K}C_i+(1-\theta)\sum_{i=1}^{K}A_i-\theta KQ\right],\\
x_2'&=\frac{\alpha\theta}{N(N-1)}\left[\sum_{i=1}^{K}A_i+KQ\right],\\
y_1'&=\frac{\alpha}{N-1}\left[KM+R-H-\sum_{i=1}^{K}B_i-\eta\sum_{i=1}^{K}A_i+(1-\eta)KQ\right]. \tag*{(195)}
\end{aligned} x 1 ′ x 2 ′ y 1 ′ = N α [ G + i = 1 ∑ K C i + ( 1 − θ ) i = 1 ∑ K A i − θ K Q ] , = N ( N − 1 ) α θ [ i = 1 ∑ K A i + K Q ] , = N − 1 α [ K M + R − H − i = 1 ∑ K B i − η i = 1 ∑ K A i + ( 1 − η ) K Q ] . (195)
Therefore, with ρ = N / ( N − 1 ) \rho=N/(N-1) ρ = N / ( N − 1 ) ,
N α d d t log W = 2 ρ ( K M + R ) + ( ( p − 1 ) G − 2 ρ H ) + ∑ i = 1 K ( ( p − 1 ) C i − 2 ρ B i ) + ( ( p − 1 ) ( 1 − θ ) − 2 ρ η − p θ N − 1 ) ∑ i = 1 K A i + K ( 2 ρ ( 1 − η ) − ( p − 1 ) θ − p θ N − 1 ) Q . (196) \begin{aligned}
\frac{N}{\alpha}\frac{\mathrm{d}}{\mathrm{d}t}\log W
={}&2\rho(KM+R)+\bigl((p-1)G-2\rho H\bigr)+\sum_{i=1}^{K}\bigl((p-1)C_i-2\rho B_i\bigr)\\
&+\left((p-1)(1-\theta)-2\rho\eta-\frac{p\theta}{N-1}\right)\sum_{i=1}^{K}A_i\\
&+K\left(2\rho(1-\eta)-(p-1)\theta-\frac{p\theta}{N-1}\right)Q. \tag*{(196)}
\end{aligned} α N d t d log W = 2 ρ ( K M + R ) + ( ( p − 1 ) G − 2 ρ H ) + i = 1 ∑ K ( ( p − 1 ) C i − 2 ρ B i ) + ( ( p − 1 ) ( 1 − θ ) − 2 ρ η − N − 1 pθ ) i = 1 ∑ K A i + K ( 2 ρ ( 1 − η ) − ( p − 1 ) θ − N − 1 pθ ) Q . (196)
The two denominator terms involving D 1 D_1 D 1 and D 2 , i D_{2,i} D 2 , i have the same favorable structure as in the interior calculation. Namely, with
a = e x 1 , b = e y 1 , δ y = 1 + 2 ρ − p > 0 , (197) a=e^{x_1},\qquad b=e^{y_1},\qquad\delta_y=1+2\rho-p>0, \tag*{(197)} a = e x 1 , b = e y 1 , δ y = 1 + 2 ρ − p > 0 , (197)
we have
( p − 1 ) G − 2 ρ H = ( p − 1 ) ( N − 2 ) − δ y b D 1 ≤ − δ y H + C 1 D 1 , (198) (p-1)G-2\rho H=\frac{(p-1)(N-2)-\delta_y b}{D_1}\le-\delta_y H+C\frac{1}{D_1}, \tag*{(198)} ( p − 1 ) G − 2 ρ H = D 1 ( p − 1 ) ( N − 2 ) − δ y b ≤ − δ y H + C D 1 1 , (198)
and, for every i i i ,
( p − 1 ) C i − 2 ρ B i = ( p − 1 ) ( N − K − 1 ) − δ y b D 2 , i ≤ − δ y B i + C 1 D 2 , i . (199) (p-1)C_i-2\rho B_i=\frac{(p-1)(N-K-1)-\delta_y b}{D_{2,i}}\le-\delta_y B_i+C\frac{1}{D_{2,i}}. \tag*{(199)} ( p − 1 ) C i − 2 ρ B i = D 2 , i ( p − 1 ) ( N − K − 1 ) − δ y b ≤ − δ y B i + C D 2 , i 1 . (199)
The coefficients of the A i A_i A i and Q Q Q terms in Equation (196) are uniformly bounded above, independently of θ \theta θ and η \eta η . Hence Equations (196) , (198) and (199) imply
N α d d t log W ≤ C ( M + R + 1 D 1 + ∑ i = 1 K 1 D 2 , i + ∑ i = 1 K A i + Q ) − c ( H + ∑ i = 1 K B i ) . (200) \frac{N}{\alpha}\frac{\mathrm{d}}{\mathrm{d}t}\log W \le C\left(M+R+\frac{1}{D_1}+\sum_{i=1}^{K}\frac{1}{D_{2,i}}+\sum_{i=1}^{K}A_i+Q\right)-c\left(H+\sum_{i=1}^{K}B_i\right). \tag*{(200)} α N d t d log W ≤ C ( M + R + D 1 1 + i = 1 ∑ K D 2 , i 1 + i = 1 ∑ K A i + Q ) − c ( H + i = 1 ∑ K B i ) . (200)
This estimate is valid for every Filippov choice of θ \theta θ and η \eta η on u = 0 u=0 u = 0 .
We now show that the negative term in Equation (200) dominates whenever W = Z W=Z W = Z is large. Since y 1 ( t ) → ∞ y_1(t)\to\infty y 1 ( t ) → ∞ , choose t 0 t_0 t 0 so large that
b = e y 1 ( t ) ≥ B 0 (201) b=e^{y_1(t)}\ge B_0 \tag*{(201)} b = e y 1 ( t ) ≥ B 0 (201)
for all t ≥ t 0 t\ge t_0 t ≥ t 0 , where B 0 B_0 B 0 will be fixed below. On u = 0 u=0 u = 0 and for t ≥ t 0 t\ge t_0 t ≥ t 0 , E E E is bounded and b b b is large, so
M ≤ C b , R ≤ C b , Q ≤ C b . (202) M\le\frac{C}{b},\qquad R\le\frac{C}{b},\qquad Q\le\frac{C}{b}. \tag*{(202)} M ≤ b C , R ≤ b C , Q ≤ b C . (202)
Moreover, since every D 2 , i D_{2,i} D 2 , i contains the term ( K − i + 1 ) a (K-i+1)a ( K − i + 1 ) a and K − i + 1 ≥ 1 K-i+1\ge1 K − i + 1 ≥ 1 , while E E E is bounded,
∑ i = 1 K A i ≤ C a + b + 1 . (203) \sum_{i=1}^{K}A_i\le\frac{C}{a+b+1}. \tag*{(203)} i = 1 ∑ K A i ≤ a + b + 1 C . (203)
and
1 D 1 + ∑ i = 1 K 1 D 2 , i ≤ C a + b + 1 . (204) \frac{1}{D_1}+\sum_{i=1}^{K}\frac{1}{D_{2,i}}\le\frac{C}{a+b+1}. \tag*{(204)} D 1 1 + i = 1 ∑ K D 2 , i 1 ≤ a + b + 1 C . (204)
Finally,
H + ∑ i = 1 K B i ≥ c b a + b + 1 . (205) H+\sum_{i=1}^{K}B_i\ge c\frac{b}{a+b+1}. \tag*{(205)} H + i = 1 ∑ K B i ≥ c a + b + 1 b . (205)
Combining Equation (200) –Equation (205) , every admissible boundary velocity satisfies
N α d d t log W ≤ C b + C a + b + 1 − c b a + b + 1 . (206) \frac{N}{\alpha}\frac{\mathrm{d}}{\mathrm{d}t}\log W\le\frac{C}{b}+\frac{C}{a+b+1}-c\frac{b}{a+b+1}. \tag*{(206)} α N d t d log W ≤ b C + a + b + 1 C − c a + b + 1 b . (206)
It remains only to check the two possible relative sizes of a a a and b b b .
Boundary case 1: a ≤ L ( b + 1 ) a\le L(b+1) a ≤ L ( b + 1 ) . Choose L > 1 L>1 L > 1 fixed. Then
b a + b + 1 ≥ c L > 0. (207) \frac{b}{a+b+1}\ge c_L>0. \tag*{(207)} a + b + 1 b ≥ c L > 0. (207)
Thus the negative term in Equation (206) is bounded below by a positive constant, whereas the positive terms are O ( b − 1 ) O(b^{-1}) O ( b − 1 ) . After increasing B 0 B_0 B 0 if necessary, Equation (206) gives
d d t log W < 0 (208) \frac{\mathrm{d}}{\mathrm{d}t}\log W<0 \tag*{(208)} d t d log W < 0 (208)
throughout this boundary subcase.
Boundary case 2: a > L ( b + 1 ) a > L(b + 1) a > L ( b + 1 ) . Here
b a + b + 1 ≲ b a . (209) \frac{b}{a+b+1} \lesssim\frac{b}{a}. \tag*{(209)} a + b + 1 b ≲ a b . (209)
Since W = Z = b 2 / a W = Z = b^2/a W = Z = b 2 / a on u = 0 u = 0 u = 0 , this becomes
b a = W b . (210) \frac{b}{a} = \frac{W}{b}. \tag*{(210)} a b = b W . (210)
Also ( a + b + 1 ) − 1 ≤ C / a ≤ C / b (a+b+1)^{-1} \le C/a \le C/b ( a + b + 1 ) − 1 ≤ C / a ≤ C / b . Hence Equation (206) yields
N α d d t log W ≤ C b − W b . (211) \frac{N}{\alpha}\frac{\mathrm{d}}{\mathrm{d}t}\log W \le\frac{C}{b}-\frac{W}{b}. \tag*{(211)} α N d t d log W ≤ b C − b W . (211)
Therefore the boundary derivative is strictly negative in this subcase whenever W ≥ C ∂ W \ge C_{\partial} W ≥ C ∂ , where C ∂ C_{\partial} C ∂ depends only on the fixed parameters and on the bound for v + v_{+} v + .
The two boundary cases show that, for t ≥ t 0 t \ge t_0 t ≥ t 0 , no Filippov velocity at u = 0 u = 0 u = 0 can point outward through a sufficiently large level surface of W W W . More precisely, after choosing C ∂ C_{\partial} C ∂ and B 0 B_0 B 0 large enough,
W ≥ C ∂ ⟹ d d t log W < 0 (212) W \ge C_{\partial} \quad\Longrightarrow\quad\frac{\mathrm{d}}{\mathrm{d}t}\log W < 0 \tag*{(212)} W ≥ C ∂ ⟹ d t d log W < 0 (212)
for every admissible boundary velocity. This is the required boundary version of the Lyapunov estimate.
Combining the interior and boundary estimates: for a.e. t ≥ t 0 t \ge t_0 t ≥ t 0 , W ( t ) ≥ C ∗ : = max { C W , C ∂ } W(t) \ge C_* := \max\{C_W,C_{\partial}\} W ( t ) ≥ C ∗ := max { C W , C ∂ } implies ( log W ) ′ ( t ) < 0 (\log W)'(t) < 0 ( log W ) ′ ( t ) < 0 , whether u ( t ) > 0 u(t) > 0 u ( t ) > 0 or u ( t ) = 0 u(t) = 0 u ( t ) = 0 . Since W W W is absolutely continuous (the vector field is bounded, so x i , y i x_i,y_i x i , y i are Lipschitz), the one-sided barrier argument gives W ( t ) ≤ max { W ( t 0 ) , C ∗ } W(t) \le\max\{W(t_0),C_*\} W ( t ) ≤ max { W ( t 0 ) , C ∗ } for all t ≥ t 0 t \ge t_0 t ≥ t 0 , and W W W is bounded on the compact interval [ 0 , t 0 ] [0,t_0] [ 0 , t 0 ] by continuity. Hence
W ( t ) = Z ( t ) s ( t ) p ≤ C ∀ t ≥ 0. (213) W(t) = Z(t)s(t)^p \le C \qquad\forall t \ge0. \tag*{(213)} W ( t ) = Z ( t ) s ( t ) p ≤ C ∀ t ≥ 0. (213)
Since s ≥ 1 s \ge1 s ≥ 1 , we immediately obtain
Z ( t ) ≤ C . (214) Z(t) \le C. \tag*{(214)} Z ( t ) ≤ C . (214)
Equivalently,
2 y 1 ( t ) − x 1 ( t ) ≤ C Z ∀ t ≥ 0. (215) 2y_1(t)-x_1(t) \le C_Z \qquad\forall t \ge0. \tag*{(215)} 2 y 1 ( t ) − x 1 ( t ) ≤ C Z ∀ t ≥ 0. (215)
Upper bound of x 1 − x 2 x_1-x_2 x 1 − x 2 . We now prove that the upper barrier for
s = e u = e x 1 − x 2 (216) s = e^u = e^{x_1-x_2} \tag*{(216)} s = e u = e x 1 − x 2 (216)
is finite. We already know u ≥ 0 u \ge0 u ≥ 0 , hence s ≥ 1 s \ge1 s ≥ 1 . It remains to exclude escape of s s s to infinity. It suffices to consider the region u > 0 u > 0 u > 0 . There E = e u + v + = s e v + E = e^{u+v_+} = se^{v_+} E = e u + v + = s e v + , and the formulae for x 1 ′ x_1' x 1 ′ and x 2 ′ x_2' x 2 ′ give
u ′ = x 1 ′ − x 2 ′ = α N [ G + ∑ i = 1 K C i − 1 N − 1 ∑ i = 1 K A i − K N N − 1 Q ] ≤ α N [ G + ∑ i = 1 K C i − Q ] . (217) \begin{aligned}
u' &= x_1' - x_2' \\
&= \frac{\alpha}{N}\left[G+\sum_{i=1}^{K}C_i-\frac{1}{N-1}\sum_{i=1}^{K}A_i-\frac{K N}{N-1}Q\right]
\le\frac{\alpha}{N}\left[G+\sum_{i=1}^{K}C_i-Q\right]. \tag*{(217)}
\end{aligned} u ′ = x 1 ′ − x 2 ′ = N α [ G + i = 1 ∑ K C i − N − 1 1 i = 1 ∑ K A i − N − 1 K N Q ] ≤ N α [ G + i = 1 ∑ K C i − Q ] . (217)
where we dropped A i ≥ 0 A_i \ge0 A i ≥ 0 and used K N / ( N − 1 ) ≥ 1 KN/(N-1) \ge1 K N / ( N − 1 ) ≥ 1 . Since y 1 ≥ 0 y_1 \ge0 y 1 ≥ 0 and x 2 ≥ 0 x_2 \ge0 x 2 ≥ 0 , we have b ≥ 1 b \ge1 b ≥ 1 and s ≤ a s \le a s ≤ a , so for the positive terms G G G and C i C_i C i ,
G ≤ C b + 1 a + b + 1 ≤ C b a , C i ≤ C b + 1 a + b + 1 ≤ C b a . (218) G \le C\frac{b+1}{a+b+1} \le C\frac{b}{a}, \qquad C_i \le C\frac{b+1}{a+b+1} \le C\frac{b}{a}. \tag*{(218)} G ≤ C a + b + 1 b + 1 ≤ C a b , C i ≤ C a + b + 1 b + 1 ≤ C a b . (218)
For Q Q Q , since E ≤ e C v s E \le e^{C_v}s E ≤ e C v s we have the lower bound
Q ≥ C ′ s s + b + 1 . (219) Q \ge C'\frac{s}{s+b+1}. \tag*{(219)} Q ≥ C ′ s + b + 1 s . (219)
Therefore u ′ < 0 u' < 0 u ′ < 0 as soon as
( K + 1 ) C b a < C ′ s s + b + 1 , i.e. ( K + 1 ) C ( 1 + e y 1 − u + e − u ) < C ′ e x 1 − y 1 . (220) (K+1)C\frac{b}{a} < C'\frac{s}{s+b+1}, \qquad\text{i.e.} \qquad(K+1)C(1+e^{y_1-u}+e^{-u}) < C'e^{x_1-y_1}. \tag*{(220)} ( K + 1 ) C a b < C ′ s + b + 1 s , i.e. ( K + 1 ) C ( 1 + e y 1 − u + e − u ) < C ′ e x 1 − y 1 . (220)
We claim that Equation (220) holds whenever
u ≥ 2 y 1 − x 1 + C 2 , C 2 : = 2 log ( 3 ( K + 1 ) C / C ′ ) . (221) u \ge2y_1-x_1+C_2, \qquad C_2 := 2\log(3(K+1)C/C'). \tag*{(221)} u ≥ 2 y 1 − x 1 + C 2 , C 2 := 2 log ( 3 ( K + 1 ) C / C ′ ) . (221)
Indeed, since x 2 ≥ 0 x_2 \ge0 x 2 ≥ 0 we have x 1 ≥ u x_1 \ge u x 1 ≥ u , so Equation (221) implies x 1 ≥ 2 y 1 − x 1 + C 2 x_1 \ge2y_1-x_1+C_2 x 1 ≥ 2 y 1 − x 1 + C 2 , i.e. x 1 − y 1 ≥ C 2 / 2 x_1-y_1 \ge C_2/2 x 1 − y 1 ≥ C 2 /2 ; hence 1 + e − u ≤ 2 ≤ 2 e − C 2 / 2 e x 1 − y 1 1+e^{-u} \le2 \le2e^{-C_2/2}e^{x_1-y_1} 1 + e − u ≤ 2 ≤ 2 e − C 2 /2 e x 1 − y 1 and e y 1 − u ≤ e − C 2 e x 1 − y 1 ≤ e − C 2 / 2 e x 1 − y 1 e^{y_1-u} \le e^{-C_2}e^{x_1-y_1} \le e^{-C_2/2}e^{x_1-y_1} e y 1 − u ≤ e − C 2 e x 1 − y 1 ≤ e − C 2 /2 e x 1 − y 1 , so the left-hand side of Equation (220) is at most 3 ( K + 1 ) C e − C 2 / 2 e x 1 − y 1 = C ′ e x 1 − y 1 3(K+1)Ce^{-C_2/2}e^{x_1-y_1}=C'e^{x_1-y_1} 3 ( K + 1 ) C e − C 2 /2 e x 1 − y 1 = C ′ e x 1 − y 1 , with strict inequality after enlarging C 2 C_2 C 2 slightly. (Note that the two extra terms 1 + e − u 1+e^{-u} 1 + e − u cannot be dropped: without the input x 1 ≥ u x_1 \ge u x 1 ≥ u the implication fails.)
Therefore, we argue that
u ( t ) ≤ U ( t ) : = max { u ( 0 ) , sup 0 ≤ τ ≤ t ( 2 y 1 ( τ ) − x 1 ( τ ) ) + C 2 , ϵ } . (222) u(t) \le U(t) := \max\left\{u(0),\ \sup_{0\le\tau\le t}(2y_1(\tau)-x_1(\tau))+C_2,\ \epsilon\right\}. \tag*{(222)} u ( t ) ≤ U ( t ) := max { u ( 0 ) , 0 ≤ τ ≤ t sup ( 2 y 1 ( τ ) − x 1 ( τ )) + C 2 , ϵ } . (222)
where ϵ \epsilon ϵ is a small positive constant. To prove this, suppose the opposite and consider the first violation time
t 0 = inf { t : u ( t ) > U ( t ) } . (223) t_0=\inf\{t:u(t)>U(t)\}. \tag*{(223)} t 0 = inf { t : u ( t ) > U ( t )} . (223)
Since u u u and U U U are continuous, u ( t 0 ) = U ( t 0 ) u(t_0)=U(t_0) u ( t 0 ) = U ( t 0 ) , and U U U is non-decreasing. Since u ( t 0 ) ≥ ϵ > 0 u(t_0)\ge\epsilon>0 u ( t 0 ) ≥ ϵ > 0 , the state lies in the open region { u > 0 } \{u>0\} { u > 0 } , where u u u is continuously differentiable (the x x x -equations contain no v v v -indicator and E = e u + v + E=e^{u+v_+} E = e u + v + is continuous), and u ( t 0 ) ≥ 2 y 1 ( t 0 ) − x 1 ( t 0 ) + C 2 u(t_0)\ge2y_1(t_0)-x_1(t_0)+C_2 u ( t 0 ) ≥ 2 y 1 ( t 0 ) − x 1 ( t 0 ) + C 2 gives u ′ ( t 0 ) < 0 u'(t_0)<0 u ′ ( t 0 ) < 0 by Equations (220) and (221) . Hence u ( t ) < u ( t 0 ) = U ( t 0 ) ≤ U ( t ) u(t)<u(t_0)=U(t_0)\le U(t) u ( t ) < u ( t 0 ) = U ( t 0 ) ≤ U ( t ) on a right neighborhood of t 0 t_0 t 0 , contradicting the definition of t 0 t_0 t 0 .
Further, since 2 y 1 − x 1 ≤ C Z 2y_1-x_1\le C_Z 2 y 1 − x 1 ≤ C Z for all t ≥ 0 t\ge0 t ≥ 0 , we conclude that u u u is also upper-bounded, i.e.,
0 ≤ x 1 − x 2 ≤ C u : = max { u ( 0 ) , C Z + C 2 , ϵ } , ∀ t ≥ 0. (224) 0\le x_1-x_2\le C_u:=\max\{u(0),C_Z+C_2,\epsilon\},\qquad\forall t\ge0. \tag*{(224)} 0 ≤ x 1 − x 2 ≤ C u := max { u ( 0 ) , C Z + C 2 , ϵ } , ∀ t ≥ 0. (224)
Logarithm growth rate The averaged x x x -variable for the system is
X = x 1 + ( N − 1 ) x 2 N , Y = y 1 . (225) X=\frac{x_1+(N-1)x_2}{N}, \qquad Y=y_1. \tag*{(225)} X = N x 1 + ( N − 1 ) x 2 , Y = y 1 . (225)
Since 0 ≤ u = x 1 − x 2 ≤ C u 0\le u=x_1-x_2\le C_u 0 ≤ u = x 1 − x 2 ≤ C u ,
X = x 1 + O ( 1 ) = x 2 + O ( 1 ) , more precisely X = x 1 − N − 1 N u . (226) X=x_1+O(1)=x_2+O(1), \qquad\text{more precisely}\qquad X=x_1-\frac{N-1}{N}u. \tag*{(226)} X = x 1 + O ( 1 ) = x 2 + O ( 1 ) , more precisely X = x 1 − N N − 1 u . (226)
Also u u u and v + v_+ v + are bounded, so
1 ≤ E = e u + v + ≤ C E . (227) 1 \le E=e^{u+v_+}\le C_E. \tag*{(227)} 1 ≤ E = e u + v + ≤ C E . (227)
The estimate Z = e 2 Y − x 1 ≤ C Z=e^{2Y-x_1}\le C Z = e 2 Y − x 1 ≤ C gives
e 2 Y ≤ C e x 1 ≍ C e X , (228) e^{2Y}\le Ce^{x_1}\asymp Ce^X, \tag*{(228)} e 2 Y ≤ C e x 1 ≍ C e X , (228)
and in particular, since Y → ∞ Y\to\infty Y → ∞ ,
e y 1 − x 1 = Z e − y 1 ≤ C e − Y → 0 , e − X ≤ e − N − 1 N C u e − x 1 ≤ C e − 2 Y ≤ C e − Y . (229) e^{y_1-x_1}=Ze^{-y_1}\le Ce^{-Y}\to0,\qquad e^{-X}\le e^{-\frac{N-1}{N}C_u}e^{-x_1}\le Ce^{-2Y}\le Ce^{-Y}. \tag*{(229)} e y 1 − x 1 = Z e − y 1 ≤ C e − Y → 0 , e − X ≤ e − N N − 1 C u e − x 1 ≤ C e − 2 Y ≤ C e − Y . (229)
Consequently D 1 = e x 1 ( 1 + o ( 1 ) ) D_1=e^{x_1}(1+o(1)) D 1 = e x 1 ( 1 + o ( 1 )) and D 2 , i = ( K − i + 1 ) e x 1 ( 1 + o ( 1 ) ) D_{2,i}=(K-i+1)e^{x_1}(1+o(1)) D 2 , i = ( K − i + 1 ) e x 1 ( 1 + o ( 1 )) ; this is where the bound 2 y 1 − x 1 ≤ C 2y_1-x_1\le C 2 y 1 − x 1 ≤ C enters.
Writing θ ∈ [ 0 , 1 ] \theta\in[0,1] θ ∈ [ 0 , 1 ] for the Filippov weight of 1 u ≥ 0 \mathbf{1}_{u\ge0} 1 u ≥ 0 (so θ = 1 \theta=1 θ = 1 on { u > 0 } \{u>0\} { u > 0 } ),
( x 1 + ( N − 1 ) x 2 ) ′ = α N [ G + ∑ i = 1 K C i + ( 1 − θ ) ∑ i = 1 K A i − θ K Q + θ ∑ i = 1 K A i + θ K Q ] = α N [ G + ∑ i = 1 K ( C i + A i ) ] , (230) \begin{aligned}
(x_1+(N-1)x_2)'&=\frac{\alpha}{N}\left[G+\sum_{i=1}^{K}C_i+(1-\theta)\sum_{i=1}^{K}A_i-\theta KQ+\theta\sum_{i=1}^{K}A_i+\theta KQ\right]\\
&=\frac{\alpha}{N}\left[G+\sum_{i=1}^{K}(C_i+A_i)\right], \tag*{(230)}
\end{aligned} ( x 1 + ( N − 1 ) x 2 ) ′ = N α [ G + i = 1 ∑ K C i + ( 1 − θ ) i = 1 ∑ K A i − θ K Q + θ i = 1 ∑ K A i + θ K Q ] = N α [ G + i = 1 ∑ K ( C i + A i ) ] , (230)
independently of θ \theta θ ; the same identity therefore holds for the Filippov boundary field at u = 0 u=0 u = 0 . Hence
X ′ = α N 2 [ G + ∑ i = 1 K ( C i + A i ) ] . (231) X'=\frac{\alpha}{N^2}\left[G+\sum_{i=1}^{K}(C_i+A_i)\right]. \tag*{(231)} X ′ = N 2 α [ G + i = 1 ∑ K ( C i + A i ) ] . (231)
Because E ≤ C E E\le C_E E ≤ C E and D 2 , i ≥ e x 1 D_{2,i}\ge e^{x_1} D 2 , i ≥ e x 1 , the A i A_i A i terms are O ( e − x 1 ) = O ( e − X ) O(e^{-x_1})=O(e^{-X}) O ( e − x 1 ) = O ( e − X ) . Moreover
G + ∑ i = 1 K C i = e y 1 ( 1 D 1 + ∑ i = 1 K 1 D 2 , i ) + O ( e − x 1 ) = a 0 ( t ) e Y − X + O ( e − X ) , (232) G+\sum_{i=1}^{K}C_i=e^{y_1}\left(\frac{1}{D_1}+\sum_{i=1}^{K}\frac{1}{D_{2,i}}\right)+O(e^{-x_1})=a_0(t)e^{Y-X}+O(e^{-X}), \tag*{(232)} G + i = 1 ∑ K C i = e y 1 ( D 1 1 + i = 1 ∑ K D 2 , i 1 ) + O ( e − x 1 ) = a 0 ( t ) e Y − X + O ( e − X ) , (232)
where, by (229) , a 0 ( t ) = e X − x 1 e x 1 ( 1 / D 1 + ∑ i 1 / D 2 , i ) a_0(t)=e^{X-x_1}e^{x_1}(1/D_1+\sum_i1/D_{2,i}) a 0 ( t ) = e X − x 1 e x 1 ( 1/ D 1 + ∑ i 1/ D 2 , i ) is bounded above and below by positive constants once Y Y Y is large: e X − x 1 = e u ∈ [ e − C u , 1 ] e^{X-x_1}=e^u\in[e^{-C_u},1] e X − x 1 = e u ∈ [ e − C u , 1 ] and e x 1 ( 1 / D 1 + ∑ i 1 / D 2 , i ) → 1 + ∑ i = 1 K 1 K − i + 1 e^{x_1}(1/D_1+\sum_i1/D_{2,i})\to1+\sum_{i=1}^{K}\frac{1}{K-i+1} e x 1 ( 1/ D 1 + ∑ i 1/ D 2 , i ) → 1 + ∑ i = 1 K K − i + 1 1 . The O ( e − X ) O(e^{-X}) O ( e − X ) term is absorbed into the coefficient because e − X ≤ e Y − X e^{-X}\le e^{Y-X} e − X ≤ e Y − X for Y ≥ 0 Y\ge0 Y ≥ 0 . Thus, for all large t t t ,
X ′ = a ( t ) e Y − X , (233) X'=a(t)e^{Y-X}, \tag*{(233)} X ′ = a ( t ) e Y − X , (233)
with
0 < a − ≤ a ( t ) ≤ a + < ∞ . (234) 0<a_-\le a(t)\le a_+<\infty. \tag*{(234)} 0 < a − ≤ a ( t ) ≤ a + < ∞. (234)
Next, writing η ∈ [ 0 , 1 ] \eta\in[0,1] η ∈ [ 0 , 1 ] for the Filippov weight of 1 v ≥ 0 \mathbf{1}_{v\ge0} 1 v ≥ 0 (so η = 1 \eta=1 η = 1 on { v > 0 } \{v>0\} { v > 0 } and η = 0 \eta=0 η = 0 on { v < 0 } \{v<0\} { v < 0 } ), the Y = y 1 Y=y_1 Y = y 1 equation is
Y ′ = α N − 1 [ K M + R − H − ∑ i = 1 K B i − η ∑ i = 1 K A i + ( 1 − η ) K Q ] . (235) Y'=\frac{\alpha}{N-1}\left[KM+R-H-\sum_{i=1}^{K}B_i-\eta\sum_{i=1}^{K}A_i+(1-\eta)KQ\right]. \tag*{(235)} Y ′ = N − 1 α [ K M + R − H − i = 1 ∑ K B i − η i = 1 ∑ K A i + ( 1 − η ) K Q ] . (235)
Here
K M = K N − 1 e Y + N − 1 = : b 0 ( t ) e − Y , (236) KM = K\frac{N-1}{e^{Y}+N-1}=:b_0(t)e^{-Y}, \tag*{(236)} K M = K e Y + N − 1 N − 1 =: b 0 ( t ) e − Y , (236)
with b 0 ( t ) ∈ [ K ( N − 1 ) / 2 , K ( N − 1 ) ] b_0(t)\in[K(N-1)/2,K(N-1)] b 0 ( t ) ∈ [ K ( N − 1 ) /2 , K ( N − 1 )] once e Y ≥ N − 1 e^{Y}\ge N-1 e Y ≥ N − 1 . Also
H + ∑ i = 1 K B i = e Y ( 1 D 1 + ∑ i = 1 K 1 D 2 , i ) = a ~ 0 ( t ) e Y − X , (237) H+\sum_{i=1}^{K}B_i=e^{Y}\left(\frac{1}{D_1}+\sum_{i=1}^{K}\frac{1}{D_{2,i}}\right)=\widetilde{a}_0(t)e^{Y-X}, \tag*{(237)} H + i = 1 ∑ K B i = e Y ( D 1 1 + i = 1 ∑ K D 2 , i 1 ) = a 0 ( t ) e Y − X , (237)
where a ~ 0 ( t ) = e X ( 1 / D 1 + ∑ i 1 / D 2 , i ) \widetilde{a}_0(t)=e^{X}(1/D_1+\sum_i1/D_{2,i}) a 0 ( t ) = e X ( 1/ D 1 + ∑ i 1/ D 2 , i ) is bounded above and below by positive constants for large t t t , by the same computation as for a 0 a_0 a 0 . The remaining terms satisfy
0 ≤ R ≤ ( N − K − 1 ) e − Y , 0 ≤ ( 1 − η ) K Q ≤ K C E e − Y , (238) 0\le R\le(N-K-1)e^{-Y},\qquad0\le(1-\eta)KQ\le KCE e^{-Y}, \tag*{(238)} 0 ≤ R ≤ ( N − K − 1 ) e − Y , 0 ≤ ( 1 − η ) K Q ≤ K C E e − Y , (238)
0 ≤ η ∑ i A i ≤ C e − x 1 ≤ C e − 2 Y , (239) 0\le\eta\sum_i A_i\le Ce^{-x_1}\le Ce^{-2Y}, \tag*{(239)} 0 ≤ η i ∑ A i ≤ C e − x 1 ≤ C e − 2 Y , (239)
using D 4 ≥ e Y D_4\ge e^{Y} D 4 ≥ e Y , E ≤ C E E\le C_E E ≤ C E , D 2 , i ≥ e x 1 D_{2,i}\ge e^{x_1} D 2 , i ≥ e x 1 and Equation (229) . Therefore
Y ′ = b ( t ) e − Y − a ~ ( t ) e Y − X + r ( t ) , r ( t ) : = α N − 1 [ R + ( 1 − η ) K Q − η ∑ i A i ] , (240) Y'=b(t)e^{-Y}-\widetilde{a}(t)e^{Y-X}+r(t),\qquad r(t):=\frac{\alpha}{N-1}\left[R+(1-\eta)KQ-\eta\sum_i A_i\right], \tag*{(240)} Y ′ = b ( t ) e − Y − a ( t ) e Y − X + r ( t ) , r ( t ) := N − 1 α [ R + ( 1 − η ) K Q − η i ∑ A i ] , (240)
where b ( t ) = α N − 1 b 0 ( t ) b(t)=\frac{\alpha}{N-1}b_0(t) b ( t ) = N − 1 α b 0 ( t ) , a ~ ( t ) = α N − 1 a ~ 0 ( t ) \widetilde{a}(t)=\frac{\alpha}{N-1}\widetilde{a}_0(t) a ( t ) = N − 1 α a 0 ( t ) satisfy
0 < b − ≤ b ( t ) ≤ b + < ∞ , 0 < a ~ − ≤ a ~ ( t ) ≤ a ~ + < ∞ , (241) 0<b_-\le b(t)\le b_+<\infty,\qquad0<\widetilde{a}_-\le\widetilde{a}(t)\le\widetilde{a}_+<\infty, \tag*{(241)} 0 < b − ≤ b ( t ) ≤ b + < ∞ , 0 < a − ≤ a ( t ) ≤ a + < ∞ , (241)
and
∣ r ( t ) ∣ ≤ C e − Y . (242) |r(t)|\le Ce^{-Y}. \tag*{(242)} ∣ r ( t ) ∣ ≤ C e − Y . (242)
We now check that the effective input β ( t ) = b ( t ) + e Y r ( t ) \beta(t)=b(t)+e^{Y}r(t) β ( t ) = b ( t ) + e Y r ( t ) is eventually bounded above and below by positive constants. We have
e Y r = α N − 1 [ ( N − K − 1 ) e Y D 4 + ( 1 − η ) K E e Y D 4 − η e Y ∑ i A i ] . e^{Y}r=\frac{\alpha}{N-1}\left[\frac{(N-K-1)e^{Y}}{D_4}+(1-\eta)\frac{KEe^{Y}}{D_4}-\eta e^{Y}\sum_i A_i\right]. e Y r = N − 1 α [ D 4 ( N − K − 1 ) e Y + ( 1 − η ) D 4 K E e Y − η e Y i ∑ A i ] .
The first two terms lie in [ 0 , N − K − 1 + K C E ] [0,N-K-1+KC_E] [ 0 , N − K − 1 + K C E ] , since D 4 ≥ e Y D_4\ge e^{Y} D 4 ≥ e Y and 1 ≤ E ≤ C E 1\le E\le C_E 1 ≤ E ≤ C E ; they are of the same order as b b b , but they are nonnegative. For the last term, e Y ∑ i A i ≤ C e Y − x 1 = C Z e − Y ≤ C e − Y → 0 e^{Y}\sum_i A_i\le Ce^{Y-x_1}=CZe^{-Y}\le Ce^{-Y}\to0 e Y ∑ i A i ≤ C e Y − x 1 = C Z e − Y ≤ C e − Y → 0 by the bound Z ≤ C Z\le C Z ≤ C . Since b ( t ) ≥ α N − 1 K ( N − 1 ) 2 = α K 2 b(t)\ge\frac{\alpha}{N-1}\frac{K(N-1)}{2}=\frac{\alpha K}{2} b ( t ) ≥ N − 1 α 2 K ( N − 1 ) = 2 α K once e Y ≥ N − 1 e^{Y}\ge N-1 e Y ≥ N − 1 , we obtain
α K 4 ≤ β ( t ) ≤ α K + α N − 1 ( N − K − 1 + K C E ) (243) \frac{\alpha K}{4}\le\beta(t)\le\alpha K+\frac{\alpha}{N-1}(N-K-1+KC_E) \tag*{(243)} 4 α K ≤ β ( t ) ≤ α K + N − 1 α ( N − K − 1 + K C E ) (243)
for all t ≥ t 0 t\ge t_0 t ≥ t 0 , where t 0 t_0 t 0 is chosen, using Y → ∞ Y\to\infty Y → ∞ , so that all the above estimates hold for t ≥ t 0 t\ge t_0 t ≥ t 0 .
All hypotheses of Theorem C.4 are therefore satisfied on [ t 0 , ∞ ) [t_0,\infty) [ t 0 , ∞ ) (the equations hold a.e. and for every Filippov selection, since all bounds are uniform in θ , η ∈ [ 0 , 1 ] \theta,\eta\in[0,1] θ , η ∈ [ 0 , 1 ] ). Applying the lemma to X ~ ( t ) = X ( t + t 0 ) \widetilde{X}(t)=X(t+t_0) X ( t ) = X ( t + t 0 ) , Y ~ ( t ) = Y ( t + t 0 ) \widetilde{Y}(t)=Y(t+t_0) Y ( t ) = Y ( t + t 0 ) and using log ( t + t 0 ) = log t + O ( 1 ) \log(t+t_0)=\log t+O(1) log ( t + t 0 ) = log t + O ( 1 ) for t ≥ 1 t\ge1 t ≥ 1 gives
X ( t ) = 2 log t + O ( 1 ) , Y ( t ) = log t + O ( 1 ) . (244) X(t)=2\log t+O(1),\qquad Y(t)=\log t+O(1). \tag*{(244)} X ( t ) = 2 log t + O ( 1 ) , Y ( t ) = log t + O ( 1 ) . (244)
Since X = x 1 + O ( 1 ) = x 2 + O ( 1 ) X=x_1+O(1)=x_2+O(1) X = x 1 + O ( 1 ) = x 2 + O ( 1 ) and Y = y 1 Y=y_1 Y = y 1 , we obtain
x 1 ( t ) , x 2 ( t ) = 2 log t + O ( 1 ) , y 1 ( t ) = log t + O ( 1 ) . (245) x_1(t),x_2(t)=2\log t+O(1),\qquad y_1(t)=\log t+O(1). \tag*{(245)} x 1 ( t ) , x 2 ( t ) = 2 log t + O ( 1 ) , y 1 ( t ) = log t + O ( 1 ) . (245)
Concerning the dependence on α \alpha α : if ( x 1 , x 2 , y 1 , y 2 ) (x_1,x_2,y_1,y_2) ( x 1 , x 2 , y 1 , y 2 ) solves the system with learning rate α \alpha α , then t ↦ ( x 1 , x 2 , y 1 , y 2 ) ( t / α ) t \mapsto(x_1,x_2,y_1,y_2)(t/\alpha) t ↦ ( x 1 , x 2 , y 1 , y 2 ) ( t / α ) solves the system with α = 1 \alpha=1 α = 1 from the same initial state. Hence ( x 1 , x 2 , y 1 , y 2 ) ( t ) = ( x ^ 1 , x ^ 2 , y ^ 1 , y ^ 2 ) ( α t ) (x_1,x_2,y_1,y_2)(t)=(\hat{x}_1,\hat{x}_2,\hat{y}_1,\hat{y}_2)(\alpha t) ( x 1 , x 2 , y 1 , y 2 ) ( t ) = ( x ^ 1 , x ^ 2 , y ^ 1 , y ^ 2 ) ( α t ) , where the hats denote the solution for α = 1 \alpha=1 α = 1 , and all the O ( 1 ) O(1) O ( 1 ) terms above can be taken independent of α \alpha α when the growth is expressed in terms of α t \alpha t α t :
x 1 ( t ) , x 2 ( t ) = 2 log ( α t ) + O ( 1 ) , y 1 ( t ) = log ( α t ) + O ( 1 ) . (246) x_1(t),x_2(t)=2\log(\alpha t)+O(1), \qquad y_1(t)=\log(\alpha t)+O(1). \tag*{(246)} x 1 ( t ) , x 2 ( t ) = 2 log ( α t ) + O ( 1 ) , y 1 ( t ) = log ( α t ) + O ( 1 ) . (246)
Since y 1 − y 2 ≤ C y_1-y_2\le C y 1 − y 2 ≤ C , hence
y 2 ( t ) ≥ y 1 ( t ) − C = log ( α t ) + O ( 1 ) . (247) y_2(t)\ge y_1(t)-C=\log(\alpha t)+O(1). \tag*{(247)} y 2 ( t ) ≥ y 1 ( t ) − C = log ( α t ) + O ( 1 ) . (247)
For the matching upper bound on y 2 y_2 y 2 , we use an elementary barrier. Let
m ( t ) = max 0 ≤ r ≤ t y 1 ( r ) , h ( t ) = ( y 2 ( t ) − m ( t ) ) + . (248) m(t)=\max_{0\le r\le t}y_1(r), \qquad h(t)=(y_2(t)-m(t))_+. \tag*{(248)} m ( t ) = 0 ≤ r ≤ t max y 1 ( r ) , h ( t ) = ( y 2 ( t ) − m ( t ) ) + . (248)
Both y 2 y_2 y 2 and m m m are Lipschitz (the vector field is bounded), hence so is h h h . At almost every time at which h ( t ) > 0 h(t)>0 h ( t ) > 0 , we have y 2 ( t ) > m ( t ) ≥ y 1 ( t ) y_2(t)>m(t)\ge y_1(t) y 2 ( t ) > m ( t ) ≥ y 1 ( t ) , hence v < 0 v<0 v < 0 strictly, so the state is in the open region { v < 0 } \{v<0\} { v < 0 } where y ˙ 2 = 0 \dot{y}_2=0 y ˙ 2 = 0 for every Filippov selection, while m ′ ( t ) ≥ 0 m'(t)\ge0 m ′ ( t ) ≥ 0 . Therefore h ′ ( t ) ≤ 0 h'(t)\le0 h ′ ( t ) ≤ 0 for almost every such time, and consequently h h h is non-increasing (if h ( t 2 ) > h ( t 1 ) h(t_2)>h(t_1) h ( t 2 ) > h ( t 1 ) for some t 1 < t 2 t_1<t_2 t 1 < t 2 , let t ∗ = sup { t ∈ [ t 1 , t 2 ] : h ( t ) ≤ h ( t 1 ) } t_*=\sup\{t\in[t_1,t_2]:h(t)\le h(t_1)\} t ∗ = sup { t ∈ [ t 1 , t 2 ] : h ( t ) ≤ h ( t 1 )} ; then h > h ( t 1 ) ≥ 0 h>h(t_1)\ge0 h > h ( t 1 ) ≥ 0 on ( t ∗ , t 2 ] (t_*,t_2] ( t ∗ , t 2 ] and h ( t 2 ) = h ( t ∗ ) + ∫ t ∗ t 2 h ′ ≤ h ( t 1 ) h(t_2)=h(t_*)+\int_{t_*}^{t_2}h'\le h(t_1) h ( t 2 ) = h ( t ∗ ) + ∫ t ∗ t 2 h ′ ≤ h ( t 1 ) , a contradiction). Hence y 2 ( t ) ≤ m ( t ) + h ( 0 ) y_2(t)\le m(t)+h(0) y 2 ( t ) ≤ m ( t ) + h ( 0 ) , where h ( 0 ) = ( y 2 ( 0 ) − y 1 ( 0 ) ) + h(0)=(y_2(0)-y_1(0))_+ h ( 0 ) = ( y 2 ( 0 ) − y 1 ( 0 ) ) + is a constant (it vanishes for the trajectory of Theorem C.2, along which v > 0 v>0 v > 0 at the initial time). Thus
y 2 ( t ) ≤ max 0 ≤ r ≤ t y 1 ( r ) + h ( 0 ) ≤ log ( α t ) + O ( 1 ) for t ≥ t 0 , (249) y_2(t)\le\max_{0\le r\le t}y_1(r)+h(0)\le\log(\alpha t)+O(1) \qquad\text{for }t\ge t_0, \tag*{(249)} y 2 ( t ) ≤ 0 ≤ r ≤ t max y 1 ( r ) + h ( 0 ) ≤ log ( α t ) + O ( 1 ) for t ≥ t 0 , (249)
where we used that y 1 ( r ) ≤ log ( α r ) + C ≤ log ( α t ) + C y_1(r)\le\log(\alpha r)+C\le\log(\alpha t)+C y 1 ( r ) ≤ log ( α r ) + C ≤ log ( α t ) + C for t 0 ≤ r ≤ t t_0\le r\le t t 0 ≤ r ≤ t and that y 1 y_1 y 1 is bounded on [ 0 , t 0 ] [0,t_0] [ 0 , t 0 ] . Therefore
y 2 ( t ) = log ( α t ) + O ( 1 ) . (250) y_2(t)=\log(\alpha t)+O(1). \tag*{(250)} y 2 ( t ) = log ( α t ) + O ( 1 ) . (250)
This completes the proof.
By Theorem C.5 , we can prove Theorem 4.2 .
Proof of Theorem 4.2. By Theorems C.2, C.3 and C.5 , for t ≥ T 0 t\ge T_0 t ≥ T 0 we have
a 1 ≡ c 0 , ( b 2 ) + = ( b 2 − b 1 ) + = 0 , d 2 ≤ 0 , (251) a_1\equiv c_0,\qquad(b_2)_+=(b_2-b_1)_+=0,\qquad d_2\le0, \tag*{(251)} a 1 ≡ c 0 , ( b 2 ) + = ( b 2 − b 1 ) + = 0 , d 2 ≤ 0 , (251)
a 2 , b 1 = 2 log ( α t ) + O ( 1 ) , g 1 − d 1 = log ( α t ) + O ( 1 ) , g 2 − d 2 = log ( α t ) + O ( 1 ) , (252) a_2,b_1=2\log(\alpha t)+O(1),\qquad g_1-d_1=\log(\alpha t)+O(1),\qquad g_2-d_2=\log(\alpha t)+O(1), \tag*{(252)} a 2 , b 1 = 2 log ( α t ) + O ( 1 ) , g 1 − d 1 = log ( α t ) + O ( 1 ) , g 2 − d 2 = log ( α t ) + O ( 1 ) , (252)
and 0 ≤ u = a 2 − b 1 ≤ C u 0\le u=a_2-b_1\le C_u 0 ≤ u = a 2 − b 1 ≤ C u , v + = ( g 1 − d 1 − g 2 + d 2 ) + ≤ C v v_+=(g_1-d_1-g_2+d_2)_+\le C_v v + = ( g 1 − d 1 − g 2 + d 2 ) + ≤ C v for constants C u , C v C_u,C_v C u , C v given by items 1 and 3 of Theorem C.5 . The two-sided bound ∣ g 1 − d 1 − g 2 + d 2 ∣ ≤ C |g_1-d_1-g_2+d_2|\le C ∣ g 1 − d 1 − g 2 + d 2 ∣ ≤ C follows from the last two growth rates for t ≥ t 0 t\ge t_0 t ≥ t 0 and from continuity on [ 0 , t 0 ] [0,t_0] [ 0 , t 0 ] ; this proves the first part of the theorem. By Equation (127) the individual parameters satisfy
g 1 , g 2 = N − 1 N log ( α t ) + O ( 1 ) , d 1 , d 2 = − 1 N log ( α t ) + O ( 1 ) ; (253) g_1,g_2=\frac{N-1}{N}\log(\alpha t)+O(1),\qquad d_1,d_2=-\frac{1}{N}\log(\alpha t)+O(1); \tag*{(253)} g 1 , g 2 = N N − 1 log ( α t ) + O ( 1 ) , d 1 , d 2 = − N 1 log ( α t ) + O ( 1 ) ; (253)
in particular d 2 → − ∞ d_2\to-\infty d 2 → − ∞ . All constants below depend only on N , K , c 0 , c 0 ′ N,K,c_0,c'_0 N , K , c 0 , c 0 ′ .
Logits on a general graph. Consider any directed graph on N N N vertices, any valid nonterminated DFS trace, and the current node n t n_t n t with stack parent p p p (if n t ≠ n R o o t n_t\ne n_{\mathrm{Root}} n t = n Root ). We read the post-ReLU logits off Tables 1 and 2, using b 2 = 0 b_2=0 b 2 = 0 , ( d 2 ) + = 0 (d_2)_+=0 ( d 2 ) + = 0 (so b 2 d 2 i n v , t o t a l = 0 b_2d_2^{\mathrm{inv,total}}=0 b 2 d 2 inv , total = 0 and ( b 2 − b 1 m v ) + = 0 (b_2-b_1m_v)_+=0 ( b 2 − b 1 m v ) + = 0 ), ( g 1 − d 1 ) + = g 1 − d 1 (g_1-d_1)_+=g_1-d_1 ( g 1 − d 1 ) + = g 1 − d 1 and the dynamic readout, which sets the logit of n t n_t n t to 0 while keeping n t n_t n t in the softmax with weight 1:
an unvisited child v v v : logit ( a 2 + b 2 ( d v i n , t o t a l − 1 ) ) + = a 2 (a_2 + b_2(d_v^{\mathrm{in,total}} - 1))_+ = a_2 ( a 2 + b 2 ( d v in , total − 1 ) ) + = a 2 , since b 2 = 0 b_2 = 0 b 2 = 0 for t ≥ T 0 t \ge T_0 t ≥ T 0 ;
the parent p p p : logit ( g 1 − d 1 + d 2 1 [ p = n R o o t ] ) + + 1 [ ( n t → p ) ∈ E ] ( a 2 − b 1 m p ) + (g_1 - d_1 + d_2 \mathbf{1}[p = n_{\mathrm{Root}}])_+ + \mathbf{1}[(n_t \to p) \in\mathcal{E}](a_2 - b_1m_p)_+ ( g 1 − d 1 + d 2 1 [ p = n Root ] ) + + 1 [( n t → p ) ∈ E ] ( a 2 − b 1 m p ) + , which lies between ( g 1 − d 1 + d 2 1 [ p = n R o o t ] ) + (g_1 - d_1 + d_2 \mathbf{1}[p = n_{\mathrm{Root}}])_+ ( g 1 − d 1 + d 2 1 [ p = n Root ] ) + and g 1 − d 1 + u + g_1 - d_1 + u_+ g 1 − d 1 + u + ;
Let m m m be the number of unvisited children, m 1 m_1 m 1 the number of visited children other than the parent, and m 2 ≤ N m_2 \le N m 2 ≤ N the number of remaining vertices. The softmax normalizer Z Z Z therefore satisfies
Z ≤ m e a 2 + m 1 e C u + C v + e g 1 − d 1 + C u + m 2 ≤ m e a 2 + C e g 1 − d 1 + C . (254) Z \le m e^{a_2} + m_1 e^{C u + C v} + e^{g_1-d_1+C u} + m_2 \le m e^{a_2} + C e^{g_1-d_1} + C. \tag*{(254)} Z ≤ m e a 2 + m 1 e C u + C v + e g 1 − d 1 + C u + m 2 ≤ m e a 2 + C e g 1 − d 1 + C . (254)
Case 1: n t n_t n t has unvisited children (m ≥ 1 m \ge1 m ≥ 1 ). The admissible set C t C_t C t consists of the unvisited children, each with logit L v = a 2 L_v = a_2 L v = a 2 , and the normalizer is Z = ∑ v ∈ C t e L v + Z ′ Z = \sum_{v \in C_t} e^{L_v} + Z' Z = ∑ v ∈ C t e L v + Z ′ with Z ′ ≤ C e g 1 − d 1 + C Z' \le C e^{g_1-d_1} + C Z ′ ≤ C e g 1 − d 1 + C by Equation (254) . Hence
− log ( ∑ v ∈ C t p θ ( v ) ) = log ( 1 + Z ′ ∑ v ∈ C t e L v ) ≤ log ( 1 + C e g 1 − d 1 + C m e a 2 ) (255) -\log\left(\sum_{v \in C_t} p_\theta(v)\right) = \log\left(1 + \frac{Z'}{\sum_{v \in C_t} e^{L_v}}\right) \le\log\left(1 + \frac{C e^{g_1-d_1} + C}{m e^{a_2}}\right) \tag*{(255)} − log ( v ∈ C t ∑ p θ ( v ) ) = log ( 1 + ∑ v ∈ C t e L v Z ′ ) ≤ log ( 1 + m e a 2 C e g 1 − d 1 + C ) (255)
≤ 2 C e g 1 − d 1 − a 2 = O ( ( α t ) − 1 ) , (256) \le2C e^{g_1-d_1-a_2} = O\left((\alpha t)^{-1}\right), \tag*{(256)} ≤ 2 C e g 1 − d 1 − a 2 = O ( ( α t ) − 1 ) , (256)
where we used g 1 − d 1 > 0 g_1 - d_1 > 0 g 1 − d 1 > 0 (so C ≤ C e g 1 − d 1 C \le C e^{g_1-d_1} C ≤ C e g 1 − d 1 ) and a 2 − ( g 1 − d 1 ) = log ( α t ) + O ( 1 ) a_2 - (g_1 - d_1) = \log(\alpha t) + O(1) a 2 − ( g 1 − d 1 ) = log ( α t ) + O ( 1 ) .
Case 2: no unvisited child and p ≠ n R o o t p \ne n_{\mathrm{Root}} p = n Root . The admissible set is { p } \{p\} { p } . Writing L p L_p L p for the parent’s logit, L p ≥ g 1 − d 1 L_p \ge g_1 - d_1 L p ≥ g 1 − d 1 , and the normalizer is e L p + Z ′ ′ e^{L_p} + Z'' e L p + Z ′′ with Z ′ ′ ≤ m 1 e C u + C v + m 2 ≤ C Z'' \le m_1 e^{C u+C v} + m_2 \le C Z ′′ ≤ m 1 e C u + C v + m 2 ≤ C (the parent’s own term, including its forward contribution when ( n t → p ) ∈ E (n_t \to p) \in\mathcal{E} ( n t → p ) ∈ E , is in the numerator as well). Hence
− log p θ ( p ) = log ( 1 + Z ′ ′ e L p ) ≤ log ( 1 + C e − ( g 1 − d 1 ) ) = O ( ( α t ) − 1 ) . (257) -\log p_\theta(p) = \log\left(1 + \frac{Z''}{e^{L_p}}\right) \le\log\left(1 + C e^{-(g_1-d_1)}\right) = O\left((\alpha t)^{-1}\right). \tag*{(257)} − log p θ ( p ) = log ( 1 + e L p Z ′′ ) ≤ log ( 1 + C e − ( g 1 − d 1 ) ) = O ( ( α t ) − 1 ) . (257)
Case 3: no unvisited child and p = n R o o t p = n_{\mathrm{Root}} p = n Root . Now the parent’s logit is L p ≥ g 1 − d 1 + d 2 L_p \ge g_1 - d_1 + d_2 L p ≥ g 1 − d 1 + d 2 , and the same computation gives
− log p θ ( p ) ≤ log ( 1 + C e − ( g 1 − d 1 + d 2 ) ) = O ( e − N − 1 N log ( α t ) ) = O ( ( α t ) − N − 1 N ) , (258) -\log p_\theta(p) \le\log\left(1 + C e^{-(g_1-d_1+d_2)}\right) = O\left(e^{-\frac{N-1}{N}\log(\alpha t)}\right) = O\left((\alpha t)^{-\frac{N-1}{N}}\right), \tag*{(258)} − log p θ ( p ) ≤ log ( 1 + C e − ( g 1 − d 1 + d 2 ) ) = O ( e − N N − 1 l o g ( α t ) ) = O ( ( α t ) − N N − 1 ) , (258)
since g 1 − d 1 + d 2 = N − 1 N log ( α t ) + O ( 1 ) g_1 - d_1 + d_2 = \frac{N-1}{N}\log(\alpha t) + O(1) g 1 − d 1 + d 2 = N N − 1 log ( α t ) + O ( 1 ) . This rate cannot be improved: the masked current node contributes 1 1 1 to the normalizer, so − log p θ ( p ) ≥ log ( 1 + e − L p ) -\log p_\theta(p) \ge\log(1 + e^{-L_p}) − log p θ ( p ) ≥ log ( 1 + e − L p ) with L p = g 1 − d 1 + d 2 L_p = g_1 - d_1 + d_2 L p = g 1 − d 1 + d 2 exactly when ( n t → n R o o t ) ∉ E (n_t \to n_{\mathrm{Root}}) \notin\mathcal{E} ( n t → n Root ) ∈ / E , and the graph with the single edge ( n R o o t → a ) (n_{\mathrm{Root}} \to a) ( n Root → a ) and the trace n R o o t , a , n R o o t n_{\mathrm{Root}}, a, n_{\mathrm{Root}} n Root , a , n Root realizes this case.
A valid DFS trace on N N N vertices has at most 2 ( N − 1 ) 2(N - 1) 2 ( N − 1 ) steps, and in every step one of the three cases applies (the terminated state, in which the root has no unvisited children, carries no loss term). Summing over the steps and taking the supremum over graphs and traces,
L g e n ≤ 2 ( N − 1 ) C ( α t ) − N − 1 N (259) \mathcal{L}_{\mathrm{gen}} \le2(N - 1) C (\alpha t)^{-\frac{N-1}{N}} \tag*{(259)} L gen ≤ 2 ( N − 1 ) C ( α t ) − N N − 1 (259)
for all t ≥ t ∗ / α t \ge t_*/\alpha t ≥ t ∗ / α , where t ∗ t_* t ∗ is such that all the O ( 1 ) O(1) O ( 1 ) estimates above are in force. This proves Equation (18) .
Additional experimental results
Protocol. All runs use float64. The fixed first layer is implemented by its exact output on each token type (Equation (7) ); the second layer, its value and output matrices, and the dynamic readout are those of § 4 , and the softmax is taken over the N N N vertex tokens. The reduced flow integrates the eight-scalar system of Theorem C.1 with an implicit solver in logarithmic time (relative tolerance 10 − 9 10^{-9} 1 0 − 9 ), replacing each indicator 1 x ≥ 0 \mathbf{1}_{x\geq0} 1 x ≥ 0 by the steep ramp min { max { x / ϵ + 1 , 0 } , 1 } \min\{\max\{x/\epsilon+1,0\},1\} min { max { x / ϵ + 1 , 0 } , 1 } with ϵ = 10 − 6 \epsilon=10^{-6} ϵ = 1 0 − 6 ; changing ϵ \epsilon ϵ by two orders of magnitude moves the trajectory by O ( ϵ ) O(\epsilon) O ( ϵ ) , and plain gradient descent with η = 0.05 \eta=0.05 η = 0.05 agrees with it to within 0.012 on [ 1 , 5 × 10 5 ] [1,5\times10^5] [ 1 , 5 × 1 0 5 ] . Full-matrix training uses B = 128 B=128 B = 128 random vertex permutations per gradient step, i.e., a stochastic version of the loss Equation (13) ; the effective scalars of a matrix are the mean of its diagonal and the mean of its off-diagonal entries. The implementation was checked against a direct autograd implementation, against finite differences, against the closed-form node logits of Tables 1 and Table 2 on random graphs, and, for N = 7 N=7 N = 7 , against the exact average over all 5040 permutations.
The test set contains, for N = 16 N=16 N = 16 , 300 random graphs (Erdős–Rényi graphs with edge probability in { 0.1 , 0.2 , 0.4 } \{0.1,0.2,0.4\} { 0.1 , 0.2 , 0.4 } , random out-trees with up to three extra edges, and “root-fan” graphs in which the root starts several chains and the target is unreachable or on a late chain), together with the full canonical trace and the single-edge graph { ( n R o o t → a ) } \{(n_{\mathrm{Root}}\to a)\} {( n Root → a )} . For each graph we enumerate all valid DFS prefixes (or sample 50 traces when there are more), giving 64,325 distinct states, of which 5,625 are forward steps at a node that has an edge back to its stack parent.
Individual parameters and gap variables. Figure 2 shows the quantities behind Equation (127) and Theorem C.5 . The combination g 1 + ( N − 1 ) d 1 g_1+(N-1)d_1 g 1 + ( N − 1 ) d 1 is conserved to 5 × 10 − 12 5\times10^{-12} 5 × 1 0 − 12 along the reduced flow; g 2 + ( N − 1 ) d 2 g_2+(N-1)d_2 g 2 + ( N − 1 ) d 2 moves by 0.019 during the initial phase, while d 2 > 0 d_2>0 d 2 > 0 , and is constant afterwards, as predicted. The gap u = a 2 − b 1 u=a_2-b_1 u = a 2 − b 1 is nonnegative throughout and converges to about 4.8 after a transient maximum of 5.1 at t ≈ 400 t \approx400 t ≈ 400 ; v v v rises to 0.34 during the initial phase and then converges to 0, i.e., the trajectory slides on the switching surface v = 0 v = 0 v = 0 for all t ≳ 10 2 t \gtrsim10^2 t ≳ 1 0 2 ; and 2 ( g 1 − d 1 ) − a 2 2(g_1 - d_1) - a_2 2 ( g 1 − d 1 ) − a 2 converges to about 5.6.
Figure 2. Left: the individual parameters g 1 , g 2 g_1,g_2 g 1 , g 2 (fitted slopes 0.935; predicted ( N − 1 ) / N = 0.9375 (N-1)/N=0.9375 ( N − 1 ) / N = 0.9375 ) and d 1 , d 2 d_1,d_2 d 1 , d 2 (fitted slopes − 0.062 -0.062 − 0.062 ; predicted − 1 / N -1/N − 1/ N ), and the conserved combinations g i + ( N − 1 ) d i g_i+(N-1)d_i g i + ( N − 1 ) d i . Right: the gap variables u = a 2 − b 1 u=a_2-b_1 u = a 2 − b 1 , v = g 1 − d 1 − g 2 + d 2 v=g_1-d_1-g_2+d_2 v = g 1 − d 1 − g 2 + d 2 and 2 ( g 1 − d 1 ) − a 2 2(g_1-d_1)-a_2 2 ( g 1 − d 1 ) − a 2 , which stay bounded as in Theorem C.5 , together with b 2 b_2 b 2 , which is frozen at 0 after the initial phase.
Invariant subspace under sampled permutations. Theorem 4.1 concerns the exact permutation-averaged loss. With B = 128 B = 128 B = 128 sampled permutations per step (Figure 3 , left), the iterate leaves the subspace during the initial phase, when the dynamics slide along the switching surfaces b 2 = 0 b_2 = 0 b 2 = 0 and v = 0 v = 0 v = 0 : the relative distance of B 1 B_1 B 1 and B 2 B_2 B 2 reaches 0.7–0.9 and the effective scalars differ from the reduced flow by up to 0.9 at t ≈ 15 t \approx15 t ≈ 15 . The iterate then returns: the distance is below 10 − 2 10^{-2} 1 0 − 2 from t ≈ 70 t \approx70 t ≈ 70 on and at most 6 × 10 − 4 6 \times10^{-4} 6 × 1 0 − 4 at t = 5 × 10 5 t = 5 \times10^5 t = 5 × 1 0 5 , where the effective scalars agree with the reduced flow to within 0.03. The excursion is a sampling effect: rerunning the early phase with B ∈ { 512 , 2048 } B \in\{512, 2048\} B ∈ { 512 , 2048 } reduces it only partially, whereas B ≥ 8192 B \ge8192 B ≥ 8192 keeps the distance below 0.03 and the scalars within 0.007 of the reduced flow at all times, and reducing η \eta η tenfold at B = 128 B = 128 B = 128 has no effect.
Figure 3. Left: relative distance ∥ X − ( x ˉ d i a g I + x ˉ o f f E o f f ) ∥ F / ∥ X ∥ F \lVert X - (\bar{x}_{\mathrm{diag}} I + \bar{x}_{\mathrm{off}} E_{\mathrm{off}}) \rVert_F / \lVert X \rVert_F ∥ X − ( x ˉ diag I + x ˉ off E off ) ∥ F / ∥ X ∥ F of each trained matrix from the invariant subspace during full-matrix training with B = 128 B = 128 B = 128 sampled permutations per step. Right: fraction of test states at which the argmax is a valid DFS move, by step type.
Exponent versus N N N . The table in § 5 reports the exponent of the root-backtracking loss fitted on t ∈ [ 10 6 , 10 9 ] t \in[10^6, 10^9] t ∈ [ 1 0 6 , 1 0 9 ] for N ∈ { 8 , 10 , 12 , 16 , 24 , 32 } N \in\{8, 10, 12, 16, 24, 32\} N ∈ { 8 , 10 , 12 , 16 , 24 , 32 } ; every fit lies within 3 × 10 − 4 3 \times10^{-4} 3 × 1 0 − 4 of − ( N − 1 ) / N -(N - 1)/N − ( N − 1 ) / N . Figure 4 shows the local log–log slope of the same curves: it approaches − ( N − 1 ) / N -(N - 1)/N − ( N − 1 ) / N from below, and the transient lasts longer for larger N N N , because the O ( 1 ) O(1) O ( 1 ) terms in d 2 = − 1 N log ( α t ) + O ( 1 ) d_2 = -\frac{1}{N}\log(\alpha t) + O(1) d 2 = − N 1 log ( α t ) + O ( 1 ) then take longer to become negligible.
Accuracy and back-edge states. Figure 3 (right) shows when the argmax becomes a valid DFS move. Backtracking steps are correct from t ≈ 5 t \approx5 t ≈ 5 , forward steps without an edge back to the parent from t ≈ 40 t \approx40 t ≈ 40 , and forward steps with such an edge only from t ≈ 1.8 × 10 4 t \approx1.8 \times10^4 t ≈ 1.8 × 1 0 4 , when b 1 b_1 b 1 exceeds g 1 − d 1 g_1 - d_1 g 1 − d 1 ; the overall accuracy is 0.91 at t = 100 t = 100 t = 100 and equals 1 from t ≈ 2 × 10 4 t \approx2 \times10^4 t ≈ 2 × 1 0 4 on, and both training procedures give the same curves. The back-edge states are also the ones that attain the supremum L g e n \mathcal{L}_{\mathrm{gen}} L gen over the whole observed range: a visited child of the current node that is also its stack parent receives votes from both heads, g 1 − d 1 g_1 - d_1 g 1 − d 1 from the backtracking head and a 2 − b 1 a_2 - b_1 a 2 − b 1 from the forward head, so its logit gap to the correct unexplored child is b 1 − ( g 1 − d 1 ) = log ( α t ) − u + O ( 1 ) b_1 - (g_1 - d_1) = \log(\alpha t) - u + O(1) b 1 − ( g 1 − d 1 ) = log ( α t ) − u + O ( 1 ) and the loss is Θ ( e u / t ) \Theta(e^{u}/t) Θ ( e u / t ) with e u ≈ e 5 e^{u} \approx e^{5} e u ≈ e 5 . Both effects are consequences of the two-head mechanism itself rather than of the training horizon.
Initialization. Figure 5 and Figure 6 examine the role of the initialization Equation (16) . The all-constant initialization c 0 E c_{0}E c 0 E lies on the two switching surfaces b 2 = b 1 b_{2}=b_{1} b 2 = b 1 and g 1 = d 1 g_{1}=d_{1} g 1 = d 1 , where the vector field is discontinuous, and its behaviour depends on the convention for ReLU ′ ( 0 ) \operatorname{ReLU}'(0) ReLU ′ ( 0 ) and on ( N , K ) (N,K) ( N , K ) . With ReLU ′ ( 0 ) = 0 \operatorname{ReLU}'(0)=0 ReLU ′ ( 0 ) = 0 (the default of common autograd libraries) and exactly cancelling head sums, g 1 − d 1 g_{1}-d_{1} g 1 − d 1 remains exactly 0 and the backtracking head never activates; the forward parameters then grow only half as fast (a 2 , b 1 ≈ 11.5 a_{2},b_{1}\approx11.5 a 2 , b 1 ≈ 11.5 at t = 5 × 10 5 t=5\times10^{5} t = 5 × 1 0 5 against 22.0 and 17.2). With ReLU ′ ( 0 ) = 1 \operatorname{ReLU}'(0)=1 ReLU ′ ( 0 ) = 1 the outcome depends on ( N , K ) (N,K) ( N , K ) : at ( 16 , 8 ) (16,8) ( 16 , 8 ) the trajectory enters the region { b 2 < b 1 , g 1 > d 1 } \{b_{2}<b_{1},\ g_{1}>d_{1}\} { b 2 < b 1 , g 1 > d 1 } at once, but at ( 16 , 2 ) (16,2) ( 16 , 2 ) and ( 40 , 3 ) (40,3) ( 40 , 3 ) it first exits towards b 2 > b 1 b_{2}>b_{1} b 2 > b 1 with b 1 < 0 b_{1}<0 b 1 < 0 for a transient of several hundred time units, after which it approaches the trajectory from Equation (16) in a 2 , b 1 , g 1 − d 1 a_{2},b_{1},g_{1}-d_{1} a 2 , b 1 , g 1 − d 1 but keeps a positive b 2 b_{2} b 2 that decays only logarithmically slowly (0.34 at t = 5 × 10 5 t=5\times10^{5} t = 5 × 1 0 5 for ( 16 , 2 ) (16,2) ( 16 , 2 ) ), its gradient being O ( 1 / t ) O(1/t) O ( 1/ t ) . Across 60 pairs ( N , K ) (N,K) ( N , K ) with 8 ≤ N ≤ 64 8\le N\le64 8 ≤ N ≤ 64 and 2 ≤ K ≤ 10 2\le K\le10 2 ≤ K ≤ 10 , the wrong-side exit occurs in all but one case in which N > ( K + 2 ) 2 / 2 N>(K+2)^{2}/2 N > ( K + 2 ) 2 /2 , and in none of the others. In practice, floating-point round-off or permutation sampling is enough to release the frozen case (Figure 5 , D3 with round-off and D4), but the trajectory is then selected by numerical noise rather than by the gradient flow. The initialization Equation (16) avoids all of these effects: it starts inside the region that Theorem C.2 shows to be invariant and stays there from t = 0 t=0 t = 0 in every setting we tried.
Figure 5. Initialization ablation at N = 16 N = 16 N = 16 , K = 8 K = 8 K = 8 (reduced flow by gradient descent with η = 0.05 \eta= 0.05 η = 0.05 ; D4 is full-matrix training with three seeds). D1: the initialization Equation (16) . D2, D3: the all-constant initialization c 0 E c_0E c 0 E with the conventions ReLU ′ ( 0 ) = 1 \operatorname{ReLU}'(0) = 1 ReLU ′ ( 0 ) = 1 and ReLU ′ ( 0 ) = 0 \operatorname{ReLU}'(0) = 0 ReLU ′ ( 0 ) = 0 . With ReLU ′ ( 0 ) = 0 \operatorname{ReLU}'(0) = 0 ReLU ′ ( 0 ) = 0 and exactly cancelling head sums, g 1 − d 1 g_1-d_1 g 1 − d 1 stays at 0 forever and the backtracking head never activates; floating-point round-off in the head sums, or sampling noise (D4), releases it.
Figure 6. The all-constant initialization c 0 E c_0E c 0 E with ReLU ′ ( 0 ) = 1 \operatorname{ReLU}'(0)=1 ReLU ′ ( 0 ) = 1 at ( N , K ) = ( 16 , 2 ) (N,K)=(16,2) ( N , K ) = ( 16 , 2 ) , ( 40 , 3 ) (40,3) ( 40 , 3 ) , and ( 16 , 8 ) (16,8) ( 16 , 8 ) . When N N N is large relative to K K K , the trajectory first leaves the switching surface b 2 = b 1 b_2=b_1 b 2 = b 1 towards b 2 > b 1 b_2>b_1 b 2 > b 1 and b 1 b_1 b 1 becomes negative, before re-entering the region { b 2 < b 1 , g 1 > d 1 } \{b_2<b_1,\,g_1>d_1\} { b 2 < b 1 , g 1 > d 1 } at t ≈ 300 t\approx300 t ≈ 300 and t ≈ 10 3 t\approx10^3 t ≈ 1 0 3 , respectively; at ( 16 , 8 ) (16,8) ( 16 , 8 ) it enters the region immediately. The initialization Equation (16) stays in the region from t = 0 t=0 t = 0 in all three cases.
Summary. The experiments agree with the training-dynamics theory in every quantitative prediction: the forward-head parameters grow at 2 log t 2\log t 2 log t , the backtracking separation at log t \log t log t , the individual parameters at N − 1 N log t \frac{N-1}{N}\log t N N − 1 log t and − 1 N log t -\frac{1}{N}\log t − N 1 log t , the gap variables remain bounded, and backtracking to the root decays at exactly t − ( N − 1 ) / N t^{-(N-1)/N} t − ( N − 1 ) / N for N N N between 8 and 32. They also show which parts of the statements are asymptotic: the supremum loss is dominated over all practical horizons by t − 1 t^{-1} t − 1 terms with large constants, and sampled-permutation training approaches the invariant subspace only after the initial phase.