Introduction
A recurrent model compresses a history under its current parameters. Replacing those parameters can change which historical distinctions matter. An expressive migration map cannot recover distinctions that the old state has discarded. We ask what information a processor must retain before the replacement is known to reconstruct the state that the new model would compute from the same inputs.
The target is retrospective. A query at time t t t requests the state obtained by applying one replacement model to the complete prefix x 1 : t x_{1:t} x 1 : t , not the trajectory generated by historical weight changes. This question arises when a running filter is recalibrated or an existing recurrent context is reinterpreted. We study linear recurrent models with fixed input injections. Arbitrary changes to a multilayer network are outside this setting.
One relevant case is transition-only adaptation of a linear recurrent block, such as an LRU, with its input projection and upstream features frozen [17 ] . The general causal characterization permits vector inputs. The sharper k d kd k d bound additionally requires a controllable scalar-input anchor.
Our main result identifies the minimum continuous causal memory for local approximation of order k k k . It equals the reachable dimension of a lifted coefficient system minus the information already present in the anchor state. For a controllable d d d -state model with scalar input, the sharp worst-case bound is k d kd k d additional real coordinates. The encoder can be fixed before any matrix replacement direction is specified, while one rank-one direction can already require all k d kd k d coordinates. Unknown arbitrary directions and one suitably chosen direction therefore have the same worst-case local information cost. The result counts independent historical information, which can be smaller than the coefficient array that represents it.
Exact local order, finite-tolerance accuracy, and practical storage are distinct. We give conditional singular-value and auxiliary-bit bounds for a fixed finite query set, then construct causal coefficient memories under an invariant-subspace condition. An output quotient separates preservation of future predictions from preservation of every state coordinate. The experiments compare coefficient, interpolation, and history-basis representations at matched memory budgets and prescribed prediction tolerances. The memory-width study relates the required storage to the requested output and the anchor’s pole structure, with shared setup and decoder costs accounted for separately.
Relation to prior work. Recurrent-state staleness is studied by R2D2 (Kapturowski et al., [12 ] . RTRL propagates exact first derivatives Williams and Zipser [21 ] , UORO approximates them online Tallec and Ollivier [19 ] , and structured recurrent learning reduces their maintenance cost Farr et al. [5 ] . Expansion, invariant projection, rational realization, and residual estimation are established tools of parametric model reduction [2 , 10 ] . Continuous nonlinear widths supply the fixed-horizon language [4 , 18 ] . Laguerre models and HiPPO retain historical signals in prescribed bases [20 , 8 ] , whereas stitching transforms a representation using model-pair calibration [1 , 14 ] . Context composition within an SSM [13 ] and state compression during training [3 ] concern different targets. The contribution is a causal information characterization for a replacement revealed after streaming. The differentiation rules, orthogonal filter families, and generic residual bounds are established ingredients. Appendix § 25 gives result-level comparisons.
Causal information before a replacement
Let h t ( a ) h_t(a) h t ( a ) be the state produced from zero by processing x 1 : t x_{1:t} x 1 : t with the same coordinate a a a . The anchor is h t ( 0 ) h_t(0) h t ( 0 ) . An auxiliary summary updates from its previous value, the anchor, and the current input without knowing future queries. A query returns an estimate of h t ( a ) h_t(a) h t ( a ) but does not change the underlying summary. Queries at a a a and then a ′ a' a ′ each concern the full prefix. They do not describe a trajectory that first used a a a and then a ′ a' a ′ . After a deployment change, maintaining the anchor adds storage and computation to those of the active model.
Two histories already show why auxiliary information is necessary. The scalar old state is h 2 ( 0 ) = λ x 1 + x 2 h_2(0)=\lambda x_1+x_2 h 2 ( 0 ) = λ x 1 + x 2 . Histories ( x 1 , x 2 ) (x_1,x_2) ( x 1 , x 2 ) and ( x 1 + v , x 2 − λ v ) (x_1+v,x_2-\lambda v) ( x 1 + v , x 2 − λ v ) collide in that state, but replacing λ \lambda λ by λ + a \lambda+a λ + a separates their targets by a v av a v . Retaining x 1 x_1 x 1 before the query resolves the ambiguity. Whether this distinction matters depends on the tolerance and readout.
The reachable coefficient space
Consider the time-invariant family
h t ( a ) = ( A 0 + ∑ i = 1 p a i A i ) h t − 1 ( a ) + B x t , h 0 ( a ) = 0. (1) h_t(a)=\left(A_0+\sum_{i=1}^{p}a_iA_i\right)h_{t-1}(a)+Bx_t,\qquad h_0(a)=0. \tag*{(1)} h t ( a ) = ( A 0 + i = 1 ∑ p a i A i ) h t − 1 ( a ) + B x t , h 0 ( a ) = 0. (1)
An auxiliary summary supports order-k k k local migration when, for every finite history,
∥ h t ( a ) − D a ( h t ( 0 ) , S t ) ∥ = O ( ∥ a ∥ k + 1 ) ( a → 0 ) . (2) \lVert h_t(a)-D_a(h_t(0),S_t)\rVert=O(\lVert a\rVert^{k+1})\qquad(a\to0). \tag*{(2)} ∥ h t ( a ) − D a ( h t ( 0 ) , S t )∥ = O (∥ a ∥ k + 1 ) ( a → 0 ) . (2)
Table 1. Four different memory questions. Local results count continuous real coordinates, not bits. All targets reprocess the same historical inputs under one replacement model.
The implied constant may depend on the history. The parameter set contains a neighborhood of zero. This definition concerns an asymptotic order, not a fixed-radius numerical error.
For two update directions, the first-order state and sensitivity coefficients obey
\begin{bmatrix} h_t^0 \\ z_{t,1} \\ z_{t,2} \end{bmatrix}
=
\begin{bmatrix} A_0 & 0 & 0 \\ A_1 & A_0 & 0 \\ A_2 & 0 & A_0 \end{bmatrix}
\begin{bmatrix} h_{t-1}^0 \\ z_{t-1,1} \\ z_{t-1,2} \end{bmatrix}
+
\begin{bmatrix} Bx_t \\ 0 \\ 0 \end{bmatrix}. \tag*{(3)}\end{bmatrix}
The three blocks can contain redundant historical information. With d = 2 d=2 d = 2 and scalar input, a controllable example has only four reachable coordinates in this six-coordinate array. Its anchor accounts for two, leaving two auxiliary coordinates rather than four. Appendix P.4 gives explicit noncommuting matrices, exact ranks through degree three, and a numerical realization.
Stack all coefficients of degrees zero through k k k into j t j_t j t . Its exact recurrence is j t = T k j t − 1 + B k x t j_t=T_kj_{t-1}+B_kx_t j t = T k j t − 1 + B k x t , with A 0 A_0 A 0 on every diagonal block and A i A_i A i from block α − e i \alpha-e_i α − e i to block α \alpha α . The input acts only on the constant block. Let n k = d ( p + k k ) n_k=d^{\binom{p+k}{k}} n k = d ( k p + k ) , let π 0 \pi_0 π 0 extract that block, and define the reachable coefficient space
K k = ∑ s = 0 n k − 1 im ( T k s B k ) . (4) \mathcal{K}_k=\sum_{s=0}^{n_k-1}\operatorname{im}\left(T_k^sB_k\right). \tag*{(4)} K k = s = 0 ∑ n k − 1 im ( T k s B k ) . (4)
Theorem 1 (Minimum causal local memory). For real-vector inputs, the minimum number of continuous causal auxiliary coordinates beyond the anchor that supports (2) at every finite horizon is
ν k = dim K k − dim ( π 0 K k ) . (5) \nu_k=\dim\mathcal{K}_k-\dim\left(\pi_0\mathcal{K}_k\right). \tag*{(5)} ν k = dim K k − dim ( π 0 K k ) . (5) A linear causal encoder attains this minimum, even when the competing decoders are unrestricted.
Histories that share a summary must agree in every coefficient through degree k k k to satisfy local accuracy. The antipodal argument gives a lower bound once the coefficient system is fully reached. For the upper bound, choose a linear complement to π 0 \pi_0 π 0 on K k \mathcal{K}_k K k , retain its coordinates, and update them before any query. Appendix H proves both directions. The minimum concerns exact local order, not numerical conditioning or update cost.
Theorem 2 (A rank-one direction can be worst case). For every d , k ≥ 1 d,k\ge1 d , k ≥ 1 , a uniformly contractive scalar-input family
h s ( a ) = ( A + a v v T ) h s − 1 ( a ) + b u s (6) h_s(a)=(A+avv^\mathsf{T})h_{s-1}(a)+bu_s \tag*{(6)} h s ( a ) = ( A + a v v T ) h s − 1 ( a ) + b u s (6) requires k d kd k d additional continuous coordinates for (2) at horizon d ( k + 1 ) d(k+1) d ( k + 1 ) . Retaining its first k k k coefficient vectors attains this count.
Choose A A A diagonal with distinct eigenvalues in ( 0 , 1 ) (0,1) ( 0 , 1 ) and all entries of v , b v,b v , b nonzero. Its coefficient lift has A A A on each diagonal block and v v T vv^{\mathsf{T}} v v T immediately below. Controllability makes its d ( k + 1 ) d(k+1) d ( k + 1 ) coordinates independent functions of a sufficiently long history. The anchor retains only d d d . Appendix § 12 proves this by the PBH criterion. This is an existence result, not a claim about every rank-one family.
The same k d kd k d bound suffices for a scalar-input model permitting arbitrary transition replacements.
Corollary 3 (Sharp direction-independent scalar-input memory). If ( A 0 , b ) (A_0,b) ( A 0 , b ) is controllable and x t x_t x t is scalar , k d kd k d auxiliary real coordinates suffice for order-k k k local migration under every matrix replacement A 0 + Δ A_0+\Delta A 0 + Δ . The causal encoder can be chosen from ( A 0 , b , k ) (A_0,b,k) ( A 0 , b , k ) before Δ \Delta Δ or any update directions are known. No smaller worst-case count is possible. In particular, for every finite affine family, ν k ≤ k d \nu_k \le kd ν k ≤ k d , independently of p p p .
Let P P P be the characteristic polynomial of A 0 A_0 A 0 . The matrix P ( T k ) P(T_k) P ( T k ) strictly increases coefficient degree, so P ( T k ) k + 1 = 0 P(T_k)^{k+1}=0 P ( T k ) k + 1 = 0 . A scalar input can therefore reach at most d ( k + 1 ) d(k+1) d ( k + 1 ) lifted dimensions, regardless of the number of monomials. Subtracting the controllable anchor gives k d kd k d .
To construct a direction-independent encoder, put p ( z ) = det ( I − z A 0 ) p(z)=\det(I-zA_0) p ( z ) = det ( I − z A 0 ) and J ( z ) = adj ( I − z A 0 ) J(z)=\operatorname{adj}(I-zA_0) J ( z ) = adj ( I − z A 0 ) . The order-k k k response to Δ \Delta Δ has transfer function
H k ( z , Δ ) = ∑ ℓ = 0 k [ J ( z ) z Δ ] ℓ J ( z ) b p ( z ) k − ℓ p ( z ) k + 1 . (7) H_k(z,\Delta)=\frac{\sum_{\ell=0}^{k}[J(z)z\Delta]^\ell J(z)b\,p(z)^{k-\ell}}{p(z)^{k+1}}. \tag*{(7)} H k ( z , Δ ) = p ( z ) k + 1 ∑ ℓ = 0 k [ J ( z ) z Δ ] ℓ J ( z ) b p ( z ) k − ℓ . (7)
Its numerator degree is less than d ( k + 1 ) d(k+1) d ( k + 1 ) and its denominator is independent of Δ \Delta Δ . One scalar filter therefore supports all replacement directions. A coordinate transformation incorporates the ordinary anchor and leaves k d kd k d auxiliary coordinates. Appendix § 21 proves the construction and truncation bound and discusses conditioning. The information result assumes exact arithmetic. It does not justify using a high-degree companion filter in a large model; a smaller declared family can reduce implementation cost.
Fixed-horizon tolerance and precision
Throughout, vector and operator norms are Euclidean. The continuous lower bounds apply to real histories and need not extend to finite-alphabet tokens. For a fixed horizon, write the old state as R 0 u R_0u R 0 u , with u ∈ B 2 n u\in B_2^n u ∈ B 2 n , and stack the target maps into R u Ru R u . Let P 0 = I − R 0 T R 0 P_0=I-R_0^{\mathsf{T}}R_0 P 0 = I − R 0 T R 0 . The auxiliary encoder S : B 2 n → R m S:B_2^n\to\mathbb{R}^m S : B 2 n → R m must be continuous. This excludes discontinuous infinite-precision encodings of an entire history into one real. The decoder need not be continuous.
Theorem 4 (State side information). With singular-value lists extended by zeros,
inf S c o n t i n u o u s , D sup ∥ u ∥ ≤ 1 ∥ R u − D ( R 0 u , S ( u ) ) ∥ = σ m + 1 ( R P 0 ) . (8) \inf_{S\ \mathrm{continuous},D}\ \sup_{\lVert u\rVert\le1}\lVert Ru-D(R_0u,S(u))\rVert=\sigma_{m+1}(RP_0). \tag*{(8)} S continuous , D inf ∥ u ∥ ≤ 1 sup ∥ R u − D ( R 0 u , S ( u ))∥ = σ m + 1 ( R P 0 ) . (8) In particular, exact recovery requires and is attained by
rank [ R 0 R ] − rank R 0 (9) \operatorname{rank}\begin{bmatrix}R_0\\R\end{bmatrix}-\operatorname{rank}R_0 \tag*{(9)} rank [ R 0 R ] − rank R 0 (9) additional coordinates at that horizon.
The old state supplies R ( I − P 0 ) u R(I-P_0)u R ( I − P 0 ) u . Leading right-singular coordinates recover the remaining component optimally. For the lower bound, Borsuk–Ulam on a sphere in ker R 0 \ker R_0 ker R 0 gives indistinguishable antipodal histories with separated targets. This is a specialization of classical width arguments, not a new topological method. Appendix § 11 gives the full proof. The attaining sketch need not admit an inexpensive causal update.
Normalize a finite query set by R Q = J − 1 / 2 [ R 1 ⊤ , … , R J ⊤ ] ⊤ R_Q=J^{-1/2}[R_1^\top,\ldots,R_J^\top]^\top R Q = J − 1/2 [ R 1 ⊤ , … , R J ⊤ ] ⊤ . Its Euclidean error is RMS error over the J J J queries. The next statement turns exact rank into an accuracy requirement.
Theorem 5 (Tolerance and auxiliary bits). For ϵ > 0 \epsilon>0 ϵ > 0 , the minimum continuous auxiliary dimension for worst-history RMS error at most ϵ \epsilon ϵ is
m ϵ = # { i : σ i ( R Q P 0 ) > ϵ } . (10) m_\epsilon=\#\{i:\sigma_i(R_QP_0)>\epsilon\}. \tag*{(10)} m ϵ = # { i : σ i ( R Q P 0 ) > ϵ } . (10) If instead the auxiliary message has at most 2 b 2^b 2 b values, even a discontinuous encoder requires
b ≥ ∑ i log 2 + ( σ i ( R Q P 0 ) ϵ ) . (11) b\geq\sum_i\log_2^+\left(\frac{\sigma_i(R_QP_0)}{\epsilon}\right). \tag*{(11)} b ≥ i ∑ log 2 + ( ϵ σ i ( R Q P 0 ) ) . (11) The old state is supplied exactly in both statements.
The bit bound follows by covering the conditional target ellipsoid with 2 b 2^b 2 b radius-ϵ \epsilon ϵ balls. Appendix § 19 gives the proof and an explicit quantized upper bound. Per-query error ϵ \epsilon ϵ implies the RMS condition, but the converse need not hold. Neither statement identifies a finite-bit, causal optimum over a continuum of queries.
Exact recovery can be much more demanding. A contractive scalar recurrence h s ( a ) = ( λ + a ) h s − 1 ( a ) + u s h_s(a)=(\lambda+a)h_{s-1}(a)+u_s h s ( a ) = ( λ + a ) h s − 1 ( a ) + u s can require t − 1 t-1 t − 1 extra coordinates to recover every allowed target exactly. Evaluations at t t t distinct parameters recover the entire history through a Vandermonde system (Proposition 9 ). Small conditional singular values can nevertheless make far fewer coordinates sufficient at a nonzero tolerance.
Structured memory and output preservation
For time-invariant matrices the smallest A 0 A_0 A 0 -invariant space containing the update images is
V ∗ = span { A 0 j im ( A i ) : 0 ≤ j < d , 1 ≤ i ≤ p } . (12) \mathcal{V}_*=\operatorname{span}\{A_0^j\operatorname{im}(A_i):0\leq j<d,\ 1\leq i\leq p\}. \tag*{(12)} V ∗ = span { A 0 j im ( A i ) : 0 ≤ j < d , 1 ≤ i ≤ p } . (12)
Its dimension can be d d d for a rank-one update. Conversely, a low-dimensional space permits coefficient propagation without realizing the entire lift. This construction is sufficient even when the retained coefficients contain redundant information.
A fixed-anchor coefficient memory
Consider the possibly input-dependent affine recurrence
h t ( a ) = ( A 0 , t + ∑ i a i U D i , t ) h t − 1 ( a ) + b t , ∥ a ∥ 1 ≤ R , (13) h_t(a)=\left(A_{0,t}+\sum_i a_iUD_{i,t}\right)h_{t-1}(a)+b_t,\qquad\lVert a\rVert_1\leq R, \tag*{(13)} h t ( a ) = ( A 0 , t + i ∑ a i U D i , t ) h t − 1 ( a ) + b t , ∥ a ∥ 1 ≤ R , (13)
where U ∈ R d × r U\in\mathbb{R}^{d\times r} U ∈ R d × r is orthonormal and fixed before the stream. The matrices and injections may depend causally on inputs, but not on a a a . Write
T t = U ⊤ A 0 , t U , G i , t = D i , t U , E t = A 0 , t U − U T t . (14) T_t=U^\top A_{0,t}U,\qquad G_{i,t}=D_{i,t}U,\qquad E_t=A_{0,t}U-UT_t. \tag*{(14)} T t = U ⊤ A 0 , t U , G i , t = D i , t U , E t = A 0 , t U − U T t . (14)
For I k = { α : 1 ≤ ∣ α ∣ ≤ k } \mathcal{I}_k=\{\alpha:1\le|\alpha|\le k\} I k = { α : 1 ≤ ∣ α ∣ ≤ k } , maintain an anchor and reduced polynomial coefficients, initialized to zero,
h t 0 = A 0 , t h t − 1 0 + b t , (15) h_t^0=A_{0,t}h_{t-1}^0+b_t, \tag*{(15)} h t 0 = A 0 , t h t − 1 0 + b t , (15)
z t , α = T t z t − 1 , α + ∑ i : α i > 0 { D i , t h t − 1 0 , α = e i , G i , t z t − 1 , α − e i , ∣ α ∣ > 1. (16) z_{t,\alpha}=T_tz_{t-1,\alpha}+\sum_{i:\alpha_i>0}
\begin{cases}
D_{i,t}h_{t-1}^0, & \alpha=e_i,\\
G_{i,t}z_{t-1,\alpha-e_i}, & |\alpha|>1.
\end{cases} \tag*{(16)} z t , α = T t z t − 1 , α + i : α i > 0 ∑ { D i , t h t − 1 0 , G i , t z t − 1 , α − e i , α = e i , ∣ α ∣ > 1. (16)
Every right-hand side uses quantities from the previous time step. The resulting update-closed memory (UCM) answers a query with
h ^ t ( k ) ( a ) = h t 0 + U ∑ α ∈ I k a α z t , α . (17) \widehat{h}_t^{(k)}(a)=h_t^0+U\sum_{\alpha\in\mathcal{I}_k}a^\alpha z_{t,\alpha}. \tag*{(17)} h t ( k ) ( a ) = h t 0 + U α ∈ I k ∑ a α z t , α . (17)
Under exact closure, E t = 0 E_t=0 E t = 0 , these are the exact coefficients of a α a^\alpha a α . The array uses r N p , k rN_{p,k} r N p , k scalars, where N p , k = ( p + k p ) − 1 N_{p,k}=\binom{p+k}{p}-1 N p , k = ( p p + k ) − 1 , and degree k ≥ t − 1 k\ge t-1 k ≥ t − 1 recovers the state exactly (Proposition 11 ). These are derivatives divided by α ! \alpha! α ! , and mixed coefficients include every ordering of noncommuting matrices. The array can exceed the minimum ν k \nu_k ν k . For a scalar anchor block and scalar input, different noncommuting update words can share the same history moments, as shown in Appendix § 21 .§ 2 .
Beyond an independently maintained active d d d -state model, generic auxiliary storage is d + r N p , k d+rN_{p,k} d + r N p , k scalars. The first d d d store the anchor and incur additional storage after a deployment change. Exact closure allows sharing that reduces extra storage to r + r N p , k r+rN_{p,k} r + r N p , k . A scalar certificate adds one coordinate. For fixed dense matrices, maintenance beyond the anchor costs O ( p r d + p r 2 N p , k + r N p , k ) O(prd+pr^2N_{p,k}+rN_{p,k}) O ( p r d + p r 2 N p , k + r N p , k ) operations and a query costs O ( r N p , k + d r + p N p , k ) O(rN_{p,k}+dr+pN_{p,k}) O ( r N p , k + d r + p N p , k ) . Preprocessing, input-dependent reductions, and shared calibration are charged separately in Appendix § 22 .
Finite-radius error and a streaming certificate
Assume
∥ A 0 , t ∥ ≤ ρ < 1 , max i ∥ D i , t ∥ ≤ L , ∥ b t ∥ ≤ B , ∥ E t ∥ ≤ δ , (18) \lVert A_{0,t}\rVert\le\rho<1,\qquad\max_i\lVert D_{i,t}\rVert\le L,\qquad\lVert b_t\rVert\le B,\qquad\lVert E_t\rVert\le\delta, \tag*{(18)} ∥ A 0 , t ∥ ≤ ρ < 1 , i max ∥ D i , t ∥ ≤ L , ∥ b t ∥ ≤ B , ∥ E t ∥ ≤ δ , (18)
and set κ = ρ + L R < 1 \kappa=\rho+LR<1 κ = ρ + L R < 1 and q = L R / ( 1 − ρ ) q=LR/(1-\rho) q = L R / ( 1 − ρ ) . These norm assumptions are stronger than eigenvalue stability [16 ] .
Theorem 6 (Truncation and leakage). Uniformly in the horizon and in ∥ a ∥ 1 ≤ R \lVert a\rVert_1\le R ∥ a ∥ 1 ≤ R ,
∥ h t ( a ) − h ^ t ( k ) ( a ) ∥ ≤ B 1 − κ q k + 1 + δ L R B ( 1 − ρ ) ( 1 − κ ) 2 . (19) \left\lVert h_t(a)-\widehat{h}_t^{(k)}(a)\right\rVert\le\frac{B}{1-\kappa}q^{k+1}+\frac{\delta LRB}{(1-\rho)(1-\kappa)^2}. \tag*{(19)} h t ( a ) − h t ( k ) ( a ) ≤ 1 − κ B q k + 1 + ( 1 − ρ ) ( 1 − κ ) 2 δ L R B . (19) The first term is a geometric coefficient tail. The second bounds the error of the untruncated projected response. Increasing degree leaves the leakage term unchanged. This upper bound does not imply an unavoidable positive error floor.
For a path-dependent bound that can be tighter, define
M t , ℓ = max ∣ α ∣ = ℓ ∥ z t , α ∥ ( ℓ α ) . (20) M_{t,\ell}=\max_{|\alpha|=\ell}\frac{\lVert z_{t,\alpha}\rVert}{\binom{\ell}{\alpha}}. \tag*{(20)} M t , ℓ = ∣ α ∣ = ℓ max ( α ℓ ) ∥ z t , α ∥ . (20)
Choose upper bounds κ t \kappa_t κ t on the family transition norm, δ t \delta_t δ t on ∥ E t ∥ \lVert E_t\rVert ∥ E t ∥ , and ℓ t \ell_t ℓ t on max i ∥ G i , t ∥ \max_i\lVert G_{i,t}\rVert max i ∥ G i , t ∥ .
Theorem 7 (Radius-uniform residual certificate). With c 0 = 0 c_0=0 c 0 = 0 , the causal update
c t = κ t c t − 1 + δ t ∑ j = 1 k R j M t − 1 , j + ℓ t R k + 1 M t − 1 , k (21) c_t=\kappa_tc_{t-1}+\delta_t\sum_{j=1}^{k}R^jM_{t-1,j}+\ell_tR^{k+1}M_{t-1,k} \tag*{(21)} c t = κ t c t − 1 + δ t j = 1 ∑ k R j M t − 1 , j + ℓ t R k + 1 M t − 1 , k (21)
The one-step defect consists of retained-response leakage and an omitted degree-( k + 1 ) (k+1) ( k + 1 ) term. Propagating it gives an exact-arithmetic guarantee, not validated floating-point certification. Each later query uses the original memory and its current bound, so earlier migration errors do not accumulate in that memory. Changing the anchor, injection, or update family is a different problem. Contraction also helps replay. If H = B / ( 1 − κ ) H = B/(1-\kappa) H = B / ( 1 − κ ) , replaying a retained suffix of length L b u f L_{\mathrm{buf}} L buf from zero has error at most H κ L b u f H\kappa^{L_{\mathrm{buf}}} H κ L buf . Appendix § 19 .§ 4 compares this sufficient buffer budget with (19) . Degree must be provisioned before the stream. Selecting a lower-degree answer from a previously maintained larger array does not retrospectively save memory or maintenance.
Preserving future outputs
Current-output equality does not imply future-output equality. For a fixed A A A , C C C , the kernel of O A , C = [ C T , ( C A ) T , … , ( C A d − 1 ) T ] T \mathcal{O}_{A,C} = [C^{\mathsf{T}}, (CA)^{\mathsf{T}}, \ldots, (CA^{d-1})^{\mathsf{T}}]^{\mathsf{T}} O A , C = [ C T , ( C A ) T , … , ( C A d − 1 ) T ] T identifies states with equal future outputs under common inputs. Theorem 4 applies to the corresponding stacked output maps.
A causal reduction valid under later parameter switches requires a common quotient. If Q Q Q has orthonormal rows, Q A i = A ˉ i Q QA_i = \bar{A}_iQ Q A i = A ˉ i Q for every i = 0 , … , p i = 0,\ldots,p i = 0 , … , p , and C = C ˉ Q C = \bar{C}Q C = C ˉ Q , then Q h t Qh_t Q h t is an exact recurrent state for those outputs. Under update closure U U U , only rank ( Q U ) N p , k \operatorname{rank}(QU)N_{p,k} rank ( Q U ) N p , k coefficient scalars are needed (Corollary 14). Row-space closure constructs Q Q Q before streaming. This does not cover arbitrary nonlinear heads. Appendix § 16 also bounds cumulative predictive KL under common future inputs, not ground-truth loss or closed-loop generated trajectories.
Experiments
The experiments separate exact local dimension, finite-tolerance fidelity, predictive usefulness, and complete cost. Each reported mean and sample SD aggregates within a fitted-model seed first. Streams and updates are paired observations within a seed, not independent training runs. The source, retained measurements, and new diagnostics are included with the experimental materials.
Dimension is not a tolerance budget
The four-state rank-one example has the k d = 8 kd = 8 k d = 8 local requirement for k = 2 k = 2 k = 2 . At horizon 12 and query radius 0.2, however, it needs 0, 1, 1, 3, 5 continuous coordinates at RMS tolerances 10 − 2 10^{-2} 1 0 − 2 , 10 − 3 10^{-3} 1 0 − 3 , 10 − 4 10^{-4} 1 0 − 4 , 10 − 6 10^{-6} 1 0 − 6 , 10 − 8 10^{-8} 1 0 − 8 . In particular, one coordinate suffices at 10 − 4 10^{-4} 1 0 − 4 . The spectrum, precision bounds, and normalization are given in Appendix § 24 .§ 1 . Independent null-space and projector calculations agree within 1.1 × 10 − 16 1.1 \times10^{-16} 1.1 × 1 0 − 16 .
A separate dense noncommuting study has d = 24 d = 24 d = 24 , r = 4 r = 4 r = 4 , p = 2 p = 2 p = 2 , five model seeds, and 21,600 queries spanning degree, radius, leakage, and history length. Observed small-radius slopes are 2.00, 2.99, 3.99, 4.99 for degrees one through four. There are no sampled certificate violations with the declared 10 − 10 10^{-10} 1 0 − 10 tolerance, but leakage can prevent certification at every retained degree. Appendix § 24 .8 reports this failure rate and certificate conservatism. These finite numerical checks do not establish validated floating-point certification.
Equal-memory language migration
Five trained 64-state byte models undergo eight learned replacements on each of eight held-out streams. The family modifies eight separable coordinates through two fixed directions. Calibration, training, tuning, and evaluation histories are disjoint. A direct input-to-logit path weakens dependence on the migrated state. This experiment tests state reconstruction, not a general need for migration in language modeling (Appendix § 23 .§ 2 ).
The cubic Chebyshev bank is fixed before queries and matches degree-three coefficients’ 256-byte budget. Mean state NRMSE is 5.30 × 10 − 4 5.30 \times10^{-4} 5.30 × 1 0 − 4 , versus 7.49 × 10 − 4 7.49 \times10^{-4} 7.49 × 1 0 − 4 for degree three, with lower error in 3 of five seeds. Neither method changes predictive loss meaningfully. Recorded state-only stream medians are 1.163 ms for the bank and 1.123 ms for coefficients, with IQRs [1.152, 1.201] and [1.098, 1.209] ms. The overlapping IQRs do not establish a reliable speed ordering. Appendix § 26 .§ 5 shows the paired seed outcomes.
Using fitted seeds as the paired units gives a two-sided t t t -test p = 0.475 p = 0.475 p = 0.475 and a sign-flip p = 0.75 p = 0.75 p = 0.75 . These five models do not establish an interpolation advantage.
Useful adaptation and competing history representations
The coupled study fits a 128-state scalar-input model with a known injection and fixed known readout, without a direct input skip. Full-state transitions identify an anchor. Six independent pilot domains determine two update directions and an eight-dimensional image through a fixed singular-value energy rule. Four new domains then provide independent calibration data for fitting replacements. Matrices are noncommuting and mixed coefficients are nonzero. The image is estimated, but the synthetic true family deliberately occupies a slow invariant block and the readout lies in that block. The task is retrospective identification with observed calibration states, not latent-state fine-tuning.
Each of five seeds has eight held-out streams of 4096 inputs, with replacements every 512 inputs. The resulting 280 events per method are evaluated over 32 subsequent positions. Task NMSE measures squared prediction error against each generating model’s clean conditional mean over the full prefix. Restricted exact replay reaches 2.50 × 10 − 9 2.50 \times10^{-9} 2.50 × 1 0 − 9 NMSE, close to unrestricted fits at 2.01 × 10 − 9 2.01 \times10^{-9} 2.01 × 1 0 − 9 using the same calibration data. The permitted family therefore retains the useful adaptation available to the unrestricted fit in this setting.
Stale reuse has mean NMSE 0.375, compared with 0.134 without adaptation. Degree three reaches 2.15 × 10 − 5 2.15 \times10^{-5} 2.15 × 1 0 − 5 . Seed 4 contributes 95.7% of the summed degree-three seed NMSE, making the paired outcomes important for interpreting the mean. The analytic Laguerre history bank ([20 ] ) matches replay’s prediction error at the same 1600-byte budget and is faster on the measured stream. Its shared decoder depends on the replacement dynamics and injection, but not on the readout or prediction labels. It requires setup but no historical replay. Appendix § 24 .§ 3 gives the basis, exact-arithmetic residual formula, and complete calibration protocol.
Table 3: Coupled task. Prediction NMSE is mean ± SD across five fitted seeds. Extra storage excludes the common 1024-byte active state but includes a retained anchor when required. Times include one complete stream, seven migrations, initialization, and readouts. Shared setup is separate. A dash is untimed, not free.
Table 3.
How much of the history bank is needed?
We evaluate the prespecified widths 1, 2, 4, 8, 16, 32, 64, 128, 200, 400, 800 on the same five fitted systems. Each bank processes the stream before targets are revealed, with pole fixed at 0.995. Eight float64 coordinates, or 64 extra bytes, suffice for output NRMSE below 10 − 4 10^{-4} 1 0 − 4 in every seed, while mean full-state NRMSE remains 10.7%. Output fidelity here is relative to the fitted replacement’s exact replay, rather than the clean target used for task NMSE. Table 4 uses pooled event-window error within each seed and requires the tolerance in all five seeds, not at every individual event.
Table 4: Smallest tested history-bank width meeting each tolerance in every fitted seed. “Repeated” and “split” refer to the true slow poles. Each coordinate costs eight per-stream bytes. These are empirical widths for one prescribed basis, not minimum-memory theorems.
Table 4.
To test the repeated-pole advantage, a paired structural variant replaces the true slow block 0.995 I 8 0.995I_{8} 0.995 I 8 by eight poles equally spaced from 0.955 to 0.995. All models and update families are refitted from independent calibration data using the same procedure and seeds. Adaptation remains useful, but the eight-coordinate bank’s maximum seed output NRMSE rises to 6.56 × 10 − 2 6.56 \times10^{-2} 6.56 × 1 0 − 2 . Among tested widths, 64 coordinates are now needed at 10 − 4 10^{-4} 1 0 − 4 . This deterioration concerns the prescribed history basis, not every migration method. Degree-three coefficients have mean task NMSE 3.54 × 10 − 9 3.54 \times10^{-9} 3.54 × 1 0 − 9 in the variant. The full protocol and all widths appear in Appendix § 26 .§ 2 .
At output NRMSE tolerance 10 − 3 10^{-3} 1 0 − 3 , degree-three memory and a 32-coordinate bank both qualify, using 1600 and 256 extra bytes per stream. At 10 − 4 10^{-4} 1 0 − 4 , degree three fails the all-five-seed criterion and a 64-coordinate bank succeeds. Appendix § 26 .§ 3 reports the stitching comparison and shared-storage costs.
Maintenance, regimes, and numerical realization
Whole-stream times include every input, retained-memory maintenance, migrations, initialization, and readouts. In the coupled instance, 21 timing repetitions give medians of 24.67 ms for degree three, 30.50 ms for replay, and 8.78 ms for the 200-coordinate analytic bank. Its four decoders occupy 819,200 shared bytes and take 4.83 ms to construct, versus 0.57 ms for coefficient reductions. The timings use one fitted model and one stream, not five timing seeds. Separately measured medians cannot be added into a joint latency quantile. The new width sweep makes no new runtime claim.
Appendix § 24 .10 gives total cost as a function of stream count and query frequency, including shared setup and decoder storage.
Appendix § 24 .7 reports the contraction and update-frequency sweeps. Relative leakage, history exposure, and domain mixture also change across settings. The comparisons are therefore descriptive rather than isolated effects of either factor. On the two-state example, an orthogonal reachable-space realization uses the exact local minimum of two, four, or six extra coordinates at degrees one through three. Its maximum polynomial-response NRMSE is 4.79 × 10 − 8 4.79 \times10^{-8} 4.79 × 1 0 − 8 in float32 and 2.49 × 10 − 16 2.49 \times10^{-16} 2.49 × 1 0 − 16 in float64. Appendix § 26 .§ 4 reports exact ranks, numerical rank thresholds, closure residuals, and conditioning. This small fixed-family check does not supply a general finite-precision theorem for the direction-independent construction.
Appendix § 24 .11 bounds accumulated arithmetic error for a specified realization. The bound identifies conditioning quantities to assess before choosing a minimal rather than a larger representation.
Discussion
The causal coefficient space identifies what must be remembered for exact local migration. For controllable scalar-input models, unknown arbitrary matrix directions and a suitable rank-one direction have the same sharp worst-case k d kd k d cost. Neither coefficient-array size nor update rank alone determines the number of independent historical distinctions.
Representation choice remains separate. Interpolation, coefficients, and orthogonal history banks have different maintenance and approximation costs. The experiments show how the readout and pole structure affect that comparison. The bit bound conditions on an exact old state; residual certificates concern exact arithmetic; general upstream updates can invalidate the injection model. The language task shows negligible predictive benefit, and both coupled variants retain full-state calibration and planted structure. A memory-width study on a naturally trained linear recurrent model with transition-only rank-one adaptation remains future work. The information theorem remains an exact local result; practical migration must also account for these restrictions.
Ethics statement
This study concerns preservation of a model’s computation, not a guarantee that its outputs are accurate, fair, or safe. A replay-free summary is not necessarily privacy-preserving and should not be treated as a substitute for deletion or access-control policies. The experiments use synthetic numerical streams and an existing small news-text corpus, with no newly recruited human participants. The underlying news articles are not redistributed. No deployment in a consequential decision system is evaluated.
AI use statement
Generative AI was used for grammatical checks and proofreading. It was also used for reference/literature search and verification. The author assumes full responsibility for the content.
Reproducibility statement
The supplement contains the implementations, trained models, numerical results, and calibration data. Complete proofs and experimental protocols are given in the appendices.
References
Information limits with an existing state
The appendices give proofs of every theoretical statement in the main text. The encoder continuity assumption is imposed on the auxiliary history summary. Unless explicitly stated otherwise, no regularity is imposed on the decoder. The arguments therefore also apply to continuous online memories with arbitrary nonlinear migration rules. A fixed-horizon bound does not by itself imply that its attaining summary can be maintained using bounded computation at earlier times.
The topological ingredient
We use the following standard form of the Borsuk–Ulam theorem [15 ] .
Lemma 8 (Borsuk–Ulam). For every m ≥ 0 m \ge0 m ≥ 0 and every continuous map f : S m → R m f:S^m \to\mathbb{R}^m f : S m → R m , there is v ∈ S m v \in S^m v ∈ S m such that f ( v ) = f ( − v ) f(v) = f(-v) f ( v ) = f ( − v ) .
For m = 0 m = 0 m = 0 , the codomain is the singleton R 0 \mathbb{R}^0 R 0 , so the assertion holds for the two points of S 0 S^0 S 0 . We use this classical theorem without reproving it and give all remaining steps below.
Proof of Theorem 4
Let r 0 = rank ( R 0 ) r_0 = \operatorname{rank}(R_0) r 0 = rank ( R 0 ) and M = R P 0 M = RP_0 M = R P 0 . The Moore–Penrose identities imply that R 0 † R 0 R_0^\dagger R_0 R 0 † R 0 is the orthogonal projector onto ( ker R 0 ) ⊥ (\ker R_0)^\perp ( ker R 0 ) ⊥ , so P 0 P_0 P 0 is the orthogonal projector onto ker R 0 \ker R_0 ker R 0 . Every history decomposes as
u = R 0 † R 0 u + P 0 u , R u = R R 0 † R 0 u + M u . (22) u = R_0^\dagger R_0u + P_0u,\qquad Ru = RR_0^\dagger R_0u + Mu. \tag*{(22)} u = R 0 † R 0 u + P 0 u , R u = R R 0 † R 0 u + M u . (22)
The first target component is therefore exactly recoverable from the old state without an auxiliary sketch.
Take a singular value decomposition M = ∑ j = 1 r M σ j w j v j ⊤ M = \sum_{j=1}^{r_M} \sigma_j w_jv_j^\top M = ∑ j = 1 r M σ j w j v j ⊤ , where r M = rank ( M ) r_M = \operatorname{rank}(M) r M = rank ( M ) and σ 1 ≥ ⋯ ≥ σ r M > 0 \sigma_1 \ge\cdots\ge\sigma_{r_M} > 0 σ 1 ≥ ⋯ ≥ σ r M > 0 . Put s = min ( m , r M ) s = \min(m,r_M) s = min ( m , r M ) and choose
S ( u ) = ( v 1 ⊤ u , … , v s ⊤ u , 0 , … , 0 ) ∈ R m , D ( y , z ) = R R 0 † y + ∑ j = 1 s σ j w j z j . (23) S(u) = (v_1^\top u,\ldots,v_s^\top u,0,\ldots,0) \in\mathbb{R}^m,\qquad D(y,z) = RR_0^\dagger y + \sum_{j=1}^{s} \sigma_j w_jz_j. \tag*{(23)} S ( u ) = ( v 1 ⊤ u , … , v s ⊤ u , 0 , … , 0 ) ∈ R m , D ( y , z ) = R R 0 † y + j = 1 ∑ s σ j w j z j . (23)
This sketch and decoder are linear. Their error is
R u − D ( R 0 u , S ( u ) ) = ∑ j > s σ j w j v j ⊤ u . (24) Ru - D(R_0u,S(u)) = \sum_{j>s} \sigma_j w_jv_j^\top u. \tag*{(24)} R u − D ( R 0 u , S ( u )) = j > s ∑ σ j w j v j ⊤ u . (24)
Its supremum on the unit ball is σ m + 1 \sigma_{m+1} σ m + 1 under the zero-padding convention. This proves attainability, including the case m ≥ r M m \ge r_M m ≥ r M .
For the lower bound, it suffices to consider σ m + 1 > 0 \sigma_{m+1} > 0 σ m + 1 > 0 . A positive right-singular vector of M M M lies in ker R 0 \ker R_0 ker R 0 , because M ⊤ M = P 0 R ⊤ R P 0 M^\top M = P_0R^\top RP_0 M ⊤ M = P 0 R ⊤ R P 0 . Define the ( m + 1 ) (m+1) ( m + 1 ) -dimensional subspace V = span ( v 1 , … , v m + 1 ) ⊆ ker R 0 V = \operatorname{span}(v_1,\ldots,v_{m+1}) \subseteq\ker R_0 V = span ( v 1 , … , v m + 1 ) ⊆ ker R 0 . For an arbitrary continuous sketch, its restriction to the unit sphere of V V V is a continuous map to R m \mathbb{R}^m R m . Lemma 8 provides u ∈ V u \in V u ∈ V with ∥ u ∥ = 1 \lVert u\rVert= 1 ∥ u ∥ = 1 and S ( u ) = S ( − u ) S(u) = S(-u) S ( u ) = S ( − u ) . Both histories have old state zero. Thus, if y = D ( 0 , S ( u ) ) y = D(0,S(u)) y = D ( 0 , S ( u )) , the triangle inequality gives
max { ∥ R u − y ∥ , ∥ − R u − y ∥ } ≥ 1 2 ∥ ( R u − y ) − ( − R u − y ) ∥ = ∥ R u ∥ ≥ σ m + 1 ( M ) . (25) \max\{\lVert Ru-y\rVert,\lVert-Ru-y\rVert\} \ge\frac{1}{2}\lVert(Ru-y)-(-Ru-y)\rVert= \lVert Ru\rVert\ge\sigma_{m+1}(M). \tag*{(25)} max {∥ R u − y ∥ , ∥ − R u − y ∥} ≥ 2 1 ∥( R u − y ) − ( − R u − y )∥ = ∥ R u ∥ ≥ σ m + 1 ( M ) . (25)
The last inequality follows from u ∈ V u \in V u ∈ V and P 0 u = u P_0u = u P 0 u = u . This holds for every decoder, whether or not it is continuous or measurable, and proves (8).
Exact recovery is possible precisely when m ≥ rank ( M ) m \ge\operatorname{rank}(M) m ≥ rank ( M ) . To identify this rank, restrict R R R to ker R 0 \ker R_0 ker R 0 . Rank–nullity on that subspace gives
rank ( R P 0 ) = dim ker R 0 − dim ( ker R 0 ∩ ker R ) (26) \operatorname{rank}(RP_0) = \dim\ker R_0 - \dim(\ker R_0 \cap\ker R) \tag*{(26)} rank ( R P 0 ) = dim ker R 0 − dim ( ker R 0 ∩ ker R ) (26)
= ( n − r 0 ) − ( n − rank [ R 0 R ] ) , (27) = (n-r_0) - \left(n-\operatorname{rank}\begin{bmatrix} R_0 \\ R \end{bmatrix}\right), \tag*{(27)} = ( n − r 0 ) − ( n − rank [ R 0 R ] ) , (27)
which is (9) . This completes the proof. □
Scope of the information bound. The lower bound is deterministic and worst-case. If a random sketch must satisfy the guarantee for every realization, the same argument applies after conditioning on that realization. The theorem does not establish the same formula for expected error under randomized encoders, distribution-dependent loss, or discontinuous finite-bit encodings. For a target stack R = ( R 1 ⊤ , … , R K ⊤ ) ⊤ R = (R_1^\top,\ldots,R_K^\top)^\top R = ( R 1 ⊤ , … , R K ⊤ ) ⊤ , the error is the square root of the sum of squared target errors. Replacing this by a maximum across targets changes the optimization problem and is not covered by the singular-value formula.
The no-memory case has a simple exact interpretation. Setting m = 0 m=0 m = 0 in (9) , exact recovery of R u Ru R u from R 0 u R_0u R 0 u alone is possible if and only if ker R 0 ⊆ ker R \ker R_0 \subseteq\ker R ker R 0 ⊆ ker R . Equivalently, R = L R 0 R=LR_0 R = L R 0 for some linear L L L . Thus allowing a nonlinear decoder does not improve exact recovery over unrestricted linear histories. On a restricted data distribution, a learned nonlinear decoder can still improve average error, as the experiments illustrate.
Proof of Proposition 9
Proposition 9 (Exact recovery can retain the entire past). The scalar recurrence h s ( a ) = ( λ + a ) h s − 1 ( a ) + u s h_s(a)=(\lambda+a)h_{s-1}(a)+u_s h s ( a ) = ( λ + a ) h s − 1 ( a ) + u s , starting from zero, can be contractive on an arbitrarily narrow interval around zero while requiring t − 1 t-1 t − 1 additional continuous coordinates for exact recovery of h t ( a ) h_t(a) h t ( a ) at every permitted a a a .
Take λ = 1 / 2 \lambda=1/2 λ = 1/2 and I = [ − η , η ] I=[-\eta,\eta] I = [ − η , η ] for any 0 < η < 1 / 2 0<\eta<1/2 0 < η < 1/2 . Then ∣ λ + a ∣ ≤ 1 / 2 + η < 1 |\lambda+a|\le1/2+\eta<1 ∣ λ + a ∣ ≤ 1/2 + η < 1 for every permitted a a a , including intervals as narrow as desired. Unrolling the recurrence yields
h t ( a ) = ∑ s = 1 t ( λ + a ) t − s u s . (28) h_t(a)=\sum_{s=1}^{t}(\lambda+a)^{t-s}u_s. \tag*{(28)} h t ( a ) = s = 1 ∑ t ( λ + a ) t − s u s . (28)
Choose t t t distinct points a 1 , … , a t ∈ I a_1,\ldots,a_t\in I a 1 , … , a t ∈ I , one of which is 0 0 0 . The map from u u u to ( h t ( a 1 ) , … , h t ( a t ) ) (h_t(a_1),\ldots,h_t(a_t)) ( h t ( a 1 ) , … , h t ( a t )) has rows ( ( λ + a j ) t − 1 , … , λ + a j , 1 ) ((\lambda+a_j)^{t-1},\ldots,\lambda+a_j,1) (( λ + a j ) t − 1 , … , λ + a j , 1 ) . Reversing the columns gives a Vandermonde matrix. Its determinant, up to sign, is ∏ i < j ( a j − a i ) \prod_{i<j}(a_j-a_i) ∏ i < j ( a j − a i ) , which is nonzero. The stacked target map therefore has rank t t t , while the old-state row has rank one. Theorem 4 implies that any continuous summary supporting exact recovery for all a ∈ I a\in I a ∈ I needs at least t − 1 t-1 t − 1 extra coordinates, since it must recover these t t t particular queries.
The bound is attainable at that horizon by keeping u 1 , … , u t − 1 u_1,\ldots,u_{t-1} u 1 , … , u t − 1 and recovering u t u_t u t from the old state, whose coefficient on u t u_t u t is one. The size of this summary grows with t t t . Narrowing the interval can make the Vandermonde matrix poorly conditioned but does not change its exact rank. The proposition is therefore an exact-information obstruction, not a claim that approximate migration on a small interval always needs nearly the whole history. □
A local lower bound for rank-one updates
We first prove the controllability criterion used in the dimension argument.
Lemma 10 (A finite-dimensional controllability criterion). For T ∈ R N × N T \in\mathbb{R}^{N \times N} T ∈ R N × N and b ∈ R N b \in\mathbb{R}^{N} b ∈ R N , the matrix [ b , T b , … , T N − 1 b ] [b,Tb,\ldots,T^{N-1}b] [ b , T b , … , T N − 1 b ] has rank N N N if and only if there is no nonzero complex row vector w w w and complex scalar λ \lambda λ with w T = λ w wT=\lambda w w T = λ w and w b = 0 wb=0 w b = 0 .
Proof. If such a row exists, then w T j b = λ j w b = 0 wT^{j}b=\lambda^{j}wb=0 w T j b = λ j w b = 0 for every j j j , so the controllability matrix cannot have full rank. Conversely, suppose its column span K \mathcal{K} K is proper. By Cayley–Hamilton, T K ⊆ K T\mathcal{K}\subseteq\mathcal{K} T K ⊆ K . Its nonzero annihilator, after complexification, is invariant under the row-vector map w ↦ w T w\mapsto wT w ↦ w T . Every linear operator on a nonzero finite-dimensional complex vector space has an eigenvector. Choose such a row in the annihilator. It satisfies w b = 0 wb=0 w b = 0 and the desired eigenvector equation.
Proof of Theorem 2
Fix d , k ≥ 1 d,k\geq1 d , k ≥ 1 . Let A = diag ( λ 1 , … , λ d ) A=\operatorname{diag}(\lambda_1,\ldots,\lambda_d) A = diag ( λ 1 , … , λ d ) , with all λ i \lambda_i λ i distinct and in ( 0 , 1 ) (0,1) ( 0 , 1 ) . Choose v , b ∈ R d v,b\in\mathbb{R}^{d} v , b ∈ R d with every entry nonzero and set D = v v T D=vv^{\mathsf{T}} D = v v T . In particular, D D D has rank one and D i i = v i 2 > 0 D_{ii}=v_i^2>0 D ii = v i 2 > 0 . For any
0 < ε < 1 − ∥ A ∥ ∥ D ∥ , (29) 0<\varepsilon<\frac{1-\lVert A\rVert}{\lVert D\rVert}, \tag*{(29)} 0 < ε < ∥ D ∥ 1 − ∥ A ∥ , (29)
all transitions A + a D A+aD A + a D with ∣ a ∣ ≤ ε \lvert a\rvert\leq\varepsilon ∣ a ∣ ≤ ε have norm at most ∥ A ∥ + ε ∥ D ∥ < 1 \lVert A\rVert+\varepsilon\lVert D\rVert<1 ∥ A ∥ + ε ∥ D ∥ < 1 . Write H s , j ( u ) H_{s,j}(u) H s , j ( u ) for the coefficient of a j a^j a j in h s ( a ; u ) h_s(a;u) h s ( a ; u ) , with H s , 0 = h s ( 0 ; u ) H_{s,0}=h_s(0;u) H s , 0 = h s ( 0 ; u ) . Substitution into the recurrence gives
H s , 0 = A H s − 1 , 0 + b u s , (30) H_{s,0}=AH_{s-1,0}+bu_s, \tag*{(30)} H s , 0 = A H s − 1 , 0 + b u s , (30)
H s , j = A H s − 1 , j + D H s − 1 , j − 1 , 1 ≤ j ≤ k . (31) H_{s,j}=AH_{s-1,j}+DH_{s-1,j-1},\qquad1\leq j\leq k. \tag*{(31)} H s , j = A H s − 1 , j + D H s − 1 , j − 1 , 1 ≤ j ≤ k . (31)
The lift has the explicit form
T k = [ A 0 ⋯ 0 D A ⋱ ⋮ 0 ⋱ ⋱ 0 0 0 D A ] , b = ( b T , 0 , … , 0 ) T . (32) T_k=
\begin{bmatrix}
A & 0 & \cdots& 0\\
D & A & \ddots& \vdots\\
0 & \ddots& \ddots& 0\\
0 & 0 & D & A
\end{bmatrix},
\qquad
b=(b^{\mathsf{T}},0,\ldots,0)^{\mathsf{T}}. \tag*{(32)} T k = A D 0 0 0 A ⋱ 0 ⋯ ⋱ ⋱ D 0 ⋮ 0 A , b = ( b T , 0 , … , 0 ) T . (32)
Thus the stacked coefficients form a linear system with transition T k T_k T k and input b b b from (32) , of dimension N = d ( k + 1 ) N=d(k+1) N = d ( k + 1 ) . We prove this lifted system is controllable. Its eigenvalues are the λ i \lambda_i λ i , since it is block triangular with diagonal blocks A A A . Let w = ( w 0 , … , w k ) w=(w_0,\ldots,w_k) w = ( w 0 , … , w k ) be a left eigenvector at λ i \lambda_i λ i , with each w j w_j w j a row in C d \mathbb{C}^{d} C d . The block equations are
w j ( A − λ i I ) + w j + 1 D = 0 ( 0 ≤ j < k ) , w k ( A − λ i I ) = 0. (33) w_j(A-\lambda_i I)+w_{j+1}D=0\quad(0\leq j<k),\qquad w_k(A-\lambda_i I)=0. \tag*{(33)} w j ( A − λ i I ) + w j + 1 D = 0 ( 0 ≤ j < k ) , w k ( A − λ i I ) = 0. (33)
Distinctness of the eigenvalues gives w k = c k e i T w_k=c_ke_i^{\mathsf{T}} w k = c k e i T . Multiply the equation for j = k − 1 j=k-1 j = k − 1 on the right by e i e_i e i . Its first term is zero, so c k D i i = 0 c_kD_{ii}=0 c k D ii = 0 and hence w k = 0 w_k=0 w k = 0 . The last nontrivial equation then implies w k − 1 = c k − 1 e i T w_{k-1}=c_{k-1}e_i^{\mathsf{T}} w k − 1 = c k − 1 e i T . The equation one block earlier forces c k − 1 D i i = 0 c_{k-1}D_{ii}=0 c k − 1 D ii = 0 . Repeating gives w 1 = ⋯ = w k = 0 w_1=\cdots=w_k=0 w 1 = ⋯ = w k = 0 and w 0 = c 0 e i T w_0=c_0e_i^{\mathsf{T}} w 0 = c 0 e i T . Since w ≠ 0 w\neq0 w = 0 , c 0 ≠ 0 c_0\neq0 c 0 = 0 , and therefore w b = c 0 b i ≠ 0 wb=c_0b_i\neq0 w b = c 0 b i = 0 . Lemma 10 proves controllability.
At horizon N N N , the input-to-coefficient map is the reversed controllability matrix
( H N , 0 , … , H N , k ) = [ T k N − 1 b , … , b ] u , (34) (H_{N,0},\ldots,H_{N,k}) = [T_k^{N-1}\mathbf{b},\ldots,\mathbf{b}]u, \tag*{(34)} ( H N , 0 , … , H N , k ) = [ T k N − 1 b , … , b ] u , (34)
which is invertible. In particular, the old-state map has rank d d d and its kernel V V V has dimension N − d = k d N-d=kd N − d = k d .
Suppose a continuous sketch S : B 2 N → R m S:\mathbb{B}_2^N\to\mathbb{R}^m S : B 2 N → R m with m < k d m<kd m < k d supports (2). Choose any ( m + 1 ) (m+1) ( m + 1 ) -dimensional subspace of V V V . By Lemma 8, its unit sphere contains u u u with S ( u ) = S ( − u ) S(u)=S(-u) S ( u ) = S ( − u ) , and both histories have the same zero old state. Let y ( a ) y(a) y ( a ) be their common decoded estimate. The two local error guarantees imply
∥ h N ( a ; u ) − h N ( a ; − u ) ∥ ≤ ∥ h N ( a ; u ) − y ( a ) ∥ + ∥ h N ( a ; − u ) − y ( a ) ∥ = O ( ∣ a ∣ k + 1 ) . (35) \lVert h_N(a;u)-h_N(a;-u)\rVert\le\lVert h_N(a;u)-y(a)\rVert+\lVert h_N(a;-u)-y(a)\rVert=O(|a|^{k+1}). \tag*{(35)} ∥ h N ( a ; u ) − h N ( a ; − u )∥ ≤ ∥ h N ( a ; u ) − y ( a )∥ + ∥ h N ( a ; − u ) − y ( a )∥ = O ( ∣ a ∣ k + 1 ) . (35)
Because the recurrence is linear in the history, the left difference is 2 h N ( a ; u ) 2h_N(a;u) 2 h N ( a ; u ) . Invertibility of the coefficient map implies that a nonzero u ∈ V u\in V u ∈ V has some nonzero coefficient among H N , 1 , … , H N , k H_{N,1},\ldots,H_{N,k} H N , 1 , … , H N , k . Let j ≤ k j\le k j ≤ k be the smallest such order. Then a − j h N ( a ; u ) → H N , j ( u ) ≠ 0 a^{-j}h_N(a;u)\to H_{N,j}(u)\ne0 a − j h N ( a ; u ) → H N , j ( u ) = 0 as a → 0 a\to0 a → 0 , contradicting (35). This proves m ≥ k d m\ge kd m ≥ k d without requiring differentiability of S S S or any regularity of D a D_a D a .
Finally, retaining the k k k coefficient vectors and returning h N ( 0 ; u ) + ∑ j = 1 k a j H N , j ( u ) h_N(0;u)+\sum_{j=1}^{k}a^jH_{N,j}(u) h N ( 0 ; u ) + ∑ j = 1 k a j H N , j ( u ) attains order-k k k local accuracy with k d kd k d extra scalars. For fixed N N N , h N ( a ; u ) h_N(a;u) h N ( a ; u ) is a finite polynomial, so its remaining terms are O ( ∣ a ∣ k + 1 ) O(|a|^{k+1}) O ( ∣ a ∣ k + 1 ) . This finishes the proof. □
Sharpness and scope. For each d , k d,k d , k , the theorem constructs a family attaining the bound. It does not require k d kd k d additional coordinates for every rank-one family or make multivariate coefficient arrays minimal. Rank one, scalar parameterization, and uniform contraction are jointly insufficient to reduce the worst-case k d kd k d local-information cost. The input is real-valued and ranges over an open ball. Discrete language histories do not inherit this continuous-domain lower bound automatically.
Invariant closure and the streaming construction
The smallest structural invariant space
For time-invariant matrices A 0 , A 1 , … , A p A_0,A_1,\ldots,A_p A 0 , A 1 , … , A p , define V ∗ V_* V ∗ by (12). Every im ( A i ) \operatorname{im}(A_i) im ( A i ) is contained in V ∗ V_* V ∗ . Applying A 0 A_0 A 0 to a generator with exponent j < d − 1 j<d-1 j < d − 1 produces another generator. Applying it at exponent d − 1 d-1 d − 1 produces A 0 d im ( A i ) A_0^d\operatorname{im}(A_i) A 0 d im ( A i ) , which is contained in V ∗ V_* V ∗ by Cayley–Hamilton. Therefore V ∗ V_* V ∗ is A 0 A_0 A 0 -invariant. Conversely, any A 0 A_0 A 0 -invariant subspace containing every im ( A i ) \operatorname{im}(A_i) im ( A i ) contains A 0 j im ( A i ) A_0^j\operatorname{im}(A_i) A 0 j im ( A i ) for all j j j , hence contains V ∗ V_* V ∗ . This proves the claimed minimality.
Choose an orthonormal basis U U U of V ∗ V_* V ∗ . Then A 0 U = U ( U ⊤ A 0 U ) A_0U=U(U^\top A_0U) A 0 U = U ( U ⊤ A 0 U ) and A i = U ( U ⊤ A i ) A_i=U(U^\top A_i) A i = U ( U ⊤ A i ) . Thus the family has the form (13) with D i = U ⊤ A i D_i=U^\top A_i D i = U ⊤ A i and zero leakage. This construction gives a sufficient space for all possible anchor states. It may exceed what is necessary when injections visit only a restricted reachable set or when only certain outputs are relevant. For time-varying families, a common invariant space may instead require closure under several anchor transitions. Formula (12) is asserted only for the time-invariant case.
Proof of Proposition 11
Proposition 11 (Exact retained coefficients. ) If E t = 0 E_t=0 E t = 0 at every time, U z , α U_{z,\alpha} U z , α is the exact coefficient of a α a^\alpha a α in h t ( a ) h_t(a) h t ( a ) . The coefficient array uses r N p , k rN_{p,k} r N p , k scalars, where N p , k = ( p + k k ) − 1 N_{p,k}=\binom{p+k}{k}-1 N p , k = ( k p + k ) − 1 . At horizon t t t , degree k ≥ t − 1 k\ge t-1 k ≥ t − 1 gives exact recovery.
For a fixed finite history, the state in (13) is a polynomial in a a a . Since the initial state is zero and b t b_t b t is parameter-independent, its degree is at most t − 1 t-1 t − 1 . Write
h t ( a ) = h t 0 + ∑ α ≠ 0 a α H t , α (36) h_t(a)=h_t^0+\sum_{\alpha\ne0}a^\alpha H_{t,\alpha} \tag*{(36)} h t ( a ) = h t 0 + α = 0 ∑ a α H t , α (36)
with nonexistent coefficients understood as zero. Matching powers in (13) gives the constant recurrence (15) and
H t , α = A 0 , t H t − 1 , α + { U D i , t h t − 1 0 , α = e i , U D i , t H t − 1 , α − e i , ∣ α ∣ > 1. (37) H_{t,\alpha}=A_{0,t}H_{t-1,\alpha}+
\begin{cases}
UD_{i,t}h_{t-1}^0, & \alpha=e_i,\\
UD_{i,t}H_{t-1,\alpha-e_i}, & |\alpha|>1.
\end{cases} \tag*{(37)} H t , α = A 0 , t H t − 1 , α + { U D i , t h t − 1 0 , U D i , t H t − 1 , α − e i , α = e i , ∣ α ∣ > 1. (37)
All positive-degree coefficients start at zero. Assume inductively that H t − 1 , α = U z t − 1 , α H_{t-1,\alpha}=Uz_{t-1,\alpha} H t − 1 , α = U z t − 1 , α . If E t = 0 E_t=0 E t = 0 , then A 0 , t U = U T t A_{0,t}U=UT_t A 0 , t U = U T t , and substituting into (37) gives exactly U U U times (16) . This proves the identity at time t t t . The induction applies independently of commutativity and does not depend on higher-degree coefficients. Consequently, truncating storage at degree k k k leaves every retained coefficient exact.
The number of multi-indices of degree j j j in p p p variables is ( p + j − 1 j ) \binom{p+j-1}{j} ( j p + j − 1 ) . Summing from j = 1 j=1 j = 1 to k k k gives N p , k = ( p + k k ) − 1 N_{p,k}=\binom{p+k}{k}-1 N p , k = ( k p + k ) − 1 . Each reduced coefficient has r r r coordinates, establishing the storage count. If k ≥ t − 1 k\ge t-1 k ≥ t − 1 , no nonzero coefficient has been omitted, so (17) is exact. □
Normalization and mixed terms. For p = 2 p=2 p = 2 , the degree-two equations include
z t , ( 2 , 0 ) = T t z t − 1 , ( 2 , 0 ) + G 1 , t z t − 1 , ( 1 , 0 ) (38) z_{t,(2,0)}=T_tz_{t-1,(2,0)}+G_{1,t}z_{t-1,(1,0)} \tag*{(38)} z t , ( 2 , 0 ) = T t z t − 1 , ( 2 , 0 ) + G 1 , t z t − 1 , ( 1 , 0 ) (38)
z t , ( 1 , 1 ) = T t z t − 1 , ( 1 , 1 ) + G 1 , t z t − 1 , ( 0 , 1 ) + G 2 , t z t − 1 , ( 1 , 0 ) (39) z_{t,(1,1)}=T_tz_{t-1,(1,1)}+G_{1,t}z_{t-1,(0,1)}+G_{2,t}z_{t-1,(1,0)} \tag*{(39)} z t , ( 1 , 1 ) = T t z t − 1 , ( 1 , 1 ) + G 1 , t z t − 1 , ( 0 , 1 ) + G 2 , t z t − 1 , ( 1 , 0 ) (39)
z t , ( 0 , 2 ) = T t z t − 1 , ( 0 , 2 ) + G 2 , t z t − 1 , ( 0 , 1 ) (40) z_{t,(0,2)}=T_tz_{t-1,(0,2)}+G_{2,t}z_{t-1,(0,1)} \tag*{(40)} z t , ( 0 , 2 ) = T t z t − 1 , ( 0 , 2 ) + G 2 , t z t − 1 , ( 0 , 1 ) (40)
There is no factor of two in the repeated-coordinate equations because z ( 2 , 0 ) z_{(2,0)} z ( 2 , 0 ) represents the coefficient, not the unnormalized second derivative. The mixed equation includes both orderings, without assuming commutativity.
Uniform truncation and leakage bounds
The projected response
The projected coefficient recurrence remains defined when E t ≠ 0 E_t\ne0 E t = 0 . Define an untruncated reduced response by
z t ( a ) = ( T t + ∑ i a i G i , t ) z t − 1 ( a ) + ( ∑ i a i D i , t ) h t − 1 0 , z 0 ( a ) = 0. (41) z_t(a)=\left(T_t+\sum_i a_iG_{i,t}\right)z_{t-1}(a)+\left(\sum_i a_iD_{i,t}\right)h_{t-1}^0,\qquad z_0(a)=0. \tag*{(41)} z t ( a ) = ( T t + i ∑ a i G i , t ) z t − 1 ( a ) + ( i ∑ a i D i , t ) h t − 1 0 , z 0 ( a ) = 0. (41)
At any finite t t t this is a finite polynomial, and its positive-degree coefficients are precisely (16) . Define h ˉ t ( a ) = h t 0 + U z t ( a ) \bar{h}_t(a)=h_t^0+Uz_t(a) h ˉ t ( a ) = h t 0 + U z t ( a ) . This trajectory is used only in the analysis; the algorithm does not maintain it for unknown a a a .
From (18) and orthonormality of U U U ,
∥ h t 0 ∥ ≤ B 1 − ρ , ∥ T t ∥ ≤ ρ , ∥ G i , t ∥ ≤ L . (42) \lVert h_t^0\rVert\le\frac{B}{1-\rho},\qquad\lVert T_t\rVert\le\rho,\qquad\lVert G_{i,t}\rVert\le L. \tag*{(42)} ∥ h t 0 ∥ ≤ 1 − ρ B , ∥ T t ∥ ≤ ρ , ∥ G i , t ∥ ≤ L . (42)
Equation (41) then gives
∥ z t ( a ) ∥ ≤ κ ∥ z t − 1 ( a ) ∥ + L R B 1 − ρ ≤ L R B ( 1 − ρ ) ( 1 − κ ) . (43) \lVert z_t(a)\rVert\le\kappa\lVert z_{t-1}(a)\rVert+\frac{LRB}{1-\rho}\le\frac{LRB}{(1-\rho)(1-\kappa)}. \tag*{(43)} ∥ z t ( a )∥ ≤ κ ∥ z t − 1 ( a )∥ + 1 − ρ L R B ≤ ( 1 − ρ ) ( 1 − κ ) L R B . (43)
To compare with the true state, write A t ( a ) = A 0 , t + ∑ i a i U D i , t A_t(a)=A_{0,t}+\sum_i a_i U D_{i,t} A t ( a ) = A 0 , t + ∑ i a i U D i , t . Direct substitution shows
A t ( a ) h ˉ t − 1 ( a ) + b t − h ˉ t ( a ) = E t z t − 1 ( a ) . (44) A_t(a)\bar{h}_{t-1}(a)+b_t-\bar{h}_t(a)=E_t z_{t-1}(a). \tag*{(44)} A t ( a ) h ˉ t − 1 ( a ) + b t − h ˉ t ( a ) = E t z t − 1 ( a ) . (44)
Hence, for e t = h t ( a ) − h ˉ t ( a ) e_t=h_t(a)-\bar{h}_t(a) e t = h t ( a ) − h ˉ t ( a ) ,
e t = A t ( a ) e t − 1 + E t z t − 1 ( a ) , e 0 = 0. (45) e_t=A_t(a)e_{t-1}+E_tz_{t-1}(a),\qquad e_0=0. \tag*{(45)} e t = A t ( a ) e t − 1 + E t z t − 1 ( a ) , e 0 = 0. (45)
Using (43) and ∥ A t ( a ) ∥ ≤ κ \lVert A_t(a)\rVert\le\kappa ∥ A t ( a )∥ ≤ κ gives
∥ e t ∥ ≤ δ L R B ( 1 − ρ ) ( 1 − κ ) 2 . (46) \lVert e_t\rVert\le\frac{\delta LRB}{(1-\rho)(1-\kappa)^2}. \tag*{(46)} ∥ e t ∥ ≤ ( 1 − ρ ) ( 1 − κ ) 2 δ L R B . (46)
A multinomial coefficient bound
Lemma 12. For every nonzero multi-index α \alpha α , every time t t t , and j = ∣ α ∣ j=|\alpha| j = ∣ α ∣ ,
∥ z t , α ∥ ≤ ( j α ) L j B ( 1 − ρ ) j + 1 . (47) \lVert z_{t,\alpha}\rVert\le\binom{j}{\alpha}\frac{L^jB}{(1-\rho)^{j+1}}. \tag*{(47)} ∥ z t , α ∥ ≤ ( α j ) ( 1 − ρ ) j + 1 L j B . (47)
Proof. Set C α = ( j α ) L j B / ( 1 − ρ ) j + 1 C_\alpha=\binom{j}{\alpha}L^jB/(1-\rho)^{j+1} C α = ( α j ) L j B / ( 1 − ρ ) j + 1 . All coefficients are zero at time zero. Suppose all previous-time coefficients satisfy the bound. When j = 1 j=1 j = 1 , the forcing in (16) has norm at most L B / ( 1 − ρ ) = ( 1 − ρ ) C α LB/(1-\rho)=(1-\rho)C_\alpha L B / ( 1 − ρ ) = ( 1 − ρ ) C α . The inherited term has norm at most ρ C α \rho C_\alpha ρ C α . Their sum is at most C α C_\alpha C α .
For j > 1 j>1 j > 1 , use
∑ i : α i > 0 ( j − 1 α − e i ) = ∑ i : α i > 0 α i j ( j α ) = ( j α ) . (48) \sum_{i:\alpha_i>0}\binom{j-1}{\alpha-e_i}
=\sum_{i:\alpha_i>0}\frac{\alpha_i}{j}\binom{j}{\alpha}
=\binom{j}{\alpha}. \tag*{(48)} i : α i > 0 ∑ ( α − e i j − 1 ) = i : α i > 0 ∑ j α i ( α j ) = ( α j ) . (48)
The total forcing norm is bounded by
L ∑ i : α i > 0 C α − e i = ( j α ) L j B ( 1 − ρ ) j = ( 1 − ρ ) C α . (49) L\sum_{i:\alpha_i>0}C_{\alpha-e_i}
=\binom{j}{\alpha}\frac{L^jB}{(1-\rho)^j}
=(1-\rho)C_\alpha. \tag*{(49)} L i : α i > 0 ∑ C α − e i = ( α j ) ( 1 − ρ ) j L j B = ( 1 − ρ ) C α . (49)
Adding ρ C α \rho C_\alpha ρ C α from the propagated coefficient proves the induction. No cancellation or sign assumption is used. □
Proof of Theorem 6
For any a a a in the declared ball, the multinomial theorem and Lemma 12 imply
∑ ∣ α ∣ = j ∣ a α ∣ ∥ z t , α ∥ ≤ L j B ( 1 − ρ ) j + 1 ∑ ∣ α ∣ = j ( j α ) ∏ i ∣ a i ∣ α i (50) \sum_{|\alpha|=j}|a^\alpha|\lVert z_{t,\alpha}\rVert
\le\frac{L^jB}{(1-\rho)^{j+1}}\sum_{|\alpha|=j}\binom{j}{\alpha}\prod_i|a_i|^{\alpha_i} \tag*{(50)} ∣ α ∣ = j ∑ ∣ a α ∣ ∥ z t , α ∥ ≤ ( 1 − ρ ) j + 1 L j B ∣ α ∣ = j ∑ ( α j ) i ∏ ∣ a i ∣ α i (50)
= B 1 − ρ ( L ∥ a ∥ 1 1 − ρ ) j ≤ B 1 − ρ q j . (51) =\frac{B}{1-\rho}\left(\frac{L\lVert a\rVert_1}{1-\rho}\right)^j
\le\frac{B}{1-\rho}q^j. \tag*{(51)} = 1 − ρ B ( 1 − ρ L ∥ a ∥ 1 ) j ≤ 1 − ρ B q j . (51)
Since U U U is an isometry on R r \mathbb{R}^r R r , the difference between the untruncated projected state and its degree-k k k truncation satisfies
∥ h ˉ t ( a ) − h ^ t ( k ) ( a ) ∥ ≤ B 1 − ρ ∑ j = k + 1 ∞ q j = B 1 − κ q k + 1 . (52) \lVert\bar{h}_t(a)-\hat{h}^{(k)}_t(a)\rVert
\le\frac{B}{1-\rho}\sum_{j=k+1}^{\infty}q^j
=\frac{B}{1-\kappa}q^{k+1}. \tag*{(52)} ∥ h ˉ t ( a ) − h ^ t ( k ) ( a )∥ ≤ 1 − ρ B j = k + 1 ∑ ∞ q j = 1 − κ B q k + 1 . (52)
For finite t t t , the actual tail is finite, and the infinite sum is only an upper bound. Combining (52) with (46) proves (19). The condition κ < 1 \kappa<1 κ < 1 implies q < 1 q<1 q < 1 , justifying the geometric series. □
Degree selection. Let ε \varepsilon ε exceed the leakage bound E l e a k E_{\mathrm{leak}} E leak in (46) . For 0 < q < 1 0<q<1 0 < q < 1 , any integer k ≥ 1 k\ge1 k ≥ 1 satisfying
k + 1 ≥ log ( ( ε − E l e a k ) ( 1 − κ ) / B ) log q (53) k+1\ge\frac{\log\left((\varepsilon-E_{\mathrm{leak}})(1-\kappa)/B\right)}{\log q} \tag*{(53)} k + 1 ≥ log q log ( ( ε − E leak ) ( 1 − κ ) / B ) (53)
is sufficient whenever B > 0 B>0 B > 0 . If the logarithmic ratio is negative, the inequality is automatically satisfied by all sufficiently small positive degrees as well. When L = 0 L=0 L = 0 or B = 0 B=0 B = 0 , the parameter-dependent response is zero and no positive-degree coefficients are necessary. If the requested tolerance is at or below the leakage bound, increasing degree alone cannot certify that tolerance using this theorem. Failure of this bound to certify the tolerance does not imply that the tolerance is unattainable on the given stream.
Online residuals and successive updates
Proof of Theorem 7
For j ≥ 1 j\ge1 j ≥ 1 , write z t [ j ] ( a ) = ∑ ∣ α ∣ = j a α z t , α z_t^{[j]}(a)=\sum_{|\alpha|=j}a^\alpha z_{t,\alpha} z t [ j ] ( a ) = ∑ ∣ α ∣ = j a α z t , α and w t ( k ) ( a ) = ∑ j = 1 k z t [ j ] ( a ) w_t^{(k)}(a)=\sum_{j=1}^{k}z_t^{[j]}(a) w t ( k ) ( a ) = ∑ j = 1 k z t [ j ] ( a ) . By the definition of M t , j M_{t,j} M t , j and the multinomial theorem,
∥ z t [ j ] ( a ) ∥ ≤ M t , j ∑ ∣ α ∣ = j ( j α ) ∣ a α ∣ = M t , j ∥ a ∥ 1 j ≤ R j M t , j . (54) \left\|z_t^{[j]}(a)\right\|\le M_{t,j}\sum_{|\alpha|=j}\binom{j}{\alpha}|a^\alpha|=M_{t,j}\|a\|_1^j\le R^jM_{t,j}. \tag*{(54)} z t [ j ] ( a ) ≤ M t , j ∣ α ∣ = j ∑ ( α j ) ∣ a α ∣ = M t , j ∥ a ∥ 1 j ≤ R j M t , j . (54)
Define G t ( a ) = ∑ i a i G i , t G_t(a)=\sum_i a_iG_{i,t} G t ( a ) = ∑ i a i G i , t . Summing the retained recurrences shows that the one-step residual of the decoded polynomial is exactly
r t ( a ) = A t ( a ) h ^ t − 1 ( k ) ( a ) + b t − h ^ t ( k ) ( a ) (55) r_t(a)=A_t(a)\hat{h}_{t-1}^{(k)}(a)+b_t-\hat{h}_t^{(k)}(a) \tag*{(55)} r t ( a ) = A t ( a ) h ^ t − 1 ( k ) ( a ) + b t − h ^ t ( k ) ( a ) (55)
= E t w t − 1 ( k ) ( a ) + U G t ( a ) z t − 1 [ k ] ( a ) . (56) =E_tw_{t-1}^{(k)}(a)+UG_t(a)z_{t-1}^{[k]}(a). \tag*{(56)} = E t w t − 1 ( k ) ( a ) + U G t ( a ) z t − 1 [ k ] ( a ) . (56)
All terms of degrees one through k k k cancel by (16) . The second term is the omitted degree-( k + 1 ) (k+1) ( k + 1 ) contribution, while the first is off-subspace propagation of every retained degree.
Using ∥ G t ( a ) ∥ ≤ R ℓ t \|G_t(a)\|\le R\ell_t ∥ G t ( a ) ∥ ≤ R ℓ t in (56) and applying (54) gives
∥ r t ( a ) ∥ ≤ δ t ∑ j = 1 k R j M t − 1 , j + ℓ t R k + 1 M t − 1 , k . (57) \|r_t(a)\|\le\delta_t\sum_{j=1}^{k}R^jM_{t-1,j}+\ell_tR^{k+1}M_{t-1,k}. \tag*{(57)} ∥ r t ( a ) ∥ ≤ δ t j = 1 ∑ k R j M t − 1 , j + ℓ t R k + 1 M t − 1 , k . (57)
The migration error d t ( a ) = h t ( a ) − h ^ t ( k ) ( a ) d_t(a)=h_t(a)-\hat{h}_t^{(k)}(a) d t ( a ) = h t ( a ) − h ^ t ( k ) ( a ) satisfies d t ( a ) = A t ( a ) d t − 1 ( a ) + r t ( a ) d_t(a)=A_t(a)d_{t-1}(a)+r_t(a) d t ( a ) = A t ( a ) d t − 1 ( a ) + r t ( a ) and d 0 ( a ) = 0 d_0(a)=0 d 0 ( a ) = 0 . Taking the supremum over the ball, bounding the transition by κ t \kappa_t κ t , and inducting on t t t proves (21) . The proof uses only finite sums and does not require contraction. □ \square □
Numerical certification. Equation (21) is not, by itself, a validated floating-point certificate. If an implementation can supply a rigorous upper bound η t \eta_t η t on the additional one-step defect caused by rounded arithmetic and rounded coefficient updates, the same induction gives the valid update c t = κ t c t − 1 + residual bound + η t c_t=\kappa_tc_{t-1}+\text{residual bound}+\eta_t c t = κ t c t − 1 + residual bound + η t . Obtaining η t \eta_t η t and certified operator-norm bounds requires additional numerical analysis or interval arithmetic. The numerical experiments use ordinary float64 arithmetic. Agreement with independently replayed states at the stated tolerance does not control the rounding defect, so it does not justify setting η t = 0 \eta_t=0 η t = 0 .
Repeated queries do not re-center the approximation
Let τ 1 < τ 2 < ⋯ \tau_1 < \tau_2 < \cdots τ 1 < τ 2 < ⋯ be update times and let a j a_j a j be any permitted replacement coordinate revealed at τ j \tau_j τ j . The coordinate can depend on observations up to that point. The radius-uniform guarantees already hold for every such coordinate. At the switch, set the active state to h ^ τ j ( k ) ( a j ) \hat{h}^{(k)}_{\tau_j}(a_j) h ^ τ j ( k ) ( a j ) while continuing to maintain the same anchor and coefficients. For s s s subsequent inputs before the next switch, exact execution of the new recurrence gives
∥ h τ j + s a c t i v e − h τ j + s ( a j ) ∥ ≤ ( ∏ ν = 1 s ∥ A τ j + ν ( a j ) ∥ ) c τ j ≤ κ s c τ j (58) \left\|h^{\mathrm{active}}_{\tau_j+s}-h_{\tau_j+s}(a_j)\right\| \le\left(\prod_{\nu=1}^{s}\left\|A_{\tau_j+\nu}(a_j)\right\|\right)c_{\tau_j} \le\kappa^s c_{\tau_j} \tag*{(58)} h τ j + s active − h τ j + s ( a j ) ≤ ( ν = 1 ∏ s A τ j + ν ( a j ) ) c τ j ≤ κ s c τ j (58)
under the uniform assumptions. At time τ j + 1 \tau_{j+1} τ j + 1 the next query uses the anchor memory, not this active approximation. Its error is therefore bounded by c τ j + 1 c_{\tau_{j+1}} c τ j + 1 , independently of the number of preceding switches.
This statement depends on retaining the original anchor family. Translating a finite Taylor polynomial to a new center does not recover the omitted coefficients and is not exact re-centering of the underlying historical response. Nor does the statement recover the trajectory generated by applying the old parameters to old epochs and the new parameters only to new epochs. The target is the retrospective constant-parameter trajectory defined in § 2 .
Observable equivalence and predictive fidelity
Exact future-output equivalence
Consider the target system h s + 1 = A h s + B u s + 1 h_{s+1}=Ah_s+Bu_{s+1} h s + 1 = A h s + B u s + 1 with output y s = C h s y_s=Ch_s y s = C h s , and two initial states h , h ′ h,h' h , h ′ . Under any common future input sequence their state difference is A s ( h − h ′ ) A^s(h-h') A s ( h − h ′ ) , so their output difference is C A s ( h − h ′ ) CA^s(h-h') C A s ( h − h ′ ) . Equality of all current and future outputs implies equality for s = 0 , … , d − 1 s=0,\ldots,d-1 s = 0 , … , d − 1 , hence O A , C ( h − h ′ ) = 0 \mathcal{O}_{A,C}(h-h')=0 O A , C ( h − h ′ ) = 0 .
Conversely, if these first d d d output differences vanish, the characteristic polynomial identity expresses A d A^d A d as a linear combination of I , A , … , A d − 1 I,A,\ldots,A^{d-1} I , A , … , A d − 1 . Multiplying successively by A A A shows that every later power is in the same linear span. Therefore C A s ( h − h ′ ) = 0 CA^s(h-h')=0 C A s ( h − h ′ ) = 0 for every s ≥ 0 s\ge0 s ≥ 0 . This proves the equivalence. The input matrix and input sequence need not satisfy additional restrictions because their effects cancel in the state difference.
For a finite set of target models with history-to-state maps R j R_j R j , define
R o b s = [ O A 1 , C 1 R 1 ⋮ O A K , C K R K ] . (59) R_{\mathrm{obs}} =
\begin{bmatrix}
\mathcal{O}_{A_1,C_1}R_1\\
\vdots\\
\mathcal{O}_{A_K,C_K}R_K
\end{bmatrix}. \tag*{(59)} R obs = O A 1 , C 1 R 1 ⋮ O A K , C K R K . (59)
The exact-information dimension is rank ( [ R 0 ⊤ , R o b s ⊤ ] ⊤ ) − rank ( R 0 ) \operatorname{rank}([R_0^\top,R_{\mathrm{obs}}^\top]^\top)-\operatorname{rank}(R_0) rank ([ R 0 ⊤ , R obs ⊤ ] ⊤ ) − rank ( R 0 ) by Theorem 4 . The decoder may first recover the observable vector for target j j j , then choose any state realizing it, for example O A j , C j \mathcal{O}_{A_j,C_j} O A j , C j times that vector. Because the observable vector belongs to the range of the observability matrix, this produces an output-equivalent state. The approximate singular-value formula concerns this stacked observable-vector norm, not an unweighted infinite-horizon output norm.
Current output is not enough. Let
A = [ 0 1 0 0 ] , C = [ 1 0 ] , e = [ 0 1 ] . (60) A=\begin{bmatrix}0&1\\0&0\end{bmatrix},\qquad C=\begin{bmatrix}1&0\end{bmatrix},\qquad e=\begin{bmatrix}0\\1\end{bmatrix}. \tag*{(60)} A = [ 0 0 1 0 ] , C = [ 1 0 ] , e = [ 0 1 ] . (60)
Then C e = 0 C e = 0 C e = 0 but C A e = 1 C A e = 1 C A e = 1 . A presently invisible state difference changes the next output. In contrast, if A A A is diagonal and C C C reads only its first coordinate, all other coordinates are unobservable for this fixed model. A history direction discarded by the old model may then be unnecessary for output-equivalent migration even when exact hidden-state recovery would require retaining it. These claims concern a fixed target during subsequent execution. Requiring equivalence under arbitrary later model switches can demand a larger space involving products of different transitions.
Predictive KL and its proof
For a fixed κ \kappa κ -contractive target under common future inputs, L g L_g L g -Lipschitz logits, and an initial state-error bound c t c_t c t , we have
∑ s ≥ 0 KL ( p t + s r e p l a y ∥ p t + s m i g r a t e d ) ≤ L g 2 c t 2 4 ( 1 − κ 2 ) . (61) \sum_{s \ge0} \operatorname{KL}\left(p_{t+s}^{\mathrm{replay}} \mathbin{\|} p_{t+s}^{\mathrm{migrated}}\right) \le\frac{L_g^2 c_t^2}{4(1-\kappa^2)}. \tag*{(61)} s ≥ 0 ∑ KL ( p t + s replay ∥ p t + s migrated ) ≤ 4 ( 1 − κ 2 ) L g 2 c t 2 . (61)
Let ψ ( z ) = log ∑ i e z i \psi(z) = \log\sum_i e^{z_i} ψ ( z ) = log ∑ i e z i and p ( z ) = ∇ ψ ( z ) p(z) = \nabla\psi(z) p ( z ) = ∇ ψ ( z ) . Its Hessian is ∇ 2 ψ ( z ) = diag ( p ) − p p ⊤ \nabla^2\psi(z) = \operatorname{diag}(p) - pp^\top ∇ 2 ψ ( z ) = diag ( p ) − p p ⊤ . The absolute row sum at row i i i is
p i ( 1 − p i ) + ∑ j ≠ i p i p j = 2 p i ( 1 − p i ) ≤ 1 2 . (62) p_i(1-p_i) + \sum_{j \ne i} p_i p_j = 2p_i(1-p_i) \le\frac{1}{2}. \tag*{(62)} p i ( 1 − p i ) + j = i ∑ p i p j = 2 p i ( 1 − p i ) ≤ 2 1 . (62)
The Hessian is symmetric positive semidefinite, so its operator norm is at most 1 / 2 1/2 1/2 . The integral Taylor formula therefore implies
ψ ( z ′ ) − ψ ( z ) − ∇ ψ ( z ) ⊤ ( z ′ − z ) ≤ 1 4 ∥ z ′ − z ∥ 2 . (63) \psi(z')-\psi(z)-\nabla\psi(z)^\top(z'-z) \le\frac{1}{4}\lVert z'-z\rVert^2. \tag*{(63)} ψ ( z ′ ) − ψ ( z ) − ∇ ψ ( z ) ⊤ ( z ′ − z ) ≤ 4 1 ∥ z ′ − z ∥ 2 . (63)
The left side is KL ( p ( z ) ∥ p ( z ′ ) ) \operatorname{KL}(p(z)\mathbin{\|}p(z')) KL ( p ( z ) ∥ p ( z ′ )) . With the same subsequent inputs, contraction bounds the state discrepancy at offset s s s by κ s c t \kappa^s c_t κ s c t . If the logit map is L g L_g L g -Lipschitz in the state, the logit discrepancy is at most L g κ s c t L_g\kappa^s c_t L g κ s c t . Thus
KL ( p t + s r e p l a y ∥ p t + s m i g r a t e d ) ≤ L g 2 4 κ 2 s c t 2 . (64) \operatorname{KL}\left(p_{t+s}^{\mathrm{replay}} \mathbin{\|} p_{t+s}^{\mathrm{migrated}}\right) \le\frac{L_g^2}{4}\kappa^{2s}c_t^2. \tag*{(64)} KL ( p t + s replay ∥ p t + s migrated ) ≤ 4 L g 2 κ 2 s c t 2 . (64)
Summing the geometric series proves (61). A direct input-to-logit term is harmless when both trajectories receive the same inputs, since it cancels in the comparison. For the experimental head, a valid state-Lipschitz constant is ∥ W o ∥ ∥ W h ∥ \lVert W_o\rVert\lVert W_h\rVert ∥ W o ∥ ∥ W h ∥ because coordinatewise tanh \tanh tanh is 1-Lipschitz.
The KL comparison is an expectation under the replay model’s distribution. Ground-truth test labels need not follow that distribution exactly. Consequently the difference in empirical negative log-likelihood can have either sign, and lower KL need not imply better test loss. We report both quantities to distinguish state fidelity from task improvement.
A first-order extension to smooth recurrent models
The higher-order algorithm in the main text relies on an affine hidden-state recurrence. The following first-order result extends the analysis to smooth recurrences, without asserting the same compact higher-order structure for general nonlinear models.
Proposition 13 (Projected first-order migration). Let h t ( a ) = F t ( h t − 1 ( a ) , a ) h_t(a) = F_t(h_{t-1}(a), a) h t ( a ) = F t ( h t − 1 ( a ) , a ) with h 0 ( a ) = 0 h_0(a) = 0 h 0 ( a ) = 0 , for a a a in a convex Euclidean ball around zero. Assume F t F_t F t is twice continuously differentiable on a domain containing the trajectories for every parameter in that ball, with
∥ D h F t ∥ ≤ ρ < 1 , ∥ D a F t ∥ ≤ L , (65) \lVert D_h F_t\rVert\le\rho< 1, \qquad\lVert D_a F_t\rVert\le L, \tag*{(65)} ∥ D h F t ∥ ≤ ρ < 1 , ∥ D a F t ∥ ≤ L , (65)
uniformly there. Bound its joint second derivative by M M M under the product norm ∥ ( v , w ) ∥ = ∥ v ∥ + ∥ w ∥ \|(v,w)\|=\|v\|+\|w\| ∥ ( v , w ) ∥ = ∥ v ∥ + ∥ w ∥ .
At the anchor trajectory, set J t = D h F t ( h t − 1 ( 0 ) , 0 ) J_t=D_hF_t(h_{t-1}(0),0) J t = D h F t ( h t − 1 ( 0 ) , 0 ) , B t = D a F t ( h t − 1 ( 0 ) , 0 ) B_t=D_aF_t(h_{t-1}(0),0) B t = D a F t ( h t − 1 ( 0 ) , 0 ) , and P = U U ⊤ P=UU^\top P = U U ⊤ . Suppose
∥ ( I − P ) J t U ∥ ≤ δ , ∥ ( I − P ) B t ∥ ≤ β . (66) \|(I-P)J_tU\|\le\delta,\qquad\|(I-P)B_t\|\le\beta. \tag*{(66)} ∥ ( I − P ) J t U ∥ ≤ δ , ∥ ( I − P ) B t ∥ ≤ β . (66)
Maintain K 0 = 0 K_0=0 K 0 = 0 and K t = U ⊤ J t U K t − 1 + U ⊤ B t K_t=U^\top J_tUK_{t-1}+U^\top B_t K t = U ⊤ J t U K t − 1 + U ⊤ B t . Then
∥ h t ( a ) − h t ( 0 ) − U K t a ∥ ≤ ( δ L ( 1 − ρ ) 2 + β 1 − ρ ) ∥ a ∥ (67) \|h_t(a)-h_t(0)-UK_ta\|\le\left(\frac{\delta L}{(1-\rho)^2}+\frac{\beta}{1-\rho}\right)\|a\| \tag*{(67)} ∥ h t ( a ) − h t ( 0 ) − U K t a ∥ ≤ ( ( 1 − ρ ) 2 δ L + 1 − ρ β ) ∥ a ∥ (67)
+ M ( 1 + L / ( 1 − ρ ) ) 2 2 ( 1 − ρ ) ∥ a ∥ 2 . (68) +\frac{M(1+L/(1-\rho))^2}{2(1-\rho)}\|a\|^2. \tag*{(68)} + 2 ( 1 − ρ ) M ( 1 + L / ( 1 − ρ ) ) 2 ∥ a ∥ 2 . (68)
Proof. Let S t ( a ) = D a h t ( a ) S_t(a)=D_ah_t(a) S t ( a ) = D a h t ( a ) . Differentiating the recurrence gives S t ( a ) = D h F t S t − 1 ( a ) + D a F t S_t(a)=D_hF_tS_{t-1}(a)+D_aF_t S t ( a ) = D h F t S t − 1 ( a ) + D a F t . Thus ∥ S t ( a ) ∥ ≤ L / ( 1 − ρ ) \|S_t(a)\|\le L/(1-\rho) ∥ S t ( a ) ∥ ≤ L / ( 1 − ρ ) uniformly. The projected recurrence similarly gives ∥ K t ∥ ≤ L / ( 1 − ρ ) \|K_t\|\le L/(1-\rho) ∥ K t ∥ ≤ L / ( 1 − ρ ) . At a = 0 a=0 a = 0 , subtract the projected sensitivity from the true one to obtain
S t ( 0 ) − U K t = J t ( S t − 1 ( 0 ) − U K t − 1 ) + ( I − P ) J t U K t − 1 + ( I − P ) B t . (69) S_t(0)-UK_t=J_t(S_{t-1}(0)-UK_{t-1})+(I-P)J_tUK_{t-1}+(I-P)B_t. \tag*{(69)} S t ( 0 ) − U K t = J t ( S t − 1 ( 0 ) − U K t − 1 ) + ( I − P ) J t U K t − 1 + ( I − P ) B t . (69)
Its norm is bounded by a geometric recurrence with forcing δ L / ( 1 − ρ ) + β \delta L/(1-\rho)+\beta δ L / ( 1 − ρ ) + β , yielding ∥ S t ( 0 ) − U K t ∥ ≤ δ L / ( 1 − ρ ) 2 + β / ( 1 − ρ ) \|S_t(0)-UK_t\|\le\delta L/(1-\rho)^2+\beta/(1-\rho) ∥ S t ( 0 ) − U K t ∥ ≤ δ L / ( 1 − ρ ) 2 + β / ( 1 − ρ ) .
For unit directions v , w v,w v , w , a second differentiation yields an inherited second-state-derivative term with norm at most ρ ∥ D a 2 h t − 1 ∥ \rho\|D_a^2h_{t-1}\| ρ ∥ D a 2 h t − 1 ∥ and a joint second-derivative term bounded by M ( ∥ S t − 1 v ∥ + 1 ) ( ∥ S t − 1 w ∥ + 1 ) M(\|S_{t-1}v\|+1)(\|S_{t-1}w\|+1) M ( ∥ S t − 1 v ∥ + 1 ) ( ∥ S t − 1 w ∥ + 1 ) . Hence
∥ D a 2 h t ( a ) ∥ ≤ M ( 1 + L / ( 1 − ρ ) ) 2 1 − ρ . (70) \|D_a^2h_t(a)\|\le\frac{M(1+L/(1-\rho))^2}{1-\rho}. \tag*{(70)} ∥ D a 2 h t ( a ) ∥ ≤ 1 − ρ M ( 1 + L / ( 1 − ρ ) ) 2 . (70)
Taylor’s integral remainder along the segment from zero to a a a bounds h t ( a ) − h t ( 0 ) − S t ( 0 ) a h_t(a)-h_t(0)-S_t(0)a h t ( a ) − h t ( 0 ) − S t ( 0 ) a by one half of this constant times ∥ a ∥ 2 \|a\|^2 ∥ a ∥ 2 . Adding the projected-sensitivity discrepancy proves (68).
This is a projected sensitivity statement with explicit leakage, not an exact state-invariant nonlinear reduction. Higher derivatives of F t F_t F t can generate directions outside im U \operatorname{im}U im U even when first-order leakage vanishes. The experiments do not use this extension. It gives no theorem for arbitrary GRU, LSTM, or multilayer selective-model fine-tuning.
Exact local memory as a causal realization problem
This section gives a causal counterpart to the fixed-horizon information calculation. It concerns exact local approximation order, not a fixed nonzero error tolerance. Consider the time-invariant family
h t ( a ) = ( A 0 + ∑ i = 1 p a i A i ) h t − 1 ( a ) + B x t , h 0 ( a ) = 0 , (71) h_t(a)=\left(A_0+\sum_{i=1}^{p}a_iA_i\right)h_{t-1}(a)+Bx_t,\qquad h_0(a)=0, \tag*{(71)} h t ( a ) = ( A 0 + i = 1 ∑ p a i A i ) h t − 1 ( a ) + B x t , h 0 ( a ) = 0 , (71)
where x t ∈ R q x_t\in\mathbb{R}^q x t ∈ R q and the allowed parameter set contains a neighborhood of zero. Let J k = { α ∈ N 0 p : ∣ α ∣ ≤ k } \mathcal{J}_k=\{\alpha\in\mathbb{N}_0^p:|\alpha|\le k\} J k = { α ∈ N 0 p : ∣ α ∣ ≤ k } and n k = d ∣ J k ∣ n_k=d|\mathcal{J}_k| n k = d ∣ J k ∣ . Stack the coefficients through degree k k k into j t ∈ R n k j_t\in\mathbb{R}^{n_k} j t ∈ R n k , including the constant coefficient. Their exact recurrence is
j t = T k j t − 1 + B k x t , ( T k ) α , β = { A 0 , α = β , A i , α = β + e i , 0 , otherwise . (72) j_t=T_kj_{t-1}+B_kx_t,\qquad(T_k)_{\alpha,\beta}=
\begin{cases}
A_0,&\alpha=\beta,\\
A_i,&\alpha=\beta+e_i,\\
0,&\text{otherwise}.
\end{cases} \tag*{(72)} j t = T k j t − 1 + B k x t , ( T k ) α , β = ⎩ ⎨ ⎧ A 0 , A i , 0 , α = β , α = β + e i , otherwise . (72)
with B B B in the constant block of B k B_k B k and zeros elsewhere. The cases in (72) are disjoint. Define
K k = ∑ s = 0 n k − 1 im ( T k s B k ) , π 0 j = j 0 , ν k = dim K k − dim ( π 0 K k ) . (73) \mathcal{K}_k=\sum_{s=0}^{n_k-1}\operatorname{im}(\mathbf{T}_k^sB_k),\qquad\pi_0j=j_0,\qquad\nu_k=\dim\mathcal{K}_k-\dim(\pi_0\mathcal{K}_k). \tag*{(73)} K k = s = 0 ∑ n k − 1 im ( T k s B k ) , π 0 j = j 0 , ν k = dim K k − dim ( π 0 K k ) . (73)
The notation j 0 j_0 j 0 in the projection denotes the constant multi-index block, not time zero. Cayley–Hamilton makes K k \mathcal{K}_k K k invariant under T k \mathbf{T}_k T k . All coefficient states reached from the zero initial condition lie in this space.
Proof of Theorem 1
We first prove that local order is an information requirement on the entire coefficient vector. Suppose two fixed histories have the same old state and auxiliary summary, so the decoder returns the same function of a a a for both histories. If each approximation has error O ( ∥ a ∥ k + 1 ) O(\lVert a\rVert^{k+1}) O (∥ a ∥ k + 1 ) as a → 0 a\to0 a → 0 , their target difference has this order as well. Let ℓ ≤ k \ell\le k ℓ ≤ k be the least degree with a nonzero coefficient in their difference, and let P ℓ ( a ) P_\ell(a) P ℓ ( a ) be the corresponding vector-valued homogeneous polynomial. There is a vector v v v with P ℓ ( v ) ≠ 0 P_\ell(v)\ne0 P ℓ ( v ) = 0 . Otherwise every scalar component of this polynomial would vanish identically, contrary to the choice of ℓ \ell ℓ . Along a = s v a=sv a = s v ,
h t ( s v ; x ) − h t ( s v ; x ′ ) = s ℓ P ℓ ( v ) + O ( ∣ s ∣ ℓ + 1 ) . (74) h_t(sv;x)-h_t(sv;x')=s^\ell P_\ell(v)+O(\lvert s\rvert^{\ell+1}). \tag*{(74)} h t ( s v ; x ) − h t ( s v ; x ′ ) = s ℓ P ℓ ( v ) + O (∣ s ∣ ℓ + 1 ) . (74)
Dividing by ∣ s ∣ ℓ \lvert s\rvert^\ell ∣ s ∣ ℓ contradicts the assumed O ( ∣ s ∣ k + 1 ) O(\lvert s\rvert^{k+1}) O (∣ s ∣ k + 1 ) difference. Thus all coefficients through degree k k k agree. The constants in the local error statements may depend on the two histories. They are fixed once those histories are selected, which is all the argument requires.
For the lower bound, take the horizon n k n_k n k . Its map F F F from the concatenated input vector in R q n k \mathbb{R}^{qn_k} R q n k to j n k j_{n_k} j n k has image K k \mathcal{K}_k K k , since its block columns are the controllability columns in reverse order. Its old-state map is π 0 F \pi_0F π 0 F . The restriction of F F F to ker ( π 0 F ) \ker(\pi_0F) ker ( π 0 F ) has rank
rank F − rank ( π 0 F ) = dim K k − dim ( π 0 K k ) = ν k . (75) \operatorname{rank}F-\operatorname{rank}(\pi_0F)=\dim\mathcal{K}_k-\dim(\pi_0\mathcal{K}_k)=\nu_k. \tag*{(75)} rank F − rank ( π 0 F ) = dim K k − dim ( π 0 K k ) = ν k . (75)
If a continuous auxiliary summary has m < ν k m<\nu_k m < ν k coordinates, choose an ( m + 1 ) (m+1) ( m + 1 ) -dimensional subspace of ker ( π 0 F ) \ker(\pi_0F) ker ( π 0 F ) on which F F F is injective. Restrict the summary to its unit sphere. Borsuk–Ulam gives antipodal histories x , − x x,-x x , − x with the same summary and old state zero. Their coefficient difference 2 F x 2Fx 2 F x is nonzero. The preceding local-order argument rules this out. It follows that m ≥ ν k m\ge\nu_k m ≥ ν k for any continuous causal summary that works at every horizon. In fact, this single horizon is already enough for the lower bound.
For attainability, choose a fixed linear map L : K k → R ν k L:\mathcal{K}_k\to\mathbb{R}^{\nu_k} L : K k → R ν k such that
H : j ⟼ ( π 0 j , L j ) (76) H:j\longmapsto(\pi_0j,Lj) \tag*{(76)} H : j ⟼ ( π 0 j , L j ) (76)
is injective on K k \mathcal{K}_k K k . Such a map exists by selecting coordinates on ker ( π 0 ∣ K k ) \ker(\pi_0|_{\mathcal{K}_k}) ker ( π 0 ∣ K k ) and extending them to a direct-sum complement. The inverse of H H H on its image is linear. Maintain the ordinary anchor h t ( 0 ) h_t(0) h t ( 0 ) and the ν k \nu_k ν k additional coordinates s t = L j t s_t=Lj_t s t = L j t . At each new input, recover j t − 1 j_{t-1} j t − 1 by this inverse and set
s t = L ( T k j t − 1 + B k x t ) , h t ( 0 ) = A 0 h t − 1 ( 0 ) + B x t . (77) s_t=L(\mathbf{T}_kj_{t-1}+B_kx_t),\qquad h_t(0)=A_0h_{t-1}(0)+Bx_t. \tag*{(77)} s t = L ( T k j t − 1 + B k x t ) , h t ( 0 ) = A 0 h t − 1 ( 0 ) + B x t . (77)
This update is causal and independent of any future parameter. Its reconstructed coefficient vector is exact by induction. Evaluate the degree-k k k polynomial only when the query arrives. At every finite horizon, the omitted polynomial terms have order O ( ∥ a ∥ k + 1 ) O(\lVert a\rVert^{k+1}) O (∥ a ∥ k + 1 ) . This proves attainability and the theorem.
The characterization applies reachability and state realization to an unknown-query local-accuracy requirement; it introduces no new controllability criterion. A dense basis for K k \mathcal{K}_k K k can be expensive to construct, numerically ill-conditioned, or costly to update. The theorem identifies the minimum continuous state dimension for this exact local objective. It does not optimize runtime or bits, or guarantee uniform accuracy at a fixed radius. If the inputs are confined to a finite alphabet rather than a Euclidean ball, the topological lower bound does not automatically apply.
For the rank-one family of Theorem 2 , K k = R d ( k + 1 ) \mathcal{K}_k = \mathbb{R}^{d(k+1)} K k = R d ( k + 1 ) and π 0 K k = R d \pi_0\mathcal{K}_k = \mathbb{R}^d π 0 K k = R d , so ν k = k d \nu_k = kd ν k = k d . For an update-closed family with parameter-independent injection, the nonconstant coefficients lie in im U \operatorname{im} U im U . Hence ν k ≤ r N p , k \nu_k \le rN_{p,k} ν k ≤ r N p , k whenever the ordinary anchor is fully reachable. More generally the same bound follows by choosing the r N p , k rN_{p,k} r N p , k reduced coefficients as auxiliary coordinates. Equality need not hold. Reachability, vanishing monomials, and output equivalence can all reduce the required memory.
Finite tolerance, finite bits, and query radius
A normalized finite-query objective
Fix J J J permitted parameters a ( 1 ) , … , a ( J ) a^{(1)},\ldots,a^{(J)} a ( 1 ) , … , a ( J ) and their linear history maps R 1 , … , R J R_1,\ldots,R_J R 1 , … , R J . Define
R Q = J − 1 / 2 [ R 1 T , … , R J T ] T , M Q = R Q P 0 , P 0 = I − R 0 † R 0 . (78) R_Q = J^{-1/2}[R_1^\mathsf{T},\ldots,R_J^\mathsf{T}]^\mathsf{T}, \qquad M_Q = R_QP_0, \qquad P_0 = I - R_0^\dagger R_0. \tag*{(78)} R Q = J − 1/2 [ R 1 T , … , R J T ] T , M Q = R Q P 0 , P 0 = I − R 0 † R 0 . (78)
A stacked decoder returns all J J J target estimates from the same old state and summary. Its Euclidean error is the root mean squared error over the queries. Theorem 4 applies directly to R Q R_Q R Q . Consequently, the number of auxiliary continuous coordinates needed to achieve worst-history RMS error at most ϵ \epsilon ϵ is exactly
# { i : σ i ( M Q ) > ϵ } . (79) \#\{i:\sigma_i(M_Q)>\epsilon\}. \tag*{(79)} # { i : σ i ( M Q ) > ϵ } . (79)
The inequality is strict because a singular value equal to the tolerance may be omitted.
A uniform guarantee of ϵ \epsilon ϵ at each query implies the RMS guarantee, so (79) is a necessary dimension for that stronger objective. It is not generally sufficient. Conversely, an RMS guarantee of ϵ / J \epsilon/\sqrt{J} ϵ / J is sufficient for per-query error ϵ \epsilon ϵ on this finite set. A guarantee on this finite grid is not equivalent to a guarantee over a continuum of allowed queries.
Proof of the finite-bit statement in Theorem 5
Let the auxiliary encoder have at most 2 b 2^b 2 b possible messages. It can be discontinuous. Condition on old state zero and restrict histories to the unit ball of ker R 0 \ker R_0 ker R 0 . The set of normalized targets on this fiber is the ellipsoid
E = M Q B 2 n (80) \mathcal{E}=M_QB_2^n \tag*{(80)} E = M Q B 2 n (80)
whose positive semiaxes are the singular values σ 1 ≥ ⋯ ≥ σ s > 0 \sigma_1\ge\cdots\ge\sigma_s>0 σ 1 ≥ ⋯ ≥ σ s > 0 . Every history with a given message has the same decoded target. Thus 2 b 2^b 2 b balls of radius ϵ \epsilon ϵ must cover E \mathcal{E} E . Project these balls and the ellipsoid onto the span of the first j j j left singular vectors, for any 1 ≤ j ≤ s 1\le j\le s 1 ≤ j ≤ s . The projected balls have volume at most v j ϵ j v_j\epsilon^j v j ϵ j , whereas the ellipsoid has volume v j ∏ i = 1 j σ i v_j\prod_{i=1}^j\sigma_i v j ∏ i = 1 j σ i , with v j v_j v j the unit-ball volume in R j \mathbb{R}^j R j . Subadditivity of volume yields
2 b ϵ j ≥ ∏ i = 1 j σ i . (81) 2^b\epsilon^j\ge\prod_{i=1}^j\sigma_i. \tag*{(81)} 2 b ϵ j ≥ i = 1 ∏ j σ i . (81)
Taking logarithms and maximizing over j j j , also allowing j = 0 j=0 j = 0 , gives
b ≥ ∑ i = 1 s log 2 + ( σ i ϵ ) , log 2 + x = max ( 0 , log 2 x ) . (82) b\ge\sum_{i=1}^s\log_2^+\left(\frac{\sigma_i}{\epsilon}\right), \qquad\log_2^+x=\max(0,\log_2 x). \tag*{(82)} b ≥ i = 1 ∑ s log 2 + ( ϵ σ i ) , log 2 + x = max ( 0 , log 2 x ) . (82)
Neither continuity nor measurability of the encoder is required, because the covering centers and balls are finite in number. The old state is still supplied exactly without charge. This is therefore an auxiliary-bit lower bound conditional on exact side information, not a bound on the total precision of a deployed machine.
A quantized upper bound makes the precision requirement explicit. Choose m m m such that σ m + 1 ≤ ϵ / 2 \sigma_{m+1} \le\epsilon/2 σ m + 1 ≤ ϵ /2 . If m = 0 m=0 m = 0 , return the known side-information component and use no auxiliary bits. Otherwise retain the output singular coordinates
y i = σ i v i T u ∈ [ − σ i , σ i ] , i = 1 , … , m . (83) y_i = \sigma_i v_i^{\mathsf T}u \in[-\sigma_i,\sigma_i], \qquad i=1,\ldots,m. \tag*{(83)} y i = σ i v i T u ∈ [ − σ i , σ i ] , i = 1 , … , m . (83)
Cover each interval by a uniform grid with mesh at most ϵ / m \epsilon/\sqrt{m} ϵ / m including its endpoints, then encode the nearest grid point. There are at most 1 + ⌈ 2 σ i m / ϵ ⌉ 1+\lceil2\sigma_i\sqrt{m}/\epsilon\rceil 1 + ⌈ 2 σ i m / ϵ ⌉ grid points. The Euclidean rounding error of the retained vector is at most ϵ / 2 \epsilon/2 ϵ /2 . The discarded singular coordinates have norm at most ϵ / 2 \epsilon/2 ϵ /2 . Reconstructing
R Q R 0 T R 0 u + ∑ i = 1 m y ~ i w i (84) R_Q R_0^{\mathsf T}R_0u+\sum_{i=1}^{m}\widetilde{y}_i w_i \tag*{(84)} R Q R 0 T R 0 u + i = 1 ∑ m y i w i (84)
therefore incurs RMS error at most ϵ \epsilon ϵ using at most
∑ i = 1 m ⌈ log 2 ( 1 + ⌈ 2 σ i m ϵ ⌉ ) ⌉ (85) \sum_{i=1}^{m}\left\lceil\log_2\left(1+\left\lceil\frac{2\sigma_i\sqrt{m}}{\epsilon}\right\rceil\right)\right\rceil \tag*{(85)} i = 1 ∑ m ⌈ log 2 ( 1 + ⌈ ϵ 2 σ i m ⌉ ) ⌉ (85)
bits. This is a fixed-horizon encoding. Updating quantized singular coordinates online would require a separate roundoff analysis. □
How an exact local obstruction becomes a finite-radius obstruction
For a scalar parameter at horizon t t t , write h t ( a ; u ) = ∑ ℓ = 0 t − 1 a ℓ C ℓ u h_t(a;u)=\sum_{\ell=0}^{t-1}a^\ell C_\ell u h t ( a ; u ) = ∑ ℓ = 0 t − 1 a ℓ C ℓ u . The old map is C 0 C_0 C 0 . For J ≥ k J\ge k J ≥ k selected query coordinates, form
V j ℓ = J − 1 / 2 ( a ( j ) ) ℓ , 1 ≤ j ≤ J , 1 ≤ ℓ ≤ k , (86) V_{j\ell}=J^{-1/2}\left(a^{(j)}\right)^\ell,\qquad1\le j\le J,\quad1\le\ell\le k, \tag*{(86)} V j ℓ = J − 1/2 ( a ( j ) ) ℓ , 1 ≤ j ≤ J , 1 ≤ ℓ ≤ k , (86)
and C + = [ C 1 T , … , C k T ] T C_+=[C_1^{\mathsf T},\ldots,C_k^{\mathsf T}]^{\mathsf T} C + = [ C 1 T , … , C k T ] T . Suppose V V V has full column rank. The normalized query map restricted to the old-state kernel satisfies
M Q = ( V ⊗ I d ) C + P 0 + F k P 0 , (87) M_Q=(V\otimes I_d)C_+P_0+F_kP_0, \tag*{(87)} M Q = ( V ⊗ I d ) C + P 0 + F k P 0 , (87)
where F k F_k F k stacks the terms of degree above k k k . For every index i i i , extending lists of singular values by zeros,
σ i ( M Q ) ≥ [ σ min ( V ) σ i ( C + P 0 ) − ∥ F k P 0 ∥ ] + . (88) \sigma_i(M_Q)\ge\left[\sigma_{\min}(V)\sigma_i(C_+P_0)-\lVert F_kP_0\rVert\right]_+. \tag*{(88)} σ i ( M Q ) ≥ [ σ m i n ( V ) σ i ( C + P 0 ) − ∥ F k P 0 ∥ ] + . (88)
To prove this, the inequality V T V ⪰ σ min ( V ) 2 I V^{\mathsf T}V\succeq\sigma_{\min}(V)^2I V T V ⪰ σ m i n ( V ) 2 I gives ( ( V ⊗ I ) C ) T ( ( V ⊗ I ) C ) ⪰ σ min ( V ) 2 C T C ((V\otimes I)C)^{\mathsf T}((V\otimes I)C)\succeq\sigma_{\min}(V)^2C^{\mathsf T}C (( V ⊗ I ) C ) T (( V ⊗ I ) C ) ⪰ σ m i n ( V ) 2 C T C . The eigenvalue min–max principle bounds every singular value of the product from below. The singular-value perturbation inequality, which also follows from the norm min–max characterization, subtracts at most ∥ F k P 0 ∥ \lVert F_kP_0\rVert ∥ F k P 0 ∥ after adding the remainder. Nonnegativity gives (88). If ∣ a ( j ) ∣ ≤ R \lvert a^{(j)}\rvert\le R ∣ a ( j ) ∣ ≤ R ,
∥ F k P 0 ∥ ≤ ∑ ℓ = k + 1 t − 1 R ℓ ∥ C ℓ P 0 ∥ . (89) \lVert F_kP_0\rVert\le\sum_{\ell=k+1}^{t-1}R^\ell\lVert C_\ell P_0\rVert. \tag*{(89)} ∥ F k P 0 ∥ ≤ ℓ = k + 1 ∑ t − 1 R ℓ ∥ C ℓ P 0 ∥ . (89)
This follows by applying the triangle inequality to each block and then using the normalized stack. Together with the width theorem, (88) is a finite-radius lower bound for any retained dimension. It can be zero. Narrowing the interval worsens the conditioning of V V V , and reachability alone says nothing about the conditioning of C + P 0 C_+P_0 C + P 0 . Thus the exact k d kd k d theorem does not establish that k d kd k d coordinates are needed at an arbitrary practical tolerance.
A causal upper bound from a retained suffix
For a target recurrence with ∥ A t ( a ) ∥ ≤ κ < 1 \lVert A_t(a)\rVert\le\kappa< 1 ∥ A t ( a )∥ ≤ κ < 1 and ∥ b t ∥ ≤ B \lVert b_t\rVert\le B ∥ b t ∥ ≤ B , all states from zero satisfy ∥ h t ( a ) ∥ ≤ H = B / ( 1 − κ ) \lVert h_t(a)\rVert\le H = B/(1-\kappa) ∥ h t ( a )∥ ≤ H = B / ( 1 − κ ) . Replaying only the last L L L inputs from zero differs from full replay by
A t ( a ) ⋯ A t − L + 1 ( a ) h t − L ( a ) , (90) A_t(a)\cdots A_{t-L+1}(a)h_{t-L}(a), \tag*{(90)} A t ( a ) ⋯ A t − L + 1 ( a ) h t − L ( a ) , (90)
whose norm is at most H κ L H\kappa^L H κ L . Hence
L ≥ ⌈ log ( ϵ / H ) log κ ⌉ (91) L \ge\left\lceil\frac{\log(\epsilon/H)}{\log\kappa} \right\rceil \tag*{(91)} L ≥ ⌈ log κ log ( ϵ / H ) ⌉ (91)
suffices when 0 < ϵ < H 0 < \epsilon< H 0 < ϵ < H . An old-model checkpoint has an additional initial error, so the zero-initialized bound does not directly apply to checkpoint replay. The experiments use the stronger checkpoint baseline and measure its actual error.
For coefficient memory, write the leakage term in (19) as H f Hf H f with
f = δ L R ( 1 − ρ ) ( 1 − κ ) . (92) f = \frac{\delta L R}{(1-\rho)(1-\kappa)}. \tag*{(92)} f = ( 1 − ρ ) ( 1 − κ ) δ L R . (92)
For a normalized tolerance τ = ϵ / H \tau= \epsilon/H τ = ϵ / H , the global certificate can succeed only if f < τ f < \tau f < τ , except for degenerate zero-tail cases. If 0 < q < 1 0 < q < 1 0 < q < 1 and f < τ < 1 + f f < \tau< 1+f f < τ < 1 + f , a sufficient polynomial degree is
k ≥ max { 0 , ⌈ log ( τ − f ) log q ⌉ − 1 } . (93) k \ge\max\left\{0,\left\lceil\frac{\log(\tau-f)}{\log q}\right\rceil-1\right\}. \tag*{(93)} k ≥ max { 0 , ⌈ log q log ( τ − f ) ⌉ − 1 } . (93)
The case k = 0 k=0 k = 0 returns the anchor, and the general coefficient formula extends with no nonconstant coefficients. Degree zero is included only in the sufficient-budget diagnostic, not in the residual implementation that assumes k ≥ 1 k \ge1 k ≥ 1 . If q = 0 q=0 q = 0 , the updates vanish and the anchor is exact. These are competing sufficient causal budgets with different storage units and maintenance costs. Neither is a minimax characterization at finite tolerance.
Causal memory for a common output quotient
Corollary 14 (Output-oriented retained memory). Suppose Q Q Q has orthonormal rows, Q A i = A ˉ i Q QA_i=\bar{A}_iQ Q A i = A ˉ i Q for i = 0 , … , p i=0,\ldots,p i = 0 , … , p , and C = C ˉ Q C=\bar{C}Q C = C ˉ Q . Then Q h t Qh_t Q h t is an exact recurrent state for all outputs, including under subsequent parameter switches. If the original family has exact update closure U U U , its reduced coefficient memory needs only r o u t N p , k r_{\mathrm{out}}N_{p,k} r out N p , k scalars, where r o u t = rank ( Q U ) ≤ r r_{\mathrm{out}}=\operatorname{rank}(QU)\le r r out = rank ( Q U ) ≤ r .
Consider time-invariant matrices A 0 , A 1 , … , A p A_0,A_1,\ldots,A_p A 0 , A 1 , … , A p , an input matrix B B B , and a fixed linear readout C C C . Let Q ∈ R q × d Q\in\mathbb{R}^{q\times d} Q ∈ R q × d have orthonormal rows and suppose
Q A i = A ˉ i Q ( 0 ≤ i ≤ p ) , C = C ˉ Q . (94) QA_i=\bar{A}_iQ \quad(0\le i\le p), \qquad C=\bar{C}Q. \tag*{(94)} Q A i = A ˉ i Q ( 0 ≤ i ≤ p ) , C = C ˉ Q . (94)
These conditions mean that ker Q \ker Q ker Q is invariant under every transition component and invisible to the readout. This is a stronger condition than matching the present output or being unobservable for one fixed target parameter.
Proof of Corollary 14
Multiplying the full recurrence by Q Q Q gives the exact reduced recurrence
h ˉ t ( a ) = ( A ˉ 0 + ∑ i a i A ˉ i ) h ˉ t − 1 ( a ) + Q B x t , h ˉ t ( a ) = Q h t ( a ) . (95) \bar{h}_t(a)=\left(\bar{A}_0+\sum_i a_i\bar{A}_i\right)\bar{h}_{t-1}(a)+QB x_t,\qquad\bar{h}_t(a)=Qh_t(a). \tag*{(95)} h ˉ t ( a ) = ( A ˉ 0 + i ∑ a i A ˉ i ) h ˉ t − 1 ( a ) + QB x t , h ˉ t ( a ) = Q h t ( a ) . (95)
The output is C h t ( a ) = C ˉ h ˉ t ( a ) Ch_t(a)=\bar{C}\bar{h}_t(a) C h t ( a ) = C ˉ h ˉ t ( a ) . This identity continues to hold when the parameter changes between subsequent inputs, because (94) holds separately for each transition component.
Suppose A i = U D i A_i=UD_i A i = U D i for i ≥ 1 i\ge1 i ≥ 1 and A 0 U = U T A_0U=UT A 0 U = U T . Let U ˉ \bar{U} U ˉ be an orthonormal basis for im ( Q U ) \operatorname{im}(QU) im ( Q U ) , with r o u t = rank ( Q U ) r_{\mathrm{out}}=\operatorname{rank}(QU) r out = rank ( Q U ) . Since Q ⊤ Q^{\top} Q ⊤ is a right inverse of Q Q Q , the reduced matrices can be taken as A ˉ i = Q A i Q ⊤ \bar{A}_i=QA_iQ^{\top} A ˉ i = Q A i Q ⊤ . Define
D ˉ i = U ˉ ⊤ Q U D i Q ⊤ . (96) \bar{D}_i=\bar{U}^{\top}QUD_iQ^{\top}. \tag*{(96)} D ˉ i = U ˉ ⊤ Q U D i Q ⊤ . (96)
Then A ˉ i = U ˉ D ˉ i \bar{A}_i=\bar{U}\bar{D}_i A ˉ i = U ˉ D ˉ i , because Q A i Q ⊤ QA_iQ^{\top} Q A i Q ⊤ has image contained in im ( Q U ) \operatorname{im}(QU) im ( Q U ) . Also
A ˉ 0 ( Q U ) = Q A 0 U = Q U T , (97) \bar{A}_0(QU)=QA_0U=QUT, \tag*{(97)} A ˉ 0 ( Q U ) = Q A 0 U = Q U T , (97)
so im U ˉ \operatorname{im}\bar{U} im U ˉ is invariant under A ˉ 0 \bar{A}_0 A ˉ 0 . Proposition 11 therefore applies to the reduced family, retaining exactly its degree-k k k coefficients in r o u t N p , k r_{\mathrm{out}}N_{p,k} r out N p , k scalar locations. Differentiating the identity h ˉ t ( a ) = Q h t ( a ) \bar{h}_t(a)=Qh_t(a) h ˉ t ( a ) = Q h t ( a ) shows that these coefficients are the projections of the full coefficients. The query reproduces the degree-k k k output expansion without retaining state distinctions in ker Q \ker Q ker Q . This proves the corollary.
The reduction retains all input information needed for the outputs. The injection is explicitly Q B x t QBx_t QB x t , and the intertwining conditions ensure that discarded state directions never re-enter observable ones. When these conditions fail, simply dropping components of U U U according to the current readout is invalid.
Constructing the common quotient
Starting with the row space of C C C , repeatedly append the rows of Q A i QA_i Q A i for every i = 0 , … , p i=0,\ldots,p i = 0 , … , p , taking a basis after each step. The row space increases until it is invariant under right multiplication by all the matrices. At most d d d strict dimension increases are possible. At stabilization, an orthonormal row basis Q Q Q satisfies (94). Its kernel is the largest common invariant subspace contained in ker C \ker C ker C . Indeed, its rows are spanned by C C C followed by finite words in the matrices, and every common invariant subspace invisible to C C C is annihilated by all these words.
The construction uses simultaneous observability reduction. Its migration-specific consequence is the count of pre-query coefficients and the validity of subsequent outputs under parameter replacements. It is not a general claim that an optimal nonlinear or distribution-dependent output memory has this dimension. In floating point, numerical rank thresholds define an approximate quotient; an exact theorem requires exact intertwining, or additional error terms for its violation.
A sharp scalar-input bound independent of update dimension
This result separates persistent memory from the number of allowed update coordinates. The local-order statement is algebraic and does not provide a precision-independent stable numerical realization.
Proof of Corollary 3
Assume the anchor pair ( A 0 , b ) (A_0,b) ( A 0 , b ) is controllable and the input is scalar. Let P P P be the characteristic polynomial of A 0 A_0 A 0 , of degree d d d . Group the coefficient lift (72) by total degree. Its block diagonal consists of copies of A 0 A_0 A 0 , and every off-diagonal block increases total degree. By Cayley–Hamilton, all diagonal blocks of P ( T k ) P(T_k) P ( T k ) vanish. Hence P ( T k ) P(T_k) P ( T k ) increases the degree filtration by at least one, and
P ( T k ) k + 1 = 0. (98) P(T_k)^{k+1}=0. \tag*{(98)} P ( T k ) k + 1 = 0. (98)
This is a monic annihilating polynomial of degree d ( k + 1 ) d(k+1) d ( k + 1 ) . The reachable Krylov space of a single input has dimension at most that degree, because every higher power of T k T_k T k applied to its input vector is a linear combination of the first d ( k + 1 ) d(k+1) d ( k + 1 ) powers. Controllability of the anchor gives dim ( π 0 K k ) = d \dim(\pi_0\mathcal{K}_k)=d dim ( π 0 K k ) = d . Theorem 1 now yields
ν k ≤ d ( k + 1 ) − d = k d . (99) \nu_k \le d(k+1)-d=kd. \tag*{(99)} ν k ≤ d ( k + 1 ) − d = k d . (99)
The rank-one family in Theorem 2 attains k d kd k d , making this a sharp worst-case bound. No commutativity among A 0 , A 1 , … , A p A_0,A_1,\ldots,A_p A 0 , A 1 , … , A p was used.
The stronger direction-independent construction below shows that the retained state can be fixed without specifying the update matrices. Let z z z denote a formal delay variable, set
p ( z ) = det ( I − z A 0 ) , J ( z ) = adj ( I − z A 0 ) , (100) p(z)=\det(I-zA_0),\qquad J(z)=\operatorname{adj}(I-zA_0), \tag*{(100)} p ( z ) = det ( I − z A 0 ) , J ( z ) = adj ( I − z A 0 ) , (100)
and let Δ \Delta Δ be the matrix replacement revealed at query time. The transfer kernel of the degree-k k k Taylor expansion in Δ \Delta Δ is
H k ( z , Δ ) = ∑ ℓ = 0 k ( I − z A 0 ) − 1 [ z Δ ( I − z A 0 ) − 1 ] ℓ b (101) H_k(z,\Delta)=\sum_{\ell=0}^{k}(I-zA_0)^{-1}\left[z\Delta(I-zA_0)^{-1}\right]^\ell b \tag*{(101)} H k ( z , Δ ) = ℓ = 0 ∑ k ( I − z A 0 ) − 1 [ z Δ ( I − z A 0 ) − 1 ] ℓ b (101)
= N k ( z , Δ ) p ( z ) k + 1 , (102) =\frac{N_k(z,\Delta)}{p(z)^{k+1}}, \tag*{(102)} = p ( z ) k + 1 N k ( z , Δ ) , (102)
N k ( z , Δ ) = ∑ ℓ = 0 k [ J ( z ) z Δ ] ℓ J ( z ) b p ( z ) k − ℓ . (103) N_k(z,\Delta)=\sum_{\ell=0}^{k}[J(z)z\Delta]^\ell J(z)b\,p(z)^{k-\ell}. \tag*{(103)} N k ( z , Δ ) = ℓ = 0 ∑ k [ J ( z ) z Δ ] ℓ J ( z ) b p ( z ) k − ℓ . (103)
The order of matrix multiplication is fixed as written. Each numerator term has degree at most ℓ d + ( d − 1 ) + ( k − ℓ ) d = d ( k + 1 ) − 1 \ell d+(d-1)+(k-\ell)d=d(k+1)-1 ℓ d + ( d − 1 ) + ( k − ℓ ) d = d ( k + 1 ) − 1 . Thus all possible Δ \Delta Δ share the scalar denominator p ( z ) k + 1 p(z)^{k+1} p ( z ) k + 1 and a numerator of degree less than m = d ( k + 1 ) m=d(k+1) m = d ( k + 1 ) .
Write p ( z ) k + 1 = 1 + q 1 z + ⋯ + q m z m p(z)^{k+1}=1+q_1z+\cdots+q_mz^m p ( z ) k + 1 = 1 + q 1 z + ⋯ + q m z m , padding with zeros if necessary. Maintain the scalar filter
w t = x t − ∑ j = 1 m q j w t − j , w t = 0 ( t ≤ 0 ) , (104) w_t=x_t-\sum_{j=1}^{m}q_jw_{t-j},\qquad w_t=0\quad(t\le0), \tag*{(104)} w t = x t − j = 1 ∑ m q j w t − j , w t = 0 ( t ≤ 0 ) , (104)
and its m m m most recent values. This memory depends only on A 0 A_0 A 0 and k k k . At a query, evaluate the numerator coefficients in (103) and take their linear combination of these filter values. Formal power-series multiplication proves exact agreement with the degree-k k k truncated target for every finite history. All filter updates precede the query and require no discarded inputs.
To reduce the additional count from m m m to m − d m-d m − d , let y t = ( w t , … , w t − m + 1 ) y_t=(w_t,\ldots,w_{t-m+1}) y t = ( w t , … , w t − m + 1 ) . The constant-parameter transfer kernel J ( z ) b / p ( z ) J(z)b/p(z) J ( z ) b / p ( z ) has a numerator of degree below m m m when expressed with denominator p ( z ) k + 1 p(z)^{k+1} p ( z ) k + 1 , so h t ( 0 ) = H y t h_t(0)=Hy_t h t ( 0 ) = H y t for a fixed matrix H H H . The scalar filter has a reachable companion realization, and anchor controllability implies rank H = d \operatorname{rank}H=d rank H = d . Choose m − d m-d m − d rows S S S such that the square matrix [ H ⊤ S ⊤ ] ⊤ \begin{bmatrix}H^\top&S^\top\end{bmatrix}^\top [ H ⊤ S ⊤ ] ⊤ is invertible. Store ( h t ( 0 ) , S y t ) (h_t(0),Sy_t) ( h t ( 0 ) , S y t ) and update this coordinate transform of the companion realization. There are exactly k d kd k d auxiliary coordinates. Its query reconstructs y t y_t y t linearly and then evaluates (103) . This completes the proof. □
Accuracy and conditioning. At a fixed horizon, the target is a polynomial in the entries of Δ \Delta Δ , so the omitted terms are O ( ∥ Δ ∥ k + 1 ) O(\lVert\Delta\rVert^{k+1}) O (∥ Δ ∥ k + 1 ) in a neighborhood of zero. If ∥ A 0 ∥ ≤ ρ < 1 \lVert A_0\rVert\le\rho<1 ∥ A 0 ∥ ≤ ρ < 1 , ∥ Δ ∥ ≤ R < 1 − ρ \lVert\Delta\rVert\le R<1-\rho ∥ Δ ∥ ≤ R < 1 − ρ , and ∥ b x t ∥ ≤ B \lVert bx_t\rVert\le B ∥ b x t ∥ ≤ B , the same insertion-count argument as the coefficient-tail proof gives the horizon-uniform truncation bound
∥ h t ( A 0 + Δ ) − h ^ t ( k ) ( Δ ) ∥ ≤ B 1 − ρ − R ( R 1 − ρ ) k + 1 (105) \left\lVert h_t(A_0+\Delta)-\widehat{h}_t^{(k)}(\Delta)\right\rVert
\le\frac{B}{1-\rho-R}\left(\frac{R}{1-\rho}\right)^{k+1} \tag*{(105)} h t ( A 0 + Δ ) − h t ( k ) ( Δ ) ≤ 1 − ρ − R B ( 1 − ρ R ) k + 1 (105)
in exact arithmetic. The input filter and coordinate transform can be severely ill-conditioned near repeated poles. Their roundoff is not controlled by (105) . The implementation is therefore tested on small well-conditioned instances, not substituted for the high-dimensional empirical method.
Forming the universal numerator after an update can also be expensive. A declared-family method instead evaluates the retained coefficients directly. The bound is about persistent real-coordinate memory, not about universal superiority in bits, maintenance cost, or query time. In a scalar-input update-closed family, both constructions give the informative upper bound ν k ≤ min { k d , r N p , k } \nu_k \le\min\{kd,rN_{p,k}\} ν k ≤ min { k d , r N p , k } , but either can exceed the actual reachable dimension.
Why noncommuting updates can still share a small history basis
For a scalar anchor transition λ I r \lambda I_r λ I r and scalar injection b x t b x_t b x t , let G ( a ) = ∑ i a i G i G(a) = \sum_i a_iG_i G ( a ) = ∑ i a i G i with arbitrary, possibly noncommuting G i G_i G i . Since λ I r \lambda I_r λ I r commutes with G ( a ) G(a) G ( a ) ,
h t ( a ) = ∑ ℓ = 0 t − 1 G ( a ) ℓ b m t , ℓ , m t , ℓ = ∑ s = 0 t − 1 ( s ℓ ) λ s − ℓ x t − s , (106) h_t(a)=\sum_{\ell=0}^{t-1}G(a)^\ell b m_{t,\ell},\qquad m_{t,\ell}=\sum_{s=0}^{t-1}\binom{s}{\ell}\lambda^{s-\ell}x_{t-s}, \tag*{(106)} h t ( a ) = ℓ = 0 ∑ t − 1 G ( a ) ℓ b m t , ℓ , m t , ℓ = s = 0 ∑ t − 1 ( ℓ s ) λ s − ℓ x t − s , (106)
where terms with s < ℓ s < \ell s < ℓ vanish. The causal update is m t , 0 = λ m t − 1 , 0 + x t m_{t,0}=\lambda m_{t-1,0}+x_t m t , 0 = λ m t − 1 , 0 + x t and m t , ℓ = λ m t − 1 , ℓ + m t − 1 , ℓ − 1 m_{t,\ell}=\lambda m_{t-1,\ell}+m_{t-1,\ell-1} m t , ℓ = λ m t − 1 , ℓ + m t − 1 , ℓ − 1 . A degree-k k k response therefore uses k + 1 k+1 k + 1 scalar history moments, not one vector for every multivariate monomial. If b ≠ 0 b \ne0 b = 0 , the anchor determines m t , 0 m_{t,0} m t , 0 . Noncommutativity alone does not make the full coefficient count necessary. This shared-moment structure helps explain the efficiency of the history-basis alternative in the structured system-identification experiment.
Algorithm and complete memory accounting
A causal implementation
Algorithm 1 gives the general update without relying on a particular experimental architecture. The reference implementation uses a separate next-coefficient array for clarity. An implementation can instead update coefficients in decreasing total degree, because every nonconstant forcing term depends only on a lower degree from the previous time. The anchor must be updated last. Certificate statistics are computed from the previous coefficients before any overwrite.
An active deployed state can be advanced in parallel with this memory. Under exact closure, represent it as h 0 + U v h^0+Uv h 0 + U v for a reduced active response v ∈ R r v\in\mathbb{R}^r v ∈ R r . While the deployed coordinate is a a a , update
v t = ( T t + ∑ i a i G i , t ) v t − 1 + ∑ i a i D i , t h t − 1 0 . (107) v_t=\left(T_t+\sum_i a_iG_{i,t}\right)v_{t-1}+\sum_i a_iD_{i,t}h^0_{t-1}. \tag*{(107)} v t = ( T t + i ∑ a i G i , t ) v t − 1 + i ∑ a i D i , t h t − 1 0 . (107)
At a switch, replace v t v_t v t by ∑ α a α z t , α \sum_\alpha a^\alpha z_{t,\alpha} ∑ α a α z t , α . This shares the anchor between the auxiliary memory and deployed state. With leakage, (107) no longer equals the full-model transition. A generic implementation must then retain a separate d d d -dimensional active state to execute the full target recurrence exactly between switches.
Storage and arithmetic
The generic auxiliary memory alone contains d + r N p , k + 1 d+rN_{p,k}+1 d + r N p , k + 1 real scalars, including the anchor and radius-uniform certificate. Keeping an additional full active state gives total storage 2 d + r N p , k + 1 2d+rN_{p,k}+1 2 d + r N p , k + 1 . Under exact closure, sharing the anchor as in (107) reduces this total to d + r N p , k + r + 1 d+rN_{p,k}+r+1 d + r N p , k + r + 1 . Compared with the d d d scalars of a conventional active state, the extra storage is then r N p , k + r + 1 rN_{p,k}+r+1 r N p , k + r + 1 . Omitting
Algorithm 1 Pre-update polynomial memory
Require: A declared family (13), basis U, radius R, degree k ≥ 1
1: Enumerate I_k and precompute each predecessor α − e_i.
2: Initialize h^0 = 0, z_α = 0 for α ∈ I_k, and c = 0.
3: for each arriving input at time t do
4: Obtain A_{0,t}, D_{i,t}, b_t without using a future query parameter.
5: Obtain T_t, G_{i,t}, κ_t, δ_t, ℓ_t from the declared family.
6: Compute M_{t−1,j} from (20) and update c by (21).
7: for α ∈ I_k in decreasing total degree do
8: v ← T_t z_α
9: for i with α_i > 0 do
10: if α = e_i then
11: v ← v + D_{i,t}h^0
12: else
13: v ← v + G_{i,t}z_{α−e_i}
14: end if
15: end for
16: z_α ← v
17: end for
18: h^0 ← A_{0,t}h^0 + b_t
19: if a migration query a is revealed then
20: Check that ∥a∥_1 ≤ R.
21: Return h^0 + U ∑_{α∈I_k} a^α z_α and the bound c.
22: end if
23: end for
the optional certificate subtracts one scalar. Sparse coefficients can reduce these counts, as in the language experiment.
The model description is shared across streams and is counted separately from per-stream memory. It includes A 0 A_0 A 0 , U U U , the D i D_i D i , and any precomputed reduced matrices and norm bounds. For a dense time-invariant family, computing A 0 U A_0U A 0 U and U T A 0 U U^\mathsf{T}A_0U U T A 0 U costs O ( d 2 r + d r 2 ) O(d^2r + dr^2) O ( d 2 r + d r 2 ) , and computing the G i = D i U G_i = D_iU G i = D i U costs O ( p r 2 d ) O(pr^2d) O ( p r 2 d ) . Obtaining exact operator norms may require further matrix factorizations. These are offline costs for fixed matrices, not free operations when a family changes with each input.
For a fixed dense family, the anchor transition costs O ( d 2 ) O(d^2) O ( d 2 ) . The products D i h 0 D_i h^0 D i h 0 cost O ( p r d ) O(prd) O ( p r d ) and can be shared between coefficient and active-response updates. Propagating N p , k N_{p,k} N p , k reduced coefficients and their at most p p p predecessors costs O ( p r 2 N p , k ) O(pr^2N_{p,k}) O ( p r 2 N p , k ) . Computing all coefficient norms and degree maxima costs O ( r N p , k ) O(rN_{p,k}) O ( r N p , k ) . Polynomial evaluation costs O ( p N p , k + r N p , k ) O(pN_{p,k} + rN_{p,k}) O ( p N p , k + r N p , k ) and lifting to a full state costs O ( d r ) O(dr) O ( d r ) . None of these operations requires the discarded history. For time-varying matrices, the cost of producing the reductions must be added unless the model supplies them directly.
Storage grows with p p p , so this construction is intended for a small declared coordinate family. Low-rank matrix parameterizations with freely trainable factors can still have many update coordinates, can change the update image, and need not preserve the chosen U U U . A fixed basis of update matrices is a stricter assumption than unconstrained low-rank fine-tuning.
The implementation is included in the supplement.
Table 5. Number N p , k N_{p,k} N p , k of nonconstant monomials before structural sparsity. Multiply each entry by the reduced dimension r r r for coefficient storage.
Experimental protocols and complete results
We describe the geometry and language experiments, including data partitions, training, migration baselines, and resource accounting. The language experiment uses the Lee corpus and five independently trained models.
Dense noncommuting state-recovery experiment
For each seed s ∈ { 0 , 1 , 2 , 3 , 4 } s \in\{0,1,2,3,4\} s ∈ { 0 , 1 , 2 , 3 , 4 } , we draw an orthogonal matrix Q Q Q from a Gaussian QR factorization using NumPy’s default generator with seed s + 7301 s + 7301 s + 7301 . In the Q Q Q coordinates, begin with a 24 × 24 24 \times24 24 × 24 diagonal matrix whose diagonal ranges linearly from 0.72 to 0.96. Replace its leading 4 × 4 4 \times4 4 × 4 block by 0.94 I + 0.008 Z 0.94I + 0.008Z 0.94 I + 0.008 Z , where Z Z Z has independent standard normal entries. Fill the top-right block with 0.004 times independent standard normals. The lower-left block is λ l e a k E \lambda_{\mathrm{leak}}E λ leak E , where E E E is an independent Gaussian matrix normalized to operator norm one and
λ l e a k ∈ { 0 , 0.001 , 0.003 , 0.01 , 0.03 } . (108) \lambda_{\mathrm{leak}} \in\{0, 0.001, 0.003, 0.01, 0.03\}. \tag*{(108)} λ leak ∈ { 0 , 0.001 , 0.003 , 0.01 , 0.03 } . (108)
Normalize the entire matrix M M M to operator norm 0.97 and set A 0 = Q M Q T A_0 = QMQ^{\mathsf{T}} A 0 = QM Q T and U = Q : , 1 : 4 U = Q_{:,1:4} U = Q : , 1 : 4 . Independently draw two 4 × 24 4 \times24 4 × 24 Gaussian reduced update matrices, scale each to operator norm 0.02, and rotate their input coordinates by Q T Q^{\mathsf{T}} Q T . The resulting update matrices U D i UD_i U D i are dense and generally do not commute with each other or with A 0 A_0 A 0 . The leakage scalar controls the nominal value before normalization. The actual δ = ∥ A 0 U − U U T A 0 U ∥ \delta= \left\|A_0U - UU^{\mathsf{T}}A_0U\right\| δ = A 0 U − U U T A 0 U is stored for every measurement.
The family radius is R = 0.8 R = 0.8 R = 0.8 , giving the uniform bounds ρ = 0.97 \rho= 0.97 ρ = 0.97 , L = 0.02 L = 0.02 L = 0.02 , κ = 0.986 \kappa= 0.986 κ = 0.986 , and q = 8 / 15 q = 8/15 q = 8/15 . Each of twelve streams receives a correlated injection. Starting from s 0 = 0 s_0 = 0 s 0 = 0 , draw independent standard Gaussian vectors ξ t , η t \xi_t,\eta_t ξ t , η t and set
s t = 0.95 s t − 1 + 0.05 ξ t , b t = 0.05 tanh ( s t + 0.4 η t ) . (109) s_t = 0.95s_{t-1} + 0.05\xi_t,\qquad b_t = 0.05\tanh(s_t + 0.4\eta_t). \tag*{(109)} s t = 0.95 s t − 1 + 0.05 ξ t , b t = 0.05 tanh ( s t + 0.4 η t ) . (109)
The coordinatewise nonlinearity makes ∥ b t ∥ ≤ 0.05 24 \|b_t\| \le0.05\sqrt{24} ∥ b t ∥ ≤ 0.05 24 deterministically. For a given seed, the same input draws and query direction are used across leakage levels. A two-dimensional Gaussian direction is normalized in ℓ 1 \ell_1 ℓ 1 and scaled to each of the six radii 0.025, 0.05, 0.1, 0.2, 0.4, and 0.8. This parameter is never passed to the memory-update method. The algorithm knows only the declared family and its radius.
The degree-four memory is advanced once per input. Lower-order queries use its retained lower-degree coefficients, which are unaffected by higher degrees. Separate certificate recurrences are advanced for degrees one through four. Reference states are computed by independent full recurrences at the selected query parameters. Errors are measured at prefixes 256, 1024, and 4096. The Cartesian product of five seeds, five leakage levels, three prefixes, four degrees, six query radii, and twelve streams yields exactly 21,600 measurements.
Figure 2. State-recovery error versus history length at zero nominal leakage and query radius 0.4. Error bars are the sample standard deviation of five per-seed stream averages. The guarantee is uniform in the horizon, not monotonic in the horizon. The fluctuations do not contradict a horizon-uniform bound.
For each plotted setting, the twelve stream errors are averaged within seed, and the five resulting averages determine the plotted mean and sample standard deviation. The slopes in Figure 3 are least-squares slopes of log mean error on log radius using only the three smallest query radii at zero leakage and prefix 4096. The fitted slopes are 1.9962, 2.9926, 3.9913, and 4.9875. The highest-order errors at small radii approach floating-point scales; the slopes are not arbitrary-precision estimates.
Figure 3. Dense noncommuting structural test of approximation order. State error versus query radius at zero leakage. Error bars are the sample SD of five seed-level averages over twelve streams.
All measured errors lie below both (19) and (21) , allowing an absolute numerical tolerance of 10 − 10 10^{-10} 1 0 − 10 . The online bound uses the full radius 0.8 even when the query radius is 0.025, so it is not expected to be tight for every query. Absence of sampled violations does not imply probabilistic coverage. The bounds hold deterministically in exact arithmetic; these comparisons use finite-precision calculations.
Language data and architecture
We use the Lee news corpus distributed with Gensim ((Gensim developers, n.d.), containing 360,082 bytes and 300 newline-delimited records. We operate directly on bytes, with all 256 values assigned embeddings and logits. No text decoding, Unicode normalization, vocabulary fitting, or document-level sampling is applied. The four disjoint contiguous ranges, in zero-based half-open byte offsets, are
[ 0 , 216049 ) , [ 216049 , 252057 ) , [ 252057 , 288065 ) , [ 288065 , 360082 ) . (110) [0, 216049),\quad[216049, 252057),\quad[252057, 288065),\quad[288065, 360082). \tag*{(110)} [ 0 , 216049 ) , [ 216049 , 252057 ) , [ 252057 , 288065 ) , [ 288065 , 360082 ) . (110)
They are used respectively for base training, adapter tuning, stitching calibration, and evaluation. The remaining evaluation bytes beyond the eight selected streams are unused. Corpus access details are provided in the supplement; the source news text is not redistributed.
The recurrence has d = 64 d = 64 d = 64 real states. With a learned embedding e ( x ) ∈ R 64 e(x) \in\mathbb{R}^{64} e ( x ) ∈ R 64 and fixed base decays λ j \lambda_j λ j , it is
h t , j ( a ) = ( λ j + 0.002 a j m o d 21 { j < 8 } ) h t − 1 , j ( a ) + ( 1 − λ j ) e j ( x t ) , j = 0 , … , 63. (111) h_{t,j}(a) = \left(\lambda_j + 0.002a_j \bmod21\{j < 8\}\right)h_{t-1,j}(a) + (1-\lambda_j)e_j(x_t), \qquad j = 0,\ldots,63. \tag*{(111)} h t , j ( a ) = ( λ j + 0.002 a j mod 21 { j < 8 } ) h t − 1 , j ( a ) + ( 1 − λ j ) e j ( x t ) , j = 0 , … , 63. (111)
The first eight decays are evenly spaced from 0.980 to 0.996, and the other 56 from 0.05 to 0.95. The injection always uses the base decay, so it is independent of the adapter coordinate. The nonlinear logits are
g ( h t , x t ) = W o tanh ( W h h t + b h + W s e ( x t ) ) + b o , (112) g(h_t,x_t) = W_o \tanh(W_hh_t + b_h + W_se(x_t)) + b_o, \tag*{(112)} g ( h t , x t ) = W o tanh ( W h h t + b h + W s e ( x t )) + b o , (112)
with hidden width 128 and output width 256. The skip matrix W s W_s W s has no bias. Base training learns the embedding and the prediction head, not the decay or update-direction buffers. Embeddings are initialized from a zero-mean Gaussian with standard deviation 0.35 and the linear layers use PyTorch’s default initialization. The model is small, and its direct input-to-readout path limits the predictions’ dependence on the migrated state.
Each base model is trained from an independent seed in { 0 , 1 , 2 , 3 , 4 } \{0,1,2,3,4\} { 0 , 1 , 2 , 3 , 4 } for 600 AdamW steps, with learning rate 0.002, weight decay 0.01, batch size 12, and sequence length 256. The optimizer uses its library-default moment coefficients. Random contiguous training blocks are sampled with replacement. Every block starts from state zero, and cross-entropy excludes its first 64 positions. Gradients are clipped to norm one. Training computes the diagonal recurrence by FFT convolution, while evaluation uses a causal filtering implementation. The saved checkpoints contain the trained embedding and head, together with the fixed recurrence buffers.
After base training, these weights are frozen. Starting from a = 0 a = 0 a = 0 , Adam with learning rate 0.035 performs eight stages of 15 steps, using only the tuning split. Each step samples 12 length-512 blocks, excludes the first 128 positions from its loss, and projects a a a onto the ℓ 1 \ell_1 ℓ 1 ball of radius 0.75. This gives eight successive learned coordinates per seed. The learned update schedules are included in the supplement. The projection and training weights use float32, so the stored coordinates can exceed the nominal radius by at most 4.5 × 10 − 8 4.5 \times10^{-8} 4.5 × 1 0 − 8 in these runs. This excess is due to numerical rounding and does not change the adaptation task. Exact-arithmetic statements about the experiment can use the enclosing radius 0.750001; no machine-validated certificate is claimed.
Figure 4. Increasing degree under controlled off-subspace leakage. Error bars are the sample SD of five seed-level averages over twelve streams. Displayed leakage is nominal before normalization.
Migration protocol and baselines
Each seed is evaluated on eight contiguous held-out streams, each with 8192 inputs and one additional byte for next-byte labels. The last label byte of one stream is the first input byte of the next, but their 8192-input histories do not overlap. Each history starts from state zero. The successive replacement times are
1024 , 2048 , 3072 , 4096 , 5120 , 6144 , 7168 , 7936. (113) 1024,\ 2048,\ 3072,\ 4096,\ 5120,\ 6144,\ 7168,\ 7936. \tag*{(113)} 1024 , 2048 , 3072 , 4096 , 5120 , 6144 , 7168 , 7936. (113)
Between replacements, every baseline advances its own active state with the deployed coordinate. At a replacement, the reference is a full replay of the complete current prefix under the new coordinate. Evaluation copies each migrated state forward for 31 additional inputs, producing the current next-byte prediction and 31 subsequent predictions. These scoring copies do not overwrite the active state or change the next epoch’s starting time.
Full replay, reset, and stale reuse. Full replay is the exact counterfactual reference. Since only eight coordinates depend on the update, all methods preserve the unaffected 56 coordinates exactly. Reset zeros only the eight affected coordinates, making it a stronger baseline than resetting the entire state. Stale reuse retains its current affected state without transformation. After migration, both baselines continue with the new transition.
Checkpoint replay. For a suffix of length L ∈ { 64 , 256 } L \in\{64,256\} L ∈ { 64 , 256 } , the baseline retains the affected active state at time t − L t-L t − L and the next L L L raw bytes. When the replacement is revealed at t t t , it replays those bytes from the retained checkpoint under the new coordinate. The checkpoint itself was generated under the previous deployed model and can include error from earlier migrations. The protocol uses neither zero initialization nor an updated-model checkpoint that would require replaying earlier inputs. It assumes update times can be announced L L L inputs ahead, but does not reveal the new coordinate early.
Linear and nonlinear stitching. For each adjacent pair of learned coordinates, exact old and new states are computed from the distinct calibration split. The full 64-dimensional old state is used as a feature, and the target is the correction to its first eight coordinates. States are sampled every four positions after a 512-position warmup, yielding 8874 candidate pairs. Only the first 80% of these pairs are used for fitting, and the final 20% are unused. No hyperparameters are selected on either those unused pairs or the test streams.
Features are centered and scaled by their fitting-set standard deviation plus 10 − 5 10^{-5} 1 0 − 5 . The linear stitch is ridge regression with an intercept and penalty 10 − 3 10^{-3} 1 0 − 3 applied to all design coefficients. The nonlinear stitch has widths 64 → 64 → 8 64 \to64 \to8 64 → 64 → 8 with a tanh hidden activation. Its targets are scaled by their fitting-set standard deviation plus 10 − 6 10^{-6} 1 0 − 6 . It is trained for 250 AdamW steps with batch size 256, learning rate 0.002, and weight decay 0.001. These maps are shared across evaluation streams. Each adjacent model pair has its own maps. Calibration replays are permitted on the separate calibration data, not on unavailable deployment histories.
Sensitivities and UCM. The diagonal structure makes each affected state coordinate depend on only one adapter coordinate. With z t , 0 , j = h t , j 0 z_{t,0,j}=h^0_{t,j} z t , 0 , j = h t , j 0 and zero initial coefficients,
z t , ℓ , j = λ j z t − 1 , ℓ , j + 0.002 z t − 1 , ℓ − 1 , j , ℓ ≥ 1 , j < 8. (114) z_{t,\ell,j}=\lambda_jz_{t-1,\ell,j}+0.002z_{t-1,\ell-1,j}, \qquad\ell\ge1,\ j<8. \tag*{(114)} z t , ℓ , j = λ j z t − 1 , ℓ , j + 0.002 z t − 1 , ℓ − 1 , j , ℓ ≥ 1 , j < 8. (114)
The degree-k k k query is h t , j 0 + ∑ ℓ = 1 k a j ℓ m o d 2 z t , ℓ , j h^0_{t,j}+\sum_{\ell=1}^{k}a_j^{\ell\ \mathrm{mod}\ 2}z_{t,\ell,j} h t , j 0 + ∑ ℓ = 1 k a j ℓ mod 2 z t , ℓ , j . All other multivariate coefficients are identically zero. Exact first-order sensitivity is this formula with k = 1 k=1 k = 1 , rather than a truncated-history approximation. UCM uses k = 2 k=2 k = 2 and k = 3 k=3 k = 3 . None is re-centered after an update.
The UORO-style baseline approximates the 8 × 2 8 \times2 8 × 2 anchor sensitivity by an outer product u v T uv^{\mathsf{T}} u v T . It uses independent Rademacher vectors for each stream, propagates the first factor through the diagonal anchor Jacobian, and applies a norm-balanced rank-one update. A symmetric 10 − 30 10^{-30} 1 0 − 30 stabilization is used in scale denominators. The random seed is 100 s + j + 666 100s+j+666 100 s + j + 666 for model seed s s s and stream j j j . The baseline uses UORO-style rank-one sensitivity updates, but does not reimplement the full recurrent-training procedure. Its migration query remains first-order in a a a , so it does not remove Taylor truncation error.
Quadratic interpolation bank. Before the stream, maintain the affected anchor and two auxiliary filters per affected coordinate, one at coordinate value + 0.75 +0.75 + 0.75 and one at − 0.75 -0.75 − 0.75 . For a query scalar a a a associated with that state coordinate, return
h 0 + a 2 R b ( h + − h − ) + a 2 2 R b 2 ( h + + h − − 2 h 0 ) , R b = 0.75. (115) h^0+\frac{a}{2R_b}(h^+-h^-)+\frac{a^2}{2R_b^2}(h^++h^--2h^0), \qquad R_b=0.75. \tag*{(115)} h 0 + 2 R b a ( h + − h − ) + 2 R b 2 a 2 ( h + + h − − 2 h 0 ) , R b = 0.75. (115)
This is ordinary three-node quadratic interpolation with exactly two extra filter states per affected coordinate, matching the coefficient budget of degree-two UCM. Because coordinates are independent, the two auxiliary global filters can assign both coordinates + R b +R_b + R b or both − R b -R_b − R b . Those auxiliary vectors are outside the permitted replacement ℓ 1 \ell_1 ℓ 1 ball, but their scalar filters are stable and are defined before the stream. The method never uses a future replacement value to choose a filter. The comparison tests Taylor memory against an interpolation alternative at the same coefficient budget, beyond the first-order approximation baseline.
Table 6. All-seed state NRMSE, in percent. Each entry aggregates the eight streams and eight update events belonging to one trained model.
Aggregation and predictive-loss results
For a seed s s s , let E s \mathcal{E}_s E s be its 64 stream/update pairs. With errors measured on the affected coordinates, define
NRMSE s = ( ∑ e ∈ E s ∥ h ^ e − h e r e p l a y ∥ 2 ∑ e ∈ E s ∥ h e r e p l a y ∥ 2 ) 1 / 2 . (116) \operatorname{NRMSE}_s = \left(\frac{\sum_{e\in\mathcal{E}_s}\left\|\widehat{h}_e-h_e^{\mathrm{replay}}\right\|^2}{\sum_{e\in\mathcal{E}_s}\left\|h_e^{\mathrm{replay}}\right\|^2}\right)^{1/2}. \tag*{(116)} NRMSE s = ∑ e ∈ E s h e replay 2 ∑ e ∈ E s h e − h e replay 2 1/2 . (116)
The manuscript reports the arithmetic mean and sample standard deviation of these five seed-level ratios. It does not average per-event relative errors, whose small denominators can distort the comparison. Predictive KL is averaged over each 32-position window, then over the 64 events, then across model seeds. All scoring uses float64 filtering and logits. Tiny negative KL values caused by rounding are clipped to zero. Differences in bits per byte are signed differences from the same seed and event’s full-replay loss, not absolute discrepancies.
Reset produces a predictive discrepancy, whereas several alternatives already yield small KL. Linear stitching gives lower state error than degree-two UCM, and the relative ordering of state error and predictive loss is not identical. The equal-memory cubic bank has smaller mean state error and KL than degree three, while both methods change true predictive loss negligibly. These results do not support claims of improved perplexity, adaptation quality, or general language-model performance.
Timing, calibration, and resource accounting
Timing uses the first trained model, its eight-update replacement schedule, and one 8192-byte held-out stream. The CPU is an AMD EPYC 9V74 80-Core Processor. The measured kernels use Numba and float64, with single-threaded execution throughout. The recorded software environment includes Python 3.13, NumPy 2.3.5, and PyTorch 2.10.0+cpu. A warmup compiles each kernel. configuration. Twenty-one subsequent repetitions randomize the order of methods using a fixed trial-dependent permutation.
Table 7. Ground-truth predictive loss relative to full replay. The middle column is in micro-bits per byte and reports mean ± \pm ± sample standard deviation across five seeds. The right column also gives the seed-level minimum and maximum in micro-bits per byte. Negative values are lower empirical loss than replay, not necessarily better state fidelity.
Figure 5. State fidelity versus additional logical memory in the language diagnostic. Points and error bars are mean and sample SD across five model seeds. The equal-memory cubic bank is included. Shared fitting costs are not per-stream history bytes.
Every measured whole-stream call includes state initialization, embedding lookup, active-state updates, anchor and auxiliary-state maintenance when required, and all eight migrations. The state-only version does not evaluate logits. The second version evaluates the full nonlinear prediction head at every input for every method. All methods share the unaffected state coordinates, and methods that do not need the affected anchor do not update those anchor coordinates. Common-kernel scratch arrays are not a measured peak-memory claim. Logical retained-state counts in Table 2 reflect the information needed by each method, not incidental allocations in the reference implementation.
Table 2. Language migration over 320 events per method. Mean ± \pm ± sample SD across five models. Extra bytes exclude the common active state. Predictive KL averages the first 32 post-update positions.
Table 8. Complete timing results. Entries are median [25th percentile, 75th percentile] across 21 repetitions. Times are for one complete stream, not one migration.
The certificate timing variant adds the degree-three radius-uniform residual update with enclosing radius 0.750001, using operator-norm bounds computed from the stored decay and direction arrays and the highest-degree coefficient norm over each parameter group. It is an optional cost beyond the coefficient-only variant. Timing excludes training, compilation, calibration, and loading the data and models. Training and calibration costs are reported separately, without assuming a deployment count over which to amortize them. The calibration measurements include exact state-pair construction and fitting both the linear and nonlinear maps. They were collected while other experiment processes could be active, so they should not be used as isolated hardware performance claims.
For logical memory, float64 states use eight bytes per coordinate and retained raw inputs use one byte per token. A degree-k k k method stores an additional eight-dimensional anchor and 8 k 8k 8 k coefficients beyond the active state, giving 64 ( k + 1 ) 64(k + 1) 64 ( k + 1 ) bytes. Degree three therefore needs 256 extra bytes, or 264 with the optional scalar certificate. The quadratic bank needs 192 bytes and the cubic bank needs 256. Checkpoint replay needs 64 + L 64 + L 64 + L bytes. The rank-one sensitivity baseline needs eight anchor values, eight values for one factor, and two for the other, or 144 bytes. Full replay needs t t t retained input bytes at prefix t t t . Stitching has no additional per-stream historical state, but requires shared maps and calibration data. A linear map has 65 × 8 = 520 65 \times8 = 520 65 × 8 = 520 regression coefficients plus feature normalization parameters. The nonlinear map has 4680 network parameters plus feature and target normalization parameters. These are incurred per adjacent model pair, not per stream. The timing comparison includes both fitted stitchers and the cubic bank, with calibration recorded separately. UORO is included in the accuracy table but is not timed in this deterministic serving comparison. Model weights and the current global adapter coordinate are common deployment costs.
The work measurements do not support a general speed advantage. For this small model, checkpoint replay is a strong inexpensive alternative, and the prediction head largely masks state-maintenance differences. The timing table reports this trade-off across the whole stream, rather than only the polynomial query’s latency.
Evidence boundaries
The controlled numerical streams test the mathematical mechanism, including dense noncommuting transitions and failure under leakage. They are not a substitute for forecasting, control, or anomaly-detection benchmarks. The language study tests five small learned models with a particularly favorable invariant structure and a modest update radius. It does not test pretrained models, unknown update subspaces, multi-layer input changes, or arbitrary continual fine-tuning. The corpus is small, and no claim about broad linguistic generalization is made. The direct prediction-head skip path is another reason not to infer task improvements from the large relative state-error reductions.
The experiments provide evidence about the implemented retained-memory mechanism within the stated assumptions. The theoretical lower bounds explain why those assumptions matter. Whether a practical application benefits from the mechanism depends on the closure dimension, the tolerated predictive discrepancy, the update frequency, the available replay buffer, and the maintenance cost paid before each update.
Additional protocols and evidence
We give the finite-accuracy calculations, equal-memory comparisons, coupled identification task, and cost accounting used in the main text. The language comparisons use the same five trained models described in Appendix M.2, changing only the migration method. The models and numerical results are included in the supplement.
Finite-query spectra and precision budgets
The rank-one spectral diagnostic takes d ∈ { 1 , 4 , 8 } d \in\{1,4,8\} d ∈ { 1 , 4 , 8 } ,
A = diag ( linspace ( 0.3 , 0.9 , d ) ) , v = ( 1 , 2 , … , d ) T ∥ ( 1 , 2 , … , d ) ∥ , D = 0.15 v v T , b = 1 d . (117) A = \operatorname{diag}(\operatorname{linspace}(0.3,0.9,d)), \qquad v = \frac{(1,2,\ldots,d)^{\mathsf{T}}}{\lVert(1,2,\ldots,d)\rVert}, \qquad D = 0.15vv^{\mathsf{T}}, \qquad b = \frac{1}{\sqrt{d}}. \tag*{(117)} A = diag ( linspace ( 0.3 , 0.9 , d )) , v = ∥( 1 , 2 , … , d )∥ ( 1 , 2 , … , d ) T , D = 0.15 v v T , b = d 1 . (117)
For d = 1 d=1 d = 1 , the single diagonal entry is 0.3. The horizon is t ∈ { 3 d , 8 d , 64 d } t \in\{3d,8d,64d\} t ∈ { 3 d , 8 d , 64 d } and the query radius is R ∈ { 0.01 , 0.05 , 0.2 , 0.5 } R \in\{0.01,0.05,0.2,0.5\} R ∈ { 0.01 , 0.05 , 0.2 , 0.5 } . For each configuration, use 33 nodes
a j = R cos ( ( 2 j + 1 ) π 66 ) , j = 0 , … , 32. (118) a_j = R\cos\left(\frac{(2j+1)\pi}{66}\right), \qquad j=0,\ldots,32. \tag*{(118)} a j = R cos ( 66 ( 2 j + 1 ) π ) , j = 0 , … , 32. (118)
All queried recurrences are contractive, since ∥ A ∥ + 0.15 R ≤ 0.975 \lVert A\rVert+0.15R \le0.975 ∥ A ∥ + 0.15 R ≤ 0.975 . The horizon map is formed explicitly as [ A ( a ) t − 1 b , … , b ] [A(a)^{t-1}b,\ldots,b] [ A ( a ) t − 1 b , … , b ] . An orthonormal basis Z Z Z for ker R 0 \ker R_0 ker R 0 is obtained by singular-value decomposition, and the reported spectrum is that of R Q Z R_QZ R Q Z . It has the same nonzero singular values as R Q P 0 R_QP_0 R Q P 0 and avoids forming a large projector.
The d = 4 d=4 d = 4 , t = 12 t=12 t = 12 , R = 0.2 R=0.2 R = 0.2 example has necessary continuous dimensions 0, 1, 1, 3, 5 at RMS tolerances 10 − 2 10^{-2} 1 0 − 2 , 10 − 3 10^{-3} 1 0 − 3 , 10 − 4 10^{-4} 1 0 − 4 , 10 − 6 10^{-6} 1 0 − 6 , 10 − 8 10^{-8} 1 0 − 8 . The auxiliary-bit lower bounds are 0.00000, 1.90465, 5.22657, 20.78339, 49.74601 before rounding to an integer. Table 9 gives the tolerance budgets, and Figure 6 shows how the conditional spectrum varies with query radius. These are fixed-horizon calculations on the unit-energy history ball, not claims of a cheap causal encoder.
The leading five conditional singular values at radius 0.2 are 3.74416862 × 10 − 3 3.74416862 \times10^{-3} 3.74416862 × 1 0 − 3 , 6.95036610 × 10 − 5 6.95036610 \times10^{-5} 6.95036610 × 1 0 − 5 , 6.93525064 × 10 − 6 6.93525064 \times10^{-6} 6.93525064 × 1 0 − 6 , 8.47191936 × 10 − 7 8.47191936 \times10^{-7} 8.47191936 × 1 0 − 7 , and 6.17494116 × 10 − 8 6.17494116 \times10^{-8} 6.17494116 × 1 0 − 8 . One implementation uses explicit powers and an orthonormal null-space basis. A second builds the maps recursively and removes the anchor row space. Their maximum absolute discrepancy is 1.1 × 10 − 16 1.1 \times10^{-16} 1.1 × 1 0 − 16 . The full numerical results are included in the supplement.
Table 9. Conditional information for the stated four-state, horizon-12, radius-0.2 example. The bit column is a real-valued lower bound on an integer message length, with an exact old state supplied separately.
Figure 6. RMS-normalized conditional singular values for the four-state rank-one family at horizon 12. The query radius changes finite-tolerance requirements even when exact rank is unchanged. The 33 nodes are stated above.
For the language recurrence, we also evaluate the continuous relaxation in which each of the eight affected coordinates has an independently varying injection. For each base decay λ j \lambda_j λ j , take the scalar injection gain 1 − λ j 1-\lambda_j 1 − λ j , update slope 0.002, 17 Chebyshev nodes, radii 0.1, 0.4, 0.75, and horizons 256, 1024, 4096. The direct-sum conditional spectrum is obtained by combining the eight scalar spectra. This history ball contains injection sequences that actual byte embeddings cannot generate. The spectrum therefore describes the recurrent operators, not an information lower bound for the discrete language data.
The memory-budget comparison uses the sufficient bounds of Appendix § 19 .§ 4 . It varies ρ ∈ { 0.9 , 0.97 , 0.99 , 0.995 , 0.999 } \rho\in\{0.9, 0.97, 0.99, 0.995, 0.999\} ρ ∈ { 0.9 , 0.97 , 0.99 , 0.995 , 0.999 } , q ∈ { 0.05 , 0.2 , 0.4 , 0.7 , 0.9 , 0.99 } q \in\{0.05, 0.2, 0.4, 0.7, 0.9, 0.99\} q ∈ { 0.05 , 0.2 , 0.4 , 0.7 , 0.9 , 0.99 } , p ∈ { 1 , 2 , 4 } p \in\{1, 2, 4\} p ∈ { 1 , 2 , 4 } , r ∈ { 2 , 4 , 8 } r \in\{2, 4, 8\} r ∈ { 2 , 4 , 8 } , normalized tolerance τ ∈ { 0.1 , 0.01 , 0.001 } \tau\in\{0.1, 0.01, 0.001\} τ ∈ { 0.1 , 0.01 , 0.001 } , and relative leakage δ / ( 1 − ρ ) ∈ { 0 , 10 − 4 , 10 − 3 , 10 − 2 } \delta/(1-\rho) \in\{0, 10^{-4}, 10^{-3}, 10^{-2}\} δ / ( 1 − ρ ) ∈ { 0 , 1 0 − 4 , 1 0 − 3 , 1 0 − 2 } . An entry marked infeasible means that the displayed sufficient bound cannot certify the tolerance, not that migration itself is impossible.
Table 10. Complete language comparison, including all replay, stitching, sensitivity, coefficient, and interpolation baselines. Entries are mean ± \pm ± sample SD across the five trained models. State errors are measured only on affected coordinates. KL is in nats.
Equal-memory cubic interpolation in the language model
The cubic bank uses four fixed nodes
ξ j = 0.750001 cos ( ( 2 j + 1 ) π 8 ) , j = 0 , 1 , 2 , 3. (119) \xi_j = 0.750001 \cos\left(\frac{(2j+1)\pi}{8}\right), \qquad j=0,1,2,3. \tag*{(119)} ξ j = 0.750001 cos ( 8 ( 2 j + 1 ) π ) , j = 0 , 1 , 2 , 3. (119)
For each affected coordinate, it retains the four scalar filter states at those nodes and evaluates the ordinary Lagrange interpolant when the replacement is revealed. The nodes are neither selected from a test query nor calibrated on evaluation histories. A separable coordinate uses only its own parameter, so each scalar node is also a valid query on that coordinate axis. Bundling independent coordinate filters into arrays does not require treating a vector with both coordinates equal to a node as an allowed model replacement.
The bank stores four float64 values per affected coordinate, or 256 extra bytes in total, with no affected anchor needed by the bank. Degree-three coefficient memory stores an affected anchor and three coefficients per coordinate, also 256 bytes. Both methods keep an active state for ordinary execution between replacements. The unaffected 56 coordinates are shared. Neither method is given the replacement coordinate during historical maintenance. The bank and coefficient methods may receive multiple subsequent queries without changing their underlying retained memory.
The cubic bank improves mean state NRMSE and mean KL relative to degree-three coefficients. Its state error is lower in three seeds and higher in two (Figure 10 ). The ordering is specific to this experiment. Interpolation accuracy depends on node placement, parameter dimension, and query location. A comparison against first-order sensitivity alone cannot identify the best use of the extra memory. The signed empirical loss differences for both methods remain on the micro-bit scale.
Figure 10. The same five language models evaluated with degree-three coefficient memory and equal-memory cubic interpolation. Lines join methods within a fitted-model seed. The figure reports migration-state fidelity, not a meaningful improvement in predictive loss.
Linear and nonlinear stitching maps use the independent calibration split, seeds, optimizer settings, and sample counts described above. Map application is timed separately from fitting. The timing comparison includes both stitchers and cubic interpolation. UORO remains in the accuracy comparison but has no measurement in the deterministic kernel timing table; its cost is not zero.
Learning the coupled model and its update family
The coupled study is a fully observed synthetic system-identification problem. Its observation model provides states during calibration, and its data-generating system deliberately contains an invariant slow block. These assumptions permit learning a useful restricted update family. They do not establish that comparable structure can be learned in an arbitrary pretrained neural model. The matrices are dense in the reported coordinates, and the perturbations need not commute. The scalar-input setting also provides an appropriate test of history features that can be shared across coefficient directions.
For seed s ∈ { 0 , 1 , 2 , 3 , 4 } s \in\{0,1,2,3,4\} s ∈ { 0 , 1 , 2 , 3 , 4 } , initialize NumPy’s generator with 18001 + s 18001+s 18001 + s . Obtain an orthogonal 128 × 128 128 \times128 128 × 128 matrix Q ∗ Q_* Q ∗ from a Gaussian QR factorization, and let U ∗ U_* U ∗ be its first eight columns. For a selected ρ ∈ { 0.97 , 0.995 , 0.999 } \rho\in\{0.97,0.995,0.999\} ρ ∈ { 0.97 , 0.995 , 0.999 } , set
A ∗ = Q ∗ diag ( ρ I 8 , linspace ( 0.5 , 0.9 , 120 ) ) Q ∗ T . (120) A_* = Q_* \operatorname{diag}(\rho I_8,\operatorname{linspace}(0.5,0.9,120))Q_*^{\mathsf{T}}. \tag*{(120)} A ∗ = Q ∗ diag ( ρ I 8 , linspace ( 0.5 , 0.9 , 120 )) Q ∗ T . (120)
Draw independent 8 × 8 8 \times8 8 × 8 Gaussian matrices, normalize their operator norms to 0.6 ( 1 − ρ ) 0.6(1-\rho) 0.6 ( 1 − ρ ) , and lift them through U ∗ U_* U ∗ to obtain two true update matrices D 1 ∗ , D 2 ∗ D_1^*,D_2^* D 1 ∗ , D 2 ∗ . Their images are in the slow block. The scalar injection vector has independent Gaussian slow coordinates and Gaussian fast coordinates scaled by 0.2 0.2 0.2 , is normalized by the norm of its slow coordinates, and is then rotated by Q ∗ Q_* Q ∗ and multiplied by 1 − ρ 2 \sqrt{1-\rho^2} 1 − ρ 2 . The scalar readout is a fixed known random unit vector in im U ∗ \operatorname{im} U_* im U ∗ and is not estimated from outputs alone. There is no direct input-to-output path.
The learner first estimates the entire source transition A A A from 4096 independent, randomly initialized one-step observations. The initial states and scalar inputs are standard Gaussian, and the next-state observations have independent Gaussian noise with standard deviation 0.6 ( 1 − ρ ) × 10 − 3 0.6(1-\rho) \times10^{-3} 0.6 ( 1 − ρ ) × 1 0 − 3 . The known injection b x b_x b x is subtracted before least squares. These are observed-state calibration data, not latent-state sequence supervision.
Six pilot domains use coefficients ( 0.6 , 0 ) , ( − 0.6 , 0 ) , ( 0 , 0.6 ) , ( 0 , − 0.6 ) , ( 0.3 , 0.3 ) , ( − 0.3 , − 0.3 ) (0.6,0),(-0.6,0),(0,0.6),(0,-0.6),(0.3,0.3),(-0.3,-0.3) ( 0.6 , 0 ) , ( − 0.6 , 0 ) , ( 0 , 0.6 ) , ( 0 , − 0.6 ) , ( 0.3 , 0.3 ) , ( − 0.3 , − 0.3 ) in the true update basis. Each supplies 2048 independent observations and an unrestricted transition estimate. Subtract the fitted source transition, flatten the six differences, and retain the least number of right-singular directions explaining 99.9% of squared singular-value mass. This selects p = 2 p=2 p = 2 for every reported fit. Concatenate the images of those directions and apply the same energy rule to obtain r = 8 r=8 r = 8 . No evaluation history participates in either SVD. The retained image basis is the learned U U U , and its two reduced directions are normalized to operator norm 0.6 ( 1 − ρ ) 0.6(1-\rho) 0.6 ( 1 − ρ ) .
Four further domains use the true coefficient pairs ( 0.55 , 0.25 ) , ( − 0.45 , 0.35 ) , ( 0.25 , − 0.55 ) (0.55,0.25),(-0.45,0.35),(0.25,-0.55) ( 0.55 , 0.25 ) , ( − 0.45 , 0.35 ) , ( 0.25 , − 0.55 ) , and ( − 0.45 , − 0.35 ) (-0.45,-0.35) ( − 0.45 , − 0.35 ) . Each supplies 2048 new observed transitions. The restricted model fits two coordinates in A + U ∑ i α i D i A+U\sum_i \alpha_iD_i A + U ∑ i α i D i by least squares. The declared rule scales a coordinate vector onto the unit ℓ 1 \ell_1 ℓ 1 ball if its norm exceeds one. No projection is triggered in the recorded fits. The unrestricted comparator estimates a full transition from the same observations. Both are evaluated against the true domain’s conditional mean. This comparison isolates task loss from errors due to using a restricted adaptation family.
The fitted bases are not exactly invariant. At the main setting ρ = 0.995 \rho=0.995 ρ = 0.995 , measured leakage ranges from 1.80 × 10 − 4 1.80 \times10^{-4} 1.80 × 1 0 − 4 to 5.01 × 10 − 4 5.01 \times10^{-4} 5.01 × 1 0 − 4 , or 3.59% to 10.01% of the fitted source stability margin. Relative leakage rises as the margin narrows. All fitted families satisfy the recorded operator-norm contraction bound for the full declared radius. Gaussian inputs are not uniformly bounded over all realizations, so the global bounded-injection theorem is not asserted uniformly over this data distribution. A finite run has a finite realized injection bound, and the residual theorem does not require a fixed global injection bound. The experiments report measured errors separately from the conservative leakage bound.
Independent calibration and streaming evaluation
The separate decoder-calibration history contains 8192 independent standard Gaussian inputs, cast to float32, from seed 26001 + s s s . It is processed under the fitted source and four target matrices. State-pair samples use positions 1024, 1028, . . . , 8188. The first 75% of these samples fit each regression and the last 25% select among ridge penalties 10 − 8 10^{-8} 1 0 − 8 , 10 − 6 10^{-6} 1 0 − 6 , 10 − 4 10^{-4} 1 0 − 4 , 10 − 2 10^{-2} 1 0 − 2 . Feature means and standard deviations are estimated only on the fitting portion. Five stitching maps cover source-to-first-target and the four cyclic target transitions. The fitted history baseline uses four separate maps from its retained history coordinates to the target state. All calibration is independent of evaluation histories.
For held-out stream j j j in seed s s s , the generator seed is 37001 + 100s s s + j j j . Each stream has 4096 independent standard Gaussian float32 inputs and starts from zero. The main update period is 512 inputs. The seven replacement times are 512, 1024, . . . , 3584, and the four domain coordinates repeat cyclically. The memory is initialized before the stream and knows the fitted family and its radius, but no query coordinate is supplied to its update. At each event, the reference processes the whole prefix under that single target matrix. The prediction window comprises the current output and 31 subsequent common inputs. Scoring copies do not overwrite a method’s active state or advance its next event.
The contract is retrospective re-estimation under a newly selected domain model. The protocol does not change physical dynamics retrospectively or assume that a filter should always discard the true historical parameter sequence. Each baseline advances its own active state between events. Checkpoint replay retains its actual state L L L inputs before the announced event, even if that suffix crosses an earlier replacement. It then replays the suffix with the new target transition. Replacement times are announced for checkpoint capture, but replacement coordinates are not.
For every seed and method, sum squared prediction errors across all events, streams, and 32-position windows, and divide by the corresponding sum of squared clean target outputs. This defines the reported normalized MSE. The five seed ratios then determine the mean and sample SD.
Table 12: All methods on the coupled task at ρ = 0.995 \rho= 0.995 ρ = 0.995 and update period 512. Prediction NMSE is relative to the true conditional mean. Stream times are measured only for the indicated serving kernels. The analytic history bank and full replay have nearly identical task error, which is then dominated by fitting the replacement model rather than migration.
Table 12.
Table 13: Every seed’s coupled-task NMSE. The individual values show the between-seed variation hidden by a pooled result. The paired adaptation-benefit calculation uses each seed’s own no-adaptation and full-replay errors.
Table 13.
The target is a conditional mean, not a noisy test observation. A separate fidelity column compares with the fitted target’s exact replay rather than the true generating system. Both are retained in the raw event table.
Let L s , m L_{s,m} L s , m denote a seed’s NMSE and define the recovered adaptation benefit as
1 − L s , m − L s , r e p l a y L s , a n c h o r − L s , r e p l a y . (121) 1 - \frac{L_{s,m} - L_{s,\mathrm{replay}}}{L_{s,\mathrm{anchor}} - L_{s,\mathrm{replay}}}. \tag*{(121)} 1 − L s , anchor − L s , replay L s , m − L s , replay . (121)
Every denominator is positive in the recorded main setting. Degree three has mean recovered benefit 99.99045% and minimum 99.98134%. This is a paired descriptive result on five constructed systems, not a confidence bound for a population of deployed models. The analytic history bank also attains essentially the full replay benefit.
Equal-memory multivariate interpolation
The nonconstant monomials of total degree at most three in two variables give N 2 , 3 = 9 N_{2,3} = 9 N 2 , 3 = 9 . Form candidate points ( i / 8 , j / 8 ) (i/8, j/8) ( i /8 , j /8 ) for integers i , j ∈ [ − 8 , 8 ] i, j \in[-8, 8] i , j ∈ [ − 8 , 8 ] satisfying 0 < ∣ i ∣ + ∣ j ∣ ≤ 8 0 < |i| + |j| \le8 0 < ∣ i ∣ + ∣ j ∣ ≤ 8 . Evaluate the nine nonconstant monomials at those points and use column-pivoted QR on the transposed evaluation matrix to select nine nodes. The anchor value at the origin is already known. The selected square evaluation matrix V V V has condition number 26.68 in the recorded construction. The selection uses neither target coordinates nor test inputs.
At each selected node ξ \xi ξ , maintain the full projected reduced response
r t ( ξ ) = ( T + ∑ i ξ i G i ) r t − 1 ( ξ ) + ∑ i ξ i D i h t − 1 0 . (122) r_t(\xi)=\left(T+\sum_i \xi_iG_i\right)r_{t-1}(\xi)+\sum_i \xi_iD_i h^0_{t-1}. \tag*{(122)} r t ( ξ ) = ( T + i ∑ ξ i G i ) r t − 1 ( ξ ) + i ∑ ξ i D i h t − 1 0 . (122)
For a query, let m ( a ) m(a) m ( a ) be its vector of nine monomials and return h t 0 + U m ( a ) ⊤ V − 1 r t h_t^0+Um(a)^\top V^{-1}r_t h t 0 + U m ( a ) ⊤ V − 1 r t . The bank uses the same projected dynamics and possible leakage as coefficient memory, not a larger full-state filter bank. It retains the full 128-state anchor and nine eight-state responses. Its 1600-byte auxiliary budget matches degree-three coefficient memory exactly. The original separable language bank can use scalar filters without this projection.
A strong history-basis baseline and its analytic decoder
A model-independent history basis is a direct competitor in the scalar-input task. Following the classical Laguerre-model construction [20 ] , we use a discrete Laguerre bank with pole α = ρ \alpha=\rho α = ρ and width M = 200 M=200 M = 200 , fixed before the stream. Define formal delay-domain basis functions
ϕ j ( z ) = 1 − α 2 1 − α z ( z − α 1 − α z ) j , j ≥ 0. (123) \phi_j(z)=\frac{\sqrt{1-\alpha^2}}{1-\alpha z}\left(\frac{z-\alpha}{1-\alpha z}\right)^j,\qquad j\ge0. \tag*{(123)} ϕ j ( z ) = 1 − α z 1 − α 2 ( 1 − α z z − α ) j , j ≥ 0. (123)
They can be maintained without the past inputs. With all coordinates initially zero,
y t , 0 = α y t − 1 , 0 + 1 − α 2 x t , (124) y_{t,0}=\alpha y_{t-1,0}+\sqrt{1-\alpha^2}x_t, \tag*{(124)} y t , 0 = α y t − 1 , 0 + 1 − α 2 x t , (124)
y t , j = α y t − 1 , j + y t − 1 , j − 1 − α y t , j − 1 , j ≥ 1. (125) y_{t,j}=\alpha y_{t-1,j}+y_{t-1,j-1}-\alpha y_{t,j-1},\qquad j\ge1. \tag*{(125)} y t , j = α y t − 1 , j + y t − 1 , j − 1 − α y t , j − 1 , j ≥ 1. (125)
The second equation uses the newly computed preceding coordinate. A shared-history baseline of this form follows the general approach of retaining a basis of the past rather than a basis of one model’s hidden representation [8 ] . It is a discrete Laguerre construction, not an implementation claim about the particular HiPPO-LegS algorithm.
The fitted variant learns a state decoder from independent calibration histories. A stronger variant needs no historical calibration data. For a new stable target matrix A A A and known input vector b b b , its analytic state-decoder columns are
l 0 = 1 − α 2 ( I − α A ) − 1 b , l j = ( I − α A ) − 1 ( A − α I ) l j − 1 . (126) l_0=\sqrt{1-\alpha^2}(I-\alpha A)^{-1}b,\qquad l_j=(I-\alpha A)^{-1}(A-\alpha I)l_{j-1}. \tag*{(126)} l 0 = 1 − α 2 ( I − α A ) − 1 b , l j = ( I − α A ) − 1 ( A − α I ) l j − 1 . (126)
At a query, return h ^ t = ∑ j = 0 M − 1 l j y t , j \hat{h}_t=\sum_{j=0}^{M-1}l_jy_{t,j} h ^ t = ∑ j = 0 M − 1 l j y t , j . Only the decoder depends on the new model. The history coordinates were already maintained before it was known. Between queries, the active state follows the exact target recurrence, as for all migration methods.
The matrix L M = [ l 0 , … , l M − 1 ] L_M=[l_0,\ldots,l_{M-1}] L M = [ l 0 , … , l M − 1 ] reconstructs the full state. It is constructed from A A A , b b b , and α \alpha α , without the readout C C C , prediction labels, or an output-fitting objective. Changing C C C therefore requires no decoder refit, although it can change the width needed for accurate outputs. The construction is readout-agnostic, not replacement-model-independent.
To justify the decoder, write ϕ j ( z ) = ∑ s ≥ 0 ϕ j [ s ] z s \phi_j(z)=\sum_{s\ge0}\phi_j[s]z^s ϕ j ( z ) = ∑ s ≥ 0 ϕ j [ s ] z s . For ∣ α ∣ < 1 |\alpha|<1 ∣ α ∣ < 1 , these impulse responses form an orthonormal basis of one-sided square-summable sequences. On the unit circle the ratio ( z − α ) / ( 1 − α z ) (z-\alpha)/(1-\alpha z) ( z − α ) / ( 1 − α z ) is all-pass, and its change of angular variable has Jacobian ( 1 − α 2 ) / ∣ 1 − α z ∣ 2 (1-\alpha^2)/|1-\alpha z|^2 ( 1 − α 2 ) /∣1 − α z ∣ 2 . Substitution in the Fourier inner product gives orthogonality and unit norm. Completeness follows by the corresponding invertible weighted composition operator applied to the ordinary nonnegative Fourier basis. The target impulse response is A s b A^s b A s b , so its projection coefficient is
∑ s ≥ 0 ϕ j [ s ] A s b = ϕ j ( A ) b = l j . (127) \sum_{s\ge0}\phi_j[s]A^s b=\phi_j(A)b=l_j. \tag*{(127)} s ≥ 0 ∑ ϕ j [ s ] A s b = ϕ j ( A ) b = l j . (127)
The matrix functions are rational functions of the same A A A and therefore commute. This yields (126) . Stability ensures convergence. Truncating the basis approximates the target impulse response, not the parameter Taylor expansion.
The same construction gives a finite-tolerance causal certificate for the chosen bank. If W c = A W c A ⊤ + b b ⊤ W_c=AW_cA^\top+bb^\top W c = A W c A ⊤ + b b ⊤ and L M = [ l 0 , … , l M − 1 ] L_M=[l_0,\ldots,l_{M-1}] L M = [ l 0 , … , l M − 1 ] , then the residual impulse-response Gram matrix in exact arithmetic is E M = W c − L M L M ⊤ ⪰ 0 E_M=W_c-L_ML_M^\top\succeq0 E M = W c − L M L M ⊤ ⪰ 0 . For a history of Euclidean input norm at most one, the worst state error of this bank is λ max ( E M ) \sqrt{\lambda_{\max}(E_M)} λ m a x ( E M ) . For root-sum-squared future output error with fixed target A , C A,C A , C , replace this by λ max ( E M 1 / 2 W o E M 1 / 2 ) \sqrt{\lambda_{\max}(E_M^{1/2}W_oE_M^{1/2})} λ m a x ( E M 1/2 W o E M 1/2 ) , where W o = A ⊤ W o A + C ⊤ C W_o=A^\top W_oA+C^\top C W o = A ⊤ W o A + C ⊤ C . The claim follows by taking the operator norm of the omitted-history map, followed by the observability operator. At a fixed finite prefix it remains an upper bound. This is a certificate for a chosen scalar-input basis and target, not a minimum over all causal encoders or a uniform guarantee over an unbounded input distribution. Its Gramian computation and numerical validation are not included in the timing comparison, and no machine-level certificate is claimed for the nearly zero residuals of the main task.
The 200 retained float64 history coordinates use 1600 bytes, equal to the generic leaky degree-three memory. Their analytic decoder is shared model data, not per-stream memory. Its storage exceeds that of the polynomial query coefficients, and constructing it incurs a separate cost. Both are reported below. The basis’s success is consistent with the shared-moment calculation in Appendix § 21 .§ 2 . A dense coordinate rotation and noncommuting perturbations do not eliminate historical redundancy in the underlying scalar-input slow-block model.
Contraction, frequency, and leakage regimes
We fit the full pipeline for each of five seeds at ρ = 0.97 , 0.995 , 0.999 \rho=0.97,0.995,0.999 ρ = 0.97 , 0.995 , 0.999 with period 512. The same seed conventions and true update coordinates are used, while update norms scale with the stability margin. This changes contraction and history length, not just the absolute size of a fixed transition perturbation. At ρ = 0.995 \rho=0.995 ρ = 0.995 we additionally use periods 128 and 1024. Reusing the period-512 run gives 25 distinct fitted/evaluated configurations, without counting that run twice. Periods 128,512,1024 give 31,7,3 migration events per stream. There are 33,000 event-method measurements in the complete coupled study.
The tables below give the full variability across fitted models. Figure 7 and Figure 8 show the corresponding means.
Figure 7. Contraction-design sweep. Means across five fitted-model seeds. The update scale and relative learned leakage also vary with the nominal contraction target, so this is not an isolated causal effect of contraction.
Figure 8. Replacement-frequency sweep. The periods change historical exposure and the mixture of target domains as well as the number of queries. All methods use the same schedule within a setting.
The original noncommuting geometry study varies leakage independently of the other model parameters and sweeps six query radii. It therefore complements rather than duplicates the learned study. Neither study supports extrapolating a monotone error trend to arbitrary neural models. In particular, different frequencies expose different histories and different mixtures of the four replacement domains.
Certificate tightness and its use as a decision rule
At horizon 4096 and query radius 0.4, Table 16 reports the ratio of the radius-0.8 certificate to the actual error, pooled over the finite set of five seeds and twelve streams at each setting. These describe conservatism, not probabilistic coverage. The certificate is uniform over a larger radius than the evaluated query, so it need not be close to one.
Table 16. Conservatism of the online certificate at prefix 4096 and query radius 0.4. Degree and nominal leakage identify the setting. The certificate was maintained for the full radius 0.8.
Table 14. Contraction sweep. Mean ± \pm ± sample SD across five fitted models per setting. The relative leakage of the learned subspace also changes with the stability margin, as shown in Table 11 .
Measured structure of the fitted coupled families. The last column is the mixed coefficient norm at the end of the final held-out stream. Matrices are time invariant, so their leakage norm is the same on every stream. Nonzero mixed coefficients and noncommutators do not by themselves establish independent historical information.
Table 11. Measured structure of the fitted coupled families. The last column is the mixed coefficient norm at the end of the final held-out stream. Matrices are time invariant, so their leakage norm is the same on every stream. Nonzero mixed coefficients and noncommutators do not by themselves establish independent historical information.
Table 15. Update-frequency sweep at ρ = 0.995 \rho= 0.995 ρ = 0.995 . Windows contain 32 predictions after each replacement. Query schedules are matched across methods within a setting.
To assess whether the certificate can select a degree, each run maintains certificates for degrees one through four before querying. At prefix 4096, choose the smallest already-stored degree whose radius-uniform certificate is below the desired absolute state tolerance. If none qualifies, record failure to certify rather than using unavailable replay or claiming a successful migration. The three tolerances are 0.001, 0.01, and 0.1.
At zero leakage, every run certifies tolerance 0.01 with degree two. At nominal leakage 0.003, only 55% can certify that tolerance with the available array. At leakage 0.01 or 0.03, none can. The failures show where the exact-arithmetic sufficient condition is too conservative to certify the requested tolerance. The ordinary floating-point experiments check inequalities with tolerance 10 − 10 10^{-10} 1 0 − 10 .
Output-oriented retention diagnostic
The common-output quotient experiment uses five seeds, d = 12 d = 12 d = 12 , an eight-dimensional update image, two noncommuting directions, and a four-dimensional linear readout. Four source coordinates are visible, only two of which lie in the update image. Six further update-image coordinates are invisible and are dynamically separated from the visible block. A Gaussian orthogonal rotation makes the reported matrices dense. Iterated row-space closure recovers a four-dimensional quotient with r o u t = 2 r_{\mathrm{out}} = 2 r out = 2 .
For degrees one through three, four independent input streams of length 1024 compare full and quotient coefficient propagation under the same query ( 0.35 , − 0.4 ) (0.35, -0.4) ( 0.35 , − 0.4 ) . The reduced version retains one quarter as many coefficient scalars. Its outputs agree with the full coefficient method within 5.66 × 10 − 15 5.66 \times10^{-15} 5.66 × 1 0 − 15 in the recorded runs, although lifting the quotient state by setting invisible coordinates to zero does not reconstruct the full hidden state. This numerical agreement illustrates the distinction between preserving outputs and preserving every coordinate under the common-invariance assumptions. It is not a compression result for nonlinear readouts.
Complete serving, setup, and shared-memory costs
The coupled serving benchmark uses seed zero at ρ = 0.995 \rho= 0.995 ρ = 0.995 , a single 4096-input stream, and seven replacements. Every timed call includes all dense active-state transitions, all required anchor and auxiliary updates, migration work, initialization, and a scalar readout at each input. All methods receive the same float32 inputs and perform state arithmetic in float64. Each kernel is compiled and warmed before 21 trials, whose method order is randomly permuted. BLAS and OpenMP are restricted to one thread. The benchmark records timings, checksums, and the machine/software metadata. The benchmark times the serving computation, excluding the evaluation routine’s stored oracle trajectories.
Table 19 measures model-side construction separately on the same fitted model, without amortizing it over an unspecified number of client streams. The analytic construction is summed over all four target decoders. The fitted history and stitching entries include processing the independent calibration history, generating exact calibration target states, fitting, and selecting the ridge penalty. The model-reduction entries are once per source family. Because the coefficient setup uses the same generic node-preparation routine as interpolation in the reference code, its reported time conservatively includes that routine rather than subtracting a hypothetical optimized cost. All setup measurements use 21 repetitions and exclude compilation and file I/O. The original base and domain model fitting is common to migration strategies and is recorded separately in each setting’s metadata.
Table 19. Shared setup measured independently of per-stream serving. These costs must be added for a single fresh deployment or amortized over an explicitly stated number of streams.
Adding the median analytic setup for all four targets to its median serving time gives 13.61 ms, versus 25.24 ms for the coefficient setup plus degree-three serving on this instance. This sum of separately measured medians is a descriptive cost decomposition, not a new jointly measured latency quantile or a general hardware performance claim. Linear stitching serves quickly once fitted but has a larger measured calibration cost.
Table 17. Certificate-based degree selection among degrees already maintained before the query. Each fraction is over 60 runs. A missing mean degree means no run was certified. The memory and historical maintenance cost remain those of degree four.
Shared model data are separate from stream-specific retained history. Table 20 counts actual saved float64 arrays. A single analytic decoder contains 128 × 200 128 \times200 128 × 200 coefficients, or 204,800 bytes, and four cached decoders use 819,200 bytes. A deployment can construct and cache fewer model decoders at a time, trading storage against recomputation. The raw calibration history and temporary state-pair arrays can be discarded after fitting. Their work is included in setup, but they are not claimed to be part of a 1600-byte streaming summary.
Table 20. Shared auxiliary model storage for the coupled benchmark. These are not per-stream history bytes. Base and target weights are common model data and are excluded from every method’s auxiliary column.
Table 18. Complete coupled serving times. Median and interquartile range over 21 repetitions on one fitted model and one stream. These are timing repetitions, not independent model seeds.
The main per-stream counts exclude a common 128-coordinate active state, or 1024 bytes. Learned leakage prevents exact sharing of its complement with the anchor, so the coefficient and projected-node methods retain a full extra 128-state anchor. Degrees one, two, and three then require 1152,1344,1600 extra bytes. The generic cubic bank and both width-200 history banks also require 1600. A checkpoint stores a full 1024-byte state plus 4 L 4L 4 L raw input bytes. Full replay stores 4 t 4t 4 t raw bytes at prefix t t t . Stitching has no additional history state but uses the shared map data. Logical counts exclude temporary work arrays and are not measurements of peak Python-process memory.
Amortization over streams and queries. For method j j j , let S j S_j S j be shared setup work, H j H_j H j shared auxiliary bytes, and c j c_j c j , q j q_j q j , m j m_j m j per-input work, per-query work, and per-stream auxiliary bytes. For N N N streams with T T T inputs and Q Q Q queries each, the corresponding totals are
W j = S j + N ( T c j + Q q j ) , M j = H j + N m j . W_j = S_j + N(Tc_j + Qq_j), \qquad M_j = H_j + Nm_j. W j = S j + N ( T c j + Q q j ) , M j = H j + N m j .
In the work expression, N N N counts processed streams sharing the prepared targets. In the memory expression it counts histories retained simultaneously. All methods must first meet the same accuracy requirement. When replay cost grows with prefix length, Q q j Qq_j Q q j denotes the sum of its query-specific costs. Each newly encountered target also adds its decoder-construction or calibration cost to S j S_j S j . These formulas separate query frequency from the per-input maintenance charged even when no replacement occurs.
For the measured four-target, 4096-input, seven-query schedule, the retained medians give the following estimates, in milliseconds:
W b a n k = 4.833 + 8.776 N , W r e p l a y = 30.495 N , \mathcal{W}_{\mathrm{bank}} = 4.833 + 8.776N,\quad\mathcal{W}_{\mathrm{replay}} = 30.495N, W bank = 4.833 + 8.776 N , W replay = 30.495 N ,
W c h e c k p o i n t = 9.777 N , W s t i t c h = 136.456 + 7.065 N . \mathcal{W}_{\mathrm{checkpoint}} = 9.777N,\quad\mathcal{W}_{\mathrm{stitch}} = 136.456 + 7.065N. W checkpoint = 9.777 N , W stitch = 136.456 + 7.065 N .
The bank is cheaper than replay from one stream, cheaper than checkpoint-256 from five streams, and cheaper than stitching through 76 streams. Beyond that, stitching amortizes its calibration cost. These estimates add separately measured medians, rather than reporting medians of jointly measured deployment times. Concurrent execution was not measured. Checkpoint replay and stitching have larger errors in this experiment, so their lower cost is useful only when those errors are acceptable.
At prefix 4096, the corresponding auxiliary-memory counts are 819200 + 1600 N 819200 + 1600N 819200 + 1600 N bytes for the bank, 16384 N 16384N 16384 N for full replay, and 2048 N 2048N 2048 N for checkpoint-256. The bank becomes smaller at 56 and 1829 simultaneously retained streams, respectively. Stitching has no additional stream history but retains the shared maps in Table 20 . These comparisons use the stated cached target set and exclude common model weights, common active states, and temporary construction arrays.
Conditioning of small universal realizations
The direction-independent filter is deliberately tested only at small dimensions. For a two-state example with source eigenvalues 0.2, 0.55, scalar input vector ( 1 , 0.7 ) T (1, 0.7)^\mathsf{T} ( 1 , 0.7 ) T , and degrees one through three, the required auxiliary counts are two, four, and six. The respective coordinate-transform condition numbers are approximately 15.6, 27.5, and 38.4. They do not justify a numerical-stability claim at the 128-state experimental dimension. The high-dimensional experiment instead uses the explicitly described projected coefficient and Laguerre realizations.
A sufficient precision condition for a chosen realization. Let z t = F z t − 1 + g x t z_t = Fz_{t-1} + gx_t z t = F z t − 1 + g x t , z 0 = 0 z_0 = 0 z 0 = 0 , be an exact retained-state realization, with degree-k k k query response h t ( k ) ( Δ ) = D Δ z t h_t^{(k)}(\Delta) = D_{\Delta}z_t h t ( k ) ( Δ ) = D Δ z t . For a computed trajectory initialized at zero, define its one-step defect against these exact matrices by
r t = z ^ t − F z ^ t − 1 − g x t . r_t = \widehat{z}_t - F\widehat{z}_{t-1} - gx_t. r t = z t − F z t − 1 − g x t .
Suppose ∥ r t ∥ ≤ η \lVert r_t\rVert\le\eta ∥ r t ∥ ≤ η and decoder evaluation contributes error at most η D \eta_D η D . Subtracting the recurrences and iterating gives
z ^ t − z t = ∑ s = 1 t F t − s r s , ∥ h ^ t − h t ( k ) ( Δ ) ∥ ≤ η ∑ j = 0 t − 1 ∥ D Δ F j ∥ + η D . \widehat{z}_t - z_t = \sum_{s=1}^{t} F^{t-s}r_s,\qquad\left\lVert\widehat{h}_t - h_t^{(k)}(\Delta)\right\rVert\le\eta\sum_{j=0}^{t-1}\left\lVert D_{\Delta}F^j\right\rVert+ \eta_D. z t − z t = s = 1 ∑ t F t − s r s , h t − h t ( k ) ( Δ ) ≤ η j = 0 ∑ t − 1 D Δ F j + η D .
If ∥ F j ∥ ≤ K F ρ F j \lVert F^{j}\rVert\le K_{F}\rho_{F}^{j} ∥ F j ∥ ≤ K F ρ F j for 0 ≤ ρ F < 1 0 \le\rho_{F} < 1 0 ≤ ρ F < 1 , this numerical contribution is at most η ∥ D Δ ∥ K F / ( 1 − ρ F ) + η D \eta\lVert D_{\Delta}\rVert K_{F}/(1-\rho_{F})+\eta_{D} η ∥ D Δ ∥ K F / ( 1 − ρ F ) + η D . Adding a valid truncation bound yields a sufficient absolute-error budget for that realization. Uniform accuracy over replacement queries requires uniform bounds on these quantities. A numerical projection onto an approximate reachable subspace adds its own representation error.
The defect allowance must include stored-coefficient rounding, arithmetic, and the input and state ranges. Stable eigenvalues alone do not make K F K_{F} K F or the decoder norm small. Non-normal transients and poorly scaled coordinates can make a minimum-dimensional realization less accurate than a larger one. Computing the reachable basis also requires numerical separation of its retained subspace. The condition above explains which quantities must be controlled, but it does not estimate them for a large universal realization or provide validated floating-point certification. When their amplification is excessive, a larger scaled coefficient or orthogonal-history representation may be preferable.
Relationship to earlier methods and the scope of the contribution
Widths and conditional information. DeVore et al. [4 ] develop continuous nonlinear widths, and Siegel [18 ] discusses sharp manifold-width lower bounds using topological arguments. The fixed-horizon result here uses the same kind of singular-value upper bound and antipodal lower bound, conditioned on the old state. It does not introduce Borsuk–Ulam, a new general nonlinear width, or a lower bound for discontinuous real encodings. The finite-bit covering argument likewise uses an elementary ellipsoid-volume bound. The migration-specific distinction is between information already retained, information a future replacement can expose, and whether its attaining summary can be updated causally.
Realization and parametric model reduction. The coefficient lift, its reachable subspace, invariant projection, and common-denominator construction use classical linear-system tools. Projection-based parametric reduction and offline/online error estimation are established subjects [2 , 10 ] . The causal minimum follows by combining reachability with the requirement that histories sharing a summary agree in every coefficient demanded by local order. The sharp scalar-input comparison shows that an unknown arbitrary matrix direction and one suitably chosen rank-one direction have the same worst-case local information cost. The contribution lies in these information statements, not in sensitivity propagation or residual estimation.
Recurrent differentiation and restricted adaptation. RTRL, UORO, and more recent structured recurrent learning maintain or approximate derivatives for online learning [21 , 19 , 5 ] . The present query asks how an unavailable historical prefix would be represented after a subsequently selected parameter replacement. First-order sensitivities can answer it approximately, but they are one retained representation rather than a universal minimum. Parameter-efficient adaptation motivates small families [11 ] , yet a freely trained low-rank factorization need not preserve a fixed image or a small number of scalar coordinates. A fixed image and a small scalar-coordinate family are additional assumptions, not consequences of the term “adapter.”
Stitching and history bases. Stitching exploits compatibility of existing representations, with its behavior depending on training objectives, feature layers, and calibration [1 , 14 ] . It is not ruled out by worst-case collisions when real data occupy a restricted set of histories. Both linear and nonlinear versions are therefore tested. HiPPO provides a principled family of causal history-basis representations [8 ] . The Laguerre baseline retains model-independent history coordinates and illustrates how a compact shared history basis can replace parameter-specific coefficients. It is not claimed as a new family of orthogonal filters.
Context composition and state compression. PICASO composes context states within state-space dynamics [13 ] , whereas CompreSSM uses control-theoretic state reduction during training [3 ] . The target here is a counterfactual state under changed parameters, and the memory must be chosen before those parameters are revealed. A smaller state under one model need not contain the distinctions exposed by another. Structured recurrent models and selective SSMs provide architectural motivation [9 , 17 , 7 ] , not automatic instances of the theorem. In particular, parameter-dependent input gates or upstream activations can invalidate the assumption that injections and base matrices are shared across replacements.
History width, structural variation, and numerical realization
This section studies two distinct questions: the width needed by a prescribed history representation at a fixed tolerance, and a small numerical realization of the causal information minimum. A finite-width Laguerre bank is an approximate representation with an independently chosen pole. A reachable-space realization represents a specified truncated polynomial response without approximation in real arithmetic. Neither experiment characterizes the optimal finite-bit causal encoder over all representations.
Information timing and evaluation objects
Table 21 states when information becomes available. “Calibration” refers to independent transition observations, not to recovery of the evaluation stream. A decoder may be constructed after a target is fitted, but it may not use the past evaluation inputs unless those inputs were explicitly retained and charged as replay memory. Independent target fits are computed before evaluating the held-out
Table 21. Information available at each stage of the coupled identification experiment.
streams. Neither family selection nor the retained-memory recurrence receives those fitted target coordinates. Moving the target fits after historical maintenance would therefore leave the summaries unchanged. The retrospective reference at each query runs the fitted target over the entire input prefix. The true conditional-mean target instead runs the generating target model over that prefix. They coincide only to the extent that system identification is accurate. Comparing against these two targets separates migration fidelity from task error.
Prespecified bank-width comparison
The repeated-pole arm uses the five frozen fitted systems from the main ρ = 0.995 \rho= 0.995 ρ = 0.995 , period-512 experiment. All fitted systems and seeds are retained without refitting or selection. The input seed for fitted seed s s s and stream j j j is 37001 + 100 s + j 37001 + 100s + j 37001 + 100 s + j , as in the original experiment. Each of eight streams contains 4096 independent standard Gaussian inputs stored in float32. State calculations and retained bank coordinates use float64. Queries follow 512, 1024, … \ldots … , 3584 inputs, and cycle through the four fitted target domains. The evaluation window includes the query output and 31 subsequent outputs.
Before running this width comparison, the configuration fixes
M = { 1 , 2 , 4 , 8 , 16 , 32 , 64 , 128 , 200 , 400 , 800 } , E = { 10 − 2 , 10 − 3 , 10 − 4 } . \mathcal{M} = \{1, 2, 4, 8, 16, 32, 64, 128, 200, 400, 800\}, \qquad\mathcal{E} = \{10^{-2}, 10^{-3}, 10^{-4}\}. M = { 1 , 2 , 4 , 8 , 16 , 32 , 64 , 128 , 200 , 400 , 800 } , E = { 1 0 − 2 , 1 0 − 3 , 1 0 − 4 } .
Every bank uses pole α = 0.995 \alpha= 0.995 α = 0.995 , taken from the anchor design, without tuning to a replacement, a test history, or a target tolerance. Width is an experimental resource variable. The smallest successful tested width describes an empirical frontier. It is not a deployed selection rule with a population guarantee.
The discrete Laguerre transfer functions are
ϕ m ( z ) = 1 − α 2 1 − α z ( z − α 1 − α z ) m , m = 0 , 1 , … (128) \phi_m(z) = \frac{\sqrt{1-\alpha^2}}{1-\alpha z}\left(\frac{z-\alpha}{1-\alpha z}\right)^m,\qquad m = 0,1,\ldots \tag*{(128)} ϕ m ( z ) = 1 − α z 1 − α 2 ( 1 − α z z − α ) m , m = 0 , 1 , … (128)
This is a classical system-identification basis [20 ] . The causal recursion and analytic decoder are given in Appendix § 24 .§ 3 . All bank states start at zero. For each target matrix A A A , decoder columns satisfy
ℓ 0 = 1 − α 2 ( I − α A ) − 1 b , ℓ m = ( I − α A ) − 1 ( A − α I ) ℓ m − 1 . (129) \ell_0 = \sqrt{1-\alpha^2}(I-\alpha A)^{-1}b,\qquad\ell_m = (I-\alpha A)^{-1}(A-\alpha I)\ell_{m-1}. \tag*{(129)} ℓ 0 = 1 − α 2 ( I − α A ) − 1 b , ℓ m = ( I − α A ) − 1 ( A − α I ) ℓ m − 1 . (129)
A width-M M M reconstruction is h ^ t ( M ) = ∑ m = 0 M − 1 ℓ m s t , m \hat{h}_t^{(M)} = \sum_{m=0}^{M-1}\ell_m s_{t,m} h ^ t ( M ) = ∑ m = 0 M − 1 ℓ m s t , m , where s t , m s_{t,m} s t , m is the bank’s response to the evaluation prefix. The decoder uses only the target model and the prescribed basis. It is not fitted on evaluation histories. A width-800 bank is maintained once for efficient evaluation, and each smaller width uses its prefix. The first M M M coordinates have a closed causal recursion that never uses later coordinates, so this evaluation gives the same states as an independently maintained width-M M M bank. The width-M M M memory count excludes the other coordinates in the 800-coordinate evaluation array.
Three error measures. For one fitted-model seed, sum over its 56 events, denoted by e e e . Let h e h_e h e be the fitted target’s exact replay state at the event, and h ^ e \hat{h}_e h ^ e a migrated state. Let y e , s ∗ y_{e,s}^{*} y e , s ∗ , y e , s r e f y_{e,s}^{\mathrm{ref}} y e , s ref , and y ^ e , s \hat{y}_{e,s} y ^ e , s be the generating conditional mean, fitted-target replay output, and evaluated method’s output at offset s ∈ { 0 , … , 31 } s \in\{0,\ldots,31\} s ∈ { 0 , … , 31 } . We report
Table 22. Complete width comparison. Columns show maximum seed-level NRMSE over five independently fitted systems. “Rep.” denotes repeated slow poles. Byte counts are additional per-stream bank storage.
NMSE t a s k = ∑ e , s ( y ^ e , s − y e , s ∗ ) 2 ∑ e , s ( y e , s ∗ ) 2 , (130) \operatorname{NMSE}_{\mathrm{task}} = \frac{\sum_{e,s}(\hat{y}_{e,s}-y^{*}_{e,s})^2}{\sum_{e,s}(y^{*}_{e,s})^2}, \tag*{(130)} NMSE task = ∑ e , s ( y e , s ∗ ) 2 ∑ e , s ( y ^ e , s − y e , s ∗ ) 2 , (130)
NRMSE o u t = [ ∑ e , s ( y ^ e , s − y e , s r e f ) 2 ∑ e , s ( y e , s r e f ) 2 ] 1 / 2 , (131) \operatorname{NRMSE}_{\mathrm{out}} = \left[\frac{\sum_{e,s}(\hat{y}_{e,s}-y^{\mathrm{ref}}_{e,s})^2}{\sum_{e,s}(y^{\mathrm{ref}}_{e,s})^2}\right]^{1/2}, \tag*{(131)} NRMSE out = [ ∑ e , s ( y e , s ref ) 2 ∑ e , s ( y ^ e , s − y e , s ref ) 2 ] 1/2 , (131)
NRMSE s t a t e = [ ∑ e ∥ h ^ e − h e ∥ 2 ∑ e ∥ h e ∥ 2 ] 1 / 2 . (132) \operatorname{NRMSE}_{\mathrm{state}} = \left[\frac{\sum_e \lVert\hat{h}_e-h_e\rVert^2}{\sum_e \lVert h_e\rVert^2}\right]^{1/2}. \tag*{(132)} NRMSE state = [ ∑ e ∥ h e ∥ 2 ∑ e ∥ h ^ e − h e ∥ 2 ] 1/2 . (132)
Thus state fidelity is measured at migration and output fidelity over a subsequent common-input window. The two normalized quantities use different denominators and measure different losses. All task NMSE means and sample SDs use five seed-level ratios. Tolerance tables use the maximum of the five seed-level NRMSEs . They do not bound the worst event, an unseen seed, an arbitrary input sequence, or an infinite future horizon. The nonzero readout in every tested stream gives positive denominators.
Memory accounting. A history bank needs 8 M 8M 8 M per-stream bytes in float64, beyond the common active 128-state model. It needs no retained anchor. Its four target-specific 128 × M 128 \times M 128 × M decoder matrices require 4 ⋅ 128 ⋅ M ⋅ 8 4 \cdot128 \cdot M \cdot8 4 ⋅ 128 ⋅ M ⋅ 8 shared bytes. At M = 8 , 64 , 200 M=8, 64, 200 M = 8 , 64 , 200 , these are 32,768, 262,144, and 819,200 shared bytes, respectively. Direct decoder multiplication costs O ( d M ) O(dM) O ( d M ) per query, bank maintenance costs O ( M ) O(M) O ( M ) per input, and analytic decoder construction is shared work after the target is known. The degree-three coefficient and multivariate interpolation methods retain 200 scalars, including their anchor, or 1600 extra bytes. Full replay retains the input prefix. The width sweep reports accuracy and logical storage, not newly measured timings or a claim that model-dependent decoder setup is free.
Eight coordinates have maximum seed output NRMSE 3.07 × 10 − 7 3.07 \times10^{-7} 3.07 × 1 0 − 7 in the repeated-pole arm, while their mean full-state NRMSE is 10.7%. Their task NMSE is 2.50 × 10 − 9 2.50 \times10^{-9} 2.50 × 1 0 − 9 , approximately the identification error achieved by replay. The 200-coordinate budget therefore exceeds the requirement of this tested prediction criterion. It is not a claim that eight is the optimal continuous dimension or that the retained history preserves every state coordinate. At output tolerance 10 − 2 10^{-2} 1 0 − 2 , the smallest successful tested width is four; at 10 − 3 10^{-3} 1 0 − 3 and 10 − 4 10^{-4} 1 0 − 4 it is eight. Table 4 contrasts those counts with full-state tolerances.
Figure 9. Full-state fidelity for the same bank widths and queries as Figure 1 . The common readout can be accurate while many state coordinates remain inaccurate. These results concern a finite empirical design; the common-invariant quotient theorem is a separate exact statement.
Figure 1. Maximum seed-level output NRMSE across the five fitted systems at each prescribed bank width. Inputs, replacement schedules, and calibration seeds are paired between structural variants. Each ordinate is the maximum of five pooled within-seed window errors, not a worst-query or population bound.
A paired split-pole structural variant
The baseline generator uses a repeated slow pole 0.995 on an eight-dimensional block. With scalar input, many update coefficients then share the same weighted historical moments. To weaken that property while keeping a useful adaptation problem, replace only the true slow diagonal by
λ j = 0.995 − 0.04 j 7 , j = 0 , … , 7. (133) \lambda_j = 0.995 - 0.04\frac{j}{7}, \qquad j = 0,\ldots,7. \tag*{(133)} λ j = 0.995 − 0.04 7 j , j = 0 , … , 7. (133)
The 120 fast poles, Gaussian rotation, injection and readout construction, update-direction scale, calibration observation noise, number of calibration observations, and independent seed rules are unchanged. We fit all five source models, pilot families, and target domains anew. No method receives the true split-pole invariant basis. The same fixed singular-value energy rule chooses the fitted image and update directions. Each fitted model and its metadata are retained.
The bank pole remains 0.995 in this arm. Its relative mismatch to some of the true poles therefore increases. Update perturbations are still confined by construction to the eight-dimensional slow block, and calibration still exposes the full state. The variant tests sensitivity to a repeated-pole assumption, not removal of all favorable structure. Different pole locations also change trajectory energy and the importance of old inputs. Consequently a between-arm error ratio is not an isolated causal estimate of “number of distinct poles” in an otherwise invariant prediction distribution.
Useful adaptation survives. Mean task NMSE is 1.40 × 10 − 2 1.40 \times10^{-2} 1.40 × 1 0 − 2 without adaptation and 2.90 × 10 − 2 2.90 \times10^{-2} 2.90 × 1 0 − 2 with stale reuse, versus 3.08 × 10 − 10 3.08 \times10^{-10} 3.08 × 1 0 − 10 for restricted replay and 2.66 × 10 − 10 2.66 \times10^{-10} 2.66 × 1 0 − 10 for unrestricted replay.
Prediction and fidelity in the repeated-pole arm. Task NMSE is mean ± \pm ± sample SD across five fitted seeds. Output and state NRMSE are maxima of the five seed-level ratios. Baselines are evaluated on the identical histories and target schedule.
Table 23.
Degree-three coefficients attain 3.54 × 10 − 9 3.54 \times10^{-9} 3.54 × 1 0 − 9 . The analytic bank’s eight-coordinate maximum seed output NRMSE is 6.56 × 10 − 2 6.56 \times10^{-2} 6.56 × 1 0 − 2 , and 64 coordinates reduce it to 1.86 × 10 − 7 1.86 \times10^{-7} 1.86 × 1 0 − 7 . Among the prescribed widths, 16, 32, and 64 meet output tolerances 10 − 2 10^{-2} 1 0 − 2 , 10 − 3 10^{-3} 1 0 − 3 , 10 − 4 10^{-4} 1 0 − 4 in every seed. The split-pole setting weakens the small history bank without establishing a general advantage for another representation.
The study contains two structural arms, five fitted seeds per arm, eight streams, seven events per stream, and 17 methods per event. Its 9,520 event-method records retain both squared-error numerators and target-energy denominators. The records include every stream and width, including those that fail the tolerances. The repeated-pole arm holds the fitted models fixed, while the split-pole arm fits a separate model family for each seed.
Stitching and first-order memory. A follow-up holds the five fitted split-pole systems fixed and evaluates these two methods on the same eight streams and seven queries. Stitching uses the original independent calibration inputs: 8192 Gaussian values with seed 26001 + s s s for fitted seed s s s , retaining every fourth state after the first 1024 inputs. The first 75% of retained rows fit the map, and the remaining 25% select the ridge penalty from 10 − 8 10^{-8} 1 0 − 8 , 10 − 6 10^{-6} 1 0 − 6 , 10 − 4 10^{-4} 1 0 − 4 , 10 − 2 10^{-2} 1 0 − 2 . Model-pair maps cover the anchor-to-first-target transition and successive target transitions, including the return from target four to target one. Only these maps are fitted. The recurrent models, held-out histories, and 32-position evaluation windows are unchanged. Repeating the comparison in the repeated-pole arm gives 1120 event-method records across both arms.
Table 24 reports the split-pole results. Stitching has mean task NMSE 1.59 × 10 − 6 1.59 \times10^{-6} 1.59 × 1 0 − 6 , compared with 7.22 × 10 − 5 7.22 \times10^{-5} 7.22 × 1 0 − 5 for first-order memory. Its maximum seed output NRMSE is 2.04 × 10 − 3 2.04 \times10^{-3} 2.04 × 1 0 − 3 , so it fails the 10 − 3 10^{-3} 1 0 − 3 criterion across all five seeds. First-order memory truncates the response about the anchor. Stitching instead fits the finite replacement from full calibration states and can exploit correlations among histories in that distribution. The local-order lower bound does not rank those two approximations on this distribution.
The corresponding split-pole arm. A method can preserve output fidelity more accurately than full-state fidelity. The unrestricted replay row evaluates its own fitted transition, not a migration error under the restricted family. Stitching and first-order memory are evaluated on the frozen fitted systems as described below.
Table 24.
Comparison at the same tolerance. At maximum seed output NRMSE 10 − 3 10^{-3} 1 0 − 3 , degree three and the width-32 bank both qualify, with errors 1.19 × 10 − 4 1.19 \times10^{-4} 1.19 × 1 0 − 4 and 2.36 × 10 − 4 2.36 \times10^{-4} 2.36 × 1 0 − 4 . They use 1600 and 256 extra per-stream bytes. At 10 − 4 10^{-4} 1 0 − 4 , degree three fails, while width 64 qualifies with error 1.86 × 10 − 7 1.86 \times10^{-7} 1.86 × 1 0 − 7 and 512 bytes. Small task NMSE alone does not imply this output-fidelity requirement.
Including shared storage changes the 10 − 3 10^{-3} 1 0 − 3 comparison. Four 128 × 32 float64 decoders use 131072 bytes, whereas the coefficient reductions use 9728 bytes under the auxiliary-storage convention of Appendix § 24 .10. Thus the totals are 131072 + 256N N N and 9728 + 1600N N N bytes for N N N simultaneously retained streams. The bank becomes smaller at 91 streams. Its recurrence costs O ( M ) O(M) O ( M ) work per input and full-state decoding costs O ( d M ) O(dM) O ( d M ) per query. The coefficient method also maintains the anchor and reduced responses. Width-specific runtimes were not measured, so these counts do not determine a runtime crossover.
For full-state NRMSE 10 − 4 10^{-4} 1 0 − 4 instead, degree three qualifies at 5.04 × 10 − 5 5.04 \times10^{-5} 5.04 × 1 0 − 5 with 1600 extra bytes. The smallest successful tested bank has width 800 and uses 6400 bytes. This is a different reconstruction objective, not a predictive advantage or an optimum over all history bases.
An orthogonal realization of the causal minimum
For a tractable fixed-family example, use
A 0 = [ 1 / 5 0 0 2 / 5 ] , b = [ 1 2 ] , A 1 = 1 50 [ 1 2 3 4 ] , A 2 = 1 50 [ 3 − 1 2 1 ] . (134) A_0 = \begin{bmatrix} 1/5 & 0 \\ 0 & 2/5 \end{bmatrix}, \quad b = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad A_1 = \frac{1}{50}\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}, \quad A_2 = \frac{1}{50}\begin{bmatrix} 3 & -1 \\ 2 & 1 \end{bmatrix}. \tag*{(134)} A 0 = [ 1/5 0 0 2/5 ] , b = [ 1 2 ] , A 1 = 50 1 [ 1 3 2 4 ] , A 2 = 50 1 [ 3 2 − 1 1 ] . (134)
The two directions do not commute. Their raw degree-one, degree-two, and degree-three lifts have dimensions 6, 12, and 20. Rational arithmetic gives reachable dimensions 4, 6, and 8, respectively. Since the anchor is controllable, Theorem 1 gives exactly 2, 4, and 6 auxiliary coordinates. Replacing A 0 A_0 A 0 by ( 1 / 5 ) I 2 (1/5)I_2 ( 1/5 ) I 2 leaves anchor rank one and gives reachable dimensions 2, 3, and 4, with only 1, 2, and 3 auxiliary coordinates. Table 25 reports both cases. These are exact rank calculations on stated rational matrices, not floating-point rank estimates.
Table 25. Redundant lifts and an anchor-preserving numerical realization. “Rep.” replaces the distinct anchor poles by a repeated pole. Auxiliary dimensions are computed in rational arithmetic. NRMSE columns are maximum seed discrepancies from the full polynomial response in the distinct-pole case.
Here is an anchor-preserving realization for the distinct-pole case. Let W W W have orthonormal columns spanning K k K_k K k in the lifted coordinates, and put H = π 0 W H = \pi_0 W H = π 0 W . Then H H H has row rank d d d . Let N N N have orthonormal columns spanning ker H \ker H ker H , and define
J = [ H N T ] , q t = J W T j ~ t . (135) J = \begin{bmatrix} H \\ N^{\mathsf{T}} \end{bmatrix}, \qquad q_t = JW^{\mathsf{T}}\tilde{j}_t. \tag*{(135)} J = [ H N T ] , q t = J W T j ~ t . (135)
If J v = 0 Jv = 0 J v = 0 , then H v = 0 Hv = 0 H v = 0 and N T v = 0 N^{\mathsf{T}}v = 0 N T v = 0 . The first identity puts v v v in the column space of N N N , and the second then forces v = 0 v = 0 v = 0 . Thus J J J is invertible. The first d d d coordinates of q t q_t q t are exactly the anchor; the remaining dim K k − d \dim\mathcal{K}_k - d dim K k − d are auxiliary. In real arithmetic the causal dynamics are
q t = J W T T k W J − 1 q t − 1 + J W T B k x t , j t = W J − 1 q t . (136) q_t = JW^{\mathsf{T}}T_kWJ^{-1}q_{t-1} + JW^{\mathsf{T}}B_kx_t,\qquad j_t = WJ^{-1}q_t. \tag*{(136)} q t = J W T T k W J − 1 q t − 1 + J W T B k x t , j t = W J − 1 q t . (136)
Reachability ensures j t ∈ K k j_t \in\mathcal{K}_k j t ∈ K k , so W W T j t = j t WW^{\mathsf{T}}j_t = j_t W W T j t = j t ; invariance of that subspace proves the recurrence. This is the linear-complement construction in the causal theorem expressed in orthogonal coordinates. It is not a new general model-reduction algorithm.
The conditioning of this coordinate choice has an explicit form. With nonzero auxiliary dimension, J T J = H T H + N N T J^{\mathsf{T}}J = H^{\mathsf{T}}H + NN^{\mathsf{T}} J T J = H T H + N N T acts as the identity on ker H \ker H ker H and as H T H H^{\mathsf{T}}H H T H on its orthogonal complement. Its eigenvalues are therefore one on the auxiliary subspace and the squared nonzero singular values of H H H on the complement. Hence
κ 2 ( J ) = max { 1 , σ max ( H ) } min { 1 , σ min ( H ) } . \kappa_2(J) = \frac{\max\{1,\sigma_{\max}(H)\}}{\min\{1,\sigma_{\min}(H)\}}. κ 2 ( J ) = min { 1 , σ m i n ( H )} max { 1 , σ m a x ( H )} .
Here σ min ( H ) \sigma_{\min}(H) σ m i n ( H ) is the smallest of the d d d positive singular values. Full row rank does not prevent this value from being small. Appendix § 24 .11 gives the separate bound for accumulated step and decoder errors. Neither bound supplies a uniform conditioning guarantee for the direction-independent filter.
For numerical implementation, W W W is obtained by SVD of the finite Krylov matrix with relative singular-value threshold 10 − 12 10^{-12} 1 0 − 12 . Its selected dimension agrees with the rational calculation in every tested case. We report ∥ W T W − I ∥ 2 \lVert W^{\mathsf{T}}W - I\rVert_2 ∥ W T W − I ∥ 2 , the closure residual ∥ ( I − W W T ) T k W ∥ 2 \lVert(I - WW^{\mathsf{T}})T_kW\rVert_2 ∥( I − W W T ) T k W ∥ 2 , and cond 2 ( J ) \operatorname{cond}_2(J) cond 2 ( J ) rather than equating a chosen threshold with exact rank. The coordinate condition numbers are approximately 1.124, 1.152, and 1.166 for degrees one through three. Closure residuals range from 2.9 × 10 − 16 2.9 \times10^{-16} 2.9 × 1 0 − 16 to 2.5 × 10 − 13 2.5 \times10^{-13} 2.5 × 1 0 − 13 . Setup uses float64; the serving recurrence is evaluated separately in float32 and float64. The anchor block is updated directly under A 0 A_0 A 0 at each step, removing an unnecessary numerical discrepancy in the coordinates designated as side information.
Five Gaussian input sequences of length 4096 are generated with seeds 62001, …, 62005. At every 128th input, a two-coordinate query is drawn uniformly from [ − 0.5 , 0.5 ] 2 [-0.5, 0.5]^2 [ − 0.5 , 0.5 ] 2 using the same generator. The reduced state is decoded to its polynomial response and compared with an independently maintained full coefficient array. NRMSE pools squared response discrepancies and squared full-array response norms within a seed. This comparison excludes the common Taylor truncation error against the nonlinear-in-parameter target response. The maximum discrepancies across degrees and seeds are 2.49 × 10 − 16 2.49 \times10^{-16} 2.49 × 1 0 − 16 in float64 and 4.79 × 10 − 8 4.79 \times10^{-8} 4.79 × 1 0 − 8 in float32.
The computation realizes the local minimum with small numerical error in this fixed family. It does not prove that the same construction is well conditioned for large systems, near-defective dynamics, or arbitrary subsequently specified directions. The universal filter in Corollary 3 has a stronger direction-independent information contract and a different conditioning problem. The numerical check does not extend to that stronger contract.
Paired seed comparisons
Seed-level averages can conceal variation across fitted models. Figure 10 retains all five language-model seeds. Cubic interpolation has lower state error in three and higher error in two. Its lower mean does not imply lower error for each model. Figure 11 shows the same paired structure for the original coupled study. Seed 4 contributes 95.7% of the summed degree-three seed NMSE. The task’s adaptation benefit is consequently computed within each seed before summarization, as in the original analysis. These plots are descriptive paired observations, not simultaneous confidence bands or evidence from newly independent training samples.
Figure 11. Seed-paired task NMSE in the original coupled identification study. The analytic bank here has 200 coordinates. The individual points show the between-seed variation in coefficient and interpolation errors.
For the language comparison, let d s d_s d s be cubic-interpolation NRMSE minus degree-three NRMSE for fitted seed s s s . The five differences have mean − 2.19 × 10 − 4 -2.19 \times10^{-4} − 2.19 × 1 0 − 4 and sample SD 6.23 × 10 − 4 6.23 \times10^{-4} 6.23 × 1 0 − 4 . A descriptive paired Student-t t t interval, using four degrees of freedom, is [ − 9.92 × 10 − 4 , 5.54 × 10 − 4 ] [-9.92 \times10^{-4}, 5.54 \times10^{-4}] [ − 9.92 × 1 0 − 4 , 5.54 × 1 0 − 4 ] at 95%, with two-sided p = 0.475 p = 0.475 p = 0.475 . Enumerating all 2 5 2^5 2 5 sign patterns gives two-sided p = 0.75 p = 0.75 p = 0.75 under the paired-difference sign-symmetry assumption. Neither test establishes superiority. Migration events within a fitted seed are not independent trained models, and no multiplicity-adjusted claim is made.