betaFeedback

WorksComputer Science

A Log-Timescale Perturbation Bound for Selective State Space Models at Every Sequence Length

Abstract

Mamba models deployed on streaming data, such as limit order book updates, run far past the sequence lengths they were trained on. A guarantee on how much a small change of input can move the hidden state therefore has to hold at every length. The closest existing bound we know of measures the change of the timescale additively, and it grows linearly with the sequence length unless the timescale is bounded away from zero. In this paper, we measure the change on a log scale instead. The key observation is that the decay factor of each step then moves by at most a fixed fraction of that step's input weight, so the per-step errors cannot pile up. Using this, we prove that two runs of a selective-scan cell from a common start stay within a bound that holds at every length, shrinks to zero with the size of the change and needs no lower bound on the timescale. We give a construction showing that, for small changes, this bound is within a factor of about 2 of the best possible, and we extend it to the update used in Mamba's released code when the timescale is bounded above. We then evaluate block-level bounds built from the theorem on two small models trained on one day of order book data. With a bounded timescale, a bound built on the existing analysis is tighter than ours for at most the first 8 steps. With Mamba's softplus timescale, whose certified lower bound is as small as 10−2110^{-21}, it is looser at every length. Our bounds hold for every input to a block, but they remain more than five orders of magnitude above the changes that gradient attacks find.

Introduction

Limit order book updates arrive as an unbounded stream. A selective state space model such as Mamba [11] can carry its state along such a stream far beyond its training windows. A bound on how far a small change of input moves that state then has to hold at every length. We prove such a bound for one cell, a scalar recurrence inside the selective scan, and measure how tight it is.

The closest bound we know of, Cao et al.’s Theorem 3.3(ii) [6], grows linearly with the position. The growth comes from its time-warp term, which accounts for the change of the timescale, the step size that the model computes from each input. Their Remark 3.4 caps the growth under a floor, a positive lower bound Δo\Delta_{\mathrm{o}} on the timescale, but the cap scales like 1/Δo1/\Delta_{\mathrm{o}} as the floor shrinks. Mamba’s softplus timescale has a floor only when its pre-activations are bounded below, and on one of our trained models we certify a floor of about 10−2110^{-21}.

We measure the change of the timescale on a log scale, whereas Cao et al. measure it additively. In their bound, both the linear growth and the need for a floor come from the additive measure. We require the timescales Δt\Delta_t and Δ~t\widetilde{\Delta}_t of a nominal and a perturbed run to differ by at most a factor eρe^\rho at every step, that is, ∣log⁡Δt−log⁡Δ~t∣≤ρ\lvert\log\Delta_t - \log\widetilde{\Delta}_t \rvert\leq\rho. We call this the log condition.

With an exact zero-order hold, which our models use, each step of the cell replaces the state by a weighted average of itself and an input-dependent equilibrium. The weight on the equilibrium, which we call the input weight, tends to zero with the timescale. Under the log condition, the two runs’ weights on the previous state differ by at most eρ−1e^{\rho}-1 times the input weight (Lemma 2). Under an additive bound, this difference does not shrink with the input weight and can exceed it at small timescales (Figure 1).

Figure 1. (a) Change of the transition e−λΔe^{-\lambda\Delta} against λΔ\lambda\Delta when log⁡Δ\log\Delta changes by 0.1 and when λΔ\lambda\Delta changes by 0.01, with the input weight 1−Aˉ1-\bar{A}. The two changes of λΔ\lambda\Delta are equal at λΔ≈0.095\lambda\Delta\approx0.095, where the two curves of the change of the transition cross. The first change stays below e0.1−1≈0.105e^{0.1}-1\approx0.105 times the input weight. The second exceeds the whole input weight for λΔ\lambda\Delta below about 0.0099. At λΔ=10−4\lambda\Delta=10^{-4} it is 99.5 times the input weight. (b) The gap ∣ht−h~t∣|h_t-\tilde{h}_t| in the construction of Proposition 4 for ρ=0.5\rho=0.5, M=1M=1 and η=0\eta=0. The nominal λΔt\lambda\Delta_t alternates between 0.05e0.50.05e^{0.5} and 0.05. The gap settles at about 0.4895, just below the value 2Mtanh⁡(ρ/2)≈0.48982M\tanh(\rho/2)\approx0.4898 that it approaches as the timescales tend to 0. The panel shows the bound E≈1.297E\approx1.297 of Theorem 3 and the trivial bound 2M+η=22M+\eta=2.

With Lemma 2, the gap between the two runs obeys an inequality of the same form as the update of the state, with a bounded equilibrium. The standard invariant argument for convex recursions, which Horbatko states for a Mamba channel [14 Lemma 4.4], bounds this gap at every length, as it bounds the state. From a common start, the variation-of-constants argument of Cao et al.’s Lemma B.2 also gives this bound after a change of time variable (Appendix § 10).

Our main result, Theorem 3, assumes the log condition, a bound η\eta on the change of the equilibrium and a bound MM on the equilibrium of the perturbed run. Two runs that start from the same state in [−M,M][-M,M] then stay within η+2M(eρ−1)\eta+2M(e^{\rho}-1) of each other at every step, with no floor on the timescale. For every MM, η\eta and ρ\rho, Proposition 4 constructs such runs whose gaps come arbitrarily close to η+2Mtanh⁡(ρ/2)\eta+2M\tanh(\rho/2). From a common start, the theorem’s constant is therefore within a factor eρ+1e^\rho+1 of the best possible, a factor that tends to 2 as ρ→0\rho\to0.

If ρ\rho is derived from Cao et al.’s additive bound and floor, our time-warp term 2M(eρ−1)2M(e^\rho-1) is no larger in each cell than that of their Remark 3.4. It is also no smaller than the limit, as the length grows, of a slightly sharper form of their bound, and the three are close when the floor is small (§ 3.3 and Appendix § 10). The gain in our experiments comes instead from bounding log⁡Δ\log\Delta directly from the bound on the timescale’s pre-activation.

A gradient search in one cell, with both runs started from 0, finds no gap above the value that the construction of Proposition 4 approaches (§ 5.2). From random initial sequences it comes within 0.4 per cent of that value at the longest lengths searched, and closer still from initial sequences shaped like the construction.

We then train a small two-block model on one day of limit order book data, once with a timescale bounded to a fixed range and once with Mamba’s softplus timescale. From the theorem we build a bound for each block on how far its output can move when its normalised input changes (§ 4). With the bounded timescale, the same bound with an additive time-warp term of Cao et al.’s form in place of ours is tighter for at most the first 8 steps. Without a floor, it then grows linearly with the length. Under the floor that we certify, it levels off at a multiple of ours that is set mainly by the chosen timescale range (§ 5.3). With the softplus timescale, it is looser than ours at every length, and the certified floor is too small to stop its linear growth at any length we test.

Gradient attacks give lower bounds on the true sensitivity, and our block bounds, which hold for every input of a block, lie far above them. Along a given input the realised timescale is known, and the log and additive bounds on its change agree to first order. The log scale therefore matters for bounds that hold for every input, where no realised timescale is available (§ 5.4). Code and the recorded runs are at https://github.com/jackdoe/java/mamba-v-lob.

Section 2 sets up the cell. Section 3 proves the theorem, states the lower bound and extends the theorem to the exponential-Euler step of the released Mamba implementations, with the proofs of the lower bound and the extension in Appendix § 9. Section 4 builds the block bounds, and Section 5 reports the measurements.

Setting

The selective cell

A selective state space model computes a timescale Δt>0\Delta_t>0 and matrices BtB_t and CtC_t from the layer input at every position tt. It uses the timescale to discretise a fixed diagonal matrix AA with negative diagonal entries [11]. We certify a Mamba block that uses the exact zero-order hold for this discretisation. The block applies a layer normalisation, an input projection, a causal depthwise convolution with a SiLU, the projections that produce Δ\Delta, BB and CC, and the scan. A SiLU gate, an output projection and a residual connection follow, and Appendix § 9.§ 4 gives the block in full.

Because AA is diagonal, the scan splits into independent scalar recurrences, one for each channel ii and state index nn. We call each of them a cell. A cell updates a scalar state hth_t from the input utu_t of its channel, and we write λ=∣Ai,n∣>0\lambda= \lvert A_{i,n}\rvert> 0 for its decay rate. We state our results on the scan for one cell.

We consider two maps from the timescale pre-activation zz to Δ\Delta. The bounded map Δ=Δmin⁡(Δmax⁡/Δmin⁡)sigmoid⁡(z)\Delta= \Delta_{\min}(\Delta_{\max}/\Delta_{\min})^{\operatorname{sigmoid}(z)} keeps Δ\Delta in [Δmin⁡,Δmax⁡][\Delta_{\min}, \Delta_{\max}]. Mamba’s map Δ=softplus⁡(z)\Delta= \operatorname{softplus}(z) is positive, but it has no positive lower bound over real zz. We call a positive lower bound on the timescale a floor.

Under the exact zero-order hold, one step of a cell uses

Aˉt=e−λΔt,Bˉt=e−λΔt−1−λBt=(1−Aˉt)Btλ,(1)\bar{A}_t = e^{-\lambda\Delta_t}, \qquad\bar{B}_t = \frac{e^{-\lambda\Delta_t}-1}{-\lambda}B_t = (1-\bar{A}_t)\frac{B_t}{\lambda}, \tag*{(1)}

and with ct=Btut/λc_t = B_tu_t/\lambda the update ht=Aˉtht−1+Bˉtuth_t = \bar{A}_th_{t-1} + \bar{B}_tu_t becomes

ht=Aˉtht−1+(1−Aˉt)ct.(2)h_t = \bar{A}_t h_{t-1} + (1-\bar{A}_t)c_t. \tag*{(2)}

Since Aˉt∈(0,1)\bar{A}_t \in(0,1) for every input, each step is a convex combination of the previous state and ctc_t. We call ctc_t the equilibrium and 1−Aˉt1-\bar{A}_t the input weight of step tt. Gu and Dao [11 Theorem 1] give (2) as a gated recurrence for λ=1\lambda= 1 and Bt=1B_t = 1 with a softplus timescale.

The released implementations of Mamba use a different input term, which we treat in § 3.4.

The forward invariant

Because each step is a convex combination, a bound on the equilibria and on the initial state bounds the state at every step.

Proposition 1 (Forward invariant). If ∣ct∣≤M\lvert c_t\rvert\le M for every t≥0t \ge0, then ∣ht∣≤max⁡(∣h−1∣,M)\lvert h_t\rvert\le\max(\lvert h_{-1}\rvert, M) for every t≥0t \ge0.

Proof. We argue by induction on tt: if ∣ht−1∣≤max⁡(∣h−1∣,M)\lvert h_{t-1}\rvert\le\max(\lvert h_{-1}\rvert, M), then ∣ht∣≤Aˉt∣ht−1∣+(1−Aˉt)∣ct∣≤max⁡(∣h−1∣,M)\lvert h_t\rvert\le\bar{A}_t\lvert h_{t-1}\rvert+ (1-\bar{A}_t)\lvert c_t\rvert\le\max(\lvert h_{-1}\rvert, M).

Proposition 1 is Lemma 4.4 of Horbatko [14], stated there for a diagonal Mamba channel. For h−1=0h_{-1} = 0 it also follows from the state bound of Cao et al.’s Theorem 3.3(i) and Lemma B.1 [6]. The reading of a convex gated update as a leaky integrator with a learned step goes back to Tallec and Ollivier [27] and Gu et al. [12].

Since BtB_t and utu_t are activations, the hypothesis ∣ct∣≤M\lvert c_t\rvert\le M has to hold for every input the model can receive. Within the scan and the readout yt=Ctht+Duty_t = C_th_t + D u_t that follows it, only AA and the skip weight DD are fixed parameters. The timescale Δt\Delta_t, the matrices BtB_t and CtC_t and the scan input utu_t are all computed from the block input.

Each block opens with a layer normalisation. By the Laguerre–Samuelson inequality [18, 26], the output of the normalisation lies in a bounded set fixed by its own parameters (Lemma 9 in Appendix § 11). Interval arithmetic carries this set through the projections, the convolution and the activations. For the projection that follows the normalisation, we use the fused bound of Proposition 10 in Appendix § 11, which follows from the geometry of layer normalisation that Riechers [25] describes. The result is a box, with one interval per coordinate, for the output of each of these stages, including BB, uu and zz. We call these the certified boxes. The boxes of BB and uu give the radius M=sup⁡∣Bu∣/λM = \sup\lvert Bu\rvert/\lambda of each cell. The box ZZ of the timescale pre-activation of a channel gives its certified floor Δ(inf⁡Z)\Delta(\inf Z) and certified ceiling Δ(sup⁡Z)\Delta(\sup Z), the smallest and largest timescales that the box allows.

We have machine-checked (1), Proposition 1, Lemma 9 and the first part of Proposition 10 in Isabelle/HOL [22], and the theory is in our repository. Lemma 2, Theorem 3 and Proposition 4 are not machine-checked. We compute MM for a trained model by interval propagation in Python code, which we test but have not formally verified.

A sensitivity bound at every length

We bound how far apart two runs of one cell can be at any length. The bound holds when their equilibria are bounded and their timescales differ by at most a factor eρe^{\rho} at every step. The nominal run has timescales Δt>0\Delta_t > 0, equilibria ctc_t, transitions Aˉt=e−λΔt\bar{A}_t = e^{-\lambda\Delta_t} and states hth_t given by (2). The perturbed run has Δ~t>0\widetilde{\Delta}_t > 0, c~t\widetilde{c}_t, A~t=e−λΔ~t\widetilde{A}_t = e^{-\lambda\widetilde{\Delta}_t} and h~t\widetilde{h}_t. We write et=ht−h~te_t = h_t - \widetilde{h}_t for the error and call ∣et∣\lvert e_t\rvert the gap. The hypothesis on the timescales is ∣log⁡Δt−log⁡Δ~t∣≤ρ\lvert\log\Delta_t - \log\widetilde{\Delta}_t\rvert\le\rho for every tt, where ρ≥0\rho\ge0 and log⁡\log is the natural logarithm. We write η=sup⁡t∣ct−c~t∣\eta= \sup_t\lvert c_t - \widetilde{c}_t\rvert for the change of the equilibrium.

The ratio lemma

Lemma 2 shows that a relative change of the timescale moves the transition by at most a fixed fraction of the input weight.

Lemma 2 (Relative perturbation of the transition). For x>0x > 0, s∈Rs \in\mathbb{R} and x~=xes\widetilde{x} = xe^s we have

∣e−x−e−x~∣≤∣es−1∣(1−e−x).\lvert e^{-x} - e^{-\widetilde{x}}\rvert\le\lvert e^s - 1\rvert(1 - e^{-x}).

In particular the left-hand side is at most (eρ−1)(1−e−x)(e^\rho- 1)(1 - e^{-x}) whenever ∣s∣≤ρ\lvert s\rvert\le\rho.

Proof. We compare chord slopes of the concave function y↦1−e−yy \mapsto1 - e^{-y}, which vanishes at 0 and whose chord slope from the origin, m(y)=(1−e−y)/ym(y) = (1 - e^{-y})/y, is therefore decreasing on (0,∞)(0,\infty). For s≥0s \ge0 we have x~≥x\widetilde{x} \ge x, and the decrease of mm gives 0≤e−x−e−x~=x~m(x~)−xm(x)≤(x~−x)m(x)=(es−1)(1−e−x)0 \le e^{-x} - e^{-\widetilde{x}} = \widetilde{x}m(\widetilde{x}) - xm(x) \le(\widetilde{x} - x)m(x) = (e^s - 1)(1 - e^{-x}).

For s<0s < 0 we have x~<x\widetilde{x} < x and m(x~)≥m(x)m(\widetilde{x}) \ge m(x), hence 0≤e−x~−e−x=xm(x)−x~m(x~)≤(x−x~)m(x)=(1−es)(1−e−x)0 \le e^{-\widetilde{x}} - e^{-x} = xm(x) - \widetilde{x}m(\widetilde{x}) \le(x-\widetilde{x})m(x) = (1-e^s)(1-e^{-x}), which together with ∣es−1∣≤e∣s∣−1\lvert e^s - 1\rvert\le e^{\lvert s\rvert}-1 proves both bounds.

With x=λΔtx = \lambda\Delta_t and s=log⁡(Δ~t/Δt)s = \log(\widetilde{\Delta}_t/\Delta_t), Lemma 2 gives

∣Aˉt−Aˉ~t∣≤(eρ−1)(1−Aˉt),(3)\lvert\bar{A}_t - \widetilde{\bar{A}}_t\rvert\le(e^\rho- 1)(1-\bar{A}_t), \tag*{(3)}

and with the runs exchanged it gives the same bound with 1−Aˉ~t1-\widetilde{\bar{A}}_t on the right. Cao et al.’s Lemma B.3 bounds the same change by λ∣Δt−Δ~t∣\lambda\lvert\Delta_t-\widetilde{\Delta}_t\rvert, which lacks the factor 1−Aˉt1-\bar{A}_t. Under an additive bound on ∣Δt−Δ~t∣\lvert\Delta_t-\widetilde{\Delta}_t\rvert, the term that each step adds to a bound on the gap does not shrink with the input weight. The later transitions damp these terms, but their sum grows like tt unless a floor makes the damping geometric.

Figure 1(a) shows the two cases at a single step. When log⁡Δ\log\Delta changes by a fixed amount, the change of the transition vanishes with the input weight as λΔ→0\lambda\Delta\to0. When λΔ\lambda\Delta changes by a fixed amount, the change of the transition does not vanish, and for small λΔ\lambda\Delta it exceeds the whole input weight.

The cell theorem

Theorem 3 bounds the gap at every step by a constant that depends only on η\eta, MM and ρ\rho, or by the initial gap if that is larger.

Theorem 3 (Cell bound). Suppose that ∣log⁡Δt−log⁡Δ~t∣≤ρ|\log\Delta_t-\log\widetilde{\Delta}_t|\le\rho and ∣c~t∣≤M|\widetilde{c}_t|\le M for every t≥0t\ge0, and that ∣h~−1∣≤M|\widetilde{h}_{-1}|\le M. With η=sup⁡t∣ct−c~t∣\eta=\sup_t|c_t-\widetilde{c}_t|, E=η+2M(eρ−1)E=\eta+2M(e^\rho-1) and Pt=∏k=0tAˉkP_t=\prod_{k=0}^{t}\bar{A}_k, for every t≥0t\ge0

∣ht−h~t∣≤Pt∣h−1−h~−1∣+(1−Pt)E≤max⁡(∣h−1−h~−1∣,E).|h_t-\widetilde{h}_t|\le P_t|h_{-1}-\widetilde{h}_{-1}|+(1-P_t)E\le\max(|h_{-1}-\widetilde{h}_{-1}|,E).

Proof. We write the error as a convex recursion of the form (2) with a bounded equilibrium and then argue as for Proposition 1, which also gives ∣h~t∣≤M|\widetilde{h}_t|\le M for every t≥−1t\ge-1. Adding and subtracting Aˉth~t−1\bar{A}_t\widetilde{h}_{t-1} and (1−Aˉt)c~t(1-\bar{A}_t)\widetilde{c}_t in ht−h~th_t-\widetilde{h}_t gives

et=Aˉtet−1+(1−Aˉt)(ct−c~t)+(Aˉt−A~t)(h~t−1−c~t).(4)e_t=\bar{A}_t e_{t-1}+(1-\bar{A}_t)(c_t-\widetilde{c}_t)+(\bar{A}_t-\widetilde{A}_t)(\widetilde{h}_{t-1}-\widetilde{c}_t). \tag*{(4)}

Since ∣h~t−1−c~t∣≤2M|\widetilde{h}_{t-1}-\widetilde{c}_t|\le2M, the bound (3) gives ∣et∣≤Aˉt∣et−1∣+(1−Aˉt)E|e_t|\le\bar{A}_t|e_{t-1}|+(1-\bar{A}_t)E. Unrolling this inequality gives ∣et∣≤Pt∣e−1∣+E∑k=0t(1−Aˉk)∏r=k+1tAˉr|e_t|\le P_t|e_{-1}|+E\sum_{k=0}^{t}(1-\bar{A}_k)\prod_{r=k+1}^{t}\bar{A}_r, where the sum telescopes to 1−Pt1-P_t.

The passage from the one-step inequality to the bound of Theorem 3 is a standard contraction estimate, sampled at the end of each step. In the accumulated time τt=∑s≤tΔs\tau_t=\sum_{s\le t}\Delta_s, the cell contracts at the constant rate λ\lambda (Appendix § 10). The estimate compares a system that contracts at a constant rate μ\mu with the same system plus a disturbance dd [20, 5]. After a time σ\sigma, the distance between their trajectories is at most e−μσe^{-\mu\sigma} times its initial value plus (1−e−μσ)sup⁡∣d∣/μ(1-e^{-\mu\sigma})\sup|d|/\mu. With μ=λ\mu=\lambda, ∣d∣≤λE|d|\le\lambda E and σ=τt\sigma=\tau_t, this is the bound of Theorem 3.

Per step, the cell contracts by the factor Aˉt=e−λΔt\bar{A}_t=e^{-\lambda\Delta_t}, and without a floor the rate λΔt\lambda\Delta_t has no positive lower bound. The bound is still uniform in tt, because the disturbance at each step carries the same factor 1−Aˉt1-\bar{A}_t as the contraction, so their ratio stays bounded as Δt→0\Delta_t\to0. A bound that treats the transition and the input gain separately loses this cancellation (Appendix § 11).

The proof uses the hypotheses one step at a time. The bounds ρ\rho and η\eta may therefore depend on tt, with EE replaced by max⁡s≤t(ηs+2M(eρs−1))\max_{s\le t}(\eta_s+2M(e^{\rho_s}-1)).

If ∣h−1∣≤M+η|h_{-1}| \le M+\eta, Proposition 1 applied to both runs gives ∣ht∣≤M+η|h_t| \le M+\eta and ∣h~t∣≤M|\tilde{h}_t| \le M, hence the trivial bound ∣et∣≤2M+η|e_t| \le2M+\eta. For M>0M>0, the trivial bound exceeds EE exactly when ρ<log⁡2\rho< \log2.

Proposition 4 (Lower bound). Let λ>0\lambda> 0, M>0M > 0, η≥0\eta\ge0 and ρ>0\rho> 0. For every ε>0\varepsilon> 0 there are timescales Δt,Δ~t>0\Delta_t,\tilde{\Delta}_t > 0 with ∣log⁡Δt−log⁡Δ~t∣=ρ|\log\Delta_t-\log\tilde{\Delta}_t|=\rho and equilibria with ∣c~t∣=M|\tilde{c}_t|=M and ct−c~t=ηc_t-\tilde{c}_t=\eta for every t≥0t\ge0 such that the two runs from h−1=h~−1=0h_{-1}=\tilde{h}_{-1}=0 satisfy ∣ht−h~t∣>η+2Mtanh⁡(ρ/2)−ε|h_t-\tilde{h}_t|>\eta+2M\tanh(\rho/2)-\varepsilon at some step tt.

The runs in the proof of Proposition 4 (Appendix § 9.§ 1) satisfy the hypotheses of Theorem 3. The perturbed equilibrium c~t\tilde{c}_t alternates between MM and −M-M. The perturbed timescale is e−ρe^{-\rho} times the nominal one on the steps with c~t=M\tilde{c}_t=M and eρe^\rho times it on the others. The nominal run therefore has the larger input weight on the steps with c~t=M\tilde{c}_t=M and the smaller one on the others. As t→∞t\to\infty and then the timescales tend to 00, the gap after a step with c~t=M\tilde{c}_t=M tends to η+2Mtanh⁡(ρ/2)\eta+2M\tanh(\rho/2). Figure 1(b) shows the gap for ρ=0.5\rho=0.5, M=1M=1 and η=0\eta=0. It settles close to 2Mtanh⁡(ρ/2)2M\tanh(\rho/2), below the bound EE of Theorem 3.

Proposition 4 limits how far Theorem 3 can be improved. Every bound on the gap from a common start that holds under the hypotheses of the theorem and depends only on MM, η\eta and ρ\rho is at least η+2Mtanh⁡(ρ/2)\eta+2M\tanh(\rho/2). Since tanh⁡(ρ/2)=(eρ−1)/(eρ+1)\tanh(\rho/2)=(e^\rho-1)/(e^\rho+1), the term 2M(eρ−1)2M(e^\rho-1) of EE is eρ+1e^\rho+1 times 2Mtanh⁡(ρ/2)2M\tanh(\rho/2). The bound EE is therefore within a factor eρ+1e^\rho+1 of the smallest such bound, and this factor tends to 22 as ρ→0\rho\to0.

From a common start in [−M,M][-M,M], the largest possible gap thus lies between η+2Mtanh⁡(ρ/2)\eta+2M\tanh(\rho/2) and min⁡(E,2M+η)\min(E,2M+\eta). We do not prove that η+2Mtanh⁡(ρ/2)\eta+2M\tanh(\rho/2) is the largest possible gap. The numerical search of § 5.2, from the common start 00, finds none above it.

Comparison with Cao et al.

We replace the bound on log⁡Δ\log\Delta by an additive bound ∣Δt−Δ~t∣≤ηΔ|\Delta_t-\tilde{\Delta}_t|\le\eta_\Delta and keep the other hypotheses of Theorem 3. We also assume a lower bound Δlo≥0\Delta_{\mathrm{lo}}\ge0 on both timescales, which is a floor when it is positive. By the mean value theorem, the transitions then satisfy ∣Aˉt−Aˉ~t∣≤λe−λΔloηΔ|\bar{A}_t-\tilde{\bar{A}}_t|\le\lambda e^{-\lambda\Delta_{\mathrm{lo}}}\eta_\Delta. We carry the last term of (4) forward with a factor of at most e−λΔloe^{-\lambda\Delta_{\mathrm{lo}}} per step. From a common start in [−M,M][-M,M], this gives

∣hL−1−h~L−1∣≤η+2Mλe−λΔloηΔ∑k=0L−1e−kλΔlo.(5)|h_{L-1}-\tilde{h}_{L-1}|\le\eta+2M\lambda e^{-\lambda\Delta_{\mathrm{lo}}}\eta_\Delta\sum_{k=0}^{L-1}e^{-k\lambda\Delta_{\mathrm{lo}}}. \tag*{(5)}

We call (5) the additive baseline. It has the structure of Cao et al.’s Theorem 3.3(ii) and Remark 3.4. It keeps the factor e−λΔloe^{-\lambda\Delta_{\mathrm{lo}}} that their Lemma B.3 drops, which makes it slightly smaller than their constant. As Cao et al. do, we call the term of a bound that comes from the change of the timescale its time-warp term. In EE this term is 2M(eρ−1)2M(e^\rho-1), and in (5) it is the term with the sum.

Without a floor, the sum equals LL, so the additive baseline grows linearly in the length LL. With a floor it saturates at η+2MληΔ/(eλΔlo−1)\eta+2M\lambda\eta_\Delta/(e^{\lambda\Delta_{\mathrm{lo}}}-1) as L→∞L\to\infty. Its time-warp term is then the one-step cost 2Mλe−λΔloηΔ2M\lambda e^{-\lambda\Delta_{\mathrm{lo}}}\eta_\Delta times 1/(1−e−λΔlo)1/(1-e^{-\lambda\Delta_{\mathrm{lo}}}). This factor is of order 1/(λΔlo)1/(\lambda\Delta_{\mathrm{lo}}) as the floor shrinks.

Theorem 3 needs neither a floor nor an additive bound on the timescales, only a bound on the change of log⁡Δ\log\Delta. Both timescale maps give one from a bound on the change of the pre-activation zz. By Lemma 6 in Appendix § 9, dlog⁡Δ/dzd\log\Delta/dz is at most 11 for the softplus map and at most 14log⁡(Δmax⁡/Δmin⁡)\frac{1}{4}\log(\Delta_{\max}/\Delta_{\min}) for the bounded map. Cao et al. derive the Lipschitz constant of Δ\Delta that they assume from the same bound on zz.

With a floor, the additive bound alone also bounds the change of log⁡Δ\log\Delta, by ρ=log⁡(1+ηΔ/Δlo)\rho=\log(1+\eta_\Delta/\Delta_{\mathrm{lo}}). With this ρ\rho, our time-warp term lies between the time-warp term of the additive baseline as L→∞L \to\infty and that of Cao et al.’s Remark 3.4. This follows from 1−e−x≤x≤ex−11-e^{-x}\le x\le e^x-1 for x=λΔlox=\lambda\Delta_{\mathrm{lo}} (Appendix § 10). In our experiments, the gain over the additive baseline comes from bounding log⁡Δ\log\Delta directly from the bound on zz, not through an additive bound on Δ\Delta.

For the bounded map, Remark 5 compares the two time-warp terms in one cell once the additive baseline has saturated.

Remark 5 (The ratio for the bounded map). Let Δlo≥Δmin⁡\Delta_{\mathrm{lo}}\ge\Delta_{\min} be a floor, R=Δmax⁡/Δmin⁡R=\Delta_{\max}/\Delta_{\min} and x=λΔlox=\lambda\Delta_{\mathrm{lo}}, let ZZ be the certified box of the pre-activation of one channel, and let a bound ζ\zeta on the change of zz within ZZ give ηΔ=ζsup⁡Z∣Δ′∣\eta_\Delta=\zeta\sup_Z|\Delta'| and ρ=ζsup⁡Z∣(log⁡Δ)′∣\rho=\zeta\sup_Z|(\log\Delta)'|. To first order in ζ\zeta, the time-warp term of (5) as L→∞L\to\infty is

sup⁡Z∣Δ′∣Δlosup⁡Z∣(log⁡Δ)′∣⋅xex−1\frac{\sup_Z|\Delta'|}{\Delta_{\mathrm{lo}}\sup_Z|(\log\Delta)'|}\cdot\frac{x}{e^x-1}

times ours (Appendix § 10). If ZZ contains 0, then sup⁡Z∣(log⁡Δ)′∣=14log⁡R\sup_Z|(\log\Delta)'|=\frac{1}{4}\log R by Lemma 6. With p=sigmoid⁡(z)p=\operatorname{sigmoid}(z) we have Δ′=Δ(log⁡Δ)′=Δmin⁡Rpp(1−p)log⁡R\Delta'=\Delta(\log\Delta)'=\Delta_{\min}R^p p(1-p)\log R, and since Δmin⁡/Δlo≤1\Delta_{\min}/\Delta_{\mathrm{lo}}\le1 and x/(ex−1)≤1x/(e^x-1)\le1, the ratio is then at most 4max⁡p∈(0,1)Rpp(1−p)4\max_{p\in(0,1)}R^p p(1-p), which for R=100R=100 is 25.8, attained at p≈0.83p\approx0.83. With the floor Δlo=Δ(inf⁡Z)\Delta_{\mathrm{lo}}=\Delta(\inf Z) that the box certifies, this ratio depends on the trained weights only through ZZ and xx.

The exponential-Euler step

The released implementations of Mamba keep the transition Aˉt=e−λΔt\bar{A}_t=e^{-\lambda\Delta_t} but replace the input term Bˉtut\bar{B}_t u_t by ΔtBtut\Delta_t B_tu_t [19 Section 3.1]. This exponential-Euler step combines the exact transition with Euler’s rule for the input integral. It differs from the exponential Euler method of the literature on exponential integrators, which for this cell gives (1).

The update of this step is still a convex combination of the previous state and an equilibrium. The equilibrium is now enlarged by the factor Ψ(λΔt)\Psi(\lambda\Delta_t), where Ψ(x)=x/(1−e−x)\Psi(x)=x/(1-e^{-x}) lies in (1,1+x)(1,1+x) for x>0x>0. This factor grows without bound in Δt\Delta_t. The analogues of Proposition 1 and Theorem 3 for this step therefore need an upper bound Δhi\Delta_{\mathrm{hi}} on the timescale, but they still need no floor. Corollary 7 in Appendix § 9 gives the analogue of the theorem for a common zero start. With G=Ψ(λΔhi)G=\Psi(\lambda\Delta_{\mathrm{hi}}), the bound EE holds for this cell with η\eta replaced by GηbG\eta^b, MM by GMGM and the factor 2 by 3. Here ηb\eta^b bounds the change of bt=Btut/λb_t=B_tu_t/\lambda.

From the cell to a certificate

Corollary 8 in Appendix § 9.§ 4 turns Theorem 3 into a bound on the change of a whole block’s output. The bound holds in real arithmetic for every input of the block, and we call such a bound a certificate. It needs no input data, so we also call it data-free.

The corollary follows a perturbation of the normalised input through the stages of the block in order. It composes Theorem 3 with sup-norm Lipschitz constants of the other stages, taken over the certified boxes that also give the radius MM. The input projection, the convolution and the SiLU bound the change of the scan input uu. The projections that follow bound the changes of BB, of CC and of the timescale pre-activation zz. Lemma 6 (Appendix § 9) turns the bound on zz into ρ\rho. A product rule gives the bound η\eta on the change of the equilibrium ct=Btut/λc_t=B_tu_t/\lambda. Theorem 3 then bounds the change of the state in each cell. We carry this bound through the readout Ctht+DtutC_th_t+D_tu_t, the gate and the output projection.

Every stage of the certificate takes the worst case over its box and ignores the dependence between its inputs. Section 5.4 bounds from above how much is lost up to each stage.

Experiments

Setup

Our data are the Databento MBP-10 records of SPY on 28 January 2021. After feature construction they give about 4.55 million rows and 43 model inputs. We train a small model on windows of 100 ticks. From each window it predicts the log return of the mid-price over the next 100 ticks.

We divide the records chronologically into a training split of 70 per cent and validation and test splits of 15 per cent each. We purge each boundary by the prediction horizon of 100 ticks. We winsorise every feature at the 0.001 and 0.999 quantiles of its training values, in training and at inference alike.

The model maps the 43 inputs to width dmodel=64d_{\mathrm{model}} = 64 with a linear layer, a SiLU and a layer normalisation. Two Mamba blocks follow, each with dinner=128d_{\mathrm{inner}} = 128 channels and dstate=16d_{\mathrm{state}} = 16 state indices per channel. Each block has a convolution of width 4 and a timescale projection of rank 4. A final layer normalisation and a two-layer head read the prediction at the last position.

We train two versions of this model, with set-ups that differ only in the timescale map. We call each of the two trained models an arm. The bounded arm uses the bounded map of § 2.1 with Δ\Delta in [10−3,10−1][10^{-3}, 10^{-1}], the range from which Mamba initialises its timescales [11]. The softplus arm uses Mamba’s map Δ=softplus⁡(z)\Delta= \operatorname{softplus}(z). Both arms use the exact zero-order hold (1), not the exponential-Euler input term of the released implementations (§ 3.4). After training, the decay rates λ\lambda of the bounded arm lie between 0.912 and 17.14.

We score predictions by the rank information coefficient, the Spearman correlation between predicted and realised returns. For each arm we keep the checkpoint with the best rank information coefficient on a fixed slice of 12,800 windows of the validation split. Training ran for at most 3000 steps, and the kept checkpoints come from step 750 on the bounded arm and step 2000 on the softplus arm.

Each arm is a single training run, and the two kept checkpoints come from different training steps, so we draw no conclusion about the timescale maps from differences between the arms. We keep the softplus arm to show what a very small certified floor does to the additive baseline.

On the first 25,600 windows of the test split the bounded arm has a rank information coefficient of 0.13. Its block bootstrap 95 per cent interval is [0.075, 0.197], so this arm has learned some signal. We do not compare it with other architectures.

We bound how much a perturbation of a block’s input changes the block’s output before the residual connection. This output is F=out_proj⁡(y⊙SiLU⁡(g))F = \operatorname{out\_proj}(y \odot\operatorname{SiLU}(g)), where yy is the readout and gg the gate pre-activation (Appendix § 9.§ 4). The perturbation moves the normalised input of the block by at most δ\delta in sup-norm at every position. We divide each bound, and each change that an attack finds, by δ\delta and call the quotient a slope. For a certificate we take the limit of this quotient as δ→0\delta\to0.

When we compare a certificate with an attack, we compute the certificate on boxes widened to cover every point within δ\delta of a normalised vector (Appendix § 9.§ 4). The widening is needed because an attacked point need not be a normalised vector. At δ=10−2\delta= 10^{-2} it raises the certificate by less than 5 per cent.

We compute the boxes and radii of the certificate by interval arithmetic in float32, and the rest of the certificate in round-to-nearest float64. No stage uses outward rounding. The relative rounding error per operation is of order 10−710^{-7} in float32 and 10−1610^{-16} in float64.

Each attack runs sign-gradient ascent in float64, and we re-evaluate the change it finds with a reference forward pass. Every attack value is therefore a lower bound on the true sensitivity. We call the largest slope that an attack finds the attack slope. The ratio of a certificate’s slope to the attack slope is an upper bound on the certificate’s looseness.

One cell

We test the tightness of Theorem 3 on a single cell by searching numerically for the largest gap between two runs. The cell has λ=1\lambda= 1 and M=1M = 1, and both runs start from 0. The gap after TT steps is ∣hT−1−h~T−1∣|h_{T-1} - \tilde{h}_{T-1}|. The search runs gradient ascent with Adam in float64 for 600 steps from 64 random initial sequences and from initial sequences shaped like the construction of Proposition 4. In the shaped sequences, the perturbed equilibrium c~t\tilde{c}_t follows a square wave with runs of 1, 2, 4, 8 or 16 steps. We re-run the best sequence found with the plain recursion (2). Every gap we report is therefore attained by two admissible runs and is a lower bound on the true worst case.

We first search the hypothesis class of Theorem 3, with the nominal timescale λΔt\lambda\Delta_t free in [10−4,10][10^{-4}, 10]. We take ρ∈{0.01,0.03,0.1,0.3,0.6}\rho\in\{0.01, 0.03, 0.1, 0.3, 0.6\} and η∈{0,0.1}\eta\in\{0, 0.1\} at lengths TT from 10 to 1000.

No gap found at any searched length exceeds η+2Mtanh⁡(ρ/2)\eta+ 2M \tanh(\rho/2), the value that the construction of Proposition 4 approaches. For T≥300T \ge300 Adam from the initial sequences shaped like the construction comes within 0.01 per cent of this value at every ρ\rho and η\eta. At the same lengths, Adam from the 64 random initial sequences comes within 0.4 per cent of it. At T=10T = 10 the largest gap found lies less than 1.1 per cent below the value.

For η=0\eta= 0 and T≥300T \ge300, the theorem’s bound E=η+2M(eρ−1)E = \eta+ 2M(e^\rho- 1) equals eρ+1e^\rho+ 1 times the largest gap found, to four significant figures. This factor is about 2.0 at ρ=0.01\rho= 0.01 and about 2.8 at ρ=0.6\rho= 0.6 (Figure 2, right).

Figure 2. Largest gap found by numerical search in one cell with λ=1\lambda= 1 and M=1M = 1, with both runs started from 0. The left and middle panels show the largest gap found against the length TT for the bounded and the softplus map, at ρ=0.01\rho= 0.01 and η=0\eta= 0. They also show the bound of Theorem 3, the additive baseline (5) with and without the floor Δ(inf⁡Z)\Delta(\inf Z), and the trivial bound 2M+η=22M + \eta= 2. The gap levels off at about half of the theorem’s bound. On the softplus map the two forms of the baseline coincide at every length shown. The right panel shows the largest gap found at T=1000T = 1000, divided by the theorem’s bound EE, against ρ\rho for η=0\eta= 0 and η=0.1\eta= 0.1. The curve for each η\eta is the value η+2Mtanh⁡(ρ/2)\eta+2M\tanh(\rho/2) that the construction of Proposition 4 approaches, divided by EE. Every point lies less than 0.01 per cent below the curve, and none lies above it.

We then set η=0\eta= 0 and search the perturbation model of our certificates. In this model each timescale is the image of a pre-activation under the timescale map. The nominal pre-activation ztz_t lies in a box ZZ, and the perturbed one lies in ZZ within ζ\zeta of ztz_t. For the bounded map we take Δ∈[10−3,10−1]\Delta\in[10^{-3}, 10^{-1}] and Z=[−8,8]Z = [-8, 8]. For the softplus map we take Z=[−48,2]Z = [-48, 2]. On both maps we choose ζ\zeta so that the bound on the change of log⁡Δ\log\Delta is ρ=ζsup⁡Z∣(log⁡Δ)′∣=0.01\rho= \zeta\sup_Z |(\log\Delta)'| = 0.01. The additive baseline (5) with L=TL = T uses the bound ηΔ=ζsup⁡Z∣Δ′∣\eta_\Delta= \zeta\sup_Z |\Delta'| on the change of Δ\Delta, and its floored form uses the floor Δlo=Δ(inf⁡Z)\Delta_{\mathrm{lo}} = \Delta(\inf Z). We search at lengths TT from 3 to 3000.

On both maps the largest gap found grows with TT and levels off at about half of the theorem’s bound (Figure 2, left and middle). This is the value that the search over the hypothesis class gives for ρ=0.01\rho= 0.01 and η=0\eta= 0.

Since M=1M = 1 and η=0\eta= 0 here, every gap between two admissible runs is at most the trivial bound 2M+η=22M + \eta= 2 of § 3. This bound does not tend to zero with the perturbation, so it gives no finite slope and plays no part in the block certificates of § 5.3.

For the bounded map the additive baseline is above the theorem’s bound from T=39T = 39 without the floor and from T=40T = 40 with it. Both lengths are about inversely proportional to λ\lambda. With the same box and ζ\zeta, at λ=17.14\lambda= 17.14, the largest decay rate of the trained bounded arm, the baseline would be above the theorem’s bound from T=3T = 3. Without the floor the baseline keeps growing linearly. With the floor it tends to about 25.6 times the theorem’s bound as T→∞T \to\infty, below the value 25.8 of Remark 5. Both forms of the baseline stay below the trivial bound at every searched length.

For the softplus map the floor Δ(inf⁡Z)\Delta(\inf Z) is about 1.4×10−211.4 \times10^{-21}. At every searched length the baseline with this floor coincides with the one without it. The baseline is above the theorem’s bound from T=2T = 2 and above the trivial bound from T=114T = 114. It grows linearly to more than 2600 times the theorem’s bound at T=3000T = 3000.

Comparison with the additive baseline

Table 1 and Figure 3 compare our data-free certificate with one that uses the additive baseline (5) in place of Theorem 3 in each cell. Our certificate does not depend on the length LL, while the baseline without a floor grows linearly with LL. On the bounded arm the baseline passes our certificate after about 8 steps, the crossing LL in Table 1. It grows by about 0.11 times our certificate per step. At L=5000L = 5000 it is about 540 times ours on both blocks. On the softplus arm the baseline is looser at every length. It grows by about 3.9 times our certificate per step, and at L=5000L = 5000 it is about 2×1042 \times10^{4} times ours. These ratios at L=5000L = 5000 depend on the chosen length, whereas the crossing LL and the growth per step are properties of the trained model.

Figure 3. Data-free output slopes of block 0 of each arm against the sequence length LL. The flat line, labelled log-timescale, is our certificate, which uses the constant EE of Theorem 3 in every cell. Without a floor the additive baseline (5) grows linearly, and with the certified floor it levels off as L→∞L \to\infty. On the bounded arm both forms of the baseline pass the flat line near L=8L = 8. On the softplus arm the smallest certified floor is about 10−2110^{-21}, and the two forms of the baseline overlap at every length shown.

ArmBlockTheorem 3Additive, no floor, L=5000L = 5000RatioAdditive, floor, L→∞L \to\inftyRatioCrossing
bounded09.34×1069.34 \times10^{6}5.04×1095.04 \times10^{9}5392.16×1082.16 \times10^{8}23.18.3
bounded16.08×1066.08 \times10^{6}3.27×1093.27 \times10^{9}5381.37×1081.37 \times10^{8}22.48.1
softplus01.85×1071.85 \times10^{7}3.59×10113.59 \times10^{11}1.95×1041.95 \times10^{4}3.93×10263.93 \times10^{26}2.1×10192.1 \times10^{19}none
softplus11.46×1071.46 \times10^{7}2.87×10112.87 \times10^{11}1.97×1041.97 \times10^{4}4.11×10224.11 \times10^{22}2.8×10152.8 \times10^{15}none

Table 1. Data-free output slopes of each block of the two arms, maximised over output coordinates and rounded up. The additive columns use the baseline (5) without a floor at L=5000L = 5000 and with the certified floor as L→∞L \to\infty. Each ratio divides the unrounded additive slope to its left by the unrounded slope in the Theorem 3 column. The crossing LL is the length at which the affine baseline without a floor reaches the flat bound of Theorem 3. A crossing of none means that the baseline is the larger bound at every length.

Our certificate, the flat line in Figure 3, uses the constant EE of the theorem in every cell. The theorem also gives the finite-horizon bound (1−Pt)E(1 - P_t)E, which is smaller than EE, so the baseline passes this bound no later than it passes EE. We also compare the two time-warp terms cell by cell, with our finite-horizon term at its worst over admissible timescales. Over the cells of the bounded arm, the baseline’s term reaches ours after a median of 5 steps (n∗n^* in Appendix § 10).

With the certified floor the baseline levels off as L→∞L \to\infty. On the bounded arm its limit is 23 times our certificate on block 0 and 22 times on block 1. This ratio is set mainly by our choice of Δmax⁡/Δmin⁡=100\Delta_{\max}/\Delta_{\min} = 100 for the bounded map. On the bounded arm every certified floor is below 1.1×10−31.1 \times10^{-3} and every certified ceiling is above 0.095. Since the bounded map is increasing and gives Δ=10−2\Delta= 10^{-2} at z=0z = 0, every box ZZ contains 0. Remark 5 then bounds the ratio of the two time-warp terms in each cell by 25.8, a cap that depends only on Δmax⁡/Δmin⁡\Delta_{\max}/\Delta_{\min}. Over the cells of the trained model the ratio lies between 24.1 and 25.8.

The ratio for the block is slightly below the ratio for a single cell. The two certificates differ only in the time-warp term, and the terms they share make up 0.11 to 0.13 of ours at its largest output coordinate.

With a floor taken from an attack on the block’s input, the baseline on the bounded arm is still 8.5 to 12 times our certificate (Appendix § 10). This floor, which we call the empirical floor, is the smallest timescale that the attack reaches in a channel. Any valid floor of a channel is at most its empirical floor, and a larger floor lowers the baseline. The empirical floor is therefore at least as favourable to the baseline as any valid floor.

On the softplus arm the certified floors are too small to matter at the lengths we test. The smallest are 1.3×10−211.3 \times10^{-21} on block 0 and 1.9×10−171.9 \times10^{-17} on block 1. Up to L=5000L = 5000 the baseline with these floors stays within 1 per cent of the baseline without one. Its limit as L→∞L \to\infty is 2.1×10192.1 \times10^{19} times our certificate on block 0 and 2.8×10152.8 \times10^{15} times on block 1.

Comparison with gradient attacks

We attack each block on four held-out windows at each of the lengths L=10L = 10, 100, 1000 and 5000. The windows of all four lengths end at the same rows, and the two longest are 10 and 50 times as long as the training windows. From L=10L = 10 to L=5000L = 5000 at δ=10−6\delta= 10^{-6}, the attack slope grows by a factor of 1.4 to 7.3, while the baseline without a floor grows by a factor of at least 445. The largest change found lies at the last position in only 1 of the 16 combinations of arm, block and length. At the last position alone, the attack slope grows by a factor of 1.06 to 2.3. The growth therefore comes partly from searching more positions and partly from the longer history before a single position.

No attack value exceeds a certificate, and our data-free certificate lies far above what the attacks find. At δ=10−6\delta= 10^{-6} it is 3.06×1053.06 \times10^{5} to 3.23×1063.23 \times10^{6} times the attack slope. We also compare the certified bound after each stage of the block with the largest change that random perturbations of the block input produce there. Random perturbations do not search for the worst case, so each such ratio is an upper bound on the looseness after its stage. On block 0 of the bounded arm the ratio is 8.2 after the scan input uu, 134 after BB, 537 after the equilibrium cc, 1.54×1041.54 \times10^{4} after the state hh, 3.24×1053.24 \times10^{5} after the readout yy and 4.65×1074.65 \times10^{7} at the output. At the output this ratio exceeds the ratio to the attack slope.

The factor of a stage is the ratio after it divided by the ratio before it. In every block the gate and the output projection together have the largest factor, 98 to 147. The scan follows with 18 to 39, and then the readout with 14 to 29. The step that forms the equilibrium from BtB_t and utu_t has a factor of only 4 to 7. Each factor mixes the slack of its stage with how far the random samples fall short of that stage’s worst case.

Most of this slack comes from taking the worst case over the certified boxes. Around the realised input of each window, we also propagate the perturbation through every stage of the block. In the scan we apply the triangle inequality to the error identity (4) with the two runs exchanged. The realised distance ∣ht−1−ct∣\lvert h_{t-1} - c_t\rvert then takes the place of the bound 2M2M that the proof uses, and we take the result at the worst transition that the perturbed run can have. That transition lies in the intersection of the intervals given by the log-timescale bound and by the additive bound on ∣Δt−Δ~t∣\lvert\Delta_t - \widetilde{\Delta}_t\rvert. For δ≤10−4\delta\le10^{-4} this local bound is at most 12.7 times the attack slope, and dropping the log-timescale interval from the intersection changes it by at most 0.034 per cent. The local bound and its code are in our repository.

To first order in the perturbation, the choice between the log and the additive metric matters only for the data-free certificate. A bound on the change of the timescale pre-activation ztz_t bounds the change of Δt\Delta_t through the derivative Δ′\Delta' and that of log⁡Δt\log\Delta_t through (log⁡Δ)′(\log\Delta)'. Since Δ′=Δ(log⁡Δ)′\Delta' = \Delta(\log\Delta)', the first bound is Δt\Delta_t times the second to first order. Along a given input the realised timescale Δt\Delta_t is known at every step, so either bound gives the other and the two agree to first order. The data-free certificate has no realised timescale to use, and there the two metrics give the different certificates of Section 5.3.

Related work

Bounds on selective scans. Stability analyses of selective scans differ in how they treat the input-dependent timescale. Halloran, Gulati and Roysdon [13] show that the maximal Lyapunov exponent of the Mamba recurrence is nonpositive when Δt\Delta_t, BtB_t and CtC_t are held at their nominal values. Zubić and Scaramuzza [31] work in continuous time under a common quadratic Lyapunov function. Cao et al. [6] keep the timescale input-dependent and treat one layer without normalisation, for inputs in a known bounded set. Section 3.3 compares their bound with ours.

Analyses of training and generalisation for selective state space models also rely on a floor. Piaszczyny et al. [24] bound parameter gradients uniformly in the length under a floor on the softplus timescale. The robustness bound of Tang et al. [28 Theorem 1] takes its decay factor from a floor on the timescale.

Incremental stability of gated recurrences. A factor that plays the role of a floor also appears in incremental stability results for gated recurrent networks. The incremental stability conditions of Bonassi et al. [2] for networks of gated recurrent units contain the inverse of one minus an upper bound on the retention gate. The incremental gains of De Carli et al. [7, 8] contain the inverse of one minus a contraction bound, which includes an upper bound on the forget gate.

Orvieto et al. [23 Section 4] observe that the factor Δ\Delta in the input gain cancels the growth of order 1/Δ1/\Delta that the transition alone would produce. Zucchet and Orvieto [32] use logarithmic timescale coordinates to control the parameter sensitivity of diagonal linear recurrences. Their coordinates describe the fixed eigenvalues of the recurrence, whereas the timescale of our cell changes at every step with the input.

The nearest argument we know is that of Matsubara et al. [21 Lemma A.5]. They bound the error between two exponential smoothers through a recursion like (4), with a constant proportional to the relative difference of their smoothing parameters. Each cell is such a smoother on the irregular grid of accumulated times [9 Section 4.1], but its input weight varies from step to step. Lemma 2 bounds the difference of the two transitions at each step by a multiple of that step’s input weight, which keeps these differences from accumulating.

Verification of recurrent models. Verifiers for recurrent networks either propagate an input set through a fixed number of steps or search for an invariant. Propagation certifies one length at a time, whether it unrolls the network into a feed-forward one or proceeds step by step with star sets [30]. A search for an inductive invariant may iterate the transition up to a depth limit [15] or check candidates with Marabou [16, 17]. It succeeds only if it finds an invariant. For state space models, Thapliyal [29] propagates intervals exactly through a linear time-invariant classifier and observes that the analysis does not carry over to Mamba.

Brucker and Stell [4] give Isabelle/HOL proofs for feed-forward classifiers, and George et al. [10] give Lean proofs for unrolled recurrent computation. Our Isabelle theory is narrower than both.

Limitations

Theorem 3 concerns one scalar cell, so it requires a diagonal AA. It also needs a real transition Aˉ\bar{A} in (0,1)(0,1), which makes each step a convex combination. The theorem therefore does not cover the complex-valued state update of Mamba-3 [19 Section 3.2], whose transition is not a real number in (0,1)(0,1).

We do not prove that the largest gap from a common start equals η+2Mtanh⁡(ρ/2)\eta+ 2M \tanh(\rho/2). The search of § 5.2 gives only lower bounds on this gap. For η>0\eta> 0, the construction of Proposition 4 puts the nominal equilibrium outside [−M,M][-M,M], at ct=M+ηc_t = M + \eta on half of the steps. When the equilibria of both runs are restricted to [−M,M][-M,M], so are both states, and the gap is at most 2M2M. Under this restriction no gap can approach η+2Mtanh⁡(ρ/2)\eta+ 2M \tanh(\rho/2) once η>4M/(eρ+1)\eta> 4M/(e^\rho+ 1), because this value then exceeds 2M2M. With c~t\tilde{c}_t alternating between M−ηM-\eta and −M-M instead, the construction satisfies the restriction and gives gaps that approach η+(2M−η)tanh⁡(ρ/2)\eta+ (2M-\eta)\tanh(\rho/2). We do not know the largest gap under this restriction.

Our certificates bound one block at a time, with respect to its normalised input. We do not compose them through the residual stream and the head. A certificate with respect to the raw input of a block would also have to pass through its layer normalisation (Lemma 9). This normalisation is globally Lipschitz because its variance offset ε\varepsilon is positive. In the sup-norm of our certificates, its Jacobian on the blocks of the bounded arm has norm at most 3.9 at realised inputs but about 653 at a constant input. Because a certificate holds for every input, one with respect to the raw input would carry the global Lipschitz constant, which is at least the norm of about 653 at a constant input. We do not compute one.

Theorem 3 and its corollaries hold in real arithmetic, and we evaluate our certificates without outward rounding (§ 5.1). A scan computed in float32 adds a rounding error at every step. In a cell this error decays at the rate Aˉ\bar{A}, so it accumulates to at most about 1/(1−sup⁡Aˉ)1/(1-\sup\bar{A}) times the error added per step. On the bounded arm this factor is about 1100. One run there accumulates an error of at most about 2×10−4M2 \times10^{-4}M, where MM is the invariant radius of the cell. At δ=10−6\delta= 10^{-6}, the errors of the two runs together exceed the certified change of the state in 49 and 61 per cent of the cells of the two blocks. A float32 scan is therefore covered only once a rounding term is added to the bound.

A transition can also round to exactly 1. In bfloat16 this happens to the slowest transitions of the bounded arm. In float32 it happens to the slowest transitions of the softplus arm, where λΔ\lambda\Delta reaches 10−2110^{-21} on the certified boxes. Where this happens, the computed update is not a convex combination, and the theorem does not apply to the scan as computed.

Our order book data come from one instrument on one trading day, and our attacks are heuristic searches on four windows. The two arms come from one training set-up with one learning rate. They use the exact zero-order hold rather than the exponential-Euler step of the released implementations. We do not certify a released language model here.

Conclusion

We have shown that measuring timescale perturbations in log⁡Δ\log\Delta rather than in Δ\Delta gives a sensitivity bound for a selective-scan cell that holds at every sequence length and needs no floor on the timescale (Theorem 3). From a common start, its constant is within a factor eρ+1e^\rho+ 1 of the best possible (Proposition 4).

How much this helps in practice depends on the timescale map. With the bounded map, the additive baseline is tighter than our certificate only over the first 8 steps or fewer, and once it uses its certified floor, the chosen timescale range caps its ratio to ours in each cell (Remark 5). With Mamba’s softplus map, the certified floor is the only thing that limits the baseline, and on our model it is too small to matter at any length we test. Along a single input, the log and additive ways of measuring the change agree to first order, so the gain shows up only in bounds that must hold for every input of a block.

This paper does not settle whether η+2Mtanh⁡(ρ/2)\eta+ 2M \tanh(\rho/2) is the largest possible gap from a common start, although our search finds none larger, nor what the largest gap is when the equilibria of both runs lie in [−M,M][-M, M] (§ 7). Natural next steps are to extend the theorem to the complex-valued transitions of Mamba-3 and to tighten the block certificate, which remains far above what attacks find.

Use of generative AI

Claude (Anthropic), a large language model, assisted the authors throughout this project. It assisted with searching and summarising the related literature, drafting and checking proofs, writing and debugging code, running experiments, and drafting and revising the text. It was also used to review the paper, including tracing every reported number to the code and the recorded runs in our repository. The authors reviewed all AI-assisted material and take full responsibility for the content of this paper.

The lower bound, the corollaries and their proofs

Proof of the lower bound

Proof of Proposition 4. Put Q=eρ>1Q=e^{\rho}>1, fix τ>0\tau>0 and put a=e−Qτa=e^{-Q\tau} and b=e−τb=e^{-\tau}, so that 0<a<b<10<a<b<1. On the even steps we take c~t=M\widetilde{c}_t=M, λΔt=Qτ\lambda\Delta_t=Q\tau and Δt~=Δt/Q\Delta\widetilde{t}=\Delta_t/Q, and on the odd steps c~t=−M\widetilde{c}_t=-M, λΔt=τ\lambda\Delta_t=\tau and Δt~=QΔt\Delta\widetilde{t}=Q\Delta_t. On every step ct=c~t+ηc_t=\widetilde{c}_t+\eta. Then ∣log⁡Δt−log⁡Δt~∣=ρ\lvert\log\Delta_t-\log\Delta\widetilde{t}\rvert=\rho, ∣c~t∣=M\lvert\widetilde{c}_t\rvert=M and ct−c~t=ηc_t-\widetilde{c}_t=\eta for every tt. The nominal transitions are a,b,a,b,…a,b,a,b,\ldots and the perturbed ones b,a,b,a,…b,a,b,a,\ldots, starting at t=0t=0.

Let h^\widehat{h} be the run on the nominal timescales with the equilibria of the perturbed run, h^t=A‾th^t−1+(1−A‾t)c~t\widehat{h}_t=\overline{A}_t\widehat{h}_{t-1}+(1-\overline{A}_t)\widetilde{c}_t with h^−1=0\widehat{h}_{-1}=0, and split ht−h~t=(ht−h^t)+(h^t−h~t)h_t-\widetilde{h}_t=(h_t-\widehat{h}_t)+(\widehat{h}_t-\widetilde{h}_t). The first part dt=ht−h^td_t=h_t-\widehat{h}_t satisfies dt=A‾tdt−1+(1−A‾t)ηd_t=\overline{A}_td_{t-1}+(1-\overline{A}_t)\eta with d−1=0d_{-1}=0, so η−dt=A‾t(η−dt−1)\eta-d_t=\overline{A}_t(\eta-d_{t-1}) and dt=η(1−Pt)d_t=\eta(1-P_t), with PtP_t as in Theorem 3. Since Pt≤bt+1P_t\le b^{t+1}, dtd_t tends to η\eta.

For the second part we put Xk=h^2kX_k=\widehat{h}_{2k} and X~k=h~2k\widetilde{X}_k=\widetilde{h}_{2k} for k≥0k\ge0, so that X0=(1−a)MX_0=(1-a)M and X~0=(1−b)M\widetilde{X}_0=(1-b)M. Step 2k−12k-1 has c~=−M\widetilde{c}=-M and step 2k2k has c~=M\widetilde{c}=M, so for k≥1k\ge1

Xk=a(bXk−1−(1−b)M)+(1−a)M=abXk−1+M(1−2a+ab),X_k=a\bigl(bX_{k-1}-(1-b)M\bigr)+(1-a)M=abX_{k-1}+M(1-2a+ab),

and in the same way, with the two transitions exchanged, X~k=b(aX~k−1−(1−a)M)+(1−b)M=abX~k−1+M(1−2b+ab)\widetilde{X}_k=b\bigl(a\widetilde{X}_{k-1}-(1-a)M\bigr)+(1-b)M=ab\widetilde{X}_{k-1}+M(1-2b+ab). Since 0<ab<10<ab<1, both sequences converge to the fixed points of these affine maps, and

lim⁡k→∞(h^2k−h~2k)=M(1−2a+ab)−M(1−2b+ab)1−ab=2Me−τ−e−Qτ1−e−(Q+1)τ.\lim_{k\to\infty}(\widehat{h}_{2k}-\widetilde{h}_{2k})=\frac{M(1-2a+ab)-M(1-2b+ab)}{1-ab}=2M\frac{e^{-\tau}-e^{-Q\tau}}{1-e^{-(Q+1)\tau}}.

As τ→0\tau\to0 the numerator of the last fraction is (Q−1)τ+O(τ2)(Q-1)\tau+O(\tau^2) and its denominator (Q+1)τ+O(τ2)(Q+1)\tau+O(\tau^2), so the limit tends to 2M(Q−1)/(Q+1)=2Mtanh⁡(ρ/2)2M(Q-1)/(Q+1)=2M\tanh(\rho/2).

Given ϵ>0\epsilon>0, we choose τ\tau such that 2M(b−a)/(1−ab)>2Mtanh⁡(ρ/2)−ϵ/22M(b-a)/(1-ab)>2M\tanh(\rho/2)-\epsilon/2. Since h2k−h~2k=d2k+(Xk−X~k)h_{2k}-\widetilde{h}_{2k}=d_{2k}+(X_k-\widetilde{X}_k) tends to η+2M(b−a)/(1−ab)\eta+2M(b-a)/(1-ab) as k→∞k\to\infty, some kk gives h2k−h~2k>η+2M(b−a)/(1−ab)−ϵ/2h_{2k}-\widetilde{h}_{2k}>\eta+2M(b-a)/(1-ab)-\epsilon/2. At t=2kt=2k we then have ∣ht−h~t∣>η+2Mtanh⁡(ρ/2)−ϵ\lvert h_t-\widetilde{h}_t\rvert>\eta+2M\tanh(\rho/2)-\epsilon.

The log-timescale slopes

Lemma 6 (Log-timescale slopes). (i) For Δ=softplus⁡(z)\Delta=\operatorname{softplus}(z), the slope ddzlog⁡Δ\frac{d}{dz}\log\Delta, which equals sigmoid⁡(z)/softplus⁡(z)\operatorname{sigmoid}(z)/\operatorname{softplus}(z), lies in (0,1)(0,1), is strictly decreasing in zz and tends to 11 as z→−∞z\to-\infty. Hence ∣log⁡Δ(z)−log⁡Δ(z~)∣≤∣z−z~∣\lvert\log\Delta(z)-\log\Delta(\widetilde{z})\rvert\le\lvert z-\widetilde{z}\rvert, and for z,z~≥z0z,\widetilde{z}\ge z_0 the constant 11 may be replaced by sigmoid⁡(z0)/softplus⁡(z0)\operatorname{sigmoid}(z_0)/\operatorname{softplus}(z_0). (ii) For Δ=Δmin⁡(Δmax⁡/Δmin⁡)sigmoid⁡(z)\Delta=\Delta_{\min}(\Delta_{\max}/\Delta_{\min})^{\operatorname{sigmoid}(z)}, the slope ddzlog⁡Δ=sigmoid⁡′(z)log⁡(Δmax⁡/Δmin⁡)\frac{d}{dz}\log\Delta=\operatorname{sigmoid}'(z)\log(\Delta_{\max}/\Delta_{\min}) lies in (0,14log⁡(Δmax⁡/Δmin⁡)](0,\frac14\log(\Delta_{\max}/\Delta_{\min})], with the maximum at z=0z=0.

Proof. For (i) we put y=ezy=e^z, for which sigmoid⁡(z)=y/(1+y)\operatorname{sigmoid}(z)=y/(1+y) and softplus⁡(z)=log⁡(1+y)\operatorname{softplus}(z)=\log(1+y). Since log⁡(1+y)−y/(1+y)\log(1+y)-y/(1+y) vanishes at y=0y=0 and has derivative y/(1+y)2>0y/(1+y)^2>0, the ratio lies in (0,1)(0,1). Its derivative has the sign of (1−sigmoid⁡)softplus⁡−sigmoid⁡=(log⁡(1+y)−y)/(1+y)<0(1-\operatorname{sigmoid})\operatorname{softplus}-\operatorname{sigmoid}=(\log(1+y)-y)/(1+y)<0, and as z→−∞z\to-\infty both functions equal ez(1+o(1))e^z(1+o(1)).

For (ii), log⁡Δ=log⁡Δmin⁡+sigmoid⁡(z)log⁡(Δmax⁡/Δmin⁡)\log\Delta=\log\Delta_{\min}+\operatorname{sigmoid}(z)\log(\Delta_{\max}/\Delta_{\min}) and sigmoid⁡′=sigmoid⁡(1−sigmoid⁡)\operatorname{sigmoid}'=\operatorname{sigmoid}(1-\operatorname{sigmoid}), which lies in (0,1/4](0,1/4].

Our float32 implementation clamps Δ\Delta to [Δmin⁡,Δmax⁡][\Delta_{\min},\Delta_{\max}] on the bounded arm. In the reals the clamp is inactive, and it never increases ∣log⁡Δ−log⁡Δ~∣|\log\Delta-\log\widetilde{\Delta}|.

The exponential-Euler cell

With xt=λΔtx_t=\lambda\Delta_t, bt=Btut/λb_t=B_tu_t/\lambda and Ψ\Psi as in § 3.4 we have xt=(1−Aˉt)Ψ(xt)x_t=(1-\bar{A}_t)\Psi(x_t), and the exponential-Euler update ht=Aˉtht−1+ΔtBtuth_t=\bar{A}_th_{t-1}+\Delta_tB_tu_t is the convex combination

ht=Aˉtht−1+(1−Aˉt)ctee,ctee=Ψ(xt)bt.h_t=\bar{A}_th_{t-1}+(1-\bar{A}_t)c_t^{\mathrm{ee}},\qquad c_t^{\mathrm{ee}}=\Psi(x_t)b_t.

Corollary 7 (Exponential-Euler cell). Suppose that Δt,Δ~t∈(0,Δhi]\Delta_t,\widetilde{\Delta}_t\in(0,\Delta_{\mathrm{hi}}], ∣log⁡Δt−log⁡Δ~t∣≤ρ|\log\Delta_t-\log\widetilde{\Delta}_t|\le\rho, ∣b~t∣≤M|\widetilde{b}_t|\le M and ∣bt−b~t∣≤ηb|b_t-\widetilde{b}_t|\le\eta^b for every tt, and that h−1=h~−1=0h_{-1}=\widetilde{h}_{-1}=0. With G=Ψ(λΔhi)G=\Psi(\lambda\Delta_{\mathrm{hi}}), the exponential-Euler cell satisfies ∣ht−h~t∣≤Gηb+3GM(eρ−1)|h_t-\widetilde{h}_t|\le G\eta^b+3GM(e^\rho-1) for every t≥0t\ge0.

Proof. Since log⁡Ψ(x)=log⁡x−log⁡(1−e−x)\log\Psi(x)=\log x-\log(1-e^{-x}), we have dlog⁡Ψ(x)/dlog⁡x=1−x/(ex−1)d\log\Psi(x)/d\log x=1-x/(e^x-1), which lies in (0,1)(0,1) for x>0x>0. Hence Ψ\Psi is increasing and ∣log⁡Ψ(x)−log⁡Ψ(x~)∣≤∣log⁡x−log⁡x~∣|\log\Psi(x)-\log\Psi(\widetilde{x})|\le|\log x-\log\widetilde{x}|. With xt,x~t≤λΔhix_t,\widetilde{x}_t\le\lambda\Delta_{\mathrm{hi}} this gives ∣c~tee∣=Ψ(x~t)∣b~t∣≤GM|\widetilde{c}^{\mathrm{ee}}_t|=\Psi(\widetilde{x}_t)|\widetilde{b}_t|\le GM and Ψ(xt)≤G\Psi(x_t)\le G, and Theorem 3 applies with GMGM in place of MM.

The same bound gives ∣Ψ(xt)−Ψ(x~t)∣≤(eρ−1)Ψ(xt)|\Psi(x_t)-\Psi(\widetilde{x}_t)|\le(e^\rho-1)\Psi(x_t), hence ∣ctee−c~tee∣≤Ψ(xt)∣bt−b~t∣+∣Ψ(xt)−Ψ(x~t)∣∣b~t∣≤Gηb+(eρ−1)GM|c^{\mathrm{ee}}_t-\widetilde{c}^{\mathrm{ee}}_t|\le\Psi(x_t)|b_t-\widetilde{b}_t|+|\Psi(x_t)-\Psi(\widetilde{x}_t)||\widetilde{b}_t|\le G\eta^b+(e^\rho-1)GM. Adding the time-warp term 2GM(eρ−1)2GM(e^\rho-1) gives the claim.

The block and the data-free block certificate

We certify a block that computes

ξ^=LN⁡(ξ),[u0,g]=in_proj⁡(ξ^),u=SiLU⁡(DepthwiseCausalConv1d⁡(u0)),[p,B,C]=x_proj⁡(u),Δ=Δmin⁡(Δmax⁡Δmin⁡)sigmoid⁡(dt_proj⁡(p)),A=−exp⁡(Alog⁡),Aˉt=exp⁡(ΔtA),Bˉt=A−1(exp⁡(ΔtA)−I)Bt,ht=Aˉtht−1+Bˉtut,h−1=0,yt=∑nCt,nht,:,n+Dut,out⁡=out_proj⁡(y⊙SiLU⁡(g))+ξ.\begin{aligned} \widehat{\xi}&=\operatorname{LN}(\xi),\qquad[u_0,g]=\operatorname{in\_proj}(\widehat{\xi}),\qquad u=\operatorname{SiLU}(\operatorname{DepthwiseCausalConv1d}(u_0)),\\ [p,B,C]&=\operatorname{x\_proj}(u),\qquad\Delta=\Delta_{\min}\left(\frac{\Delta_{\max}}{\Delta_{\min}}\right)^{\operatorname{sigmoid}(\operatorname{dt\_proj}(p))},\\ A&=-\exp(A_{\log}),\qquad\bar{A}_t=\exp(\Delta_tA),\qquad\bar{B}_t=A^{-1}\bigl(\exp(\Delta_tA)-I\bigr)B_t,\\ h_t&=\bar{A}_th_{t-1}+\bar{B}_tu_t,\qquad h_{-1}=0,\qquad y_t=\sum_n C_{t,n}h_{t,:,n}+Du_t,\\ \operatorname{out}&=\operatorname{out\_proj}(y\odot\operatorname{SiLU}(g))+\xi. \end{aligned}

with all products elementwise and broadcast over the state axis; on the softplus arm Δ=softplus⁡(dt_proj⁡(p))\Delta=\operatorname{softplus}(\operatorname{dt\_proj}(p)). Both A=−exp⁡(Alog⁡)<0A=-\exp(A_{\log})<0 and Δ>0\Delta>0 hold by construction, for every checkpoint of the architecture.

With LN the layer normalisation of Lemma 9, of width dd, gain γ\gamma and offset β\beta, we write Z={β+γ⊙r:1Tr=0, ∥r∥2≤d}\mathcal{Z}=\{\beta+\gamma\odot r:\mathbf{1}^{\mathsf{T}}r=0,\ \|r\|_2\le\sqrt{d}\} for the set on which the boxes of § 2 are certified, which contains LN⁡(ξ)\operatorname{LN}(\xi) for every ξ\xi, and ∣W∣|W| for the entrywise absolute value of a matrix WW. We write u^i\widehat{u}_i, B^n\widehat{B}_n, C^n\widehat{C}_n and s^i\widehat{s}_i for the suprema of ∣ui∣|u_i|, ∣Bn∣|B_n|, ∣Cn∣|C_n| and ∣SiLU⁡(gi)∣|\operatorname{SiLU}(g_i)| over the certified boxes, Mi,nM_{i,n} for the invariant radius of cell (i,n)(i,n), and y^i=∑nC^nMi,n+∣Di∣u^i\widehat{y}_i=\sum_n\widehat{C}_nM_{i,n}+|D_i|\widehat{u}_i. We write sius_i^u and sigs_i^g for the suprema of ∣SiLU⁡′∣|\operatorname{SiLU}'| over the boxes of the convolution output and the gate pre-activation of channel ii, and ℓi\ell_i for the supremum of the log⁡Δi/dzi\log\Delta_i/dz_i over the box of ziz_i (Lemma 6). Each Λ\Lambda below is a sup-norm Lipschitz constant of the named stage over its certified box (Λiu\Lambda_i^u, for example, bounds the change of uiu_i per unit change of the normalised input).

Corollary 8 (Block). Let ξ\xi and ξ~\tilde{\xi} be two input sequences of the block, both runs starting from h−1=0h_{-1}=0, and let ∥LN⁡(ξt)−LN⁡(ξ~t)∥∞≤δ\|\operatorname{LN}(\xi_t)-\operatorname{LN}(\tilde{\xi}_t)\|_{\infty}\le\delta for every tt. Let wi,⋅w_{i,\cdot} be the convolution kernel of channel ii, let WBW^B, WCW^C and WpW^p be the blocks of x-proj that produce BB, CC and pp, let WinW^{\mathrm{in}}, WdtW^{\mathrm{dt}} and WoutW^{\mathrm{out}} be the weights of in-proj, dt-proj and out-proj, and define

Λiu=siu∥wi,⋅∥1∑j∣Wijin∣,Λig=∑j∣Wdinner+i,jin∣,ΛB=∣WB∣Λu,ΛC=∣WC∣Λu,Λz=∣WdtWp∣Λu,ρi=ℓiΛizδ,ηi,n=(B^nΛiu+u^iΛnB)δ/λi,n,Ei,n=ηi,n+2Mi,n(eρi−1),Yi=∑n(C^nEi,n+Mi,nΛCδ)+∣Di∣Λiuδ,Oi=y^isigΛigδ+s^iYi,Γk=∑i∣Wkiout∣Oi.\begin{aligned} \Lambda_i^u &= s_i^u\|w_{i,\cdot}\|_1\sum_j |W_{ij}^{\mathrm{in}}|, & \Lambda_i^g &= \sum_j |W_{d_{\mathrm{inner}}+i,j}^{\mathrm{in}}|,\\ \Lambda^B &= |W^B|\Lambda^u, & \Lambda^C &= |W^C|\Lambda^u,\\ \Lambda^z &= |W^{\mathrm{dt}}W^p|\Lambda^u, & \rho_i &= \ell_i\Lambda_i^z\delta,\\ \eta_{i,n} &= (\widehat{B}_n\Lambda_i^u+\widehat{u}_i\Lambda_n^B)\delta/\lambda_{i,n}, & E_{i,n} &= \eta_{i,n}+2M_{i,n}(e^{\rho_i}-1),\\ Y_i &= \sum_n(\widehat{C}_nE_{i,n}+M_{i,n}\Lambda^C\delta)+|D_i|\Lambda_i^u\delta, & O_i &= \widehat{y}_i s_i^g\Lambda_i^g\delta+\widehat{s}_iY_i,\\ \Gamma_k &= \sum_i|W_{ki}^{\mathrm{out}}|O_i. \end{aligned}

Then for every t≥0t\ge0, channel ii, state nn and output coordinate kk, ∣log⁡Δt,i−log⁡Δ~t,i∣≤ρi|\log\Delta_{t,i}-\log\widetilde{\Delta}_{t,i}|\le\rho_i, ∣ct,i,n−c~t,i,n∣≤ηi,n|c_{t,i,n}-\widetilde{c}_{t,i,n}|\le\eta_{i,n}, ∣ht,i,n−h~t,i,n∣≤Ei,n|h_{t,i,n}-\widetilde{h}_{t,i,n}|\le E_{i,n}, ∣yt,i−y~t,i∣≤Yi|y_{t,i}-\widetilde{y}_{t,i}|\le Y_i and ∣out⁡t,k−out⁡~t,k∣≤Γk+∣ξt,k−ξ~t,k∣|\operatorname{out}_{t,k}-\widetilde{\operatorname{out}}_{t,k}|\le\Gamma_k+|\xi_{t,k}-\widetilde{\xi}_{t,k}|.

Proof. We follow the perturbation through the stages in order. Since both normalised sequences lie in Z\mathcal{Z}, every intermediate quantity of both runs, and every segment between corresponding values, then lies in its certified box. Through the input projection, ui0u_i^0 moves by at most ∑j∣Wijin∣δ\sum_j|W_{ij}^{\mathrm{in}}|\delta and gig_i by at most Λigδ\Lambda_i^g\delta. With zero left padding in both runs, the convolution multiplies that bound by ∥wi,⋅∥1\|w_{i,\cdot}\|_1, and the mean value theorem for SiLU gives Λiuδ\Lambda_i^u\delta. Linearity of x-proj and dt-proj gives ΛBδ\Lambda^B\delta, ΛCδ\Lambda^C\delta and Λizδ\Lambda_i^z\delta, and Lemma 6 gives ρi\rho_i. Writing Bnui−B~nu~i=Bn(ui−u~i)+u~i(Bn−B~n)B_nu_i-\widetilde{B}_n\widetilde{u}_i=B_n(u_i-\widetilde{u}_i)+\widetilde{u}_i(B_n-\widetilde{B}_n) gives ηi,n\eta_{i,n}, and Theorem 3 gives Ei,nE_{i,n}. The same product rule applied to Cnhi,nC_nh_{i,n} with ∣h~i,n∣≤Mi,n|\widetilde{h}_{i,n}|\le M_{i,n} gives YiY_i. The gate, the output projection and the residual then give OiO_i, Γk\Gamma_k and the term ∣ξt,k−ξ~t,k∣|\xi_{t,k}-\widetilde{\xi}_{t,k}|.

As a function of δ\delta, Γk/δ\Gamma_k/\delta increases through eρi−1e^{\rho_i}-1 and tends, as δ→0\delta\to0, to the value with ρi\rho_i in place of eρi−1e^{\rho_i}-1. Perturbations that leave Z\mathcal{Z} need the boxes widened to Z+[−δ,δ]d\mathcal{Z}+[-\delta,\delta]^d (§ 5.1 gives the effect on our model).

Relation to Cao et al. in detail

A derivation from their lemmas. In the setting of Cao et al. [6], AA is Hurwitz with ∥eσA∥≤MAe−ασ\|e^{\sigma A}\|\le M_Ae^{-\alpha\sigma}, one timescale per step is shared by the whole state, and by their Lemma 3.1 the scan samples a continuous flow at the grid of accumulated times. We follow their argument for one cell of (2), whose flow is h˙=−λh+w\dot{h}=-\lambda h+w with the forcing ws=λcsw_s=\lambda c_s on step ss. We take both runs from the same state with ∣h~−1∣≤M|\widetilde{h}_{-1}|\le M. With τ−1=τ~−1=0\tau_{-1}=\widetilde{\tau}_{-1}=0, τt=∑s≤tΔs\tau_t=\sum_{s\le t}\Delta_s and τ~t=∑s≤tΔ~s\widetilde{\tau}_t=\sum_{s\le t}\widetilde{\Delta}_s we have ht=h(τt)h_t=h(\tau_t) and h~t=h~(τ~t)\widetilde{h}_t=\widetilde{h}(\widetilde{\tau}_t), where hh and h~\widetilde{h} solve h˙=−λh+w\dot{h}=-\lambda h+w and h~˙=−λh~+w~\dot{\widetilde{h}}=-\lambda\widetilde{h}+\widetilde{w} from the common start, with forcing wsw_s on [τs−1,τs)[\tau_{s-1},\tau_s) and w~s=λc~s\widetilde{w}_s=\lambda\widetilde{c}_s on [τ~s−1,τ~s)[\widetilde{\tau}_{s-1},\widetilde{\tau}_s).

We map each interval [τ~s−1,τ~s)[\widetilde{\tau}_{s-1},\widetilde{\tau}_s) linearly onto [τs−1,τs)[\tau_{s-1},\tau_s) and let g^(σ)\widehat{g}(\sigma) be the value of h~\widetilde{h} at the corresponding time, which gives g^(τt)=h~t\widehat{g}(\tau_t)=\widetilde{h}_t and, on [τs−1,τs)[\tau_{s-1},\tau_s),

g^˙=rs(w~s−λg^)=−λg^+w~s+(rs−1)(w~s−λg^),rs=Δ~sΔs.\dot{\widehat{g}}=r_s(\widetilde{w}_s-\lambda\widehat{g})=-\lambda\widehat{g}+\widetilde{w}_s+(r_s-1)(\widetilde{w}_s-\lambda\widehat{g}),\qquad r_s=\frac{\widetilde{\Delta}_s}{\Delta_s}.

With the two runs now on one grid, their forcings differ by w‾s=ws−w~s=(rs−1)(w~s−λg^)\overline{w}_s=w_s-\widetilde{w}_s=(r_s-1)(\widetilde{w}_s-\lambda\widehat{g}). Since ∣ws−w~s∣≤λη|w_s-\widetilde{w}_s|\le\lambda\eta, ∣rs−1∣≤eρ−1|r_s-1|\le e^{\rho}-1, ∣w~s∣≤λM|\widetilde{w}_s|\le\lambda M and ∣g^∣≤M|\widehat{g}|\le M, we have ∣w‾s∣≤λη(eρ−1)|\overline{w}_s|\le\lambda\eta(e^\rho-1). The bound ∣g^∣≤M|\widehat{g}|\le M holds because between grid points h^\widehat{h} is a convex combination of h~s−1\widetilde{h}_{s-1} and c~s\widetilde{c}_s, with ∣h~s−1∣≤M|\widetilde{h}_{s-1}|\le M by Proposition 1. The mismatch depends on g^\widehat{g} itself, but that is admissible because g^\widehat{g} takes the values of h~\widetilde{h}.

By variation of constants (the argument of their Lemma B.2, which applies to any bounded measurable forcing), two solutions of h˙=−λh+w\dot{h}=-\lambda h+w from a common start differ at time σ\sigma by at most (1−e−λσ)/λ(1-e^{-\lambda\sigma})/\lambda times the supremum of the difference of their forcings. Since e−λτt=Pte^{-\lambda\tau_t}=P_t, this gives ∣ht−h~t∣≤(1−Pt)(η+2M(eρ−1))|h_t-\tilde{h}_t| \le(1-P_t)(\eta+2M(e^\rho-1)) for every tt, which is Theorem 3 for a common initial state.

In the proof of their Theorem 3.3, Cao et al. state that a direct bound on the forcing mismatch is unavailable because the two forcings are piecewise constant on different grids, but our change of time variable aligns the grids. Its size, the largest ∣log⁡rs∣|\log r_s|, has the form of the norm that Billingsley [1, Section 12] assigns to a time change of [0,1][0,1] in the Skorokhod topology, here for a map between intervals of different lengths.

Comparison regime by regime. Their time-warp argument assumes one timescale shared by the whole state, which holds for each channel of the block separately. Treating a diagonal AA one cell at a time is the most favourable form of their bound, since their statement for the whole state uses α=min⁡λ\alpha=\min\lambda and ∥A∥=max⁡λ\lVert A\rVert=\max\lambda. On the bounded arm, for example, the decay rates lie between 0.912 and 17.14, while a statement for the whole state replaces all of them by the two extremes. Both the additive baseline (5) and Theorem 3 pass through the readout in the same way, which lets us compare them at the level of the state.

Without a floor, the additive time-warp term grows linearly in the number of steps and ours does not, although for short sequences ours can be the larger. Let n∗n^* be the smallest n≥1n\ge1 at which the additive time-warp term without a floor after nn steps, 2MληΔn2M\lambda\eta\Delta n, is at least our finite-horizon term at its worst over admissible timescales, 2M(eρ−1)(1−e−λΔhin)2M(e^\rho-1)(1-e^{-\lambda\Delta h_i^n}), to first order in the size of the perturbation, where Δhi\Delta_{h_i} is the certified ceiling. For fewer than n∗n^* steps the additive term is the smaller. Because the finite-horizon term is at most 2M(eρ−1)2M(e^\rho-1), n∗n^* is never later than the step at which the additive term passes the flat bound of Theorem 3 in the same cell. Over the cells of the bounded arm n∗n^* ranges from 3 to 42 with median 5, while on the softplus arm it is at most 2.

With a floor and only the additive information ηΔ\eta\Delta, the admissible value ρ=log⁡(1+ηΔ/Δlo)\rho=\log(1+\eta\Delta/\Delta_{\mathrm{lo}}) turns our time-warp term into 2λMηΔ/x2\lambda M\eta\Delta/x with x=λΔlox=\lambda\Delta_{\mathrm{lo}}. This lies between the additive baseline 2λMηΔ/(ex−1)2\lambda M\eta\Delta/(e^x-1) and the form 2λMηΔ/(1−e−x)2\lambda M\eta\Delta/(1-e^{-x}) of their Remark 3.4. Under their hypotheses our term is thus no larger than theirs, and the ratios of the three terms tend to 1 as x→0x\to0. At the certified floors of our trained model, where xx is below 0.018, they differ by less than 2 per cent.

We obtain the ratio of Remark 5 from the two terms directly. If zz changes by at most ζ\zeta on the certified box ZZ of its channel, then ηΔ=ζsup⁡Z∣Δ′∣\eta_\Delta=\zeta\sup_Z|\Delta'| and ρ=ζsup⁡Z∣(log⁡Δ)′∣\rho=\zeta\sup_Z|(\log\Delta)'| are admissible. The limit 2MληΔ/(ex−1)2M\lambda\eta_\Delta/(e^x-1) of the additive time-warp term, divided by our 2M(eρ−1)=2Mρ+O(ζ2)2M(e^\rho-1)=2M\rho+O(\zeta^2), gives the ratio of the remark. Its numerator sup⁡Z∣Δ′∣\sup_Z|\Delta'| is the counterpart, restricted to the box, of the factor that multiplies ∥WΔ∥\lVert W_\Delta\rVert in the Lipschitz constant of Δ\Delta that Cao et al. assume (their Assumption 3.2 (A4)), bounded in their Proposition 4.6 and Remark 4.7 with the supremum over all zz. When ZZ excludes 0 the argument of the remark does not apply. Since Δ\Delta increases in zz, on ZZ we have ∣Δ′∣=Δ∣(log⁡Δ)′∣≤Δ(sup⁡Z)∣(log⁡Δ)′∣|\Delta'|=\Delta|(\log\Delta)'|\le\Delta(\sup Z)|(\log\Delta)'|, so with the floor Δlo=Δ(inf⁡Z)\Delta_{\mathrm{lo}}=\Delta(\inf Z) that the box itself certifies, the first factor of the ratio is at most Δ(sup⁡Z)/Δ(inf⁡Z)\Delta(\sup Z)/\Delta(\inf Z), a bound that tends to 1 as the box narrows. For the bounded map with R=100R=100 and Z=[3,4]Z=[3,4], the first factor itself equals 1, because sup⁡Z∣Δ′∣\sup_Z|\Delta'| and sup⁡Z∣(log⁡Δ)′∣\sup_Z|(\log\Delta)'| are both attained at z=3z=3 (Δ′\Delta' peaks at z≈1.57z\approx1.57). The whole ratio is then x/(ex−1)<1x/(e^x-1)<1, and the floored baseline is the tighter bound in such a cell.

At the empirical floor of § 5.3 (the smallest timescale in a channel that an attack on the block’s input reaches) the baseline is 8.5 to 12 times our bound at the output of a bounded block. Per cell, however, the ratio there ranges from 0.16 to 25, and in about a tenth of the cells an exact per-channel floor could make the additive term smaller than ours.

For the softplus arm the leading factor is about 1/Δlo1/\Delta_{\mathrm{lo}} when the floor is small. With the certified floors of the trained softplus arm the per-cell ratio, which compares the limits as L→∞L\to\infty, lies between 1.3×1041.3\times10^4 and 7.9×10207.9\times10^{20}, with medians of 9.1×10109.1\times10^{10} and 4.0×10104.0\times10^{10} on the two blocks.

Interval arithmetic and the normalisation bounds

The separated bound

A verifier handed the update without (1) bounds Aˉ\bar{A} and Bˉ\bar{B} separately and obtains the geometric bound sup⁡∣Bˉu∣/(1−sup⁡Aˉ)\sup|\bar{B}u|/(1-\sup\bar{A}). In the true update the input term and the contraction cancel, because the same factor 1−Aˉt1-\bar{A}_t multiplies the one and defines the other. A separated bound instead takes the input term at the ceiling Δhi\Delta_{\mathrm{hi}} of the timescale box and the contraction at its floor Δlo\Delta_{\mathrm{lo}}, which gives the per-cell ratio to the invariant

gap⁡(λ)=1−e−λΔhi1−e−λΔlo.(6)\operatorname{gap}(\lambda)=\frac{1-e^{-\lambda\Delta_{\mathrm{hi}}}}{1-e^{-\lambda\Delta_{\mathrm{lo}}}}. \tag*{(6)}

This ratio is at least 1, with equality only for a one-point box. As Δlo→0\Delta_{\mathrm{lo}}\to0 it grows like (1−e−λΔhi)/(λΔlo)(1-e^{-\lambda\Delta_{\mathrm{hi}}})/(\lambda\Delta_{\mathrm{lo}}) while the invariant does not move, and since (1−e−y)/y(1-e^{-y})/y is decreasing it never exceeds Δhi/Δlo\Delta_{\mathrm{hi}}/\Delta_{\mathrm{lo}}. It decreases in λ\lambda, because y↦y/(ey−1)y\mapsto y/(e^y-1) is decreasing and the log-derivative of (6) in λ\lambda is Δhi/(eλΔhi−1)−Δlo/(eλΔlo−1)<0\Delta_{\mathrm{hi}}/(e^{\lambda\Delta_{\mathrm{hi}}}-1)-\Delta_{\mathrm{lo}}/(e^{\lambda\Delta_{\mathrm{lo}}}-1)<0.

Interval arithmetic meets the same loss when it checks the induction step of Proposition 1. Because the variable aa occurs twice in aMh+(1−a)McaM_h+(1-a)M_c, an interval evaluator given a candidate box MhM_h on the state and the bound McM_c on ctc_t returns sup⁡AˉMh+(1−inf⁡Aˉ)Mc\sup\bar{A}M_h+(1-\inf\bar{A})M_c. This check passes exactly when Mh≥(1−inf⁡Aˉ)Mc/(1−sup⁡Aˉ)M_h\ge(1-\inf\bar{A})M_c/(1-\sup\bar{A}), that is, for boxes at least the geometric fixed point. For the invariant’s own box, Mh=Mc>0M_h=M_c>0, it fails whenever the timescale box is not a single point.

With aa kept as one variable, the supremum over a∈[0,1]a\in[0,1] of aMh+(1−a)McaM_h+(1-a)M_c is max⁡(Mh,Mc)\max(M_h,M_c). The box is then preserved exactly when Mc≤MhM_c\le M_h (our implementation checks the residual max⁡(Mh,Mc)−Mh\max(M_h,M_c)-M_h).

On the bounded arm of our order book model, generic interval propagation iterated to convergence agrees with the geometric fixed point, which is 95.6 and 95.2 times the invariant radius on the two blocks.

The normalisation bounds

Lemma 9 (Coordinate bound). Let LN⁡(x)=γ⊙(x−μ)/σ2+ε+β\operatorname{LN}(x)=\gamma\odot(x-\mu)/\sqrt{\sigma^2+\varepsilon}+\beta on Rd\mathbb{R}^d, with d≥2d\ge2, ε>0\varepsilon>0 and σ2\sigma^2 the variance with divisor dd. Then ∣LN⁡(x)i−βi∣≤∣γi∣d−1|\operatorname{LN}(x)_i-\beta_i|\le|\gamma_i|\sqrt{d-1} for every xx and every ii.

Proof. With z=(x−μ)/σ2+εz=(x-\mu)/\sqrt{\sigma^2+\varepsilon} we have ∑jzj=0\sum_j z_j=0 and ∑jzj2=dσ2/(σ2+ε)<d\sum_j z_j^2=d\sigma^2/(\sigma^2+\varepsilon)<d. Cauchy–Schwarz on ∑j≠izj=−zi\sum_{j\ne i}z_j=-z_i gives ∑j≠izj2≥zi2/(d−1)\sum_{j\ne i}z_j^2\ge z_i^2/(d-1), hence d>zi2d/(d−1)d>z_i^2d/(d-1) and zi2<d−1z_i^2<d-1. □

With ε=0\varepsilon=0, Lemma 9 is the Laguerre–Samuelson inequality [18, 26], which bounds one coordinate at a time, whereas the fused bound of Proposition 10 combines the zero-mean and norm constraints with the following linear map and equals the supremum over all inputs.

Proposition 10 (Fused bound). Let WkW_k be row kk of the linear map applied to the normalised output, let vk=Wk⊙γv_k=W_k\odot\gamma with vˉk\bar{v}_k the mean of its entries, and let S={z:1Tz=0, ∥z∥2≤d}S=\{z: \mathbf{1}^{\mathsf{T}}z=0,\ \lVert z\rVert_2\le\sqrt{d}\}. Then sup⁡z∈S⟨vk,z⟩=∥vk−vˉk1∥2d\sup_{z\in S}\langle v_k,z\rangle=\lVert v_k-\bar{v}_k\mathbf{1}\rVert_2\sqrt{d}, attained, when vkv_k is not constant, at z∗=d(vk−vˉk1)/∥vk−vˉk1∥2z^*=\sqrt{d}(v_k-\bar{v}_k\mathbf{1})/\lVert v_k-\bar{v}_k\mathbf{1}\rVert_2. For ε>0\varepsilon>0 the supremum over the normalised vectors of all inputs is the same value, but when vkv_k is not constant it is not attained.

Proof. Since 1Tz=0\mathbf{1}^{\mathsf{T}}z=0, Cauchy–Schwarz over ∥z∥2≤d\lVert z\rVert_2\le\sqrt{d} applied to ⟨vk,z⟩=⟨vk−vˉk1,z⟩\langle v_k,z\rangle=\langle v_k-\bar{v}_k\mathbf{1},z\rangle gives the bound, which z∗∈Sz^*\in S attains. Every normalised vector has ∥z∥22=dσ2/(σ2+ε)<d\lVert z\rVert_2^2=d\sigma^2/(\sigma^2+\varepsilon)<d, and when vkv_k is not constant Cauchy–Schwarz then gives ⟨vk,z⟩≤∥vk−vˉk1∥2∥z∥2<∥vk−vˉk1∥2d\langle v_k,z\rangle\le\lVert v_k-\bar{v}_k\mathbf{1}\rVert_2\lVert z\rVert_2<\lVert v_k-\bar{v}_k\mathbf{1}\rVert_2\sqrt{d} for every normalised vector zz, and the value is not attained. For the input x=tz∗x=t z^* the normalised vector is tz∗/t2+εt z^*/\sqrt{t^2+\varepsilon}, which tends to z∗z^* as t→∞t\to\infty and shows that the supremum is the same value. □

Riechers [25] and Brody et al. [3] describe this geometry of layer normalisation (a projection onto the hyperplane orthogonal to 1\mathbf{1} followed by a scaling). Bounding the normalisation and the linear map separately gives ∥vk∥1d−1\lVert v_k\rVert_1\sqrt{d-1} instead, an ℓ1\ell_1 quantity where an ℓ2\ell_2 one suffices. This difference carries over directly to the radius sup⁡∣Bu∣/λ\sup|Bu|/\lambda, which is why our certificates use the fused bound.

References

  1. [2]Fabio Bonassi, Marcello Farina, and Riccardo Scattolini. On the stability properties of gated recurrent units neural networks. Systems & Control Letters, 157:105049, 2021. doi: 10.1016/j.sysconle.2021.105049.DOI
  2. [3]Shaked Brody, Uri Alon, and Eran Yahav. On the expressivity role of LayerNorm in transformers’ attention. In Findings of the Association for Computational Linguistics: ACL 2023, pages 14211–14221, Toronto, Canada, 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.findings-acl.895. arXiv:2305.02582.
  3. [4]Achim D. Brucker and Amy Stell. Verifying feedforward neural networks for classification in Isabelle/HOL. In Formal Methods: 25th International Symposium, FM 2023, volume 14000 of Lecture Notes in Computer Science, pages 427–444. Springer, 2023. doi: 10.1007/978-3-031-27481-7_24.DOI
  4. [5]Francesco Bullo. Contraction Theory for Dynamical Systems. Kindle Direct Publishing, 2026. ISBN 979-8836646806. URL https://fbullo.github.io/ctds. Edition 1.3.
  5. [6]Liang Cao, Weide Liu, Zhuo Chen, and Yan Qin. A control-theoretic view of Mamba on stability and robustness. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pages 11350–11364. PMLR, 2026.
  6. [7]Stefano De Carli, Davide Previtali, Mirko Mazzoleni, and Fabio Previdi. Chaos-free networks are stable recurrent neural networks. IEEE Control Systems Letters, 10:751–756, 2026. doi: 10.1109/LC SYS.2026.3702800. arXiv:2603.14106.arxiv.org/abs/2603.14106
  7. [8]Stefano De Carli, Davide Previtali, Mirko Mazzoleni, and Fabio Previdi. Stability properties of Minimal Gated Unit neural networks. arXiv preprint arXiv:2603.03017, 2026.arxiv.org/abs/2603.03017
  8. [9]Andreas Eckner. Algorithms for unevenly spaced time series: Moving averages and other rolling operators. Working paper, April 2019. URL http://www.eckner.com/papers/Algorithms%20for%20Unevenly%20Spaced%20Time%20Series.pdf. First version January 2010; current version 10 April 2019.
  9. [10]Robert Joseph George, Jennifer Cruden, Will Adkisson, Xiangrui Zhong, Huan Zhang, and Anima Anandkumar. TorchLean: Formalizing neural networks in Lean. arXiv preprint arXiv:2602.22631, 2026.arxiv.org/abs/2602.22631
  10. [11]Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. In First Conference on Language Modeling (COLM), 2024. arXiv:2312.00752.arxiv.org/abs/2312.00752
  11. [12]Albert Gu, Isys Johnson, Karan Goel, Khaled Saab, Tri Dao, Atri Rudra, and Christopher Ré. Combining recurrent, convolutional, and continuous-time models with linear state space layers. In Advances in Neural Information Processing Systems, volume 34, pages 572–585. Curran Associates, Inc., 2021. arXiv:2110.13985.arxiv.org/abs/2110.13985
  12. [13]John T. Halloran, Manbir Gulati, and Paul F. Roysdon. Mamba state-space models are Lyapunov-stable learners. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL https://openreview.net/forum?id=wzsYQYs3d0. arXiv:2406.00209.arxiv.org/abs/2406.00209
  13. [14]Liubomyr Horbatko. Sessa: Selective state space attention. arXiv preprint arXiv:2604.18580, 2026.arxiv.org/abs/2604.18580
  14. [15]Mehran Hosseini, Alessio Lummuscio, and Nicola Paoletti. LTL verification of memoryful neural agents. In Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2025), pages 978–987, Detroit, Michigan, USA, 2025. International Foundation for Autonomous Agents and Multiagent Systems. arXiv:2503.02512.arxiv.org/abs/2503.02512
  15. [16]Yuval Jacoby, Clark Barrett, and Guy Katz. Verifying recurrent neural networks using invariant inference. In Automated Technology for Verification and Analysis (ATVA), volume 12302 of Lecture Notes in Computer Science, pages 57–74. Springer, 2020. Full version: arXiv:2004.02462.arxiv.org/abs/2004.02462
  16. [17]Guy Katz, Derek A. Huang, Duligur Ibeling, Kyle Julian, Christopher Lazarus, Rachel Lim, Parth Shah, Shantanu Thakoor, Haoze Wu, Aleksandar Zeljić, David L. Dill, Mykel J. Kochenderfer, and Clark Barrett. The Marabou framework for verification and analysis of deep neural networks. In Computer Aided Verification (CAV), volume 11561 of Lecture Notes in Computer Science, pages 443–452. Springer, 2019.DOI
  17. [18]Edmond Laguerre. Sur une méthode pour obtenir par approximation les racines d’une équation algébrique qui a toutes ses racines réelles. Nouvelles Annales de Mathématiques, 2e série, 19:161–171, 1880. URL https://www.numdam.org/item/NAM_1880_2_19__161_1/.
  18. [19]Aakash Lahoti, Kevin Y. Li, Berlin Chen, Caitlin Wang, Aviv Bick, J. Zico Kolter, Tri Dao, and Albert Gu. Mamba-3: Improved sequence modeling using state space principles. In International Conference on Learning Representations (ICLR), 2026. URL https://openreview.net/forum?id=HwCvaJ0iCj. arXiv:2603.15569.arxiv.org/abs/2603.15569
  19. [20]Winfried Lohmiller and Jean-Jacques E. Slotine. On contraction analysis for non-linear systems. Automatica, 34(6):683–696, 1998. doi: 10.1016/S0005-1098(98)00019-3.DOI
  20. [21]Takuo Matsubara, Peiwen Jiang, Wilson Ye Chen, and Minh-Ngoc Tran. Exponential smoothing for time series of random objects. arXiv preprint arXiv:2609.20274, 2026.arxiv.org/abs/2609.20274
  21. [22]Tobias Nipkow, Lawrence C. Paulson, and Markus Wenzel. Isabelle/HOL: A Proof Assistant for Higher-Order Logic, volume 2283 of Lecture Notes in Computer Science. Springer, 2002.DOI
  22. [23]Antonio Orvieto, Samuel L. Smith, Albert Gu, Anushan Fernando, Caglar Gulcehre, Razvan Pascanu, and Soham De. Resurrecting recurrent neural networks for long sequences. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 26670–26698. PMLR, 2023. arXiv:2303.06349.arxiv.org/abs/2303.06349
  23. [24]Adam Piaseczny, Md Kamran Chowdhury Shisher, Shiqiang Wang, and Christopher G. Brinton. Distributed learning with selective state space models: Architecture-aware convergence analysis. arXiv preprint arXiv:2610.02659, 2026.arxiv.org/abs/2610.02659
  24. [25]Paul M. Riechers. Geometry and dynamics of LayerNorm. arXiv preprint arXiv:2405.04134, 2024.arxiv.org/abs/2405.04134
  25. [26]Paul A. Samuelson. How deviant can you be? Journal of the American Statistical Association, 63(324):1522–1525, 1968. doi: 10.1080/01621459.1968.10480944.DOI
  26. [27]Corentin Tallec and Yann Ollivier. Can recurrent neural networks warp time? In International Conference on Learning Representations (ICLR), 2018. URL https://openreview.net/forum?id=SJcKhk-Ab. arXiv:1804.11188.arxiv.org/abs/1804.11188
  27. [28]Alex Tang, M. Emrullah İldiz, Batın Kurt, Samet Oymak, and Necmiye Ozay. On the generalization properties of selective state-space models for filtering tasks for unknown systems. arXiv preprint arXiv:2604.23818, 2026. Accepted to the 65th IEEE Conference on Decision and Control (CDC 2026).arxiv.org/abs/2604.23818
  28. [29]Omanshu Thapliyal. Bounded reachability & jailbreak detection via contraction-constrained state space models. arXiv preprint arXiv:2610.02853, 2026. Accepted at the First Workshop in AI Measurement Science (AIMS), COLM 2026.arxiv.org/abs/2610.02853
  29. [30]Hoang-Dung Tran, Sung Woo Choi, Xiaodong Yang, Tomoya Yamaguchi, Bardh Hoxha, and Danil Prokhorov. Verification of recurrent neural networks with star reachability. In Hybrid Systems: Computation and Control (HSCC). ACM, 2023. doi: 10.1145/3575870.3587128.DOI
  30. [31]Nikola Zubić and Davide Scaramuzza. Regularity and stability properties of selective SSMs with discontinuous gating. Transactions on Machine Learning Research, 2026. ISSN 2835-8856. URL https://openreview.net/forum?id=7Vav53cDeN. arXiv:2505.11602.arxiv.org/abs/2505.11602
  31. [32]Nicolas Zucchet and Antonio Orvieto. Recurrent neural networks: vanishing and exploding gradients are not the end of the story. In Advances in Neural Information Processing Systems, volume 37, pages 139402–139443. Curran Associates, Inc., 2024. doi: 10.52202/079017-4425. arXiv:2405.21064.DOI

Paper details

vlnc:W_7xayr9et
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Claude (Anthropic)

View statement
Looking for related papers…
Contents