WorksComputer Science
A Log-Timescale Perturbation Bound for Selective State Space Models at Every Sequence Length
Abstract
Mamba models deployed on streaming data, such as limit order book updates, run far past the sequence lengths they were trained on. A guarantee on how much a small change of input can move the hidden state therefore has to hold at every length. The closest existing bound we know of measures the change of the timescale additively, and it grows linearly with the sequence length unless the timescale is bounded away from zero. In this paper, we measure the change on a log scale instead. The key observation is that the decay factor of each step then moves by at most a fixed fraction of that step's input weight, so the per-step errors cannot pile up. Using this, we prove that two runs of a selective-scan cell from a common start stay within a bound that holds at every length, shrinks to zero with the size of the change and needs no lower bound on the timescale. We give a construction showing that, for small changes, this bound is within a factor of about 2 of the best possible, and we extend it to the update used in Mamba's released code when the timescale is bounded above. We then evaluate block-level bounds built from the theorem on two small models trained on one day of order book data. With a bounded timescale, a bound built on the existing analysis is tighter than ours for at most the first 8 steps. With Mamba's softplus timescale, whose certified lower bound is as small as , it is looser at every length. Our bounds hold for every input to a block, but they remain more than five orders of magnitude above the changes that gradient attacks find.
Introduction
Limit order book updates arrive as an unbounded stream. A selective state space model such as Mamba [11] can carry its state along such a stream far beyond its training windows. A bound on how far a small change of input moves that state then has to hold at every length. We prove such a bound for one cell, a scalar recurrence inside the selective scan, and measure how tight it is.
The closest bound we know of, Cao et al.’s Theorem 3.3(ii) [6], grows linearly with the position. The growth comes from its time-warp term, which accounts for the change of the timescale, the step size that the model computes from each input. Their Remark 3.4 caps the growth under a floor, a positive lower bound on the timescale, but the cap scales like as the floor shrinks. Mamba’s softplus timescale has a floor only when its pre-activations are bounded below, and on one of our trained models we certify a floor of about .
We measure the change of the timescale on a log scale, whereas Cao et al. measure it additively. In their bound, both the linear growth and the need for a floor come from the additive measure. We require the timescales and of a nominal and a perturbed run to differ by at most a factor at every step, that is, . We call this the log condition.
With an exact zero-order hold, which our models use, each step of the cell replaces the state by a weighted average of itself and an input-dependent equilibrium. The weight on the equilibrium, which we call the input weight, tends to zero with the timescale. Under the log condition, the two runs’ weights on the previous state differ by at most times the input weight (Lemma 2). Under an additive bound, this difference does not shrink with the input weight and can exceed it at small timescales (Figure 1).
Figure 1. (a) Change of the transition against when changes by 0.1 and when changes by 0.01, with the input weight . The two changes of are equal at , where the two curves of the change of the transition cross. The first change stays below times the input weight. The second exceeds the whole input weight for below about 0.0099. At it is 99.5 times the input weight. (b) The gap in the construction of Proposition 4 for , and . The nominal alternates between and 0.05. The gap settles at about 0.4895, just below the value that it approaches as the timescales tend to 0. The panel shows the bound of Theorem 3 and the trivial bound .
With Lemma 2, the gap between the two runs obeys an inequality of the same form as the update of the state, with a bounded equilibrium. The standard invariant argument for convex recursions, which Horbatko states for a Mamba channel [14 Lemma 4.4], bounds this gap at every length, as it bounds the state. From a common start, the variation-of-constants argument of Cao et al.’s Lemma B.2 also gives this bound after a change of time variable (Appendix § 10).
Our main result, Theorem 3, assumes the log condition, a bound on the change of the equilibrium and a bound on the equilibrium of the perturbed run. Two runs that start from the same state in then stay within of each other at every step, with no floor on the timescale. For every , and , Proposition 4 constructs such runs whose gaps come arbitrarily close to . From a common start, the theorem’s constant is therefore within a factor of the best possible, a factor that tends to 2 as .
If is derived from Cao et al.’s additive bound and floor, our time-warp term is no larger in each cell than that of their Remark 3.4. It is also no smaller than the limit, as the length grows, of a slightly sharper form of their bound, and the three are close when the floor is small (§ 3.3 and Appendix § 10). The gain in our experiments comes instead from bounding directly from the bound on the timescale’s pre-activation.
A gradient search in one cell, with both runs started from 0, finds no gap above the value that the construction of Proposition 4 approaches (§ 5.2). From random initial sequences it comes within 0.4 per cent of that value at the longest lengths searched, and closer still from initial sequences shaped like the construction.
We then train a small two-block model on one day of limit order book data, once with a timescale bounded to a fixed range and once with Mamba’s softplus timescale. From the theorem we build a bound for each block on how far its output can move when its normalised input changes (§ 4). With the bounded timescale, the same bound with an additive time-warp term of Cao et al.’s form in place of ours is tighter for at most the first 8 steps. Without a floor, it then grows linearly with the length. Under the floor that we certify, it levels off at a multiple of ours that is set mainly by the chosen timescale range (§ 5.3). With the softplus timescale, it is looser than ours at every length, and the certified floor is too small to stop its linear growth at any length we test.
Gradient attacks give lower bounds on the true sensitivity, and our block bounds, which hold for every input of a block, lie far above them. Along a given input the realised timescale is known, and the log and additive bounds on its change agree to first order. The log scale therefore matters for bounds that hold for every input, where no realised timescale is available (§ 5.4). Code and the recorded runs are at https://github.com/jackdoe/java/mamba-v-lob.
Section 2 sets up the cell. Section 3 proves the theorem, states the lower bound and extends the theorem to the exponential-Euler step of the released Mamba implementations, with the proofs of the lower bound and the extension in Appendix § 9. Section 4 builds the block bounds, and Section 5 reports the measurements.
Setting
The selective cell
A selective state space model computes a timescale and matrices and from the layer input at every position . It uses the timescale to discretise a fixed diagonal matrix with negative diagonal entries [11]. We certify a Mamba block that uses the exact zero-order hold for this discretisation. The block applies a layer normalisation, an input projection, a causal depthwise convolution with a SiLU, the projections that produce , and , and the scan. A SiLU gate, an output projection and a residual connection follow, and Appendix § 9.§ 4 gives the block in full.
Because is diagonal, the scan splits into independent scalar recurrences, one for each channel and state index . We call each of them a cell. A cell updates a scalar state from the input of its channel, and we write for its decay rate. We state our results on the scan for one cell.
We consider two maps from the timescale pre-activation to . The bounded map keeps in . Mamba’s map is positive, but it has no positive lower bound over real . We call a positive lower bound on the timescale a floor.
Under the exact zero-order hold, one step of a cell uses
and with the update becomes
Since for every input, each step is a convex combination of the previous state and . We call the equilibrium and the input weight of step . Gu and Dao [11 Theorem 1] give (2) as a gated recurrence for and with a softplus timescale.
The released implementations of Mamba use a different input term, which we treat in § 3.4.
The forward invariant
Because each step is a convex combination, a bound on the equilibria and on the initial state bounds the state at every step.
Proposition 1 (Forward invariant). If for every , then for every .
Proof. We argue by induction on : if , then .
Proposition 1 is Lemma 4.4 of Horbatko [14], stated there for a diagonal Mamba channel. For it also follows from the state bound of Cao et al.’s Theorem 3.3(i) and Lemma B.1 [6]. The reading of a convex gated update as a leaky integrator with a learned step goes back to Tallec and Ollivier [27] and Gu et al. [12].
Since and are activations, the hypothesis has to hold for every input the model can receive. Within the scan and the readout that follows it, only and the skip weight are fixed parameters. The timescale , the matrices and and the scan input are all computed from the block input.
Each block opens with a layer normalisation. By the Laguerre–Samuelson inequality [18, 26], the output of the normalisation lies in a bounded set fixed by its own parameters (Lemma 9 in Appendix § 11). Interval arithmetic carries this set through the projections, the convolution and the activations. For the projection that follows the normalisation, we use the fused bound of Proposition 10 in Appendix § 11, which follows from the geometry of layer normalisation that Riechers [25] describes. The result is a box, with one interval per coordinate, for the output of each of these stages, including , and . We call these the certified boxes. The boxes of and give the radius of each cell. The box of the timescale pre-activation of a channel gives its certified floor and certified ceiling , the smallest and largest timescales that the box allows.
We have machine-checked (1), Proposition 1, Lemma 9 and the first part of Proposition 10 in Isabelle/HOL [22], and the theory is in our repository. Lemma 2, Theorem 3 and Proposition 4 are not machine-checked. We compute for a trained model by interval propagation in Python code, which we test but have not formally verified.
A sensitivity bound at every length
We bound how far apart two runs of one cell can be at any length. The bound holds when their equilibria are bounded and their timescales differ by at most a factor at every step. The nominal run has timescales , equilibria , transitions and states given by (2). The perturbed run has , , and . We write for the error and call the gap. The hypothesis on the timescales is for every , where and is the natural logarithm. We write for the change of the equilibrium.
The ratio lemma
Lemma 2 shows that a relative change of the timescale moves the transition by at most a fixed fraction of the input weight.
Lemma 2 (Relative perturbation of the transition). For , and we have
In particular the left-hand side is at most whenever .
Proof. We compare chord slopes of the concave function , which vanishes at 0 and whose chord slope from the origin, , is therefore decreasing on . For we have , and the decrease of gives .
For we have and , hence , which together with proves both bounds.
With and , Lemma 2 gives
and with the runs exchanged it gives the same bound with on the right. Cao et al.’s Lemma B.3 bounds the same change by , which lacks the factor . Under an additive bound on , the term that each step adds to a bound on the gap does not shrink with the input weight. The later transitions damp these terms, but their sum grows like unless a floor makes the damping geometric.
Figure 1(a) shows the two cases at a single step. When changes by a fixed amount, the change of the transition vanishes with the input weight as . When changes by a fixed amount, the change of the transition does not vanish, and for small it exceeds the whole input weight.
The cell theorem
Theorem 3 bounds the gap at every step by a constant that depends only on , and , or by the initial gap if that is larger.
Theorem 3 (Cell bound). Suppose that and for every , and that . With , and , for every
Proof. We write the error as a convex recursion of the form (2) with a bounded equilibrium and then argue as for Proposition 1, which also gives for every . Adding and subtracting and in gives
Since , the bound (3) gives . Unrolling this inequality gives , where the sum telescopes to .
The passage from the one-step inequality to the bound of Theorem 3 is a standard contraction estimate, sampled at the end of each step. In the accumulated time , the cell contracts at the constant rate (Appendix § 10). The estimate compares a system that contracts at a constant rate with the same system plus a disturbance [20, 5]. After a time , the distance between their trajectories is at most times its initial value plus . With , and , this is the bound of Theorem 3.
Per step, the cell contracts by the factor , and without a floor the rate has no positive lower bound. The bound is still uniform in , because the disturbance at each step carries the same factor as the contraction, so their ratio stays bounded as . A bound that treats the transition and the input gain separately loses this cancellation (Appendix § 11).
The proof uses the hypotheses one step at a time. The bounds and may therefore depend on , with replaced by .
If , Proposition 1 applied to both runs gives and , hence the trivial bound . For , the trivial bound exceeds exactly when .
Proposition 4 (Lower bound). Let , , and . For every there are timescales with and equilibria with and for every such that the two runs from satisfy at some step .
The runs in the proof of Proposition 4 (Appendix § 9.§ 1) satisfy the hypotheses of Theorem 3. The perturbed equilibrium alternates between and . The perturbed timescale is times the nominal one on the steps with and times it on the others. The nominal run therefore has the larger input weight on the steps with and the smaller one on the others. As and then the timescales tend to , the gap after a step with tends to . Figure 1(b) shows the gap for , and . It settles close to , below the bound of Theorem 3.
Proposition 4 limits how far Theorem 3 can be improved. Every bound on the gap from a common start that holds under the hypotheses of the theorem and depends only on , and is at least . Since , the term of is times . The bound is therefore within a factor of the smallest such bound, and this factor tends to as .
From a common start in , the largest possible gap thus lies between and . We do not prove that is the largest possible gap. The numerical search of § 5.2, from the common start , finds none above it.
Comparison with Cao et al.
We replace the bound on by an additive bound and keep the other hypotheses of Theorem 3. We also assume a lower bound on both timescales, which is a floor when it is positive. By the mean value theorem, the transitions then satisfy . We carry the last term of (4) forward with a factor of at most per step. From a common start in , this gives
We call (5) the additive baseline. It has the structure of Cao et al.’s Theorem 3.3(ii) and Remark 3.4. It keeps the factor that their Lemma B.3 drops, which makes it slightly smaller than their constant. As Cao et al. do, we call the term of a bound that comes from the change of the timescale its time-warp term. In this term is , and in (5) it is the term with the sum.
Without a floor, the sum equals , so the additive baseline grows linearly in the length . With a floor it saturates at as . Its time-warp term is then the one-step cost times . This factor is of order as the floor shrinks.
Theorem 3 needs neither a floor nor an additive bound on the timescales, only a bound on the change of . Both timescale maps give one from a bound on the change of the pre-activation . By Lemma 6 in Appendix § 9, is at most for the softplus map and at most for the bounded map. Cao et al. derive the Lipschitz constant of that they assume from the same bound on .
With a floor, the additive bound alone also bounds the change of , by . With this , our time-warp term lies between the time-warp term of the additive baseline as and that of Cao et al.’s Remark 3.4. This follows from for (Appendix § 10). In our experiments, the gain over the additive baseline comes from bounding directly from the bound on , not through an additive bound on .
For the bounded map, Remark 5 compares the two time-warp terms in one cell once the additive baseline has saturated.
Remark 5 (The ratio for the bounded map). Let be a floor, and , let be the certified box of the pre-activation of one channel, and let a bound on the change of within give and . To first order in , the time-warp term of (5) as is
times ours (Appendix § 10). If contains 0, then by Lemma 6. With we have , and since and , the ratio is then at most , which for is 25.8, attained at . With the floor that the box certifies, this ratio depends on the trained weights only through and .
The exponential-Euler step
The released implementations of Mamba keep the transition but replace the input term by [19 Section 3.1]. This exponential-Euler step combines the exact transition with Euler’s rule for the input integral. It differs from the exponential Euler method of the literature on exponential integrators, which for this cell gives (1).
The update of this step is still a convex combination of the previous state and an equilibrium. The equilibrium is now enlarged by the factor , where lies in for . This factor grows without bound in . The analogues of Proposition 1 and Theorem 3 for this step therefore need an upper bound on the timescale, but they still need no floor. Corollary 7 in Appendix § 9 gives the analogue of the theorem for a common zero start. With , the bound holds for this cell with replaced by , by and the factor 2 by 3. Here bounds the change of .
From the cell to a certificate
Corollary 8 in Appendix § 9.§ 4 turns Theorem 3 into a bound on the change of a whole block’s output. The bound holds in real arithmetic for every input of the block, and we call such a bound a certificate. It needs no input data, so we also call it data-free.
The corollary follows a perturbation of the normalised input through the stages of the block in order. It composes Theorem 3 with sup-norm Lipschitz constants of the other stages, taken over the certified boxes that also give the radius . The input projection, the convolution and the SiLU bound the change of the scan input . The projections that follow bound the changes of , of and of the timescale pre-activation . Lemma 6 (Appendix § 9) turns the bound on into . A product rule gives the bound on the change of the equilibrium . Theorem 3 then bounds the change of the state in each cell. We carry this bound through the readout , the gate and the output projection.
Every stage of the certificate takes the worst case over its box and ignores the dependence between its inputs. Section 5.4 bounds from above how much is lost up to each stage.
Experiments
Setup
Our data are the Databento MBP-10 records of SPY on 28 January 2021. After feature construction they give about 4.55 million rows and 43 model inputs. We train a small model on windows of 100 ticks. From each window it predicts the log return of the mid-price over the next 100 ticks.
We divide the records chronologically into a training split of 70 per cent and validation and test splits of 15 per cent each. We purge each boundary by the prediction horizon of 100 ticks. We winsorise every feature at the 0.001 and 0.999 quantiles of its training values, in training and at inference alike.
The model maps the 43 inputs to width with a linear layer, a SiLU and a layer normalisation. Two Mamba blocks follow, each with channels and state indices per channel. Each block has a convolution of width 4 and a timescale projection of rank 4. A final layer normalisation and a two-layer head read the prediction at the last position.
We train two versions of this model, with set-ups that differ only in the timescale map. We call each of the two trained models an arm. The bounded arm uses the bounded map of § 2.1 with in , the range from which Mamba initialises its timescales [11]. The softplus arm uses Mamba’s map . Both arms use the exact zero-order hold (1), not the exponential-Euler input term of the released implementations (§ 3.4). After training, the decay rates of the bounded arm lie between 0.912 and 17.14.
We score predictions by the rank information coefficient, the Spearman correlation between predicted and realised returns. For each arm we keep the checkpoint with the best rank information coefficient on a fixed slice of 12,800 windows of the validation split. Training ran for at most 3000 steps, and the kept checkpoints come from step 750 on the bounded arm and step 2000 on the softplus arm.
Each arm is a single training run, and the two kept checkpoints come from different training steps, so we draw no conclusion about the timescale maps from differences between the arms. We keep the softplus arm to show what a very small certified floor does to the additive baseline.
On the first 25,600 windows of the test split the bounded arm has a rank information coefficient of 0.13. Its block bootstrap 95 per cent interval is [0.075, 0.197], so this arm has learned some signal. We do not compare it with other architectures.
We bound how much a perturbation of a block’s input changes the block’s output before the residual connection. This output is , where is the readout and the gate pre-activation (Appendix § 9.§ 4). The perturbation moves the normalised input of the block by at most in sup-norm at every position. We divide each bound, and each change that an attack finds, by and call the quotient a slope. For a certificate we take the limit of this quotient as .
When we compare a certificate with an attack, we compute the certificate on boxes widened to cover every point within of a normalised vector (Appendix § 9.§ 4). The widening is needed because an attacked point need not be a normalised vector. At it raises the certificate by less than 5 per cent.
We compute the boxes and radii of the certificate by interval arithmetic in float32, and the rest of the certificate in round-to-nearest float64. No stage uses outward rounding. The relative rounding error per operation is of order in float32 and in float64.
Each attack runs sign-gradient ascent in float64, and we re-evaluate the change it finds with a reference forward pass. Every attack value is therefore a lower bound on the true sensitivity. We call the largest slope that an attack finds the attack slope. The ratio of a certificate’s slope to the attack slope is an upper bound on the certificate’s looseness.
One cell
We test the tightness of Theorem 3 on a single cell by searching numerically for the largest gap between two runs. The cell has and , and both runs start from 0. The gap after steps is . The search runs gradient ascent with Adam in float64 for 600 steps from 64 random initial sequences and from initial sequences shaped like the construction of Proposition 4. In the shaped sequences, the perturbed equilibrium follows a square wave with runs of 1, 2, 4, 8 or 16 steps. We re-run the best sequence found with the plain recursion (2). Every gap we report is therefore attained by two admissible runs and is a lower bound on the true worst case.
We first search the hypothesis class of Theorem 3, with the nominal timescale free in . We take and at lengths from 10 to 1000.
No gap found at any searched length exceeds , the value that the construction of Proposition 4 approaches. For Adam from the initial sequences shaped like the construction comes within 0.01 per cent of this value at every and . At the same lengths, Adam from the 64 random initial sequences comes within 0.4 per cent of it. At the largest gap found lies less than 1.1 per cent below the value.
For and , the theorem’s bound equals times the largest gap found, to four significant figures. This factor is about 2.0 at and about 2.8 at (Figure 2, right).
Figure 2. Largest gap found by numerical search in one cell with and , with both runs started from 0. The left and middle panels show the largest gap found against the length for the bounded and the softplus map, at and . They also show the bound of Theorem 3, the additive baseline (5) with and without the floor , and the trivial bound . The gap levels off at about half of the theorem’s bound. On the softplus map the two forms of the baseline coincide at every length shown. The right panel shows the largest gap found at , divided by the theorem’s bound , against for and . The curve for each is the value that the construction of Proposition 4 approaches, divided by . Every point lies less than 0.01 per cent below the curve, and none lies above it.
We then set and search the perturbation model of our certificates. In this model each timescale is the image of a pre-activation under the timescale map. The nominal pre-activation lies in a box , and the perturbed one lies in within of . For the bounded map we take and . For the softplus map we take . On both maps we choose so that the bound on the change of is . The additive baseline (5) with uses the bound on the change of , and its floored form uses the floor . We search at lengths from 3 to 3000.
On both maps the largest gap found grows with and levels off at about half of the theorem’s bound (Figure 2, left and middle). This is the value that the search over the hypothesis class gives for and .
Since and here, every gap between two admissible runs is at most the trivial bound of § 3. This bound does not tend to zero with the perturbation, so it gives no finite slope and plays no part in the block certificates of § 5.3.
For the bounded map the additive baseline is above the theorem’s bound from without the floor and from with it. Both lengths are about inversely proportional to . With the same box and , at , the largest decay rate of the trained bounded arm, the baseline would be above the theorem’s bound from . Without the floor the baseline keeps growing linearly. With the floor it tends to about 25.6 times the theorem’s bound as , below the value 25.8 of Remark 5. Both forms of the baseline stay below the trivial bound at every searched length.
For the softplus map the floor is about . At every searched length the baseline with this floor coincides with the one without it. The baseline is above the theorem’s bound from and above the trivial bound from . It grows linearly to more than 2600 times the theorem’s bound at .
Comparison with the additive baseline
Table 1 and Figure 3 compare our data-free certificate with one that uses the additive baseline (5) in place of Theorem 3 in each cell. Our certificate does not depend on the length , while the baseline without a floor grows linearly with . On the bounded arm the baseline passes our certificate after about 8 steps, the crossing in Table 1. It grows by about 0.11 times our certificate per step. At it is about 540 times ours on both blocks. On the softplus arm the baseline is looser at every length. It grows by about 3.9 times our certificate per step, and at it is about times ours. These ratios at depend on the chosen length, whereas the crossing and the growth per step are properties of the trained model.
Figure 3. Data-free output slopes of block 0 of each arm against the sequence length . The flat line, labelled log-timescale, is our certificate, which uses the constant of Theorem 3 in every cell. Without a floor the additive baseline (5) grows linearly, and with the certified floor it levels off as . On the bounded arm both forms of the baseline pass the flat line near . On the softplus arm the smallest certified floor is about , and the two forms of the baseline overlap at every length shown.
| Arm | Block | Theorem 3 | Additive, no floor, | Ratio | Additive, floor, | Ratio | Crossing |
| bounded | 0 | 539 | 23.1 | 8.3 | |||
| bounded | 1 | 538 | 22.4 | 8.1 | |||
| softplus | 0 | none | |||||
| softplus | 1 | none |
Table 1. Data-free output slopes of each block of the two arms, maximised over output coordinates and rounded up. The additive columns use the baseline (5) without a floor at and with the certified floor as . Each ratio divides the unrounded additive slope to its left by the unrounded slope in the Theorem 3 column. The crossing is the length at which the affine baseline without a floor reaches the flat bound of Theorem 3. A crossing of none means that the baseline is the larger bound at every length.
Our certificate, the flat line in Figure 3, uses the constant of the theorem in every cell. The theorem also gives the finite-horizon bound , which is smaller than , so the baseline passes this bound no later than it passes . We also compare the two time-warp terms cell by cell, with our finite-horizon term at its worst over admissible timescales. Over the cells of the bounded arm, the baseline’s term reaches ours after a median of 5 steps ( in Appendix § 10).
With the certified floor the baseline levels off as . On the bounded arm its limit is 23 times our certificate on block 0 and 22 times on block 1. This ratio is set mainly by our choice of for the bounded map. On the bounded arm every certified floor is below and every certified ceiling is above 0.095. Since the bounded map is increasing and gives at , every box contains 0. Remark 5 then bounds the ratio of the two time-warp terms in each cell by 25.8, a cap that depends only on . Over the cells of the trained model the ratio lies between 24.1 and 25.8.
The ratio for the block is slightly below the ratio for a single cell. The two certificates differ only in the time-warp term, and the terms they share make up 0.11 to 0.13 of ours at its largest output coordinate.
With a floor taken from an attack on the block’s input, the baseline on the bounded arm is still 8.5 to 12 times our certificate (Appendix § 10). This floor, which we call the empirical floor, is the smallest timescale that the attack reaches in a channel. Any valid floor of a channel is at most its empirical floor, and a larger floor lowers the baseline. The empirical floor is therefore at least as favourable to the baseline as any valid floor.
On the softplus arm the certified floors are too small to matter at the lengths we test. The smallest are on block 0 and on block 1. Up to the baseline with these floors stays within 1 per cent of the baseline without one. Its limit as is times our certificate on block 0 and times on block 1.
Comparison with gradient attacks
We attack each block on four held-out windows at each of the lengths , 100, 1000 and 5000. The windows of all four lengths end at the same rows, and the two longest are 10 and 50 times as long as the training windows. From to at , the attack slope grows by a factor of 1.4 to 7.3, while the baseline without a floor grows by a factor of at least 445. The largest change found lies at the last position in only 1 of the 16 combinations of arm, block and length. At the last position alone, the attack slope grows by a factor of 1.06 to 2.3. The growth therefore comes partly from searching more positions and partly from the longer history before a single position.
No attack value exceeds a certificate, and our data-free certificate lies far above what the attacks find. At it is to times the attack slope. We also compare the certified bound after each stage of the block with the largest change that random perturbations of the block input produce there. Random perturbations do not search for the worst case, so each such ratio is an upper bound on the looseness after its stage. On block 0 of the bounded arm the ratio is 8.2 after the scan input , 134 after , 537 after the equilibrium , after the state , after the readout and at the output. At the output this ratio exceeds the ratio to the attack slope.
The factor of a stage is the ratio after it divided by the ratio before it. In every block the gate and the output projection together have the largest factor, 98 to 147. The scan follows with 18 to 39, and then the readout with 14 to 29. The step that forms the equilibrium from and has a factor of only 4 to 7. Each factor mixes the slack of its stage with how far the random samples fall short of that stage’s worst case.
Most of this slack comes from taking the worst case over the certified boxes. Around the realised input of each window, we also propagate the perturbation through every stage of the block. In the scan we apply the triangle inequality to the error identity (4) with the two runs exchanged. The realised distance then takes the place of the bound that the proof uses, and we take the result at the worst transition that the perturbed run can have. That transition lies in the intersection of the intervals given by the log-timescale bound and by the additive bound on . For this local bound is at most 12.7 times the attack slope, and dropping the log-timescale interval from the intersection changes it by at most 0.034 per cent. The local bound and its code are in our repository.
To first order in the perturbation, the choice between the log and the additive metric matters only for the data-free certificate. A bound on the change of the timescale pre-activation bounds the change of through the derivative and that of through . Since , the first bound is times the second to first order. Along a given input the realised timescale is known at every step, so either bound gives the other and the two agree to first order. The data-free certificate has no realised timescale to use, and there the two metrics give the different certificates of Section 5.3.
Related work
Bounds on selective scans. Stability analyses of selective scans differ in how they treat the input-dependent timescale. Halloran, Gulati and Roysdon [13] show that the maximal Lyapunov exponent of the Mamba recurrence is nonpositive when , and are held at their nominal values. Zubić and Scaramuzza [31] work in continuous time under a common quadratic Lyapunov function. Cao et al. [6] keep the timescale input-dependent and treat one layer without normalisation, for inputs in a known bounded set. Section 3.3 compares their bound with ours.
Analyses of training and generalisation for selective state space models also rely on a floor. Piaszczyny et al. [24] bound parameter gradients uniformly in the length under a floor on the softplus timescale. The robustness bound of Tang et al. [28 Theorem 1] takes its decay factor from a floor on the timescale.
Incremental stability of gated recurrences. A factor that plays the role of a floor also appears in incremental stability results for gated recurrent networks. The incremental stability conditions of Bonassi et al. [2] for networks of gated recurrent units contain the inverse of one minus an upper bound on the retention gate. The incremental gains of De Carli et al. [7, 8] contain the inverse of one minus a contraction bound, which includes an upper bound on the forget gate.
Orvieto et al. [23 Section 4] observe that the factor in the input gain cancels the growth of order that the transition alone would produce. Zucchet and Orvieto [32] use logarithmic timescale coordinates to control the parameter sensitivity of diagonal linear recurrences. Their coordinates describe the fixed eigenvalues of the recurrence, whereas the timescale of our cell changes at every step with the input.
The nearest argument we know is that of Matsubara et al. [21 Lemma A.5]. They bound the error between two exponential smoothers through a recursion like (4), with a constant proportional to the relative difference of their smoothing parameters. Each cell is such a smoother on the irregular grid of accumulated times [9 Section 4.1], but its input weight varies from step to step. Lemma 2 bounds the difference of the two transitions at each step by a multiple of that step’s input weight, which keeps these differences from accumulating.
Verification of recurrent models. Verifiers for recurrent networks either propagate an input set through a fixed number of steps or search for an invariant. Propagation certifies one length at a time, whether it unrolls the network into a feed-forward one or proceeds step by step with star sets [30]. A search for an inductive invariant may iterate the transition up to a depth limit [15] or check candidates with Marabou [16, 17]. It succeeds only if it finds an invariant. For state space models, Thapliyal [29] propagates intervals exactly through a linear time-invariant classifier and observes that the analysis does not carry over to Mamba.
Brucker and Stell [4] give Isabelle/HOL proofs for feed-forward classifiers, and George et al. [10] give Lean proofs for unrolled recurrent computation. Our Isabelle theory is narrower than both.
Limitations
Theorem 3 concerns one scalar cell, so it requires a diagonal . It also needs a real transition in , which makes each step a convex combination. The theorem therefore does not cover the complex-valued state update of Mamba-3 [19 Section 3.2], whose transition is not a real number in .
We do not prove that the largest gap from a common start equals . The search of § 5.2 gives only lower bounds on this gap. For , the construction of Proposition 4 puts the nominal equilibrium outside , at on half of the steps. When the equilibria of both runs are restricted to , so are both states, and the gap is at most . Under this restriction no gap can approach once , because this value then exceeds . With alternating between and instead, the construction satisfies the restriction and gives gaps that approach . We do not know the largest gap under this restriction.
Our certificates bound one block at a time, with respect to its normalised input. We do not compose them through the residual stream and the head. A certificate with respect to the raw input of a block would also have to pass through its layer normalisation (Lemma 9). This normalisation is globally Lipschitz because its variance offset is positive. In the sup-norm of our certificates, its Jacobian on the blocks of the bounded arm has norm at most 3.9 at realised inputs but about 653 at a constant input. Because a certificate holds for every input, one with respect to the raw input would carry the global Lipschitz constant, which is at least the norm of about 653 at a constant input. We do not compute one.
Theorem 3 and its corollaries hold in real arithmetic, and we evaluate our certificates without outward rounding (§ 5.1). A scan computed in float32 adds a rounding error at every step. In a cell this error decays at the rate , so it accumulates to at most about times the error added per step. On the bounded arm this factor is about 1100. One run there accumulates an error of at most about , where is the invariant radius of the cell. At , the errors of the two runs together exceed the certified change of the state in 49 and 61 per cent of the cells of the two blocks. A float32 scan is therefore covered only once a rounding term is added to the bound.
A transition can also round to exactly 1. In bfloat16 this happens to the slowest transitions of the bounded arm. In float32 it happens to the slowest transitions of the softplus arm, where reaches on the certified boxes. Where this happens, the computed update is not a convex combination, and the theorem does not apply to the scan as computed.
Our order book data come from one instrument on one trading day, and our attacks are heuristic searches on four windows. The two arms come from one training set-up with one learning rate. They use the exact zero-order hold rather than the exponential-Euler step of the released implementations. We do not certify a released language model here.
Conclusion
We have shown that measuring timescale perturbations in rather than in gives a sensitivity bound for a selective-scan cell that holds at every sequence length and needs no floor on the timescale (Theorem 3). From a common start, its constant is within a factor of the best possible (Proposition 4).
How much this helps in practice depends on the timescale map. With the bounded map, the additive baseline is tighter than our certificate only over the first 8 steps or fewer, and once it uses its certified floor, the chosen timescale range caps its ratio to ours in each cell (Remark 5). With Mamba’s softplus map, the certified floor is the only thing that limits the baseline, and on our model it is too small to matter at any length we test. Along a single input, the log and additive ways of measuring the change agree to first order, so the gain shows up only in bounds that must hold for every input of a block.
This paper does not settle whether is the largest possible gap from a common start, although our search finds none larger, nor what the largest gap is when the equilibria of both runs lie in (§ 7). Natural next steps are to extend the theorem to the complex-valued transitions of Mamba-3 and to tighten the block certificate, which remains far above what attacks find.
Use of generative AI
Claude (Anthropic), a large language model, assisted the authors throughout this project. It assisted with searching and summarising the related literature, drafting and checking proofs, writing and debugging code, running experiments, and drafting and revising the text. It was also used to review the paper, including tracing every reported number to the code and the recorded runs in our repository. The authors reviewed all AI-assisted material and take full responsibility for the content of this paper.
The lower bound, the corollaries and their proofs
Proof of the lower bound
Proof of Proposition 4. Put , fix and put and , so that . On the even steps we take , and , and on the odd steps , and . On every step . Then , and for every . The nominal transitions are and the perturbed ones , starting at .
Let be the run on the nominal timescales with the equilibria of the perturbed run, with , and split . The first part satisfies with , so and , with as in Theorem 3. Since , tends to .
For the second part we put and for , so that and . Step has and step has , so for
and in the same way, with the two transitions exchanged, . Since , both sequences converge to the fixed points of these affine maps, and
As the numerator of the last fraction is and its denominator , so the limit tends to .
Given , we choose such that . Since tends to as , some gives . At we then have .
The log-timescale slopes
Lemma 6 (Log-timescale slopes). (i) For , the slope , which equals , lies in , is strictly decreasing in and tends to as . Hence , and for the constant may be replaced by . (ii) For , the slope lies in , with the maximum at .
Proof. For (i) we put , for which and . Since vanishes at and has derivative , the ratio lies in . Its derivative has the sign of , and as both functions equal .
For (ii), and , which lies in .
Our float32 implementation clamps to on the bounded arm. In the reals the clamp is inactive, and it never increases .
The exponential-Euler cell
With , and as in § 3.4 we have , and the exponential-Euler update is the convex combination
Corollary 7 (Exponential-Euler cell). Suppose that , , and for every , and that . With , the exponential-Euler cell satisfies for every .
Proof. Since , we have , which lies in for . Hence is increasing and . With this gives and , and Theorem 3 applies with in place of .
The same bound gives , hence . Adding the time-warp term gives the claim.
The block and the data-free block certificate
We certify a block that computes
with all products elementwise and broadcast over the state axis; on the softplus arm . Both and hold by construction, for every checkpoint of the architecture.
With LN the layer normalisation of Lemma 9, of width , gain and offset , we write for the set on which the boxes of § 2 are certified, which contains for every , and for the entrywise absolute value of a matrix . We write , , and for the suprema of , , and over the certified boxes, for the invariant radius of cell , and . We write and for the suprema of over the boxes of the convolution output and the gate pre-activation of channel , and for the supremum of the over the box of (Lemma 6). Each below is a sup-norm Lipschitz constant of the named stage over its certified box (, for example, bounds the change of per unit change of the normalised input).
Corollary 8 (Block). Let and be two input sequences of the block, both runs starting from , and let for every . Let be the convolution kernel of channel , let , and be the blocks of x-proj that produce , and , let , and be the weights of in-proj, dt-proj and out-proj, and define
Then for every , channel , state and output coordinate , , , , and .
Proof. We follow the perturbation through the stages in order. Since both normalised sequences lie in , every intermediate quantity of both runs, and every segment between corresponding values, then lies in its certified box. Through the input projection, moves by at most and by at most . With zero left padding in both runs, the convolution multiplies that bound by , and the mean value theorem for SiLU gives . Linearity of x-proj and dt-proj gives , and , and Lemma 6 gives . Writing gives , and Theorem 3 gives . The same product rule applied to with gives . The gate, the output projection and the residual then give , and the term .
As a function of , increases through and tends, as , to the value with in place of . Perturbations that leave need the boxes widened to (§ 5.1 gives the effect on our model).
Relation to Cao et al. in detail
A derivation from their lemmas. In the setting of Cao et al. [6], is Hurwitz with , one timescale per step is shared by the whole state, and by their Lemma 3.1 the scan samples a continuous flow at the grid of accumulated times. We follow their argument for one cell of (2), whose flow is with the forcing on step . We take both runs from the same state with . With , and we have and , where and solve and from the common start, with forcing on and on .
We map each interval linearly onto and let be the value of at the corresponding time, which gives and, on ,
With the two runs now on one grid, their forcings differ by . Since , , and , we have . The bound holds because between grid points is a convex combination of and , with by Proposition 1. The mismatch depends on itself, but that is admissible because takes the values of .
By variation of constants (the argument of their Lemma B.2, which applies to any bounded measurable forcing), two solutions of from a common start differ at time by at most times the supremum of the difference of their forcings. Since , this gives for every , which is Theorem 3 for a common initial state.
In the proof of their Theorem 3.3, Cao et al. state that a direct bound on the forcing mismatch is unavailable because the two forcings are piecewise constant on different grids, but our change of time variable aligns the grids. Its size, the largest , has the form of the norm that Billingsley [1, Section 12] assigns to a time change of in the Skorokhod topology, here for a map between intervals of different lengths.
Comparison regime by regime. Their time-warp argument assumes one timescale shared by the whole state, which holds for each channel of the block separately. Treating a diagonal one cell at a time is the most favourable form of their bound, since their statement for the whole state uses and . On the bounded arm, for example, the decay rates lie between 0.912 and 17.14, while a statement for the whole state replaces all of them by the two extremes. Both the additive baseline (5) and Theorem 3 pass through the readout in the same way, which lets us compare them at the level of the state.
Without a floor, the additive time-warp term grows linearly in the number of steps and ours does not, although for short sequences ours can be the larger. Let be the smallest at which the additive time-warp term without a floor after steps, , is at least our finite-horizon term at its worst over admissible timescales, , to first order in the size of the perturbation, where is the certified ceiling. For fewer than steps the additive term is the smaller. Because the finite-horizon term is at most , is never later than the step at which the additive term passes the flat bound of Theorem 3 in the same cell. Over the cells of the bounded arm ranges from 3 to 42 with median 5, while on the softplus arm it is at most 2.
With a floor and only the additive information , the admissible value turns our time-warp term into with . This lies between the additive baseline and the form of their Remark 3.4. Under their hypotheses our term is thus no larger than theirs, and the ratios of the three terms tend to 1 as . At the certified floors of our trained model, where is below 0.018, they differ by less than 2 per cent.
We obtain the ratio of Remark 5 from the two terms directly. If changes by at most on the certified box of its channel, then and are admissible. The limit of the additive time-warp term, divided by our , gives the ratio of the remark. Its numerator is the counterpart, restricted to the box, of the factor that multiplies in the Lipschitz constant of that Cao et al. assume (their Assumption 3.2 (A4)), bounded in their Proposition 4.6 and Remark 4.7 with the supremum over all . When excludes 0 the argument of the remark does not apply. Since increases in , on we have , so with the floor that the box itself certifies, the first factor of the ratio is at most , a bound that tends to 1 as the box narrows. For the bounded map with and , the first factor itself equals 1, because and are both attained at ( peaks at ). The whole ratio is then , and the floored baseline is the tighter bound in such a cell.
At the empirical floor of § 5.3 (the smallest timescale in a channel that an attack on the block’s input reaches) the baseline is 8.5 to 12 times our bound at the output of a bounded block. Per cell, however, the ratio there ranges from 0.16 to 25, and in about a tenth of the cells an exact per-channel floor could make the additive term smaller than ours.
For the softplus arm the leading factor is about when the floor is small. With the certified floors of the trained softplus arm the per-cell ratio, which compares the limits as , lies between and , with medians of and on the two blocks.
Interval arithmetic and the normalisation bounds
The separated bound
A verifier handed the update without (1) bounds and separately and obtains the geometric bound . In the true update the input term and the contraction cancel, because the same factor multiplies the one and defines the other. A separated bound instead takes the input term at the ceiling of the timescale box and the contraction at its floor , which gives the per-cell ratio to the invariant
This ratio is at least 1, with equality only for a one-point box. As it grows like while the invariant does not move, and since is decreasing it never exceeds . It decreases in , because is decreasing and the log-derivative of (6) in is .
Interval arithmetic meets the same loss when it checks the induction step of Proposition 1. Because the variable occurs twice in , an interval evaluator given a candidate box on the state and the bound on returns . This check passes exactly when , that is, for boxes at least the geometric fixed point. For the invariant’s own box, , it fails whenever the timescale box is not a single point.
With kept as one variable, the supremum over of is . The box is then preserved exactly when (our implementation checks the residual ).
On the bounded arm of our order book model, generic interval propagation iterated to convergence agrees with the geometric fixed point, which is 95.6 and 95.2 times the invariant radius on the two blocks.
The normalisation bounds
Lemma 9 (Coordinate bound). Let on , with , and the variance with divisor . Then for every and every .
Proof. With we have and . Cauchy–Schwarz on gives , hence and . □
With , Lemma 9 is the Laguerre–Samuelson inequality [18, 26], which bounds one coordinate at a time, whereas the fused bound of Proposition 10 combines the zero-mean and norm constraints with the following linear map and equals the supremum over all inputs.
Proposition 10 (Fused bound). Let be row of the linear map applied to the normalised output, let with the mean of its entries, and let . Then , attained, when is not constant, at . For the supremum over the normalised vectors of all inputs is the same value, but when is not constant it is not attained.
Proof. Since , Cauchy–Schwarz over applied to gives the bound, which attains. Every normalised vector has , and when is not constant Cauchy–Schwarz then gives for every normalised vector , and the value is not attained. For the input the normalised vector is , which tends to as and shows that the supremum is the same value. □
Riechers [25] and Brody et al. [3] describe this geometry of layer normalisation (a projection onto the hyperplane orthogonal to followed by a scaling). Bounding the normalisation and the linear map separately gives instead, an quantity where an one suffices. This difference carries over directly to the radius , which is why our certificates use the fused bound.
References
- [2]Fabio Bonassi, Marcello Farina, and Riccardo Scattolini. On the stability properties of gated recurrent units neural networks. Systems & Control Letters, 157:105049, 2021. doi: 10.1016/j.sysconle.2021.105049.DOI
- [3]Shaked Brody, Uri Alon, and Eran Yahav. On the expressivity role of LayerNorm in transformers’ attention. In Findings of the Association for Computational Linguistics: ACL 2023, pages 14211–14221, Toronto, Canada, 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.findings-acl.895. arXiv:2305.02582.
- [4]Achim D. Brucker and Amy Stell. Verifying feedforward neural networks for classification in Isabelle/HOL. In Formal Methods: 25th International Symposium, FM 2023, volume 14000 of Lecture Notes in Computer Science, pages 427–444. Springer, 2023. doi: 10.1007/978-3-031-27481-7_24.DOI
- [5]Francesco Bullo. Contraction Theory for Dynamical Systems. Kindle Direct Publishing, 2026. ISBN 979-8836646806. URL https://fbullo.github.io/ctds. Edition 1.3.
- [6]Liang Cao, Weide Liu, Zhuo Chen, and Yan Qin. A control-theoretic view of Mamba on stability and robustness. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pages 11350–11364. PMLR, 2026.
- [7]Stefano De Carli, Davide Previtali, Mirko Mazzoleni, and Fabio Previdi. Chaos-free networks are stable recurrent neural networks. IEEE Control Systems Letters, 10:751–756, 2026. doi: 10.1109/LC SYS.2026.3702800. arXiv:2603.14106.arxiv.org/abs/2603.14106
- [8]Stefano De Carli, Davide Previtali, Mirko Mazzoleni, and Fabio Previdi. Stability properties of Minimal Gated Unit neural networks. arXiv preprint arXiv:2603.03017, 2026.arxiv.org/abs/2603.03017
- [9]Andreas Eckner. Algorithms for unevenly spaced time series: Moving averages and other rolling operators. Working paper, April 2019. URL http://www.eckner.com/papers/Algorithms%20for%20Unevenly%20Spaced%20Time%20Series.pdf. First version January 2010; current version 10 April 2019.
- [10]Robert Joseph George, Jennifer Cruden, Will Adkisson, Xiangrui Zhong, Huan Zhang, and Anima Anandkumar. TorchLean: Formalizing neural networks in Lean. arXiv preprint arXiv:2602.22631, 2026.arxiv.org/abs/2602.22631
- [11]Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. In First Conference on Language Modeling (COLM), 2024. arXiv:2312.00752.arxiv.org/abs/2312.00752
- [12]Albert Gu, Isys Johnson, Karan Goel, Khaled Saab, Tri Dao, Atri Rudra, and Christopher Ré. Combining recurrent, convolutional, and continuous-time models with linear state space layers. In Advances in Neural Information Processing Systems, volume 34, pages 572–585. Curran Associates, Inc., 2021. arXiv:2110.13985.arxiv.org/abs/2110.13985
- [13]John T. Halloran, Manbir Gulati, and Paul F. Roysdon. Mamba state-space models are Lyapunov-stable learners. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL https://openreview.net/forum?id=wzsYQYs3d0. arXiv:2406.00209.arxiv.org/abs/2406.00209
- [14]Liubomyr Horbatko. Sessa: Selective state space attention. arXiv preprint arXiv:2604.18580, 2026.arxiv.org/abs/2604.18580
- [15]Mehran Hosseini, Alessio Lummuscio, and Nicola Paoletti. LTL verification of memoryful neural agents. In Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2025), pages 978–987, Detroit, Michigan, USA, 2025. International Foundation for Autonomous Agents and Multiagent Systems. arXiv:2503.02512.arxiv.org/abs/2503.02512
- [16]Yuval Jacoby, Clark Barrett, and Guy Katz. Verifying recurrent neural networks using invariant inference. In Automated Technology for Verification and Analysis (ATVA), volume 12302 of Lecture Notes in Computer Science, pages 57–74. Springer, 2020. Full version: arXiv:2004.02462.arxiv.org/abs/2004.02462
- [17]Guy Katz, Derek A. Huang, Duligur Ibeling, Kyle Julian, Christopher Lazarus, Rachel Lim, Parth Shah, Shantanu Thakoor, Haoze Wu, Aleksandar Zeljić, David L. Dill, Mykel J. Kochenderfer, and Clark Barrett. The Marabou framework for verification and analysis of deep neural networks. In Computer Aided Verification (CAV), volume 11561 of Lecture Notes in Computer Science, pages 443–452. Springer, 2019.DOI
- [18]Edmond Laguerre. Sur une méthode pour obtenir par approximation les racines d’une équation algébrique qui a toutes ses racines réelles. Nouvelles Annales de Mathématiques, 2e série, 19:161–171, 1880. URL https://www.numdam.org/item/NAM_1880_2_19__161_1/.
- [19]Aakash Lahoti, Kevin Y. Li, Berlin Chen, Caitlin Wang, Aviv Bick, J. Zico Kolter, Tri Dao, and Albert Gu. Mamba-3: Improved sequence modeling using state space principles. In International Conference on Learning Representations (ICLR), 2026. URL https://openreview.net/forum?id=HwCvaJ0iCj. arXiv:2603.15569.arxiv.org/abs/2603.15569
- [20]Winfried Lohmiller and Jean-Jacques E. Slotine. On contraction analysis for non-linear systems. Automatica, 34(6):683–696, 1998. doi: 10.1016/S0005-1098(98)00019-3.DOI
- [21]Takuo Matsubara, Peiwen Jiang, Wilson Ye Chen, and Minh-Ngoc Tran. Exponential smoothing for time series of random objects. arXiv preprint arXiv:2609.20274, 2026.arxiv.org/abs/2609.20274
- [22]Tobias Nipkow, Lawrence C. Paulson, and Markus Wenzel. Isabelle/HOL: A Proof Assistant for Higher-Order Logic, volume 2283 of Lecture Notes in Computer Science. Springer, 2002.DOI
- [23]Antonio Orvieto, Samuel L. Smith, Albert Gu, Anushan Fernando, Caglar Gulcehre, Razvan Pascanu, and Soham De. Resurrecting recurrent neural networks for long sequences. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 26670–26698. PMLR, 2023. arXiv:2303.06349.arxiv.org/abs/2303.06349
- [24]Adam Piaseczny, Md Kamran Chowdhury Shisher, Shiqiang Wang, and Christopher G. Brinton. Distributed learning with selective state space models: Architecture-aware convergence analysis. arXiv preprint arXiv:2610.02659, 2026.arxiv.org/abs/2610.02659
- [25]Paul M. Riechers. Geometry and dynamics of LayerNorm. arXiv preprint arXiv:2405.04134, 2024.arxiv.org/abs/2405.04134
- [26]Paul A. Samuelson. How deviant can you be? Journal of the American Statistical Association, 63(324):1522–1525, 1968. doi: 10.1080/01621459.1968.10480944.DOI
- [27]Corentin Tallec and Yann Ollivier. Can recurrent neural networks warp time? In International Conference on Learning Representations (ICLR), 2018. URL https://openreview.net/forum?id=SJcKhk-Ab. arXiv:1804.11188.arxiv.org/abs/1804.11188
- [28]Alex Tang, M. Emrullah İldiz, Batın Kurt, Samet Oymak, and Necmiye Ozay. On the generalization properties of selective state-space models for filtering tasks for unknown systems. arXiv preprint arXiv:2604.23818, 2026. Accepted to the 65th IEEE Conference on Decision and Control (CDC 2026).arxiv.org/abs/2604.23818
- [29]Omanshu Thapliyal. Bounded reachability & jailbreak detection via contraction-constrained state space models. arXiv preprint arXiv:2610.02853, 2026. Accepted at the First Workshop in AI Measurement Science (AIMS), COLM 2026.arxiv.org/abs/2610.02853
- [30]Hoang-Dung Tran, Sung Woo Choi, Xiaodong Yang, Tomoya Yamaguchi, Bardh Hoxha, and Danil Prokhorov. Verification of recurrent neural networks with star reachability. In Hybrid Systems: Computation and Control (HSCC). ACM, 2023. doi: 10.1145/3575870.3587128.DOI
- [31]Nikola Zubić and Davide Scaramuzza. Regularity and stability properties of selective SSMs with discontinuous gating. Transactions on Machine Learning Research, 2026. ISSN 2835-8856. URL https://openreview.net/forum?id=7Vav53cDeN. arXiv:2505.11602.arxiv.org/abs/2505.11602
- [32]Nicolas Zucchet and Antonio Orvieto. Recurrent neural networks: vanishing and exploding gradients are not the end of the story. In Advances in Neural Information Processing Systems, volume 37, pages 139402–139443. Curran Associates, Inc., 2024. doi: 10.52202/079017-4425. arXiv:2405.21064.DOI


