WorksComputer Science
Halting in Recurrent Sequence Models
Abstract
Recurrent sequence models, such as recurrent neural networks (RNNs) and state space models (SSMs), are often trained to emit an end-of-sequence (EOS) token when their output sequence is complete. During training, EOS is treated as another token without a special loss term or incentive. Although this training paradigm is widely used, it is not understood how models decide to output EOS. We study a minimal version of the EOS halting problem in which networks are trained to emit a given input sequence an instructed number of times, followed by EOS. We train tanh RNNs and linear SSMs on this task, finding that the RNNs encode the number of tokens left to copy in a linear subspace, of which two principal components capture most of the variation. Hidden states with the same remaining count converge to a shared representation regardless of their instructed repeat count. By modulating the hidden state using an affine transformation at the boundary between copied sequences, we shift the RNN's state to the representation shared by states with one fewer copy remaining. The unchanged network emits EOS one copy earlier while reproducing the shortened target sequence in 94.4% of new sequences, revealing that this representation causally determines the stopping time. In the SSMs, the difference between trajectories with different repeat counts is separated across two-dimensional rotation planes whose contributions to the EOS comparison cancel during copying and align near the stopping time. Rotating the planes independently removes most of the intended stopping while preserving the copied content, which shows that the stopping time depends on the planes' relative phases. These results identify two candidate primitives of EOS halting in recurrent models: a shared encoding of the number of remaining tokens in tanh RNNs and phase alignment across fixed rotations in linear SSMs.
Introduction
How does a recurrent neural network (RNN) know when to stop outputting new tokens when trained on a task with a variable sequence length? In such a task setting, RNNs commonly end generation by emitting an end-of-sequence (EOS) token after their content. Given an input and hidden state , the model updates its state and assigns a score to every output token according to
Here is the recurrent update with parameters , is the output matrix (decoder), and is a vector containing the scores for each token in the task’s vocabulary . The EOS margin
provides us with a mathematical object that defines when EOS is output: the model emits content while and emits EOS once . When subsequent inputs are fixed, the current state determines the entire future trajectory and can therefore determine the stopping time even while its current margin remains negative. The margin only records when EOS is emitted. Why its sign changes at a particular step has to be explained by the hidden dynamics, which carry the count instruction until then. Existing attempts at characterizing EOS prediction do not provide a transition-level mechanism of how a count instruction is transported into the EOS comparisons (§ 2).
The repeat-copy task augments the copy task by adding a count instruction signaling that the sequence to be copied should be copied times, followed by EOS [11]. Under different count instructions and , , the same model produces hidden trajectories and . At each output step shared by such trajectories, the trajectory difference
isolates the effect of the repeat count, since both trajectories store the same sequence. Its evolution describes how the count reaches the EOS comparisons.
We consider a minimal RNN architecture and show that its states carry the number of output steps remaining, which at the fixed sequence length is equivalent to the position within a copy together with the number of copies remaining. Trajectories with the same number of remaining steps approach the same states regardless of the count instruction. Within each copy, mixes , and the tanh activation then attenuates each of its components according to the current pair of hidden states. An affine state transformation learned from separate sequences shifts a state at a copy boundary to the states with one fewer copy remaining, which causes EOS to be emitted one copy earlier without changing the tokens output until that point. For comparison, experiments using a linear SSM reveal that evolves within two-dimensional rotation planes whose contributions to the EOS comparisons cancel during copying and increase near the correct stopping time. Independently rotating these contributions removes most of the intended stopping behavior while preserving the copied content (Figure 1).
Figure 1. The count instruction changes the EOS time while the stored sequence remains fixed. A, RNN trajectory differences on held-out sequences projected onto two principal components obtained from separate sequences. Open circles mark the state after the count instruction, stars mark the earlier target EOS, and arrows follow the autonomous recurrence. B, mean contributions of the 64 SSM rotation blocks to the limiting EOS comparison for count change 4 → 2. Between copy boundaries the contributions have mixed signs. At the target EOS, most contributions have the same sign and the EOS margin crosses zero. C, repeat-copy examples ending after two or four copies of . RNN seed 5, the model with the median energy in the two plotted components, and SSM seed 1 are shown.
Related work
Prior work on halting using an EOS token has examined sequence probability, decoding criteria, learned stopping, and length generalization [24, 26, 31, 33]. Newman et al. [27] find that EOS training organizes recurrent states into length manifolds and terminal attractors. Grant et al. [9] show that in models trained on tasks using a latent count variable, hidden state interventions on the network’s representation of the count variable leads to altered output behavior.
Adaptive computation methods add an explicit halting variable that controls the number of internal updates a model makes along its trajectory [1, 3, 5, 7, 10]. Structured state space models (SSMs) implement recurrent sequence processing through constrained linear transitions [14, 15, 28, 30], including the scaled rotation blocks in the linear SSM studied here.
Controlled sequence tasks reveal memory, position, and algorithmic structure in recurrent states [2, 6, 12, 17, 20, 21, 22]. Fixed points and local linearizations describe their evolution under the recurrent transition [16, 25, 29, 32]. Causal interventions test the behavioral role of variables identified by linear decoders [4, 8, 18, 19].
Stopping as a hitting-time problem
Task. We use the repeat-copy task [11], in which each input sequence consists of symbols with , yielding sixteen possible content tokens. After the symbols and a delimiter, an instruction with a single entry encodes and requests the output
Unlike Graves et al. [11], who supply the repeat count as a scalar on a separate input channel, we present it as a one-hot token after the delimiter.
Models. We use two models: an RNN with one hidden layer and tanh activation, and a linear SSM. Both models have width . A matrix first maps each input into the hidden state before the bias-free output matrix returns the token scores. The recurrent operator is for the RNN and for the SSM:
The SSM transition contains 64 learned and scaled rotation blocks. We train five seeds for each architecture, using 100,000 updates for the RNNs and 5,000 updates for the SSMs. Every checkpoint emits EOS at the intended step on the held-out sequences used in the analyses below, and only RNN seed 4 miscopies a single content token, in one of 512 sequences.
Categorical stopping. Two paradigms are generally used to signal completion. The first is categorical stopping, which defines EOS as one token of the model’s vocabulary and compares its score directly with the scores assigned to all other . The second encodes the EOS signal in a separate binary output “head” that outputs 1 when the model signals completion. We consider categorical stopping throughout, where the stopping time depends on how the recurrent state reaches the part of the output space that favors EOS.
Under categorical stopping, the comparison between EOS and each content token is linear in the hidden state. Let denote row of . For each content token , the difference defines the EOS comparison
and stacking the rows gives with , so that . We call the comparison that attains this minimum the limiting EOS comparison. Note that the EOS token is assigned the largest value in the network’s output layer exactly when . Generally, many distinct hidden states lead to the model outputting EOS [13, 27]. Thus, we can define the set containing all valid hidden states associated with emission of EOS.
Stopping can then be viewed as a hitting-time problem. Once the model has finished reading all input symbols, its input is set to zero and the hidden-state recurrence becomes autonomous, , and stopping must therefore be encoded by alone. Hence, chained application of determines when the trajectory first enters . We call this first entry time the EOS hitting time,
so that counts the steps that remain before EOS. Correct stopping requires to equal the number of output steps the count instruction specifies, and carries the difference between the two required hitting times along the trajectory.
Metrics. All accuracies below are measured on held-out sequences. Stopping accuracy is the fraction of sequences that emit EOS at the intended step, content accuracy is the fraction of content tokens that match the target, and complete-sequence accuracy is the fraction of sequences whose entire output, content and EOS position together, matches the target. Where an intervention is applied part way through a rollout, we additionally report continuation accuracy, the fraction of reconstructed rollouts whose decoded output matches the one the unmodified network produces.
How the hidden state encodes stopping
Fix a model , let denote the step at which it reads the count instruction, and consider two hidden-state rollouts of the same input sequence under count instructions and . The hidden states are identical until the count instruction creates the preactivation difference
Because subsequent inputs are zero, the autonomous recurrence needs to carry this difference until EOS is output.
RNN
During autonomous output, both RNN trajectories satisfy
For scalar preactivations and , we define the tanh secant
Now let be the diagonal matrix containing these secants for the corresponding preactivations in the two rollouts, and define the resulting change in the EOS comparisons as . This difference then follows the exact recurrence
This identity holds for any pair of trajectories. From its injection through , this difference moves along the autonomous hidden trajectory before affecting the EOS comparisons (Figure 6). Across the five RNNs, the first two components of an uncentered PCA basis obtained from separate sequences contain on average of the energy of . Projected onto these components, the differences on new sequences trace coherent curves, with the two adjacent count changes following nearly the same path (Figure 1A).
Figure 6. Propagation of the RNN trajectory difference from the instruction to the EOS comparisons for model seed 5 and count change . A, the preactivation response and resulting hidden response at the instruction. Hidden units are ordered by the step of their largest response on separate sequences. B, the hidden response during autonomous output, with horizontal lines at copy boundaries. C, the norm of , its effect on the EOS comparisons, and the difference between the two EOS margins, each scaled by its largest absolute value.
The transport of consists of a fixed mixing step followed by a state-dependent scaling step. At each step, first mixes the current difference. Then, scales each hidden unit by a value in . Since depends on the two current states, this scaling changes with the stored sequence, the repeat count, and the output position. To test which of these dependencies matter, we replace the secants in Equation (4) and propagate with the replaced secants. Note that these replacements run open loop: the propagated difference is decoded through , but the network itself never receives the resulting states. Removing the scaling altogether, or moving each secant to a different hidden unit, leaves almost no correct continuation (Table 1 and Figure 2). Shifting the same secants in time retains a higher continuation accuracy. Secants taken from other sequences retain more, and their fitting-set mean at the same step yields the highest continuation accuracy of all. Hidden-unit identity and temporal order thus have the largest effects, while the values themselves are largely shared across sequences. The fixed mixing by admits a further separation into normal and nonnormal parts, neither of which reproduces the stopping times on its own (Section A).
| Secants used in the transport | Continuation accuracy |
| Exact | 1.000 |
| Identity, | 0.000 |
| Permuted across hidden units | 0.010 |
| Mean across hidden units, same sequence and step | 0.091 |
| Circularly shifted in time | 0.316 |
| Taken from other sequences | 0.669 |
| Fitting-set mean, same step | 0.859 |
Table 1. Replacements for the tanh secants in Equation (4), evaluated across the five RNNs and the three count changes. Each replacement propagates over the final copy of the shorter target. We add the result to the trajectory of the longer count and decode it through , without rerunning the network. Continuation accuracy is the fraction of these reconstructions whose output matches the output of the unmodified network. The exact row is a numerical check.
Figure 2. Transport of the RNN trajectory difference. A, mean tanh secants for model seed 5 and count change , with hidden units ordered by the step of strongest attenuation on separate sequences. B, multiplication by expands the trajectory difference before tanh attenuates it. Faint lines denote model means and solid lines denote the cohort mean. C, EOS margins along the two trajectories. D, continuation accuracy under the exact secants and their controlled replacements.
The transported difference separates trajectories by the number of output steps that remain, which we describe by the current position within the copied sequence and the number of copies remaining.
Indexing the output states from , for a count instruction let
with the index within the current copy of and copies of remaining. Applying the network’s autonomous transition function to ,
Equation (5) describes the change in and within a single rollout. The position advances until the copy ends, and the number of copies remaining then decreases by one. Because is fixed, is equivalent to the number of output steps remaining, which equals the hitting time of § 3.
Rollouts that began with different count instructions can contain the same value for . A model that is asked for four copies of the input, for example, reaches after completing two copies. The same model asked for three copies reaches after completing one copy of . States that agree on and should then coincide regardless of the count instruction:
We measure this proximity across 5 RNN seeds using 512 held-out sequences. For each model, the distance between the states in Equation (6) is averaged and divided by the mean distance between states with adjacent values of (Section A). In all five models, states that agree on and lie close to each other regardless of the count instruction that produced them. Their mean distance is 0.165 of this reference, with full-state cosine between 0.996 and 0.998 (Panel A of Figure 3). The largest ratio of this distance to the distance between adjacent values of at any count and position is 0.862.
If we average the hidden states over the positions of each copy, we obtain one cycle center per copy. Cycle centers with the same lie even closer together, at 0.109 of the distance between cycle centers with adjacent . Thus, the states agree more closely once the variation within a copy is removed. Panel B of Figure 3 shows the cycle centers ordered by in a PCA projection obtained from separate sequences. The individual states carry the number of output steps remaining in a similarly low-dimensional form once the mean state of each emitted symbol, which dominates their leading principal components, is subtracted. A linear readout from the first two principal components of these centered states then recovers on new sequences with between 0.73 and 0.98 across the five models, which exceeds every one of 32 random two-dimensional projections per model (Panel C of Figure 3).
Figure 3. RNN states indexed by the number of output steps remaining. A, distances between states that share and under different count instructions, between states one copy apart, and between states storing other content, each relative to the distance between states one copy apart. Gray lines denote models and the blue line denotes their mean. B, cycle centers projected onto two principal components obtained from separate sequences. Arrows follow one complete copy. C, evaluation states of model seed 3, the model with the median , after subtracting the fitting-set mean state of each emitted symbol, projected onto the first two principal components of the fitting states and colored by . D, stopping and content accuracy after the affine state transformation of Equation (7) as a function of its rank. E, EOS margins after the native recurrence advances a state by two copies.
Subtracting the cycle center from each state leaves the motion through . We call the resulting sequence of centered hidden states across the positions of a centered path, since it describes how the state moves within one copy. The position-wise average of the centered paths from the other copies of the same held-out sequence recovers 0.872 of the variation of each path. Centered paths of successive copies of the same sequence have a mean cosine of 0.939. Shifting the path by one position drops this cosine to , and a path from another sequence gives . Thus, the centered path follows and depends on . This is consistent with a position that advances through and resets after each complete copy, while the number of copies remaining decreases. Note that it is equally consistent with a single countdown of .
Moving a state from to should remove one copy from the future output. We test this with an affine state transformation learned from separate fitting sequences. For each sequence in the fitting set, we take the source state and the target state at the same output step under adjacent count instructions:
Let and denote the corresponding means. A rank linear map then estimates the centered difference from the centered source state, which yields the state transformation
We evaluate on new sequences by giving it only the hidden state that lies at a copy boundary, i.e., after one copy has completed and another is started at the next time step. The unchanged RNN outputs EOS one copy earlier when continuing from . The tokens that the RNN copies before EOS are not affected by the transformation (Table 2 and Figure 3). does not isolate the number of copies remaining as a separate variable, since it acts on the complete hidden state. The complete target is reproduced less often when we only add the mean change of the fitting sequences with the same count and cycle (Table 2). Thus, most of the change that removes one copy is shared by all sequences. We measure how much depends on the stored sequence by using two further controls. The first control shuffles the target states across the fitting sequences, and the second control replaces by the mean target state. Neither control produces a single complete output. The accuracy of the copied content then drops to roughly the level expected by chance over the sixteen content tokens. Both controls keep the term in . Hence, they only remove the relation between the stored sequence and its target state.
| Intervention | Stopping | Content | Complete |
| RNN, affine state transformation from to copies remaining | |||
| Fitted transformation, new sequences | 0.995 | 0.994 | 0.944 |
| Mean change, same count and cycle | 0.829 | 0.976 | 0.728 |
| Targets shuffled across sequences | 0.456 | 0.049 | 0.000 |
| replaced by mean target state | 0.520 | 0.046 | 0.000 |
| SSM, amplitude-preserving phase rotations | |||
| Independent rotation per plane | 0.015–0.047 | 0.999–1.000 | 0.015–0.047 |
| Rotation shared by all planes | 0.268–0.316 | 1.000 | 0.268–0.316 |
| SSM, fixed direction at the first output state, | |||
| Fitted direction | 0.911–0.976 | 0.998–1.000 | 0.911–0.974 |
| Mean count-instruction difference | 0.548–0.724 | 0.950–0.991 | 0.547–0.724 |
| Norm-matched random direction | 0.000–0.001 | 0.953–0.975 | 0.000–0.001 |
Table 2. Interventions on the stopping mechanism, evaluated on held-out sequences. All accuracies are computed over every rollout. RNN entries are means over the five models, and SSM entries are ranges across the five models. The fixed-direction rows pool the targets , starting from a rollout of three copies. Since the SSM rotations preserve almost all of the copied content, their complete-sequence accuracy rounds to the same range as their stopping accuracy.
Linear SSM
The linear SSM defined in Equation (3) carries the trajectory difference through a fixed linear recurrence. After autonomous output steps,
Each transition block has the form . Let select block , and let and be the amplitude and phase of . The trajectory difference in this plane is
The output matrix sums the contribution from every plane:
The fixed relative phases make the plane contributions cancel during copying and align near the stopping time, increasing the effect of the count instruction on the EOS comparisons. Their coherence lies between 2.61 and 3.60 at EOS across the five models (Figure 4).
Figure 4. Coherence of the rotation-plane contributions at EOS in each SSM, compared with independent amplitude-preserving rotations of the planes. Stopping and content accuracy under these rotations are given in Table 2.
Amplitude and phase are separable, meaning we can intervene on them independently. Rotating the trajectory difference within each plane on its own (Section A) preserves every plane amplitude and changes only the phases relative to the other planes, which removes almost all of the intended stopping behavior while leaving the copied content intact (Table 2). In contrast, a rotation shared by all planes keeps the phases of the planes relative to each other, i.e., every plane is rotated by the same angle. The model then still stops at the intended step in between 0.268 and 0.316 of the rollouts (Table 2). Thus, the relative phase across planes is necessary for correct EOS timing. Note that it is not sufficient, since the shared rotation keeps every relative phase and still removes most of the intended stopping. Hence, the orientation of the trajectory difference relative to matters as well. Both interventions keep the amplitude of every plane fixed. Thus, they cannot be used to test whether the amplitudes contribute to EOS timing.
Because the SSM is linear, the stopping time can also be set directly by adding a fixed direction to the hidden state. For each model, we fit one direction such that adding to the first output state of a rollout with three copies produces copies of , followed by EOS. We fit on separate sequences for by requiring the target token to win every future comparison by a margin of 0.1, and hold out (Section A). On new sequences, the same direction sets the number of copies for every from 1 to 5, including , one copy more than the model was trained on (Table 2). The held-out is reached in every sequence of every model. The direction meets the pre-specified success criteria, a complete-sequence accuracy of at least 0.75 at every target and 0.9 on average, in four of the five models, and it fails at . Moving the state along the mean difference that the count instruction itself produces per copy reaches the target less often, and norm-matched random directions almost never do.
What the two mechanisms share
Because both architectures use a linear decoder to generate the EOS token, each must associate the count with EOS emission. We find that the models achieve this association via opposite changes in the trajectory difference. We measure its magnitude as and its visibility through the EOS comparisons as . During the last copy of , the trajectory difference grows in the RNN by factors between 1.52 and 2.16 and contracts in the SSM by factors between 0.61 and 0.65. Nonetheless, visibility rises in four of the five RNNs, by factors between 1.08 and 5.52, and in every SSM, by factors between 1.12 and 1.29. Every model also reaches EOS with positive alignment to the limiting comparison. The cosines lie between 0.214 and 0.622 in the RNN and between 0.527 and 0.580 in the SSM (Figure 5). The trajectory difference thus has to become visible to at the intended step, while its magnitude at that step remains a property of the recurrent transition. A separate comparison shows that choosing among several stopping times requires more principal components than copying with a fixed stopping time (Section B and Figure 7).
Figure 5. Magnitude of and its visibility through the EOS comparisons. A, magnitude of relative to the start of the last copy. B, visibility through all EOS comparisons. C, cosine with the weakest EOS comparison at the target state. D, coherence of the SSM plane contributions. Faint lines denote models and dark lines denote their means.
Figure 7. Fixed short-copy dynamics under reduced recurrences. A, agreement of the rank three system across five models. B, EOS margins from the full and reduced systems on the mixed sequences. C, contributions of output position and symbols to the exhaustive state table. D, output agreement for the reduced system, random subspaces, and the linearized RNN. Rank three preserves every output in four of the five RNNs and the stopping time in four of the five SSMs.
Discussion
Stopping with an EOS token requires the nonlinear RNNs and linear SSMs studied here to encode a timing signal in their hidden states. Using the repeat-copy task, this signal is visible as a difference between trajectories generated from the same sequence under different repeat counts. The difference persists throughout the model’s autonomously generated output, including steps at which the EOS margin remains negative. Its presence at these steps shows that the hidden dynamics contain information about the future stopping time before this information can affect the current output. The EOS margin thus acts as an observable of the effect of the model’s hidden dynamics, which implement the timing computation.
The form of this timing computation depends on the recurrent transition. The RNNs we consider here transport the trajectory difference through repeated mixing by their recurrent matrix and state-dependent scaling by the activation function’s secants. Their hidden states carry the remaining step count , which at the fixed sequence length is equivalent to the pair of , the number of copies of remaining, and , the current position within , so that our experiments do not determine whether the network represents the two indices separately. An affine state transformation estimated on separate sequences shifts a state at a copy boundary to the states with one fewer copy remaining, and the unmodified network then ends its autonomous output one copy earlier while copying every preceding token correctly.
We find that the linear SSM transports through fixed rotation planes. Contributions from these planes cancel in the EOS comparisons during copying before reinforcing near the stopping time. Independent phase rotations preserve the response amplitudes and copied symbols; most of the intended stopping behavior disappears. A shared rotation retains more stopping behavior because it preserves the relative phases. Relative phase across planes and the orientation of the response toward therefore both govern when the trajectory difference affects the EOS comparisons. A single fixed direction in the SSM state also sets the stopping time, to any number of copies from one to five.
Limitations
Because is fixed, there exists a bijection from to the remaining step count , making our nested description of the RNN’s dynamics indistinguishable from a single counter of the remaining number of output steps. The affine state transformation, applied at copy boundaries, does not distinguish these either. Varying sequence length independently of repeat count would distinguish the aforementioned axes. Besides, the explicit count tokens and deterministic sequences also make the timing response identifiable. The analysis in this paper concerns instructed halting in small tanh RNNs and linear SSMs. Experiments using gated RNNs and nonlinear SSMs will be needed to determine whether the same forms of transport occur in other recurrent architectures.
Experimental details
The nonlinear RNNs we use to train on repeat-copy have width 128 and use the default dense PyTorch initialization for . The RNNs are optimized for 100,000 updates and the SSMs for 5,000, chosen based on how long they took to converge. We train both architectures with Adam [23] using batch size 128, unit gradient clipping, and a cosine learning rate schedule from to over 100,000 updates. Training uses the cross-entropy loss. We do not provide the models with a ground truth signal or output feedback. All training, fitting, evaluation, and control streams are derived from the single data seed 123, and the five models of each cohort use the model seeds 1 to 5.
RNN transport of the trajectory difference. We evaluate five RNN seeds, 1 through 5, for the count changes , , and from the shorter target’s final copy through EOS. Separate streams supply 256 fitting sequences, 512 evaluation sequences, and the hidden unit permutations, content exchanges, and temporal shifts. The cycle comparison uses the same fitting and evaluation sequences. It reuses eight position-specific maps from the first copy at later copies and includes an absolute time control fitted separately at every output step.
SSM phase interventions. The intervention parameters are fixed in advance, and we evaluate seeds 1 through 5 on new sequences and phase draws. The phase controls rotate at 8, 4, or 1 steps before the target EOS, using either an independent rotation for each plane or one common rotation for all planes, with angles drawn uniformly from . Neither control uses output weights or token labels. Stopping accuracy is computed over all rollouts, without filtering. The fixed direction is fitted on 64 sequences and evaluated on 256 new sequences, against 32 norm-matched random directions per model.
Remaining copy geometry. RNN seeds 1 through 5 use the same 256 fitting and 512 evaluation sequences. A wider cohort, again with model seeds 1 through 5, is trained on repeat counts from one to six and evaluated on the same sequences. We estimate the affine state transformation from the fitting sequences and apply it at copy boundaries in the evaluation sequences. The cycle centers and centered paths use the same 512 evaluation sequences, with the principal component axes obtained from the 256 fitting sequences. For each sequence, the history-collapse statistic averages distances between states that share under different initial counts and normalizes them by the distance between adjacent values of . For the remaining-step readout, the symbol means, the principal components, and the linear readout of are all estimated on the fitting sequences and evaluated on the evaluation sequences, and the random projections are 32 uniformly drawn orthonormal two-dimensional bases per model. We predict each centered path from the other copies of the same sequence. The controls shift its position by one or substitute a different sequence. Of the wider cohort, four models solve every count from one to six. The fifth, seed 2, does not and is excluded. Two of the four, seeds 1 and 3, show the same state organization and the same causal edit at every count, while the other two keep substantial history dependence at high counts. The main cohort does not exclude any model. The stronger state-transformation controls use the same sequences and a separate stream for the target permutations.
Task compression. The cross-width comparison uses model seeds 1 through 5. The short-copy cohort uses model seeds 6 through 10, since seeds 1 through 5 would reproduce the width-four models of the cross-width comparison. For short copy, the principal component basis contains only and . The reduced rollout is evaluated on the held-out inputs and , against sixty-four seeded Haar control bases per model. All four inputs enter the native competence check and the state factorization.
Transport verification. Using for the count input step, at step , changes the EOS comparisons by
Across the float32 trajectories in Figure 2, the maximum state and EOS comparison discrepancies are and . The discrepancies measure the numerical agreement between the direct network rollout and the exact pairwise factorization.
Normal and nonnormal mixing. The transport in (4) combines fixed mixing by with state-dependent scaling by . To examine the fixed mixing, we obtain a spectrum-preserving normal surrogate from the real Schur decomposition by replacing each two-dimensional block with a scaled rotation that has the same conjugate eigenvalues. Its residual separates the response as
Across the five RNNs and four fixed Schur orderings, and have mean Frobenius norms of 0.357 and 0.931 relative to . After tanh scaling, their contributions have root mean square norms of 0.863 and 0.391 relative to the complete next response. The normal surrogate alone preserves 0.019 of the required EOS hitting times. The residual alone preserves 0.0004. Their sum recovers every hitting time. Removing either component from the autonomous recurrence reduces complete-sequence accuracy to zero in every model. The normal surrogate carries most of the one-step count response. The residual supplies the correction required for the hidden difference to reach the EOS region.
Advancing a trajectory. For hidden state on an unmodified network trajectory and a desired remaining count , let . Advancing the autonomous recurrence by steps reaches , as shown in the following:
Under Equation (8), the same trajectory advances by complete copies without changing the stored sequence or current position. The remaining stopping time then changes from to :
At any boundary between copies of , the endpoint map reaches the EOS state through the unmodified recurrence. The affine state transformation in Equation (7) approximates the one-copy transition from separate sequences and applies the resulting state relation to new content.
The SSM implements this advancement with the fixed matrix . For the RNN, let , , and . Define for , , and . Repeated substitution expresses as a matrix that depends on the current state:
The entries of depend on the intermediate states along the chosen trajectory. For , this matrix maps a state with four copies remaining to the state reached two copies later. Across all five models and 512 evaluation sequences per model, the unchanged recurrence produces fifteen negative EOS margins followed by a positive margin on step sixteen. Every intervening content token remains correct except one, which RNN seed 4 also miscopies without the transformation.
The exact cycle transformation defines the target state . For , a rank-one change to the recurrent matrix sends to the successor of this target in one update:
For this choice, . After a single edited update reaches , restoring continues along the original dynamics from this successor. The rank-one construction depends on the current state and target. The affine state transformation in Equation (7) is fitted on separate sequences and applies across content.
Reduced dynamics for simpler tasks
We compare fixed short-copy and cued-delay tasks by projecting the first output state onto a principal component basis obtained from separate trajectories, then advancing it through the original recurrence. The smallest rank that reproduces every target token defines the autonomous projection rank. At widths 4, 8, 16, and 32, the median short-copy ranks are 2, 3, 3, and 3 in the RNN and 3, 4, 5, and 4 in the SSM. The corresponding cued-delay ranks are 4, 6, 14, and 17 in the RNN and 4, 7, 11, and 32 in the SSM. The cued-delay rank exceeds the short-copy rank at every width in both architectures under the exact output criterion (Figure 7).
At width four, both architectures solve the short-copy and cued-delay tasks. For the fixed three-output copy program, let be the mean of the six RNN states produced by the homogeneous sequences and . The first three singular directions of these states form , with . The reduced recurrence is
Starting from the first output state, this rank three recurrence reproduces every output for the separate mixed sequences and in four of the five RNNs. Mean output agreement is 0.768 for random rank three subspaces and 0.933 for a first order linearization of at .
The corresponding SSM reduction replaces with . At rank three, it preserves the EOS position in four of the five models and reproduces the complete mixed sequences in one. The full four-dimensional state reproduces every output in every model. For output position and input symbols , the exhaustive state table has the decomposition
Here is the mean state at position , and are the two symbol effects, and is their interaction. Output position accounts for median fractions 0.759 and 0.743 of centered variation in the RNN and SSM. The corresponding interaction fractions are 0.018 and numerical zero.
References
- [1]Andrea Banino, Jan Balaguer, and Charles Blundell. PonderNet: Learning to ponder. arXiv preprint arXiv:2107.05407, 2021. URL https://arxiv.org/abs/2107.05407.
- [2]Róbert Csordás, Christopher Potts, Christopher D. Manning, and Atticus Geiger. Recurrent neural networks learn to store and generate sequences using non-linear representations. In Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 248–262. Association for Computational Linguistics, 2024. doi: 10.18653/v1/2024.blackboxnlp-1.17.DOI
- [3]Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Lukasz Kaiser. Universal transformers. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=HyzdRiR9Y7.arxiv.org/abs/1807.03819
- [4]Yanai Elazar, Shauli Ravfogel, Alon Jacovi, and Yoav Goldberg. Amnesic probing: Behavioral explanation with amnesic counterfactuals. Transactions of the Association for Computational Linguistics, 9:160–175, 2021. doi: 10.1162/tacl_a_00359.DOI
- [5]Maha Elbayad, Jiatao Gu, Edouard Grave, and Michael Auli. Depth-adaptive transformer. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=SJg7KhVKPH.
- [6]Jeffrey L. Elman. Finding structure in time. Cognitive Science, 14(2):179–211, 1990. doi: 10.1207/s15516709cog1402_1.DOI
- [7]Michael Figurnov, Artem Sobolev, and Dmitry Vetrov. Probabilistic adaptive computation time. Bulletin of the Polish Academy of Sciences: Technical Sciences, 66(6):811–820, 2018. doi: 10.24425/bpas.2018.125928.DOI
- [8]Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. Causal abstractions of neural networks. In Advances in Neural Information Processing Systems, volume 34, pages 9574–9586, 2021.arxiv.org/abs/2106.02997
- [9]Satchel Grant, Noah D. Goodman, and James L. McClelland. Emergent symbol-like number variables in artificial neural networks. Transactions on Machine Learning Research, 2025. URL https://arxiv.org/abs/2501.06141.
- [10]Alex Graves. Adaptive computation time for recurrent neural networks. arXiv preprint arXiv:1603.08983, 2016. URL https://arxiv.org/abs/1603.08983.
- [11]Alex Graves, Greg Wayne, and Ivo Danihelka. Neural Turing machines. arXiv preprint arXiv:1410.5401, 2014. URL https://arxiv.org/abs/1410.5401.
- [12]Edward Grefenstette, Karl Moritz Hermann, Mustafa Suleyman, and Phil Blunsom. Learning to transduce with unbounded memory. In Advances in Neural Information Processing Systems, volume 28, 2015.arxiv.org/abs/1506.02516
- [13]Andreas Grivas, Nikolay Bogoychev, and Adam Lopez. Low-rank softmax can have ungrammable classes in theory but rarely in practice. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6738–6758, 2022. doi: 10.18653/v1/2022.acl-long.465.DOI
- [14]Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023. URL https://arxiv.org/abs/2312.00752.
- [15]Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=uYLFoz1vlAC.arxiv.org/abs/2111.00396
- [16]Doron Haviv, Alexander Rivkind, and Omri Barak. Understanding and controlling memory in recurrent neural networks. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 2663–2671. PMLR, 2019. URL https://proceedings.mlr.press/v97/haviv19a.html.arxiv.org/abs/1902.07275
- [17]Mikael Henaff, Arthur Szlam, and Yann LeCun. Recurrent orthogonal networks and long-memory tasks. In Proceedings of the 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 2034–2042. PMLR, 2016. URL https://proceedings.mlr.press/v48/henaff16.html.arxiv.org/abs/1602.06662
- [18]John Hewitt and Percy Liang. Designing and interpreting probes with control tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2733–2743. Association for Computational Linguistics, 2019. doi: 10.18653/v1/D19-1275.DOI
- [19]John Hewitt and Christopher D. Manning. A structural probe for finding syntax in word representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4129–4138. Association for Computational Linguistics, 2019. doi: 10.18653/v1/N19-1419.DOI
- [20]Armand Joulin and Tomas Mikolov. Inferring algorithmic patterns with stack-augmented recurrent nets. In Advances in Neural Information Processing Systems, volume 28, 2015. URL https://proceedings.neurips.cc/paper/2015/hash/26657d5ff9020d2abefe558796b99584-Abstract.html.arxiv.org/abs/1503.01007
- [21]Andrej Karpathy, Justin Johnson, and Li Fei-Fei. Visualizing and understanding recurrent networks. arXiv preprint arXiv:1506.02078, 2015. URL https://arxiv.org/abs/1506.02078.
- [22]Arjun Karuvally, Peter DelMastro, and Hava T. Siegelmann. Episodic memory theory of recurrent neural networks: Insights into long-term information storage and manipulation. In Proceedings of 2nd Annual Workshop on Topology, Algebra, and Geometry in Machine Learning (TAG-ML), volume 221 of Proceedings of Machine Learning Research, pages 371–383. PMLR, 2023. URL https://proceedings.mlr.press/v221/karuvally23a.html.
- [23]Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015. URL https://arxiv.org/abs/1412.6980.
- [24]Mingbo Ma, Renjie Zheng, and Liang Huang. Learning to stop in structured prediction for neural machine translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1884–1889. Association for Computational Linguistics, 2019. doi: 10.18653/v1/N19-1187.DOI
- [25]Niru Maheswaranathan, Alex Williams, Matthew Golub, Surya Ganguli, and David Sussillo. Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics. In Advances in Neural Information Processing Systems, volume 32, 2019. URL https://proceedings.neurips.cc/paper/2019/hash/d921c3c762b1522c475ac8fc0811bb0f-Abstract.html.arxiv.org/abs/1906.10720
- [26]Kenton Murray and David Chiang. Correcting length bias in neural machine translation. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 212–223. Association for Computational Linguistics, 2018. doi: 10.18653/v1/W18-6322.DOI
- [27]Benjamin Newman, John Hewitt, Percy Liang, and Christopher D. Manning. The EOS decision and length extrapolation. In Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 276–291. Association for Computational Linguistics, 2020. doi: 10.18653/v1/2020.blackboxnlp-1.26.DOI
- [28]Antonio Orvieto, Samuel L. Smith, Albert Gu, Anushan Fernando, Caglar Gulcehre, Razvan Pascanu, and Soham De. Resurrecting recurrent neural networks for long sequences. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 26670–26698. PMLR, 2023. URL https://proceedings.mlr.press/v202/orvieto23a.html.arxiv.org/abs/2303.06349
- [29]Jimmy Smith, Scott Linderman, and David Sussillo. Reverse engineering recurrent neural networks with Jacobian switching linear dynamical systems. In Advances in Neural Information Processing Systems, volume 34, pages 16700–16713, 2021. URL https://proceedings.neurips.cc/paper/2021/hash/8b77b4b5156dc11dec152c6c71481565-Abstract.html.arxiv.org/abs/2111.01256
- [30]Jimmy T. H. Smith, Andrew Warrington, and Scott W. Linderman. Simplified state space layers for sequence modeling. In International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=Ai8Hw3AXqks.arxiv.org/abs/2208.04933
- [31]Pavel Sountsov and Sunita Sarawagi. Length bias in encoder decoder models and a case for global conditioning. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1516–1525. Association for Computational Linguistics, 2016. doi: 10.18653/v1/D16-1158.DOI
- [32]David Sussillo and Omri Barak. Opening the black box: Low-dimensional dynamics in high-dimensional recurrent neural networks. Neural Computation, 25(3):626–649, 2013. doi: 10.1162/NECO_a_00409.DOI
- [33]Yilin Yang, Liang Huang, and Mingbo Ma. Breaking the beam search curse: A study of (re-)scoring methods and stopping criteria for neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3054–3059. Association for Computational Linguistics, 2018. doi: 10.18653/v1/D18-1342.DOI






