WorksPhysics and Astronomy
Memory flows: geometry and dynamics of sequential retrieval in input-driven Hopfield networks
Abstract
Associative memory models classically describe retrieval as convergence to stored patterns, but the theory of sequential retrieval remains comparatively limited. We develop a two-timescale input-driven Hopfield network in which fast associative retrieval is modulated by a slow feedback variable evolving over a prescribed memory-transition graph. This yields autonomous, graph-constrained transitions while preserving analytical tractability. Using slow-fast and geometric singular perturbation theory, we characterize memory fixation via memory manifolds and identify the geometric regions that govern stability loss and escape from retrieved memories. We further derive reduced slow dynamics and explicit criteria for transition onset, self-sustained retrieval, and collapse, expressed through escape times, gain thresholds, and fixed points. Together, these results provide a tractable framework for analyzing feedback-driven sequential retrieval in Hopfield networks.
Introduction
Associative memory has been a foundational concept in theoretical neuroscience and machine learning for over four decades. In its classical formulation, associative memory retrieval is modeled as convergence toward stable equilibria of an energy landscape [1, 2, 3], whereby a corrupted cue relaxes to a stored pattern through recurrent dynamics, providing a canonical example of distributed computation in neural networks. A large body of work has focused on characterizing the memory capacity of such models [4, 5, 6], revealing a fundamental limitation of the original Hopfield network, whose capacity scales sub-linearly with the number of neurons. Modern Hopfield models [7, 8, 9] overcome this limitation by achieving exponential capacity within an energy-based framework, and have been linked to contemporary architectures such as Transformers [10, 11] and diffusion models [12, 13]. However, they largely remain static retrieval systems: dynamics stop at convergence, so sequential retrieval requires repeated reinitialization. This contrasts with cognitive and algorithmic tasks, including those addressed by Transformers, where reasoning depends on structured transitions across sequences of memories.
Pioneering works by [14, 15] already identified the limitation of a theory accounting only for static retrieval. They proposed a continuous-time mechanism for sequential memory transitions via delayed asymmetric interactions, which introduce a destabilizing dynamic component driving transitions between attractors. This idea has since motivated extensive efforts to model memory transitions [16, 17], establishing mean-field results on the gain sustaining sequentiality. More recently, [Karuvally et al., 2023] introduced a continuous-time, slow-fast energy-based model with delayed asymmetric interactions, while [19] investigated long-sequence retrieval and capacity in a discrete-time setting. The softmax-based model of [20] further provides capacity results and escape-time estimates based primarily on energy arguments. Related approaches use slowly evolving parameters to control attractor switching [21, 22]. Our aim is complementary: rather than improving retrieval performance, we focus on developing an analytically tractable model that provides a mechanistic understanding of sequential memory transitions.
In this paper, we propose and analyze a two-timescale extension of the input-driven plasticity (IDP) Hopfield model of [23] that enables autonomous sequential retrieval. In the original IDP Hopfield model, external inputs modulate the saliency of stored memories and thereby reshape the retrieval landscape. Here, we replace this externally prescribed modulation with a slow feedback variable whose dynamics are driven by the currently retrieved memory and structured by a transition graph. The slow feedback can be viewed as a phenomenological representation of activity-dependent biological processes, such as short-term synaptic plasticity, neuronal adaptation, or neuromodulatory gain control, that modulate memory stability [24]. The resulting sequential IDP Hopfield network produces graph-constrained transitions between memories while remaining analytically tractable. Our contributions are threefold. First, using slow-fast and geometric singular perturbation theory, we characterize the geometry of memory fixation through memory manifolds and the attracting and saddle-type regions that govern entry and escape. Second, we derive reduced saliency dynamics and explicit escape conditions, including transition times, gain thresholds, and collapse regimes. Third, we show how these conditions support self-sustained sequential retrieval over structured memory graphs. Figure 1 provides a schematic overview of the sequential memory transition mechanism introduced in this work. Together, these results establish a tractable geometric and dynamical framework for sequential retrieval in input-driven Hopfield networks, providing explicit conditions under which slow internal feedback induces, sustains, or fails to sustain transitions between memories.
Figure 1. Schematic illustration of the proposed two-timescale mechanism for sequential memory transitions. At the initial time, the fast state variable is at a stable equilibrium corresponding to memory . The slow variable progressively destabilizes this equilibrium while simultaneously stabilizing the equilibrium associated with memory . At the escape time , the equilibrium corresponding to loses stability and the system transitions toward . This transition occurs only when the gain parameter of the slow dynamics exceeds a critical threshold . We analytically characterize both and the resulting escape time .
The IDP Hopfield model for sequential retrieval
The IDP Hopfield model introduced in [23] extends the classical Hopfield model by incorporating an explicit modulation of synaptic interactions through external inputs. This formulation enables a controlled and interpretable mechanism for memory retrieval, where the input does not merely select an initial condition but directly shapes the energy landscape governing the dynamics. We next briefly review the IDP Hopfield model, deferring technical details to Appendix § 9, and then introduce a sequential extension that enables autonomous, structured transitions between stored memories.
The IDP Hopfield model. The IDP Hopfield dynamics, illustrated schematically in Figure 2(A), are
Figure 2. (A) Block diagram of the IDP Hopfield model; setting recovers (1). (B) Block diagram of the two-timescale sequential IDP Hopfield model (3). (C) Example of a deterministic transition graph , where nodes represent stored memories and edges encode admissible transitions. The graph decomposes into multiple connected components (highlighted by dashed red contours), each defining a distinct memory sequence; one, selected by , is highlighted in blue. (D) One-timescale model with orthogonal memories (): sequential retrieval proceeds through mixed memory states, with partial overlaps and unpredictable escape times across memories. (E-F) Two-timescale sequential IDP Hopfield model (). (E) Orthogonal memories: transitions occur between individual memories without mixing, with uniform escape times and overlaps reaching their maximal value, indicating exact memory alignment. (F) Random memories: transitions occur without mixing, reaching maximum overlap, but escape times are no longer uniform. Numerical experiments are shown for visualization purposes with memories and a limit cycle over 5 of them, with , and whose overlaps are highlighted in different colors in the plots.
where is the network state and is a slowly varying external input. The map is a saliency function that acts as a positive filter on the input , while is the neural activation function. The synaptic matrix depends on the filtered input through
where is a set of mutually orthogonal stored patterns. We stress that the orthogonality constraint is necessary to derive rigorous results on the geometry and dynamics of the discussed model, but numerical experiments support their validity also in the canonical case of random memories.
Throughout the paper, we assume that acts componentwise and homogeneously, i.e., for some scalar function , which is even and monotone with respect to . Similarly, acts componentwise as , where is odd, continuously-differentiable and monotone non-decreasing (e.g., ). Under these assumptions, the map assigns a non-negative saliency coefficient to each stored pattern, thereby modulating its contribution to the synaptic matrix , while retains the standard symmetry and monotonicity properties of Hopfield-type activation functions. The classical Hopfield model is recovered as a special case when all input-dependent coefficients , , are all equal to one, in which case the synaptic matrix reduces to a fixed matrix independent of the input.
Under the above-mentioned standard assumptions on , the IDP Hopfield model admits the energy function [3, 8]
This energy provides a geometric interpretation of retrieval: the input-dependent saliencies reshape the energy landscape, thereby controlling the existence and stability of memory equilibria.
A key feature of the IDP formulation is that the input-dependent coefficients directly control the existence and stability of memory states. In general, the system admits multiple equilibria, some of which are associated with the stored patterns.1 In particular, to each memory vector one can associate a memory equilibrium of the form , which exists if and only if . In that case, the scaling factor is uniquely determined by , where . The stability of these equilibria is also governed by the coefficients : if , the corresponding equilibrium is stable only if is sufficiently large relative to the maximum coefficient (see Appendix § 9 for details). Overall, the coefficients provide a direct and tunable mechanism to determine which memories exist and are stable, thereby enabling selective activation and suppression within the same network.
The Two-Timescale Sequential IDP Hopfield Model. While the IDP formulation enables input-modulated retrieval and sharp control over memory stability, it remains fundamentally static in nature: for a fixed input, the dynamics converge to a single equilibrium and do not exhibit autonomous transitions between memories. In particular, the model lacks an internal mechanism to encode temporal or semantic relationships among stored patterns.
To overcome this limitation, we introduce a minimal two-timescale extension of (1) that couples the fast retrieval dynamics to a slower internal state:
where is a gain parameter and encodes structured associations among memories. Specifically, is the adjacency matrix of a directed graph , whose nodes represent stored memories and whose edges represent admissible transitions, with if and only if . We focus on deterministic graphs, where each node has out-degree at most one, and assume , so that evolves rapidly while provides slow modulation.
Figure 2(B) illustrates the block diagram of the sequential IDP Hopfield model (3). In contrast to the original IDP formulation (1), where the input directly modulates the synaptic weights, here the modulation is mediated by the slow, feedback-driven variable , which endows the system with an internal memory of past retrieval states. Figure 2(C) shows an example of deterministic graph . Such graphs may consist of multiple connected components, each corresponding to a distinct memory sequence that can be selectively retrieved through appropriate choice of the initial condition .
In the regime , a clear separation of timescales emerges in (3): the fast variable rapidly relaxes toward a memory determined by the current , while the slow variable evolves in response to the retrieved pattern and progressively reshapes the energy landscape. This interaction gives rise to autonomous sequential retrieval, whereby the system transitions between stored memories according to the graph . As illustrated in Figure 2(E-F) on a simple cyclic graph with memories, the proposed two-timescale sequential IDP Hopfield model exhibits robust and predictable sequential retrieval when is sufficiently large. For orthogonal memories (Figure 2(E)), transitions occur directly between individual memories without intermediate mixing: overlaps reach their maximal value, indicating exact memory alignment, and escape times are uniform across memories. For random memories (Figure 2(F)), the dynamics similarly transition without mixing and attain maximal overlap, although escape times are no longer uniform. To highlight the role of timescale separation, we compare this behavior with the classical sequential Hopfield dynamics obtained in a single-timescale setting, , where transitions are encoded directly in the synaptic matrix [25]. As shown in Figure 2(D), this model exhibits qualitatively different dynamics: trajectories evolve through mixed states with partial overlaps, display irregular escape times, and are highly sensitive to the gain . Depending on its value, the system may collapse to the origin, converge to a fixed point, or sustain a limit cycle, consistent with previous observations.
Finally, we point out that (3) admits a natural modern Hopfield reformulation, detailed in Appendix § 10.
Memory manifolds: the highway of sequential retrieval
Memory manifolds: the highway of sequential retrieval
Sequential memory retrieval is the continuous evolution of neural activity in state space, from one stable memory state to another one. The direction of motion is shaped by the semantic relationships between memories, which at any given time constrain neural activity to a lower-dimensional memory subspace. As semantic relationships change, the binding is broken, allowing neural trajectories to leave one memory subspace and evolve toward another.
From a geometric perspective, as sketched in Figure 3(A), sequential retrieval therefore requires two complementary structures: (i) memory subspaces (red and shaded surfaces) that transiently trap neural activity (blue curves), and (ii) entry and exit pathways that enable transitions (purple branches) between them. In this section, we formalize this intuition using slow-fast and geometric singular perturbation theory (GSPT) [26, 27]. Leveraging the analytical tractability of the IDP Hopfield model, we provide a geometric characterization of how sequential retrieval arises in high-dimensional associative memory systems.
Figure 3. (A) Sketch of the transitions between different memory manifolds in the slow-fast Hopfield model. The blue curves corresponds to the slow movement of the system close to a stable memory manifold, whereas the purple curves correspond to the fast switching between different stable memory manifolds. Each memory manifold is decomposed into its attracting (red) and saddle-type (green) memory manifolds , . (B) Simulated trajectory projected onto the fast subspace in a slow-fast Hopfield model storing two memories. Continuous curves indicate the flow along memory branches (magenta) and (blue), whereas dashed curves indicate rapid transitions from one memory branch to another. Switching occurs after the trajectory has moved from the attracting part of the memory branches and , respectively, to the saddle-type section and . The black dots indicate the relevant point of separation between and , and and , respectively (red/green). (C) Temporal series of the variable . Colors and line types as in panel B.
Geometric singular perturbation theory Fenichel, [26] provides a theoretical framework to analyze dynamical systems with well-separated time scales. The central idea is that, when fast and slow variables interact weakly, the full dynamics can be approximated as a sequence of slow movements, solutions of a simpler system, interspersed by fast jumps. We introduce the time-scale parameter and consider the slow-fast system
We refer to the -ODE with the term fast subsystem and denote with its right-hand side. Taking the limit constrains the fast dynamics onto , yielding the reduced system. The algebraic constraint effectively tracks the minimum over the energy as the slow neural feedback evolves, and admits multiple solutions, reflecting that Hopfield networks are designed to store multiple stable patterns. In slow-fast theory, the set of the fast-subsystem equilibria is called critical set, and defined as
Memory branches, memory manifolds, and stability. Given the fast dynamics with stored memories, a memory branch describes a -dimensional subspace in which neural activity can be temporarily confined. In the IDP Hopfield model, these memory branches take the explicit form
for and . The normally hyperbolic subsets and of are known as the attracting and saddle-type memory manifolds. They can be identified by studying the stability of the fast subsystem equilibria in , which are related to the saliency weights. Another relevant branch of is the origin manifold,
GSPT ensures that, for , each (, , and ) perturbs to a nearby invariant manifold (, , and ) of the full system (4). In particular, the manifolds and provide a precise geometric description of memory fixation and transitions.
Fixation, loss of stability, and transitions. Sequential retrieval requires understanding how trajectories enter and leave , which consists in studying the stability of its constitutive memory manifolds. As presented in Figure 3(B-C), during retrieval of the memory , the neural activity converges to rapidly, fixation occurs, and escape is impossible. As the saliency decreases, an escape direction appears when crosses the threshold from above. Geometrically, this corresponds to crossing (an -perturbation of) a non-hyperbolic point on (black circle in Figure 3(B)), corresponding to the trajectory moving from the attracting manifold to the saddle manifold . Beyond this point, a typical trajectory is ejected via an unstable fiber of towards a new attracting manifold associated to the memory . However, the jump towards a different memory can occur only if the associated . Otherwise, if for all it holds then the only attracting manifold is , which drives the full system towards the origin.
Slow saliency dynamics: the driving engine
In § 4, we characterized the geometric structure underlying memory retrieval, including the manifolds that support retrieval and the instabilities and fibers that enable transitions between them. We now turn to how the reduced slow dynamics in (4) govern sequential transitions as a function of the gain parameter , the initial condition , and the structure of the transition matrix . We present the main results below, deferring technical derivations and proofs to Appendix § 12.
Reduced dynamics and retrieval regimes. Consider the sequential IDP Hopfield model (3) in the limit . The fast variable rapidly relaxes to a stable equilibrium determined by the current saliency . Substitution into the slow equation of (3) gives
This reduced equation describes the slow flow along the branches of the fast Hopfield subsystem: for each fixed , the fast dynamics select an equilibrium and sticks to the associated attracting manifold until it loses stability. Only then the trajectory may jump to another memory branch, as pictorially depicted in the introductory Figure 1. By the geometric picture of § 4, the saliency state selects which memory branches contain attracting manifolds and can therefore support fixation. This gives three regimes:
(i) No active memories. If for all , no memory equilibrium exists and trajectories converge to the origin by evolving in .
(ii) Unique active memory. If there is a unique such that , there is only active memory equilibrium, and trajectories rapidly converge to . This equilibrium is stable and given by
(iii) Competing memories. If several components exceed the threshold, multiple memory equilibria may coexist, allowing transitions between the corresponding memory states.
Two-memory reduction. We next restrict to the case in which at most two memories compete. Specifically, we assume that, along the relevant portion of the trajectory, only two components of are significantly different from zero. Without loss of generality, let the initial time be and assume , where denotes the -th vector of the canonical basis of and is the initial value of . 2
If , no memory is active: the fast state converges to , and the slow dynamics reduce to , driving all saliencies exponentially to zero in . We therefore focus on , where is initially active and the fast state is trapped on its associated memory equilibrium,
In this case, the system initially evolves on , and the reduced saliency dynamics (8) satisfy
where is the unique successor of in the transition graph . Hence for all . Since only is nonzero, all components except and remain identically zero, with . The successor saliency component satisfies
Note that, on the interval where the source memory exists, , so is driven by a positive input. Thus, the transition from to reduces to a two-dimensional saliency dynamics: a decaying source component and a positively driven successor component.
Transition criterion and induced map. We now determine whether the two-dimensional saliency dynamics produces a transition to the successor memory or decays to inactivity. Let be the time at which the source memory ceases to exist, namely . Since , the source memory exists for and disappears for . The outcome is determined by whether the successor saliency becomes large enough before this loss of existence occurs. Two possible cases arise (see also Figure 6 of the Appendix for a graphical illustration).
Figure 6. Graphical illustration of the two possible outcomes of the source-successor saliency dynamics. Left: no transition occurs and the dynamics decay to inactivity. Right: the successor saliency catches up with the source saliency, triggering a transition between memories.
Case 1: no transition. Assume that
Then the successor component never catches up with the source component before the source memory disappears. Hence, after , both saliencies fall below the existence threshold and the system enters the inactive regime , where the origin is the only stable equilibrium.
Case 2: transition. Assume instead that there exists a first time such that
In this case, the following result characterizes the loss of stability of the source branch from to , and the induced update of the successor saliency. The proof is given in Appendix D.
Theorem 1 (Escape condition and saliency update). Under the above condition, there exists an escape time at which the equilibrium of the fast Hopfield subsystem associated with the source memory loses stability. Moreover, is the smallest solution in of
*where . 3 At escape, the successor saliency is*
At time , the trajectory leaves the branch associated with the source memory and jumps to the manifold associated with the successor memory . Theorem 1 therefore defines a one-dimensional saliency update . For later use, it is convenient to rewrite this update in terms of . Since
the transition map is equivalently described by4
where
Thus, each saliency update is computed by first selecting the smallest satisfying , when such a solution exists, and then setting . If no such exists, the transition fails and the trajectory collapses to the inactive state.
Analytical illustration and numerical validation
We illustrate the results of § 5 with an analytically tractable example and further assess their validity and robustness through numerical experiments. See also Appendix § 17.
We consider the saliency update (12)-(13) with the following nonlinearities
In particular, we will refer to such as SmoothF. On the positive branch , we have , hence . Therefore and for . Substituting into (13), for and ,
Hence the equation is equivalent to
Thus, given , the update is computed by first selecting the smallest admissible solution of this equation, and then setting . If no such solution exists, the transition fails.
Fixed points of this scalar map characterize self-sustained retrieval across repeated transitions: means that each jump regenerates the same saliency amplitude. Since , fixed points satisfy , and the implicit equation for becomes
Thus fixed points exist when the level intersects the curve . Since has a unique maximum at , with , fixed points exist only for . In particular, for , two fixed points appear and , , with unstable and stable. Since and at the stable fixed point, the asymptotic escape time is . Further details are given in Appendix § 12.
Fig. 4 illustrates the preceding analysis and extends it numerically to the case . Fig. 4(B) compares the SmoothF and Tanh activation functions, while Fig. 4(A) shows the induced saliency update map for SmoothF, revealing the emergence of two fixed points for gains exceeding the critical value . Fig. 4(C)-(F) present representative trajectories for subcritical and supercritical values of using the SmoothF activation. For orthogonal memories (Fig. 4(C)-(D)), subcritical gains lead to collapse into an inactive state, whereas supercritical gains give rise to self-sustained sequential retrieval. In contrast, for random memories (Fig. 4(E)-(F)), both gain regimes exhibit sustained transitions, likely due to cross-talk noise and the consequent lack of entrainment to a fixed asymptotic value. In the subcritical regime, the resulting -oscillations have small amplitude and may therefore lead to collapse only on longer timescales. Finally, Fig. 4(G)-(H) show that the qualitative features of the slow -dynamics persist with Tanh activation and random memories.
We next validate the theoretical escape-time predictions through simulations of the full model. For orthogonal memories, numerical escape times closely match the predictions (Figure 5(A)), while the switching times , defined as the times at which the network transitions to the successor memory, occur slightly later due to the finite time required to leave the unstable memory state. Agreement between theoretical and numerical asymptotic escape times improves with (Figure 5(B)). Adding Gaussian noise to the fast dynamics reduces the mean switching delay , eventually making it negative due to premature switching (Figure 5(C)).
Figure 5. (A) Theoretical and numerical escape times and observed switching times across consecutive transitions (, ). (B) Asymptotic escape time versus : theory and numerical estimates. (C) Mean switching delay versus Gaussian noise amplitude . (D) Probability of correct transitions in the plane. (E) Asymptotic escape time versus and memory bias . (F) Asymptotic saliency amplitude versus and . The numerical configuration is the same as in Figure 4. Panels (A-C) use orthogonal memories, while (D-F) use biased random memories, with 10 independent realizations per parameter pair. In (B), asymptotic escape times are estimated over the last 8 time units of simulations of duration (approximately 5-7 transitions). Panel (C) uses Gaussian noise with standard deviation and 64 realizations per noise amplitude. Hatched regions in (D-F) indicate no successful transitions.
We then investigate robustness to non-orthogonal memories by independently sampling biased binary patterns, as in [19], according to , where induces an expected normalized pairwise overlap of . Figure 5(D) shows that the critical gain for successful retrieval increases only slightly with memory bias , while retrieval reliability decreases sharply for . No successful transitions are observed for over the investigated gains. Conditional on successful transitions, both the asymptotic escape time (Figure 5(E)) and saliency amplitude (Figure 5(F)) increase with . As memory bias increases, escape times decrease, while saliency remains approximately constant. Overall, these results suggest that the theoretical estimates derived for orthogonal memories remain informative for moderately correlated memories (), while stronger correlations lead to significant deviations and eventually compromise sequential retrieval. Additional numerical results for the Tanh activation are provided in Appendix § 13.
Limitations and future research
While this work presents an analytically tractable framework for sequential retrieval in Hopfield networks, several important limitations remain. First, our analysis is restricted to canonical Tanh-type activation functions and does not address the practically relevant case . This regime is of particular interest, as it directly links Hopfield dynamics to self-attention mechanisms in Transformer architectures [10] and is known to enable exponential memory capacity [9]. Extending the present theory to non-diagonal activation functions represents a promising direction for future work, with the potential to provide dynamical interpretations of inference in modern Transformer-based models. Second, although we provide a qualitative geometric characterization of the invariant sets and unstable structures supporting sequential retrieval, we do not yet precisely quantify the ranges of slow-fast timescale separation for which transitions emerge intrinsically, without reliance on numerical error or small-amplitude noise to trigger adherence to unstable fibers. A more refined geometric analysis is needed to delineate the parameter regimes that permit robust, noise-free sequential transitions. Third, while our results establish conditions for sequential retrieval, they do not characterize how retrieval reliability scales with the number of stored memories and the length of the retrieved sequence. Quantifying these capacity limits remains an important open problem, preliminarily explored numerically in Appendix § 14. Fourth, our theoretical analysis is restricted to deterministic transition graphs with at most one outgoing edge per memory. Extending the theory to branching graphs is another promising direction, as preliminary simulations in Appendix § 15 suggest that memory overlap can influence branch selection. Finally, the role of external inputs in controlling retrieval sequences remains largely unexplored. Developing a theoretical framework for steering memory transitions through external signals is an interesting direction for future research, illustrated by the associative-memory finite-state machine in Appendix § 16.
Appendix
The IDP Hopfield model
The IDP Hopfield model presented in Betteti et al. [ [23] is characterized by the O.D.E.
where is the synaptic matrix of neural interaction built from the memories of the model. The memories and associated synaptic matrix obey the following assumption.
Assumption 1 (Memory and synaptic matrix). Let and define the set of orthogonal memories
where is the Kronecker delta for indices and . For positive we define the input-modulated synaptic matrix as
The orthogonal memories constraint is necessary to derive rigorous results on existence and stability of fixed points, but can be relaxed to orthogonality in expectation in implementations:
Remark 1. Notice that by constructing the memory matrix
we can rewrite the input-modulated synaptic matrix as
The canonical synaptic matrix of the Hopfield model is a specific instance of (18) for the choice (or a scalar multiple of it).
We now introduce an assumption on the saliency function going from the space of the slow neural feedback to the saliency space such that .
Assumption 2. The saliency function satisfies the following properties:
(i) (diagonality and homogeneity) for all we have ;
(ii) (symmetry and positivity) , with ;
(iii) (symmetric monotonicity) is monotonically increasing on .
Likewise, we introduce some smoothness assumptions on the activation function of the IDP Hopfield network, which will be useful for the characterization of the existence and local stability conditions for the equilibria of (15).
Assumption 3. The activation function satisfies the following properties:
(i) (diagonality, homogeneity and smoothness) for all , we have with ;
(ii) (odd and saturating) for all and .
(iii) (bounded monotonicity) for all and ;
(iv) (monotonicity of the derivative) is monotonically decreasing on .
We will now state some of the main analytical results presented in [23 Supplementary Information], starting from the existence conditions for the equilibria of (15). The presented results are stated with respect to the saliency states .
Theorem 2 (Existence of retrievable memories). Let be a memory as in Assumption 1. Then is an equilibrium for (15) for some unique if and only if . In this case, satisfies
We now provide the main result on the local stability of the equilibria for the fast IDP Hopfield dynamics.
Theorem 3 (Local stability of equilibria and memories). Consider the dynamics (15). Then
an equilibrium is locally exponentially stable if and only if
if with is an equilibrium, then is locally exponentially stable if and only if
In particular, the study of the local stability condition allows to provide exact conditions on the points where the Jacobian becomes has eigenvalues with null real part, which will later be useful for the characterization of the memory manifold (see Section C). We will restrict ourselves to equilibrium conditions characterized by , as the case is associated to the so called anti-memory and is a symmetric artifact of classical Hopfield models. Notice in particular that by property 3(iv) the derivative is locally invertible in , and we can define the following stability function.
Definition 1 (Saliency stability function). We define the saliency stability function as
Before providing the key results on the eigenvalues of the Jacobian of the IDP Hopfield vector field, we establish the basic invertibility and monotonicity properties of the saliency stability function.
Lemma 1 (Invertibility of the saliency stability function). The saliency stability function is invertible, and in particular
Furthermore, it holds that
;
.
Proof. The paragraph above Definition 1 establishes the invertibility of in . Considering then the function we immediately observe that for any we have
From Assumption 3(iv) we have that is monotonically decreasing on and in particular we have
Consequently for all and in particular is invertible on . Taking we observe that
The proof of points 1-2 is straightforward, and relies on the fact that for all , as proven in (28).
:
We are interested in proving that
Taking , we can express , and therefore our condition reduces to
:
We are interested in proving that
Taking , we can express , and therefore our condition reduces to
The saliency stability function allows us to state an important proposition on the eigenvalues of the Jacobian of [15] at the equilibrium points.
Proposition 1 (Eigenvalues of the IDP Hopfield Jacobian). Let and define . Let such that for the equilibrium exists. Then the Jacobian matrix
has one eigenvalue with iff
Proof. From [23 Theorems 2, SI] we have that
This has the same spectrum of the matrix
This matrix has the same inertia [28] of the matrix
that has spectrum
From this it is immediate to see that has null eigenvalues iff there exists such that
Exploiting the invertibility of in , we get the equivalent condition
Since , then exploiting the uniqueness of in the equilibrium condition in (22) we have that the zero eigenvalue condition in (40) coincides with
In particular, notice that from Lemma 1 for all it holds that , so that whenever at least a memory exists, at least one is also locally exponentially stable. Notice in addition that the number of eigenvalues with positive and negative real part is obtained by counting the instances where the condition is satisfied for .
A modern reformulation of the sequential IDP Hopfield model
The IDP Hopfield model admits a modern Hopfield Krotov and Hopfield, [8] interpretation in which retrieval is decomposed into three interacting processes: a memory layer that reads out the similarity between the current feature state and the stored memories, a saliency layer that modulates this readout, and a feature layer that reconstructs the retrieved representation Betteti et al., [23]. In the original IDP model, the saliency layer is driven by the external input. Here we show that the sequential IDP Hopfield model (3) admits the same interpretation, with the saliency layer driven instead by internal graph-structured feedback.
We introduce an auxiliary memory-index variable and consider
The variable is the memory-layer activity: its components encode the alignment between the current activated feature state and the stored memories. The map is a slow feedback map prescribing the target saliency profile toward which relaxes.
Under the timescale separation , the memory layer rapidly satisfies
Substituting this quasi-steady relation into the feature dynamics gives
which is exactly the fast equation of the sequential IDP Hopfield model (3).
The slow equation in (3) is recovered by choosing
Indeed, replacing by its fast quasi-steady value yields
which coincides with the slow feedback dynamics of (3).
This reformulation clarifies the relation with the modern IDP interpretation. In the original IDP model, saliencies are prescribed by an external input decomposition. In the sequential IDP model, saliencies are generated autonomously by the retrieved memory through the transition graph.
Memory manifolds: pouring the concrete
In the classic slow-fast characterization from geometric singular perturbation theory (GSPT) presented in Fenichel [ [26], the timescale separation between coupled dynamics is extremized in order to derive geometric information about the loci in state-space where the system evolves. In the case of the IDP Hopfield model, the coupled system can be written as
and taking the limit , the problem simplifies to studying how the dynamics
evolve under the solution constraint, parametrized as a function of the slow variable , . We start by studying the solutions of the algebraic constraint, which characterize the critical set
The origin manifold
Memory branches and memory manifolds
Exploiting Theorem 2 for the existence of equilibria in the IDP Hopfield model, we proceed to define the memory branches, which are disjoint sets in associated each to a different memory.
Definition 2. Let . Then the memory branch is the -dimensional set defined as
Notice that the memory branch is not defined for , as the equilibria associated to the memory does not exists. The memory manifold is then defined as a -dimensional manifold comprising the points in where normal hyperbolicity holds.
Definition 3. Let and assume . Then the memory manifold is the set of points such that
Of particular relevance are two disjoint regions of the manifold , which play qualitatively distinct roles in shaping the system’s long-term behavior. Rather than pursuing formal definitions, we give an intuitive description aimed at conveying their dynamical function:
Attracting manifold : this region consists of points that act as local sinks for the dynamics. Trajectories initialized in a neighborhood of these points in the ambient space are drawn toward the manifold and remain confined to it over time. Geometrically, behaves as a stabilizing surface that captures and retains nearby trajectories.
Saddle manifold : this region contains points that are simultaneously attracting and repelling, depending on direction. Trajectories may approach the manifold transiently, but the presence of unstable directions eventually pushes them away. As a result, acts as a transient conduit, enabling trajectories to leave one attracting region and move toward another.
One of the advantages of relying on the IDP Hopfield model to capture sequentiality and memory transition is that there already exists a complete theory on the stability of system’s equilibria. We exploit exactly this theory to characterize the attracting and saddle memory manifolds.
Lemma 2 (Attracting and saddle manifolds). Let be the saliency stability function in Definition 1. Then the point on the memory manifold is
attracting if for all ;
saddle type if there exists at least one such that .
Proof. The manifold is attracting at if for all
and a saddle if there exists at least an eigenvalue such that . From Proposition 1, we have that the eigenvalues of the Jacobian at have the same sign of with . Thus, is attracting at iff for all
Conversely, it is a saddle if there exists such that
The phenomena known as retrieval and fixation to the memory occur for points such that the memory manifold is attracting, i.e., the respective saliency weight is greater than the saliency stability function computed at any . Escape direction emerge instead when the dynamics evolve towards a saddle region of the memory manifold. In general, we have that the attracting manifold for the memory is the subset . Conversely, the saddle manifold is .
The origin manifold
In our description of the geometric sets where the slow-fast activity of the system is constrained, we considered all the equilibria solutions of the fast IDP Hopfield dynamics that are associated to one of the memories. However, there is another solution that is independent from any of the memory vectors: the origin.
Definition 4 (Origin manifold). The origin manifold is defined as the set
In particular, for any point in the set we have that
Notice that the structure of the Jacobian allows for a clean characterization of the attracting and saddle regions. Noticing that the spectrum of Jacobian is
we can characterize the not normally hyperbolic points as such that there exists for which . Similarly, we can characterize the:
attracting origin manifold: ;
saddle origin manifold: .
Of particular relevance is the attracting origin manifold , since it is a sink for the slow-fast dynamics, and once trajectories enter this set they cannot escape it. Thereby, from the activity inevitably collapses to the origin , as formalized in the following Lemma.
Lemma 3 (Collapse to the origin). Let . Then exponentially fast.
Proof. For the slow dynamics reduce to
Therefore, the slow dynamics are completely dissipative and .
It is easy to observe that, in particular, is also forward-invariant for the reduced dynamics.
Slow saliency dynamics: under the hood
We provide the technical details for § 5. We first prove Theorem 1, and then analyze the explicit scalar map introduced in the example.
Proof of Theorem 1
We start with two auxiliary results. The first one is a corollary of Lemma 1.
Corollary 4. For every such that , we have
Proof. From Lemma 1 we know that
Taking and the definition we conclude.
In the two-memory reduction of § 5, the source and successor saliencies satisfy
Hence
Let be defined by . The next lemma shows that, once the successor saliency catches up with the source saliency, the source branch must lose stability before the source memory disappears (see also Figure 6, right plot).
Lemma 4. Assume that there exists such that . Then there exists such that
Proof. Observe first that the inequality on the left is always true by Corollary 4. We need to prove that the equality holds true for some . To this aim we will prove that
which would prove that the equality should hold between and .
Note that, again by Corollary 4, we have
We want to prove now (58). To this aim let and observe that satisfies the same differential equation satisfied by and hence we can argue that is strictly increasing and . This implies that and hence
On the other hand observe that so that and hence
where we used the assumption .
This yields the thesis.
We now prove Theorem 1.
Proof of Theorem 1. By Lemma 4 and the IDP Hopfield stability condition in Theorem 3, if the successor saliency catches up with the source saliency before , then there exists a time in at which the source branch loses stability. The first such time, , satisfies the identity in (56), namely
Equivalently,
Using and (55), and evaluating the previous identity at yields
Thus is the smallest admissible solution of the previous equation, and the updated saliency is
Extended derivations for the illustrative example
We give the details for the scalar iteration associated with
In this case it can be proved that for all so that, for all we have that . Moreover for we have . Thus, for ,
Consequently, for and ,
Substituting this expression into yields
so that the saliency update map in (12) is determined by
where
For a given , the admissible value of is the smallest solution of in , if such a solution exists.
Existence of the iteration. We present a condition on ensuring that the iteration in (59) exists, namely that there exists a solution of the equation (see also Figure 7). Observe that and that . This implies that, by taking such that

Figure 7. Plot of the function in (60) for and .
a solution must exist. Such solution is unique. Indeed, since
the sign of the this derivative coincides with the sign of the function that is a concave parabola. Since and that are both positive, we have that is positive in the interval . Hence is strictly increasing in that interval, so whenever (61) holds there is a unique solution and is uniquely defined.
Fixed points of the iteration. Fixed points satisfy . Since , this gives . Substituting into yields (see also Figure 8)
Figure 8. Plot of the function in (63) and its intersections with the line .
Observe that
If we let , then we have that is strictly increasing for the function, and strictly decreasing for , so that reaches its maximum in . Since , we can argue that we have fixed points only when
For , there are exactly two solutions, and , of (63) which satisfy . Such solutions in turn yield two fixed points, and which satisfy with .
Monotonicity of the iteration. We will now show that the iteration is strictly increasing in , which proves that the sequence of iterates, denoted by , is monotonic and hence it converges to a limit point . First notice that
Using
we obtain
and hence
That shows that the sign of only depends on the sign of . But, if the existence condition (61) holds, the derivative in (62) evaluated at must be positive. Thus, we conclude that iteration is strictly increasing if .
Convergence of the iteration. We show finally that the fixed point is repelling (or unstable) and that is attractive (or stable). To this aim observe that, at the fixed points, where , we have
Then . Moreover, comparing the expression of with (64), we see that, since , then has the same sign of showing in this way that in and in . This proves that the fixed point is repelling while the fixed point is attractive.
The previous analysis, together with the monotonicity of the iteration, shows that amplitudes above the unstable threshold converge to the attracting fixed point , which corresponds to self-sustained sequential retrieval. In contrast, amplitudes below decay until subsequent transitions can no longer be triggered, ultimately leading the dynamics to collapse into the inactive state. This is illustrated in Figure 4 in the main text.

Figure 4. (A) Fixed points of the discrete map with for the SmoothF activation, with predicted (see Figure 9 in Appendix § 13 for ). Real solutions emerge only for ; self-sustained transitions require . (B) SmoothF (blue) and Tanh (green) activation functions. (C, D) Orthogonal memories with SmoothF activation. (C) Subcritical gain (): oscillations decay and retrieval collapses to the origin. (D) Supercritical gain (): persistent oscillations converge to the stable fixed point , close to theoretical prediction, enabling reliable sequential retrieval. (E, F) Random memories with SmoothF activation. (E) Subcritical gain: cross-talk noise sustains small-amplitude transitions. (F) Supercritical gain: transitions persist but exhibit amplitude fluctuations without entrainment to . (G,H) Random memories with Tanh activation. (G) Subcritical gain () leads to collapse, with slower decay due to cross-talk. (H) Supercritical gain () yields self-sustained transitions. Panels (C-H) use , , and a directed cycle of length .
Additional results for Tanh activation
We complement the analysis of § 5 by considering the Tanh activation , while retaining . Unlike SmoothF, the corresponding saliency map and escape-time predictions do not admit simple closed-form expressions, but can be computed numerically from the implicit equations derived in § 5. Figure 9 illustrates the resulting discrete map and representative saliency dynamics for .
Figure 9. Discrete iteration map for the activation function and associated -components for the case of .
We also repeat the numerical experiments of Figure 5 using Tanh activation (Figure 10). For orthogonal memories, theoretical and numerical escape times remain in good agreement (A, B), while increasing the noise amplitude reduces the switching delay and eventually leads to premature switching (C). For non-orthogonal memories, retrieval reliability decreases with memory bias and exhibits a non-monotonic dependence on the gain , with excessively large gains reducing the probability of successful transitions (D). Conditional on successful transitions, both the asymptotic escape time and saliency generally increase with (E, F). These results support the applicability of the theoretical framework to different activation functions, while highlighting activation-dependent differences in retrieval robustness.
Figure 10. Numerical validation and robustness analysis for Tanh activation. (A) Theoretical and numerical escape times and observed switching times across consecutive transitions. (B) Asymptotic escape time versus : theory and numerical estimates. (C) Mean switching delay versus Gaussian noise amplitude. (D) Probability of correct transitions in the plane. (E, F) Asymptotic escape time and saliency amplitude, respectively. All experiments use the same numerical configuration as Figure 5. Hatched regions in (D-F) indicate no successful transitions.
Numerical investigation of sequential retrieval capacity
An important open question is how sequential retrieval reliability scales with the number of stored memories and the length of the retrieved sequence. In classical Hopfield networks, the synaptic matrix is fixed, whereas in the proposed slow-fast model it is dynamically modulated by the saliency variables:
In the classical setting, all stored memories contribute simultaneously to cross-talk. In contrast, during sequential retrieval, only a small subset of saliency coefficients typically remains significant, while the others decay toward zero. Thus, the nominal memory load need not reflect the number of memories effectively contributing to the synaptic dynamics at a given time.
To investigate this effect, we consider two numerical protocols and measure the probability of successfully completing an entire prescribed retrieval cycle, following the sequential retrieval criterion of [19 Eq. (4]. Successful retrieval requires the prescribed sequence of memory transitions, rather than merely escaping from the current memory state. In the growing-cycle protocol, a directed cycle of length is embedded among stored random i.i.d. memories with , with . In the fixed-cycle protocol, the cycle length is kept at while the memory load is increased substantially, with . Both protocols are evaluated with SmoothF and Tanh activations and , using 10 independent memory realizations per configuration.
Figure 11 reports the resulting retrieval probabilities. In the growing-cycle setting (panels A,B), SmoothF achieves reliable retrieval and , while retrieval probability constantly decreases for over the studied memory load range. Tanh exhibits greater variability, particularly for smaller network sizes. In contrast, with a fixed cycle (panels C,D), retrieval remains highly reliable even for , with occasional failures for Tanh. This difference reflects, in part, the increasing number of transitions required when grows with , whereas the additional memories in the fixed-cycle setting remain largely inactive under the saliency dynamics.
Figure 11. Full-sequence retrieval probability versus memory load . (A, B) Growing cycles with and for SmoothF and for Tanh, respectively. (C, D) Fixed cycles with and , for SmoothF and Tanh, respectively. Shaded regions indicate 95% confidence intervals across independent memory realizations.
These results highlight the distinction between nominal memory load and the number of memories actively involved in sequential retrieval. In particular, reliable retrieval of a fixed cycle at high does not imply that all stored memories can be reliably retrieved. Rather, it suggests that dynamic saliency modulation can mitigate interference from memories outside the active sequence. Moreover, since both memory load and sequence length increase in the growing-cycle protocol, their individual effects cannot be isolated from these experiments. A precise theoretical characterization of sequential retrieval capacity therefore remains an open problem.
Branching memory transitions
Our theoretical analysis focuses on transition graphs with at most one outgoing edge per memory, which enables a scalar reduction of the slow saliency dynamics and explicit conditions for sequential retrieval. However, this restriction is not inherent to the proposed model, which also accommodates branching graphs with multiple possible successors. In this setting, several successor saliencies are driven simultaneously, and branch selection depends on their competing dynamics.
To investigate this behavior, we consider two directed three-memory cycles sharing a common source memory, with equal outgoing transition weights. We use the SmoothF activation, set , , , and generate 50 independent sets of random i.i.d. memories. In the deterministic setting, the selected successor is the one with the larger absolute overlap with the source memory in all 45 realizations with unequal overlaps. We then introduce noise of amplitude per integration step in the slow dynamics and perform 20 trials per memory realization. The successor with the larger absolute overlap remains the more frequently selected branch in 36 out of 43 realizations with unequal overlaps.
These results indicate that branch selection can be influenced by the geometry of the stored memories, even when outgoing transition weights are identical. In particular, differences in memory overlaps can break the symmetry between competing successors and favor one transition over another. This preference persists in most noisy trials, although stochastic perturbations can alter the selected branch. Thus, branching introduces a competition between successor memories that depends not only on the transition graph, but also on memory correlations and perturbations. A general characterization of this competition would require extending the scalar slow reduction to account for the simultaneous evolution of multiple successor saliencies.
A motivating example for input-controlled transitions: Associative Memory Finite State Machine (AM-FSM)
Finite state machines (FSMs) are canonical models of sequential computation, in which the evolution of the system is jointly determined by current inputs and a compact internal representation of past states. At any time, an FSM is in one state drawn from a finite set, and incoming inputs trigger transitions according to a predefined state-dependent transition structure. In this sense, FSMs implement a form of implicit memory: past information is compressed into discrete internal states that condition future behavior. The state-dependent interpretation of inputs makes FSMs natural abstractions for a wide range of engineered systems, including elevators, network protocols, and traffic lights with pedestrian signals. In all such cases, effective control requires integrating current observations with the system’s internal configuration, thereby generating context-appropriate actions over time. More broadly, decision-making processes exhibit a similar structure: actions depend not only on present stimuli, but also on the internal state shaped by prior events. This observation motivates viewing sequential decision-making as the evolution of an FSM.
As an illustrative example, consider a train driver operating a metro line in the Tokyo subway system. Under normal conditions, the driver repeatedly traverses a fixed route (the "red-black" line), corresponding to a stable sequence of internal states. Due to an unexpected strike, however, the driver receives instructions to deviate from the habitual route and instead pass through Shinjuku station (on the "green-black" line) and subsequently Tokyo station (on the "blue-black" line). Only two transfer hubs permit switching from the red-black line to the other lines. At each hub, the driver must choose between alternative turns, using prior knowledge of the network and the current location to guide the decision. Although the map is outdated and uncertainty remains, the driver assigns high confidence to a left turn at each hub, while maintaining a lower probability for the alternative right turn.
This scenario exemplifies sequential decision-making under uncertainty, where each choice depends on both external cues and an internally maintained representation of the route taken so far. In the figure below, we instantiate the driver’s behavior using a two-timescale IDP Hopfield model, illustrating how stable internal states and slower adaptation processes jointly support FSM-like dynamics. In particular, we instantiate an input-driven temporal network over the transition graph, where each outgoing edge from a memory towards a set of memories is suitably weighted by an external input . The magnitude of the input components along each transition edge will ultimately determine the sequential memory path that is traversed by the two-timescale IDP Hopfield model. The exemplified control over sequential memory transitions provides solid insp
Experimental setup
The code of the experimental campaign is available at https://github.com/simlbet/Memory-Flows-NeurIPS2026.
We integrate the fast neural state and slow saliency dynamics by explicit Euler steps on the dimensionless fast clock. For , an integration step corresponds to slow time ; the numerical step and horizon are specified separately for each experiment in configs/paper/ and fast noise with amplitude .
The experiments are implemented in the accompanying repository through separate configuration-driven simulation and plotting scripts (experiments/, plotting/); the master command run_all.sh reproduces the configured workflows. For all experiments, successful retrieval requires a persistent target alignment of at least 0.90, separated from the runner-up by at least 0.10, over a 0.05 slow-time hold window in the capacity tests. Uncertainty is computed over independent pattern sets, not over multiple source trajectories as if they were independent data sets.
All numerical campaigns for the camera-ready figures were executed on the Leonardo high-performance computing system, on the AI Factory project EHPC-AIF-2026PG02-1086. The repository includes resolved experimental configurations, seed policies, CSV outputs and figure-generation scripts; numerical parallelism and worker counts are configurable through the command-line interface. Figure generation reads saved experimental data and does not rerun the underlying dynamics.
Figure 12. Finite state machine dynamics with uncertain inputs realized by a two-timescale IDP Hopfield model. (A) Schematic of a simplified Tokyo subway graph comprising three metro lines connected by two hub stations. The metro driver (red marker) must adapt the route in response to an external rerouting request, while facing uncertainty at each hub regarding the correct branch to take. (B) Sequential retrieval of stable memory states corresponding to stations along the habitual red-black line, illustrating deterministic FSM dynamics under standard operating conditions. (C) Input- and state-dependent sequential retrieval during rerouting: external inputs bias the transition dynamics at each hub toward the green-black line (via Shinjuku station) and subsequently the blue-black line (via Tokyo station), while preserving residual uncertainty across alternative transitions.
References
- [1]J. J. Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79(8):2554–2558, April 1982. ISSN 1091-6490. doi:10.1073/pnas.79.8.2554.DOI
- [2]M. A. Cohen and S. Grossberg. Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Transactions on Systems, Man, and Cybernetics, SMC-13(5):815–826, September 1983. ISSN 2168-2909. doi:10.1109/tsmc.1983.6313075.DOI
- [3]J. J. Hopfield. Neurons with graded response have collective computational properties like those of two-state neurons. Proceedings of the National Academy of Sciences, 81(10):3088–3092, May 1984. ISSN 1091-6490. doi:10.1073/pnas.81.10.3088.DOI
- [4]D. J. Amit, H. Gutfreund, and H. Sompolinsky. Information storage in neural networks with low levels of activity. Physical Review A, 35(5):2293–2303, March 1987. ISSN 0556-2791. doi:10.1103/physreva.35.2293.DOI
- [5]R. McEliece, E. Posner, E. Rodemich, and S. Venkatesh. The capacity of the hopfield associative memory. IEEE Transactions on Information Theory, 33(4):461–482, July 1987. ISSN 1557-9654. doi:10.1109/tit.1987.1057328.DOI
- [6]M. V. Tsodyks and M. V. Feigel’man. The enhanced storage capacity in neural networks with low activity level. Europhysics Letters (EPL), 6(2):101–105, May 1988. ISSN 1286-4854. doi:10.1209/0295-5075/6/2/002.DOI
- [7]D. Krotov and J. J. Hopfield. Dense associative memory for pattern recognition. In Advances in neural information processing systems, volume 29, 2016. doi:doi.org/10.48550/arXiv.1606.01164.arxiv.org/abs/1606.01164
- [8]D. Krotov and J. J. Hopfield. Large associative memory problem in neurobiology and machine learning. In International Conference on Learning Representations, 2020. doi:doi.org/10.48550/arXiv.2008.06996.arxiv.org/abs/2008.06996
- [9]M. Demircigil, J. Heusel, M. Lowe, S. Upgang, and F. Vermet. On a model of associative memory with huge storage capacity. Journal of Statistical Physics, 168(2):288–299, May 2017. ISSN 1572-9613. doi:10.1007/s10955-017-1806-y.DOI
- [10]H. Ramsauer, B. Schäf, J. Lehner, P. Seidl, M. Widrich, L. Gruber, M. Holzleitner, T. Adler, D. Kreil, M. K. Kopp, G. Klambauer, J. Brandstetter, and S. Hochreiter. Hopfield networks is all you need. In International Conference on Learning Representations, 2021.arxiv.org/abs/2008.02217
- [11]B. Hoover, Y. Liang, B. Pham, R. Panda, H. Strobel, D. H. Chau, M. Zaki, and D. Krotov. Energy transformer. Advances in neural information processing systems, 36:27532–27559, 2023a.arxiv.org/abs/2302.07253
- [12]B. Hoover, H. Strobel, D. Krotov, J. Hoffman, Z. Kira, and D. H. Chau. Memory in plain sight: A survey of the uncanny resemblances between diffusion models and associative memories. In Associative Memory & Hopfield Networks in 2023, 2023b.arxiv.org/abs/2309.16750
- [13]L. Ambrogioni. In search of dispersed memories: Generative diffusion models are associative memory networks. Entropy, 26(5):381, April 2024. ISSN 1099-4300. doi:10.3390/e26050381.DOI
- [14]D. Kleinfeld. Sequential state generation by model neural networks. Proceedings of the National Academy of Sciences, 83(24):9469–9473, December 1986. ISSN 1091-6490. doi:10.1073/pnas.83.24.9469.DOI
- [15]H. Sompolinsky and I. Kanter. Temporal association in asymmetric neural networks. Physical Review Letters, 57(22):2861–2864, December 1986. ISSN 0031-9007. doi:10.1103/physrevlett.57.2861.DOI
- [16]D. W. Tank and J. J. Hopfield. Neural computation by concentrating information in time. Proceedings of the National Academy of Sciences, 84(7):1896–1900, April 1987. ISSN 1091-6490. doi:10.1073/pnas.84.7.1896.DOI
- [17]D. Kleinfeld and H. Sompolinsky. Associative neural network model for the generation of temporal patterns. theory and application to central pattern generators. Biophysical Journal, 54(6):1039–1051, December 1988. ISSN 0006-3495. doi:10.1016/S0006-3495(88)83041-8.
- [19]H. T. Chaudhry, J. A. Zavatone-Veth, D. Krotov, and C. Pehlevan. Long sequence hopfield memory. Journal of Statistical Mechanics: Theory and Experiment, 2024(10):104024, October 2024. ISSN 1742-5468. doi:10.1088/1742-5468/ad6427.DOI
- [20]A. Karuvally, P. Lertsaroj, S. Terrence, and H. T. Siegelmann. Exponential dynamic energy network for high capacity sequence memory. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. doi:10.48550/arXiv.2510.24965.DOI
- [21]L. Herron, P. Sartori, and B. Xue. Robust retrieval of dynamic sequences through interaction modulation. PRX Life, 1(2), December 2023. ISSN 2835-8279. doi:10.1103/prxlife.1.023012.DOI
- [22]M. Farrell and C. Pelhevan. Recall tempo of hebbian sequences depends on the interplay of hebbian kernel with tutor signal timing. Proceedings of the National Academy of Sciences, 121(32), 2024. ISSN 1091-6490. doi:10.1073/pnas.2309876121.DOI
- [23]S. Betteti, G. Baggio, F. Bullo, and S. Zampieri. Input-driven dynamics for robust memory retrieval in hopfield networks. Science Advances, 11(17), April 2025. ISSN 2375-2548. doi:10.1126/sciadv.adu6991.DOI
- [24]M. K. Benna and S. Fusi. Computational principles of synaptic memory consolidation. Nature Neuroscience, 19(12):1697–1706, October 2016. ISSN 1546-1726. doi:10.1038/nn.4401.
- [25]M. Gillett and N. Brunel. Dynamic control of sequential retrieval speed in networks with heterogeneous learning rules. eLife, 12, August 2024. ISSN 2050-084X. doi:10.7554/eLife.88805.3.DOI
- [26]N. Fenichel. Geometric singular perturbation theory for ordinary differential equations. Journal of Differential Equations, 31(1):53–98, January 1979. ISSN 0022-0396. doi:10.1016/0022-0396(79)90152-9.DOI
- [27]G. Hek. Geometric singular perturbation theory in biological practice. Journal of Mathematical Biology, 60(3):347–386, April 2009. ISSN 1432-1416. doi:10.1007/s00285-009-0266-7.DOI
- [28]R. A. Horn and C. R. Johnson. Matrix Analysis. Cambridge University Press, December 1985. ISBN 9780511810817.









