betaFeedback

WorksPhysics and Astronomy

Memory flows: geometry and dynamics of sequential retrieval in input-driven Hopfield networks

Abstract

Associative memory models classically describe retrieval as convergence to stored patterns, but the theory of sequential retrieval remains comparatively limited. We develop a two-timescale input-driven Hopfield network in which fast associative retrieval is modulated by a slow feedback variable evolving over a prescribed memory-transition graph. This yields autonomous, graph-constrained transitions while preserving analytical tractability. Using slow-fast and geometric singular perturbation theory, we characterize memory fixation via memory manifolds and identify the geometric regions that govern stability loss and escape from retrieved memories. We further derive reduced slow dynamics and explicit criteria for transition onset, self-sustained retrieval, and collapse, expressed through escape times, gain thresholds, and fixed points. Together, these results provide a tractable framework for analyzing feedback-driven sequential retrieval in Hopfield networks.

Introduction

Associative memory has been a foundational concept in theoretical neuroscience and machine learning for over four decades. In its classical formulation, associative memory retrieval is modeled as convergence toward stable equilibria of an energy landscape [1, 2, 3], whereby a corrupted cue relaxes to a stored pattern through recurrent dynamics, providing a canonical example of distributed computation in neural networks. A large body of work has focused on characterizing the memory capacity of such models [4, 5, 6], revealing a fundamental limitation of the original Hopfield network, whose capacity scales sub-linearly with the number of neurons. Modern Hopfield models [7, 8, 9] overcome this limitation by achieving exponential capacity within an energy-based framework, and have been linked to contemporary architectures such as Transformers [10, 11] and diffusion models [12, 13]. However, they largely remain static retrieval systems: dynamics stop at convergence, so sequential retrieval requires repeated reinitialization. This contrasts with cognitive and algorithmic tasks, including those addressed by Transformers, where reasoning depends on structured transitions across sequences of memories.

Pioneering works by [14, 15] already identified the limitation of a theory accounting only for static retrieval. They proposed a continuous-time mechanism for sequential memory transitions via delayed asymmetric interactions, which introduce a destabilizing dynamic component driving transitions between attractors. This idea has since motivated extensive efforts to model memory transitions [16, 17], establishing mean-field results on the gain sustaining sequentiality. More recently, [Karuvally et al., 2023] introduced a continuous-time, slow-fast energy-based model with delayed asymmetric interactions, while [19] investigated long-sequence retrieval and capacity in a discrete-time setting. The softmax-based model of [20] further provides capacity results and escape-time estimates based primarily on energy arguments. Related approaches use slowly evolving parameters to control attractor switching [21, 22]. Our aim is complementary: rather than improving retrieval performance, we focus on developing an analytically tractable model that provides a mechanistic understanding of sequential memory transitions.

In this paper, we propose and analyze a two-timescale extension of the input-driven plasticity (IDP) Hopfield model of [23] that enables autonomous sequential retrieval. In the original IDP Hopfield model, external inputs modulate the saliency of stored memories and thereby reshape the retrieval landscape. Here, we replace this externally prescribed modulation with a slow feedback variable whose dynamics are driven by the currently retrieved memory and structured by a transition graph. The slow feedback can be viewed as a phenomenological representation of activity-dependent biological processes, such as short-term synaptic plasticity, neuronal adaptation, or neuromodulatory gain control, that modulate memory stability [24]. The resulting sequential IDP Hopfield network produces graph-constrained transitions between memories while remaining analytically tractable. Our contributions are threefold. First, using slow-fast and geometric singular perturbation theory, we characterize the geometry of memory fixation through memory manifolds and the attracting and saddle-type regions that govern entry and escape. Second, we derive reduced saliency dynamics and explicit escape conditions, including transition times, gain thresholds, and collapse regimes. Third, we show how these conditions support self-sustained sequential retrieval over structured memory graphs. Figure 1 provides a schematic overview of the sequential memory transition mechanism introduced in this work. Together, these results establish a tractable geometric and dynamical framework for sequential retrieval in input-driven Hopfield networks, providing explicit conditions under which slow internal feedback induces, sustains, or fails to sustain transitions between memories.

Figure 1. Schematic illustration of the proposed two-timescale mechanism for sequential memory transitions. At the initial time, the fast state variable is at a stable equilibrium corresponding to memory ξ1\xi_1. The slow variable zz progressively destabilizes this equilibrium while simultaneously stabilizing the equilibrium associated with memory ξ2\xi_2. At the escape time TescT_{\mathrm{esc}}, the equilibrium corresponding to ξ1\xi_1 loses stability and the system transitions toward ξ2\xi_2. This transition occurs only when the gain parameter κ\kappa of the slow dynamics exceeds a critical threshold κcrit\kappa_{\mathrm{crit}}. We analytically characterize both κcrit\kappa_{\mathrm{crit}} and the resulting escape time TescT_{\mathrm{esc}}.

The IDP Hopfield model for sequential retrieval

The IDP Hopfield model introduced in [23] extends the classical Hopfield model by incorporating an explicit modulation of synaptic interactions through external inputs. This formulation enables a controlled and interpretable mechanism for memory retrieval, where the input does not merely select an initial condition but directly shapes the energy landscape governing the dynamics. We next briefly review the IDP Hopfield model, deferring technical details to Appendix § 9, and then introduce a sequential extension that enables autonomous, structured transitions between stored memories.

The IDP Hopfield model. The IDP Hopfield dynamics, illustrated schematically in Figure 2(A), are

Figure 2. (A) Block diagram of the IDP Hopfield model; setting τx=1\tau_x = 1 recovers (1). (B) Block diagram of the two-timescale sequential IDP Hopfield model (3). (C) Example of a deterministic transition graph GA\mathcal{G}_A, where nodes represent stored memories and edges encode admissible transitions. The graph decomposes into multiple connected components (highlighted by dashed red contours), each defining a distinct memory sequence; one, selected by x(0)x(0), is highlighted in blue. (D) One-timescale model with orthogonal memories (κ=5\kappa= 5): sequential retrieval proceeds through mixed memory states, with partial overlaps and unpredictable escape times across memories. (E-F) Two-timescale sequential IDP Hopfield model (τx=1,τz=5,κ=5\tau_x = 1, \tau_z = 5, \kappa= 5). (E) Orthogonal memories: transitions occur between individual memories without mixing, with uniform escape times and overlaps reaching their maximal value, indicating exact memory alignment. (F) Random memories: transitions occur without mixing, reaching maximum overlap, but escape times are no longer uniform. Numerical experiments are shown for visualization purposes with P=15P = 15 memories and a limit cycle over 5 of them, with ψ=tanh⁡\psi= \tanh, and whose overlaps are highlighted in different colors in the plots.

x˙=−x+W(Φ(u))Ψ(x),x(0)∈RN,(1)\dot{x}=-x+W\left(\Phi(u)\right)\Psi(x), \qquad x(0)\in\mathbb{R}^{N}, \tag*{(1)}

where x∈RNx\in\mathbb{R}^{N} is the network state and u∈Rdu\in\mathbb{R}^{d} is a slowly varying external input. The map Φ:Rd→R≥0P\Phi:\mathbb{R}^{d}\to\mathbb{R}_{\geq0}^{P} is a saliency function that acts as a positive filter on the input uu, while Ψ:RN→RN\Psi:\mathbb{R}^{N}\to\mathbb{R}^{N} is the neural activation function. The synaptic matrix depends on the filtered input through

W(Φ(u))=Mdiag⁡(Φ(u))M⊤,M:=1N[ξ1,…,ξP],W\left(\Phi(u)\right)=M\operatorname{diag}\left(\Phi(u)\right)M^{\top}, \qquad M:=\frac{1}{\sqrt{N}}\left[\xi^{1},\ldots,\xi^{P}\right],

where {ξμ}μ=1P⊆{+1}N\{\xi^{\mu}\}_{\mu=1}^{P}\subseteq\{+1\}^{N} is a set of mutually orthogonal stored patterns. We stress that the orthogonality constraint is necessary to derive rigorous results on the geometry and dynamics of the discussed model, but numerical experiments support their validity also in the canonical case of random memories.

Throughout the paper, we assume that Φ(⋅)\Phi(\cdot) acts componentwise and homogeneously, i.e., Φi(u)=ϕ(ui)\Phi_i(u)=\phi(u_i) for some scalar function ϕ:R→R≥0\phi:\mathbb{R}\to\mathbb{R}_{\geq0}, which is even and monotone with respect to ∣ui∣\lvert u_i\rvert. Similarly, Ψ(⋅)\Psi(\cdot) acts componentwise as Ψi(x)=ψ(xi)\Psi_i(x)=\psi(x_i), where ψ:R→R\psi:\mathbb{R}\to\mathbb{R} is odd, continuously-differentiable and monotone non-decreasing (e.g., tanh⁡\tanh). Under these assumptions, the map Φ(⋅)\Phi(\cdot) assigns a non-negative saliency coefficient to each stored pattern, thereby modulating its contribution to the synaptic matrix W(Φ(u))W(\Phi(u)), while Ψ(⋅)\Psi(\cdot) retains the standard symmetry and monotonicity properties of Hopfield-type activation functions. The classical Hopfield model is recovered as a special case when all input-dependent coefficients ϕ(ui)\phi(u_i), i=1,…,Pi=1,\ldots,P, are all equal to one, in which case the synaptic matrix reduces to a fixed matrix independent of the input.

Under the above-mentioned standard assumptions on Ψ\Psi, the IDP Hopfield model admits the energy function [3, 8]

E(x,Φ(u))=−12Ψ(x)⊤W(Φ(u))Ψ(x)+x⊤Ψ(x)−∑i=1N∫0xiψ(s) ds.(2)E(x,\Phi(u))=-\frac{1}{2}\Psi(x)^{\top}W(\Phi(u))\Psi(x)+x^{\top}\Psi(x)-\sum_{i=1}^{N}\int_{0}^{x_i}\psi(s)\,\mathrm{d}s. \tag*{(2)}

This energy provides a geometric interpretation of retrieval: the input-dependent saliencies Φ(u)\Phi(u) reshape the energy landscape, thereby controlling the existence and stability of memory equilibria.

A key feature of the IDP formulation is that the input-dependent coefficients {ϕ(uμ)}μ=1P\{\phi(u_{\mu})\}_{\mu=1}^{P} directly control the existence and stability of memory states. In general, the system admits multiple equilibria, some of which are associated with the stored patterns.1 In particular, to each memory vector ξν\xi^{\nu} one can associate a memory equilibrium of the form x=γνξνx=\gamma_{\nu}\xi^{\nu}, which exists if and only if ϕ(uν)>1\phi(u_{\nu})>1. In that case, the scaling factor γν\gamma_{\nu} is uniquely determined by γν=f−1(ϕ(uν))\gamma_{\nu}=f^{-1}(\phi(u_{\nu})), where f(γ):=γ/ψ(γ)f(\gamma):=\gamma/\psi(\gamma). The stability of these equilibria is also governed by the coefficients ϕ(uν)\phi(u_{\nu}): if ϕ(uν)>1\phi(u_{\nu})>1, the corresponding equilibrium is stable only if ϕ(uν)\phi(u_{\nu}) is sufficiently large relative to the maximum coefficient ϕmax⁡:=max⁡μϕ(uμ)\phi_{\max}:=\max_{\mu}\phi(u_{\mu}) (see Appendix § 9 for details). Overall, the coefficients ϕ(uμ)\phi(u_{\mu}) provide a direct and tunable mechanism to determine which memories exist and are stable, thereby enabling selective activation and suppression within the same network.

The Two-Timescale Sequential IDP Hopfield Model. While the IDP formulation enables input-modulated retrieval and sharp control over memory stability, it remains fundamentally static in nature: for a fixed input, the dynamics converge to a single equilibrium and do not exhibit autonomous transitions between memories. In particular, the model lacks an internal mechanism to encode temporal or semantic relationships among stored patterns.

To overcome this limitation, we introduce a minimal two-timescale extension of (1) that couples the fast retrieval dynamics to a slower internal state:

{τxx˙=−x+Mdiag⁡(Φ(z))M⊤Ψ(x),x(0)∈RN,τzz˙=−z+κNAM⊤Ψ(x),z(0)∈RP,(3)\begin{cases} \tau_x\dot{x}=-x+M\operatorname{diag}\left(\Phi(z)\right)M^{\top}\Psi(x), & x(0)\in\mathbb{R}^{N},\\ \tau_z\dot{z}=-z+\dfrac{\kappa}{\sqrt{N}}AM^{\top}\Psi(x), & z(0)\in\mathbb{R}^{P}, \tag*{(3)} \end{cases}

where κ>0\kappa>0 is a gain parameter and A∈{0,1}P×PA\in\{0,1\}^{P\times P} encodes structured associations among memories. Specifically, AA is the adjacency matrix of a directed graph GA=(V,E)\mathcal{G}_{A}=(\mathcal{V},\mathcal{E}), whose nodes V={1,…,P}\mathcal{V}=\{1,\ldots,P\} represent stored memories and whose edges represent admissible transitions, with (i,j)∈E(i,j)\in\mathcal{E} if and only if Aij=1A_{ij}=1. We focus on deterministic graphs, where each node has out-degree at most one, and assume τx≪τz\tau_x\ll\tau_z, so that xx evolves rapidly while zz provides slow modulation.

Figure 2(B) illustrates the block diagram of the sequential IDP Hopfield model (3). In contrast to the original IDP formulation (1), where the input directly modulates the synaptic weights, here the modulation is mediated by the slow, feedback-driven variable zz, which endows the system with an internal memory of past retrieval states. Figure 2(C) shows an example of deterministic graph GA\mathcal{G}_A. Such graphs may consist of multiple connected components, each corresponding to a distinct memory sequence that can be selectively retrieved through appropriate choice of the initial condition x(0)x(0).

In the regime τx≪τz\tau_x \ll\tau_z, a clear separation of timescales emerges in (3): the fast variable xx rapidly relaxes toward a memory determined by the current Φ(z)\Phi(z), while the slow variable zz evolves in response to the retrieved pattern and progressively reshapes the energy landscape. This interaction gives rise to autonomous sequential retrieval, whereby the system transitions between stored memories according to the graph GA\mathcal{G}_A. As illustrated in Figure 2(E-F) on a simple cyclic graph with P=5P = 5 memories, the proposed two-timescale sequential IDP Hopfield model exhibits robust and predictable sequential retrieval when κ\kappa is sufficiently large. For orthogonal memories (Figure 2(E)), transitions occur directly between individual memories without intermediate mixing: overlaps reach their maximal value, indicating exact memory alignment, and escape times are uniform across memories. For random memories (Figure 2(F)), the dynamics similarly transition without mixing and attain maximal overlap, although escape times are no longer uniform. To highlight the role of timescale separation, we compare this behavior with the classical sequential Hopfield dynamics obtained in a single-timescale setting, x˙=−x+κMAMTΨ(x)\dot{x} = -x + \kappa MAM^{\mathsf{T}}\Psi(x), where transitions are encoded directly in the synaptic matrix [25]. As shown in Figure 2(D), this model exhibits qualitatively different dynamics: trajectories evolve through mixed states with partial overlaps, display irregular escape times, and are highly sensitive to the gain κ\kappa. Depending on its value, the system may collapse to the origin, converge to a fixed point, or sustain a limit cycle, consistent with previous observations.

Finally, we point out that (3) admits a natural modern Hopfield reformulation, detailed in Appendix § 10.

Memory manifolds: the highway of sequential retrieval

Memory manifolds: the highway of sequential retrieval

Sequential memory retrieval is the continuous evolution of neural activity in state space, from one stable memory state to another one. The direction of motion is shaped by the semantic relationships between memories, which at any given time constrain neural activity to a lower-dimensional memory subspace. As semantic relationships change, the binding is broken, allowing neural trajectories to leave one memory subspace and evolve toward another.

From a geometric perspective, as sketched in Figure 3(A), sequential retrieval therefore requires two complementary structures: (i) memory subspaces (red and shaded surfaces) that transiently trap neural activity (blue curves), and (ii) entry and exit pathways that enable transitions (purple branches) between them. In this section, we formalize this intuition using slow-fast and geometric singular perturbation theory (GSPT) [26, 27]. Leveraging the analytical tractability of the IDP Hopfield model, we provide a geometric characterization of how sequential retrieval arises in high-dimensional associative memory systems.

Figure 3. (A) Sketch of the transitions between different memory manifolds in the slow-fast Hopfield model. The blue curves corresponds to the slow movement of the system close to a stable memory manifold, whereas the purple curves correspond to the fast switching between different stable memory manifolds. Each memory manifold Nμ\mathcal{N}_{\mu} is decomposed into its attracting (red) and saddle-type (green) memory manifolds Mμa\mathcal{M}^{a}_{\mu}, Mμs\mathcal{M}^{s}_{\mu}. (B) Simulated trajectory projected onto the fast subspace in a slow-fast Hopfield model storing two memories. Continuous curves indicate the flow along memory branches Nμ\mathcal{N}_{\mu} (magenta) and Nν\mathcal{N}_{\nu} (blue), whereas dashed curves indicate rapid transitions from one memory branch to another. Switching occurs after the trajectory has moved from the attracting part of the memory branches Mνa,ε\mathcal{M}^{a,\varepsilon}_{\nu} and Mμa,ε\mathcal{M}^{a,\varepsilon}_{\mu}, respectively, to the saddle-type section Mνs,ε\mathcal{M}^{s,\varepsilon}_{\nu} and Mμs,ε\mathcal{M}^{s,\varepsilon}_{\mu}. The black dots indicate the relevant point of separation between Mνa\mathcal{M}^{a}_{\nu} and Mνs\mathcal{M}^{s}_{\nu}, and Mμa\mathcal{M}^{a}_{\mu} and Mμs\mathcal{M}^{s}_{\mu}, respectively (red/green). (C) Temporal series of the variable xνx_{\nu}. Colors and line types as in panel B.

Geometric singular perturbation theory Fenichel, [26] provides a theoretical framework to analyze dynamical systems with well-separated time scales. The central idea is that, when fast and slow variables interact weakly, the full dynamics can be approximated as a sequence of slow movements, solutions of a simpler system, interspersed by fast jumps. We introduce the time-scale parameter ε=τx/τz\varepsilon= \tau_x/\tau_z and consider the slow-fast system

εx˙=−x+M diag⁡(Φ(z)) MTΨ(x),x(0)∈RN,z˙=−z+κNAMTΨ(x),z(0)∈RP.(4)\begin{aligned} \varepsilon\dot{x} &= -x + M\,\operatorname{diag}(\Phi(z))\,M^{\mathsf{T}}\Psi(x), \qquad x(0) \in\mathbb{R}^{N}, \\ \dot{z} &= -z + \frac{\kappa}{\sqrt{N}} A M^{\mathsf{T}}\Psi(x), \qquad z(0) \in\mathbb{R}^{P}. \tag*{(4)} \end{aligned}

We refer to the xx-ODE with the term fast subsystem and denote with F(x,z)F(x,z) its right-hand side. Taking the limit ε→0+\varepsilon\to0^{+} constrains the fast dynamics onto DN=F(x,z)\mathcal{D}_{N} = F(x,z), yielding the reduced system. The algebraic constraint effectively tracks the minimum over the energy as the slow neural feedback evolves, and admits multiple solutions, reflecting that Hopfield networks are designed to store multiple stable patterns. In slow-fast theory, the set of the fast-subsystem equilibria is called critical set, and defined as

S0={(x,z)∈RN×RP:DN=F(x,z)}.(5)\mathcal{S}_{0} = \left\{(x,z) \in\mathbb{R}^{N} \times\mathbb{R}^{P} : \mathcal{D}_{N} = F(x,z)\right\}. \tag*{(5)}

Memory branches, memory manifolds, and stability. Given the fast dynamics with PP stored memories, a memory branch Nν⊆S0\mathcal{N}_{\nu} \subseteq\mathcal{S}_0 describes a PP-dimensional subspace in which neural activity can be temporarily confined. In the IDP Hopfield model, these memory branches take the explicit form

Nν={(x,z)∈RN×RP:∃γν≠0, x=γνξν, ϕ(zν)=f(γν)},(6)\mathcal{N}_{\nu}=\left\{(x,z)\in\mathbb{R}^{N}\times\mathbb{R}^{P}:\exists\gamma_{\nu}\ne0,\ x=\gamma_{\nu}\xi^{\nu},\ \phi(z_{\nu})=f(\gamma_{\nu})\right\}, \tag*{(6)}

for ν=1,…,P\nu=1,\ldots,P and f(γ)=γ/ψ(γ)f(\gamma)=\gamma/\psi(\gamma). The normally hyperbolic subsets Mνa\mathcal{M}_{\nu}^{a} and Mνs\mathcal{M}_{\nu}^{s} of Nν\mathcal{N}_{\nu} are known as the attracting and saddle-type memory manifolds. They can be identified by studying the stability of the fast subsystem equilibria in Nν\mathcal{N}_{\nu}, which are related to the saliency weights. Another relevant branch of S0\mathcal{S}_0 is the origin manifold,

M0={(x,z)∈RN×RP:x=0}.(7)\mathcal{M}_0=\left\{(x,z)\in\mathbb{R}^{N}\times\mathbb{R}^{P}:x=0\right\}. \tag*{(7)}

GSPT ensures that, for 0<ε≪10<\varepsilon\ll1, each Nν\mathcal{N}_{\nu} (Mνa\mathcal{M}_{\nu}^{a}, Mνs\mathcal{M}_{\nu}^{s}, and M0\mathcal{M}_0) perturbs to a nearby invariant manifold Nνε\mathcal{N}_{\nu}^{\varepsilon} (Mνa,ε\mathcal{M}_{\nu}^{a,\varepsilon}, Mνs,ε\mathcal{M}_{\nu}^{s,\varepsilon}, and M0ε\mathcal{M}_{0}^{\varepsilon}) of the full system (4). In particular, the manifolds Mνa,ε\mathcal{M}_{\nu}^{a,\varepsilon} and Mνs,ε\mathcal{M}_{\nu}^{s,\varepsilon} provide a precise geometric description of memory fixation and transitions.

Fixation, loss of stability, and transitions. Sequential retrieval requires understanding how trajectories enter and leave Nνε\mathcal{N}_{\nu}^{\varepsilon}, which consists in studying the stability of its constitutive memory manifolds. As presented in Figure 3(B-C), during retrieval of the memory ξν\xi^{\nu}, the neural activity converges to Mνa,ε\mathcal{M}_{\nu}^{a,\varepsilon} rapidly, fixation occurs, and escape is impossible. As the saliency ϕ(zν)\phi(z_{\nu}) decreases, an escape direction appears when ϕ(zν)\phi(z_{\nu}) crosses the threshold ϕmax⁡\phi_{\max} from above. Geometrically, this corresponds to crossing (an ε\varepsilon-perturbation of) a non-hyperbolic point on Nν\mathcal{N}_{\nu} (black circle in Figure 3(B)), corresponding to the trajectory moving from the attracting manifold Mνa,ε\mathcal{M}_{\nu}^{a,\varepsilon} to the saddle manifold Mνs,ε\mathcal{M}_{\nu}^{s,\varepsilon}. Beyond this point, a typical trajectory is ejected via an unstable fiber of Mνs,ε\mathcal{M}_{\nu}^{s,\varepsilon} towards a new attracting manifold Mμa,ε\mathcal{M}_{\mu}^{a,\varepsilon} associated to the memory ξμ\xi^{\mu}. However, the jump towards a different memory can occur only if the associated ϕ(zμ)>1\phi(z_{\mu})>1. Otherwise, if for all μ\mu it holds ϕ(zμ)<1\phi(z_{\mu})<1 then the only attracting manifold is M0ε\mathcal{M}_{0}^{\varepsilon}, which drives the full system towards the origin.

Slow saliency dynamics: the driving engine

In § 4, we characterized the geometric structure underlying memory retrieval, including the manifolds that support retrieval and the instabilities and fibers that enable transitions between them. We now turn to how the reduced slow dynamics in (4) govern sequential transitions as a function of the gain parameter κ\kappa, the initial condition z(0)z(0), and the structure of the transition matrix AA. We present the main results below, deferring technical derivations and proofs to Appendix § 12.

Reduced dynamics and retrieval regimes. Consider the sequential IDP Hopfield model (3) in the limit τx/τz→0+\tau_x/\tau_z\to0^+. The fast variable xx rapidly relaxes to a stable equilibrium x∗(z)x^*(z) determined by the current saliency zz. Substitution into the slow equation of (3) gives

z˙=−z+κNAMTΨ(x∗(z)).(8)\dot{z}=-z+\frac{\kappa}{\sqrt{N}}AM^{\mathsf{T}}\Psi\left(x^*(z)\right). \tag*{(8)}

This reduced equation describes the slow flow along the branches {Nμ}μ=1P\{\mathcal{N}_{\mu}\}_{\mu=1}^{P} of the fast Hopfield subsystem: for each fixed zz, the fast dynamics select an equilibrium xν∗(z)x_{\nu}^*(z) and sticks to the associated attracting manifold Mνa\mathcal{M}_{\nu}^{a} until it loses stability. Only then the trajectory may jump to another memory branch, as pictorially depicted in the introductory Figure 1. By the geometric picture of § 4, the saliency state zz selects which memory branches contain attracting manifolds and can therefore support fixation. This gives three regimes:

(i) No active memories. If ϕ(zμ)<1\phi(z_{\mu})<1 for all μ\mu, no memory equilibrium exists and trajectories converge to the origin by evolving in M0\mathcal{M}_0.

(ii) Unique active memory. If there is a unique ν\nu such that ϕ(zν)>1\phi(z_{\nu})>1, there is only active memory equilibrium, and trajectories rapidly converge to Mνa\mathcal{M}_{\nu}^{a}. This equilibrium is stable and given by

x∗(z)=f−1(ϕ(zν))ξν,where f(γ)=γ/ψ(γ).x^*(z)=f^{-1}\left(\phi(z_{\nu})\right)\xi^{\nu},\qquad\text{where }f(\gamma)=\gamma/\psi(\gamma).

(iii) Competing memories. If several components ϕ(zν)\phi(z_{\nu}) exceed the threshold, multiple memory equilibria may coexist, allowing transitions between the corresponding memory states.

Two-memory reduction. We next restrict to the case in which at most two memories compete. Specifically, we assume that, along the relevant portion of the trajectory, only two components of z(t)z(t) are significantly different from zero. Without loss of generality, let the initial time be t=0t=0 and assume z(0)=Ze(ν)z(0)=Z e^{(\nu)}, where e(ℓ)e^{(\ell)} denotes the ℓ\ell-th vector of the canonical basis of RP\mathbb{R}^{P} and Z>0Z>0 is the initial value of zνz_{\nu}. 2

If ϕ(Z)<1\phi(Z)<1, no memory is active: the fast state converges to x(t)=0x(t)=0, and the slow dynamics reduce to z˙=−z\dot{z}=-z, driving all saliencies exponentially to zero in M0\mathcal{M}_0. We therefore focus on ϕ(Z)>1\phi(Z)>1, where ξν\xi^{\nu} is initially active and the fast state is trapped on its associated memory equilibrium,

xν⋆(t)=f−1(ϕ(zν(t)))ξν.x_{\nu}^{\star}(t)=f^{-1}\left(\phi(z_{\nu}(t))\right)\xi^{\nu}.

In this case, the system initially evolves on Mνα\mathcal{M}_{\nu}^{\alpha}, and the reduced saliency dynamics (8) satisfy

z˙=−z+κψ(f−1(ϕ(zν)))e(μ),(9)\dot{z}=-z+\kappa\psi\left(f^{-1}\left(\phi(z_{\nu})\right)\right)e^{(\mu)}, \tag*{(9)}

where μ\mu is the unique successor of ν\nu in the transition graph GA\mathcal{G}_{A}. Hence z˙σ=−zσ\dot{z}_{\sigma}=-z_{\sigma} for all σ≠μ\sigma\ne\mu. Since only zν(0)z_{\nu}(0) is nonzero, all components except zνz_{\nu} and zμz_{\mu} remain identically zero, with zν(t)=Ze−tz_{\nu}(t)=Ze^{-t}. The successor saliency component satisfies

z˙μ=−zμ+κg(Ze−t),g:=ψ∘f−1∘ϕ.\dot{z}_{\mu}=-z_{\mu}+\kappa g(Ze^{-t}),\qquad g:=\psi\circ f^{-1}\circ\phi.

Note that, on the interval where the source memory exists, g(Ze−t)>0g(Ze^{-t})>0, so zμz_{\mu} is driven by a positive input. Thus, the transition from ξν\xi^{\nu} to ξμ\xi^{\mu} reduces to a two-dimensional saliency dynamics: a decaying source component and a positively driven successor component.

Transition criterion and induced map. We now determine whether the two-dimensional saliency dynamics produces a transition to the successor memory or decays to inactivity. Let Tmax⁡T_{\max} be the time at which the source memory ceases to exist, namely ϕ(zν(Tmax⁡))=1\phi(z_{\nu}(T_{\max}))=1. Since zν(t)=Ze−tz_{\nu}(t)=Ze^{-t}, the source memory exists for t<Tmax⁡t<T_{\max} and disappears for t>Tmax⁡t>T_{\max}. The outcome is determined by whether the successor saliency zμ(t)z_{\mu}(t) becomes large enough before this loss of existence occurs. Two possible cases arise (see also Figure 6 of the Appendix for a graphical illustration).

Figure 6. Graphical illustration of the two possible outcomes of the source-successor saliency dynamics. Left: no transition occurs and the dynamics decay to inactivity. Right: the successor saliency catches up with the source saliency, triggering a transition between memories.

Case 1: no transition. Assume that

zμ(t)<zν(t)∀t∈[0,Tmax⁡).z_{\mu}(t)<z_{\nu}(t)\qquad\forall t\in[0,T_{\max}).

Then the successor component never catches up with the source component before the source memory disappears. Hence, after Tmax⁡T_{\max}, both saliencies fall below the existence threshold and the system enters the inactive regime M0\mathcal{M}_0, where the origin is the only stable equilibrium.

Case 2: transition. Assume instead that there exists a first time Tˉ∈[0,Tmax⁡)\bar{T}\in[0,T_{\max}) such that

zμ(Tˉ)=zν(Tˉ).z_{\mu}(\bar{T})=z_{\nu}(\bar{T}).

In this case, the following result characterizes the loss of stability of the source branch from Mνα\mathcal{M}_{\nu}^{\alpha} to Mμα\mathcal{M}_{\mu}^{\alpha}, and the induced update of the successor saliency. The proof is given in Appendix D.

Theorem 1 (Escape condition and saliency update). Under the above condition, there exists an escape time Tesc∈[Tˉ,Tmax⁡)T_{\mathrm{esc}}\in[\bar{T},T_{\max}) at which the equilibrium of the fast Hopfield subsystem associated with the source memory ξν\xi^{\nu} loses stability. Moreover, TescT_{\mathrm{esc}} is the smallest solution in [0,Tmax⁡)[0,T_{\max}) of

κ∫0Tesce−(Tesc−τ)g(Ze−τ) dτ=h(Ze−Tesc),(10)\kappa\int_{0}^{T_{\mathrm{esc}}}e^{-(T_{\mathrm{esc}}-\tau)}g(Ze^{-\tau})\,\mathrm{d}\tau=h(Ze^{-T_{\mathrm{esc}}}), \tag*{(10)}

*where h(z):=ϕ−1(1ψ(f−1(ϕ(z))))h(z):=\phi^{-1}\left(\frac{1}{\psi(f^{-1}(\phi(z)))}\right). 3 At escape, the successor saliency is*

Z+:=zμ(Tesc)=h(Ze−Tesc).(11)Z^{+}:=z_{\mu}(T_{\mathrm{esc}})=h(Ze^{-T_{\mathrm{esc}}}). \tag*{(11)}

At time TescT_{\mathrm{esc}}, the trajectory leaves the branch associated with the source memory ξν\xi^{\nu} and jumps to the manifold Mμα\mathcal{M}_{\mu}^{\alpha} associated with the successor memory ξμ\xi^{\mu}. Theorem 1 therefore defines a one-dimensional saliency update Z→Z+Z\to Z^{+}. For later use, it is convenient to rewrite this update in terms of X:=eTescX:=e^{T_{\mathrm{esc}}}. Since

∫0Tesce−(Tesc−τ)g(Ze−τ) dτ=1X∫1Xg(Zs) ds,\int_{0}^{T_{\mathrm{esc}}}e^{-(T_{\mathrm{esc}}-\tau)}g(Ze^{-\tau})\,\mathrm{d}\tau=\frac{1}{X}\int_{1}^{X}g\left(\frac{Z}{s}\right)\,\mathrm{d}s,

the transition map is equivalently described by4

\begin{cases} R(Z,X)=0,\\ Z^{+}=h(Z/X), \end{cases} \qquad X\in[1,Z/\phi^{-1}(1)), \tag*{(12)}\end{cases}

where

R(Z,X):=κX∫1Xg(Zs) ds−h(ZX).(13)R(Z,X):=\frac{\kappa}{X}\int_{1}^{X}g\left(\frac{Z}{s}\right)\,\mathrm{d}s-h\left(\frac{Z}{X}\right). \tag*{(13)}

Thus, each saliency update is computed by first selecting the smallest X∈[1,Z/ϕ−1(1))X\in[1,Z/\phi^{-1}(1)) satisfying R(Z,X)=0R(Z,X)=0, when such a solution exists, and then setting Z+=h(Z/X)Z^{+}=h(Z/X). If no such XX exists, the transition fails and the trajectory collapses to the inactive state.

Analytical illustration and numerical validation

We illustrate the results of § 5 with an analytically tractable example and further assess their validity and robustness through numerical experiments. See also Appendix § 17.

We consider the saliency update (12)-(13) with the following nonlinearities

ψ(x)=x1+∣x∣,ϕ(z)=∣z∣.(14)\psi(x)=\frac{x}{1+|x|},\qquad\phi(z)=|z|. \tag*{(14)}

In particular, we will refer to such ψ\psi as SmoothF. On the positive branch z≥1z\ge1, we have f(γ)=γ/ψ(γ)=γ+1f(\gamma)=\gamma/\psi(\gamma)=\gamma+1, hence f−1(z)=z−1f^{-1}(z)=z-1. Therefore g(z)=ψ(f−1(ϕ(z)))=1−1/zg(z)=\psi(f^{-1}(\phi(z)))=1-1/z and h(z)=z2h(z)=z^{2} for z≥1z\ge1. Substituting into (13), for Z>1Z>1 and X∈[1,Z)X\in[1,Z),

∫1Xg(Zs) ds=(X−1)(1−X+12Z).\int_{1}^{X}g\left(\frac{Z}{s}\right)\,\mathrm{d}s=(X-1)\left(1-\frac{X+1}{2Z}\right).

Hence the equation R(Z,X)=0R(Z,X)=0 is equivalent to

G(Z,X)=1κ,G(Z,X):=−X(X−1)(X+1−2Z)2Z2.G(Z,X)=\frac{1}{\kappa},\qquad G(Z,X):=-\frac{X(X-1)(X+1-2Z)}{2Z^{2}}.

Thus, given ZZ, the update is computed by first selecting the smallest admissible solution X∈[1,Z)X\in[1,Z) of this equation, and then setting Z+=Z2/X2Z^{+}=Z^{2}/X^{2}. If no such solution exists, the transition fails.

Fixed points of this scalar map characterize self-sustained retrieval across repeated transitions: Z+=ZZ^{+}=Z means that each jump regenerates the same saliency amplitude. Since Z+=Z2/X2Z^{+}=Z^{2}/X^{2}, fixed points satisfy Z=X2Z=X^{2}, and the implicit equation for XX becomes

1κ=G(X2,X)=(X−1)2(2X+1)2X5.\frac{1}{\kappa}=G(X^{2},X)=\frac{(X-1)^{2}(2X+1)}{2X^{5}}.

Thus fixed points exist when the level 1/κ1/\kappa intersects the curve GG. Since GG has a unique maximum at Xˉ=(5+105)/8\bar{X}=(5+\sqrt{105})/8, with G(Xˉ2,Xˉ)≃0.0785G(\bar{X}^{2},\bar{X})\simeq0.0785, fixed points exist only for κ≥κcrit:=1/G(Xˉ2,Xˉ)≃12.74\kappa\ge\kappa_{\mathrm{crit}}:=1/G(\bar{X}^{2},\bar{X})\simeq12.74. In particular, for κ>κcrit\kappa>\kappa_{\mathrm{crit}}, two fixed points appear Zˉ1\bar{Z}_{1} and Zˉ2\bar{Z}_{2}, Zˉ1<Zˉ2\bar{Z}_{1}<\bar{Z}_{2}, with Zˉ1\bar{Z}_{1} unstable and Zˉ2\bar{Z}_{2} stable. Since X=eTescX=e^{T_{\mathrm{esc}}} and Zˉ2=X2\bar{Z}_{2}=X^{2} at the stable fixed point, the asymptotic escape time is Tesc∞=12log⁡Zˉ2T_{\mathrm{esc}}^{\infty}=\frac{1}{2}\log\bar{Z}_{2}. Further details are given in Appendix § 12.

Fig. 4 illustrates the preceding analysis and extends it numerically to the case ψ=tanh⁡\psi=\tanh. Fig. 4(B) compares the SmoothF and Tanh activation functions, while Fig. 4(A) shows the induced saliency update map for SmoothF, revealing the emergence of two fixed points for gains exceeding the critical value κ>κcrit≃12.74\kappa>\kappa_{\mathrm{crit}}\simeq12.74. Fig. 4(C)-(F) present representative trajectories for subcritical and supercritical values of κ\kappa using the SmoothF activation. For orthogonal memories (Fig. 4(C)-(D)), subcritical gains lead to collapse into an inactive state, whereas supercritical gains give rise to self-sustained sequential retrieval. In contrast, for random memories (Fig. 4(E)-(F)), both gain regimes exhibit sustained transitions, likely due to cross-talk noise and the consequent lack of entrainment to a fixed asymptotic value. In the subcritical regime, the resulting zz-oscillations have small amplitude and may therefore lead to collapse only on longer timescales. Finally, Fig. 4(G)-(H) show that the qualitative features of the slow zz-dynamics persist with Tanh activation and random memories.

We next validate the theoretical escape-time predictions through simulations of the full model. For orthogonal memories, numerical escape times closely match the predictions (Figure 5(A)), while the switching times TswitchT_{\mathrm{switch}}, defined as the times at which the network transitions to the successor memory, occur slightly later due to the finite time required to leave the unstable memory state. Agreement between theoretical and numerical asymptotic escape times Tesc∞=12log⁡Zˉ2T_{\mathrm{esc}}^{\infty}=\frac{1}{2}\log\bar{Z}_2 improves with κ\kappa (Figure 5(B)). Adding Gaussian noise to the fast dynamics reduces the mean switching delay Tswitch−TescT_{\mathrm{switch}}-T_{\mathrm{esc}}, eventually making it negative due to premature switching (Figure 5(C)).

Figure 5. (A) Theoretical and numerical escape times TescT_{\mathrm{esc}} and observed switching times TswitchT_{\mathrm{switch}} across consecutive transitions (κ=15\kappa= 15, Z0=8Z_0 = 8). (B) Asymptotic escape time versus κ\kappa: theory and numerical estimates. (C) Mean switching delay Tswitch−TescT_{\mathrm{switch}} - T_{\mathrm{esc}} versus Gaussian noise amplitude σ\sigma. (D) Probability of correct transitions in the (κ,ϵ)(\kappa, \epsilon) plane. (E) Asymptotic escape time versus κ\kappa and memory bias ϵ\epsilon. (F) Asymptotic saliency amplitude versus κ\kappa and ϵ\epsilon. The numerical configuration is the same as in Figure 4. Panels (A-C) use orthogonal memories, while (D-F) use biased random memories, with 10 independent realizations per parameter pair. In (B), asymptotic escape times are estimated over the last 8 time units of simulations of duration T=40T = 40 (approximately 5-7 transitions). Panel (C) uses Gaussian noise with standard deviation σ/N\sigma/\sqrt{N} and 64 realizations per noise amplitude. Hatched regions in (D-F) indicate no successful transitions.

We then investigate robustness to non-orthogonal memories by independently sampling biased binary patterns, as in [19], according to P(ξiμ=±1)=(1±ϵ)/2P(\xi_i^\mu=\pm1)=(1\pm\epsilon)/2, where ϵ∈[0,1]\epsilon\in[0,1] induces an expected normalized pairwise overlap of ϵ\epsilon. Figure 5(D) shows that the critical gain for successful retrieval increases only slightly with memory bias ϵ\epsilon, while retrieval reliability decreases sharply for ϵ≳0.4\epsilon\gtrsim0.4. No successful transitions are observed for ϵ>0.5\epsilon>0.5 over the investigated gains. Conditional on successful transitions, both the asymptotic escape time (Figure 5(E)) and saliency amplitude (Figure 5(F)) increase with κ\kappa. As memory bias ϵ\epsilon increases, escape times decrease, while saliency remains approximately constant. Overall, these results suggest that the theoretical estimates derived for orthogonal memories remain informative for moderately correlated memories (ϵ≲0.35\epsilon\lesssim0.35), while stronger correlations lead to significant deviations and eventually compromise sequential retrieval. Additional numerical results for the Tanh activation are provided in Appendix § 13.

Limitations and future research

While this work presents an analytically tractable framework for sequential retrieval in Hopfield networks, several important limitations remain. First, our analysis is restricted to canonical Tanh-type activation functions and does not address the practically relevant case ψ=softmax⁡\psi=\operatorname{softmax}. This regime is of particular interest, as it directly links Hopfield dynamics to self-attention mechanisms in Transformer architectures [10] and is known to enable exponential memory capacity [9]. Extending the present theory to non-diagonal activation functions represents a promising direction for future work, with the potential to provide dynamical interpretations of inference in modern Transformer-based models. Second, although we provide a qualitative geometric characterization of the invariant sets and unstable structures supporting sequential retrieval, we do not yet precisely quantify the ranges of slow-fast timescale separation for which transitions emerge intrinsically, without reliance on numerical error or small-amplitude noise to trigger adherence to unstable fibers. A more refined geometric analysis is needed to delineate the parameter regimes that permit robust, noise-free sequential transitions. Third, while our results establish conditions for sequential retrieval, they do not characterize how retrieval reliability scales with the number of stored memories and the length of the retrieved sequence. Quantifying these capacity limits remains an important open problem, preliminarily explored numerically in Appendix § 14. Fourth, our theoretical analysis is restricted to deterministic transition graphs with at most one outgoing edge per memory. Extending the theory to branching graphs is another promising direction, as preliminary simulations in Appendix § 15 suggest that memory overlap can influence branch selection. Finally, the role of external inputs in controlling retrieval sequences remains largely unexplored. Developing a theoretical framework for steering memory transitions through external signals is an interesting direction for future research, illustrated by the associative-memory finite-state machine in Appendix § 16.

Appendix

The IDP Hopfield model

The IDP Hopfield model presented in Betteti et al. [ [23] is characterized by the O.D.E.

x˙=−x+W(α)Ψ(x)α∈R>0P(15)\dot{x}=-x+W(\alpha)\Psi(x) \qquad\alpha\in\mathbb{R}_{>0}^{P} \tag*{(15)}

where W(α)W(\alpha) is the synaptic matrix of neural interaction built from the memories of the model. The memories ξ1,…,ξP\xi^{1},\ldots,\xi^{P} and associated synaptic matrix W(α)W(\alpha) obey the following assumption.

Assumption 1 (Memory and synaptic matrix). Let P∈NP\in\mathbb{N} and define the set of orthogonal memories

Σ:={ξ1,…,ξP}⊆{−1,1}N(16)\Sigma:=\{\xi^{1},\ldots,\xi^{P}\}\subseteq\{-1,1\}^{N} \tag*{(16)}
ξμ⊤ξν=Nδμ,νμ,ν∈{1,…,P}(17)\xi^{\mu\top}\xi^{\nu}=N\delta_{\mu,\nu}\qquad\mu,\nu\in\{1,\ldots,P\} \tag*{(17)}

where δμ,ν\delta_{\mu,\nu} is the Kronecker delta for indices μ\mu and ν\nu. For α∈R>0P\alpha\in\mathbb{R}_{>0}^{P} positive we define the input-modulated synaptic matrix W(α)∈RN×NW(\alpha)\in\mathbb{R}^{N\times N} as

W(α)=1N∑μ=1Pαμξμξμ⊤.(18)W(\alpha)=\frac{1}{N}\sum_{\mu=1}^{P}\alpha_{\mu}\xi^{\mu}\xi^{\mu\top}. \tag*{(18)}

The orthogonal memories constraint is necessary to derive rigorous results on existence and stability of fixed points, but can be relaxed to orthogonality in expectation in implementations:

P[ξiν=1]=P[ξiν=−1]=12∀ν=1,…,P,∀i=1,…,N.(19)\mathbb{P}[\xi_{i}^{\nu}=1]=\mathbb{P}[\xi_{i}^{\nu}=-1]=\frac{1}{2} \qquad\forall\nu=1,\ldots,P,\quad\forall i=1,\ldots,N. \tag*{(19)}

Remark 1. Notice that by constructing the memory matrix

M=1N[ξ1,…,ξP](20)M=\frac{1}{\sqrt{N}}[\xi^{1},\ldots,\xi^{P}] \tag*{(20)}

we can rewrite the input-modulated synaptic matrix as

W(α)=Mdiag⁡(α)M⊤(21)W(\alpha)=M\operatorname{diag}(\alpha)M^{\top} \tag*{(21)}

The canonical synaptic matrix WW of the Hopfield model is a specific instance of (18) for the choice α=1\alpha=1 (or a scalar multiple of it).

We now introduce an assumption on the saliency function Φ\Phi going from the space of the slow neural feedback z∈RPz\in\mathbb{R}^{P} to the saliency space α∈RP\alpha\in\mathbb{R}^{P} such that α=Φ(z)\alpha=\Phi(z).

Assumption 2. The saliency function Φ:RP→RP\Phi:\mathbb{R}^{P}\to\mathbb{R}^{P} satisfies the following properties:

  • (i) (diagonality and homogeneity) for all ν=1,…,P\nu=1,\ldots,P we have Φν(z)=ϕ(zν)\Phi_{\nu}(z)=\phi(z_{\nu});

  • (ii) (symmetry and positivity) ϕ(s)=ϕ(−s)≥0\phi(s)=\phi(-s)\geq0, with αν(0)=0\alpha_{\nu}(0)=0;

  • (iii) (symmetric monotonicity) ϕ(s)\phi(s) is monotonically increasing on (0,+∞)(0,+\infty).

Likewise, we introduce some smoothness assumptions on the activation function Ψ\Psi of the IDP Hopfield network, which will be useful for the characterization of the existence and local stability conditions for the equilibria of (15).

Assumption 3. The activation function Ψ:RN→RN\Psi:\mathbb{R}^{N}\to\mathbb{R}^{N} satisfies the following properties:

  • (i) (diagonality, homogeneity and smoothness) for all i=1,…,Ni=1,\ldots,N, we have Ψi(x)=ψ(xi)\Psi_{i}(x)=\psi(x_{i}) with ψ∈C1(R)\psi\in C^{1}(\mathbb{R});

  • (ii) (odd and saturating) ψ(s)=−ψ(−s)\psi(s)=-\psi(-s) for all s∈Rs\in\mathbb{R} and lim⁡s→±∞ψ(s)=±1\lim_{s\to\pm\infty}\psi(s)=\pm1.

  • (iii) (bounded monotonicity) ψ′(s)∈(0,1]\psi'(s)\in(0,1] for all s∈Rs\in\mathbb{R} and ψ′(0)=1\psi'(0)=1;

  • (iv) (monotonicity of the derivative) ψ′\psi' is monotonically decreasing on (0,+∞)(0,+\infty).

We will now state some of the main analytical results presented in [23 Supplementary Information], starting from the existence conditions for the equilibria of (15). The presented results are stated with respect to the saliency states α=Φ(z)\alpha=\Phi(z).

Theorem 2 (Existence of retrievable memories). Let ξρ\xi^{\rho} be a memory as in Assumption 1. Then xρ=γρξρx_{\rho}=\gamma_{\rho}\xi^{\rho} is an equilibrium for (15) for some unique γρ∈R∖{0}\gamma_{\rho}\in\mathbb{R}\setminus\{0\} if and only if αρ>1\alpha_{\rho}>1. In this case, γρ\gamma_{\rho} satisfies

αρ=f(γρ)=γρψ(γρ)(22)\alpha_{\rho}=f(\gamma_{\rho})=\frac{\gamma_{\rho}}{\psi(\gamma_{\rho})} \tag*{(22)}

We now provide the main result on the local stability of the equilibria for the fast IDP Hopfield dynamics.

Theorem 3 (Local stability of equilibria and memories). Consider the dynamics (15). Then

  1. an equilibrium x⋆∈RNx^{\star}\in\mathbb{R}^{N} is locally exponentially stable if and only if

−[DΨ(x⋆)]−1+W(α)≺0(23)-[D\Psi(x^{\star})]^{-1}+W(\alpha)\prec0 \tag*{(23)}
  1. if xρ=γρξρx_{\rho}=\gamma_{\rho}\xi^{\rho} with γρ∈R∖{0}\gamma_{\rho}\in\mathbb{R}\setminus\{0\} is an equilibrium, then xρx_{\rho} is locally exponentially stable if and only if

ψ′(γρ)<1max⁡μ=1,…,Pαμ(24)\psi'(\gamma_{\rho})<\frac{1}{\max_{\mu=1,\ldots,P}\alpha_{\mu}} \tag*{(24)}

In particular, the study of the local stability condition allows to provide exact conditions on the points where the Jacobian becomes has eigenvalues with null real part, which will later be useful for the characterization of the memory manifold (see Section C). We will restrict ourselves to equilibrium conditions characterized by γρ>0\gamma_{\rho}>0, as the case γρ<0\gamma_{\rho}<0 is associated to the so called anti-memory −ξρ-\xi^{\rho} and is a symmetric artifact of classical Hopfield models. Notice in particular that by property 3(iv) the derivative ψ′\psi' is locally invertible in (0,+∞)(0,+\infty), and we can define the following stability function.

Definition 1 (Saliency stability function). We define the saliency stability function ω:R>0→R>0\omega:\mathbb{R}_{>0}\to\mathbb{R}_{>0} as

ω(s)=f∘ψ′−1(1s)=ψ′−1(1s)ψ(ψ′−1(1s))(25)\omega(s)=f\circ\psi'^{-1}\left(\frac{1}{s}\right)=\frac{\psi'^{-1}\left(\frac{1}{s}\right)}{\psi\left(\psi'^{-1}\left(\frac{1}{s}\right)\right)} \tag*{(25)}

Before providing the key results on the eigenvalues of the Jacobian of the IDP Hopfield vector field, we establish the basic invertibility and monotonicity properties of the saliency stability function.

Lemma 1 (Invertibility of the saliency stability function). The saliency stability function ω:R>0→R>0\omega:\mathbb{R}_{>0}\to\mathbb{R}_{>0} is invertible, and in particular

ω−1(r)=1ψ′∘f−1(r),r>0.(26)\omega^{-1}(r)=\frac{1}{\psi'\circ f^{-1}(r)},\qquad r>0. \tag*{(26)}

Furthermore, it holds that

  1. ω(s)<s\omega(s)<s;

  1. ω−1(r)>r\omega^{-1}(r)>r.

Proof. The paragraph above Definition 1 establishes the invertibility of ψ′\psi' in (0,+∞)(0,+\infty). Considering then the function f(γ)=γψ(γ)f(\gamma)=\frac{\gamma}{\psi(\gamma)} we immediately observe that for any γ>0\gamma>0 we have

f′(γ)=ψ(γ)−γψ′(γ)ψ′(γ)2.(27)f'(\gamma)=\frac{\psi(\gamma)-\gamma\psi'(\gamma)}{\psi'(\gamma)^2}. \tag*{(27)}

From Assumption 3(iv) we have that ψ′\psi' is monotonically decreasing on (0,+∞)(0,+\infty) and in particular we have

ψ(γ)=∫0γψ′(s) ds>∫0γψ′(γ) ds=γψ′(γ).(28)\psi(\gamma)=\int_{0}^{\gamma}\psi'(s)\,\mathrm{d}s>\int_{0}^{\gamma}\psi'(\gamma)\,\mathrm{d}s=\gamma\psi'(\gamma). \tag*{(28)}

Consequently f′(γ)>0f'(\gamma)>0 for all γ>0\gamma>0 and in particular ff is invertible on (0,+∞)(0,+\infty). Taking ω−1(r)=1/(ψ′∘f−1(r))\omega^{-1}(r)=1/(\psi'\circ f^{-1}(r)) we observe that

ω−1∘ω(s)=1ψ′∘f−1∘f∘ψ′−1(1s)(29)\omega^{-1}\circ\omega(s)=\frac{1}{\psi'\circ f^{-1}\circ f\circ\psi'^{-1}\left(\frac{1}{s}\right)} \tag*{(29)}
=1s−1=s(30)=\frac{1}{s^{-1}}=s \tag*{(30)}

The proof of points 1-2 is straightforward, and relies on the fact that γψ′(γ)−ψ(γ)<0\gamma\psi'(\gamma)-\psi(\gamma)<0 for all γ>0\gamma>0, as proven in (28).

  1. ω(s)<s\omega(s) < s:

We are interested in proving that

ω(s)−s=f∘ψ−1(1s)−s<0.(31)\omega(s) - s = f \circ\psi^{-1}\left(\frac{1}{s}\right) - s < 0. \tag*{(31)}

Taking γ=ψ−1(1s)\gamma= \psi^{-1}\left(\frac{1}{s}\right), we can express s=1ψ′(γ)s = \frac{1}{\psi'(\gamma)}, and therefore our condition reduces to

f(γ)−1ψ′(γ)=γψ(γ)−1ψ′(γ)<0.(32)f(\gamma) - \frac{1}{\psi'(\gamma)} = \frac{\gamma}{\psi(\gamma)} - \frac{1}{\psi'(\gamma)} < 0. \tag*{(32)}
  1. ω−1(r)>r\omega^{-1}(r) > r:

We are interested in proving that

1ω−1(r)−1r=ψ′∘f−1(r)−1r<0.(33)\frac{1}{\omega^{-1}(r)} - \frac{1}{r} = \psi' \circ f^{-1}(r) - \frac{1}{r} < 0. \tag*{(33)}

Taking γ=f−1(r)\gamma= f^{-1}(r), we can express r=f(γ)r = f(\gamma), and therefore our condition reduces to

ψ′(γ)−1f(γ)=ψ′(γ)−ψ(γ)γ<0.(34)\psi'(\gamma) - \frac{1}{f(\gamma)} = \psi'(\gamma) - \frac{\psi(\gamma)}{\gamma} < 0. \tag*{(34)}

The saliency stability function allows us to state an important proposition on the eigenvalues of the Jacobian of [15] at the equilibrium points.

Proposition 1 (Eigenvalues of the IDP Hopfield Jacobian). Let α∈R>0\alpha\in\mathbb{R}_{>0} and define F(x)=−x+W(α)Ψ(x)F(x) = -x + W(\alpha)\Psi(x). Let αν>1\alpha_\nu> 1 such that for γν=f−1(αν)\gamma_\nu= f^{-1}(\alpha_\nu) the equilibrium xν=γνξνx_\nu= \gamma_\nu\xi^\nu exists. Then the Jacobian matrix

∂F∂x(xν)=−I+W(α)∂Ψ∂x(xν)(35)\frac{\partial F}{\partial x}(x_\nu) = -I + W(\alpha)\frac{\partial\Psi}{\partial x}(x_\nu) \tag*{(35)}

has one eigenvalue λμ\lambda_\mu with ℜ{λμ}=0\Re\{\lambda_\mu\} = 0 iff

∃μ∈{1,…,P}:αν=ω(αμ).(36)\exists\mu\in\{1,\ldots,P\}: \qquad\alpha_\nu= \omega(\alpha_\mu). \tag*{(36)}

Proof. From [23 Theorems 2, SI] we have that

∂F∂x(xν)=−I+W(α)∂Ψ∂x(xν)(37)\frac{\partial F}{\partial x}(x_\nu) = -I + W(\alpha)\frac{\partial\Psi}{\partial x}(x_\nu) \tag*{(37)}

This has the same spectrum of the matrix

∂Ψ∂x(xν)1/2∂F∂x(xν)∂Ψ∂x(xν)−1/2=∂Ψ∂x(xν)1/2(−∂Ψ∂x(xν)−1+W(α))∂Ψ∂x(xν)1/2\frac{\partial\Psi}{\partial x}(x_\nu)^{1/2}\frac{\partial F}{\partial x}(x_\nu)\frac{\partial\Psi}{\partial x}(x_\nu)^{-1/2} = \frac{\partial\Psi}{\partial x}(x_\nu)^{1/2} \left(-\frac{\partial\Psi}{\partial x}(x_\nu)^{-1}+W(\alpha)\right) \frac{\partial\Psi}{\partial x}(x_\nu)^{1/2}

This matrix has the same inertia [28] of the matrix

DF(xν)=−∂Ψ∂x(xν)−1+W(α)DF(x_\nu)=-\frac{\partial\Psi}{\partial x}(x_\nu)^{-1}+W(\alpha)

that has spectrum

ρ(DF(xν)):={α1−1ψ′(γν),…,αP−1ψ′(γν),−1,…,−1⏟N−P}.(38)\rho\left(DF(x_\nu)\right) := \left\{\alpha_1-\frac{1}{\psi'(\gamma_\nu)},\ldots,\alpha_P-\frac{1}{\psi'(\gamma_\nu)},\underbrace{-1,\ldots,-1}_{N-P}\right\}. \tag*{(38)}

From this it is immediate to see that ∂F∂x(xν)\frac{\partial F}{\partial x}(x_\nu) has null eigenvalues iff there exists μ∈{1,…,P}\mu\in\{1,\ldots,P\} such that

ψ′(γν)=1αμ.(39)\psi'(\gamma_\nu)=\frac{1}{\alpha_\mu}. \tag*{(39)}

Exploiting the invertibility of ψ′\psi' in (0,+∞)(0,+\infty), we get the equivalent condition

γν=ψ′−1(1αμ)∃μ∈{1,…,P}.(40)\gamma_\nu=\psi'^{-1}\left(\frac{1}{\alpha_\mu}\right)\quad\exists\mu\in\{1,\ldots,P\}. \tag*{(40)}

Since αν>1\alpha_\nu> 1, then exploiting the uniqueness of γν>0\gamma_\nu> 0 in the equilibrium condition in (22) we have that the zero eigenvalue condition in (40) coincides with

αν=f(γν)=f∘ψ′−1(1αμ)=ω(αμ)∃μ∈{1,…,P}.(41)\begin{aligned} \alpha_\nu&= f(\gamma_\nu)\\ &= f\circ\psi'^{-1}\left(\frac{1}{\alpha_\mu}\right)=\omega(\alpha_\mu)\quad\exists\mu\in\{1,\ldots,P\}. \tag*{(41)} \end{aligned}

In particular, notice that from Lemma 1 for all ν=1,…,P\nu=1,\ldots,P it holds that αν>ω(αν)\alpha_\nu>\omega(\alpha_\nu), so that whenever at least a memory exists, at least one is also locally exponentially stable. Notice in addition that the number of eigenvalues with positive and negative real part is obtained by counting the instances where the condition αν≶ω(αμ)\alpha_\nu\lessgtr\omega(\alpha_\mu) is satisfied for μ=1,…,P\mu=1,\ldots,P.

A modern reformulation of the sequential IDP Hopfield model

The IDP Hopfield model admits a modern Hopfield Krotov and Hopfield, [8] interpretation in which retrieval is decomposed into three interacting processes: a memory layer that reads out the similarity between the current feature state and the stored memories, a saliency layer that modulates this readout, and a feature layer that reconstructs the retrieved representation Betteti et al., [23]. In the original IDP model, the saliency layer is driven by the external input. Here we show that the sequential IDP Hopfield model (3) admits the same interpretation, with the saliency layer driven instead by internal graph-structured feedback.

We introduce an auxiliary memory-index variable y∈RPy \in\mathbb{R}^{P} and consider

τxx˙=−x+Mdiag⁡(Φ(z))y,τyy˙=−y+MTΨ(x),τzz˙=−z+χ(y).\begin{aligned} \tau_x \dot{x} &= -x + M\operatorname{diag}(\Phi(z))y,\\ \tau_y \dot{y} &= -y + M^{\mathsf{T}}\Psi(x),\\ \tau_z \dot{z} &= -z + \chi(y). \end{aligned}

The variable yy is the memory-layer activity: its components encode the alignment between the current activated feature state and the stored memories. The map χ:RP→RP\chi: \mathbb{R}^{P} \to\mathbb{R}^{P} is a slow feedback map prescribing the target saliency profile toward which zz relaxes.

Under the timescale separation τy≪τx≪τz\tau_y \ll\tau_x \ll\tau_z, the memory layer rapidly satisfies

y≃MTΨ(x).y \simeq M^{\mathsf{T}}\Psi(x).

Substituting this quasi-steady relation into the feature dynamics gives

τxx˙=−x+Mdiag⁡(Φ(z))MTΨ(x),\tau_x \dot{x} = -x + M\operatorname{diag}(\Phi(z))M^{\mathsf{T}}\Psi(x),

which is exactly the fast equation of the sequential IDP Hopfield model (3).

The slow equation in (3) is recovered by choosing

χ(y)=κNAy.\chi(y) = \frac{\kappa}{\sqrt{N}}Ay.

Indeed, replacing yy by its fast quasi-steady value yields

τzz˙=−z+κNAMTΨ(x),\tau_z \dot{z} = -z + \frac{\kappa}{\sqrt{N}}AM^{\mathsf{T}}\Psi(x),

which coincides with the slow feedback dynamics of (3).

This reformulation clarifies the relation with the modern IDP interpretation. In the original IDP model, saliencies are prescribed by an external input decomposition. In the sequential IDP model, saliencies are generated autonomously by the retrieved memory through the transition graph.

Memory manifolds: pouring the concrete

In the classic slow-fast characterization from geometric singular perturbation theory (GSPT) presented in Fenichel [ [26], the timescale separation between coupled dynamics is extremized in order to derive geometric information about the loci in state-space where the system evolves. In the case of the IDP Hopfield model, the coupled system can be written as

εx˙=−x+Mdiag⁡(Φ(z))MTΨ(x),x(0)∈RN,z˙=−z+κNAMTΨ(x),z(0)∈RP.(42)\begin{aligned} \varepsilon\dot{x} &= -x + M\operatorname{diag}(\Phi(z))M^{\mathsf{T}}\Psi(x), \qquad x(0) \in\mathbb{R}^{N},\\ \dot{z} &= -z + \frac{\kappa}{\sqrt{N}}AM^{\mathsf{T}}\Psi(x), \qquad z(0) \in\mathbb{R}^{P}. \tag*{(42)} \end{aligned}

and taking the limit ε→0+\varepsilon\to0^{+}, the problem simplifies to studying how the dynamics

z˙=−z+κNAMTΨ(x⋆(z))(43)\dot{z} = -z + \frac{\kappa}{\sqrt{N}}AM^{\mathsf{T}}\Psi(x^{\star}(z)) \tag*{(43)}

evolve under the solution constraint, parametrized as a function of the slow variable zz, ΩN=−x⋆(z)+Mdiag⁡(Φ(z))MTΨ(x⋆(z))\Omega_N = -x^{\star}(z) + M\operatorname{diag}(\Phi(z))M^{\mathsf{T}}\Psi(x^{\star}(z)). We start by studying the solutions of the algebraic constraint, which characterize the critical set

S0:={(x,z)∈RN×RP:x=Mdiag⁡(Φ(z))MTΨ(x)}(44)\mathcal{S}_0 := \left\{(x,z) \in\mathbb{R}^{N} \times\mathbb{R}^{P} : x = M\operatorname{diag}(\Phi(z))M^{\mathsf{T}}\Psi(x)\right\} \tag*{(44)}

The origin manifold

Memory branches and memory manifolds

Exploiting Theorem 2 for the existence of equilibria in the IDP Hopfield model, we proceed to define the memory branches, which are disjoint sets in S0S_0 associated each to a different memory.

Definition 2. Let ϕ(zν)>1\phi(z_\nu)>1. Then the νth\nu^{\mathrm{th}} memory branch is the PP-dimensional set Nν⊆S0\mathcal{N}_\nu\subseteq S_0 defined as

Nν={(x,z)∈RN×RP:∃γν≠0, x=γνξν, ϕ(zν)=f(γν)},(45)\mathcal{N}_\nu= \left\{(x,z)\in\mathbb{R}^{N}\times\mathbb{R}^{P}:\exists\gamma_\nu\ne0,\ x=\gamma_\nu\xi^\nu,\ \phi(z_\nu)=f(\gamma_\nu)\right\}, \tag*{(45)}

Notice that the memory branch Nν\mathcal{N}_\nu is not defined for ϕ(zν)<1\phi(z_\nu)<1, as the equilibria associated to the memory ξν\xi^\nu does not exists. The memory manifold is then defined as a PP-dimensional manifold comprising the points in Nν\mathcal{N}_\nu where normal hyperbolicity holds.

Definition 3. Let F(x,z)=−x+W(Φ(z))Ψ(x)F(x,z)=-x+W(\Phi(z))\Psi(x) and assume ϕ(zν)>1\phi(z_\nu)>1. Then the νth\nu^{\mathrm{th}} memory manifold Mν\mathcal{M}_\nu is the set of points (x,z)∈Nν(x,z)\in\mathcal{N}_\nu such that

∄λ∈ρ(∂F∂x(x,z)): ℜ(λ)=0.(46)\not\exists\lambda\in\rho\left(\frac{\partial F}{\partial x}(x,z)\right):\ \Re(\lambda)=0. \tag*{(46)}

Of particular relevance are two disjoint regions of the manifold Mν\mathcal{M}_\nu, which play qualitatively distinct roles in shaping the system’s long-term behavior. Rather than pursuing formal definitions, we give an intuitive description aimed at conveying their dynamical function:

  • Attracting manifold Mνa\mathcal{M}^{a}_\nu: this region consists of points that act as local sinks for the dynamics. Trajectories initialized in a neighborhood of these points in the ambient space are drawn toward the manifold and remain confined to it over time. Geometrically, Mνa\mathcal{M}^{a}_\nu behaves as a stabilizing surface that captures and retains nearby trajectories.

  • Saddle manifold Mνs\mathcal{M}^{s}_\nu: this region contains points that are simultaneously attracting and repelling, depending on direction. Trajectories may approach the manifold transiently, but the presence of unstable directions eventually pushes them away. As a result, Mνs\mathcal{M}^{s}_\nu acts as a transient conduit, enabling trajectories to leave one attracting region and move toward another.

One of the advantages of relying on the IDP Hopfield model to capture sequentiality and memory transition is that there already exists a complete theory on the stability of system’s equilibria. We exploit exactly this theory to characterize the attracting and saddle memory manifolds.

Lemma 2 (Attracting and saddle manifolds). Let ω:R>0→R>0\omega:\mathbb{R}_{>0}\to\mathbb{R}_{>0} be the saliency stability function in Definition 1. Then the point (x,z)∈Mν(x,z)\in\mathcal{M}_\nu on the memory manifold is

  • attracting if ϕ(zν)>ω(ϕ(zμ))\phi(z_\nu)>\omega(\phi(z_\mu)) for all μ∈{1,…,P}\mu\in\{1,\ldots,P\};

  • saddle type if there exists at least one μ∈{1,…,P}/{ν}\mu\in\{1,\ldots,P\}/\{\nu\} such that ϕ(zν)<ω(ϕ(zμ))\phi(z_\nu)<\omega(\phi(z_\mu)).

Proof. The manifold Mν\mathcal{M}_\nu is attracting at (x,z)(x,z) if for all

λ∈ρ(∂F∂x(x,z))ℜ(λ)<0(47)\lambda\in\rho\left(\frac{\partial F}{\partial x}(x,z)\right)\qquad\Re(\lambda)<0 \tag*{(47)}

and a saddle if there exists at least an eigenvalue λ\lambda such that ℜ(λ)>0\Re(\lambda)>0. From Proposition 1, we have that the eigenvalues of the Jacobian ∂F∂x\frac{\partial F}{\partial x} at (x,z)∈Mν(x,z)\in\mathcal{M}_\nu have the same sign of {ϕ(zμ)−1ψ′(γν)}μ=1P\left\{\phi(z_\mu)-\frac{1}{\psi'(\gamma_\nu)}\right\}_{\mu=1}^{P} with γν=f−1(ϕ(zν))\gamma_\nu=f^{-1}(\phi(z_\nu)). Thus, Mν\mathcal{M}_\nu is attracting at (x,z)(x,z) iff for all μ=1,…,P\mu=1,\ldots,P

ϕ(zμ)−1ψ′(γν)<0⟺ϕ(zν)>ω(ϕ(zμ)).(48)\phi(z_\mu)-\frac{1}{\psi'(\gamma_\nu)}<0 \quad\Longleftrightarrow\quad \phi(z_\nu)>\omega(\phi(z_\mu)). \tag*{(48)}

Conversely, it is a saddle if there exists μ∈{1,…,P}/{ν}\mu\in\{1,\ldots,P\}/\{\nu\} such that

ϕ(zμ)−1ψ′(γν)>0⟺ϕ(zν)<ω(ϕ(zμ)).(49)\phi(z_\mu)-\frac{1}{\psi'(\gamma_\nu)}>0 \quad\Longleftrightarrow\quad \phi(z_\nu)<\omega(\phi(z_\mu)). \tag*{(49)}

The phenomena known as retrieval and fixation to the memory ξν\xi^\nu occur for points (x,z)∈Mν(x,z)\in\mathcal{M}_\nu such that the memory manifold is attracting, i.e., the respective saliency weight ϕ(zν)\phi(z_\nu) is greater than the saliency stability function computed at any ϕ(zμ)\phi(z_\mu). Escape direction emerge instead when the dynamics evolve towards a saddle region of the memory manifold. In general, we have that the attracting manifold for the memory ξν\xi^\nu is the subset Mνa:={(x,z)∈Mν:ϕ(zν)>ω(ϕ(zμ)) ∀μ=1,…,P}\mathcal{M}^{a}_\nu:=\{(x,z)\in\mathcal{M}_\nu:\phi(z_\nu)>\omega(\phi(z_\mu))\ \forall\mu=1,\ldots,P\}. Conversely, the saddle manifold is Mνs:={(x,z)∈Mν:∃μ=1,…,P ϕ(zν)<ω(ϕ(zμ))}\mathcal{M}^{s}_\nu:=\{(x,z)\in\mathcal{M}_\nu:\exists\mu=1,\ldots,P\ \phi(z_\nu)<\omega(\phi(z_\mu))\}.

The origin manifold

In our description of the geometric sets where the slow-fast activity of the system is constrained, we considered all the equilibria solutions of the fast IDP Hopfield dynamics that are associated to one of the memories. However, there is another solution that is independent from any of the memory vectors: the origin.

Definition 4 (Origin manifold). The origin manifold is defined as the set

M0:={(0N,z)∈RN×RP:∄λ∈ρ(∂F∂x(0N,z)), ℜ{λ}=0}.(50)\mathcal{M}_0 := \left\{(0_N,z) \in\mathbb{R}^{N} \times\mathbb{R}^{P} : \nexists\lambda\in\rho\left(\frac{\partial F}{\partial x}(0_N,z)\right),\ \Re\{\lambda\}=0\right\}. \tag*{(50)}

In particular, for any point in the set M0\mathcal{M}_0 we have that

∂F∂x(0N,z)=−I+Mdiag⁡(Φ(z))MT(51)\frac{\partial F}{\partial x}(0_N,z)=-I+M\operatorname{diag}(\Phi(z))M^{\mathsf{T}} \tag*{(51)}

Notice that the structure of the Jacobian ∂F∂x(0N,z)\frac{\partial F}{\partial x}(0_N,z) allows for a clean characterization of the attracting and saddle regions. Noticing that the spectrum of Jacobian is

ρ(∂F∂x(0N,z)):={−1+ϕ(z1),…,−1+ϕ(zP),−1,…,−1⏟N−P},(52)\rho\left(\frac{\partial F}{\partial x}(0_N,z)\right):=\{-1+\phi(z_1),\ldots,-1+\phi(z_P),\underbrace{-1,\ldots,-1}_{N-P}\}, \tag*{(52)}

we can characterize the not normally hyperbolic points as (0N,z)(0_N,z) such that there exists μ=1,…,P\mu=1,\ldots,P for which ϕ(zμ)=1\phi(z_\mu)=1. Similarly, we can characterize the:

  1. attracting origin manifold: M0a:={(0N,z):∥Φ(z)∥∞<1}\mathcal{M}_0^a:=\{(0_N,z):\|\Phi(z)\|_\infty<1\};

  2. saddle origin manifold: M0s:=M0∖M0a\mathcal{M}_0^s:=\mathcal{M}_0\setminus\mathcal{M}_0^a.

Of particular relevance is the attracting origin manifold M0a\mathcal{M}_0^a, since it is a sink for the slow-fast dynamics, and once trajectories enter this set they cannot escape it. Thereby, from M0a\mathcal{M}_0^a the activity inevitably collapses to the origin (0N,0P)∈RN×RP(0_N,0_P)\in\mathbb{R}^{N}\times\mathbb{R}^{P}, as formalized in the following Lemma.

Lemma 3 (Collapse to the origin). Let (x0,z0)∈M0a(x_0,z_0)\in\mathcal{M}_0^a. Then (x(t),z(t))→(0N,0P)(x(t),z(t))\to(0_N,0_P) exponentially fast.

Proof. For (x0,z0)∈M0a(x_0,z_0)\in\mathcal{M}_0^a the slow dynamics reduce to

z˙=−z+κNAMTΨ(0N)=−z.(53)\begin{aligned} \dot{z}=-z+\frac{\kappa}{\sqrt{N}}AM^{\mathsf{T}}\Psi(0_N) \\ &=-z. \tag*{(53)} \end{aligned}

Therefore, the slow dynamics are completely dissipative and z(t)=z0e−tz(t)=z_0e^{-t}.

It is easy to observe that, in particular, M0a\mathcal{M}_0^a is also forward-invariant for the reduced dynamics.

Slow saliency dynamics: under the hood

We provide the technical details for § 5. We first prove Theorem 1, and then analyze the explicit scalar map introduced in the example.

Proof of Theorem 1

We start with two auxiliary results. The first one is a corollary of Lemma 1.

Corollary 4. For every zz such that ϕ(z)>1\phi(z)>1, we have

ψ′(f−1(ϕ(z)))<1ϕ(z).\psi'\left(f^{-1}(\phi(z))\right)<\frac{1}{\phi(z)}.

Proof. From Lemma 1 we know that

ω−1(r)>r.(54)\omega^{-1}(r)>r. \tag*{(54)}

Taking r=ϕ(z)r=\phi(z) and the definition ω−1(r)=1/(ψ′∘f−1(r))\omega^{-1}(r)=1/(\psi'\circ f^{-1}(r)) we conclude.

In the two-memory reduction of § 5, the source and successor saliencies satisfy

zν(t)=Ze−t,z˙μ=−zμ+κg(Ze−t),g:=ψ∘f−1∘ϕ.z_{\nu}(t) = Ze^{-t}, \qquad\dot{z}_{\mu} = -z_{\mu} + \kappa g(Ze^{-t}), \qquad g := \psi\circ f^{-1} \circ\phi.

Hence

zμ(t)=κ∫0te−(t−τ)g(Ze−τ) dτ.(55)z_{\mu}(t) = \kappa\int_{0}^{t} e^{-(t-\tau)}g(Ze^{-\tau})\,\mathrm{d}\tau. \tag*{(55)}

Let Tmax⁡T_{\max} be defined by ϕ(Ze−Tmax⁡)=1\phi(Ze^{-T_{\max}})=1. The next lemma shows that, once the successor saliency catches up with the source saliency, the source branch must lose stability before the source memory disappears (see also Figure 6, right plot).

Lemma 4. Assume that there exists T‾∈[0,Tmax⁡)\overline{T} \in[0,T_{\max}) such that zμ(T‾)=zν(T‾)z_{\mu}(\overline{T})=z_{\nu}(\overline{T}). Then there exists T∈[T‾,Tmax⁡)T \in[\overline{T},T_{\max}) such that

ω−1(ϕ(zμ(T)))>ϕ(zμ(T))=ω−1(ϕ(zν(T))).(56)\omega^{-1}\bigl(\phi(z_{\mu}(T))\bigr) > \phi(z_{\mu}(T)) = \omega^{-1}\bigl(\phi(z_{\nu}(T))\bigr). \tag*{(56)}

Proof. Observe first that the inequality on the left is always true by Corollary 4. We need to prove that the equality holds true for some T∈[T‾,Tmax⁡)T \in[\overline{T},T_{\max}). To this aim we will prove that

ϕ(zμ(t))<ω−1(ϕ(zν(t)))for t=T‾(57)\phi(z_{\mu}(t)) < \omega^{-1}\bigl(\phi(z_{\nu}(t))\bigr) \quad\text{for } t=\overline{T} \tag*{(57)}
ϕ(zμ(t))>ω−1(ϕ(zν(t)))for t=Tmax⁡−(58)\phi(z_{\mu}(t)) > \omega^{-1}\bigl(\phi(z_{\nu}(t))\bigr) \quad\text{for } t=T_{\max}^{-} \tag*{(58)}

which would prove that the equality should hold between T‾\overline{T} and Tmax⁡T_{\max}.

Note that, again by Corollary 4, we have

ϕ(zμ(T‾))=ϕ(zν(T‾))<ω−1(ϕ(zν(T‾)))\phi(z_{\mu}(\overline{T})) = \phi(z_{\nu}(\overline{T})) < \omega^{-1}\bigl(\phi(z_{\nu}(\overline{T}))\bigr)

We want to prove now (58). To this aim let δ(t):=zμ(t)−zν(t)\delta(t):=z_{\mu}(t)-z_{\nu}(t) and observe that δ(t)\delta(t) satisfies the same differential equation satisfied by zμ(t)z_{\mu}(t) and hence we can argue that δ(t)\delta(t) is strictly increasing and δ(T‾)=0\delta(\overline{T})=0. This implies that δ(Tmax⁡−)>0\delta(T_{\max}^{-})>0 and hence

1ϕ(zμ(Tmax⁡−))=1ϕ(zν(Tmax⁡−)+δ(Tmax⁡−))<1ϕ(zν(Tmax⁡−))=1\frac{1}{\phi(z_{\mu}(T_{\max}^{-}))} = \frac{1}{\phi(z_{\nu}(T_{\max}^{-})+\delta(T_{\max}^{-}))} < \frac{1}{\phi(z_{\nu}(T_{\max}^{-}))} =1

On the other hand observe that ϕ(zν(Tmax⁡−))=1\phi(z_{\nu}(T_{\max}^{-}))=1 so that f−1(ϕ(zν(Tmax⁡−)))=0f^{-1}(\phi(z_{\nu}(T_{\max}^{-})))=0 and hence

ω−1(ϕ(zν(Tmax⁡−)))=1\omega^{-1}\bigl(\phi(z_{\nu}(T_{\max}^{-}))\bigr)=1

where we used the assumption ψ′(0)=1\psi'(0)=1.

This yields the thesis.

We now prove Theorem 1.

Proof of Theorem 1. By Lemma 4 and the IDP Hopfield stability condition in Theorem 3, if the successor saliency catches up with the source saliency before Tmax⁡T_{\max}, then there exists a time in [T‾,Tmax⁡)[\overline{T},T_{\max}) at which the source branch loses stability. The first such time, TescT_{\mathrm{esc}}, satisfies the identity in (56), namely

ϕ(zμ(Tesc))=ω−1(ϕ(zν(Tesc))).\phi(z_{\mu}(T_{\mathrm{esc}})) = \omega^{-1}\bigl(\phi(z_{\nu}(T_{\mathrm{esc}}))\bigr).

Equivalently,

zμ(Tesc)=h(zν(Tesc)),h(z):=ϕ−1(ω−1(ϕ(z))).z_{\mu}(T_{\mathrm{esc}})=h\left(z_{\nu}(T_{\mathrm{esc}})\right), \qquad h(z):=\phi^{-1}\left(\omega^{-1}(\phi(z))\right).

Using zν(t)=Ze−tz_{\nu}(t)=Ze^{-t} and (55), and evaluating the previous identity at t=Tesct=T_{\mathrm{esc}} yields

κ∫0Tesce−(Tesc−τ)g(Ze−τ) dτ=h(Ze−Tesc).\kappa\int_{0}^{T_{\mathrm{esc}}}e^{-(T_{\mathrm{esc}}-\tau)}g\left(Ze^{-\tau}\right)\,\mathrm{d}\tau=h\left(Ze^{-T_{\mathrm{esc}}}\right).

Thus TescT_{\mathrm{esc}} is the smallest admissible solution of the previous equation, and the updated saliency is

Z+:=zμ(Tesc)=h(Ze−Tesc).Z^{+}:=z_{\mu}(T_{\mathrm{esc}})=h\left(Ze^{-T_{\mathrm{esc}}}\right).

Extended derivations for the illustrative example

We give the details for the scalar iteration associated with

ψ(x)=x1+∣x∣,ϕ(z)=∣z∣.\psi(x)=\frac{x}{1+\lvert x\rvert}, \qquad\phi(z)=\lvert z\rvert.

In this case it can be proved that f(x)=xψ(x)=x+1f(x)=\frac{x}{\psi(x)}=x+1 for all x≥0x\geq0 so that, for all x≥1x\geq1 we have that f−1(x)=x−1f^{-1}(x)=x-1. Moreover for x≥0x\geq0 we have ψ′(x)=1(x+1)2\psi'(x)=\frac{1}{(x+1)^2}. Thus, for z≥1z\geq1,

g(z)=1−z−1h(z)=z2.g(z)=1-z^{-1} \qquad h(z)=z^2.

Consequently, for Z>1Z>1 and X∈[1,Z)X\in[1,Z),

∫1Xg(Zs) ds=∫1X(1−sZ) ds=(X−1)(1−X+12Z).\int_{1}^{X}g\left(\frac{Z}{s}\right)\,\mathrm{d}s=\int_{1}^{X}\left(1-\frac{s}{Z}\right)\,\mathrm{d}s=(X-1)\left(1-\frac{X+1}{2Z}\right).

Substituting this expression into R(Z,X)=0R(Z,X)=0 yields

κX(X−1)(1−X+12Z)=Z2X2,\frac{\kappa}{X}(X-1)\left(1-\frac{X+1}{2Z}\right)=\frac{Z^2}{X^2},

so that the saliency update map in (12) is determined by

\begin{cases} G(Z,X)=\frac{1}{\kappa},\\ Z^{+}=\frac{Z^2}{X^2}, \end{cases} \qquad X\in[1,Z), \tag*{(59)}\end{cases}

where

G(Z,X):=−X(X−1)(X+1−2Z)2Z3.(60)G(Z,X):=-\frac{X(X-1)(X+1-2Z)}{2Z^3}. \tag*{(60)}

For a given ZZ, the admissible value of XX is the smallest solution of G(Z,X)=1/κG(Z,X)=1/\kappa in [1,Z)[1,Z), if such a solution exists.

Existence of the iteration. We present a condition on ZZ ensuring that the iteration in (59) exists, namely that there exists a solution X(Z)∈[1,Z)X(Z)\in[1,Z) of the equation G(Z,X)=1/κG(Z,X)=1/\kappa (see also Figure 7). Observe that G(Z,1)=0G(Z,1)=0 and that G(Z,Z)=12(Z−1Z)2G(Z,Z)=\frac{1}{2}\left(\frac{Z-1}{Z}\right)^2. This implies that, by taking ZZ such that

Plot of the function $G(Z,X)$

Figure 7. Plot of the function G(Z,X)G(Z,X) in (60) for κ=15\kappa= 15 and Z=4Z = 4.

1κ<12(Z−1Z)2⟺Z>11−2/κ(61)\frac{1}{\kappa}<\frac{1}{2}\left(\frac{Z-1}{Z}\right)^2 \qquad\Longleftrightarrow\qquad Z>\frac{1}{1-\sqrt{2/\kappa}} \tag*{(61)}

a solution X(Z)∈[1,Z)X(Z)\in[1,Z) must exist. Such solution is unique. Indeed, since

∂G∂X=−3X2−4ZX−1+2Z2Z3(62)\frac{\partial G}{\partial X}=-\frac{3X^2-4ZX-1+2Z}{2Z^3} \tag*{(62)}

the sign of the this derivative coincides with the sign of the function qZ(X):=−(3X2−4ZX−1+2Z)q_Z(X):=-(3X^2-4ZX-1+2Z) that is a concave parabola. Since qZ(1)=2(Z−1)q_Z(1)=2(Z-1) and qZ(Z)=(Z−1)2q_Z(Z)=(Z-1)^2 that are both positive, we have that qZ(X)q_Z(X) is positive in the interval [1,Z)[1,Z). Hence G(Z,X)G(Z,X) is strictly increasing in that interval, so whenever (61) holds there is a unique solution X(Z)∈[1,Z)X(Z)\in[1,Z) and Z+=Z2/X(Z)2Z^{+}=Z^2/X(Z)^2 is uniquely defined.

Fixed points of the iteration. Fixed points satisfy Z+=ZZ^{+}=Z. Since Z+=Z2/X2Z^{+}=Z^{2}/X^{2}, this gives Z=X2Z=X^{2}. Substituting into G(Z,X)=1/κG(Z,X)=1/\kappa yields (see also Figure 8)

Figure 8. Plot of the function G(X2,X)G(X^{2},X) in (63) and its intersections with the line 1/κ1/\kappa.

1κ=G(X2,X)=(X−1)2(2X+1)2X5(63)\frac{1}{\kappa}=G(X^{2},X)=\frac{(X-1)^{2}(2X+1)}{2X^{5}} \tag*{(63)}

Observe that

dG(X2,X)dX=−(X−1)2X6(4X2−5X−5)(64)\frac{dG(X^{2},X)}{dX}=-\frac{(X-1)}{2X^{6}}(4X^{2}-5X-5) \tag*{(64)}

If we let Xˉ=5+1058≃1.9059\bar{X}=\frac{5+\sqrt{105}}{8}\simeq1.9059, then we have that G(X2,X)G(X^{2},X) is strictly increasing for 1≤X<Xˉ1\leq X<\bar{X} the function, and strictly decreasing for X>XˉX>\bar{X}, so that G(X2,X)G(X^{2},X) reaches its maximum in Xˉ\bar{X}. Since G(Xˉ2,Xˉ)≃0.0785G(\bar{X}^{2},\bar{X})\simeq0.0785, we can argue that we have fixed points only when

κ≥κcrit:=1G(Xˉ2,Xˉ)≃12.74\kappa\geq\kappa_{\mathrm{crit}}:=\frac{1}{G(\bar{X}^{2},\bar{X})}\simeq12.74

For κ>κcrit\kappa>\kappa_{\mathrm{crit}}, there are exactly two solutions, Xˉ1\bar{X}_{1} and Xˉ2\bar{X}_{2}, of (63) which satisfy 1<Xˉ1<Xˉ<Xˉ21<\bar{X}_{1}<\bar{X}<\bar{X}_{2}. Such solutions in turn yield two fixed points, Zˉ1=Xˉ12\bar{Z}_{1}=\bar{X}_{1}^{2} and Zˉ2=Xˉ22\bar{Z}_{2}=\bar{X}_{2}^{2} which satisfy 1<Zˉ1<Zˉ<Zˉ21<\bar{Z}_{1}<\bar{Z}<\bar{Z}_{2} with Zˉ=Xˉ2=3.6323\bar{Z}=\bar{X}^{2}=3.6323.

Monotonicity of the iteration. We will now show that the iteration Z+=Z2X(Z)2Z^{+}=\frac{Z^{2}}{X(Z)^{2}} is strictly increasing in ZZ, which proves that the sequence of iterates, denoted by Z0,Z1,Z2,…Z_{0},Z_{1},Z_{2},\ldots, is monotonic and hence it converges to a limit point Z∞Z_{\infty}. First notice that

∂Z+∂Z=2ZX2−2Z2X3∂X∂Z=2ZX2+2Z2X3∂G/∂Z∂G/∂X=2ZX31∂G/∂X(X∂G∂X+Z∂G∂Z)\frac{\partial Z^{+}}{\partial Z}=\frac{2Z}{X^{2}}-\frac{2Z^{2}}{X^{3}}\frac{\partial X}{\partial Z}=\frac{2Z}{X^{2}}+\frac{2Z^{2}}{X^{3}}\frac{\partial G/\partial Z}{\partial G/\partial X}=\frac{2Z}{X^{3}}\frac{1}{\partial G/\partial X}\left(X\frac{\partial G}{\partial X}+Z\frac{\partial G}{\partial Z}\right)

Using

∂G∂Z=X(X−1)(3X−4Z+3)2Z4\frac{\partial G}{\partial Z}=\frac{X(X-1)(3X-4Z+3)}{2Z^4}

we obtain

X∂G∂X+Z∂G∂Z=X(Z−1)Z3,X\frac{\partial G}{\partial X}+Z\frac{\partial G}{\partial Z}=\frac{X(Z-1)}{Z^3},

and hence

∂Z+∂Z=2(Z−1)X2Z21∂G/∂X.\frac{\partial Z^+}{\partial Z}=\frac{2(Z-1)}{X^2Z^2}\frac{1}{\partial G/\partial X}.

That shows that the sign of ∂Z+/∂Z\partial Z^+/\partial Z only depends on the sign of ∂G/∂X\partial G/\partial X. But, if the existence condition (61) holds, the derivative ∂G/∂X\partial G/\partial X in (62) evaluated at X(Z)∈[1,Z)X(Z)\in[1,Z) must be positive. Thus, we conclude that iteration Z+=Z2X(Z)2Z^+=\frac{Z^2}{X(Z)^2} is strictly increasing if Z>1/(1−2/κ)Z>1/(1-\sqrt{2/\kappa}).

Convergence of the iteration. We show finally that the fixed point Zˉ1\bar{Z}_1 is repelling (or unstable) and that Zˉ2\bar{Z}_2 is attractive (or stable). To this aim observe that, at the fixed points, where Z=X2Z=X^2, we have

∂Z+∂Z=4(X+1)4X2−X−1⟹∂Z+∂Z−1=−4X2−5X−54X2−X−1.\frac{\partial Z^+}{\partial Z}=\frac{4(X+1)}{4X^2-X-1} \quad\Longrightarrow\quad \frac{\partial Z^+}{\partial Z}-1=-\frac{4X^2-5X-5}{4X^2-X-1}.

Then ∂Z+∂Z>0\frac{\partial Z^+}{\partial Z}>0. Moreover, comparing the expression of ∂Z+∂Z−1\frac{\partial Z^+}{\partial Z}-1 with (64), we see that, since 4X2−X−1>04X^2-X-1>0, then ∂Z+∂Z−1\frac{\partial Z^+}{\partial Z}-1 has the same sign of dG(X2,X)dX\frac{dG(X^2,X)}{dX} showing in this way that ∂Z+∂Z>1\frac{\partial Z^+}{\partial Z}>1 in X=Xˉ1X=\bar{X}_1 and 0<∂Z+∂Z<10<\frac{\partial Z^+}{\partial Z}<1 in X=Xˉ2X=\bar{X}_2. This proves that the fixed point Zˉ1\bar{Z}_1 is repelling while the fixed point Zˉ2\bar{Z}_2 is attractive.

The previous analysis, together with the monotonicity of the iteration, shows that amplitudes above the unstable threshold Zˉ1\bar{Z}_1 converge to the attracting fixed point Zˉ2\bar{Z}_2, which corresponds to self-sustained sequential retrieval. In contrast, amplitudes below Zˉ1\bar{Z}_1 decay until subsequent transitions can no longer be triggered, ultimately leading the dynamics to collapse into the inactive state. This is illustrated in Figure 4 in the main text.

Fixed points, activation functions, and sequential retrieval dynamics

Figure 4. (A) Fixed points of the discrete map Z+=h(Z/X)Z^{+}=h(Z/X) with κ=15\kappa=15 for the SmoothF activation, with predicted Zˉ2≃6.78\bar{Z}_2 \simeq6.78 (see Figure 9 in Appendix § 13 for ψ=tanh⁡\psi=\tanh). Real solutions Zˉ1,Zˉ2>1\bar{Z}_1,\bar{Z}_2>1 emerge only for κ≥κcrit≃12.74\kappa\geq\kappa_{\rm crit} \simeq12.74; self-sustained transitions require Z0>Zˉ1Z_0>\bar{Z}_1. (B) SmoothF (blue) and Tanh (green) activation functions. (C, D) Orthogonal memories with SmoothF activation. (C) Subcritical gain (κ=9\kappa=9): oscillations decay and retrieval collapses to the origin. (D) Supercritical gain (κ=15\kappa=15): persistent oscillations converge to the stable fixed point Zˉ2≃6.95\bar{Z}_2 \simeq6.95, close to theoretical prediction, enabling reliable sequential retrieval. (E, F) Random memories with SmoothF activation. (E) Subcritical gain: cross-talk noise sustains small-amplitude transitions. (F) Supercritical gain: transitions persist but exhibit amplitude fluctuations without entrainment to Zˉ2\bar{Z}_2. (G,H) Random memories with Tanh activation. (G) Subcritical gain (κ=3\kappa=3) leads to collapse, with slower decay due to cross-talk. (H) Supercritical gain (κ=5\kappa=5) yields self-sustained transitions. Panels (C-H) use N=256N=256, P=15P=15, and a directed cycle of length L=5L=5.

Additional results for Tanh activation

We complement the analysis of § 5 by considering the Tanh activation ψ(x)=tanh⁡(x)\psi(x)=\tanh(x), while retaining ϕ(z)=∣z∣\phi(z)=|z|. Unlike SmoothF, the corresponding saliency map and escape-time predictions do not admit simple closed-form expressions, but can be computed numerically from the implicit equations derived in § 5. Figure 9 illustrates the resulting discrete map and representative saliency dynamics for κ=15\kappa=15.

Figure 9. Discrete iteration map Z+=h(Z/X)Z^+=h(Z/X) for the ψ=tanh⁡\psi=\tanh activation function and associated zz-components for the case of κ=15\kappa=15.

We also repeat the numerical experiments of Figure 5 using Tanh activation (Figure 10). For orthogonal memories, theoretical and numerical escape times remain in good agreement (A, B), while increasing the noise amplitude reduces the switching delay and eventually leads to premature switching (C). For non-orthogonal memories, retrieval reliability decreases with memory bias and exhibits a non-monotonic dependence on the gain κ\kappa, with excessively large gains reducing the probability of successful transitions (D). Conditional on successful transitions, both the asymptotic escape time and saliency generally increase with κ\kappa (E, F). These results support the applicability of the theoretical framework to different activation functions, while highlighting activation-dependent differences in retrieval robustness.

Figure 10. Numerical validation and robustness analysis for Tanh activation. (A) Theoretical and numerical escape times and observed switching times across consecutive transitions. (B) Asymptotic escape time versus κ\kappa: theory and numerical estimates. (C) Mean switching delay versus Gaussian noise amplitude. (D) Probability of correct transitions in the (κ,ϵ)(\kappa,\epsilon) plane. (E, F) Asymptotic escape time and saliency amplitude, respectively. All experiments use the same numerical configuration as Figure 5. Hatched regions in (D-F) indicate no successful transitions.

Numerical investigation of sequential retrieval capacity

An important open question is how sequential retrieval reliability scales with the number of stored memories and the length of the retrieved sequence. In classical Hopfield networks, the synaptic matrix is fixed, whereas in the proposed slow-fast model it is dynamically modulated by the saliency variables:

WHopfield=1N∑μ=1Pξμ(ξμ)T,W(∣z∣)=1N∑μ=1P∣zμ∣ξμ(ξμ)T.W_{\mathrm{Hopfield}} = \frac{1}{N}\sum_{\mu=1}^{P}\xi^\mu(\xi^\mu)^{\mathsf{T}}, \qquad W(|z|) = \frac{1}{N}\sum_{\mu=1}^{P}|z_\mu|\xi^\mu(\xi^\mu)^{\mathsf{T}}.

In the classical setting, all stored memories contribute simultaneously to cross-talk. In contrast, during sequential retrieval, only a small subset of saliency coefficients typically remains significant, while the others decay toward zero. Thus, the nominal memory load P/NP/N need not reflect the number of memories effectively contributing to the synaptic dynamics at a given time.

To investigate this effect, we consider two numerical protocols and measure the probability of successfully completing an entire prescribed retrieval cycle, following the sequential retrieval criterion of [19 Eq. (4]. Successful retrieval requires the prescribed sequence of memory transitions, rather than merely escaping from the current memory state. In the growing-cycle protocol, a directed cycle of length L=round⁡(P/4)L = \operatorname{round}(P/4) is embedded among PP stored random i.i.d. memories with P(ξiμ=±1)=1/2\mathbb{P}(\xi_i^\mu= \pm1) = 1/2, with N∈{256,512}N \in\{256, 512\}. In the fixed-cycle protocol, the cycle length is kept at L=5L = 5 while the memory load P/NP/N is increased substantially, with N=256N = 256. Both protocols are evaluated with SmoothF and Tanh activations (κ=15(\kappa= 15 and z0=8)z_0 = 8), using 10 independent memory realizations per configuration.

Figure 11 reports the resulting retrieval probabilities. In the growing-cycle setting (panels A,B), SmoothF achieves reliable retrieval P/N∈(17.5,25)P/N \in(17.5, 25) and N=512N = 512, while retrieval probability constantly decreases for N=256N = 256 over the studied memory load range. Tanh exhibits greater variability, particularly for smaller network sizes. In contrast, with a fixed cycle (panels C,D), retrieval remains highly reliable even for P/N≫1P/N \gg1, with occasional failures for Tanh. This difference reflects, in part, the increasing number of transitions required when LL grows with PP, whereas the additional memories in the fixed-cycle setting remain largely inactive under the saliency dynamics.

Figure 11. Full-sequence retrieval probability versus memory load P/NP/N. (A, B) Growing cycles with L≃P/4L \simeq P/4 and N∈{256,512}N \in\{256, 512\} for SmoothF and for Tanh, respectively. (C, D) Fixed cycles with L=5L = 5 and N=256N = 256, for SmoothF and Tanh, respectively. Shaded regions indicate 95% confidence intervals across independent memory realizations.

These results highlight the distinction between nominal memory load and the number of memories actively involved in sequential retrieval. In particular, reliable retrieval of a fixed cycle at high P/NP/N does not imply that all stored memories can be reliably retrieved. Rather, it suggests that dynamic saliency modulation can mitigate interference from memories outside the active sequence. Moreover, since both memory load and sequence length increase in the growing-cycle protocol, their individual effects cannot be isolated from these experiments. A precise theoretical characterization of sequential retrieval capacity therefore remains an open problem.

Branching memory transitions

Our theoretical analysis focuses on transition graphs with at most one outgoing edge per memory, which enables a scalar reduction of the slow saliency dynamics and explicit conditions for sequential retrieval. However, this restriction is not inherent to the proposed model, which also accommodates branching graphs with multiple possible successors. In this setting, several successor saliencies are driven simultaneously, and branch selection depends on their competing dynamics.

To investigate this behavior, we consider two directed three-memory cycles sharing a common source memory, with equal outgoing transition weights. We use the SmoothF activation, set N=256N = 256, P=5P = 5, κ=15\kappa= 15, and generate 50 independent sets of random i.i.d. ±1\pm1 memories. In the deterministic setting, the selected successor is the one with the larger absolute overlap with the source memory in all 45 realizations with unequal overlaps. We then introduce noise of amplitude 10−110^{-1} per integration step in the slow dynamics and perform 20 trials per memory realization. The successor with the larger absolute overlap remains the more frequently selected branch in 36 out of 43 realizations with unequal overlaps.

These results indicate that branch selection can be influenced by the geometry of the stored memories, even when outgoing transition weights are identical. In particular, differences in memory overlaps can break the symmetry between competing successors and favor one transition over another. This preference persists in most noisy trials, although stochastic perturbations can alter the selected branch. Thus, branching introduces a competition between successor memories that depends not only on the transition graph, but also on memory correlations and perturbations. A general characterization of this competition would require extending the scalar slow reduction to account for the simultaneous evolution of multiple successor saliencies.

A motivating example for input-controlled transitions: Associative Memory Finite State Machine (AM-FSM)

Finite state machines (FSMs) are canonical models of sequential computation, in which the evolution of the system is jointly determined by current inputs and a compact internal representation of past states. At any time, an FSM is in one state drawn from a finite set, and incoming inputs trigger transitions according to a predefined state-dependent transition structure. In this sense, FSMs implement a form of implicit memory: past information is compressed into discrete internal states that condition future behavior. The state-dependent interpretation of inputs makes FSMs natural abstractions for a wide range of engineered systems, including elevators, network protocols, and traffic lights with pedestrian signals. In all such cases, effective control requires integrating current observations with the system’s internal configuration, thereby generating context-appropriate actions over time. More broadly, decision-making processes exhibit a similar structure: actions depend not only on present stimuli, but also on the internal state shaped by prior events. This observation motivates viewing sequential decision-making as the evolution of an FSM.

As an illustrative example, consider a train driver operating a metro line in the Tokyo subway system. Under normal conditions, the driver repeatedly traverses a fixed route (the "red-black" line), corresponding to a stable sequence of internal states. Due to an unexpected strike, however, the driver receives instructions to deviate from the habitual route and instead pass through Shinjuku station (on the "green-black" line) and subsequently Tokyo station (on the "blue-black" line). Only two transfer hubs permit switching from the red-black line to the other lines. At each hub, the driver must choose between alternative turns, using prior knowledge of the network and the current location to guide the decision. Although the map is outdated and uncertainty remains, the driver assigns high confidence to a left turn at each hub, while maintaining a lower probability for the alternative right turn.

This scenario exemplifies sequential decision-making under uncertainty, where each choice depends on both external cues and an internally maintained representation of the route taken so far. In the figure below, we instantiate the driver’s behavior using a two-timescale IDP Hopfield model, illustrating how stable internal states and slower adaptation processes jointly support FSM-like dynamics. In particular, we instantiate an input-driven temporal network over the transition graph, where each outgoing edge from a memory ξv\xi^v towards a set of memories {ξσ}σ\{\xi^\sigma\}_\sigma is suitably weighted by an external input uu. The magnitude of the input components along each transition edge will ultimately determine the sequential memory path that is traversed by the two-timescale IDP Hopfield model. The exemplified control over sequential memory transitions provides solid insp

Experimental setup

The code of the experimental campaign is available at https://github.com/simlbet/Memory-Flows-NeurIPS2026.

We integrate the fast neural state and slow saliency dynamics by explicit Euler steps on the dimensionless fast clock. For τx/τz=0.01\tau_x/\tau_z = 0.01, an integration step Δs\Delta s corresponds to slow time Δθ=0.01Δs\Delta\theta= 0.01\Delta s; the numerical step and horizon are specified separately for each experiment in configs/paper/ and fast noise with amplitude σ=10−6\sigma= 10^{-6}.

The experiments are implemented in the accompanying repository through separate configuration-driven simulation and plotting scripts (experiments/, plotting/); the master command run_all.sh reproduces the configured workflows. For all experiments, successful retrieval requires a persistent target alignment of at least 0.90, separated from the runner-up by at least 0.10, over a 0.05 slow-time hold window in the capacity tests. Uncertainty is computed over independent pattern sets, not over multiple source trajectories as if they were independent data sets.

All numerical campaigns for the camera-ready figures were executed on the Leonardo high-performance computing system, on the AI Factory project EHPC-AIF-2026PG02-1086. The repository includes resolved experimental configurations, seed policies, CSV outputs and figure-generation scripts; numerical parallelism and worker counts are configurable through the command-line interface. Figure generation reads saved experimental data and does not rerun the underlying dynamics.

Figure 12. Finite state machine dynamics with uncertain inputs realized by a two-timescale IDP Hopfield model. (A) Schematic of a simplified Tokyo subway graph comprising three metro lines connected by two hub stations. The metro driver (red marker) must adapt the route in response to an external rerouting request, while facing uncertainty at each hub regarding the correct branch to take. (B) Sequential retrieval of stable memory states corresponding to stations along the habitual red-black line, illustrating deterministic FSM dynamics under standard operating conditions. (C) Input- and state-dependent sequential retrieval during rerouting: external inputs bias the transition dynamics at each hub toward the green-black line (via Shinjuku station) and subsequently the blue-black line (via Tokyo station), while preserving residual uncertainty across alternative transitions.

References

  1. [1]J. J. Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79(8):2554–2558, April 1982. ISSN 1091-6490. doi:10.1073/pnas.79.8.2554.DOI
  2. [2]M. A. Cohen and S. Grossberg. Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Transactions on Systems, Man, and Cybernetics, SMC-13(5):815–826, September 1983. ISSN 2168-2909. doi:10.1109/tsmc.1983.6313075.DOI
  3. [3]J. J. Hopfield. Neurons with graded response have collective computational properties like those of two-state neurons. Proceedings of the National Academy of Sciences, 81(10):3088–3092, May 1984. ISSN 1091-6490. doi:10.1073/pnas.81.10.3088.DOI
  4. [4]D. J. Amit, H. Gutfreund, and H. Sompolinsky. Information storage in neural networks with low levels of activity. Physical Review A, 35(5):2293–2303, March 1987. ISSN 0556-2791. doi:10.1103/physreva.35.2293.DOI
  5. [5]R. McEliece, E. Posner, E. Rodemich, and S. Venkatesh. The capacity of the hopfield associative memory. IEEE Transactions on Information Theory, 33(4):461–482, July 1987. ISSN 1557-9654. doi:10.1109/tit.1987.1057328.DOI
  6. [6]M. V. Tsodyks and M. V. Feigel’man. The enhanced storage capacity in neural networks with low activity level. Europhysics Letters (EPL), 6(2):101–105, May 1988. ISSN 1286-4854. doi:10.1209/0295-5075/6/2/002.DOI
  7. [7]D. Krotov and J. J. Hopfield. Dense associative memory for pattern recognition. In Advances in neural information processing systems, volume 29, 2016. doi:doi.org/10.48550/arXiv.1606.01164.arxiv.org/abs/1606.01164
  8. [8]D. Krotov and J. J. Hopfield. Large associative memory problem in neurobiology and machine learning. In International Conference on Learning Representations, 2020. doi:doi.org/10.48550/arXiv.2008.06996.arxiv.org/abs/2008.06996
  9. [9]M. Demircigil, J. Heusel, M. Lowe, S. Upgang, and F. Vermet. On a model of associative memory with huge storage capacity. Journal of Statistical Physics, 168(2):288–299, May 2017. ISSN 1572-9613. doi:10.1007/s10955-017-1806-y.DOI
  10. [10]H. Ramsauer, B. Schäf, J. Lehner, P. Seidl, M. Widrich, L. Gruber, M. Holzleitner, T. Adler, D. Kreil, M. K. Kopp, G. Klambauer, J. Brandstetter, and S. Hochreiter. Hopfield networks is all you need. In International Conference on Learning Representations, 2021.arxiv.org/abs/2008.02217
  11. [11]B. Hoover, Y. Liang, B. Pham, R. Panda, H. Strobel, D. H. Chau, M. Zaki, and D. Krotov. Energy transformer. Advances in neural information processing systems, 36:27532–27559, 2023a.arxiv.org/abs/2302.07253
  12. [12]B. Hoover, H. Strobel, D. Krotov, J. Hoffman, Z. Kira, and D. H. Chau. Memory in plain sight: A survey of the uncanny resemblances between diffusion models and associative memories. In Associative Memory & Hopfield Networks in 2023, 2023b.arxiv.org/abs/2309.16750
  13. [13]L. Ambrogioni. In search of dispersed memories: Generative diffusion models are associative memory networks. Entropy, 26(5):381, April 2024. ISSN 1099-4300. doi:10.3390/e26050381.DOI
  14. [14]D. Kleinfeld. Sequential state generation by model neural networks. Proceedings of the National Academy of Sciences, 83(24):9469–9473, December 1986. ISSN 1091-6490. doi:10.1073/pnas.83.24.9469.DOI
  15. [15]H. Sompolinsky and I. Kanter. Temporal association in asymmetric neural networks. Physical Review Letters, 57(22):2861–2864, December 1986. ISSN 0031-9007. doi:10.1103/physrevlett.57.2861.DOI
  16. [16]D. W. Tank and J. J. Hopfield. Neural computation by concentrating information in time. Proceedings of the National Academy of Sciences, 84(7):1896–1900, April 1987. ISSN 1091-6490. doi:10.1073/pnas.84.7.1896.DOI
  17. [17]D. Kleinfeld and H. Sompolinsky. Associative neural network model for the generation of temporal patterns. theory and application to central pattern generators. Biophysical Journal, 54(6):1039–1051, December 1988. ISSN 0006-3495. doi:10.1016/S0006-3495(88)83041-8.
  18. [19]H. T. Chaudhry, J. A. Zavatone-Veth, D. Krotov, and C. Pehle­van. Long sequence hopfield memory. Journal of Statistical Mechanics: Theory and Experiment, 2024(10):104024, October 2024. ISSN 1742-5468. doi:10.1088/1742-5468/ad6427.DOI
  19. [20]A. Karuvally, P. Lertsaroj, S. Terrence, and H. T. Siegelmann. Exponential dynamic energy network for high capacity sequence memory. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. doi:10.48550/arXiv.2510.24965.DOI
  20. [21]L. Herron, P. Sartori, and B. Xue. Robust retrieval of dynamic sequences through interaction modulation. PRX Life, 1(2), December 2023. ISSN 2835-8279. doi:10.1103/prxlife.1.023012.DOI
  21. [22]M. Farrell and C. Pelhevan. Recall tempo of hebbian sequences depends on the interplay of hebbian kernel with tutor signal timing. Proceedings of the National Academy of Sciences, 121(32), 2024. ISSN 1091-6490. doi:10.1073/pnas.2309876121.DOI
  22. [23]S. Betteti, G. Baggio, F. Bullo, and S. Zampieri. Input-driven dynamics for robust memory retrieval in hopfield networks. Science Advances, 11(17), April 2025. ISSN 2375-2548. doi:10.1126/sciadv.adu6991.DOI
  23. [24]M. K. Benna and S. Fusi. Computational principles of synaptic memory consolidation. Nature Neuroscience, 19(12):1697–1706, October 2016. ISSN 1546-1726. doi:10.1038/nn.4401.
  24. [25]M. Gillett and N. Brunel. Dynamic control of sequential retrieval speed in networks with heterogeneous learning rules. eLife, 12, August 2024. ISSN 2050-084X. doi:10.7554/eLife.88805.3.DOI
  25. [26]N. Fenichel. Geometric singular perturbation theory for ordinary differential equations. Journal of Differential Equations, 31(1):53–98, January 1979. ISSN 0022-0396. doi:10.1016/0022-0396(79)90152-9.DOI
  26. [27]G. Hek. Geometric singular perturbation theory in biological practice. Journal of Mathematical Biology, 60(3):347–386, April 2009. ISSN 1432-1416. doi:10.1007/s00285-009-0266-7.DOI
  27. [28]R. A. Horn and C. R. Johnson. Matrix Analysis. Cambridge University Press, December 1985. ISBN 9780511810817.

Paper details

vlnc:W_agqy6fj7
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents