betaFeedback

WorksAgricultural and Biological Sciences

Dynamical mean-field theory of latching Potts networks

Abstract

Spontaneous transitions between memories, as observed in internally generated cognition such as mind-wandering and free association, require neural dynamics capable of destabilizing one attractive memory state and moving towards another. We ask what minimal representational structure is required for such transitions to emerge in random associative networks. To address this question, we study Potts associative networks with SS active states per unit and slow adaptive thresholds. The S=1S=1 case can model a standard Hopfield-like network of binary units. We employ dynamical mean-field theory in the extensive-load regime, where the number of stored patterns scales linearly with network size and connectivity, and we focus on transitions between two condensed memories, which are singled out in the connectivity matrix. In the finite-load regime, we show that transitions are governed by an effective drive term, whose sign and temporal evolution distinguish high-latching (structured), low-latching (random), and non-latching regimes. The resulting phase diagram shows that adaptation alone is not sufficient: transitions only occur within certain ranges of the overlap between the two memories. In the binary case S=1S=1, in particular, no unit can be active in both memories but in different Potts states, thus strongly supressing the high-latching, structured transitions that exist for S>1S>1. In the extensive-load regime, non-condensed memories generate colored fluctuations and retarded self-interactions through an effective memory kernel. We find that latching trajectories change the effective timescale of this kernel, a dynamical signature of transitions in the extensive-load regime. These results indicate that multi-state Potts-like representations are essential for structured memory-to-memory transitions, a hallmark of spontaneous thought.

Introduction

Associative neural network models link computation to what is available in cortical circuits, such as plain dendritic summation (unlike e.g. complex logic gates), distributed representations (unlike symbolic units) and associative plasticity. Unlike supervised learning, neuronal co-activation drives local synaptic strengthening in recurrent neural networks, which can support attractor dynamics and yield retrieval of learned patterns from partial cues. This is exemplified by Hopfield networks with two-state units [1], where model Hebbian learning results in symmetric couplings, defining an energy function whose minima correspond to stored memories. Extensions to graded-response units, already in [2], have broadened the formalism toward more realistic continuous firing rate models. The statistical-mechanics analysis of Hopfield networks with extensive loading [3] has characterized the retrieval, spin-glass, and paramagnetic phases, while subsequent work has shown that sparse coding can substantially increase storage capacity and reduce interference [4].

The mammalian neocortex exhibits a strongly modular organization, with dense local recurrent connectivity within cortical patches and comparatively sparse long-range projections between them [5]. This has led to consider associative network models comprised not of single neurons but of coarse-grained multi-state variables representing the collective attractors of local cortical assemblies. Potts units provide a natural implementation of this mesoscopic description: rather than representing the activity of individual neurons, each unit can be in one among SS local attractors, together with an inactive state corresponding to the absence of local activation. A mathematical analysis of such a multi-state associative model was first proposed by Kanter [6], and demonstrated an almost quadratic increase in storage capacity with the number of locally available states. His treatment can be considered as an extension of that worked out for the Hopfield model which, since Kanter did not consider an inactive state, corresponds to setting S=2S = 2 (while e.g. the model in [4] corresponds to a Potts network with an inactive state and S=1S = 1). Subsequent work has interpreted Potts units as effective variables for cortical patches, where strong intra-module recurrence supports multiple local attractors while sparse inter-module connectivity binds them into coherent global memories [7, 8].

Building on these associative memory models, it was proposed that relatively slower adaptation processes can alter network dynamics and produce, in addition to memory retrieval, spontaneous transitions between retrieved memories [7]. In particular, activity-dependent fatigue within local cortical patches can progressively destabilize currently retrieved attractors, enabling transitions toward other memory states, in a regime referred to as latching. Such dynamics provide a potential mechanistic substrate for internally generated cognitive sequences, including free association and mind wandering, in which the system autonomously explores memory space without external input [7, 9, 10].

Despite its conceptual appeal, the dynamical origin of latching remains only partially understood. In particular, it is unclear what minimal ingredients are required for adaptation to internally generate structured transitions between memories, rather than simple decay or random wandering. A key unresolved question is whether the emergence of such transitions depends fundamentally on the Potts units representing cortical patches, or they can occur also in standard models in which units represent individual neurons (which can be included in the same formalization, just setting S=1S=1, see also the seminal study in [11]). In other words, can Hopfield-like networks engage in spontaneous yet structured latching (not reflecting externally imposed episodic transition as in [12])? While previous work has addressed this question numerically and through signal-to-noise analyses [10, 13], suggesting a negative answer, it has remained unclear whether one can draw a general and definite conclusion.

To address it, a theoretical framework is required that can separate the contributions of condensed memories (those engaging in a transition), quenched disorder, and adaptive temporal feedback in the thermodynamic limit. Dynamical mean-field theory (DMFT) provides precisely such a framework [14, 15, 16, 17]. By averaging over the non-condensed patterns, DMFT reduces the high-dimensional network dynamics to an effective stochastic single-site process with self-consistently determined memory kernels and colored noise. This allows one to derive exact dynamical equations for overlaps, correlations, and response functions in the extensive-load regime [15], going beyond finite-size simulations and heuristic arguments.

In this work, we use DMFT to study adaptive Potts associative networks with extensive loading and address two central questions. First, what minimal representational complexity is required for adaptation to generate directed memory transitions? In particular, we compare the dynamics of binary (S=1S=1) and multi-state (S>1S>1) networks and determine how the internal structure of the units allows for the emergence of latching dynamics. Second, what relations between memories enable transitions between them, and how are these conditions modified by the interference generated by non-condensed patterns?

We show that, under a two-condensed-memories ansatz, the dynamics reduces to only five effective classes of units whose stochastic evolution can be solved self-consistently within DMFT. Analyzing these effective dynamics reveals that non-random transitions are disabled in the binary case, S=1S=1, while they occur in genuine (S>1S>1) Potts networks. Our results therefore indicate that multi-state cortical representations are necessary for internally generated cognitive dynamics.

The remainder of the paper is organized as follows. In Section II, we introduce the model: an adaptive Potts associative network. We define the stored random memories, the continuous-time dynamics, and the phenomenology of high-latching and low-latching transitions. In Section III, we develop the DMFT methods, separating a finite number of condensed memories from the extensive background of non-condensed patterns, and derive the corresponding effective single-site dynamics. In Section IV, we organize results in three parts. First, we analyze the non-extensive-load limit and show how the geometry of overlap between two Potts memories determines whether high-, low-, or non-latching dynamics occurs. Second, we compare multi-state Potts networks with the binary case, showing why the transition channels required for structured latching disappear in the binary limit. Third, we study the extensive-load regime, where non-condensed memories generate colored fluctuations and retarded self-interactions, and we identify changes in the DMFT memory kernel as a dynamical signature of latching. Finally, in Sections V and VI, we discuss the implications of these results for cortical representations of semantic memories and internally generated memory sequences.

Model

We consider a network of NN units, where each unit models a local assembly of neurons and can be found in one of SS active states and one silent state denoted by 0. The network evolves in continuous time. For each unit (indexed as i=1,…,Ni=1,\ldots,N) and active state (indexed as k=1,…,Sk=1,\ldots,S), we introduce a fast variable rik(t)r_i^k(t) representing the local field, together with a slow adaptive threshold θik(t)\theta_i^k(t) modeling activity-dependent fatigue of the active fraction of the local assembly, symbolically represented by a color of a Potts unit. The dynamics are given by

τrr˙ik(t)=−rik(t)−θik(t)+hik(t),\tau_r\dot{r}_i^k(t)=-r_i^k(t)-\theta_i^k(t)+h_i^k(t),

{#eq:1}

τθθ˙ik(t)=−θik(t)+σik(t),\tau_\theta\dot{\theta}_i^k(t)=-\theta_i^k(t)+\sigma_i^k(t),

{#eq:2}

i=1,…,N,k=1,…,S.i=1,\ldots,N,\qquad k=1,\ldots,S.

with τr\tau_r and τθ\tau_\theta defining the fast and slow dynamical timescales, respectively. The input field is

hik(t)=∑j=1N∑l=1SJijklσjl(t)+w(σik−1S∑l=1Sσil),h_i^k(t)=\sum_{j=1}^{N}\sum_{l=1}^{S}J_{ij}^{kl}\sigma_j^l(t)+w\left(\sigma_i^k-\frac{1}{S}\sum_{l=1}^{S}\sigma_i^l\right),

{#eq:3}

where the parameter ww controls the local competition between active states within the same Potts unit, favoring single-state occupancy and stabilizing local categorical representations.

The instantaneous activity of each unit is obtained through a softmax nonlinearity

σik(t)=exp⁡(βrik(t))exp⁡(βU)+∑l=1Sexp⁡(βril(t)),\sigma_i^k(t)=\frac{\exp\left(\beta r_i^k(t)\right)}{\exp(\beta U)+\sum_{l=1}^{S}\exp\left(\beta r_i^l(t)\right)},

{#eq:4} where β\beta is analogous to an inverse temperature, and UU sets a global bias toward the inactive state. Larger values of UU increase the relative stability of the inactive state and therefore regulate the overall sparsity of the dynamics. The dynamical equation for θik(t)\theta_i^k(t) can be formally integrated,

θik(t)=θik(0)e−t/τθ+1τθ∫0tdt′e−(t−t′)/τθσik(t′),(1)\theta_i^k(t)=\theta_i^k(0)e^{-t/\tau_\theta}+\frac{1}{\tau_\theta}\int_0^t dt' e^{-(t-t')/\tau_\theta}\sigma_i^k(t'), \tag*{(1)}

showing that adaptation acts as a temporal filter over the past activity.

The memories stored in the network are represented by pp random Potts patterns ξμ\xi^\mu (μ=1,…,p\mu=1,\ldots,p). Each pattern assigns to every unit ii either the silent state 0 or one of the SS active states, so that ξiμ∈{0,1,…,S}\xi_i^\mu\in\{0,1,\ldots,S\}. The sparsity of the representation, or the fraction of active units in each pattern, is controlled by the coding level a∈(0,1)a\in(0,1). The pattern components are drawn with probability

Pr⁡(ξiμ=0)=1−a,\Pr(\xi_i^\mu=0)=1-a,
Pr⁡(ξiμ=k)=aS,k=1,…,S.(2)\Pr(\xi_i^\mu=k)=\frac{a}{S},\qquad k=1,\ldots,S. \tag*{(2)}

Thus, each stored pattern contains on average NaNa active units, whose active states are uniformly distributed among the SS Potts colors. We assume that at any given time the network has macroscopic overlap with a finite number BB of condensed patterns (μ=1,…,B\mu=1,\ldots,B). For a specific realization of stored patterns, we explicitly parametrize the correlations among the BB patterns for a given unit ii, whereas we treat the remaining p−Bp-B non-condensed patterns as statistically independent. To investigate how the similarity between the stored memories affects latching, we will focus on the case B=2B=2 and use as control parameters the number C1C_1 of units that are active in the same Potts state in both patterns, and the number C2C_2 of units that are active in different Potts states.

The total number of patterns scales as p=αNp=\alpha N, where α\alpha is the loading parameter (the natural scaling would be with the number of connections per unit, which itself may or may not be considered to scale with NN [18], but we leave the case of a non-fully connected network to future studies). The synaptic tensor JijklJ_{ij}^{kl} is defined by the Potts Hebbian learning rule

Jijkl=1−δijNa~∑μ=1pηikμηjlμ,(3)J_{ij}^{kl}=\frac{1-\delta_{ij}}{N\tilde{a}}\sum_{\mu=1}^{p}\eta_{ik}^{\mu}\eta_{jl}^{\mu}, \tag*{(3)}

where

ηikμ=δξiμ,k−aS,a~=a(1−aS)(4)\eta_{ik}^{\mu}=\delta_{\xi_i^\mu,k}-\frac{a}{S},\qquad\tilde{a}=a\left(1-\frac{a}{S}\right) \tag*{(4)}

with i,j=1,…,Ni,j=1,\ldots,N and k,l=1,…,Sk,l=1,\ldots,S. The factor 1−δij1-\delta_{ij} removes self-couplings, while the centering terms −a/S-a/S subtract the mean activity of each active state.

Phenomenology of latching

Under suitable conditions, the network exhibits latching dynamics, in which activity transiently remains near a retrieved memory before transitioning to another attractor. These transitions are driven by the slow adaptive field introduced above. During the retrieval of a given memory, the adaptive thresholds associated with its active local states gradually increase over the timescale τθ\tau_\theta, reducing the stability of the corresponding attractor. Once this attractor becomes sufficiently destabilized, the network relaxes toward a different memory configuration, where the same mechanism repeats. The resulting dynamics consists of a sequence of metastable memory states whose dwell times are controlled by the interplay between adaptation, recurrent reinforcement, and noise. As observed in previous work [9], depending on the overlap between stored memories, the network displays qualitatively different classes of transitions. These preliminary observations motivate the dynamical mean-field analysis developed in the following sections.

When only two patterns are stored in the connectivity matrix of a large network, numerical simulations reveal qualitatively distinct transition regimes:

  • High-overlap latching: transitions in which the overlap with the second memory increases while the first memory is still substantially active (see Figure 1(a)). These events typically occur between partially overlapping memories.

  • Low-overlap latching: transitions in which the first memory is inactivated before the second memory becomes active (see Figure 1(b)). These events are common for weakly correlated or uncorrelated memories and proceed through an intermediate low-activity network state.

  • Pathological latching: transitions in which the network alternates repeatedly between two highly correlated memories. This regime is termed pathological latching in [9].

Overlap dynamics during high- and low-latching transitions

Figure 1. Numerical simulation results for the overlap dynamics during high- and low-latching transitions in a network with N=1000N = 1000 units, S=3S = 3 Potts states, and p=2p = 2 stored patterns. The network is initialized close to the first pattern. The other model parameters are (see § 1.1): a=0.25a = 0.25, U=0.3U = 0.3, β=5\beta= 5, τr=10\tau_r = 10, τθ=100\tau_\theta= 100 and w=0.8w = 0.8. (a) High-overlap latching: X=0.8X = 0.8 and Y=0.4Y = 0.4. (b) Low-overlap latching: X=0.05X = 0.05 and Y=0.8Y = 0.8.

Examples of high- and low-overlap latching are shown in Figure 1, which illustrates the dynamics of the network overlaps with the first and second memories for different values of C1C_1 (units active in the same Potts state in the two memories) and C2C_2 (units active in different Potts states). To investigate how the overlap between the patterns influences latching dynamics, it is convenient to introduce [9]

X=C1C1+C2andY=C1+C2Na,(5)X=\frac{C_1}{C_1+C_2}\qquad\text{and}\qquad Y=\frac{C_1+C_2}{Na}, \tag*{(5)}

where XX is the fraction of units active in the same Potts state, while YY is the total fraction of units active in both memories. High-overlap latching transitions occur for larger values of XX, when the two patterns share many units active in the same Potts state.

This phenomenology remains essentially the same when introducing models of slow or fast inhibition [9, 10] or a combination of the two, as shown by extensive simulations

(Kwang Il Ryom, unpublished studies and [19]): even though the type of phase transitions between the different regimes may vary, the types of latching transitions are those described above.

Latching from one memory pattern to the next relies on the ability of each unit to represent multiple active states. This multi-state structure appears to be essential: previous numerical studies have shown that networks with binary units do not sustain robust memory transitions [7, 13]. Although adaptive thresholds destabilize the retrieved memory, they do not efficiently drive transitions toward alternative attractors in binary-state networks. These observations motivate the development of a theoretical framework capable of identifying the dynamical mechanisms underlying memory transitions.

Dynamical mean-field theory

To study the dynamics in the extensive-load regime, we employ dynamical mean-field theory (DMFT), which determines the generating functional of dynamical path-probabilities after averaging over the disorder associated with the non-condensed patterns. In the limit N→∞N \to\infty, this procedure reduces the original dynamics of NN coupled units to a set of effective single-site stochastic equations that depends self-consistently on the overlaps with the condensed patterns, as well as on correlations and response functions. The resulting equation also depends explicitly on the specific realization of the finite set of condensed patterns, which allows the dynamical equations to be further reduced to a finite number of effective site classes. In appendix A, we develop the DMFT for the present model and derive the effective stochastic equations.

The generating functional of the dynamics is defined as

Z[ψ]=⟨exp⁡(∫0Tdt∑i=1N∑k=1Sψik(t)σik(t))⟩,(6)\mathcal{Z}[\psi] = \left\langle\exp\left(\int_0^T dt \sum_{i=1}^{N}\sum_{k=1}^{S}\psi_i^k(t)\sigma_i^k(t)\right)\right\rangle, \tag*{(6)}

where ⟨⋯ ⟩\langle\cdots\rangle denotes the average with respect to the path-probability P[r]\mathcal{P}[\boldsymbol{r}] of the microscopic fields {rik(t)}\{r_i^k(t)\} over the time interval t∈[0,T]t \in[0,T]. The generating functional encodes the statistics of the time-dependent network activity, since derivatives of Z[ψ]\mathcal{Z}[\psi] with respect to the source fields ψik(t)\psi_i^k(t) generate the correlation functions of σik(t)\sigma_i^k(t). In particular, the average single-site activity and the two-time activity correlation are given by

⟨σik(t)⟩=δZδψik(t)∣ψ=0(7)\left\langle\sigma_i^k(t)\right\rangle= \left.\frac{\delta\mathcal{Z}}{\delta\psi_i^k(t)}\right|_{\psi=0} \tag*{(7)}

and

⟨σik(t)σil(t′)⟩=δ2Zδψik(t)δψil(t′)∣ψ=0.(8)\left\langle\sigma_i^k(t)\sigma_i^l(t')\right\rangle= \left.\frac{\delta^2 \mathcal{Z}}{\delta\psi_i^k(t)\delta\psi_i^l(t')}\right|_{\psi=0}. \tag*{(8)}

In appendices A and B, we show that, in the limit N→∞N \to\infty, the original network dynamics factorizes into M(B,S)M(B,S) effective stochastic single-site equations

References

  1. [1]J. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proceedings of the National Academy of Sciences of the United States of America 79, 2554 (1982).pubmed.ncbi.nlm.nih.gov/6953413
  2. [2]J. J. Hopfield, Neurons with graded response have collective computational properties like those of two-state neurons, Proceedings of the National Academy of Sciences of the United States of America 81, 3088 (1984).pubmed.ncbi.nlm.nih.gov/6587342
  3. [3]D. J. Amit, H. Gutfreund, and H. Sompolinsky, Spin-glass models of neural networks, Physical Review A 32, 1007 (1985).pubmed.ncbi.nlm.nih.gov/9896156
  4. [4]M. V. Tsodyks and M. V. Feigel’man, The enhanced storage capacity in neural networks with low activity level, Europhysics Letters 6, 101 (1988).DOI
  5. [5]Braitenberg, V., & Schüz, A. (2013). Anatomy of the cortex: statistics and geometry (Vol. 18).
  6. [6]I. Kanter, Potts-glass models of neural networks, Physical Review A 37, 2739 (1988).pubmed.ncbi.nlm.nih.gov/9900002
  7. [7]A. Treves, Frontal latching networks: a possible neural basis for infinite recursion, Cognitive Neuropsychology 22, 276 (2005).DOI
  8. [8]M. Naim, V. Boboeva, C. J. Kang, and A. Treves, Reducing a cortical network to a Potts model yields storage capacity estimates, Journal of Statistical Mechanics: Theory and Experiment 2018, 043304 (2018).arxiv.org/abs/1710.04897
  9. [9]E. Russo, V. M. K. Namboodiri, A. Treves, and E. Kropff, Free association transitions in models of cortical latching dynamics, New Journal of Physics 10, 015008 (2008).DOI
  10. [10]E. Russo and A. Treves, Cortical free-association dynamics: distinct phases of a latching network, Physical Review E 85, 051920 (2012).pubmed.ncbi.nlm.nih.gov/23004800
  11. [11]D. Horn and M. Usher, Neural networks with dynamical thresholds, Physical Review A 40, 1036 (1989).pubmed.ncbi.nlm.nih.gov/9902229
  12. [12]M. Herrmann, E. Ruppin, and M. Usher, A neural model of the dynamic activation of memory, Biological cybernetics 68, 455 (1993).DOI
  13. [13]C. J. Kang, M. Naim, V. Boboeva, and A. Treves, Life on the edge: Latching dynamics in a Potts neural network, Entropy 19, 468 (2017).DOI
  14. [14]P. C. Martin, E. D. Siggia, and H. A. Rose, Statistical dynamics of classical systems, Phys. Rev. A 8, 423 (1973).DOI
  15. [15]A. C. C. Coolen, Statistical mechanics of recurrent neural networks ii: Dynamics, in Handbook of Biological Physics, Vol. 4 (Elsevier, 2001) pp. 619–684.DOI
  16. [16]A. Crisanti and H. Sompolinsky, Path integral approach to random neural networks, Phys. Rev. E 98, 062120 (2018).DOI
  17. [17]F. L. Metz, Dynamical mean-field theory of complex systems on sparse directed networks, Phys. Rev. Lett. 134, 037401 (2025).DOI
  18. [18]C. F. Mari and A. Treves, Modeling neocortical areas with a modular neural network, Biosystems 48, 47 (1998).pubmed.ncbi.nlm.nih.gov/9886631
  19. [19]K. I. Ryom, Statistical analysis of potts neural networks and latching dynamics, PhD Thesis (2023), accessible from https://hdl.handle.net/20.500.11767/134430.hdl.handle.net/20.500.11767/134430
  20. [20]K. S. Ryu, S. I. Om, K. I. Ryom, and Y. H. Ko, Retrieval properties of attractor neural networks with regulated levels of activity, The European Physical Journal B 99, 59 (2026).DOI
  21. [21]S. H. Kim, Y. G. Hong, K. I. Ryom, and C. J. Kang, Glassy and retrieval properties of Potts neural networks with constrained mean activity, AIP Advances 16, 045114 (2026).DOI
  22. [22]D. O’Kane and A. Treves, Short-and long-range connections in autoassociative memory, Journal of Physics A: Mathematical and General 25, 5055 (1992).DOI
  23. [23]H. Itoh and M. Kimoto, Chaotic itinerancy with preferred transition routes appearing in an atmospheric model, Physica D: Nonlinear Phenomena 109, 274 (1997).DOI
  24. [24]I. Tsuda, The plausibility of a chaotic brain theory, Behavioral and Brain Sciences 24, 829 (2001).DOI
  25. [25]Kaneko, K. (2002). From coupled dynamical systems to biological irreversibility. Dynamical Systems and Irreversibility: A Special Volume of Advances in Chemical Physics, 122, 53.
  26. [26]M. Rabinovich, R. Huerta, and G. Laurent, Transient dynamics for neural processing, Science 321, 48 (2008).DOI
  27. [27]M. I. Rabinovich and P. Varona, Robust transient dynamics and brain functions, Frontiers in Computational Neuroscience 5, 24 (2011).DOI
  28. [28]C. J. Kang and A. Treves, The challenge of taming a latching network near criticality, in The Functional Role of Critical Dynamics in Neural Systems (Springer, 2019) pp. 81–94.DOI
  29. [29]Y. Y. Mun, J. U. Song, C. J. K. Kang, and K. I. Ryom, Latching dynamics instructed through an extra adaptation-related field in Potts neural network, AIP Advances 16, 065026 (2026).DOI
  30. [30]H. Sompolinsky, A. Crisanti, and H.-J. Sommers, Chaos in random neural networks, Physical Review Letters 61, 259 (1988).pubmed.ncbi.nlm.nih.gov/10039285
  31. [31]F. Mastrogiuseppe and S. Ostojic, Linking connectivity, dynamics, and computations in low-rank recurrent neural networks, Neuron 99, 609 (2018).DOI
  32. [32]R. Tremblay, S. Lee, and B. Rudy, Gabaergic interneurons in the neocortex: from cellular properties to circuits, Neuron 91, 260 (2016).DOI
  33. [33]K. I. Ryom and A. Treves, Speed inversion in a Potts glass model of cortical dynamics, PRX Life 1, 013005 (2023).DOI
  34. [34]W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes 3rd Edition: The Art of Scientific Computing, 3rd ed. (Cambridge University Press, 2007).
  35. [35]Chicone, C. (2006). Ordinary Differential Equations with Applications. In Springer.

Paper details

vlnc:W_p69h9956
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 4

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Agricultural and Biological Sciences

Not recorded for this paper

View statement
Looking for related papers…
Contents