betaFeedback

WorksPhysics and Astronomy

Cell-type-specific modularity in E/I networks shapes dynamical regimes and computational capabilities

Abstract

Structural modularity is a ubiquitous feature of brain networks that shapes neural dynamics and information processing. Yet excitatory and inhibitory populations can differ in their modular organization, and it remains unclear how this asymmetry affects network dynamics and computation. Here, using dynamical mean-field theory and simulations of networks performing complex temporal tasks, we show that independently tuning excitatory and inhibitory modularity controls the dynamical regimes of balanced E/I networks. In particular, such tuning can generate a dominant spectral outlier despite overall E/I balance, producing a topology-controlled transition between bulk- and low-rank-dominated regimes, with different dynamical bifurcations leading to chaos and bistability, respectively. In chaotic time-series prediction tasks, computational performance is enhanced near these dynamical boundaries, including both the onset of chaos and the transition to bistability. Favorable configurations exhibit strongly modular excitation and comparatively homogeneous inhibition, similar to what is observed in cortical connectivity.

Introduction

Connectivity in biological neural networks is highly structured and inhomogeneous, often organized into modules or clusters with dense intra-module and sparse inter-module connectivity [1, 2, 3]. Modular and clustered connectivity structures have been shown to significantly impact network dynamics, for example, by controlling population bursting [4], generating cascading dynamics [5], and increasing the robustness of attractor dynamics [6]. Beyond this, modular network connectivity has been linked to cortical-like activity patterns [7, 8] and shown to impact information processing [9] and task performance [10].

At the same time, biological neural networks are comprised of diverse neuron types that follow distinct and highly specific connectivity patterns [11]. In particular, excitatory and inhibitory neurons differ not only in their functional roles but also in their local and long-range connectivity, with various inhibitory subtypes forming heterogeneous and spatially structured networks [12, 13]. This cell-type specificity of wiring suggests that modular organization may differ between excitation and inhibition.

Despite this biological complexity, most theoretical studies of modular and clustered brain networks have focused on excitatory populations alone [7] [14, 15]. Models investigating interplay between excitation and inhibition have largely focused on the dynamics of balanced networks under spatial constraints [16, 17]. While recent work has demonstrated that clustered E/I architectures can substantially influence population dynamics [18–20], the impact of independently tunable, cell-type-specific modular organization of excitation and inhibition on network dynamics and information processing has not been studied.

Notably, variations in excitatory clustering alone, combined with uniform inhibition, are sufficient to induce rich dynamical regimes characterized by long autocorrelation timescales [14]. Therefore, different EE and II modular structures could have serious implications on the computational capabilities of neuronal networks. Consistent with this idea, clustered spiking E/I networks have been shown to enhance learning efficiency in reinforcement learning settings [21].

Here we study the dynamics and computational properties of a modular, balanced rate network with cell-type-specific connectivity. Using dynamic mean-field theory, we characterize how independently tuning of excitatory and inhibitory modularity shapes the network’s dynamical regime. We then relate these dynamical states to computational performance on a complex time-series prediction task using an echo state network framework, and examine how structured external inputs interact with E/I-modular recurrent connectivity. Together, this approach links cell-type-specific network architecture, emergent population dynamics, and computational performance within a unified theoretical framework.

Results

We use a network of NN rate neurons split into MM modules. Each module contains an equal number of excitatory and inhibitory neurons, which respectively form excitatory and inhibitory populations. Thus, the network has 2M2M populations, indexed by Greek letters μ∈{1,…,2M}\mu\in\{1,\ldots,2M\}. Populations μ∈{1,…,M}\mu\in\{1,\ldots,M\} correspond to excitatory ones and μ∈{M+1,…,2M}\mu\in\{M+1,\ldots,2M\} to inhibitory ones. We choose the number of neurons NN to be a multiple of 2M2M, so each population has exactly N/(2M)N/(2M) neurons. We vary the network’s modularity by redistributing connection density between intra-module and inter-module links while keeping the total connection density fixed. To this end, we introduce a modularity parameter r∈[0,1]r \in[0,1], where r=0r=0 corresponds to uniform connectivity without modular structure and r=1r=1 to fully disconnected modules. Specifically, the probability of connection from neuron jj in population ν\nu to neuron ii in population μ\mu is given by:

pijμν={pM[1+(M−1)⋅rJ(μ)J(ν)],I(μ)=I(ν)pM[1−rJ(μ)J(ν)],otherwise(1)p_{ij}^{\mu\nu} = \begin{cases} \frac{p}{M}\left[1+(M-1)\cdot r_{J(\mu)J(\nu)}\right], & I(\mu)=I(\nu) \\ \frac{p}{M}\left[1-r_{J(\mu)J(\nu)}\right], & \text{otherwise} \end{cases} \tag*{(1)}

where pp sets the overall connection density, the function J(μ)∈{E,I}J(\mu)\in\{E,I\} identifies the cell type, with J(μ)=EJ(\mu)=E for μ≤M\mu\le M and J(μ)=IJ(\mu)=I if μ>M\mu>M. I(μ)=μ mod MI(\mu)=\mu\ \mathrm{mod}\ M denotes the module index. This parametrization preserves the expected in-degree of each population while systematically biasing connections toward intra-module or inter-module wiring. For all realized connections, synaptic weights are drawn independently from a prescribed distribution (see Methods). Here, we study the effects of different modularity levels for the excitatory and inhibitory populations by varying rE=rIE=rEEr_E=r_{IE}=r_{EE} and rI=rII=rEIr_I=r_{II}=r_{EI}, thus allowing the modularity to depend only on the identity of the presynaptic neuron. This construction allows us to control the fraction of excitatory and inhibitory inputs each neuron receives from its module relative to the total number of incoming excitatory and inhibitory connections (Fig. 1, Fig. 2 a). Thus, the modularity parameters rEr_E, rIr_I act as explicit control parameters of the network topology, enabling systematic tuning of the connectivity structure.

The level of excitatory vs inhibitory modularity controls the network’s dynamics

We begin by studying the dynamics of E/I modular networks in the absence of external inputs. Specifically, we investigate the distribution of stable fixed points and their stability. In recurrent networks used for computation, these intrinsic dynamics are known to strongly influence computational performance. In particular, for echo-state networks, the echo-state property requires that perturbations decay over time, which corresponds to the largest Lyapunov exponent being negative [22–24]. At the same time, performance is often maximized near the edge of chaos, where the largest Lyapunov exponent approaches zero from below [23, 25, 26]. Identifying the location in parameter space of the transition from fixed points to chaos is therefore of central importance.

In a random heterogeneous network, the transition from a stable fixed point to chaotic dynamics is controlled by the spectral properties of the connectivity matrix. When a dense bulk of eigenvalues crosses the stability threshold, many eigenmodes destabilize simultaneously, leading to chaotic dynamics. In contrast, when a single eigenvalue crosses the threshold, only one mode is destabilized, resulting in a transition to a new stable fixed point or limit cycle. Recent work by Dahmen et al. has emphasized the qualitative differences between these two types of transitions and their impact on the correlation structure of neurons [27].

Figure 2. Dynamics of the reservoir network. a. Visual decomposition of the N×NN \times N connectivity matrix as mean and variance components. b. Phase diagram of the recurrent network in the absence of external input, as predicted analytically by the dynamic mean-field theory. Solid lines represent bifurcations where the stability of fixed points changes. The dashed line indicates a transition leading to an abrupt change in the variance around the monostable fixed point. The dotted gray horizontal and vertical lines indicate the sections shown in panels (d) and (e). c. Example trajectories of individual for networks of N=3000N = 3000 for all 4 different phases of the diagram (left) and the corresponding spectrum of the connectivity matrix (right). The theoretical prediction of the outlier eigenvalue is shown by a larger dot. The dotted gray lines indicate the threshold of instability (λcb\lambda_c^b or λclr\lambda_c^{lr}) and the gray background highlights the region of stability. d. Variance of the distribution of fixed points around their mean as a function of the clustering for a fixed spectral radius λ=1\lambda= 1. A continuous line indicates the theoretical results from DMFT, while dots are values obtained by averaging 10 networks of size N=1000N = 1000. e. Same as (d), as a function of the spectral radius with fixed r=0.1r = 0.1.

We therefore begin by determining the stability of fixed points in our system, constructing its full phase diagram. These theoretical results form the basis for our later analysis of reservoir performance. We summarize the main results and their implications here; we present the full analytical derivation step by step in Supplementary I-III.

We describe the recurrent connectivity in the network by a N×NN \times N block-structured matrix, where each block is approximated by random Gaussian weights with block-specific mean and variance (Fig. 2a). The approximation of connectivity by a random Gaussian block matrix has proven to accurately predict network dynamics in similar settings [20, 28–31]. Within this framework, the elements of the (unnormalized) connectivity matrix W~\widetilde{\mathbf{W}} can be written as

W~ijμν≃wμν+σμνzijμν,(2)\widetilde{W}^{\mu\nu}_{ij} \simeq w_{\mu\nu}+\sigma_{\mu\nu}z^{\mu\nu}_{ij}, \tag*{(2)}

where zijμν∼N(0,1)z^{\mu\nu}_{ij} \sim\mathcal{N}(0,1) are independent standard Gaussian random variables, and the reduced matrix elements wμνw_{\mu\nu} and σμν\sigma_{\mu\nu} denote the mean and standard deviation of the weights between populations μ\mu and ν\nu. The reduced matrices w\mathbf{w} and σ\boldsymbol{\sigma} are of size 2M×2M2M \times2M, with each entry corresponding to a specific population pair. Indices i,j∈{1,2,…,N/(2M)}i,j \in\{1,2,\ldots,N/(2M)\} label individual neurons within each population block.

We denote the largest eigenvalue of W~\widetilde{\mathbf{W}} by λW~\lambda_{\widetilde W}. The spectrum of W~\widetilde{\mathbf{W}} scales with system size, so normalization is required to take the thermodynamic limit N→+∞N \to+\infty. In our setup, we normalize the connectivity matrix W\mathbf{W} of our echo-state networks to have largest eigenvalue λ\lambda, which we treat as a control parameter for the reservoir. This normalization differs from the one typically used in the study of dynamical mean-field recurrent neural networks [28–30], but it yields qualitatively similar results (see Supplementary IIIB for a detailed comparison). Thus,

W=λλW~W~.(3)\mathbf{W} = \frac{\lambda}{\lambda_{\widetilde W}}\widetilde{\mathbf{W}}. \tag*{(3)}

The spectrum of the full connectivity matrix W\mathbf{W} follows directly from W~\widetilde{\mathbf{W}}. The spectrum of W~\widetilde{\mathbf{W}} is determined by the spectra of the reduced mean and variance matrices, w\mathbf{w} and σ\boldsymbol{\sigma} [30, 32]. If wμν=0w_{\mu\nu}=0, we have block-Gaussian matrices, whose spectrum falls inside a spectral disk with radius λsr=λσN/(2M)\lambda_{\mathrm{sr}}=\lambda_{\sigma}\sqrt{N/(2M)}, where λσ2\lambda_{\sigma}^{2} is the leading eigenvalue of σ2\boldsymbol{\sigma}^{2} [33–35]. If wμν≠0w_{\mu\nu}\ne0, the bulk is subject to a low-rank perturbation, potentially leading to the emergence of an outlier λot=Nλw/(2M)\lambda_{\mathrm{ot}}=N\lambda_{w}/(2M), with λw\lambda_{w} the largest eigenvalue of w\mathbf{w} [35, 36]. Therefore, the largest eigenvalue of W~\widetilde{\mathbf{W}} is related to the reduced matrices as λW~=max⁡(λot,λsr)\lambda_{\widetilde{W}}=\max(\lambda_{\mathrm{ot}},\lambda_{\mathrm{sr}}). When the largest eigenvalue is the outlier λot\lambda_{\mathrm{ot}}, we will say the network is in the low-rank-dominated phase, while if the largest eigenvalue is λsr\lambda_{\mathrm{sr}}, we will refer to a bulk-dominated phase. Notice that the finite-size scaling exponent depends on the leading eigenvalue, as λot∼O(N)\lambda_{\mathrm{ot}}\sim O(N) but λsr∼O(N)\lambda_{\mathrm{sr}}\sim O(\sqrt{N}).

Whether the network is low-rank dominated or bulk-dominated depends exclusively on the topology of the modules, which is controlled solely by rE,rIr_E,r_I. For M=2M=2, the reduced matrices w,σ\mathbf{w},\boldsymbol{\sigma} take a simple form and can be diagonalized exactly. For a sparse network with probability of connection pp, where each excitatory weight is approximated by a Gaussian with mean w0w_0 and variance σw2\sigma_w^2 (see Supplementary II), the outlier and spectral radius of W~\widetilde{\mathbf{W}} are given by

λot=14p(rE−rI)w0,(4)\lambda_{\mathrm{ot}}=\frac{1}{4}p(r_E-r_I)w_0, \tag*{(4)}
λsr=p8N[4pσw2+pw02(4−p(2+rE2+rI2))].(5)\lambda_{\mathrm{sr}}=\sqrt{\frac{p}{8N}\left[4p\sigma_w^2+pw_0^2\left(4-p(2+r_E^2+r_I^2)\right)\right]}. \tag*{(5)}

The first surprising feature is that, for rE>rIr_E>r_I, we can observe a large, positive, discrete spectral outlier. Therefore, rE>rIr_E>r_I constitutes a low-rank-dominated phase. This spectrum is typical of an excitatory-dominated network. However, our networks are balanced, with excitation and inhibition having equal weights on average. Thus, even in a balanced network, the topological structure can generate a spectral outlier, making our system behave similarly to an excitation-dominated network. As discussed above, the network cannot undergo a transition to a chaotic regime when such an isolated eigenvalue dominates. For a chaotic transition to occur, the spectral bulk must first cross the instability threshold, i.e., chaos requires the bulk-dominated phase. This is possible only when the outlier has been merged inside the spectral disk. The first point at which this can happen is when the outlier enters the spectral disk, i.e., at λot=λsr\lambda_{\mathrm{ot}}=\lambda_{\mathrm{sr}}. For a fixed rIr_I, assuming that N≫1N\gg1, this condition leads to a linear relation between rEr_E and rIr_I,

rEc≃rI+22σw2+(2−p)w02w0pN,(6)r_E^c\simeq r_I+\frac{2\sqrt{2\sigma_w^2+(2-p)w_0^2}}{w_0\sqrt{pN}}, \tag*{(6)}

so the bulk-dominated phase appears only for rE<rEcr_E<r_E^c. To further simplify the upcoming dynamical mean-field computations, we restricted ourselves to rE≡rr_E\equiv r and rI≡1−rr_I\equiv1-r, so the critical point reads

rc≃12+2σw2+(2−p)w02w0Np.(7)r_c\simeq\frac{1}{2}+\frac{\sqrt{2\sigma_w^2+(2-p)w_0^2}}{w_0\sqrt{Np}}. \tag*{(7)}

Therefore, strong clustering of excitatory neurons is accompanied by weakly structured inhibition, and vice versa. This setting preserves the overall level of modularity while selectively varying which cell type exhibits more modular connectivity, allowing us to use rr as a control parameter while retaining all the dynamical regimes identified in the full parameter space. In Supplementary IV, we numerically test that using rr and λ\lambda as control parameters, we capture all the dynamical regimes present in the system. For r>rcr>r_c, the system is in the low-rank-dominated phase, which becomes unstable when its single outlier crosses the instability threshold. Chaos requires the bulk-dominated phase, i.e., r<rcr<r_c. We remark that rcr_c holds independently of the maximum eigenvalue λ\lambda of the normalized network. Thus, the transition corresponds to a horizontal line in Fig. 2b.

In random matrix theory, the critical transition at rcr_c is called a Baik-Ben Arous-Péché (BBP) transition [35, 37]. The finite-size scaling of the system at each side of the transition is different. In the low-rank-dominated phase, the connectivity is normalized by λW~∼O(N)\lambda_{\widetilde{W}}\sim O(N), and the fluctuations of W\mathbf{W} vanish in the thermodynamic limit. Therefore, a standard stability analysis suffices, leading to a transition between a monostable fixed point and a bistable one happening at λclr=1/ϕ′(0)=4\lambda_c^{\mathrm{lr}}=1/\phi'(0)=4, with ϕ(x)\phi(x) the system nonlinearity (see Methods, Fig. 2b). On the other hand, in the bulk-dominated regime λW~∼O(N)\lambda_{\widetilde{W}}\sim O(\sqrt{N}). In this case, balance must cancel the low-rank contribution. Otherwise, the input to a neural population will diverge as O(N)O(\sqrt{N}) in the thermodynamic limit. Thus, the dynamics depend only on the heterogeneous bulk, which needs to be tackled via DMFT.

The DMFT approach substitutes all variables by a set of 2M2M representative neurons, one for each population, whose input is described by a random variable. The stability of the monostable fixed point depends on the matrix

Sμν=aμσμν2⟨[ϕ′(yν∗)]2⟩,(8)S_{\mu\nu}=a_{\mu}\sigma_{\mu\nu}^{2}\left\langle[\phi'(y_{\nu}^{*})]^{2}\right\rangle, \tag*{(8)}

where aμa_{\mu} is the fraction of neurons in a module μ\mu (in our case, all aμ=Nμ/(2M)a_{\mu}=N_{\mu}/(2M)), yν∗y_{\nu}^{*} indicates the input current at the fixed point and ⟨[ϕ′(yν∗)]2⟩\left\langle[\phi'(y_{\nu}^{*})]^{2}\right\rangle is an average over the distribution of fixed points. In the DMFT setting, we assume that all nodes in the same module behave identically. Stability is lost when λS/λσ2>1\lambda^{S}/\lambda_{\sigma}^{2}>1, consistently with previous studies [28, 29] (see Supplementary II for a complete derivation).

The fact that the low-rank contribution must be balanced, and the assumption that no module differs statistically from others, allows us to obtain a closed expression for the critical point, λcb≃6π=10.68\lambda_c^{b}\simeq6\sqrt{\pi}=10.68. This approximation coincides very well with numerical simulations. Therefore, the phase diagram in Fig. 2b contains 4 phases. The upper part (above rcr_c) corresponds to the low-rank-dominated regime. This regime contains two phases separated by λclr\lambda_c^{\mathrm{lr}}: a monostable fixed point and a bistable fixed point. The bulk-dominated regime (below rcr_c also contains two phases, separated by λcb\lambda_c^b, a monostable fixed point and a chaotic phase. Example trajectories and spectra for each of these phases can be found in Fig. 2c. It is important to note that although for low λ\lambda both regimes contain a monostable fixed point, these are different phases. In the low-rank-dominated regime, fluctuations from heterogeneity are suppressed, leading to a low-variance distribution for the fixed points. In contrast, the bulk-dominated stable fixed point has high variance. This becomes clearer when we compute the variance of the fixed-point distribution, shown in Fig. 2d alongside theoretical predictions from DMFT. For low λ\lambda (top panel), the variance of the monostable fixed point collapses at rcr_c, indicating the BBP phase transition.

Figure 3. The dynamical state impacts the computational performance of the ESN. a. Schematic of the echo state network (ESN) architecture. b. The histogram loss evaluates the ESN’s performance by comparing the distribution of values in the predicted (orange) and target (blue) time series. c. The divergence time of the predicted and target time series gives us a second measure of the ESN’s performance. The plot shows the actual (blue) and predicted (orange) trajectories from the point at which the external input is switched off and the reservoir begins operating as an echo-state network. d. Median (of 500 runs) histogram and e. divergence time losses for different EE and II modularity values and a spectral radius of λ=10\lambda= 10 indicate that, as expected from the analytics, the computational capabilities are optimized near the point of the chaotic transition (Eq. 5, marked by the dashed orange line). f. The median (500 runs) histogram and divergence time losses for different levels of EE and II modularity across the rE=1−rIr_E = 1 - r_I line for λ=7\lambda= 7. g. Same for λ=10\lambda= 10. The modularity at which performance changes coincides with the analytically predicted transition by Eq. (6). Local maximum performance is achieved close to the critical BBP transition controlled by the network topology. We plot here the ESN’s performance in reconstructing the Lorenz attractor. h. Reconstructions of four different chaotic attractors (Lorenz, Aizawa, Halvorsen, Chen) by the reservoir. The blue lines show the input data, and the orange lines show the ESN’s predictions.

Computational capabilities depend on the dynamical state

To assess how modularity influences computational performance, we use the recurrent network as the reservoir in an Echo State Network (ESN) [38]. In this framework, the recurrent connectivity remains fixed, and only a linear readout is trained. During training, the readout is optimized to predict the next step of a chaotic time series under teacher forcing. During testing, the predicted output is fed back as input, closing the loop and allowing the network to autonomously generate the time series trajectory (Fig. 3a). We quantify successful learning by the network’s ability to sustain accurate trajectory generation after switching to the autonomous regime.

We evaluate a network’s performance using two complementary metrics. First, we compute the distance between the distribution of points in the target and predicted time series (histogram loss Fig. 3b). Second, we measure the divergence time, defined as the first time point at which the generated trajectory deviates from the target trajectory beyond a prescribed threshold (Fig. 3c). Because the target dynamics is chaotic, even small prediction errors inevitably lead to trajectory divergence. After divergence, however, the generated trajectory may still remain close to the underlying chaotic attractor. The divergence time, therefore, quantifies how long the network can faithfully track a specific trajectory, whereas the histogram loss assesses how well it captures the geometry of the attractor in phase space. We report both metrics throughout.

First, we evaluate performance on the Lorenz attractor, a three-dimensional chaotic attractor commonly used for time-series prediction. We begin by testing whether performance is affected by different levels of modularity in the EE and II recurrent connections, with a fixed spectral radius λ=10\lambda= 10. Analytic derivations predict the transition between bulk- and outlier-dominated connectivity at rEc=a+rIr^{c}_{E}=a+r_{I} with a≃0.6a \simeq0.6. Interestingly, the transitions in both performance metrics are situated close to this line (Fig. 3d,e). We observe that shifting both modularities by the same value (i.e., parallel to the critical line) does not change either histogram loss or divergence time (Fig. 3d,e). This allowed us to constrain the investigation to the rE=1−rIr_{E}=1-r_{I} line instead of analyzing the full (rE,rI)(r_{E},r_{I}) plane. Here, the transition to loss of performance happens at a single rcr_{c}, which is well predicted by Eq. (6).

Therefore, we study the effect of the E/IE/I clustering on performance along the line rE=1−rI=rr_{E}=1-r_{I}=r without loss of generality. The resulting loss landscape closely mirrors the dynamical regimes predicted by the DMFT analysis (Fig. 3f,g) and reveals how network dynamics influence the computational capabilities of the network. In particular, the large-variance monostable fixed point displays generally high performance across the whole phase (Fig. 3g), whereas the fully chaotic regime is detrimental to performance (Fig. 3g). For each λ\lambda, the performance along the cut reaches a local maximum at the bistable phase, close to the BBP transition at rcr_c. This effect is most pronounced for the divergence time metric. Remarkably, the BBP transition separates different dynamical regimes at rcr_c, but performance peaks after rcr_c in both cases, highlighting the role of structure on the network’s information-processing capabilities (Fig. 3f,g). Finally, the global maximum performance happens near the interface between the monostable-bistable transition and not next to the edge of chaos, in contrast with unstructured recurrent networks.

Thus, modularity can tune the network with a sufficiently large spectral radius to a dynamical regime that supports good computational performance. The optimal configurations, characterized by clustered excitation and relatively uniform inhibition, bring the network near the heterogeneous monostable-bistable transition. This finding is consistent with established results on reservoir computing, which show that performance is maximized at criticality. Although we see a local maximum in performance at the so-called edge of chaos, as in previous studies, the optimal performance is not found at this point. Moreover, heterogeneity at the monostable point can support highly performant networks. [38, 39].

Finally, to ensure that our results generalize to other chaotic time series and are not specific to the Lorenz attractor [40], we repeat our experiment with three other chaotic attractors (Aizawa, Chen, and Halvorsen) [41–43]. Despite small changes in the average loss across attractors and small shifts in the loss distributions, the overall patterns remain the same, suggesting that our estimate of the network’s dynamical state can effectively predict the region where the ESN’s performance is maximized.

The simultaneous prediction of multiple chaotic attractors shifts the optimal connectivity structures in small networks

We test whether the reservoir can simultaneously reconstruct multiple independent chaotic time series. Specifically, we present the network with combinations of the four chaotic attractors (Lorenz, Aizawa, Halvorsen, and Chen) [40–43] we tested individually, creating higher-dimensional inputs (6, 9, or 12-dimensional for 2, 3, or 4 attractors, respectively). While the network generally succeeds in reconstructing time series consisting of combinations of multiple attractors, as would be expected due to the higher complexity of the task, the average performance decreases as the number of simultaneously predicted attractors increases (Figure 4b, Figure 4d).

Effect of network size and number of predicted attractors

Figure 4. The effect of network size and number of predicted attractors. a. Distance from the estimated transition to the critical spectral radius λcb\lambda^{b}_{c} for increasing system sizes. The network is tasked with reconstructing 2 (red) or 4 (green) chaotic attractors simultaneously. The distance to criticality shifts with the number of attractors, but decreases with the system size. To estimate the transition point, we find the point where the histogram loss increases abruptly for each value of rr below rcr_{c}. The result is averaged over these values of clustering. b. Histogram loss landscape for different system sizes and number of attractors to reconstruct. Theoretical prediction for transitions overlaid as a dashed black line. For 2 and 4 attractors, the estimation of the critical point (corresponds to the dots in panel (a)) is highlighted with a white dotted line. (c),(d) Same as (a), (b), but using the divergence time instead of the histogram loss. Note that the colormap is inverted, with blue tones indicating generally bad reservoir performance.

We test the network performance for different combinations of E/IE/I modularity (varying r=rE=1−rIr = r_E = 1 - r_I) and spectral radius λ\lambda. As we increase the number of presented attractors, the loss map shifts away from the predicted bifurcation diagram, with the chaotic transition appearing at a lower spectral radius λ\lambda. The main contribution to the shift comes from finite-size effects. To better visualize this, we performed experiments with reservoirs of increasing size. Smaller networks not only perform worse but also show a shifted transition to criticality, similar to the shift we observe for fixed-size networks with multiple attractors. We verify that performance increases in the monostable phase as the network size is gradually increased (Figure 4a, Figure 4c). In particular, we find that as we increase network size, the loss values transition of reservoirs predicting multiple attractors (Figure 4b, Figure 4d) slowly moves towards the recurrent network’s phase diagram (Fig. 2b). The finite-size shift of a critical point is a well-known effect in the theory of phase transitions [44], but here we also observe it as a function of the input dimensionality. It is important to note, however, that for a microscopic network, the sampled low-rank input is not exactly balanced, leading to the appearance of spectral outliers outside the expected radius even in the infinite-size limit (see Supplementary II). These are not only detrimental to reservoir performance but can also lead to shifts relative to theory predictions, which so far neglect such outliers.

The loss landscape in Figure 4 shows that criticality enhances computation capabilities in our system, as losses are reduced close to the bifurcations separating the monostable, bistable, and chaotic regimes. At the interface between the chaotic and bistable regions, the BBP transition, performance peaks locally just above rcr_c. The BBP separation between the two monostable regimes shows a more moderate effect. The monostable-bistable transition also increases performance at the edge of instability. The heterogeneous monostable region (lower left region, r<rcr < r_c, λ<λc\lambda< \lambda_c) reduces the histogram loss, while the divergence time improves as we move closer to the edge of chaos. Rather than a pronounced peak in performance at the edge of chaos, we have a broader region appearing before the transition. In fact, the best global performance is attained in the bistable regime, close to the corner of the bifurcation point between the four states, i.e., simultaneously close to rcr_c and λc\lambda_c.

Finally, larger networks are better at time-series prediction. Therefore, by comparing smaller networks, i.e., networks closer to failure, we can more effectively evaluate differences in computing capacity between networks with different dynamics. For example, the bistable region gives the reservoir the ability to predict the time series for the longest time while performing well on the 4-attractor reconstruction. This is clearer at N=300N = 300 (Figure 4). Observe that this region emerges for clustered excitation. Hence, modularity can enhance the reservoir’s capabilities relative to its random counterpart. Moreover, the effect seems to extend to very large spectral radii as the network size grows, allowing reservoirs to optimally solve the task with λ≫λc\lambda\gg\lambda_c.

Performance of the network is mostly independent of the modularity of input connections

So far, each neuron received randomly selected input from one of the multiple input attractors. We hypothesize that random input may be suboptimal and ask whether structured input targeted to specific EE and II populations affects the network’s performance.

To test the role of input modularity, we simulated a network of four modules that simultaneously reconstruct four chaotic attractors. Each module was assigned one attractor, and input specificity was controlled by excitatory and inhibitory input modularity, rEInputr_{E}^{\mathrm{Input}} and rIInputr_{I}^{\mathrm{Input}}, where rInput=1r^{\mathrm{Input}}=1 sets fully module-specific input and rInput=0r_{\mathrm{Input}}=0 sets random assignment.

To systematically map the space of optimal solutions, we use Bayesian inference. The added input modularity parameters create many network configurations that can solve the multi-attractor reconstruction task. Because no analytical treatment is available for input-driven networks, we use simulation-based inference (SBI) [45, 46]. We run 3×1053 \times10^{5} simulations for networks with S=4S=4 inputs and N=300N=300 recurrent units. Then we compute the resulting average histogram and divergence losses across all four attractors as ∑sHists+11+Div Times\sum_s \mathrm{Hist}_s+\frac{1}{1+\mathrm{Div\ Time}_s}, s∈{1,2,3,4}s \in\{1,2,3,4\} to infer the posterior distributions over spectral radius, recurrent modularity, and input modularity.

The posterior over recurrent parameters (Figure 5b) revealed two optimal solutions, consistent with Figure 4: one in the bistable regime with high spectral radius and excitatory modularity, and one in the analytically predicted monostable regime with low spectral radius and modularity.

Figure 5. Module-specific input connections have a minor effect on the ESN performance. a. An outline of the network with multiple attractors. We study how to optimally distribute feedforward EE and II connections from different attractors to different network modules. b. The posterior distribution of the network parameters obtained in simulation-based inference. c-f. The reservoir losses for different levels of EE and II modularity in the two indicated points of the posterior, circle rE=0.2r_E=0.2, λ=5.7\lambda=5.7, cross rE=0.95r_E=0.95, λ=1.5\lambda=1.5.

Next, we investigate the effect of input modularity within the two optimal recurrent architectures (Figure 5b and Supplementary Fig. 3). We stimulate the networks for different excitatory and inhibitory input modularity, while fixing the spectral radius and recurrent modularity to two of the most probable solutions (Figure 5c,Figure 5d,Figure 5e,Figure 5f). Input modularity had only a modest effect. In the bistable case (Figure 5e,Figure 5f), excitatory modularity showed a clear positive effect. In the monostable case, inhibitory modularity provided only a minor improvement in histogram loss (Figure 5e,Figure 5f).

Discussion

In this work, we study how cell-type–specific modular connectivity in balanced E/I networks shapes population dynamics and affects computational capabilities. Using dynamic mean-field theory, we show that varying levels of excitatory and inhibitory modularity control transitions between dynamical regimes, including the transition to chaos. Using networks with different topologies as reservoir computers, we show how these dynamical regimes directly influence performance on a complex temporal prediction task. In particular, networks with strongly modular excitatory connectivity and relatively homogeneous inhibition achieve optimal computational performance near transitions between dynamical regimes.

A central idea linking the dynamical state of the network and computational performance is that maximal processing capacity is achieved when a system operates at the edge of instability [47]. For echo-state reservoir computers, optimal performance has often been linked to the critical point separating stable and chaotic dynamics [48, 49], although criticality is not a necessary condition for optimal performance [49, 50]. In biological systems, the criticality hypothesis has also been invoked as a simple framework to explain information processing capabilities in the brain [51], including positive experimental support [52–54]. Moreover, some results suggest that the optimal state should be subcritical, leaving room for an external drive without destabilizing the system [55, 56], similar to the findings in echo-state networks.

Our results suggest that criticality is favorable for computation in the thermodynamic limit. We find that computational capabilities are enhanced close to all phase transitions in the model, although in different ways. Notably, the interface between the large-variance monostable point and the chaotic state, commonly known as the ‘edge-of-chaos’, does not have a pronounced maximum of performance before the critical point, as in the unstructured rate network case [49]. Instead, it appears smeared over a larger region. In contrast, we find performance peaks along the BBP transition happening at rcr_c, highlighting the important role of excitation-inhibition clustering for information processing. In particular, optimal performance occurs in the bistable state, near both the BBP rcr_c and the edge-of-instability λcar\lambda_c^{\mathrm{ar}}. At this point, the edge of chaos is not yet present.

Classical reservoir computing relies on uniformly connected recurrent networks [38], which, while analytically convenient, do not reflect the structured organization observed in biological circuits. Cortical networks exhibit pronounced modular and clustered connectivity, with distinct patterns across excitatory and inhibitory populations, and such structure has been shown to give rise to distinct dynamical regimes [14] and influence learning [57]. Our results build on this by demonstrating that introducing cell-type–specific modularity not only preserves computational functionality but also provides an additional degree of control over network dynamics. In particular, the regime we identify as computationally optimal, characterized by strongly modular excitatory connectivity and relatively homogeneous inhibition, is consistent with experimentally observed motifs in cortical circuits, where excitatory neurons tend to form clustered, feature-specific assemblies, while inhibitory connectivity is broader and less selective [13, 58]. This suggests that topological asymmetry between excitation and inhibition may help drive biological networks toward dynamical regimes favorable for information processing.

The application of DMFT to structured systems with excitatory and inhibitory populations is an active area of research. Our approach is based on theoretical developments for multipopulation neural networks [28–30]. However, none of these studies explicitly examine the effect of modular connectivity. Our approach follows Kadmon and Sompolinsky by approximating the modular connectivity matrix with Gaussian blocks [28] and applying the theory to elucidate the effect of topology. Other works in this line, such as Mastrogiuseppe and Ostojic [30], study E/I dynamics in detail, while Aljadeff et al. [29] consider only Gaussian networks with vanishing mean, and thus no excitatory or inhibitory behavior. A large family of structured networks was studied by Ahmadian et al. [33], but our model does not fall into this class of problems. Our results could be useful for other scenarios. Finally, Clark and Beiran [20] recently studied a multi-region model in which communication among modules occurs via low-rank, structured connectivity. However, they do not enforce Dale’s law or positive firing rates, limiting the biological plausibility of their results. Hence, our results complement and extend the previous literature. Moreover, we see that the balance state plays a fundamental role, even in the presence of structured networks. This issue arises because the low-rank contribution coming from structured connectivity does not scale the same way as the bulk [35]. To address this problem, many DMFT studies normalize the low-rank and bulk components separately [20, 28, 31, 59]. We show that this is not necessary. In particular, in the bulk-dominated regime, the input current from the low-rank contribution scales with system size, apparently leading to divergence. However, we show that network balance is able to cancel such divergences, as found in the classical literature on balanced states [60, 61].

The BBP transition may be relevant to the study of brain dynamics, bringing together the two types of criticality discussed by Dahmen [27]. A vast literature already highlights the importance of the balanced state, which appears in the bulk-dominated regime. More recently, the study of low-rank recurrent neural networks has attracted a lot of attention [20, 31, 36]. A modular network near a BBP transition, equipped with a plasticity mechanism, could switch between low-rank-dominated and bulk-dominated behavior, exploiting advantages from both states. In fact, our reservoirs seem to be generally efficient close to the BBP transition. Whether such a mechanism exists in the brain, and its relevance, remains an open question.

The theory also presents several important limitations. First, we keep reservoirs small to ensure reasonable training times. This leaves each module with only a handful of neurons. However, the theory is studied in the thermodynamic limit, for infinite sizes. Moreover, we compare the network’s autonomous behavior with the input-driven reservoir. Both effects contribute to shifting the location of the phase transitions relative to theoretical predictions. To fully describe this phenomenon, one should characterize the dynamics of the finite-size recurrent network under chaotic input. This remains an open theoretical question that falls outside the scope of our work and should be addressed separately.

A natural next step is to apply this framework to anatomical connectomes extracted from experimental data, using reservoir computing as a probe to relate measured connectivity motifs to dynamical regimes and computational capacity. More broadly, our results sugsuggest that biologically plausible RNNs are a fruitful, and still underused, lens through which computational neuroscience can study the interplay between structure, dynamics, and function.

Methods

Recurrent network

The dynamics of the network’s activity x=(x1,…,xN)\mathbf{x} = (x_1,\ldots,x_N) are given by the following equations

x˙i=−xi+ϕ(∑j=1NWijxj+si(t)),(9)\dot{x}_i=-x_i+\phi\left(\sum_{j=1}^{N}W_{ij}x_j+s_i(t)\right), \tag*{(9)}

where W\mathbf{W} is the recurrent connectivity matrix and si(t)s_i(t) is the external input to each node. The nonlinearity ϕ\phi is the sigmoid

ϕ(x)=11+e−x.(10)\phi(x)=\frac{1}{1+e^{-x}}. \tag*{(10)}

Note that in our setup, ϕ(0)=1/2≡ϕ0\phi(0)=1/2\equiv\phi_0. Initial conditions for each xix_i are drawn randomly in the interval [0,1][0,1]. We employ the Euler algorithm for integration with Δt=0.01\Delta t=0.01. We have checked that the results are numerically stable for the selected time step.

Connectivity

The matrix W\mathbf{W} contains MM modules. Each module consists of an excitatory and inhibitory population, making a total of 2M2M populations. The network is sparse, and the connection probability from a neuron ii in population μ\mu to a neuron jj in population ν\nu is given by Eq. (1).

We draw connection weights from a uniform random distribution on [w0,w0+Δw][w_0,w_0+\Delta w]. In simulations, we choose w0=Δw=0.2w_0=\Delta w=0.2. This is common to all populations, but inhibitory connections are multiplied by −1-1. After constructing the matrix W\mathbf{W}, we normalize it to have the desired largest eigenvalue λ\lambda.

For theoretical calculations, we approximate the sparse, weighted networks by a Gaussian distribution. To do so, the mean and variance of the uniform distribution w0w_0, σw2=(Δw)2/12\sigma_w^2=(\Delta w)^2/12 are employed. Please refer to Supplementary II for a detailed derivation.

Input

Each neuron receives as input one dimension of a DD-dimensional time series C⃗(t)\vec{C}(t):

s⃗(t)=WInputC⃗(t),(11)\vec{s}(t)=\mathbf{W}_{\mathrm{Input}}\vec{C}(t), \tag*{(11)}

where WInput\mathbf{W}_{\mathrm{Input}} is an N×DN\times D matrix, with a single non-zero entry per row with weight wInput=0.5w_{\mathrm{Input}}=0.5. Thus, each node receives a single dimension from the time series.

Echo state network (ESN)

The simulation of the ESN is divided in two phases:

Stimulus-driven: The time series C⃗(t)\vec{C}(t) is given as input for TT time steps, leading to the recurrent network activity X=[x⃗1,…,x⃗T]\mathbf{X}=[\vec{x}_1,\ldots,\vec{x}_T] obtained by stacking network’s state at each time. Using Ridge regression, a linear layer WR\mathbf{W}_{R} is trained to predict the trajectory of the time series. Thus, using the network activity at a given time step, x⃗t\vec{x}_t the linear layer predicts the next step of the time series C⃗(t+1)≡C⃗t+1\vec{C}(t+1)\equiv\vec{C}_{t+1}. The target is constructed by stacking the time series vectors in columns, ytarget=[C⃗2,…,C⃗T+1]\mathbf{y}^{\mathrm{target}}=[\vec{C}_2,\ldots,\vec{C}_{T+1}], and regression yields

WR=ytargetX†(XX†+βI)−1,(12)\mathbf{W}_{R}=\mathbf{y}^{\mathrm{target}}\mathbf{X}^{\dagger}\left(\mathbf{X}\mathbf{X}^{\dagger}+\beta\mathbf{I}\right)^{-1}, \tag*{(12)}

where β\beta is the Ridge regression’s normalization constant and †\dagger stands for transpose.

Closed-Loop: Once the training is complete, we replace the input from the given time series C⃗T+1\vec{C}_{T+1}, with the prediction generated via the trained linear layer WRx⃗T†\mathbf{W}_{R}\vec{x}_{T}^{\dagger}. If Ridge regression was successful, then x⃗T+1′=WRx⃗T†\vec{x}_{T+1}^{\prime}=\mathbf{W}_{R}\vec{x}_{T}^{\dagger} and the recurrent network will produce the same activity as if its input had been the actual time series.

We continue in the same way for T′T^{\prime} time steps, using the linear layer’s prediction as the next input for the recurrent network, creating a fully closed loop which results in the reconstructed time series X′=[x⃗T+1′,…,x⃗T+T′′]\mathbf{X}^{\prime}=[\vec{x}_{T+1}^{\prime},\ldots,\vec{x}_{T+T^{\prime}}^{\prime}]

Evaluation

To evaluate the performance of the ESN, we compare the reconstructed time series X′\mathbf{X}^{\prime} with the actual corresponding time series Y={C⃗T+1,…,C⃗T+T′}\mathbf{Y}=\{\vec{C}_{T+1},\ldots,\vec{C}_{T+T^{\prime}}\}. In an ideally functioning ESN, the dynamics of the reservoir should be rich enough to capture the trajectory of the time series and enable a very precise reconstruction, so that X′≈Y\mathbf{X}^{\prime}\approx\mathbf{Y} over the entire testing period. We use two metrics to quantify the ESN performance:

Divergence Time. We estimate the time τ\tau at which the reconstructed trajectory and the original one start to diverge, by obtaining the τ\tau such that ∣Cτ−xτ′∣>ϵ\lvert C_{\tau}-x_{\tau}^{\prime}\rvert>\epsilon for a given small threshold ϵ\epsilon.

Histogram Loss. We compute the histogram of each dimension of the time series separately, both for the original series and the reconstruction. Then, we compute the mean-square difference between the histograms for each dimension and sum the results, multiplied by 100 to avoid vanishing values.

Each metric measures a different aspect of reservoir prediction. On one hand, divergence time captures the quality of the prediction for individual time series. A long divergence time means the reservoir can imitate the behavior of the chaotic system for a long time. However, it does not provide information about the statistical properties of the reconstruction after divergence. On the other hand, histogram loss can be very small as long as the reservoir trajectory is predicting the stable basin of the chaotic attractor. Notice that one can have a short divergence time with small histogram loss, meaning a reservoir might correctly predict the chaotic basin of attraction without performing well on individual series, but both metrics are usually correlated.

Simulation-Based Inference

We use simulation-based inference [45] to find the optimal excitatory and inhibitory input modularity (rEinr_E^{in} and rIinr_I^{in}) together with the spectral radius (λ\lambda) and recurrent modularity (rr). We randomly sampled 250,000 parameters from a uniform prior distribution (rr, rEinr_E^{in}, and rIinr_I^{in} between 0 and 1; λ\lambda between 1 and 10). We then fed four attractors to the reservoir simultaneously and computed their divergence time and histogram loss. We then obtain an average loss for all of them as Ltotal=∑sHsts+11+Div TimessL_{\mathrm{total}} = \sum_s H_{\mathrm{sts}} + \frac{1}{1+\mathrm{Div\ Times}_s}. Next, we fit a normalizing spline flow as a neural posterior estimator to predict the distribution of four parameters given the loss function (p(rEin,rIin,r,λ∣Ltotal)p(r_E^{in}, r_I^{in}, r, \lambda\mid L_{\mathrm{total}})). Next, we sampled from this posterior given L=0L = 0. For Figure 5 c,d,e,g, we run a simulations for a rIinr_I^{in} between 0.1 and 0.9 with selected values of rEr_E and λ\lambda.

Acknowledgments

This work was supported by a Sofja Kovalevskaja Award from the Alexander von Humboldt Foundation. EG thanks the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for support, and OV thanks the International Max Planck Research School for the Mechanisms of Mental Function and Dysfunction and the Joachim Herz Foundation for support. We acknowledge support from the BMBF through the Tübingen AI Center (FKZ: 01IS18039B). AL is a member of the Cluster of Excellence “Machine Learning: New Perspectives for Science”, EXC 2064/1 (project number 39072764). EG is funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – project number 575506470. VB acknowledges funding by NextGenerationEU, in the framework of the FAIR - Future Artificial Intelligence Research project (FAIR PE00000013 – CUP B43C22000800006). We thank Enrico Ventura for useful discussions on random matrix theory and Miguel A. Muñoz for feedback on the manuscript.

References

Supplementary information

Emmanouil Giannakakis; Victor Buendía; Oleg Vinogradov; Anna Levina

Model

We consider a system with NN neurons, organized in MM clusters. Each cluster has an excitatory and an inhibitory population, the total number of populations is P=2MP = 2M. The ii-th neuron of population μ\mu is characterized by a continuous firing rate xiμ>0x_i^\mu> 0. Notice that i=1,2,…,Nμi = 1, 2, \ldots, N_\mu, where NμN_\mu is the number of neurons in population μ\mu, with μ=1,2,…,P\mu= 1, 2, \ldots, P. The equations for the dynamical state of the reservoirs read

x˙iμ=−xiμ+ϕ(∑ν=1P∑j=1NνWijμνxjν)(13)\dot{x}_i^\mu= -x_i^\mu+ \phi\left(\sum_{\nu=1}^{P}\sum_{j=1}^{N_\nu} W_{ij}^{\mu\nu}x_j^\nu\right) \tag*{(13)}

where ϕ(x)=1/(1+exp⁡(−x))\phi(x) = 1/(1 + \exp(-x)) is a sigmoidal transfer function and W\mathbf{W} the connectivity matrix with elements WijμνW_{ij}^{\mu\nu}.

Observe that we have selected our transfer function in such a way that ϕ(0)=1/2\phi(0) = 1/2 and ϕ(x)>0 ∀x\phi(x) > 0\ \forall x, which ensures Dale’s law by keeping all x>0x > 0 through the dynamics. The most common choice for nonlinearities in neural networks are those such that ϕ(x≤0)=0\phi(x \le0) = 0. However, this makes learning for the reservoir computer technically more cumbersome. Our choice facilitates learning while respecting Dale’s law.

The first step we will pursue is a change of variables, y⃗=Wx⃗\vec{y} = \mathbf{W}\vec{x}. The dynamics of yiμ(t)y_i^\mu(t) are

y˙iμ=−yiμ+∑ν=1P∑j=1NνWijμνϕ(yjν).(14)\dot{y}_i^\mu= -y_i^\mu+ \sum_{\nu=1}^{P}\sum_{j=1}^{N_\nu} W_{ij}^{\mu\nu}\phi(y_j^\nu). \tag*{(14)}

The difference between the variables xiμx_i^\mu and yiμy_i^\mu is that of rate activity to a membrane potential, and their dynamics have been shown to be equivalent [1]. In our case, yiμy_i^\mu represents the input current to the ii-th neuron in population μ\mu.

Baik-Ben Arous-Peché transition from clustered connectivity

Clustered connectivity matrix and normalization

We will follow a standard approach to model the clustered connectivity. As the clusters lead to a matrix formed by blocks, we will approximate each block itself by a Gaussian matrix, a system which is analytically tractable.

Thus, we approximate

W~ijμν≃wμν+σμνzijμν,(15)\widetilde{W}_{ij}^{\mu\nu} \simeq w_{\mu\nu}+\sigma_{\mu\nu}z_{ij}^{\mu\nu}, \tag*{(15)}

which is valid as long as each block has well-defined mean wμνw_{\mu\nu} and variance σμν2\sigma_{\mu\nu}^{2}. The next step is to find a proper normalization scheme. The dynamics of the neural network depend on the eigenvalues of W~\widetilde{\mathbf{W}}, which scales with the system size. In dynamical mean-field theory, the classical normalization procedure is

Wijμν≃wμνN+σμνNzijμν,(16)W_{ij}^{\mu\nu} \simeq\frac{w_{\mu\nu}}{N}+\frac{\sigma_{\mu\nu}}{\sqrt{N}}z_{ij}^{\mu\nu}, \tag*{(16)}

where the rationale for this normalization is the following: sums over jj yield O(N)O(N) terms when applied to the first term, but ∑jzijμν∼O(N)\sum_j z_{ij}^{\mu\nu}\sim O(\sqrt{N}). In this way, the matrix W\mathbf{W} is well-defined in the thermodynamic limit.

However, in reservoir computing it is desirable to keep control of the largest eigenvalue as a control parameter, λ\lambda. To do so, one customarily scales the matrix by λ/λW~\lambda/\lambda_{\widetilde{W}}, being λW~\lambda_{\widetilde{W}} the largest eigenvalue of W\mathbf{W}. Therefore, we normalize as

Wijμν≃λλW~W~ijμν,(17)W_{ij}^{\mu\nu} \simeq\frac{\lambda}{\lambda_{\widetilde{W}}}\widetilde{W}_{ij}^{\mu\nu}, \tag*{(17)}

The main question arising now is the finite-size scaling of λW~\lambda_{\widetilde{W}}. Our Gaussian expansion for W~\widetilde{\mathbf{W}} can be thought as a low-rank perturbation to a Gaussian random matrix by blocks. The spectra of such matrices can be dominated either by low-rank outliers or by a continuum density of eigenvalues. The discrete spectral outlier of unnormalized W~\widetilde{\mathbf{W}} is λot=Nλw/P\lambda_{\mathrm{ot}}=N\lambda_w/P, being λw\lambda_w the largest eigenvalue of w\mathbf{w}, while its spectral radius is given by λsr=λσN/P\lambda_{\mathrm{sr}}=\lambda_\sigma\sqrt{N/P}, with λσ2\lambda_\sigma^2 the largest eigenvalue of σ\boldsymbol{\sigma} [2].

Thus, the largest eigenvalue λW~=max⁡(λot,λsr)\lambda_{\widetilde{W}}=\max(\lambda_{\mathrm{ot}},\lambda_{\mathrm{sr}}) scales with NN when the low-rank contribution dominates and with N\sqrt{N} when the contribution from the spectral disk dominates. We will refer to these cases as the low-rank-dominated phase or bulk-dominated phase, respectively. We will show that modulating the network’s clustering can lead to a change of the dominant contribution, an instance of the so-called a Baik-Ben Arous-Peche critical transition in random matrix theory [2, 3].

Structure of the reduced matrices

We have seen that the spectra of the connectivity matrix W~\widetilde{\mathbf{W}} depends only on the reduced matrices w\mathbf{w} and σ\boldsymbol{\sigma}. Therefore, the first step is to characterize them for our connectivity matrix.

The weight of each link in the reservoir is sampled from a uniform distribution [w0,w0+Δw][w_0,w_0+\Delta w]. The mean and variance of such a uniform distribution are given by w=w0+Δw/2w=w_0+\Delta w/2 and σw2=(Δw)2/12\sigma_w^2=(\Delta w)^2/12, respectively. Neurons in the reservoir are connected with a probability pcp_c, so the resulting mean and variance for each block can be found by the law of total variance, giving w0pcw_0p_c for the mean and pc(σw2+w02(1−pc))p_c\left(\sigma_w^2+w_0^2(1-p_c)\right) for the variance.

The probability of connection pcp_c depends on whether the neurons belong to the same module. Neurons within the same module are connected with probability pin,X=p(1+(M−1)rX)/Mp_{\mathrm{in},X}=p(1+(M-1)r_X)/M, whereas neurons belonging to different modules are connected with probability pout,X=p(1−rX)/Mp_{\mathrm{out},X}=p(1-r_X)/M. Here, XX denotes the kind of connection (e.g., excitatory-excitatory, inhibitory-excitatory).

Using rEr_E and rIr_I for the modularity of connections for excitatory and inhibitory connections, respectively, the reduced matrices read

wμν={w0pin,E,ν<M,μ=ν mod M,w0pout,E,ν<M,μ≠ν mod M,−w0pin,I,ν>M,μ=ν mod M,−w0pout,I,ν>M,μ≠ν mod M,(18)w_{\mu\nu}= \begin{cases} w_0p_{\mathrm{in},E}, & \nu<M,\quad\mu=\nu\ \mathrm{mod}\ M,\\ w_0p_{\mathrm{out},E}, & \nu<M,\quad\mu\ne\nu\ \mathrm{mod}\ M,\\ -w_0p_{\mathrm{in},I}, & \nu>M,\quad\mu=\nu\ \mathrm{mod}\ M,\\ -w_0p_{\mathrm{out},I}, & \nu>M,\quad\mu\ne\nu\ \mathrm{mod}\ M, \end{cases} \tag*{(18)}

for the means and

σμν2={pin,E(σw2+w02(1−pin,E)),ν<M,μ=ν mod M,pout,E(σw2+w02(1−pout,E)),ν<M,μ≠ν mod M,pin,I(σw2+w02(1−pin,I)),ν>M,μ=ν mod M,pout,I(σw2+w02(1−pout,I)),ν>M,μ≠ν mod M,(19)\sigma_{\mu\nu}^{2}= \begin{cases} p_{\mathrm{in},E}\left(\sigma_w^2+w_0^2(1-p_{\mathrm{in},E})\right), & \nu<M,\quad\mu=\nu\ \mathrm{mod}\ M,\\ p_{\mathrm{out},E}\left(\sigma_w^2+w_0^2(1-p_{\mathrm{out},E})\right), & \nu<M,\quad\mu\ne\nu\ \mathrm{mod}\ M,\\ p_{\mathrm{in},I}\left(\sigma_w^2+w_0^2(1-p_{\mathrm{in},I})\right), & \nu>M,\quad\mu=\nu\ \mathrm{mod}\ M,\\ p_{\mathrm{out},I}\left(\sigma_w^2+w_0^2(1-p_{\mathrm{out},I})\right), & \nu>M,\quad\mu\ne\nu\ \mathrm{mod}\ M, \end{cases} \tag*{(19)}

for the variances.

BBP transition for M=2M = 2

For two modules, M=2M = 2, the eigenvalues of the reduced matrices w\mathbf{w} and σ\boldsymbol{\sigma} can be easily computed, since they are the eigenvalues of 4×44 \times4 matrices. Thus, the eigenvalues of W~/N\widetilde{\mathbf{W}}/N are

λot=14p(rE−rI)w0,(20)\lambda_{\mathrm{ot}} = \frac{1}{4}p(r_E-r_I)w_0, \tag*{(20)}
λsr=p8N[4σw2+pw02(4−p(2+rE2+rI2))],(21)\lambda_{\mathrm{sr}} = \sqrt{\frac{p}{8N}\left[4\sigma_w^2 + pw_0^2\left(4-p\left(2+r_E^2+r_I^2\right)\right)\right]}, \tag*{(21)}

where we have already set P=4P = 4 explicitly. Observe that although W\mathbf{W} is balanced in the thermodynamic limit (since ∑νwμν=0\sum_{\nu}w_{\mu\nu}=0) the outlier can get large and positive due to the structure of the network. In contrast, a perfectly balanced network will always have λot=0\lambda_{\mathrm{ot}} = 0. The isolated outlier will be responsible for a saddle-node bifurcation in the dynamics. For chaos to happen instead, the spectral disk must cross the instability threshold. Therefore, a necessary condition for chaos is that the outlier λot\lambda_{\mathrm{ot}} must be to the right, outside the spectral radius. This happens when the outlier enters the spectral disk, at λot=λsr\lambda_{\mathrm{ot}} = \lambda_{\mathrm{sr}}. The equality leads to a critical boundary for fixed inhibitory clustering,

rEc=NN+2rI+2(N+2)pw02(N+2)(2σw2+(2−p)w02)−pw02(N+1)rI2.(22)r_E^c = \frac{N}{N+2}r_I + \frac{2}{(N+2)pw_0^2}\sqrt{(N+2)\left(2\sigma_w^2+(2-p)w_0^2\right)-pw_0^2(N+1)r_I^2}. \tag*{(22)}

This expression is very close to a straight line in the interval rI∈[0,1]r_I \in[0,1]. Taking also N≫1N \gg1, we get

rEc≃rI+22σw2+(2−p)w02w0pN.(23)r_E^c \simeq r_I + \frac{2\sqrt{2\sigma_w^2+(2-p)w_0^2}}{w_0\sqrt{pN}}. \tag*{(23)}

We keep the order O(N−1/2)O(N^{-1/2}) as our reservoirs have a finite size. The bifurcation described by rEcr_E^c is plotted in the Main Text, Fig. 3c. Since this simply separates the space in two regions with identical dynamics, we restrict the clustering parameters to the line rE=rr_E = r, rI=1−rr_I = 1-r without loss of generality. When this is done, the eigenvalues simplify to

λot=14p(2r−1)w0,(24)\lambda_{\mathrm{ot}} = \frac{1}{4}p(2r-1)w_0, \tag*{(24)}
λsr=p8N[4σw2+w02(4−p(3−2r(1−r)))],(25)\lambda_{\mathrm{sr}} = \sqrt{\frac{p}{8N}\left[4\sigma_w^2+w_0^2\left(4-p\left(3-2r(1-r)\right)\right)\right]}, \tag*{(25)}

and the exact expression for the critical value at which both eigenvalues coincide is given by

rc=12+12w08(σw2+w02)−5pw02p(1+N).(26)r_c = \frac{1}{2}+\frac{1}{2w_0}\sqrt{\frac{8(\sigma_w^2+w_0^2)-5pw_0^2}{p(1+N)}}. \tag*{(26)}

We will use simply rr as a control parameter from now on. It is important to remark that although λot\lambda_{\mathrm{ot}} becomes negative at r=1/2r=1/2, when both excitation and inhibition populations are equally clustered, the outlier blends with the spectral disk before. At a finite size, this leads to an ”asymmetry” in the location of the bifurcation, benefiting a structure where E is clustered and I homogeneous.

Observe that the value of the critical point does not depend on our normalization choice for W~\widetilde{\mathbf{W}}, as we precisely impose λot=λsr\lambda_{\mathrm{ot}}=\lambda_{\mathrm{sr}}. Interestingly, if the usual DMFT normalization scheme is considered instead, this extra N\sqrt{N} is cancelled and then rcr_c does not scale with the system size, leading to an asymmetry between excitation and inhibition modularity even in the thermodynamic limit.

If we scale W~\widetilde{\mathbf{W}} by a control parameter λ\lambda, rcr_c remains invariant. As we will show in the next section, when the largest eigenvalue λ\lambda is large enough, rcr_c becomes the edge of chaos. Interestingly, the value rcr_c obtained for M=2M=2 seems to be a good approximation for larger values of MM, such as M=4M=4 used in the main text. A phase diagram as a function of rcr_c for different numbers of modules can be seen in Sup. Figure 1.

Figure 1. Different levels of EE and II modularity control network dynamics and computational capabilities. We investigate the impact of synapse-type-specific recurrent connectivity on the dynamics of a balanced network and evaluate its impact on the network’s computational capabilities. Using dynamic mean-field theory, we map different network topologies to distinct dynamical states and, using a reservoir computing framework, we show how the analytically-derived dynamical state controls the network’s performance in a time series prediction task.

Another feature of Sup. Figure 1 that is worth mentioning is the presence of outliers outside the spectral disk in the bulk-dominated phase. Commonly these arise from non-Gaussian contributions to the network or finite-size effects. However, here they are produced by our normalization scheme plus the lack of exact balance in the microscopic model. The input current is proportional to the sum of columns of the connectivity matrix, which in turn is proportional to ∑νwμν\sum_{\nu}w_{\mu\nu}. In theory, our network is balanced, ∑νwμν=0\sum_{\nu}w_{\mu\nu}=0, preventing any low-rank mode to emerge. In practice, samples from the microscopic model display fluctuations. In the bulk-dominated regime, the low-rank term actually scales as Nwμν\sqrt{N}w_{\mu\nu}, meaning that fluctuations are O(1)O(1). A sampled network will have ∑νwμν≠0\sum_{\nu}w_{\mu\nu}\ne0, breaking exact balance and leading to emergent outliers even at the thermodynamic limit. These would never appear in the usual DMFT normalization, in which the low-rank contribution does not scale with NN.

Figure 1 (PDF p. 17). Network’s dynamical properties are independent of number of modules (a) Spectrum two networks with 2 and 4 modules and N=3000N = 3000, normalized to have unit spectral radius. (b) Variance of the individual rate, which indicates a chaotic transition for fixed spectral radius λ=10\lambda= 10 and increasing modularity for the two previous networks. The transition to chaos happens at rcr_c (eq. (26)), which is the value computed for M=2M = 2.

Dynamic Mean-Field Theory

This section contains the detailed analytical calculations to obtain the phase diagram of the system with respect to the maximum eigenvalue λ\lambda and the critical point locations.

We start substituting the expression for the normalized connectivity Eq. (5) in the model equation for input currents, Eq. (2),

y˙iμ=−yiμ+λλ~∑ν=1Pwμν∑j=1Nνϕ(yjν)+λλ~∑ν=1Pσμν∑j=1Nνzijμνϕ(yjν).(27)\dot{y}_i^\mu= -y_i^\mu+ \frac{\lambda}{\tilde{\lambda}} \sum_{\nu=1}^{P} w_{\mu\nu} \sum_{j=1}^{N_\nu} \phi(y_j^\nu) + \frac{\lambda}{\tilde{\lambda}} \sum_{\nu=1}^{P} \sigma_{\mu\nu} \sum_{j=1}^{N_\nu} z_{ij}^{\mu\nu}\phi(y_j^\nu). \tag*{(27)}

The main idea is to assume that the sums over jj can be approximated by random Gaussian variables. When this is done, index ii becomes dummy, as interactions between neurons become a single unit subject to a stochastic (mean) field. Removing index ii, we define two random variables,

mμ(t)=1N∑jNμϕ(yjμ(t)),(28)m_\mu(t) = \frac{1}{N}\sum_{j}^{N_\mu}\phi(y_j^\mu(t)), \tag*{(28)}
sμν(t)=1N∑jNμzijμνϕ(yjμ(t)).(29)s_{\mu\nu}(t) = \frac{1}{\sqrt{N}}\sum_{j}^{N_\mu} z_{ij}^{\mu\nu}\phi(y_j^\mu(t)). \tag*{(29)}

Then, we can average over the disorder to obtain the average of these new variables. We employ a typical assumption in DMFT: correlation between zijμνz_{ij}^{\mu\nu} and any function of yμy^\mu, as well as correlations among different units, is of order 1/N1/N and can be neglected in the thermodynamic limit. Then,

⟨mμ⟩=aμ⟨ϕ(yμ)⟩,(30)\langle m_\mu\rangle= a_\mu\langle\phi(y_\mu)\rangle, \tag*{(30)}
⟨sμν⟩=0,(31)\langle s_{\mu\nu}\rangle= 0, \tag*{(31)}

where we have used ⟨ϕ(yjμ)⟩=⟨ϕ(yμ)⟩≡⟨ϕ(yμ)⟩\langle\phi(y_j^\mu)\rangle= \langle\phi(y^\mu)\rangle\equiv\langle\phi(y_\mu)\rangle, as the average over the disorder eliminates the dependence on jj, and ⟨zijμν⟩=0\langle z_{ij}^{\mu\nu}\rangle= 0. Additionally, we have defined the ratio of neurons in population μ\mu, aμ=Nμ/N=1/Pa_\mu= N_\mu/N = 1/P in our case.

Then we compute the variance of both variables. Observe that all other covariances go to zero, due to assumption that correlations between units vanish in the thermodynamic limit. The assumption allows us to simplify the calculation by breaking the double sum into two parts, as

⟨mμ2⟩−⟨mμ⟩2=1N2∑j,k[⟨ϕ(yjμ)ϕ(ykμ)⟩−⟨ϕ(yjμ)⟩⟨ϕ(ykμ)⟩]=1N2(∑j≠k(…)+∑jNμ[⟨ϕ(yjμ)2⟩−⟨ϕ(yjμ)⟩2])=aμN[⟨ϕ(yμ)2⟩−⟨ϕ(yμ)⟩2]→0.(32)\begin{aligned} \langle m_\mu^2\rangle-\langle m_\mu\rangle^2 &=\frac{1}{N^2}\sum_{j,k}\left[\langle\phi(y_j^\mu)\phi(y_k^\mu)\rangle-\langle\phi(y_j^\mu)\rangle\langle\phi(y_k^\mu)\rangle\right]\\ &=\frac{1}{N^2}\left(\sum_{j\ne k}(\ldots)+\sum_j^{N_\mu}\left[\langle\phi(y_j^\mu)^2\rangle-\langle\phi(y_j^\mu)\rangle^2\right]\right)\\ &=\frac{a_\mu}{N}\left[\langle\phi(y_\mu)^2\rangle-\langle\phi(y_\mu)\rangle^2\right]\to0. \tag*{(32)} \end{aligned}

Again, we have taken out the averages from the sum to discover that fluctuations of mμm_\mu vanish in the thermodynamic limit. Therefore, mμm_\mu converges to its mean in the thermodynamic limit, and can be considered a scalar with value mμ(t)=aμ⟨ϕ(yμ(t))⟩m_\mu(t)=a_\mu\langle\phi(y_\mu(t))\rangle

Note that we evaluated both mμ(t)m_\mu(t) at the same time tt, but it is immediate to see that the same result stands for two different time points t,t′t,t'.

For sμνs_{\mu\nu}, however, it is important to make the time dependencies explicit, as

⟨sμν(t)sμν(t′)⟩=1N∑j,k⟨zjμzkν⟩⟨ϕ(yjν(t))ϕ(ykν(t′))⟩=1N∑jNν⟨ϕ(yjν(t))ϕ(yjν(t′))⟩=aνΔϕνν(t,t′),(33)\begin{aligned} \langle s_{\mu\nu}(t)s_{\mu\nu}(t')\rangle &=\frac{1}{N}\sum_{j,k}\langle z_j^\mu z_k^\nu\rangle\langle\phi(y_j^\nu(t))\phi(y_k^\nu(t'))\rangle\\ &=\frac{1}{N}\sum_j^{N_\nu}\langle\phi(y_j^\nu(t))\phi(y_j^\nu(t'))\rangle =a_\nu\Delta_\phi^{\nu\nu}(t,t'), \tag*{(33)} \end{aligned}

where we defined Δϕνν(t,t′)=⟨ϕ(yν(t))ϕ(yν(t′))⟩\Delta_\phi^{\nu\nu}(t,t')=\langle\phi(y^\nu(t))\phi(y^\nu(t'))\rangle, the autocorrelation function of the nonlinearity. Finally, the mean-field equations read

y˙μ(t)=−yμ(t)+λNλW~∑ν=1Pwμνmν(t)+λNλW~∑ν=1Pσμνsμν(t),(34)\dot{y}_\mu(t)=-y_\mu(t)+\frac{\lambda N}{\lambda_{\widetilde W}}\sum_{\nu=1}^{P}w_{\mu\nu}m_\nu(t)+\frac{\lambda\sqrt{N}}{\lambda_{\widetilde W}}\sum_{\nu=1}^{P}\sigma_{\mu\nu}s_{\mu\nu}(t), \tag*{(34)}

and sμν(t)s_{\mu\nu}(t) is a Gaussian colored noise with zero-mean and autocorrelation function given by Eq. (33).

Reservoir normalization

Low-rank-dominated regime

To study the behaviour of Eq. (34), we just need to set the scaling of λW~\lambda_{\widetilde W}. Let’s start with the low-rank-dominated regime. In this case, we had λW~=Nλw/P\lambda_{\widetilde W}=N\lambda_w/P. Therefore, the second term of Eq. (34), vanishes in the thermodynamic limit. Since mν(t)m_\nu(t) does not fluctuate, in this case we have a fully deterministic evolution,

y˙μ(t)=−yμ(t)+λλw∑ν=1Pwμνϕ(yμ(t)).(35)\dot{y}_\mu(t)=-y_\mu(t)+\frac{\lambda}{\lambda_w}\sum_{\nu=1}^{P}w_{\mu\nu}\phi(y_\mu(t)). \tag*{(35)}

Observe that we substituted aν=1/Pa_\nu=1/P, which was simplified, and we wrote ⟨ϕ(yμ)⟩=ϕ(yμ)\langle\phi(y_\mu)\rangle=\phi(y_\mu), as there is no noise in this case. It is a PP-dimensional dynamical system.

To find fixed points, we write our nonlinearity as ϕ(x)=ϕˉ(x)+ϕ0\phi(x)=\bar{\phi}(x)+\phi_0, with ϕ0≡ϕ(0)\phi_0\equiv\phi(0). Then, since our network is balanced, it turns out that ∑νwμνϕ(yμ)=∑νwμνϕˉ(yμ)\sum_\nu w_{\mu\nu}\phi(y_\mu)=\sum_\nu w_{\mu\nu}\bar{\phi}(y_\mu). Then one can check that yμ=0y_\mu=0 is a fixed point of the dynamical system.

The Jacobian matrix of this system can be computed as usual, yielding

Jμν({yμ})=−δμν+λλwwμνϕ′(yν).(36)J_{\mu\nu}(\{y_\mu\})=-\delta_{\mu\nu}+\frac{\lambda}{\lambda_w}w_{\mu\nu}\phi'(y_\nu). \tag*{(36)}

Thus, the Jacobian at the fixed point is J=−I+λϕ′(0)w/λw\mathbf{J}=-\mathbf{I}+\lambda\phi'(0)\mathbf{w}/\lambda_w. Its eigenvectors are those of w\mathbf{w}, and the eigenvalues are −1+λϕ′(0)-1+\lambda\phi'(0), leading to a critical point

λclr=1ϕ′(0)(37)\lambda_c^{lr}=\frac{1}{\phi'(0)} \tag*{(37)}

Bulk-dominated regime

This case is more complicated. The system has heterogeneity again, as sμνs_{\mu\nu} is a fluctuating variable, and thus yμy_\mu is given by a (multidimensional) distribution of fixed points rather than a scalar. Moreover, the first term now scales badly with N\sqrt{N},

y˙μ(t)=−yμ(t)+Nλλw∑ν=1Pwμν⟨ϕˉ(yμ(t))⟩+Pλλσ∑ν=1Pσμνsμν(t).(38)\dot{y}_\mu(t)=-y_\mu(t)+\frac{\sqrt{N}\lambda}{\lambda_w}\sum_{\nu=1}^{P}w_{\mu\nu}\langle\bar{\phi}(y_\mu(t))\rangle+\frac{\sqrt{P}\lambda}{\lambda_\sigma}\sum_{\nu=1}^{P}\sigma_{\mu\nu}s_{\mu\nu}(t). \tag*{(38)}

The N\sqrt{N} term must vanish for the system to be well-defined. Since ϕˉ(x)\bar{\phi}(x) is an odd sigmoid function, a symmetric distribution of fixed points around the origin will cancel this term. Hence, the solution must fulfill ⟨yμ(t)⟩=0\langle y_\mu(t)\rangle=0. Once we have cancelled the low-rank term, fixed points satisfy

yμ=Pλλσ∑ν=1Pσμνsμν.(39)y_\mu=\frac{\sqrt{P}\lambda}{\lambda_\sigma}\sum_{\nu=1}^{P}\sigma_{\mu\nu}s_{\mu\nu}. \tag*{(39)}

Since sμνs_{\mu\nu} is a Gaussian random variable, yμy_\mu is also Gaussian. Its variance is

⟨yμ2⟩=Pλ2λσ2∑α,βσμασμβ⟨sμαsμβ⟩=Pλ2λσ2∑ασμα2aαΔϕαα=λ2λσ2∑ασμα2⟨ϕ(yα)2⟩.(40)\langle y_\mu^2\rangle=\frac{P\lambda^2}{\lambda_\sigma^2}\sum_{\alpha,\beta}\sigma_{\mu\alpha}\sigma_{\mu\beta}\langle s_{\mu\alpha}s_{\mu\beta}\rangle=\frac{P\lambda^2}{\lambda_\sigma^2}\sum_{\alpha}\sigma_{\mu\alpha}^2a_\alpha\Delta_{\phi}^{\alpha\alpha}=\frac{\lambda^2}{\lambda_\sigma^2}\sum_{\alpha}\sigma_{\mu\alpha}^2\langle\phi(y_\alpha)^2\rangle. \tag*{(40)}

Therefore, the distribution of fixed points is a centered multivariate Gaussian, where the covariance matrix is diagonal. Finally, the stability of this distribution remains to be studied.

To do so, the Jacobian must be computed. The complicated part is applying the derivative to the random variable sμνs_{\mu\nu}. The simplest approach is to compute a perturbation to sμν(t)s_{\mu\nu}(t) from the fixed point:

sμν(t)=sμν+δsμν(t)=1N∑jNνzjμνϕ(yjν+εjν(t))=1N∑jzjμν[ϕ(yjν)+ϕ′(yjν)εjν(t)]=sμν+1N∑jzjμνϕ′(yjν)εjν(t).(41)\begin{aligned} s_{\mu\nu}(t)&=s_{\mu\nu}+\delta s_{\mu\nu}(t)=\frac{1}{\sqrt{N}}\sum_{j}^{N_\nu}z_j^{\mu\nu}\phi(y_j^\nu+\varepsilon_j^\nu(t))\\ &=\frac{1}{\sqrt{N}}\sum_j z_j^{\mu\nu}\left[\phi(y_j^\nu)+\phi'(y_j^\nu)\varepsilon_j^\nu(t)\right]\\ &=s_{\mu\nu}+\frac{1}{\sqrt{N}}\sum_j z_j^{\mu\nu}\phi'(y_j^\nu)\varepsilon_j^\nu(t). \tag*{(41)} \end{aligned}

This leads immediately to the definition of

δsμν(t)=1N∑jNνzjμνϕ′(yjν)εjν(42)\delta s_{\mu\nu}(t)=\frac{1}{\sqrt{N}}\sum_j^{N_\nu}z_j^{\mu\nu}\phi'(y_j^\nu)\varepsilon_j^\nu \tag*{(42)}

as a new random Gaussian variable. Its mean and variance can be computed exactly as we did for sμν(t)s_{\mu\nu}(t), leading to

⟨δsμν(t)⟩=0,(43)\langle\delta s_{\mu\nu}(t)\rangle=0, \tag*{(43)}
⟨δsμν(t)δsμν(t′)⟩=aν⟨ϕ′(yν)2⟩Δενν(t,t′)≡aν⟨ϕ′(yν)2⟩Cν(t,t′).(44)\langle\delta s_{\mu\nu}(t)\delta s_{\mu\nu}(t')\rangle=a_\nu\langle\phi'(y_\nu)^2\rangle\Delta_{\varepsilon}^{\nu\nu}(t,t')\equiv a_\nu\langle\phi'(y_\nu)^2\rangle C_\nu(t,t'). \tag*{(44)}

where we have defined Cν(t,t′)C_\nu(t,t') to lighten the notation. Thus, δsμν(t)\delta s_{\mu\nu}(t) can be also interpreted as a Gaussian colored noise. If we now expand yμ(t)=yμ+εμ(t)y_\mu(t)=y_\mu+\varepsilon_\mu(t) in Eq. (22) the evolution for the perturbations εμ(t)\varepsilon_\mu(t) is

ε˙μ(t)=−εμ(t)+Pλλσ∑ν=1Pσμνδsμν(t).(45)\dot{\varepsilon}_\mu(t)=-\varepsilon_\mu(t)+\frac{\sqrt{P}\lambda}{\lambda_\sigma}\sum_{\nu=1}^{P}\sigma_{\mu\nu}\delta s_{\mu\nu}(t). \tag*{(45)}

Taking expectation values, we have ⟨ε˙μ⟩=−⟨εμ⟩\langle\dot{\varepsilon}_\mu\rangle=-\langle\varepsilon_\mu\rangle, so in average all perturbations will decay to the fixed point at infinite time. However, the autocorrelation function of the noise depends itself on the autocorrelation of the variables, Cμ(t,t′)C_\mu(t,t').

We can obtain an equation for the autocorrelation function by isolating the noise in the right-hand side, multiplying both sides at different time points, and then taking averages. Thus, we have

(∂t+1)(∂t′+1)Cμ(t,t′)=Pλ2λσ2∑α,βPσμασμβ⟨δsμα(t)δsμβ(t′)⟩=λ2λσ2∑α,βPσμα2⟨ϕ′(yα)2⟩Cα(t,t′),(46)(\partial_t + 1)(\partial_{t'} + 1)C_\mu(t,t') = \frac{P\lambda^2}{\lambda_\sigma^2}\sum_{\alpha,\beta}^{P}\sigma_{\mu\alpha}\sigma_{\mu\beta}\langle\delta s_{\mu\alpha}(t)\delta s_{\mu\beta}(t')\rangle = \frac{\lambda^2}{\lambda_\sigma^2}\sum_{\alpha,\beta}^{P}\sigma_{\mu\alpha}^2\langle\phi'(y_\alpha)^2\rangle C_\alpha(t,t'), \tag*{(46)}

which is obtained analogously to Eq. (40).

We are looking for stationary state solutions. Therefore, letting τ=t−t′\tau= t - t' and assuming C(t,t′)=C(τ)C(t,t') = C(\tau), we have ∂t=∂τ\partial_t = \partial_\tau, ∂t′=−∂τ\partial_{t'} = -\partial_\tau so our expression simplifies to

C¨μ(τ)=Cμ(τ)−λ2λσ2∑ν=1Pσμν2⟨ϕ′(yν)2⟩Cν(τ),(47)\ddot{C}_\mu(\tau) = C_\mu(\tau) - \frac{\lambda^2}{\lambda_\sigma^2}\sum_{\nu=1}^{P}\sigma_{\mu\nu}^2\langle\phi'(y_\nu)^2\rangle C_\nu(\tau), \tag*{(47)}

which can be cast into matrix form,

C⃗¨(τ)=(−I+λ2λσ2S)C⃗(τ).(48)\ddot{\vec{C}}(\tau) = \left(-\mathbf{I} + \frac{\lambda^2}{\lambda_\sigma^2}\mathbf{S}\right)\vec{C}(\tau). \tag*{(48)}

The elements of matrix S\mathbf{S} are given by Sμν=σμν2⟨ϕ′(yν)2⟩S_{\mu\nu} = \sigma_{\mu\nu}^2\langle\phi'(y_\nu)^2\rangle. This system has a phase transition when the largest eigenvalue of the linear coefficient changes sign. Its eigenvalues are those of S\mathbf{S}. Therefore, the transition happens at

λcb=λσ2λS.(49)\lambda_c^b = \sqrt{\frac{\lambda_\sigma^2}{\lambda_S}}. \tag*{(49)}

Approximation of critical point in the bulk-dominated regime

Our DMFT analysis reveals that in the bulk-dominated regime, the fixed points follow a distribution with zero mean and finite variance ⟨yμ2⟩\langle y_\mu^2\rangle (Eq. (40)). This point loses stability at λcb\lambda_c^b, given by Eq. (49), which depends on an expected value over the fixed points distribution.

For an exact solution, the variance of the fixed points needs to be found numerically, as Eq. (40) is a self-consistent equation: the right-hand side contains an expectation value over the distribution of fixed points, which depends on the variance we are trying to find. Finally, the integral present in Eq. (49) cannot be analytically evaluated.

We present now an approximation to compute the variance and the location of the critical point. To start with, we assume that all modules are equivalent, and hence ⟨yμ2⟩,⟨ϕ(yμ)⟩,⟨ϕ′(yμ)2⟩\langle y_\mu^2\rangle,\langle\phi(y_\mu)\rangle,\langle\phi'(y_\mu)^2\rangle do not depend on μ\mu. This ansatz is justified because our network is balanced by construction, so the input to each population ∑νwμν⟨ϕ(yν)⟩=⟨ϕ(y)⟩∑νwμν=0\sum_\nu w_{\mu\nu}\langle\phi(y_\nu)\rangle= \langle\phi(y)\rangle\sum_\nu w_{\mu\nu} = 0 becomes automatically balanced.

Under this assumption, the determination of the variance reduces to

⟨y2⟩=λ2⟨ϕ(y)2⟩,(50)\langle y^2\rangle= \lambda^2\langle\phi(y)^2\rangle, \tag*{(50)}

as one can show that ∑μσμα2=λσ2\sum_\mu\sigma_{\mu\alpha}^2 = \lambda_\sigma^2. If the variance is very low, then the input is always close to zero and ϕ(y)\phi(y) can be expanded in Taylor series. The lowest order would be ϕ(0)2≃1/4\phi(0)^2 \simeq1/4. On the other hand, if the variance was very large, most of the sigmoid is saturating, meaning that it can be approximated by a step function. The average of the square of step function is ⟨ϕ(y)2⟩≃1/2\langle\phi(y)^2\rangle\simeq1/2. Therefore,

λ24≤Δ2≤λ22.(51)\frac{\lambda^2}{4} \leq\Delta^2 \leq\frac{\lambda^2}{2}. \tag*{(51)}

For Eq. (49), our assumption leads to S=⟨ϕ′(y)2⟩σ2\mathbf{S} = \langle\phi'(y)^2\rangle\sigma^2 and therefore λS=⟨ϕ′(y)2⟩λσ2\lambda_S = \langle\phi'(y)^2\rangle\lambda_\sigma^2, so the criticality condition reduces to

λ2⟨ϕ′(y)2⟩=1.(52)\lambda^2\langle\phi'(y)^2\rangle= 1. \tag*{(52)}

Now, the nonlinearity ϕ′(y)2\phi'(y)^2 cannot be treated so easily. There are two options: first, assuming a small input current and expanding it in Taylor series. In this case, we would find the critical threshold to be at least around λc≃4\lambda_c \simeq4, leading to large Δ\Delta and finding an inconsistent result. Therefore, a Taylor expansion will perform poorly. Another option is an asymptotic expansion for large variance,

12πΔ2e−y2/(2Δ2)≃12πΔ2[1−y22Δ2+(Δ−4)].(53)\frac{1}{\sqrt{2\pi\Delta^2}}e^{-y^2/(2\Delta^2)} \simeq\frac{1}{\sqrt{2\pi\Delta^2}}\left[1-\frac{y^2}{2\Delta^2}+\left(\Delta^{-4}\right)\right]. \tag*{(53)}

At zeroth order the integral becomes simply

⟨ϕ′(y)2⟩≃∫dy2πΔ2ϕ′(y)2=162πΔ2.(54)\langle\phi'(y)^2\rangle\simeq\int\frac{dy}{\sqrt{2\pi\Delta^2}}\phi'(y)^2 = \frac{1}{6\sqrt{2\pi\Delta^2}}. \tag*{(54)}

Substituting the value of variance Δ2=λ2/2\Delta^2=\lambda^2/2 and we can solve for the critical λc≃6π=10.63\lambda_c\simeq6\sqrt{\pi}=10.63. Numerical solution of Eqs. (50), (52) leads to an exact value λc=10.18\lambda_c=10.18, so our approximation is quite accurate.

Usual DMFT normalization

In dynamical mean-field theory, the usual decomposition for the normalized connectivity matrix W\mathbf{W} is

Wijμν=λwμνN+λσμνNzijμν.(55)W_{ij}^{\mu\nu}=\frac{\lambda w_{\mu\nu}}{N}+\frac{\lambda\sigma_{\mu\nu}}{\sqrt{N}}z_{ij}^{\mu\nu}. \tag*{(55)}

In this case, the outlier eigenvalue is given by λot=λw/P\lambda_{\mathrm{ot}}=\lambda_w/P, while the spectral radius is λsr=λσ/P\lambda_{\mathrm{sr}}=\lambda_\sigma/\sqrt{P}. All dependency with NN has been cancelled, as each term is scaled by its corresponding NN. Therefore, the condition λot=λsr\lambda_{\mathrm{ot}}=\lambda_{\mathrm{sr}} leads to a critical point rc≠rc(N)r_c\ne r_c(N), in contrast with the previous case,

rc=12+12w08σw2+(8−5p)w021+p.(56)r_c=\frac{1}{2}+\frac{1}{2w_0}\sqrt{\frac{8\sigma_w^2+(8-5p)w_0^2}{1+p}}. \tag*{(56)}

We still have rc>1/2r_c>1/2, so the asymmetry between excitatory and inhibitory modularity persists in the thermodynamic limit. There will still be a low-rank-dominated regime and a bulk-regime, so qualitatively the results are identical to the previous ones.

We will just sketch the DMFT analysis, since this case is very similar to the previous computations, and it can be found also in the literature [4, 5]. Inserting the connectivity into Eq. (2) and identifying the variables mμ(t)m_\mu(t) and sμν(t)s_{\mu\nu}(t) we arrive to

y˙μ=−yμ+λ∑ν=1Pwμνmν(t)+λ∑ν=1Pσμνsμν(t).(57)\dot{y}_\mu=-y_\mu+\lambda\sum_{\nu=1}^{P}w_{\mu\nu}m_\nu(t)+\lambda\sum_{\nu=1}^{P}\sigma_{\mu\nu}s_{\mu\nu}(t). \tag*{(57)}

Now, in contrast with the previous case, both terms are of the same order and contribute equally to the dynamics. Again, mμ(t)m_\mu(t) behaves like a scalar, while sμν(t)s_{\mu\nu}(t) is a colored white noise. Then, yμ(t)y_\mu(t) must follow Gaussian statistics, as it is a combination of Gaussian variables. By taking averages, using that ⟨sμν(t)⟩=0\langle s_{\mu\nu}(t)\rangle=0 and ⟨mν⟩=⟨ϕ(yν)⟩\langle m_\nu\rangle=\langle\phi(y_\nu)\rangle, we get

⟨y˙μ⟩=−⟨yμ⟩+λ∑ν=1Pwμν⟨ϕˉ(yν)⟩(58)\langle\dot{y}_\mu\rangle=-\langle y_\mu\rangle+\lambda\sum_{\nu=1}^{P}w_{\mu\nu}\langle\bar{\phi}(y_\nu)\rangle \tag*{(58)}

where we have used again that the network is balanced to interchange ϕ(yν)\phi(y_\nu) by ϕˉ(yν)\bar{\phi}(y_\nu). Let’s start with the distribution of fixed points. If ⟨y˙μ⟩=0\langle\dot{y}_\mu\rangle=0, a distribution of fixed points with vanishing mean fulfills the equation. This is because yμy_\mu must follow Gaussian statistics, as previously discussed, so if ⟨yμ⟩=0\langle y_\mu\rangle=0 then the distribution of fixed points is symmetric around the origin and ⟨ϕˉ(yν)⟩=0\langle\bar{\phi}(y_\nu)\rangle=0 automatically. Although there might be other solutions, we will consider first this one.

The variance of the fixed points is given by

⟨yμ2⟩=∑αβwμαwμβ⟨mαmβ⟩+λ2∑αaασμα2⟨ϕ2(yα)⟩,(59)\langle y_\mu^2\rangle=\sum_{\alpha\beta}w_{\mu\alpha}w_{\mu\beta}\langle m_\alpha m_\beta\rangle+\lambda^2\sum_{\alpha}a_\alpha\sigma_{\mu\alpha}^2\langle\phi^2(y_\alpha)\rangle, \tag*{(59)}

but ⟨mαmβ⟩=δαβ⟨mα2⟩→0\langle m_\alpha m_\beta\rangle=\delta_{\alpha\beta}\langle m_\alpha^2\rangle\to0 in the thermodynamic limit, so the first term vanishes. One can observe that in this case, the variance of the monostable fixed will not change dramatically with rr as in the previous case. This can be understood intuitively because now both the outlier and bulk scale at the same order with the system, so the bulk can always contribute to the variance.

In fact, in this case the low-rank dominated regime also experiences fluctuations, so even if a single outlier becomes unstable the system will not be fully deterministic. Here, the perturbations yμ(t)=yμ+ϵμ(t)y_\mu(t) = y_\mu+ \epsilon_\mu(t) lead to

ϵ˙μ=−ϵμ+λ∑ν=1Pwμνδmν(t)+λ∑ν=1Pσμνδsμν(t),(60)\dot{\epsilon}_\mu= -\epsilon_\mu+ \lambda\sum_{\nu=1}^{P} w_{\mu\nu}\delta m_\nu(t) + \lambda\sum_{\nu=1}^{P} \sigma_{\mu\nu}\delta s_{\mu\nu}(t), \tag*{(60)}

where δsμν\delta s_{\mu\nu} is defined by Eq. (42) and δmμ(t)\delta m_\mu(t) is defined analogously by

δmμ(t)=1N∑j=1Nμϕ′(yjμ)ϵjμ(t).(61)\delta m_\mu(t) = \frac{1}{N}\sum_{j=1}^{N_\mu} \phi'(y_j^\mu)\epsilon_j^\mu(t). \tag*{(61)}

As mμ(t)m_\mu(t), δmμ(t)\delta m_\mu(t) converges to its average in the thermodynamic limit and behaves as a scalar. The perturbations ϵμ\epsilon_\mu can become unstable in two ways. First, it might happen that its average its unstable, as

⟨ϵ˙μ⟩=−⟨ϵμ⟩+λP∑ν=1Pwμν⟨ϕ′(yν)⟩⟨ϵν⟩,(62)\langle\dot{\epsilon}_\mu\rangle= -\langle\epsilon_\mu\rangle+ \frac{\lambda}{P}\sum_{\nu=1}^{P} w_{\mu\nu}\langle\phi'(y_\nu)\rangle\langle\epsilon_\nu\rangle, \tag*{(62)}

i.e., when the largest eigenvalue λclr>1/λJ\lambda_c^{\mathrm{lr}}>1/\lambda_J, being λJ\lambda_J the largest eigenvalue of the matrix given by elements Jμν=wμν⟨ϕ′(yν)⟩/PJ_{\mu\nu}=w_{\mu\nu}\langle\phi'(y_\nu)\rangle/P. Observe that this is almost the same result we got previously in the low-rank dominated regime. With the reservoir normalization, the variance of the fixed point vanishes, and so ⟨ϕ′(yν)⟩→ϕ′(0)\langle\phi'(y_\nu)\rangle\to\phi'(0). Additionally, we divided by the eigenvalue of λw\lambda_w. Therefore, this is the generalization of that formula for the case in which the system is subject to fluctuations.

The other way the monostable fixed point can be become unstable is because, despite the average ⟨ϵμ(t)⟩\langle\epsilon_\mu(t)\rangle might shrink to 0, the autocorrelations might be large. This leads to the same computation we performed already in the bulk-dominated case for the reservoir, with practically no difference. We will find a value λcb\lambda_c^{\mathrm{b}} for which the autocorrelations change their behaviour, which depends again on the matrix Sμν=σμν2⟨ϕ′(yν)2⟩/PS_{\mu\nu}=\sigma_{\mu\nu}^2\langle\phi'(y_\nu)^2\rangle/P.

Observe that the first transition requires a single mode to become unstable, while the second requires a whole bulk of eigenvalues to occur. Therefore, which transition takes place depends on modularity rr. Then, we have the same result as in the reservoir normalization. The major difference is that λclr\lambda_c^{\mathrm{lr}} may change dramatically its value with respect to the previous one. However, qualitatively the phase diagram is identical.

Dynamical state in rEr_E, rIr_I phase space

We have looked at the parameter space with rE=rr_E=r, rI=1−rr_I=1-r. Here, we show that the full (rE,rI)(r_E,r_I) does not display any qualitative difference, i.e. different dynamical regimes than the ones discussed in the Main Text. To show this, we run the dynamics of the recurrent network 100 times for each value of rEr_E and rIr_I and check that the results are symmetric to the diagonal of interest.

In order to detect different states, we obtain the average variance of individual trajectories, as well as the variance of the stationary firing rates across all neurons for two different spectral radii. The results are shown in Fig. 2. Observe that the bifurcation point in (rE,rI)(r_E,r_I) space is well predicted by Eq. (10) in the chaotic case, as expected.

For small values of the spectral radius, there’s no chaotic state (average variance of trajectories is always low), and a transition from monostable to bistable can be found, hinted by the change in rate’s variance. On the other hand, with high spectral radius the transition to chaos is visible by an increase of single trajectory variance. Variance across system’s rates is not so informative, as this measure assumes rates are all stationary –which is false for the chaotic case– but the transition is seen due to increase of spread of rate’s mean.

Attractors

We use the following 3-dimensional chaotic attractors as input time series (individually or all of them simultaneously) to our network [6, 7, 8, 9]:

Figure 2 (PDF p. 23). Phase space for (rE,rI)(r_E,r_I). (a) Average variance of the individual trajectories of neurons in the network for two spectral radii. The large one (left) shows a transition to chaos, displayed by the increase in each trajectory’s variance. The smaller one (right) has no transitions. (b) Variance of the stationary rate values for the network. Both cases display a transition. White dashed line shows rE=1−rIr_E = 1 - r_I line used in the main text. Red dashed line shows Eq. (10), which correctly predicts the transition to chaos in panel (a).

Lorenz:

x˙=σ(y−x),y˙=ρx−y−xz,z˙=xy−βz,(63)\begin{aligned} \dot{x} &= \sigma(y-x),\\ \dot{y} &= \rho x-y-xz,\\ \dot{z} &= xy-\beta z, \tag*{(63)} \end{aligned}
y˙=ρx−y−xz(64)\dot{y} = \rho x-y-xz \tag*{(64)}
z˙=xy−βz(65)\dot{z} = xy-\beta z \tag*{(65)}

for σ=10\sigma= 10, ρ=28\rho= 28 and β=2.667\beta= 2.667

Aizawa:

x˙=(z−b)x−y(66)\dot{x} = (z-b)x-y \tag*{(66)}
y˙=x+(z−b)y(67)\dot{y} = x+(z-b)y \tag*{(67)}
z˙=c+az−z33−(x2+y2)(1+ez)+fzx3,(68)\dot{z} = c+az-\frac{z^3}{3}-(x^2+y^2)(1+ez)+fzx^3, \tag*{(68)}

for a=0.95a = 0.95, b=0.7b = 0.7, c=0.6c = 0.6, d=3.5d = 3.5, e=0.25e = 0.25 and f=0.1f = 0.1

Halvorsen:

x˙=−αx−4y−4z−y2,(69)\dot{x} = -\alpha x-4y-4z-y^2, \tag*{(69)}
y˙=−αy−4x−4z−z2,(70)\dot{y} = -\alpha y-4x-4z-z^2, \tag*{(70)}
z˙=−αz−4x−4y−x2.(71)\dot{z} = -\alpha z-4x-4y-x^2. \tag*{(71)}

for α=1.4\alpha= 1.4

Chen:

x˙=a(y−x),(72)\dot{x} = a(y-x), \tag*{(72)}
y˙=(c−a)x−xz+cy,(73)\dot{y} = (c-a)x-xz+cy, \tag*{(73)}
z˙=xy−bz,(74)\dot{z} = xy-bz, \tag*{(74)}

for a=40a = 40, b=3b = 3 and c=28c = 28

Supplementary figures

This section presents additional figures supporting the main results.

Figure 3 (PDF p. 24). Estimating optimal module-specific input connections in ESN that predicts four chaotic attractors. Pairwise plot of marginal posterior distributions for four model parameters rE,λ,riinput,reinputr_E,\lambda,r_i^{\mathrm{input}},r_e^{\mathrm{input}} approximated with simulation-based inference. The plot is based on 10510^5 samples from the estimated posterior.

References

  1. [1]Kenneth D. Miller and Francesco Fumarola. Mathematical Equivalence of Two Common Forms of Firing Rate Models of Neural Networks. Neural Networks. 24(1):25–31.DOI
  2. [2]Marc Potters and Jean-Philippe Bouchaud. A First Course in Random Matrix Theory: for Physicists, Engineers and Data Scientists. Cambridge University Press, Cambridge, 2020.DOI
  3. [3]Jinwoo Baik, Gérard Ben Arous, and Sandrine Péché. Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices. The Annals of Probability, 33(5):1643–1697, September 2005.DOI
  4. [4]Jonathan Kadmon and Haim Sompolinsky. Transition to Chaos in Random Neuronal Networks. Physical Review X, 5(4):041030, November 2015.DOI
  5. [5]Francesca Mastrogiuseppe and Srdjan Ostojic. Intrinsically-generated fluctuating activity in excitatory-inhibitory networks. PLOS Computational Biology, 13(4):e1005498, April 2017.DOI
  6. [6]E. N. Lorenz. Deterministic nonperiodic flow. Journal of the Atmospheric Sciences, 20(2):130–141, 1963.
  7. [7]Guanrong Chen and Tetsushi Ueta. Yet another chaotic attractor. International Journal of Bifurcation and Chaos, 9(7):1465–1466, 1999.DOI
  8. [8]Halvorsen System, chapter Chapter 32, pages 197–202.DOI
  9. [9]Yoji Aizawa. Global aspects of the dissipative dynamical systems. i — statistical identification and fractal properties of the lorenz chaos —. Progress of Theoretical Physics, 68:64–84, 1982.DOI

Paper details

vlnc:W_b3gyr92u
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents